Local LLM with Tool Calling
Running an ollama model locally with web search tool call capability
By Chi Kit Yeung in Python LLM
November 9, 2025
Problem
Privacy is a major problem. At least to those who care about it. That’s why I opt to run my AI locally. But there is a problem, my hardware can only host small models and these small models aren’t very smart. By giving a small 1B model like the llama3.2:1b the ability to make web searches, their results can be improved dramatically.
I wanted to try giving a local model the ability to do web search but it doesn’t seem to be natively supported by Ollama. LangChain offers the solution. I mainly followed this guide by BytePlus but the sample code was already outdated and LangChain’s framework has moved on so below is the updated code snippet that should work as of the time of this writing.
Ref: LangChain Docs - Chat Ollama
Solution
Dependencies
You’ll need to already have Ollama set up with your preferred model pulled. The model should have tool calling capabilities.
- Activate your virtual environment if you have any
- Install langchain and the Ollama integration
pip install -U langchain langchain-ollama
# Requires Python 3.10+
Code
from langchain_community.tools import DuckDuckGoSearchRun
from langchain.agents import create_agent
from langchain_ollama.chat_models import ChatOllama
from langchain_core.prompts import ChatPromptTemplate
# 1. Initialize the Ollama LLM
# Make sure the model name matches one you have downloaded with Ollama.
llm = ChatOllama(model="llama3.2:1b", temperature=0)
# 2. Define the tools the agent can use
# We'll give our agent a single tool: DuckDuckGo Search.
search_tool = DuckDuckGoSearchRun()
tools = [search_tool]
# 3. Create the agent prompt
# This prompt template is designed to instruct the agent on how to use tools.
prompt = ChatPromptTemplate.from_messages(
[
("system", "You are a helpful assistant. You have access to a web search tool."),
("human", "{input}"),
]
)
# 4. Create the agent itself
# The create_agent function binds the LLM, tools, and prompt together.
agent = create_agent(
model=llm,
tools=tools,
)
# 5. Test the integration with a query
def run_query(query: str):
"""Runs the agent with a specific query."""
messages = [
("human", query),
]
print(f"Testing with query: '{query}'")
response = agent.invoke({"messages": messages})
print("\nFinal Answer:")
print(response)
return response
Now let’s try to run it
# Example queries that require web access
res = run_query("When will the AI bubble burst?")
Output:
Testing with query: 'When will the AI bubble burst?'
Final Answer:
{'messages': [HumanMessage(content='When will the AI bubble burst?', additional_kwargs={}, response_metadata={}, id='eb0f5538-109a-4b66-b3d6-5c4e2fb7e419'), AIMessage(content='', additional_kwargs={}, response_metadata={'model': 'llama3.2:1b', 'created_at': '2025-11-09T08:23:19.510209531Z', 'done': True, 'done_reason': 'stop', 'total_duration': 11275986799, 'load_duration': 4435463244, 'prompt_eval_count': 187, 'prompt_eval_duration': 3986326456, 'eval_count': 25, 'eval_duration': 2851727724, 'model_name': 'llama3.2:1b', 'model_provider': 'ollama'}, id='lc_run--fa0b3d2c-777c-49ab-93e1-4c00c4aec915-0', tool_calls=[{'name': 'duckduckgo_search', 'args': {'query': 'AI bubble burst'}, 'id': '9fd76bd2-2e75-4697-a905-d343b20cd5d5', 'type': 'tool_call'}], usage_metadata={'input_tokens': 187, 'output_tokens': 25, 'total_tokens': 212}), ToolMessage(content='2 days ago - The AI bubble is a theorised stock market bubble growing amid the current AI boom , a period of rapid progression in artificial intelligence (AI) that is affecting the broader economy. Speculation about a bubble largely originates from concerns that leading AI tech firms are involved in a circular ... October 8, 2025 - Yale SOM leadership expert Jeffrey Sonnenfeld and co-author Stephen Henriques write that the tangle of AI deals among tech giants could be signs of dangerous overinvestment in the developing technology. They outline three ways the bubble could pop. 2 weeks ago - AI may not simply be “a bubble,” or even an enormous bubble. It may be the ultimate bubble. What you might cook up in a lab if your aim was to engineer the Platonic ideal of a tech bubble. One bubble to burst them all. 18 hours ago - "The vast bet on AI infrastructure assumes surging usage, yet multiple US surveys show adoption has actually declined since the summer," Carl-Benedikt Frey, professor of AI & work at the UK\'s University of Oxford, told DW. "Unless new, durable use cases emerge quickly, something will give — and the bubble could burst." October 2, 2025 - Whereas equity prices have historically followed earnings fundamentals, today’s market is driven overwhelmingly by momentum, as retail investors pile into meme stocks and AI companies because they think everybody else is piling into meme stocks and AI companies. Every economic bubble also has tell-tale signs of financial over-engineering, like the collateralized debt obligations and subprime mortgage-backed securities that blew up during the mid-2000s housing bubble.', name='duckduckgo_search', id='2050f4f9-306a-4cdc-8ab7-e8cc83b31f32', tool_call_id='9fd76bd2-2e75-4697-a905-d343b20cd5d5'), AIMessage(content='The current AI bubble is not expected to burst soon, but rather may be the ultimate bubble. Experts warn that the rapid growth in AI technology could lead to a sudden decline in adoption if new, durable use cases do not emerge quickly. Additionally, the market is driven overwhelmingly by momentum, with retail investors piling into meme stocks and AI companies due to speculation rather than fundamental analysis.', additional_kwargs={}, response_metadata={'model': 'llama3.2:1b', 'created_at': '2025-11-09T08:23:38.527808886Z', 'done': True, 'done_reason': 'stop', 'total_duration': 17369858684, 'load_duration': 148028929, 'prompt_eval_count': 428, 'prompt_eval_duration': 8639517930, 'eval_count': 77, 'eval_duration': 8576863329, 'model_name': 'llama3.2:1b', 'model_provider': 'ollama'}, id='lc_run--910b25ae-943f-489e-af0c-faefc81fd0ff-0', usage_metadata={'input_tokens': 428, 'output_tokens': 77, 'total_tokens': 505})]}
Hmm.. that’s a lot of output our dear model spat out. Let’s see what’s going on here:
for i in res['messages']:
print(i)
Output:
content='When will the AI bubble burst?' additional_kwargs={} response_metadata={} id='eb0f5538-109a-4b66-b3d6-5c4e2fb7e419'
content='' additional_kwargs={} response_metadata={'model': 'llama3.2:1b', 'created_at': '2025-11-09T08:23:19.510209531Z', 'done': True, 'done_reason': 'stop', 'total_duration': 11275986799, 'load_duration': 4435463244, 'prompt_eval_count': 187, 'prompt_eval_duration': 3986326456, 'eval_count': 25, 'eval_duration': 2851727724, 'model_name': 'llama3.2:1b', 'model_provider': 'ollama'} id='lc_run--fa0b3d2c-777c-49ab-93e1-4c00c4aec915-0' tool_calls=[{'name': 'duckduckgo_search', 'args': {'query': 'AI bubble burst'}, 'id': '9fd76bd2-2e75-4697-a905-d343b20cd5d5', 'type': 'tool_call'}] usage_metadata={'input_tokens': 187, 'output_tokens': 25, 'total_tokens': 212}
content='2 days ago - The AI bubble is a theorised stock market bubble growing amid the current AI boom , a period of rapid progression in artificial intelligence (AI) that is affecting the broader economy. Speculation about a bubble largely originates from concerns that leading AI tech firms are involved in a circular ... October 8, 2025 - Yale SOM leadership expert Jeffrey Sonnenfeld and co-author Stephen Henriques write that the tangle of AI deals among tech giants could be signs of dangerous overinvestment in the developing technology. They outline three ways the bubble could pop. 2 weeks ago - AI may not simply be “a bubble,” or even an enormous bubble. It may be the ultimate bubble. What you might cook up in a lab if your aim was to engineer the Platonic ideal of a tech bubble. One bubble to burst them all. 18 hours ago - "The vast bet on AI infrastructure assumes surging usage, yet multiple US surveys show adoption has actually declined since the summer," Carl-Benedikt Frey, professor of AI & work at the UK\'s University of Oxford, told DW. "Unless new, durable use cases emerge quickly, something will give — and the bubble could burst." October 2, 2025 - Whereas equity prices have historically followed earnings fundamentals, today’s market is driven overwhelmingly by momentum, as retail investors pile into meme stocks and AI companies because they think everybody else is piling into meme stocks and AI companies. Every economic bubble also has tell-tale signs of financial over-engineering, like the collateralized debt obligations and subprime mortgage-backed securities that blew up during the mid-2000s housing bubble.' name='duckduckgo_search' id='2050f4f9-306a-4cdc-8ab7-e8cc83b31f32' tool_call_id='9fd76bd2-2e75-4697-a905-d343b20cd5d5'
content='The current AI bubble is not expected to burst soon, but rather may be the ultimate bubble. Experts warn that the rapid growth in AI technology could lead to a sudden decline in adoption if new, durable use cases do not emerge quickly. Additionally, the market is driven overwhelmingly by momentum, with retail investors piling into meme stocks and AI companies due to speculation rather than fundamental analysis.' additional_kwargs={} response_metadata={'model': 'llama3.2:1b', 'created_at': '2025-11-09T08:23:38.527808886Z', 'done': True, 'done_reason': 'stop', 'total_duration': 17369858684, 'load_duration': 148028929, 'prompt_eval_count': 428, 'prompt_eval_duration': 8639517930, 'eval_count': 77, 'eval_duration': 8576863329, 'model_name': 'llama3.2:1b', 'model_provider': 'ollama'} id='lc_run--910b25ae-943f-489e-af0c-faefc81fd0ff-0' usage_metadata={'input_tokens': 428, 'output_tokens': 77, 'total_tokens': 505}
Aha! The response contains a list of 4 items: User query, some metadata, search tool results, final LLM response.
Let’s try to get a cleaner response by printing just the final response.
print(res['messages'][-1].content)
Output:
The current AI bubble is not expected to burst soon, but rather may be the ultimate bubble. Experts warn that the rapid growth in AI technology could lead to a sudden decline in adoption if new, durable use cases do not emerge quickly. Additionally, the market is driven overwhelmingly by momentum, with retail investors piling into meme stocks and AI companies due to speculation rather than fundamental analysis.
Looks good!
Let me know if you have any thoughts, comments, or suggestions below. Thank you!
Next up I want to learn how to implement RAG (Retrieval Augmented Generation) on local LLMs.