What is an agent?

LLM as a reasoning engine

The shift from completion to agent paradigm.

The completion paradigm

When you first use an LLM, you're working in the completion paradigm. You send a prompt, you get a response. It's a fancy autocomplete. You provide context, the model predicts what comes next.

response = ollama.chat(
    model="llama3",
    messages=[{"role": "user", "content": "What is Python?"}]
)
print(response.message.content)
# → "Python is a high-level programming language..."

This is useful, but it's fundamentally passive. The model answers your question and stops. It doesn't do anything. It doesn't check if its answer is right. It doesn't look up information it's unsure about. It doesn't try a different approach if the first one fails.

The reasoning paradigm

The shift to the reasoning paradigm happens when you start treating the LLM not as an answering machine, but as a reasoning engine, something that can look at a situation, think about what to do, and decide on an action.

The key insight is this: LLMs are remarkably good at deciding what to do next given a description of the current situation. They can:

• Break a complex goal into steps
• Look at a tool's description and decide when to use it
• Interpret the results of an action and decide what to do next
• Recognize when they're done

This is the foundation of every agent system. The LLM doesn't just generate text. It reasons about what action to take, then you (the developer) execute that action and feed the result back.

A concrete example

Say you want to build something that answers questions about your codebase. In the completion paradigm, you'd stuff code into the prompt and ask a question. In the reasoning paradigm, the LLM decides which files to look at:

# Completion paradigm: you decide what context to provide
response = ollama.chat(
    model="llama3",
    messages=[{
        "role": "user",
        "content": f"Given this code:\n{entire_codebase}\nHow does auth work?"
    }]
)
 
# Reasoning paradigm: the model decides what to look at
tools = [
    {"name": "search_code", "description": "Search for code by keyword"},
    {"name": "read_file", "description": "Read a specific file"},
    {"name": "list_directory", "description": "List files in a directory"},
]
# The LLM reasons: "I need to find auth-related code first"
# → calls search_code("authentication")
# → reads the results
# → calls read_file("src/auth/middleware.py")
# → reads that file
# → now has enough context to answer

The difference is who makes the decisions. In the completion paradigm, you do. In the reasoning paradigm, the model does. Your job as the developer is to give it good tools and a clear goal.

Why this matters for agents

An agent is just this reasoning pattern running in a loop. The model looks at the current state, decides what to do, you execute that action, and the model looks at the result. Repeat until the goal is achieved or a limit is hit.

Every agent framework (LangChain, CrewAI, AutoGen, OpenAI's Agents SDK) is built on this same foundation. The difference is how they structure the loop, manage state, and handle tool execution. Before we reach for any framework, we're going to build this loop from scratch so you understand exactly what's happening.

Key takeaway

The mental model shift is simple: stop thinking of LLMs as text generators and start thinking of them as decision engines. Your job is to build the scaffolding around them (the loop, the tools, the memory) and let them reason about what to do next. That scaffolding is what we call an agent.