The orchestration loop
Inner loop vs outer loop
Single-turn reasoning vs multi-turn task completion.
Two loops, not one
So far we've written agents as a single loop: one iteration per LLM call. That's the right starting point, but production agents almost always have two nested loops:
- The inner loop is what we've been building. It runs one task to completion: model thinks, tool runs, model thinks, tool runs, model finalizes.
- The outer loop is the layer above. It handles things like multi-turn conversation with a user, retries on failure, plan revision, or running the inner loop multiple times against different sub-goals.
Calling them "inner" and "outer" is borrowed from optimization. The inner loop is fast and tactical. The outer loop is slow and strategic.
What lives in the inner loop
The inner loop is single-turn task completion. One goal goes in, one answer comes out (or one failure). Everything we covered in the previous lesson lives here:
def inner_loop(goal, tools, tool_registry, max_iterations=15):
messages = [{"role": "system", "content": f"Goal: {goal}"}]
for step in range(max_iterations):
response = ollama.chat(model="llama3", messages=messages, tools=tools)
messages.append(response.message)
if not response.message.tool_calls:
return {"ok": True, "answer": response.message.content}
for call in response.message.tool_calls:
result = tool_registry[call.function.name](**call.function.arguments)
messages.append({"role": "tool", "content": str(result), "tool_call_id": call.id})
return {"ok": False, "reason": "max_iterations"}This function takes a goal and returns a result. It does not know about users, sessions, or retries. Keep it that way. The inner loop should be a pure function of (goal, tools, state).
What lives in the outer loop
The outer loop wraps the inner loop and adds anything that spans multiple inner-loop runs. Three common responsibilities:
Conversation with a user
Most agents talk to a user across many turns. The user asks a question, the agent runs an inner loop to answer, the user follows up, the agent runs another inner loop. Each user turn triggers exactly one inner loop:
def chat_session(tools, tool_registry):
history = []
while True:
user_input = input("> ")
if user_input.lower() in ("exit", "quit"):
break
# Each user turn triggers ONE inner-loop run
result = inner_loop(
goal=user_input,
tools=tools,
tool_registry=tool_registry,
)
history.append({"user": user_input, "agent": result})
print(result.get("answer") or f"[error: {result.get('reason')}]")The outer loop holds the conversation history. The inner loop holds the per-task scratchpad. Mixing them is how agents get confused: tool results from question 1 leak into question 5 and pollute the reasoning.
Retry on failure
When an inner loop returns ok: False, the outer loop decides whether to retry, escalate, or give up.
def with_retry(goal, tools, tool_registry, max_attempts=3):
for attempt in range(max_attempts):
result = inner_loop(goal, tools, tool_registry)
if result["ok"]:
return result
# rewrite the goal to nudge the model
goal = f"Previous attempt failed because {result['reason']}. Try again: {goal}"
return {"ok": False, "reason": "max_attempts"}Retries belong in the outer loop because retrying inside the inner loop is a different thing (that's a single tool retrying, not the whole task).
Plan / execute
This is the pattern made famous by Plan-and-Execute agents. The outer loop calls the model to produce a plan, then runs an inner loop for each step of the plan.
def plan_and_execute(goal, tools, tool_registry):
plan = make_plan(goal) # one LLM call, returns list of subtasks
results = []
for subtask in plan:
result = inner_loop(subtask, tools, tool_registry)
results.append(result)
if not result["ok"]:
return {"ok": False, "completed": results}
return {"ok": True, "results": results}The outer loop owns the plan. The inner loop owns each step.
A picture of both loops
Outer loop (per user turn / per plan step / per retry)
┌─────────────────────────────────────────────────────┐
│ │
│ user input ─→ [inner loop] ─→ response to user │
│ │ │
│ ▼ │
│ ┌──────────────────────────────────┐ │
│ │ Inner loop (per LLM iteration) │ │
│ │ │ │
│ │ model decides → tool runs │ │
│ │ ▲ │ │ │
│ │ └──── result ──────┘ │ │
│ │ │ │
│ └──────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────┘How to decide what goes where
A useful heuristic:
| Question | Loop |
|---|---|
| Does this happen many times within one task? | Inner |
| Does this happen across tasks or sessions? | Outer |
| Does it depend on a single tool result? | Inner |
| Does it depend on the agent's overall progress? | Outer |
| Does the user see it? | Outer |
| Does the model see it directly? | Inner |
When in doubt, push logic to the outer loop. The inner loop should stay narrow and predictable.
Why this split matters in Track 2
In multi-agent systems, the outer loop is often a different agent than the inner loop. A "supervisor" agent runs in the outer loop, calling "worker" agents that run inner loops. Same pattern, just with two LLMs instead of one. Keeping inner and outer cleanly separated now makes that transition obvious later.
Key takeaway
Real agents have two loops. The inner loop completes a single task. The outer loop manages everything across tasks: conversation, retries, planning. Mixing them produces tangled state and confused reasoning. Keeping them separated produces agents that are easy to debug and easy to extend.
Done with this lesson?