Orchestration topologies

Nested orchestration loops

Supervisor loops wrapping worker loops.

Loops within loops

Track 1 Module 2 introduced the agent loop: input, reasoning, action, observation, repeat. Once you go multi-agent, you stop having one loop. You have a loop of loops. The supervisor's loop dispatches; the worker runs its own loop; control returns to the supervisor; it dispatches again.

This nesting is what hierarchies depend on. It is also where some of the most surprising bugs live, because two loops sharing nothing can still produce strange behavior together. This lesson is about the inner-loop / outer-loop split: what the boundary between them really means and how to design it on purpose.

The shape

A supervisor/worker system has two loops:

OUTER LOOP (supervisor)
  while plan.has_steps_remaining:
      step = pick_next_step(plan, results_so_far)
      result = INNER_LOOP(step)              <-- worker runs here
      results_so_far.append(result)
      maybe_replan(plan, results_so_far)
  return synthesize(results_so_far)
 
 
INNER LOOP (worker)
  messages = [system, handoff_as_user]
  while not done:
      assistant_msg = model(messages, tools)
      if assistant_msg.tool_calls:
          for call in assistant_msg.tool_calls:
              messages.append(execute(call))
      else:
          done = True
  return assistant_msg.content

The inner loop is a Track 1 ReAct loop. The outer loop sits above it and is a different shape: it does not have its own tool registry; its "actions" are dispatching to workers.

What each loop knows

The outer loop knows:

• The user's original request.
• The plan and which steps are remaining.
• The summarized results of completed steps.

The outer loop does not know:

• How any worker reasoned about its task.
• Which tools any worker called.
• Any intermediate failure or retry inside a worker.

The inner loop knows:

• The handoff (intent, context, artifacts).
• Its own tool set.
• Its own message history within this single dispatch.

The inner loop does not know:

• The user's original request, except as passed in the handoff context.
• What other workers have done.
• Anything about the outer loop's plan.

This information firewall is the entire point. Each loop reasons in isolation about its own scope. Pollution does not leak between them.

Where the boundary should be

The single most important design decision in a multi-agent system is what the inner loop returns to the outer loop. Get this wrong and you've recreated the transcript-forwarding problem at the system level.

Three rules:

1. Return a structured result, not a transcript

The inner loop should produce a typed payload (summary, artifacts, status). The outer loop should never see the inner loop's tool-call history.

2. Truncate or summarize the result if it is large

If the worker found a 5000-token report, do not pass all of it back. Summarize to the level of detail the supervisor actually needs to plan the next step. The full report can be referenced through an artifact pointer if needed later.

3. Surface failures, not exceptions

If the worker hits a tool error, the inner loop should decide whether to retry or give up, and return a structured {status: "failed", reason: "..."} rather than letting an exception bubble. The outer loop should not see Python exceptions from worker internals.

Loop control questions

Each loop has its own exit conditions. Both have to terminate cleanly.

The outer loop terminates when:

• The plan is complete and synthesis has been produced.
• A worker returns a fatal failure that cannot be re-planned.
• A maximum step count is hit (a sanity guard).

The inner loop terminates when:

• The model produces a final answer (no tool calls).
• A maximum tool-call count is hit (Track 1 Module 2 covered this).
• A tool returns a fatal error that the worker cannot handle.

Mixing these up is a real mistake. A supervisor that exits because a worker hit a tool error has bypassed the worker's own retry logic. A worker that "exits" by returning control to the supervisor mid-task forces the supervisor to either restart it or treat partial work as final.

A subtle bug: shared step counters

When the outer loop has a max-iteration limit and the inner loop has its own, you have to think about what each one counts. A common bug:

# WRONG: outer loop limits "steps" but counts tool calls
MAX_STEPS = 20
for _ in range(MAX_STEPS):
    result = worker.run(handoff)   # worker might do 8 tool calls
    ...

If each worker run uses 8 inner-loop iterations, MAX_STEPS=20 in the outer loop means up to 160 LLM calls total. That is fine if expected, dangerous if not. Track each loop's budget separately and reason about the product when planning capacity.

Outer-loop replanning

The outer loop's most powerful capability is replanning: looking at the result of the previous step and deciding whether to continue, switch, or stop.

def outer_loop(plan, request):
    results = []
    while plan.has_more():
        step = plan.pop_next()
        result = run_worker(step)
        results.append(result)
 
        # Replan if the result suggests a different path.
        if result.status == "failed":
            plan = replan(request, results, plan)
        elif result.suggests_redirect:
            plan = replan(request, results, plan)
 
    return synthesize(request, results)

replan is itself an LLM call: given what we know now, what should the rest of the plan look like? The outer loop without replanning is just a fancy pipeline. With replanning, it becomes adaptive.

When to collapse to one loop

Some workflows look multi-loop but are not. If your "outer loop" only ever runs one step before exiting, you do not have an outer loop, you have a single agent. Collapse it.

A useful test: imagine running the system on the simplest possible request. If the outer loop fires once and the worker handles the whole thing, your outer loop is doing nothing for that case. If the outer loop fires once for almost every case, you do not need it at all.

Key takeaway

Multi-agent systems run a loop of loops: the outer loop plans and dispatches; the inner loop reasons and uses tools. Each one has its own state, exit conditions, and budget. The boundary between them is an information firewall: structured results flow up, transcripts and tool histories stay below. The next lesson covers what happens when you flatten this hierarchy entirely: peer-to-peer agents that have no central orchestrator at all.