Orchestration topologies

Hierarchical orchestration

Orchestrator > sub-orchestrators > workers.

Supervisors all the way down

Supervisor/worker has one supervisor at the top and N specialist workers below. Hierarchical orchestration adds a layer: each "worker" is itself a supervisor over its own team. You end up with a tree where every internal node is an orchestrator and every leaf is an executor.

                [top orchestrator]
                  /            \
        [code lead]            [ops lead]
        /     |    \           /        \
    [reader][writer][tester] [deployer][monitor]

This is the topology you reach for when no single supervisor's planning prompt can hold all the domain knowledge needed to plan well. Splitting the planning into layers gives each level a smaller scope.

Why add a layer

Three reasons hierarchies earn their complexity:

1. The top supervisor can stay generalist

A top-level orchestrator does not need to know how to plan a code review, a database migration, and a deploy rollback. It only needs to know which sub-domain a request belongs to and hand off. Each sub-orchestrator owns the domain-specific planning.

This matters because supervisor planning prompts get long fast. A 3000-token planner that knows everything about code, ops, and comms underperforms three 1000-token planners each focused on one area.

2. Sub-domains can share state inside their team without leaking it

The code lead can pass detailed file contents to the writer agent inside its team without that data ever reaching the ops side. Hierarchies create natural information firewalls.

3. Failure can be localized

If the writer agent fails, the code lead retries or reports. The top orchestrator never sees the failure unless the code lead escalates. This is how human organizations work and it works for the same reason: most problems should be handled close to where they originate.

What changes from supervisor/worker

Mechanically, very little. The interface is identical (handoffs in, results out), but each "worker" is now an entire supervisor/worker subsystem behind that interface.

# top-level orchestrator
def top_orchestrator(user_request):
    plan = top_plan(user_request)
    results = []
    for step in plan.steps:
        team = TEAMS[step.team]   # each team is itself an orchestrator
        result = team.run(Handoff(
            from_agent="top-orchestrator",
            to_agent=step.team,
            intent=step.intent,
            context=summarize(results),
            artifacts=step.artifacts,
        ))
        results.append(result)
    return top_synthesize(user_request, results)
 
 
# code team's orchestrator (called as a "worker" by the top)
def code_team_run(handoff):
    plan = code_plan(handoff.intent)
    sub_results = []
    for step in plan.steps:
        agent = CODE_AGENTS[step.agent]
        sub_results.append(agent.run(Handoff(
            from_agent="code-team",
            to_agent=step.agent,
            intent=step.intent,
            context=handoff.context,
            artifacts=step.artifacts,
        )))
    return code_synthesize(handoff.intent, sub_results)

The shape is recursive: each layer plans, dispatches, synthesizes. The base case is a leaf agent with tools.

When hierarchies are right

Three signals:

• Distinct sub-domains. Code, ops, and comms genuinely require different domain knowledge to plan well.
• The top-level supervisor's prompt is bloating. If your single supervisor's system prompt has grown past 2000 tokens trying to know everything, splitting into a hierarchy will help.
• Sub-domains have their own internal complexity. "Code" might require a reader, a writer, and a tester all coordinated. That is itself a supervisor/worker problem, nested inside the larger one.

If those three are not all true, a flat supervisor/worker is fine.

When hierarchies are wrong

Hierarchies are heavy. Each layer adds:

• An extra planning LLM call.
• An extra synthesis call.
• An extra context boundary that summaries have to flow across.

For a two-step request, a hierarchical system might do six LLM calls where a flat supervisor would do three. If your typical workload is short, the hierarchy is overhead with no payoff.

Other things hierarchies are bad at:

• Cross-team work. If code and ops constantly need to consult each other within a single request, the rigid tree creates friction. They have to keep escalating to the top orchestrator. Flat supervisor/worker handles this better, or you accept that some flows go peer-to-peer (next-but-one lesson, swarms).
• Latency-sensitive tasks. Each layer adds turns. Three-layer hierarchies for a "what time is it" question is absurd.

Layering by abstraction, not by org chart

A subtle mistake: modeling the hierarchy after a human organization rather than after the structure of the work. If your data team has three engineers and a manager, you do not necessarily want three data agents and a data lead agent. The work the LLMs are doing might split differently from how humans split it.

The right question is "what level of planning detail does each layer need?" not "what does our company look like."

Communication shape inside a hierarchy

Two flows matter:

Top-down

Handoffs flow from the top orchestrator to team leads to leaf agents. Each handoff narrows scope: the top sends "find and fix the staging bug," the code lead sends "read auth.py and find the timeout," the file reader gets "read auth.py."

Bottom-up

Results flow up the tree, summarized at each layer. The leaf returns raw findings. The team lead synthesizes into a team-level result. The top orchestrator synthesizes into a final answer.

The summarization at each layer is what keeps the system tractable. If you forward raw findings up the entire chain, you get the transcript-forwarding problem from Module 2 last lesson, just with more layers.

When to flatten

If your hierarchy has only one team that ever does real work and the others are essentially passthrough, flatten. If two layers always run the same plan together, merge. Hierarchies should reflect actual domain boundaries, not anticipated ones.

A useful test: sketch your hierarchy, then ask "if I removed this layer, would anything get worse?" If the answer is no, remove it.

Key takeaway

Hierarchical orchestration is supervisor/worker recursed: each "worker" is itself a supervisor over its own team. It earns complexity when sub-domains have distinct planning needs and internal complexity. The cost is more LLM calls and harder cross-team flows. Layer by abstraction, not by org chart. The next lesson zooms in on the part where one loop runs inside another, the inner-loop / outer-loop split that hierarchies depend on.