The Orchestrator’s Tax

The Incident Was Not Actually About Parallelism

4 subagents had been launched in a single wave. Three had clear runtimes:
roughly twelve minutes, 5 and a half minutes, and 7 minutes. A fourth
was nonetheless operating. Checked out a technique, that already justified the delegation.
Three duties ran concurrently, so wall-clock time was round twelve minutes
as a substitute of one thing nearer to 25 if the work had been
serialized.

However velocity was the seen byproduct, not the attention-grabbing half. What I used to be
looking for was the fee, and my first intuition was that it needed to be the
duplicated effort of delegation itself: each subagent studying recordsdata,
reconstructing context, understanding the duty independently.

That was not the place the largest shock turned up.

At one level through the work, the orchestrator recommended checking on the
operating brokers. It was a small, throwaway immediate: “test on the brokers.” I
adopted it. As an alternative of a light-weight abstract, the device it used pulled again
the total uncooked transcript of a background agent: tens of 1000’s of tokens of
JSONL, intermediate reasoning, and gear output, imported wholesale into the
major thread. Then it occurred once more, for a second standing test.

There is a crucial caveat right here. The declare that this polling behaviour
value greater than the duplication tax of 4 brokers was the orchestrator grading
its personal mistake. I did not have actual per-call token accounting, so deal with that
rating because the orchestrator’s account, not a measured truth. The reliable
half is narrower: the transcript dumps had been actual, the wall-clock timings had been
actual, and the status-check path clearly launched a big, avoidable value.
Whether or not it was the one largest value stays a speculation till the tooling
can instrument it correctly.

What mattered extra was that I might discovered a price I hadn’t been on the lookout for.

The Prices Had been Not All of the Similar

As soon as I finished treating the session as one lump “subagent value,” the
image broke into items that did not belong collectively.

First, two of the 4 subagents had been working in the identical space of the
response pipeline. Completely different duties, totally different recordsdata, however each needed to
perceive the identical structure, the identical testing conventions, and far of
the identical surrounding code earlier than both may start. Every paid that
orientation value independently. That is not an argument towards delegation.
It is an argument that the work had been cut up too finely.

Second, one agent ran git stash and git stash pop whereas sibling brokers
had been writing elsewhere in the identical tree. Nothing broke, however the danger was
structural, as a result of repository-wide operations are completely cheap in a
single-threaded session and turn out to be a lot tougher to justify the second a number of
writers are energetic directly.

By now I had a rising checklist of culprits: standing polling, duplicated
orientation, unsafe git operations. I discovered myself attempting to rank them. Which
one value essentially the most? The longer I attempted to reply that, the much less satisfied I
turned that rating them was the appropriate query in any respect.

The Scarce Useful resource Is the Orchestrator’s Working Reminiscence

The transcript-polling incident stored bothering me for a purpose that had
nothing to do with token value. A token invoice is one-time, you pay it and it is
over. What occurred right here was totally different. The uncooked transcript stayed within the
orchestrator’s context after the device name accomplished, and each flip after
that carried it ahead, whether or not or not it was nonetheless helpful.

That was the second I spotted I had been treating two very totally different
sorts of value as if they had been the identical. Tokens are spent as soon as. Context
shapes each determination that follows. I wasn’t merely token
consumption anymore. I used to be trying on the high quality of the orchestrator’s
working reminiscence.

That realization was carrying two separate concepts, and I need to pull them aside
moderately than allow them to blur collectively. The primary is what I simply described.
Air pollution left in context taxes each later flip. The second just isn’t about
operating out of area in any respect. The extra that is sitting in context, competing for
consideration, the tougher it will get for a mannequin to select what issues proper now,
even with loads of room nonetheless free. A much bigger context window would not repair that.
It simply offers the noise extra room to pile up earlier than anybody notices.

Context home windows are solely going to get larger, that is a given. What issues
just isn’t how a lot room there’s, however how a lot of what is sitting in that room is
well worth the mannequin’s consideration. That is the true downside subagents want to unravel,
in the event that they’re used proper.

Seen this manner, the orchestrator is the one a part of the system that
accumulates understanding throughout a protracted session. It remembers why a design
determination was made, carries ahead architectural constraints, and is aware of which
trade-offs have already been mentioned. The subagents do not, and that is by
design. They’re presupposed to be disposable. Exploration, repeated file reads,
failed approaches, and noisy intermediate reasoning are supposed to keep in
employee contexts and by no means make the journey again to the principle thread.

Cognitive Locality Adjustments What Parallelism Is For

This reframes the duplicated-orientation downside, which wasn’t actually “two
brokers studying the identical recordsdata.” It was two brokers independently reconstructing
the identical psychological mannequin of the codebase, as a result of the work had been partitioned
by job moderately than by the information every job required. I’ve began calling
that distinction cognitive locality: Duties that want the identical psychological mannequin ought to normally keep collectively.
Splitting them simply forces a number of brokers to rebuild the identical understanding
from scratch.

Parallelism nonetheless issues right here, it is simply not the principle level. Working 4
brokers concurrently is beneficial, however bizarre. The true profit is that they
maintain noisy intermediate reasoning out of the principle thread and return solely what
it nonetheless wants. That is the isolation subagents are supposed to supply, and it
solely holds if the principle thread respects it.

My working perception now: that is what subagents are literally for. Not that
they save time, however that they allow you to offload reasoning the orchestrator
would not want to carry onto, so it has much less to hold and fewer competing for its
consideration. Get the isolation proper, maintain issues native by cognitive locality,
and subagents turn out to be the device that protects the orchestrator’s working reminiscence,
not only a value you tolerate for parallelism. That is a perception, although, not a
measurement. What I’ve truly measured is the opposite facet of it, the price of
getting the isolation improper.

Turning a Session into Standing Guidelines

My subsequent transfer was the apparent one. Encode the lesson into CLAUDE.md, the
standing instruction file each session masses.

It might have been simple to write down a big corrective coverage right here, however each
further line in a standing instruction file is a price paid once more on each future
session. So I compressed the repair into the smallest algorithm that addressed
the failures I might seen. Each is basically answering the identical query. Does
this piece of data, or this manner the work is cut up, earn a spot within the
orchestrator’s context?

1. Favor two to 4 brokers in a single wave. If the orchestrator desires 5 or
   extra, it ought to first ask whether or not duties sharing recordsdata or conventions must be
   merged.
2. Don’t ballot background brokers for standing when the reply could be given
   from what’s already identified. Don’t fetch a full transcript to reply a
   light-weight query.
3. Don’t enable repository-wide git operations inside concurrent agent
   prompts.
4. Deal with overlapping file possession as a consolidation sign, not a cue to
   spawn extra brokers.

None of those inform the orchestrator precisely what to do in each case. Every
one offers it one thing to test or ask itself earlier than performing, not a script to
run, and none of them is profound by itself. Their solely actual worth is that
they’re all aimed on the identical factor, preserving disposable reasoning disposable
and preserving room within the orchestrator’s context for what it wants later within the
session.

The Subsequent Mistake Would Have Been Extra Governance

A later session surfaced a unique hole. I had began the orchestrator
with specific abilities for the sort of work I wished, coding steering in a single
case, design steering in one other. I assumed that when a ability was energetic in
the principle thread, spawned subagents would comply with it mechanically. They do not.
A subagent would not inherit abilities energetic within the dad or mum session except the
orchestrator passes them alongside explicitly.

My first intuition was so as to add a confirm-before-spawn gate. The orchestrator
would cease, checklist which brokers it desires to launch and which abilities every ought to
load, and look forward to my approval.

I am glad I did not maintain that model. It solved the improper downside. I did not
have proof that unhealthy spawn plans had been slipping by means of for lack of a
affirmation step. I might found a lacking truth about ability propagation, and
that is a unique sort of hole. A common affirmation gate would have added
a round-trip to each comparable session, and earlier than lengthy I might virtually definitely
have began approving these prompts on autopilot.

At that time, I spotted I wasn’t actually bettering governance. I used to be
simply including one other ritual.

And it nonetheless would not have caught the true downside from earlier than, the
orchestrator polluting its personal context.

The narrower repair held up higher. Earlier than spawning, the orchestrator states
which energetic abilities are related to every agent’s job and factors the subagent
on the ability file to load, moderately than pasting the entire ability inline.
Affirmation is just required above the identical batch-size threshold already in
place, or when file possession is ambiguous.

That left me with a heuristic I now use greater than the rule itself: Earlier than
including a line to a standing instruction file, ask whether or not a fairly
competent orchestrator would make the appropriate determination as soon as it knew the one
lacking truth.

If sure, the rule ought to simply state the very fact. If the repair begins specifying a
determination process, resembling approvals, checkpoints, necessary steps, that is
normally an indication I am encoding course of the place a small clarification would have
finished the job.

I do not know but whether or not that heuristic survives tougher circumstances. For now it
stops me from turning each attention-grabbing incident right into a miniature
forms.

The place This Leaves Me

I haven’t got a settled view of how a lot governance is sufficient, and I do not
assume this piece earns one.

What I’ve as a substitute is a small flywheel, with a human nonetheless firmly within the
center of it. A session exposes a niche. Somebody has to note that it felt
improper, cease the work lengthy sufficient to examine it, resolve whether or not the issue is
actual or simply noise, and choose what deserves to turn out to be a standing rule. The
orchestrator can grade its personal session and floor clues, because it did right here, however
it can not make that judgment name itself. The selection of what to codify, what
to depart alone, and what could be an overreaction continues to be mine. The following
session then tells me whether or not that judgment improved the work or simply created
a unique sort of waste.

The artifact of this spherical is the present model of
my CLAUDE.md.
It is not a completed prescription. It is the state of the calibration after this
iteration. The thresholds in it, two to 4 brokers per wave, 5 as a
consolidation sign, match the work I used to be doing once I wrote them. I would not
current them as something like common constants, and I might be suspicious of
any orchestration write-up that did. They had been additionally calibrated towards Claude Sonnet 5,
and I have never examined how they maintain up towards others. A unique
mannequin would possibly fairly want a unique stability. A number of extra guidelines have gathered
within the file since. Deal with this file as a pattern, not a template, modify and optimize
it for no matter mannequin and workflow you are truly operating. What issues is not the
particular file, it is the behavior behind it: noticing a failure, asking what it truly
value, and writing the rule that will have caught it.

For years we optimized software program techniques round CPU, reminiscence, and throughput.
The primary wave of LLM tooling taught us to observe tokens. This session made me
suspect there is a third factor price watching in long-running agent workflows:
the standard of the orchestrator’s personal working reminiscence, the one useful resource that,
as soon as polluted, retains charging hire for the remainder of the session. I do not assume
that is a settled legislation but. It is a sample that held up within the classes I’ve
checked out up to now.

The tax I went on the lookout for was by no means on the subagents. It was on the
orchestrator, in what it selected to hold ahead. That is the query I carry
into each multi-agent design now: not what number of brokers to run, however what earns
a spot within the orchestrator’s context.

The questions I am left with are genuinely open:

• How do I measure this correctly, as a substitute of counting on the orchestrator’s personal
  account of its errors?
• When ought to a lacking truth go into the directions, and when does that flip
  into an excessive amount of course of?
• What is the subsequent orchestration mistake I am not seeing but?

I anticipate I am going to revise the file once more. That feels much less like a failure of
foresight and extra like the conventional value of working with a system opaque sufficient
that doubt itself turns into a part of the tactic.