Imagine a customer asking to change a delivery address. One AI agent reads the request. Another checks the account. A third examines the order. A fourth prepares the response. A coordinator decides whether the job is finished.
The diagram looks reassuring. There are specialists, responsibilities and a manager. It resembles an organisation we already understand.
But the customer still needs one thing: the correct address attached to the correct order, before the parcel leaves. Every additional conversation inside the system has to justify itself against that result.
That is the useful question behind multi-agent AI, where several AI systems divide and coordinate a task. When does the division of labour improve the work, and when does it simply create more work to coordinate?
A new benchmark puts the architecture under pressure
In the November proceedings of AACL-IJCNLP 2026, ServiceNow researchers Tara Bogavelli, Hari Subramani and Roshnee Sharma present AgentArch. They tested 18 configurations across six models on two simulated enterprise workflows.
The highest success rates under the benchmark’s criteria were 70.8% for requesting time off and 35.3% for routing customer requests. Passing required correct tools, correct arguments and the correct final decision together. Each workflow contained 60 requests, with eight trials per configuration.
Both best scores came from single-agent configurations. Other model–task pairs benefited from multi-agent designs. These are results for the tested systems, including GPT-4.1 and Claude Sonnet 4, rather than a verdict on today’s frontier or a measurement of deployed companies.
The best tested configuration reached 70.8% for time-off requests and 35.3% for customer-request routing. These are different simulated tasks, not a controlled estimate of the effect of adding agents. Source: AgentArch, Table 2.
The interesting implication is that model selection and system design cannot sensibly be separated. Buying a more capable component does not settle how information should move around it.
For an executive, this changes the procurement question. “How many agents do you use?” reveals very little. “What can this configuration complete reliably, under our conditions?” gets much closer to the investment decision.
Which tasks can be divided?
Consider two assignments.
The first is to investigate a possible new market. One researcher studies customers, another examines competitors, and a third explores regulation. Each can make progress independently. The final synthesis benefits from bringing those perspectives together.
The second is to amend a customer’s order. The delivery address, stock position, payment status and dispatch deadline belong to the same changing situation. If different agents work from different snapshots, their individually plausible answers can combine into an invalid action.
These are illustrative cases, but the distinction matters. A research team can often divide the search space. A transaction needs a consistent account of what is true now, what has already happened and what is authorised next.
Anthropic’s June 2025 account of its multi-agent research system describes precisely why parallel exploration can help: separate agents investigate different directions with their own working context. The company also reports coordination challenges and substantially higher token consumption. Those observations concern its research implementation, not a universal price or performance ratio.
Independent subtasks can be combined; coupled work requires an accurate shared state. Conceptual illustration.
A business should therefore examine the relationships between subtasks before choosing an architecture. Can they proceed independently? Do they share a record that changes? What must arrive intact at the handover? Who resolves a disagreement?
Giving agents job titles does not answer those questions. A “finance agent” might still lack the information needed to interpret an exception. A “reviewer” might repeat the same error as the system it is checking. Separation becomes useful when it creates a real difference in information, capability or accountability.
The cost arrives between the boxes
The visible cost of an AI system is often the model call. The less visible cost sits between calls.
Someone or something must pass on the relevant facts, preserve their meaning, wait for missing results, reconcile contradictions and decide whether to retry. A summary can save time while losing the exception that changes the decision. More context can preserve detail while making the decisive fact harder to find.
This is familiar organisational territory. Human teams also struggle with handovers, duplicated work and responsibility that becomes less clear as more people participate. AI makes it possible to create those coordination structures almost instantly. It does not make their consequences disappear.
There are good reasons to add specialists. A worker with a restricted toolset can have a narrower role. Independent checks can catch failures. Parallel tasks can reduce elapsed time. The question is whether those benefits survive measurement after the coordination costs are included.
Anthropic’s earlier guidance on building effective agents, first published in December 2024, makes a durable distinction: predefined workflows suit settled paths, while agents can choose their next steps when flexibility is required. Its recommendation to begin with simpler designs remains a useful starting hypothesis, not a substitute for testing a particular application.
What should earn another agent?
Before adding a specialist, define the outcome precisely enough to compare alternatives.
For the delivery example, success would include the right order, the right address, a permitted change and a confirmed update. A polished message telling the customer that everything is done should not count if the underlying record remains unchanged.
Then compare a simpler baseline with the proposed team on the same representative cases. Include incomplete information, conflicting records and situations that should be handed to a person. Keep those cases out of the material used to tune the system.
Measure more than the fraction of correct answers. Record completion time, total cost, human repair effort and the severity of failures. Repeat the cases: an impressive first run can conceal inconsistent behaviour.
Compare quality, cost, time and reliability against the same simpler baseline. Include human repair effort and failure severity in the review.
My preferred test is practical: what specific failure does the extra agent remove, and what new failure does the handover introduce? If the team improves a meaningful outcome within acceptable limits, keep it. If its advantage disappears when review time is counted, redesign it. Where evidence is inconclusive, retain the simpler baseline while investigating.
Different organisations will set different tolerances. A drafting assistant and a system that changes customer records should not face identical deployment decisions.
Build an organisation that understands its own automation
In October’s BCM brief, I argued that useful AI should leave something dependable behind. Architecture testing gives that idea an operational form: retained test cases, known failure patterns and a clear account of why a particular division of work helps.
This matters when models change. An organisation that knows only which product it bought must rediscover much of its reasoning at the next upgrade. One that knows why its workflow works has a stronger basis for choosing what to replace.
There is room here for ambitious AI teams, and for small, carefully bounded systems. The advantage comes from matching the design to the work and being able to demonstrate the match.
Before adding more agents, write down the job. Then make the architecture earn its place.
Follow us on Youtube and LinkedIn.
1. Bogavelli, Subramani and Sharma. AgentArch: A Benchmark for Evaluating Agent Architectures in Enterprise Workflows. AACL-IJCNLP 2026, proceedings dated 6–10 November 2026. Proceedings entry ; Primary paper . Numbers from Table 2; methods from sections 3.1–3.2. Simulated workflows, not measured enterprise production deployments. Checked 10 November 2026.
2. Anthropic. How we built our multi-agent research system. 13 June 2025. Research account . Vendor account, used for mechanisms and trade-offs rather than a general performance claim. Checked 10 November 2026.
3. Anthropic. Building effective agents. First published 19 December 2024, with a current note that tooling has changed. Architecture guidance . Used for enduring architectural definitions, not current product recommendations. Checked 10 November 2026.
4. Gonçalo Perdigão. AI & Creativity Brief — October 2026: Beyond the Agent. 3 November 2026. October BCM brief . Internal editorial connection.





