Multi-agent demos look incredible. Multi-agent production systems are often disappointing: long latencies, unpredictable token bills, debugging nightmares. Many "multi-agent" architectures could be a well-structured single agent with good tool calls. The three patterns below are the ones that justify the coordination overhead, followed by the anti-patterns worth deleting.
First, decide whether you need more than one agent
If your task fits in one good system prompt and a small tool set, ship that first. A multi-agent rewrite can easily lower quality and multiply cost when the original single-agent system was already close to optimal. Multi-agent earns its complexity when:
- sub-tasks are genuinely independent and can run in parallel;
- sub-tasks have different cost-quality trade-offs, such as deep reasoning for planning and cheap volume for extraction;
- the context one sub-task needs would crowd out or confuse another's; or
- sub-tasks need different permissions and you want hard isolation between them.
Anthropic's guide to building effective agents makes the same argument: start with the simplest workflow that works and add agentic complexity only when it demonstrably helps.
Pattern 1: Planner → executors with strict handoff
A planner LLM decomposes a task into a small set of discrete sub-tasks, then dispatches each to a specialised executor, sometimes a smaller, cheaper model. Handoff is structured JSON, never free-form chat. This works because the planner needs reasoning depth and the executors need volume and predictability — two different cost-quality curves. Don't skip the structured handoff; it's the difference between a debuggable system and a séance. A handoff can be as small as:
{
"task_id": "fin-extract-01",
"instruction": "Extract revenue and net profit for the last two financial years",
"inputs": ["doc_17", "doc_18"],
"output_schema": "FinancialSummary.v2",
"max_steps": 6
}The planner validates each executor's output against the named schema before composing the final answer, and retries or flags anything that fails.
Pattern 2: Critic loops with a hard iteration cap
A generator produces output; a critic with different prompting (and ideally a different model) scores or revises it. Cap iterations at two or three. Beyond that you are paying for diminishing returns and adding latency users notice. The critic should be cheaper than the generator, not more expensive; an inverted critic cost is a red flag in design reviews. Give the critic a rubric, not a vibe: specific checks such as "every figure cited", "no claims outside the sources" and "schema valid" produce consistent scores, while "is this good?" does not.
Pattern 3: Specialist routing via a thin classifier
A small, fast classifier routes incoming requests to the right specialist agent (legal, finance, support). Routing is deterministic where possible: keyword and metadata matching first, LLM classification only on ambiguous traffic. It is common to find a frontier model doing work a regex could do, and then wonder why the unit economics don't work. Log the routing decision and its confidence so misroutes can be found and fixed.
Choosing between the patterns
| Pattern | Use when | Main cost | Key control |
|---|---|---|---|
| Planner → executors | The task splits into independent, checkable parts | Planner tokens and orchestration latency | Structured JSON handoff, validated |
| Critic loop | Quality varies and can be scored against a rubric | An extra generation per iteration | A hard cap of two or three iterations |
| Specialist routing | Request types need different prompts, tools or permissions | Misroutes | Rules first, LLM for ambiguous cases, logged confidence |
The anti-patterns we keep deleting
- Open-ended agent conversations. Two agents "discussing" a task until they agree. Burns tokens, drifts off-topic, debugs poorly.
- Recursive sub-agent spawning. Sub-agents spawning sub-agents. Latency compounds. Cost compounds. Failure modes compound.
- Agents as an org chart. A "manager" agent, a "researcher" agent and a "writer" agent because it looks tidy on a diagram. Split by data, permissions and cost profile instead of job titles.
- Shared free-text memory. Agents writing into a shared scratchpad. Fine in research, dangerous in production because state changes are invisible.
Observability is non-negotiable
Every agent boundary needs a span. Every handoff needs a structured trace. If you can't replay an end-to-end run with all sub-agent inputs and outputs, you can't debug production failures. Build the trace before you build the second agent. The OpenTelemetry semantic conventions for generative AI give you vendor-neutral names for those spans and attributes, so traces from different frameworks line up. For each run, record tokens and cost per agent, latency per step, iteration counts and the final outcome, so you can see which agent is expensive and which is slow.
Budgets and failure handling
- Set a maximum number of steps, tokens and seconds per run, enforced by the orchestrator rather than the model.
- Decide in advance what happens when an executor fails: retry once, fall back to a simpler path, or return a partial result with a clear flag.
- Make executors idempotent where they write data, so a retry cannot create duplicates.
How we approach this at Velura Labs
Our Agentic Systems practice builds multi-agent designs when they earn their keep and pushes back when a single agent would do the job better. Pair it with Custom LLM Applications for the base layer and AI Strategy & Roadmap to decide where agents fit in your roadmap. Read our agent framework piece for the tooling layer and our evaluation playbook for how to measure these systems honestly. Talk to us before you commit to a multi-agent rewrite — often the right answer is "fix the agent you already have".
Available to businesses across the United States (Washington, California, Texas, New York), Europe (France, Italy and the wider EU), the Middle East (Dubai and the Gulf) and India. Get in touch to scope your build.