Agent Orchestration for Real World Constraints

In 2024, the question everyone was asking about AI was: how good is this model? By 2026, that question has shifted: how well do these agents coordinate? The capability ceiling for single agents is real and well-documented. The teams moving fastest now aren't chasing better models. They're building better orchestration.

This post is a written companion to my March 2026 talk at AI Community Africa × MLOps Community Lagos, where I covered the five dominant orchestration patterns, how each one maps to production constraints, and what it actually takes to build agent systems that don't collapse at 3am. The full talk is on YouTube and the slides are here.

Why orchestration is the hard part

A single agent has a ceiling. Its context window fills up, it starts to hallucinate, it loses track of the original goal. This isn't a model quality problem, it's structural. The fix is decomposition: break complex work across multiple specialised agents, each operating within a manageable scope.

But moving from one agent to five doesn't multiply complexity linearly. Dependencies multiply. Failure modes compound. Latency stacks. Costs balloon. Orchestration, the coordination layer between agents, is where 80% of production failures happen. Getting it right is the difference between a demo and a deployed system.

The five orchestration patterns

There are five architectural patterns that cover the vast majority of production use cases. Each has specific strengths, cost profiles, and failure characteristics.

1. Hierarchical (Manager to Workers)

A coordinator agent sits at the top and delegates tasks to specialised workers. The coordinator doesn't do the work itself, it decomposes requests, assigns subtasks, and synthesises outputs. Workers execute independently and report back.

This is the most intuitive pattern for engineering teams because it maps to how human teams work: a project manager distributing work to specialists. It handles approval workflows naturally, supports human-in-the-loop gates, and is relatively easy to debug because execution paths are predictable.

The coordinator is also the single point of failure, which means it needs redundancy and health checks built in from the start. Cost multiplier: 4 to 7 times a single agent, because the coordinator adds a hop and every worker-to-coordinator message consumes tokens.

Real-world fit: financial compliance review, enterprise workflows requiring audit trails, any process where clear ownership and approval gates matter.

2. Hub-and-Spoke (Gateway Routing)

A central hub acts as an intelligent router, dynamically matching incoming requests to available specialist agents based on capabilities, current load, and availability. Unlike the hierarchical pattern, agents don't report back to the hub with results. The hub routes and gets out of the way.

The key advantage is graceful degradation: if one agent is unavailable, the hub routes around it. You can add or remove specialist agents without restructuring the whole system. It also centralises observability, giving you one place to monitor routing decisions, latency, and resolution rates across all agents.

Cost multiplier: 3 to 6 times, slightly more efficient than hierarchical because the hub can optimise which model handles each request. Route simple queries to cheap small models; reserve expensive reasoning models for complex tasks.

Real-world fit: customer support platforms with multiple specialisations, multi-tenant SaaS systems, any high-variability workload where you need to scale agents independently.

3. Sequential Pipeline (DAG Workflows)

Data flows through a directed graph of agents in a predetermined order. The output of each node is the input to the next. Decision nodes can branch to different paths. Failed nodes trigger retry or fallback logic.

This is the most deterministic and debuggable pattern. You can inspect state at every node, replay from a checkpoint after a failure, and reason about execution without tracing emergent behaviour. It's how most code generation tools work under the hood: parse, generate, validate, insert, verify.

Cost multiplier: 3 to 5 times, the cheapest multi-agent pattern because there's no redundant cross-agent communication. Each step passes its result forward and stops.

Real-world fit: RAG pipelines, document processing, ETL workflows, any task with a predictable and repeatable structure.

4. Collaborative Swarm (Role-Based Teams)

Multiple agents with defined roles work on a problem together, communicating peer-to-peer. A researcher gathers information, an analyst identifies patterns, a writer drafts output, a critic identifies gaps, an editor refines. Agents iterate through multiple rounds until convergence criteria are met, then a synthesiser produces the final output.

This is the most powerful pattern for creative and exploratory tasks, and the most dangerous to run without proper controls. Without hard limits on iteration count and per-workflow cost budgets, a swarm can run away from you fast.

The numbers: a 4-agent swarm iterating 3 times, at $0.01 per 1K tokens, costs around $0.12 per query. At 10,000 queries per day, that's $36,000 per month. A sequential pipeline doing equivalent work costs roughly $100 per month. Poor orchestration bankrupts you.

Cost multiplier: 10 to 20 times. Use this pattern only when the quality difference genuinely justifies the cost, and measure that difference before you assume it does.

Real-world fit: market research reports, complex analysis requiring multiple perspectives, creative work that benefits from critique and iteration.

5. Computer-Using Agents (Action, Observe, Reflect)

The agent executes an action (keyboard input, mouse click, API call), observes the outcome (screenshot, response), reflects on whether it moved toward the goal, and repeats. No API integration required, the agent works with existing systems through their interfaces.

This is the pattern behind Claude's computer use, Codex, and OpenCode's agentic mode. You can instruct it to monitor an inbox, identify a specific email, extract data, and forward a message, all through the UI, without writing a custom integration. It also means any UI change can break a workflow, and the broad action space creates real safety risks if not properly sandboxed.

Cost multiplier: 15 to 30 times, the most expensive pattern. Every observation loop involves screenshot encoding and LLM inference. Worth it for tasks where no API exists or where the flexibility outweighs the cost.

Real-world fit: legacy system integration, desktop automation, web scraping, invoice processing, any workflow that touches systems not designed for programmatic access.

Matching patterns to production constraints

Clean diagrams obscure production realities. Here's how each constraint actually plays out.

Latency compounds

In a 5-agent sequential pipeline, each agent adds roughly 200ms of network round-trip plus 2,000ms of inference. That's 11 seconds minimum before you've added queuing, retries, or tool calls, often 20 to 30 seconds in practice. This is worse when your users are far from model API endpoints: Lagos to US East adds about 180ms per hop, São Paulo about 120ms, Jakarta about 200ms.

Design responses: use small local models (0.5 to 3B parameters) for routing and classification instead of expensive frontier models; parallelise where agents don't depend on each other; cache embeddings and repeated queries aggressively; consider self-hosting for high-volume workflows where round-trip time matters.

Failure is not a special case

Production agents fail constantly: model timeouts, rate limits, malformed tool outputs, coordination deadlocks, context drift. The architecture must expect failure, not just handle it when it happens. This means circuit breakers that stop routing to failing components, state checkpointing so pipelines can resume from the last good step, exponential backoff on retries, and human-in-the-loop escape hatches when agents get stuck.

Correlation IDs are non-negotiable: every workflow gets a unique ID that appears in every log entry across every agent. Without this, debugging a multi-agent failure at 3am is not manageable.

Security is architectural

Every agent should be treated as a semi-hostile workload. Over-privileged agents, prompt injection through user input or tool outputs, agent-to-agent manipulation in swarm patterns, and supply chain risks in public agent registries are all real. 1,184 malicious agent skills were found in a public hub recently.

The principles: least-privilege IAM (an agent that reads a database doesn't get write access), tool allowlists per agent (no dynamic function discovery), runtime validation before high-risk actions, sandbox execution with network egress controls, and audit logs for every tool call that are immutable, append-only, and queryable.

Cost is non-linear

The cost table is worth keeping in mind when choosing a pattern:

• Sequential Pipeline: 3 to 5 times single agent
• Hub-and-Spoke: 3 to 6 times
• Hierarchical: 4 to 7 times
• Collaborative Swarm: 10 to 20 times
• Computer-Using: 15 to 30 times

Start with the sequential pipeline. Add complexity only when you've validated that it's necessary. A single agent with well-designed tools often outperforms a multi-agent system that was built because multi-agent sounded more impressive.

Data governance is a constraint, not an afterthought

GDPR, NDPR (Nigeria), HIPAA, PCI-DSS: regulated data often can't move across borders or between services without explicit controls. Naive orchestration with shared centralised state makes this hard to enforce. The better architecture uses federated data where possible (agents query data via controlled APIs rather than receiving copies), per-agent identity with scoped credentials, and regional deployment topology that keeps agents co-located with the data they process.

What constrained environments get right

One of the best exchanges from the Q&A was around whether teams in constrained environments, unreliable power, high API latency, currency volatility making cloud costs unpredictable, can really compete in this space.

The honest answer is that constraints breed engineering discipline that well-resourced teams often skip. A team in Lagos that optimises for latency because 180ms round-trips are their baseline builds faster systems than a Silicon Valley team that never had to think about it. Teams that self-host because data can't leave the country discover they have 70% lower latency than their API-dependent competitors. Teams that build offline-first because power is unreliable end up with more robust systems than teams that assumed connectivity.

The patterns that survive production aren't the ones with the most agents or the largest models. They're the ones built with the tightest constraints: clear failure handling, observable execution, cost discipline, and the judgment to use the simplest pattern that solves the problem.

Build with constraints. Test at 3am. Ship what survives.