Horizontal and Vertical Agent Orchestration

by

As an added benefit of having a sandbox orchestration environment, in the past few months I have been heavily experimenting with different forms of scaling in regards to agentic workloads. In this article I will go through the basics, how I started the experimentation, specifically with development and deep research workloads, and where I ended up with my current flow. Hopefully, if you are also getting yourself into this world, you will find this useful.

Before we get started, to address the basic question first:

 

Why multiple agents?

Wouldn't having general AI models with wide expertise mean we should be able to get where we want in a single agent inference loop? The answer goes back to the fundamental limitation of working with LLMs as the technology currently stands. Where we are, almost the biggest limitation of working with LLMs is bound to context length: basically, how much information can we pass through the model before it starts generating a response to that “context”.

In the past few years we have come a long way already. right now the most capable models support up to 1M tokens of context window. To put that in perspective, that is around 2000 pages of text for each completion request to the model. While that sounds like a lot of headroom, the reality is that the context window is already taken up by growing metadata related to tools, system instructions, images, files, the history of the conversation, and even the thinking window of the model and the output tokens. So if you have been using agents for the past year, you know that a session usually hits compaction multiple times, which is the indicator that your session has passed the length of the context window as is.

So with agents, what we ideally want to achieve are context windows that are specialized in a particular topic they only care about, and their knowledge base is tuned for, a subset of a problem rather than the whole pie. And with that, the idea of sub agents was born. The side benefit, of course, is that you can also achieve a great deal of input token optimisation, and reduce the risk of the AI side-tracking. For example, if your main agent needs to run an investigation of a particular feature in your code base, the sub agent can start again from a fresh context window with the instruction of a more narrowly defined sub problem, and upon finishing the investigation return only the result of it, rather than each step in the investigation cluttering our context.

 

Sub Agents

Anthropic introduced sub agents in mid 2025, and team agents as their potential evolution in 2026. In the age of AI that's a lifetime we've been given with a technology, and dare I say it should already be mature compared to many fresh features we got in the meantime. With Anthropic the idea was simple: within the same execution scope, the main agent runs one or multiple sub agents that are specialized in a given sub task.

What comes inherently with this approach is sharing the same sandbox or machine resources and tooling with the sub agent. Although, given the task, the main agent can decide to run the sub agent on a different model for cost optimisation and latency purposes.

The evolution to sub agents comes as “team agents” with a direct line of communication established between our sub agents, where they can discuss and collaborate on a given task, inform one another of progress and share resources between one another. Team agents can really act like a team, for example, backend, frontend and QA engineers in a software development task.

This is what I refer to here as vertical orchestration: a parent agent with one or more sub agents that may or may not have cross communication. For me personally, the use of sub agents came around more naturally than team agents, especially when it comes to investigation or deep research tasks.

 

Horizontal scaling

On the other hand, Horizontal scaling fundamentally depends on cross communication between specialized agents. The idea here is to have agents that are not only not running on the same executable, but that might not even share the same file system, machine or network entirely. These agents are assumed to have different capabilities and tooling available to them, and each might have access to certain hardware that comes with its own tradeoff between cost and capability. And most importantly, they don't even necessarily need to be running on the same technology or coming from the same AI lab, like pairing Codex and Claude Code together.

Horizontal scaling is not aiming to solve the cost saving problem, but rather, in addition to specializing our agent's context and memory, to have separation of concerns in a given workload. If you have been spending more or less the same time as I have with agents, you already know that one fundamental problem of working with agents is that they tend to cut corners and find the shortest path to a given problem, even if that means bending the definition of the problem.

Having specialized agents that really care about one topic, let's say code quality, means that if another agent wants to push code with a heavy aim of just solving the issue, together they can find the best path to solve the problem while keeping the code quality high. And our code quality agent does not really need to be ephemeral for the one problem we are trying to solve at a given time. That also means specialized memory for the agent, file system, or even access, like code repository, cloud account and so on.

 

 

Lack of a widely accepted horizontal orchestration standard

While I started exploring this topic, I found very little publicly available initiative for cross agent spawning and orchestration. That's not really surprising. On one side, the lack of a standard sandboxing environment when it comes to coding agents means we can't really expect the same level of power and flexibility when it comes to spawning new agents, wake ups and having them easily live alongside one another. On the other hand, the communication would then demand implementing a hub point. Luckily for me, since I was already working quite heavily at this point on the sandboxing problem, I already had a strong base to work on. As part of the work on agent sandboxes in outgate.ai (https://outgate.ai/en/capabilities/agent-harness) I had already built a programmatic environment where I could create agent sandboxes in different environments and regions, within multiple levels of encapsulation:

  • Regions: these are the networking and virtual machine environment, so within a region you have shared network infrastructure, mainly speaking. A region here could be deployed on local hardware, which was key for me.
  • Organization: a separation of resources within a region, although an org can have access to multiple privately and publicly accessible regions at the same time.
  • Users: a subset within an org with specialized connections, like GitHub, Gmail, Google Drive and so on. Users also share the agent memory for a given region.
  • Project: has separation of file system and gets specialized on a larger picture.
  • And at the end, the sandbox, or thread, where we have the agent actually running in a container.

This level of granularity meant I could really have specialized agents that were being worked on and that built a very specific memory, given the specialized tools available to them.

On the flip side, I wanted to be able to have full flexibility: to be able to, if needed, have any agent communicate with any other agent throughout the platform. Thankfully the work of the past really paid off here and most of the groundwork was already done, programmatic orchestration and spawning of the agents, file upload and download, the “thread app” structure, which means the agents and each sandbox can actually have webhooks or other means available to them, and the scoped integration and tooling from the admin dashboard.

With that, the missing pieces were twofold: an active communication channel, built on top of the existing app and webhook structure, and the agent spawning, the ability to give an agent permission and boundaries for running other agents, given their scope in the same or another project, or even a different region. In my design, since the spawning of an agent is an automatic process, I kept it within the same user so we could avoid the potential privacy hell that might follow. And the agent comms, since the user has to manually get the comms file from one sandbox to the other, can go beyond the same org and could potentially be set up across private and public regions from completely different organisations.

Early experiments of course hit the usual issue of agents echoing to one another and getting stuck in an endless loop of asking things of one another.

Echo loops were only the most visible failure mode. The less obvious ones were stale state, duplicated ownership and acknowledgements that created another run without adding new information. The communication layer therefore needed more than message delivery: it needed idempotency, clear task ownership, bounded retries and a rule that silence can represent closure.

This was later solved with the addition of messages in the middle of a running agent loop, tightening down the system instructions, and also a run-and-share-the-result approach, plus multiple queueing systems for incoming and outgoing messages. Once the cleanup work was done, I actually ended up with a fully workable system that could really horizontally scale my workflow across agents on different machines.

 

Examples

I worked with multiple examples of workflows that really benefited from this architecture. The ones that really stand out for me:

Projects with different GitHub connections. This was related to multiple projects I have been working on where I wanted to have certain knowledge shared across agents. After asking one project to set up cross thread comms and sharing the config/secret (ogpkg) file, this file contains the credentials and tenancy information for the other agent to set up comms, with the other agent, I was amazed to see how well they collaborated to collect the knowledge files, zip them, upload them and rigorously validate them on the other side before starting to put them to use.

The big observation was how little trust the two agents had toward one another, something you don't really see in the sub agent model, and how deeply they investigated each other's claims and provided data. I see that as a really good and useful feature where you need it, you don't want the agents in a system to just blindly accept one another in many cases.

Cloud plus physical hardware. The other project was a combination of cloud work and physical hardware. I have a private region on quite a strong machine equipped with a 5090 GPU, and each sandbox is able to allocate much larger resources for cross compilation and potentially running more tooling. The idea here was to be able to run an orchestrator agent in the cloud with hourly execution to fetch stock market metrics, and then cross communicate with the agent on the GPU machine to get time series predictions (TimesFM) for the metrics it had gathered, and add the predictions to the action catalogue.

Deep research with independent critics. For a research workload, I used one agent to collect and structure the evidence while another independently challenged its claims and checked the cited sources. A vertical research subagent is effective at exploring one narrow branch without consuming the main context window. The horizontal version adds something different: the reviewing agent has its own memory, standards and incentives, and does not inherit the researcher’s chain of assumptions. The value was therefore not only parallelism, but epistemic separation.

I believe ultimately what this cross comms across agents and tooling stands for is to modify the model we currently have, of an LLM host beside an agent as the execution arm, into an LLM model, plus an agent orchestration layer, plus another layer of execution arms that can be shared on demand across the orchestration layer. This would bring flexibility and separation of concerns that the former model does not offer.

 

Communication protocol

Although I think this is aside from the point of this exercise, it might be worth going through, very briefly, how this is achieved technically. There are multiple ways communication between agents can be achieved. The biggest contestant has been the agent to agent (A2A) protocol, and more custom MCP connections through standard channels (WebSocket, HTTP and so on).

I opted for the custom MCP using HTTP, with push–push and push–poll models as a fallback. This was mainly due to how much A2A's requirements aim for untrusted environments, and the overhead that added to the orchestration layer, while what I was trying to achieve was simply adding a trusted connection between sandboxes whose connections the organizer approves, hence we don't really need all the extra overhead. The decision to use REST also moves toward the same logic of avoiding a broker model and simplifying where possible.

Where both sandboxes are on publicly exposed networks, the webhook was already part of our infra and the two sandboxes can push events to one another on demand; where that is not possible, the one that has public access creates a polling endpoint for the other to call periodically. Keep in mind we are assuming the hub sandbox has at least a publicly exposed (or same-network exposed) endpoint.

 

Hybrid model: the right move forward

To me it's clear we are just at the beginning of the agent workflow movement, and we are moving toward a more seasoned approach where we, as the supervisors of the agents, need to keep a layer of transparency while maintaining as much automation as we can toward more sophisticated and fully automated workflows. This much is very clear to me: toward that goal, in addition to vertical and horizontal scaling, we will have passive observers, circuit breakers and many more models we haven't even thought of just yet. Although, being where we are, the level of improvement in the workflow that can be achieved with relatively simple additions like horizontal scaling makes where we are so exciting.

I think ultimately you, as the owner of the task, should decide whether the task you are trying to solve has the characteristics of each scaling model. Does it easily break into smaller, from-scratch subtasks? Then maybe sub agents are the right choice. Does it need specialized fresh sub agents that coordinate on the fly? Then team agents are even better. Or if you need long-lived, specialised, domain-specific agents, then go horizontal, or maybe all of the above is the ideal topology for the pipeline or the solution. This, in addition to being something for us to get familiarized with, is a very good place to start scoping, evaluating and making the right decision before committing to progressing with the task.