Artificial intelligence companies are increasingly building systems in which multiple AI agents work together on complex tasks. One agent may handle the database, another develop the user interface, and a third test the final product. The approach promises faster development and better results by dividing work among specialised AI systems. However, new research from AI evaluation company Vals AI suggests that this strategy can significantly increase costs without consistently improving the quality of the final output.
In a study published in October 2026, Vals AI tested GPT-6 Sol and Claude Opus 5.5 on its Vibe Code Bench, comparing individual agents with teams of agents across 50 web application-building tasks. Teams cost between 1.8 and 5.1 times more than single agents. Only one of four team-versus-solo comparisons produced a statistically significant quality improvement, raising questions about whether developers should routinely use multi-agent systems when a single capable model may already be sufficient.
Key Takeaways
- Vals AI found that multi-agent teams cost 1.8 to 5.1 times more per application than single agents in its benchmark.
- Only one of four comparisons showed a statistically significant quality improvement.
- GPT-6 Sol’s medium-reasoning team improved its score by 7.3 percentage points, but its cost rose from $1.22 to $3.07 per application.
- Claude Opus 5.5’s maximum-effort team cost an average of $122 per application, compared with $23.77 for its single-agent counterpart.
- More agents can improve speed or help with parallel tasks, but coordination and repeated context can reduce cost efficiency.
- Developers should measure cost, quality and completion time before adopting multi-agent architectures.
What Did the Vals AI Study Find?
Vals AI evaluated two AI models, GPT-6 Sol and Claude Opus 5.5, under two reasoning settings: medium effort and maximum effort. Each model was tested as a single agent and as a team led by a primary agent that could delegate work to as many as five subagents at once.
The benchmark involved building complete web applications from product specifications. Applications were deployed using Docker Compose and evaluated through browser-based agents that tested their user interfaces and workflows. This design allowed researchers to measure practical application quality rather than relying only on a model’s written explanation of its work.
The central finding was that teams did not consistently outperform individual agents. Across the four comparisons, only GPT-6 Sol at medium reasoning effort delivered a statistically significant improvement. The other three comparisons showed differences that were not statistically significant.
| Model and reasoning level | Single-agent score | Team score | Single-agent cost | Team cost |
|---|---|---|---|---|
| GPT-6 Sol, medium | 77.6% | 84.9% | $1.22 | $3.07 |
| GPT-6 Sol, maximum | 89.0% | 90.4% | $4.82 | $8.54 |
| Claude Opus 5.5, medium | 91.5% | 91.2% | $4.08 | $9.10 |
| Claude Opus 5.5, maximum | 89.8% | 93.2% | $23.77 | $122.00 |
Source: Vals AI’s Vibe Code Bench study. Scores represent the average share of application UI tests passed; costs are average costs per application.
The results show that a higher team score does not automatically establish a meaningful improvement. For example, Claude Opus 5.5’s maximum-effort team scored 93.2%, compared with 89.8% for the single agent. However, the study did not find that difference statistically significant, while the team’s average cost was more than five times higher.
Why Do AI Agent Teams Consume So Many Tokens?
The main cost driver is the extra work required to coordinate multiple agents.
A lead agent typically receives the original task, breaks it into smaller assignments, sends instructions to subagents, reviews their results and combines the work into a final product. Each subagent must also understand enough of the original requirements and surrounding context to complete its assignment.
That context can be expensive. Agents may repeatedly receive specifications, project files, interface requirements, API details and previous decisions. Even when some input tokens qualify for discounted cached processing, they still contribute to overall token consumption and may carry a financial cost.
Vals AI highlighted this effect in its Claude Opus 5.5 maximum-effort experiment. Across applications, subagents read a median of 224 million cached tokens per app, compared with 17 million for the lead agent and 55 million for the single-agent setup.
These figures illustrate how a team can generate enormous amounts of model activity without producing a proportionate improvement in the final application.
The hidden coordination cost
Multi-agent systems can also duplicate work. Two agents may independently investigate the same problem, implement overlapping functionality or make decisions that conflict with each other. The lead agent then has to resolve those conflicts, potentially creating additional rounds of work.
Software development is particularly sensitive to coordination. A database, application programming interface and user interface may be developed separately, but they must still agree on data formats, endpoints, error handling and authentication. A mistake at the boundary between components can undermine otherwise successful individual contributions.
The additional work can become worthwhile when parallel development saves enough time or prevents costly errors. The problem arises when the system creates more coordination overhead than useful output.
Higher Reasoning Effort Can Be Better Than Adding Agents
The Vals AI results suggest that developers should compare multi-agent architectures with another option: allowing one agent to spend more effort on the task.
For GPT-6 Sol, increasing reasoning effort from medium to maximum raised the single-agent score from 77.6% to 89%. That was an improvement of 11.4 percentage points. By comparison, using a team at medium effort raised the score by 7.3 percentage points.
The maximum-effort single agent cost $4.82 per application, while the medium-effort team cost $3.07. The single agent therefore cost more than the medium-effort team, but it achieved a higher average score in this experiment. The score difference between these two configurations was not statistically significant, so the results do not establish that one is universally superior.
Claude Opus 5.5 showed a different pattern. Its medium-effort single agent scored 91.5%, while the maximum-effort single agent scored 89.8%. The study did not find a significant improvement from raising effort alone. The maximum-effort team reached 93.2%, but its average cost increased to $122 per application.
These differences show why AI deployment decisions should be based on measured results rather than a general assumption that more agents or more reasoning will always produce better work.
More Agents Can Help, but the Benefits Depend on the Task
The findings should not be interpreted as proof that multi-agent systems are ineffective. Their value depends on the type of work, the model’s capabilities, the way tasks are divided and the importance of speed.
Tasks that can be separated into relatively independent parts may benefit from parallel execution. For example, one agent can research competitors while another analyses financial data and a third prepares a draft. A lead agent can then combine the outputs, provided the subtasks have clear boundaries and the results can be checked efficiently.
Other tasks are harder to divide. Creative writing, complex planning and software changes involving tightly connected components may require a consistent understanding of the entire problem. Splitting the work can introduce contradictions or force the lead agent to spend considerable effort reconciling different approaches.
Vals AI also found differences in completion time. GPT-6 Sol’s medium-effort team finished in a median of 11.9 minutes, compared with 12.4 minutes for the single agent. At maximum effort, the team completed tasks in 27.6 minutes, against 38.4 minutes for the solo setup.
Claude Opus 5.5 behaved differently. Its teams took longer than the corresponding single-agent setups, with the maximum-effort team taking a median of 174.1 minutes compared with 74.7 minutes for the solo agent.
The practical lesson is that teams may be valuable when parallelism matters, but their impact on both speed and quality must be measured for the specific workflow.
Anthropic’s Research Also Points to Diminishing Returns
The Vals AI findings are consistent with results described in Anthropic’s own agent-scaling evaluations. In examples reported by The Decoder, increasing the number of agents from 10 to 100 produced relatively small additional quality gains on two tasks.
| Task using Claude Opus 5.5 | 1 agent | 10 agents | 30 agents | 100 agents |
|---|---|---|---|---|
| Knowledge base | 0.53 | 0.70 | 0.71 | 0.74 |
| Lean theorem proving | 0.39 | 0.66 | 0.66 | 0.68 |
The pattern is notable: the largest improvement came from moving beyond a single agent, while adding more agents after the initial increase produced much smaller gains. The results also show that scaling behaviour can vary across tasks and models.
These evaluations are not identical to the Vals AI benchmark, and their scores should not be treated as directly comparable. Together, however, they highlight a common challenge: adding agents can eventually produce diminishing returns as coordination costs and overlapping work increase.
What Developers and Businesses Should Do
Companies deploying AI agents should begin with the simplest architecture that can reliably complete the task. A single agent is often easier to evaluate, debug and manage because it maintains one main context and does not require extensive coordination among multiple workers.
Teams should then be introduced where there is a clear reason to split the workload. A practical evaluation should compare the same tasks using a single agent and a team, measuring the following:
- Quality: How many tests pass, and how often does the result meet the actual business requirements?
- Cost: What is the total model and tool expenditure per successful task?
- Speed: Does parallel work meaningfully reduce completion time?
- Reliability: How often do agents produce conflicting outputs or require human intervention?
- Repeatability: Do the results hold across multiple runs rather than a single successful attempt?
Businesses should also set spending limits, monitor token consumption and avoid launching subagents without clear assignments. If a team architecture is retained, each agent should have a defined responsibility and a measurable contribution to the final result.
The Bigger Picture
The AI industry is moving beyond the question of which model is most capable toward a more practical question: which system delivers the best result for the money and time invested? Multi-agent architectures can make complex workflows more flexible, but every additional agent introduces potential costs in context processing, coordination, integration and verification.
Vals AI’s benchmark is a reminder that capability and efficiency are different measures. A team can perform more computation, use more tokens and generate more intermediate work without producing a reliably better application. For businesses, the relevant metric is not the number of agents deployed but the cost of obtaining a correct, useful result.
Looking Ahead
Future AI systems may improve coordination, share context more efficiently and decide when delegation is genuinely useful. Better orchestration could reduce duplicated work and make agent teams more attractive for complex projects. More extensive testing across different software tasks, research workflows and business applications will be needed to determine where those improvements translate into measurable benefits.
Until then, developers should treat multi-agent systems as a tool rather than a default architecture. Starting with a single capable agent, measuring its performance and adding specialised agents only when the evidence supports doing so can help control spending without sacrificing quality. The strongest AI workflow will be the one that consistently delivers the required result—not necessarily the one that uses the most agents.
Frequently Asked Questions
1. What did the Vals AI study discover?
Vals AI found that teams of AI agents cost 1.8 to 5.1 times more than single agents on its web application benchmark, while only one of four comparisons produced a statistically significant quality improvement.
2. Which AI models were tested?
The study tested GPT-6 Sol and Claude Opus 5.5, each in single-agent and team configurations at medium and maximum reasoning effort.
3. Why do agent teams use more tokens?
Agents often repeat context, work on overlapping tasks and exchange instructions and results. Subagents also need sufficient information to complete their assignments, increasing total input-token consumption.
4. Are multi-agent systems useless?
No. They can help when a task divides naturally into independent subtasks or when parallel execution reduces completion time. Their value depends on the workflow, coordination design and budget.
5. Should developers use one AI agent or multiple agents?
Start with one agent when it can meet the requirements. Add agents when measured improvements in quality, speed or reliability justify the additional cost.
6. What is the main business lesson?
Measure the cost per successful result rather than counting agents or tokens alone. The best system balances quality, reliability, speed and expenditure.
Get the day’s top stories in your inbox
One concise email. No spam, unsubscribe anytime.
