Does adding coding agents make performance optimization faster? I gave one Astra agent and an Astra coordinator with GPT-5.6 Sol children the same starting code and three hours each in Codex.
After rebooting the servers, the solo run scored 1,636,308, compared with 885,334 for the team: about 1.85 times higher. Yet the team reached 500,000 points roughly 27 minutes earlier. Neither the final score nor a favorable midpoint tells the whole story.
The measured task was the ISUCON11 qualifier. I also considered ISUCON14, but these are different applications and the results should not be mixed. Here is the background for readers new to the competition.
What is ISUCON?
ISUCON is a competition in which participants improve an existing web service on limited server resources while preserving its required behavior. Changes can include reducing database queries, making computation cheaper, caching results, or redistributing work between servers.
A benchmark program simulates users, checks behavior, and assigns a performance score. The points in this article are its output, not raw requests per second. The scoring rules depend on the problem.
The measured problem was ISUCONDITION, from the ISUCON11 qualifier. It receives chair condition data and displays histories, graphs, and trends. Continuous writes compete with reads and aggregation, making databases, CPU work, caches, and server placement useful optimization targets. See the official application manual.
For context, ISUCON14's ISURIDE is a different problem: a fictional autonomous-chair dispatch service involving matching and location updates. Its official manual explains the service, but ISUCON14 was not run in this comparison.
I wanted to compare the ability to deploy working changes and produce measured improvements, beyond producing convincing explanations.
Eight EC2 instances, two independent environments
Each arm received three competition servers and one benchmark server. The two environments ran simultaneously, without sharing application databases or benchmark machines.
| Condition | Solo | Coordinated team |
|---|---|---|
| Parent model | Astra / high | Astra / high |
| Children | None | GPT-5.6 Sol / high |
| Optimization time | 180 minutes | 180 minutes |
| Independent trials | 1 | 1 |
| Competition servers | 3 × c5.large | 3 × c5.large |
| Benchmark server | 1 × c4.xlarge | 1 × c4.xlarge |
| Starting code | Same commit | Same commit |
The model identifiers in the logs were gpt-6-astra and gpt-5.6-sol. Both used high reasoning effort and Codex CLI 0.154.0. The window was September 13, 2026, 00:09:35–03:09:35 JST. Setup and baseline measurement happened beforehand. This was an equal-time comparison, not an equal-token-budget comparison.
ISUCON14 originally used an 8-vCPU, 8-GB ECS Fargate benchmark environment. For this experiment's three-competition-plus-one-benchmark EC2 layout, I used the official ISUCON11 machine configuration. It is distinct from the ISUCON14 configuration.
The original official AMIs were unavailable. I used matsuu's reconstructed environment, matching the instance types, counts, and 20-GB gp3 storage. This was not a bit-identical recreation of the original competition image. Physical CPU models also differed despite matching instance types. Baseline scores were 1,678 and 1,874.
How the team was organized
The team received an explicit operating policy:
- Start with separate investigations of the database, APIs and requirements, and runtime behavior.
- Split implementation by testable hypothesis. Allow at most two implementation tasks and five children overall at once.
- Give implementation workers separate Git worktrees and focused starting context.
- Return summaries and files for the parent to integrate.
- Serialize deployment, initialization, benchmarking, and log collection within each environment.
A Git worktree provides a separate working directory for code changes. It does not isolate a database or a running service, so code isolation and environment locking were separate mechanisms.
The executor used the existing isucon-harness make aws-deploy and make aws-bench-only commands. It pinned each candidate commit and checked the running executable hashes on all three application hosts before measuring. Decisions used public manuals, benchmark output, and our own logs. Benchmark source code was not read.
The team led in the middle; solo overtook it late
This chart shows the best normal passing score observed by each time, excluding diagnostic profiling runs, failed runs, and the post-window reboot benchmark.
| Threshold | Solo | Team |
|---|---|---|
| 100,000 | 1:06:47 | 1:16:09 |
| 250,000 | 2:17:10 | 1:58:54 |
| 500,000 | 2:31:05 | 2:04:05 |
| 1,000,000 | 2:38:23 | 2:37:29 |
The team reached 500,000 about 27 minutes sooner. At one million, its lead had narrowed to 54 seconds. Solo then made larger gains.
The solutions differed. Solo grouped writes, separated database, ingestion, and presentation responsibilities across the three servers, and adjusted trend caching late in the run. The team distributed condition data across two databases and gained ground by reducing the columns fetched for graphs.
Solo's final trend cache lifetime was 16 seconds; the team's was 250 milliseconds. The public requirements permit delayed condition updates in the trend view. This is an observed difference in implementation, but I did not isolate cache lifetime as the sole cause of the outcome.
Peak scores and final reboot scores are different measurements
| Measurement | Solo | Team |
|---|---|---|
| Best normal score within three hours | 1,621,980 | 1,114,247 |
| Pre-deadline confirmation of the selected commit | 1,608,134 | 654,392 |
| Final score after reboot | 1,636,308 | 885,334 |
“One trial per arm” means one three-hour optimization attempt. Each attempt contained successive candidate measurements, including confirmation of the selected version.
Following the ISUCON11 manual, I used the post-reboot benchmark as the final score. Both passed, although the team had nine transport errors and 71 timeouts. Its output was 885334(885350 - 16): the direct penalty was only 16 points, so penalties alone do not explain the score gap.
The team varied substantially on the same selected commit. Rebooting cannot be identified as the only cause of the drop. A further persistence check—rebooting again and reading back data written during the final benchmark—and a browser equivalence replay were not performed. The runtime evidence establishes a passing benchmark after reboot, not completion of every official follow-up check.
Count the children's tokens, too
| Usage | Solo | Team parent | Sol children | Team total |
|---|---|---|---|---|
| Input | 61,273,760 | 54,406,272 | 27,412,204 | 81,818,476 |
| Cached input | 60,027,392 | 53,330,048 | 25,752,320 | 79,082,368 |
| Uncached input | 1,246,368 | 1,076,224 | 1,659,884 | 2,736,108 |
| Output | 193,901 | 208,014 | 308,477 | 516,491 |
| Total | 61,467,661 | 54,614,286 | 27,720,681 | 82,334,967 |
Cached input is a subset of input. Total is input plus output; cached tokens are not added again. Reasoning output is likewise counted within output.
The team's parent used less input than solo. Including the children, however, total tokens increased by 34%, uncached input by about 2.20 times, and output by about 2.66 times. Reducing the parent's context burden did not reduce the system's total usage.
Fresh context did not guarantee small requests. One research child reached approximately 194,000 input tokens in a single recorded request. All 24 children had references to reading the manual, so repeated onboarding remained part of the workload.
Usage came from the experimental root threads and their descendants in native session logs. The deadline interrupted the parents' resumed turns, leaving incomplete turn records. These totals reflect recorded usage, not a finalized bill covering any unreported provider-side processing. I did not assume model prices to convert them into dollars.
Twenty-four children did not mean continuous parallel work
The logs established that:
- Solo launched no children. The team launched 24 child threads, all GPT-5.6 Sol with high reasoning effort.
- Peak concurrent child activity was three. No grandchildren were launched.
- All children used
fork_turns="none". The recorded commands showed separate implementation worktrees, with no counterexample to the two-implementation limit. - Two or more children were active for only about 14.5 minutes. Average active child count was 0.665.
“Active” here means a child turn interval, including tools and waits. It is not measured GPU inference time.
Model selection, counts, and execution intervals were auditable. Worktree ownership and meaningful task independence were not enforced as operating-system security boundaries. Observed compliance is narrower than mechanical enforcement.
There were 50 normal passing runs for solo and 46 for the team. The team used eight diagnostic runs versus four for solo. Executor occupancy was about 86 and 87 minutes respectively, with almost no lock waiting. More children did not produce more measured passing candidates.
What I would change next
For this workload, the result supports keeping Astra solo as a baseline and using children selectively for independent investigation and review. The team's earlier progress is still useful evidence.
I would make the pipeline more explicit: while the parent deploys and measures, children prepare one or two independent hypotheses for the next experiment. A concise shared map of the requirements should reduce repeated reading, while preserving links to the relevant original text. Near the deadline, I would reserve time to examine errors and repeatability rather than continue adding structural changes.
Those are proposed improvements, not a measured winning configuration. This experiment tested one team setup with an efficiency policy, not an established optimum for multi-agent orchestration.
With one trial per arm, hardware differences, unequal baselines, and different optimization paths, the result does not establish a universal model ranking or a multiplier transferable to ISUCON14. The operating metric I want to improve is how many useful changes we can measure correctly within a fixed time, rather than the cumulative number of agents launched.
Data and cleanup
The summary JSON and candidate score CSV accompany this article. Figures are generated from recorded measurements. The CSV includes diagnostics, failures, and post-reboot runs; its eligible column identifies normal passing runs within the time window.
I retained about 7.50 GB of logs, commits, patches, configuration, and parent/child sessions locally. The public data excludes AWS connection details and raw sessions. After the experiment, all eight EC2 instances, associated EBS volumes, dedicated security groups, and key pairs were destroyed and their removal verified.