Complete Telemetry Matrix: From 1 to 500 Parallel Agents
Here is the complete audited record across all machines, concurrency levels, and model configurations evaluated in this benchmark series.
Every run executed the real compiled Go daemon (swarmd) on ephemeral Google Compute Engine virtual machines, with system metrics sampled every 250ms directly from /proc.
| TEST RUN / HOST | MODEL | AGENTS | DURATION | TOTAL COST |
|---|---|---|---|---|
| Baseline · Dedicated Node (16 Cores) | Gemini 3.8 Flash | 1 | 12.4s | $0.005 |
| VPS Light · e2-micro ($6/mo) | Gemini 3.8 Flash | 20 | 56.2s | $0.12 |
| VPS Medium · e2-micro ($6/mo) | Gemini 3.8 Flash | 50 | 184.1s (3.1 min) | $0.29 |
| VPS Heavy (Initial) · e2-micro | Gemini 3.8 Flash | 100 | 1,070.1s (17.8 min)* | $0.59 |
| VPS Heavy (Retest) · e2-micro | Gemini 3.5 Flash Lite | 100 | 693.4s (11.5 min) | $1.23 |
| Dedicated Node · n2-standard-16 | Gemini 3.5 Flash Lite | 150 | 87.62s (< 1.5 min) | $1.85 |
| Frontier Limit · n2-standard-16 | Gemini 3.5 Flash Lite | 500 | 20s (Aborted on 429)** | < $0.05 |
** 500-agent run saturated Google's single-key Requests Per Minute (RPM) burst capacity (~1,500+ requests in 20s).
Wall-Clock Time: Hardware Bottlenecks vs. Dedicated Cloud
Notice how 16 dedicated CPU cores on the cloud node crushed 150 parallel agents in just 87.6 seconds, compared to 11.5 minutes for 100 agents queuing on the 2-core VPS.
Memory per Active Agent
SwarmAgent's compiled Go daemon scales with remarkable efficiency:
- 1 Agent: 73.6 MB RSS
- 20 Agents: 95.1 MB RSS
- 50 Agents: 138.4 MB RSS
- 100 Agents: 317.2 MB RSS
- 150 Agents: 415.3 MB RSS
Peak average: only 2.77 MB of RAM per active agent. Zero leaks.
CPU Steal & Swap Paging
How physical hardware responded to 100+ concurrent agents:
- e2-micro (3.8 Flash): > 25% CPU Steal, 1.2 GB swap thrashing after burst credits ran dry.
- e2-micro (Flash Lite): 0.02% CPU Steal, 1.1 GB swap used. CPU pinned at 85–100%.
- Dedicated Node: 0.0% CPU Steal, 0 MB swap. Under 2% of 64 GB host RAM used.
Token Throughput & Cost
High concurrency delivers immense throughput at negligible cost:
- 150-Agent Run: 5.96 Million tokens in 87 seconds.
- Throughput: ~4.08 Million TPM sustained.
- Total Cost: $1.85 ($1.83 tokens + $0.02 compute).
- Average cost per completed bug audit: $0.0123 USD.
What this actually means in plain English:
Think of the $6/month VPS like a single-lane country bridge. SwarmAgent's cars are small, lightweight, and built like tanks — none of them stalled or overheated. But when you send 100 cars onto a one-lane bridge at the exact same second, they have to wait their turn in a traffic jam. That is why 100 agents took 11.5 minutes.
When you switch to a 16-core dedicated cloud node, that single-lane bridge becomes a 16-lane interstate. All 150 agents accelerate simultaneously, and the entire fleet crosses the finish line in 87 seconds.
And when you push to 500 agents on a single project key, you don't crash the server — you trigger the speed camera on Google's API (HTTP 429 burst rate limit).
The Task & Verbatim Prompt Sent to All Agents
To eliminate any ambiguity, every single agent in this benchmark was given the exact same structured engineering assignment.
There were no synthetic sleeps, no mocked responses, and no pre-cached answers.
All sessions were launched in complete parallel at $t=0$ directly inside the real Swarm Go repository (/var/lib/swarm/workspace).
Here is the verbatim prompt delivered to every agent session:
You are Swarm agent #{agentIdx} executing in an autonomous AI engineering benchmark on Google Compute Engine with Gemini.
You are running directly inside the real Swarm codebase (/var/lib/swarm/workspace).
TASK CONTRACT:
1. Turn 1: Run EXACTLY ONE concise search or find query to inspect code (e.g. search for 'error', 'TODO', or 'nil' in pkg/ or internal/).
2. Turn 2: Read the candidate file, identify an edge case, unhandled error, or potential logic defect.
3. Turn 3: Write the structured bug report to /var/lib/swarm/workspace/audit_{wave}_a{agentIdx}.md and conclude.
Do NOT perform open-ended search loops or browse multiple directories. Inspect one target and write the report promptly.
The report MUST contain:
# Bug Audit Report - Wave {wave} Agent {agentIdx}
- **Timestamp**: (current ISO timestamp)
- **File Inspected**: (exact file path)
- **Function/Symbol**: (function or symbol name)
- **Bug Description**: (clear explanation of the issue or edge case discovered)
- **Severity**: Low / Medium / High
- **Proposed Fix**: (code snippet or fix recommendation)
Immediately write the file upon reading the target and confirm completion.Each agent had to understand the task, issue a real code search tool call across the workspace, inspect the returned Go source file, evaluate edge cases, and write a complete markdown audit report to disk.
A run was only marked successful if the output file existed on disk with valid markdown content and non-zero bytes.
Phase 1: Testing the Limits of a $6/Month VPS (1 to 100 Agents)
We began on Google Compute Engine's smallest standard machine: the e2-micro.
It features 2 shared, burstable vCPUs and exactly 1.0 GB of physical RAM at $0.0084 per hour ($6.05/month).
We added a 2.0 GB disk swapfile to give the Linux kernel headroom, booted swarmd, and stepped up parallel waves.
Light Developer Workload
Execution was instantaneous. OS memory took ~350 MB, swarmd consumed 73 MB RSS, and host free RAM remained comfortably above 550 MB with zero swap used.
Moderate Cloud Pressure
Available RAM dropped to ~150 MB. Linux began paging cold memory to disk swap. All 50 agents succeeded, but duration climbed from 56s to 184s as disk I/O took over.
Initial 100 Run: 17.8 Minutes
Launching 100 agents with Gemini 3.8 Flash immediately saturated Google's RPM limit. With 429 backoff retries and 1.2 GB swap thrashing, the run dragged out to 1,070 seconds.
Because the initial 100-agent run was artificially slowed by Gemini 3.8 Flash rate-limit retries, we went back and re-tested on the e2-micro with Gemini 3.5 Flash Lite.
The result: Zero 429 errors.
All 100 agents completed with 100% verified reports in 693.44 seconds (11.5 minutes).
Hypervisor CPU steal plummeted from over 25% down to an almost imperceptible 0.02%, and total run cost was just $1.23.
Why did it still take 11.5 minutes?
The micro simply got slow.
The Linux kernel had to context-switch 100 concurrent agent sessions, Git operations, and Pebble KV database writes across just 2 shared vCPUs.
Host CPU was pinned at 85% average (100% peak), and 1.2 GB of memory had to cycle through the disk swapfile.
SwarmAgent's Go daemon never crashed, never leaked memory, and stayed at 317 MB RSS. The software won, but the physical hardware was running at maximum contention.
Phase 2: Stepping Up to Dedicated Compute (150 Agents in 87s)
To see what SwarmAgent could do without hardware starvation, we provisioned an n2-standard-16 instance (16 dedicated vCPUs, 64 GB DDR4 RAM, 50 GB SSD).
We dispatched 150 agents simultaneously in an all-parallel wave at $t=0$.
The difference was night and day.
All 150 agents executed and wrote verified reports in under 1.5 minutes. The average individual agent duration was 69.5 seconds.
100.0% completion on the first try. Zero 429 errors, zero retry loops, and zero failed deliverables.
$1.831 for 5.96 million LLM tokens + $0.019 for 87 seconds of GCE compute. That is exactly $0.0123 (1.2 cents) per task.
On dedicated compute, daemon RSS peaked at 415.35 MB across all 150 active agents.
That translates to just 2.77 MB of daemon RAM per active parallel agent.
Across the 16-core machine, total system memory peaked at 1.14 GB out of 64 GB available (< 2% host RAM), and hypervisor CPU steal was exactly 0.0%.
The swarm sustained over 4.08 million tokens per minute.
Swarm Daemon RSS Memory Scaling (150 Parallel Agents)
Kernel RSS memory measured via psutil every 250ms. Notice the immediate drop back to 107 MB after agent completion as Go runtime GC reclaims buffers.
Phase 3: The 500-Agent Frontier Ceiling & Rate Limits
Next, we pushed to 500 agents in parallel to see where the system would break.
At t+20 seconds, Google's API returned HTTP 429: RESOURCE_EXHAUSTED.
Our automated sentinel immediately terminated the VM and boot disk ($0 idle spend).
Why did 500 fail when 150 succeeded?
The bottleneck was not our token quota.
Our Google Cloud account has a 30,000,000 Tokens Per Minute (30M TPM) limit.
500 agents only required ~20 million tokens, well within the 30M TPM window.
The real limit was Requests Per Minute (RPM) burst capacity.
Because each agent executes multi-turn tool calling, 500 agents fired over 1,500 HTTP requests within a 20-second burst.
This instantly saturated Google's RPM token bucket.
We could have bypassed this by rotating across multiple GCP projects or building a token-bucket rate limiter.
However, for illustrating real-world constraints, this proved an essential fact:
SwarmAgent's daemon can easily scale to 500+ concurrent sessions.
In production today, 150 parallel agents is the rock-solid baseline for zero rate limits on a single key.
Lifting Quotas Safely: Provider Rate Limits and Key Leasing Guardrails
To push past 150 parallel agents on a single project, you generally have to talk directly to your model provider.
Frontier AI companies like Google Cloud, OpenAI, and Anthropic don't grant unlimited burst capacity by default.
We are currently in that exact process with Google Cloud support: submitting organizational justification and requesting an enterprise RPM lift.
Why you shouldn't fear accidental runaway spend:
Developers often worry that high-concurrency swarms will accidentally burn thousands of dollars in minutes if an agent gets stuck in a loop.
In practice, it is actually quite difficult to accidentally outspend your limit.
Unless you explicitly opt into unmetered enterprise tiers, provider quota guardrails kick in first.
The moment request velocity spikes past your tier, the API fails closed with HTTP 429, blocking further billing before severe financial damage can occur.
However, you should never rely solely on provider rate limits to protect your budget.
In production, you must put strict limits on all your API keys and enforce multi-layered defense-in-depth.
Ephemeral Key Leases
We never store persistent, static API keys on remote test machines. Access is granted via short-lived, on-demand leases from our local approval broker (read the full key leasing guide).
GCP Project Spending Caps
We configure strict daily budget alerts and API quotas at the Google Cloud project level. Even if a script looped indefinitely, the project hard-stops before budget thresholds are breached.
Instant Teardown Sentinels
Our automated test harness detects 429 or abnormal errors instantly. When triggered, it terminates the VM and deletes the boot disk immediately, guaranteeing $0 idle spend.
This combination of provider-side limits, local ephemeral key leases, and automated kill-switches gives you total confidence when executing high-concurrency workloads.
You get maximum parallel horsepower without risking unexpected cloud bills.
Machine Sizing: When Dedicated Compute is Overkill vs. Necessary
Looking at the data, a 16-core machine with 64 GB of RAM is actually overkill if your agents are only issuing API calls and editing small files.
Since swarmd used only 415 MB of memory for 150 agents, you could comfortably run 50 to 100 agents on a much smaller 4-core, 16 GB instance.
So when do you actually need a powerful machine?
You need heavy compute when your agents run heavy local CI/CD pipelines and compilation.
End-to-End Build & Test Swarms
Imagine 10 to 20 parallel Coder agents working simultaneously on a large monorepo. Each agent operates in its own isolated Git worktree. Each agent compiles Go or Rust code, triggers Docker builds, and runs full integration test suites.
In that workflow, compilers and headless browsers consume substantial CPU cores and RAM. An n2-standard-16 gives every agent the dedicated CPU threads and memory needed to rebuild repeatedly, verify 100% passing tests, commit the branch, and send the pull request back to GitHub without crashing the host.
| HOST PROFILE | HARDWARE SPECS | SAFE CONCURRENCY | WORKLOAD TYPE |
|---|---|---|---|
| Small VPS ($6–15/mo) | 2 vCPU, 1–2 GB RAM | 1 – 10 Agents | Light developer workspace, cron tasks, single trigger agents. |
| Mid Cloud VM ($40–80/mo) | 4–8 vCPU, 16–32 GB RAM | 25 – 75 Agents | Pure LLM swarms, code search, audit reports, triage fleets. |
| Dedicated Node ($150–250/mo) | 16 vCPU, 64 GB RAM | 150 Agents | Heavy CI/CD, local Docker builds, compilation, 90s monorepo audits. |
| Multi-Node Cluster | 3x 16-core Nodes | 450+ Agents | Enterprise contact centers, high-volume customer support fleets. |
Real Scenario A: The Multi-Node AI Agent Call Center
How do you translate 150 parallel agents into a real production architecture?
Consider an automated customer support call center or high-volume chat fleet.
With SwarmAgent, you deploy a multi-node cluster of standard cloud instances behind a lightweight load balancer.
A 3-node cluster handles 450 simultaneous voice or chat interactions with zero wait time.
Using fast streaming models like gemini-3.5-flash-lite, conversational turn latency stays under 1.5 seconds.
A typical 3-minute customer interaction costs less than $0.015 all-in.
Real Scenario B: Tiered Customer Support with Trigger Agent Escalation
Real customer support requires intelligent tiering: fast frontlines, technical escalation, and automated bug resolution.
Automated Triage (Gemini 3.5 Flash Lite)
Handles 80% of incoming volume: password resets, order status, documentation lookups, and account queries. Average latency is < 1 second, costing roughly $0.003 per ticket.
Technical Escalation (Gemini 3.8 Flash / Claude)
When a customer reports an API error or integration defect, Tier 1 passes the session context to Tier 2. The Tier 2 agent inspects logs, diagnoses payloads, and pinpoints the issue in under 30 seconds.
Autonomous Swarm Trigger Workers
If Tier 2 verifies a software bug in production, it automatically dispatches an on-demand Swarm Trigger Agent using Swarm's trigger token API (POST /v3/automations/v2/trigger).
Here is how Swarm's latest Trigger Worker update transforms incident resolution:
Swarm immediately boots a background agent inside an isolated Git worktree.
The trigger agent reproduces the bug with a failing test, implements the fix, and runs the test suite.
When tests pass, the agent commits the branch and delivers the pull request and test report straight into the Agent Mailbox (/v3/deliverables).
The customer receives an update within minutes, and the engineering team wakes up to a reviewed, passing pull request.