Infinity Fleet: The Queue Drainer

Tier 3 · Everything Built 7 min read

Note

Prereqs: Fleet of Claudes and Worktree Isolation: The Hard Lesson.

Fleet of Claudes proved that parallel agents can work without trampling each other. Infinity Fleet took that and made it location-agnostic: any machine with the right credentials can attach to the task queue, claim work, run it in an isolated worktree, and ship the result. Capacity isn’t a fixed ceiling — it’s the sum of attached workers.

---

The Bottleneck Fleet of Claudes Left Unsolved

Fleet of Claudes ran workers from a single orchestrator session. The orchestrator spawned agents, watched them complete, and integrated results. This worked for a batch of 6 concurrent PRs. It didn’t scale well to a continuous queue of work — because the orchestrator itself was the bottleneck, and it was pinned to one machine’s session context.

The Infinity Fleet drainer inverts the model. There is no central orchestrator managing workers. Instead: tasks go into a Supabase queue, workers poll the queue, claim tasks via optimistic lock (Supabase PATCH with claimed_by), run them in worktrees, and report results back. Any machine that can reach Supabase and has a Claude Code session available can be a worker.

CHANGELOG entry: 2026-06-08, parallel-spawn-refactor project.

---

The Drainer (agents/shared/drainer.py)

The drainer is the worker-side loop. It does five things:

1. Poll. Query the agent_tasks Supabase table for unclaimed tasks in the fleet queue. If empty: idle.

2. Claim. Optimistic PATCH sets claimed_by to this worker’s ID. If the claim fails (another worker got there first), skip and poll again. No lock contention, no coordination.

3. Register a run. Mint a run_id for this claim and write it to the runs registry. The claimed_by FK on the task points to this run.

4. Spawn in a worktree. Call fleet.py spawn() with the task’s objective and the run ID. The agent runs in an isolated git worktree, produces output, and exits.

5. Reap and report. Poll the tmux session for completion. Read the stream-json result. Mark the task done or failed in Supabase. If failed and below the retry cap: re-queue with backoff. If at the cap: park and alert JD.

Each drainer instance runs at a per-worker concurrency cap (max=2 for the Mac Mini, configurable). It doesn’t dispatch more tasks than it can run concurrently. On an empty queue, it idles without spinning.

---

The MBP Worker

Scale-out is additive. The MacBook Pro can run a fleet worker via scripts/fleet-worker-mbp.sh, polled every 10 minutes by a cron that no-ops if the MBP is offline:

# MBP auto-join cron (installs on MBP, no-ops on Mini)
*/10 * * * * bash ~/agent-system/scripts/fleet-worker-mbp.sh

When the MBP is online and the queue has work, the worker attaches. When the MBP goes offline or the queue drains, the worker exits cleanly. The Mac Mini drainer continues unaffected.

Adding more capacity is: set up another machine with the credentials (via scripts/fleet-worker-env.sh), run the worker script. No changes to the orchestrator, the queue schema, or any other worker.

---

The “Infinity” in Infinity Fleet

The name is intentional and specific. Before the drainer, the parallel-spawn PRD had a “single Max seat ceiling” — the orchestrator’s context window limited how many workers could run simultaneously. After the drainer, the constraint is gone:

PRD/WORKPLAN re-pointed from single-Max-seat-ceiling → INFINITY (per-worker cap, scale by attaching workers + credential pool). Capacity = SUM of attached workers, no global cap.

“Infinity” doesn’t mean unlimited in practice — each worker machine has a Max plan seat and a concurrency cap. It means there’s no architectural ceiling. Adding a worker increases capacity by that worker’s cap. Removing one decreases it. The queue doesn’t care how many workers are attached.

---

Feeding the Queue

The drainer is a consumer. Something has to produce tasks.

python3.12 -m agents.shared.task_queue enqueue fleet '<objective>'

That’s the current producer. The CEO work queue, when it identifies approved evolution proposals to build, can enqueue them here rather than dispatching directly. M10 of the parallel-spawn-refactor workplan is the “continuous-shipping → enqueue” pipeline; M12 is the Telegram /fleet command for manual queue management. Both are designed but not yet installed.

---

The Live State

CHANGELOG 2026-06-08: “Mini drainer LIVE via com.clawd.fleet-drainer (worker=mini max=2, idle on empty ‘fleet’ queue).” The LaunchAgent keeps the drainer running on the Mac Mini. The fleet_workers registry migration was noted as needing JD to apply (Supabase schema change, irreversible — human-gated per protocol).

The fleet drainer is running. The MBP worker is ready to attach. The producer pipeline is the next piece.

Tip

The cleanest mental model: the drainer is a job board with self-scheduling contractors. You post tasks to the board; any available contractor picks one up, does the work in their own workspace, and posts the result. The board doesn’t care who’s on it or how many contractors exist. The only coupling is the task schema in Supabase.

---

Next: The Architect Agent — the PRD gate, intent-verify, and code review layer that keeps the Nerve Center ecosystem coherent, and the explicit scope rule for when it fires.

Enjoyed this? Get weekly agent architecture insights.

Subscribe and get a free Agent Architecture Cheatsheet.

Subscribe