Multi-Runner Setup

Distribute agent sessions across multiple runners for scalability and runtime specialization.

This is a self-hosting guide. The hosted product at https://www.orcapods.ai needs none of this — sign in and go.

Why Multiple Runners?

Running multiple runners lets you:

1. Scale horizontally — more runners = more concurrent sessions
2. Specialize by runtime — dedicate runners to specific LLM providers
3. Isolate workloads — separate runners for different tenants or use cases (future)
4. Improve resilience — conductor routes away from unhealthy runners

---

How Session Routing Works

Each runner has a hash derived from its RUNNER_BASE_URL:

hash = SHA256(RUNNER_BASE_URL)[:8]

Session IDs embed this hash: sess-<hash>-<8hex>

When any conductor receives a request targeting a session, it extracts the hash, looks it up in its remote.Pool, and routes directly to the owning runner. Routing is stateless by session ID, but membership changes are currently held in each conductor process; horizontally replicated conductors must therefore reconcile their registries independently.

---

Basic Two-Runner Setup

# Terminal 1: Runner A (claude + general)
RUNNER_PORT=7070 \
RUNNER_BASE_URL=http://localhost:7070 \
RUNNER_CAPABILITIES=claude,general \
CLAUDE_SIDECAR_URL=http://localhost:7071 \
GENERAL_SIDECAR_URL=http://localhost:7073 \
ANTHROPIC_API_KEY=sk-ant-... \
  go run ./agent-runtime/cmd/runner

# Terminal 2: Runner B (codex + general)
RUNNER_PORT=7075 \
RUNNER_BASE_URL=http://localhost:7075 \
RUNNER_CAPABILITIES=codex,general \
CODEX_SIDECAR_URL=http://localhost:7072 \
GENERAL_SIDECAR_URL=http://localhost:7073 \
OPENAI_API_KEY=sk-... \
  go run ./agent-runtime/cmd/runner

# Terminal 3: Conductor pointing at both
CONDUCTOR_PORT=8080 \
RUNNER_URLS=http://localhost:7070,http://localhost:7075 \
  go run ./agent-runtime/cmd/conductor

---

Capability-Based Routing

Runners advertise their capabilities. The conductor only routes sessions to runners that support the requested runtime:

Set capabilities via RUNNER_CAPABILITIES (comma-separated):

RUNNER_CAPABILITIES=claude,general

---

Sidecar Configuration per Runner

Each runner can have different sidecars for different runtimes:

Env Var	Purpose
SIDECAR_URL	Single poly-sidecar for all runtimes
CLAUDE_SIDECAR_URL	Claude-specific sidecar (overrides SIDECAR_URL)
CODEX_SIDECAR_URL	Codex-specific sidecar (overrides SIDECAR_URL)
GENERAL_SIDECAR_URL	General-mode sidecar (overrides SIDECAR_URL)

Per-runtime sidecars (recommended for production):

# Runner with all three specialized sidecars
CLAUDE_SIDECAR_URL=http://sidecar-claude:7071
CODEX_SIDECAR_URL=http://sidecar-codex:7072
GENERAL_SIDECAR_URL=http://sidecar-general:7073

Single poly-sidecar (simpler for development):

# MODE=all dispatches by profile.runtime
SIDECAR_URL=http://sidecar-all:7070

---

Checking Topology

The GET /api/topology endpoint shows all runners, their capabilities, health, membership state, process footprint, session breakdown, and the sidecar worker instances observed beneath each runner:

curl http://localhost:8080/api/topology | jq

[
  {
    "url": "http://localhost:7070",
    "hash": "a1b2c3d4",
    "capabilities": ["claude", "general"],
    "capabilitiesKnown": true,
    "state": "live",
    "healthy": true,
    "latencyMs": 8,
    "activeSessions": 3
  },
  {
    "url": "http://localhost:7075",
    "hash": "e5f6a7b8",
    "capabilities": ["codex", "general"],
    "healthy": true,
    "latencyMs": 10,
    "activeSessions": 1
  }
]

process, sessionsByRuntime, and sidecars can be null when a runner does not answer its topology probe. sidecars[].observedInstances is a lower bound on worker replicas, not a count, and cold means an idle or scaled-to-zero sidecar rather than a fault. The dashboard Runtime page displays these values and refreshes them manually because each read probes the fleet.

---

Runner Health & Failover

The conductor probes each runner at startup and during reconcile, and records a membership state. An individual runner that is unreachable during startup no longer prevents the conductor from booting when at least one runner answers; it remains visible as unreachable and is excluded from new session routing until a successful reconcile. A runner that is deliberately draining is also excluded from new sessions, while sessions it already owns remain routed by their session ID. Boot still fails when no runner answers at all.

Sessions are never migrated automatically. Sessions already assigned to an unreachable runner remain pinned to it and can error until it recovers; sessions on a draining runner remain routable so the runner can finish its in-flight work. Removing a runner outright can orphan those sessions.

---

Multi-Host Deployment

When runners run on different hosts, RUNNER_BASE_URL must be a routable URL that all conductors can reach:

# Runner on host-a
RUNNER_BASE_URL=http://host-a.internal:7070

# Runner on host-b
RUNNER_BASE_URL=http://host-b.internal:7070

# Conductor (anywhere)
RUNNER_URLS=http://host-a.internal:7070,http://host-b.internal:7070

Never use localhost for RUNNER_BASE_URL in multi-host setups. It needs to be a URL that resolves from the conductor’s network.

---

Adding Runners at Runtime

To add a new runner to a live cluster using the internal service plane:

1. Start the new runner with a unique RUNNER_BASE_URL
2. Call POST /internal/topology/reconcile on every conductor replica with the updated URL list, or omit the body to re-read that replica’s RUNNER_URLS

Existing sessions are unaffected. Once its probe succeeds, the new runner can receive new session assignments immediately.

To remove a runner, call POST /internal/topology/drain with its hash, wait for its active session count to reach zero, then call POST /internal/topology/remove. reconcile does not remove URLs that have disappeared from its input. These operations are per-replica and require the internal signing-plane authentication; they are not a shared autoscaling registry.

---

Runner-Specific Tool Configuration

Different runners can have different tools available. For example:

• Runner A has TAVILY_API_KEY set → web_search and web_extract work
• Runner B does not → those tools return errors

Sessions are assigned to runners based on capability, not tool availability. If you rely on specific tools, ensure all runners in the pool have the necessary env vars set.