> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orcapods.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Multi-Runner Setup

> Distribute agent sessions across multiple runners for scalability and runtime specialization.

<Note>This is a self-hosting guide. The hosted product at [https://www.orcapods.ai](https://www.orcapods.ai) needs none of this — sign in and go.</Note>

## Why Multiple Runners?

Running multiple runners lets you:

1. **Scale horizontally** — more runners = more concurrent sessions
2. **Specialize by runtime** — dedicate runners to specific LLM providers
3. **Isolate workloads** — separate runners for different tenants or use cases (future)
4. **Improve resilience** — conductor routes away from unhealthy runners

***

## How Session Routing Works

Each runner has a **hash** derived from its `RUNNER_BASE_URL`:

```
hash = SHA256(RUNNER_BASE_URL)[:8]
```

Session IDs embed this hash: `sess-<hash>-<8hex>`

When any conductor receives a request targeting a session, it extracts the hash,
looks it up in its `remote.Pool`, and routes directly to the owning runner.
Routing is stateless by session ID, but membership changes are currently held
in each conductor process; horizontally replicated conductors must therefore
reconcile their registries independently.

```mermaid theme={null}
flowchart LR
  session["sess-a1b2c3d4-e5f6g7h8"]
  hash["a1b2c3d4<br/>hash of runner-1 RUNNER_BASE_URL"]
  conductor["Conductor<br/>extracts hash"]
  runner["runner-1"]

  session --> hash --> conductor -->|"routes directly"| runner
```

***

## Basic Two-Runner Setup

```bash theme={null}
# Terminal 1: Runner A (claude + general)
RUNNER_PORT=7070 \
RUNNER_BASE_URL=http://localhost:7070 \
RUNNER_CAPABILITIES=claude,general \
CLAUDE_SIDECAR_URL=http://localhost:7071 \
GENERAL_SIDECAR_URL=http://localhost:7073 \
ANTHROPIC_API_KEY=sk-ant-... \
  go run ./agent-runtime/cmd/runner

# Terminal 2: Runner B (codex + general)
RUNNER_PORT=7075 \
RUNNER_BASE_URL=http://localhost:7075 \
RUNNER_CAPABILITIES=codex,general \
CODEX_SIDECAR_URL=http://localhost:7072 \
GENERAL_SIDECAR_URL=http://localhost:7073 \
OPENAI_API_KEY=sk-... \
  go run ./agent-runtime/cmd/runner

# Terminal 3: Conductor pointing at both
CONDUCTOR_PORT=8080 \
RUNNER_URLS=http://localhost:7070,http://localhost:7075 \
  go run ./agent-runtime/cmd/conductor
```

***

## Capability-Based Routing

Runners advertise their capabilities. The conductor only routes sessions to runners that support the requested runtime:

```mermaid theme={null}
flowchart LR
  claude["profile.runtime = claude"] --> runnerA["Runner A<br/>claude, general"]
  codex["profile.runtime = codex"] --> runnerB["Runner B<br/>codex, general"]
  general["profile.runtime = general"] --> runnerA
  general --> runnerB
```

Set capabilities via `RUNNER_CAPABILITIES` (comma-separated):

```bash theme={null}
RUNNER_CAPABILITIES=claude,general
```

***

## Sidecar Configuration per Runner

Each runner can have **different sidecars** for different runtimes:

| Env Var               | Purpose                                           |
| --------------------- | ------------------------------------------------- |
| `SIDECAR_URL`         | Single poly-sidecar for all runtimes              |
| `CLAUDE_SIDECAR_URL`  | Claude-specific sidecar (overrides `SIDECAR_URL`) |
| `CODEX_SIDECAR_URL`   | Codex-specific sidecar (overrides `SIDECAR_URL`)  |
| `GENERAL_SIDECAR_URL` | General-mode sidecar (overrides `SIDECAR_URL`)    |

**Per-runtime sidecars (recommended for production):**

```bash theme={null}
# Runner with all three specialized sidecars
CLAUDE_SIDECAR_URL=http://sidecar-claude:7071
CODEX_SIDECAR_URL=http://sidecar-codex:7072
GENERAL_SIDECAR_URL=http://sidecar-general:7073
```

**Single poly-sidecar (simpler for development):**

```bash theme={null}
# MODE=all dispatches by profile.runtime
SIDECAR_URL=http://sidecar-all:7070
```

***

## Checking Topology

The `GET /api/topology` endpoint shows all runners, their capabilities, health,
membership state, process footprint, session breakdown, and the sidecar worker
instances observed beneath each runner:

```bash theme={null}
curl http://localhost:8080/api/topology | jq
```

```json theme={null}
[
  {
    "url": "http://localhost:7070",
    "hash": "a1b2c3d4",
    "capabilities": ["claude", "general"],
    "capabilitiesKnown": true,
    "state": "live",
    "healthy": true,
    "latencyMs": 8,
    "activeSessions": 3
  },
  {
    "url": "http://localhost:7075",
    "hash": "e5f6a7b8",
    "capabilities": ["codex", "general"],
    "healthy": true,
    "latencyMs": 10,
    "activeSessions": 1
  }
]
```

`process`, `sessionsByRuntime`, and `sidecars` can be `null` when a runner does
not answer its topology probe. `sidecars[].observedInstances` is a lower bound
on worker replicas, not a count, and `cold` means an idle or scaled-to-zero
sidecar rather than a fault. The dashboard **Runtime** page displays these
values and refreshes them manually because each read probes the fleet.

***

## Runner Health & Failover

The conductor probes each runner at startup and during reconcile, and records a
membership state. An individual runner that is unreachable during startup no
longer prevents the conductor from booting when at least one runner answers;
it remains visible as `unreachable` and is excluded from new session routing
until a successful reconcile. A runner that is deliberately `draining` is also
excluded from new sessions, while sessions it already owns remain routed by
their session ID. Boot still fails when no runner answers at all.

<Warning>
  Sessions are never migrated automatically. Sessions already assigned to an
  unreachable runner remain pinned to it and can error until it recovers;
  sessions on a draining runner remain routable so the runner can finish its
  in-flight work. Removing a runner outright can orphan those sessions.
</Warning>

***

## Multi-Host Deployment

When runners run on different hosts, `RUNNER_BASE_URL` must be a routable URL that all conductors can reach:

```bash theme={null}
# Runner on host-a
RUNNER_BASE_URL=http://host-a.internal:7070

# Runner on host-b
RUNNER_BASE_URL=http://host-b.internal:7070

# Conductor (anywhere)
RUNNER_URLS=http://host-a.internal:7070,http://host-b.internal:7070
```

<Note>
  Never use `localhost` for `RUNNER_BASE_URL` in multi-host setups. It needs to be a URL that resolves from the conductor's network.
</Note>

***

## Adding Runners at Runtime

To add a new runner to a live cluster using the internal service plane:

1. Start the new runner with a unique `RUNNER_BASE_URL`
2. Call `POST /internal/topology/reconcile` on every conductor replica with
   the updated URL list, or omit the body to re-read that replica's
   `RUNNER_URLS`

Existing sessions are unaffected. Once its probe succeeds, the new runner can
receive new session assignments immediately.

To remove a runner, call `POST /internal/topology/drain` with its hash, wait
for its active session count to reach zero, then call
`POST /internal/topology/remove`. `reconcile` does not remove URLs that have
disappeared from its input. These operations are per-replica and require the
internal signing-plane authentication; they are not a shared autoscaling
registry.

***

## Runner-Specific Tool Configuration

Different runners can have different tools available. For example:

* Runner A has `TAVILY_API_KEY` set → `web_search` and `web_extract` work
* Runner B does not → those tools return errors

Sessions are assigned to runners based on capability, not tool availability. If you rely on specific tools, ensure all runners in the pool have the necessary env vars set.