> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orcapods.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Architecture

> The two-tier conductor/runner design that makes Orca horizontally scalable.

## Overview

Orca is built on a **two-tier architecture** that separates concerns cleanly:

| Tier            | Component        | Characteristics                                                             |
| --------------- | ---------------- | --------------------------------------------------------------------------- |
| Control Plane   | **Conductor**    | Stateless, horizontally scalable, HTTP API                                  |
| Data Plane      | **Runner**       | Stateful, owns sessions, session-scoped MCP                                 |
| Agent Execution | **agent-worker** | Node.js worker, one per runtime mode; inbound or sandbox-outbound transport |

```mermaid theme={null}
flowchart TB
  dashboard["Dashboard<br/>React SPA"]

  conductor["Conductor<br/>Stateless control plane<br/>REST API + SSE"]

  pool["remote.Pool<br/>Round-robin placement<br/>Hash-based session routing"]

  runnerA["Runner A<br/>Sessions, profiles, toolkit<br/>Session MCP endpoint"]
  runnerB["Runner B<br/>Same runtime contract"]
  runnerC["Runner C<br/>Same runtime contract"]

  worker["agent-worker<br/>Node.js worker<br/>claude | codex | general"]

  providers["LLM Providers<br/>Anthropic | OpenAI | Google | Groq"]

  dashboard -->|"REST + SSE"| conductor
  conductor --> pool
  pool -->|"HTTP NDJSON"| runnerA
  pool -->|"HTTP NDJSON"| runnerB
  pool -->|"HTTP NDJSON"| runnerC
  runnerA <-->|"Worker HTTP transport"| worker
  runnerB <-->|"Worker HTTP transport"| worker
  runnerC <-->|"Worker HTTP transport"| worker
  worker --> providers
```

***

## Conductor

The **Conductor** is a stateless Go binary that owns the HTTP API surface and routes work to runners.

### Responsibilities

* Accept REST requests from the dashboard and API clients
* Maintain a **run registry** (in-memory + optional JSONL append-log)
* Delegate session creation to `remote.Pool` (round-robin to capable runner)
* Fan run events to SSE subscribers with replay and heartbeat
* Manage the MCP server catalog
* Seed default profiles to runners on startup

### Key Properties

<Note>
  Because conductors are stateless, you can run any number of replicas behind a load balancer. Session routing never requires shared state — the session ID itself encodes which runner owns it.
</Note>

**Session ID format:** `sess-<runnerHash>-<8hex>`

The runner hash is `SHA256(RUNNER_BASE_URL)[:8]`. Any conductor can extract the hash from a session ID and route directly to the owning runner without a shared registry.

***

## Runner

The **Runner** is a stateful Go binary that owns all live agent state.

### Responsibilities

* Store profiles and sessions in memory
* Build **session-scoped toolkit views** (intersection of profile `tools` list and registry)
* Resolve `${VAR}` placeholders in MCP headers from its environment
* Dispatch run envelopes to agent-worker over the inbound sidecar API or a session-scoped outbound worker channel
* Expose a **session-scoped MCP streamable HTTP endpoint** per session
* Advertise capabilities and URL hash to the conductor via `/runner/info`

### Session Lifecycle

```mermaid theme={null}
stateDiagram-v2
  [*] --> Profile
  Profile --> Idle: POST /runner/sessions
  Idle --> Running: POST /runner/run
  Running --> Idle: run completed
  Running --> Errored: run failed
  Idle --> Shutdown: DELETE /runner/sessions/{id}
  Errored --> Shutdown: DELETE /runner/sessions/{id}
  Shutdown --> [*]
```

***

## agent-worker

The **agent-worker** is a Node.js process that wraps LLM provider SDKs and streams `RunEvent` objects back to the runner. It supports two HTTP transport modes without changing the runtime handlers or run envelope:

| Transport       | Activation                                             | Behavior                                                                                                                                                           |
| --------------- | ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Inbound server  | Default                                                | Listens on `PORT`; the runner calls `POST /run` and the worker returns an NDJSON response.                                                                         |
| Outbound client | `ORCA_RUNNER_URL` and `ORCA_WORKER_TOKEN` are both set | Binds no port. A session-scoped worker long-polls the runner for commands, uploads run events as NDJSON, and uploads runtime state. `ORCA_SESSION_ID` is required. |

Outbound mode is intended for workers inside per-session sandboxes that the runner cannot safely reach. Every runner call carries the session-scoped bearer token. Dropping the event upload cancels the active run, while transient poll or upload failures are retried with bounded exponential backoff.

After each outbound run, and when the runner sends `export_state`, the worker best-effort checkpoints conversation state to sandbox disk and uploads a durable copy to the runner. On restart it restores the disk copy before its first poll. The state directory defaults to `/var/orca/state` and can be changed with `ORCA_STATE_DIR`.

### Mode Dispatch

The sidecar's behavior is controlled entirely by the `MODE` environment variable:

| `MODE`    | SDK Used                         | Best For                                             |
| --------- | -------------------------------- | ---------------------------------------------------- |
| `claude`  | `@anthropic-ai/claude-agent-sdk` | Claude-specific features (native tool use, vision)   |
| `codex`   | `@openai/codex-sdk`              | OpenAI Codex models                                  |
| `general` | Vercel AI SDK                    | Multi-provider, model switching without redeployment |
| `all`     | Dispatches by `profile.runtime`  | Poly-sidecar — one process handles all runtimes      |

### Runtime Configuration Isolation

The worker runtime is configured from the run envelope, not from the host agent CLI settings. Profile prompts, attached skills, session MCP, external MCP servers, and allowed tools are injected programmatically by Orca.

* `claude` runs the Claude Agent SDK with filesystem setting sources disabled, so host `~/.claude` skills, plugins, and project settings are not discovered. Host built-in tools such as `Bash`, `Read`, `Write`, `Edit`, `WebFetch`, and subagent spawning are disabled by default, so executable work must route through the runner MCP sandbox tools.
* `codex` runs with an isolated `CODEX_HOME` and disables project-doc loading, so host `~/.codex` config and repo `AGENTS.md` files are not inherited by agent runs. Codex runs in `read-only` sandbox mode by default; set `AGENT_ORC_CODEX_HOME` only when operators need to move that isolated Codex home directory.

Spawned Claude and Codex children receive a strict allowlisted environment instead of the full worker `process.env`. Model auth and the egress-guard variables are preserved, but worker-only storage credentials, database URLs, Redis URLs, and internal signing keys are not exposed to model-generated shell commands.

Set `AGENT_WORKER_ALLOW_HOST_TOOLS=1` only as a local/debugging break-glass. It re-enables Claude host built-ins and restores Codex `danger-full-access`; unset or any other value keeps host tools disabled so execution fails closed unless it can use runner sandbox tools such as `bash`, `run_skill_script`, or VirtualFS execution.

### Run Envelope

The runner sends this JSON body to the sidecar's `POST /run` endpoint:

```json theme={null}
{
  "sessionId": "sess-a1b2c3d4-e5f6",
  "profile": {
    "name": "researcher",
    "runtime": "general",
    "systemPrompt": "You are a research assistant.",
    "tools": ["@default"],
    "model": "anthropic:claude-sonnet-4-6"
  },
  "subtask": {
    "title": "Research task",
    "prompt": "What is the capital of France?",
    "files": []
  },
  "sessionMcpUrl": "http://runner:7070/runner/sessions/sess-.../mcp",
  "mcpServers": [
    {
      "name": "my-api",
      "transport": "http",
      "url": "https://api.example.com/mcp",
      "headers": { "Authorization": "Bearer resolved-secret" }
    }
  ]
}
```

The sidecar responds with **newline-delimited JSON** `RunEvent` objects streamed as they are produced.

***

## Data Flow: End-to-End

<Steps>
  <Step title="Client submits a run">
    `POST /api/runs { profile, title, prompt }` arrives at the conductor.
  </Step>

  <Step title="Conductor routes to a runner">
    `remote.Pool` round-robins across runners that advertise the required runtime capability. If the request omitted `sessionId`, the runner creates a fresh session and the conductor returns `{ runId, sessionId }`.
  </Step>

  <Step title="Client opens SSE stream">
    `GET /api/runs/{runId}/stream` — conductor registers subscriber and begins forwarding events.
  </Step>

  <Step title="Conductor dispatches to runner">
    `POST /runner/run` with the run envelope over an NDJSON-streaming HTTP connection to the owning runner.
  </Step>

  <Step title="Runner dispatches to worker">
    In inbound mode, the runner calls `POST /run`. In outbound mode, the session worker receives the same envelope by long-polling the runner. Profiles that opt in to `@memory` first have a `--- CONTEXT FROM MEMORY ---` block prepended to `subtask.Prompt` — see [Memory Bank](/concepts/memory-bank). The worker then connects to the LLM provider and begins streaming responses.
  </Step>

  <Step title="Tool calls loop through runner">
    When the agent calls a tool, the sidecar invokes `POST /runner/sessions/{id}/toolkit/invoke`. The runner executes the tool and returns the result.
  </Step>

  <Step title="Events flow back">
    The inbound response or outbound event upload carries `RunEvent` NDJSON → runner → conductor → SSE → all subscribers.
  </Step>
</Steps>

***

## Capability Routing

Runners advertise their capabilities at startup via `RUNNER_CAPABILITIES` (default: `claude,codex,general`). The conductor's `remote.Pool` only routes sessions to runners that support the requested `profile.runtime`.

```mermaid theme={null}
flowchart LR
  claude["Profile runtime<br/>claude"] --> runnerA["Runner A<br/>claude, general"]
  codex["Profile runtime<br/>codex"] --> runnerB["Runner B<br/>codex, general"]
  general["Profile runtime<br/>general"] --> runnerA
  general --> runnerB
```

***

## Design Principles

<AccordionGroup>
  <Accordion title="Stateless control plane">
    Conductors hold no session state. The run registry is append-only (JSONL) and can be rebuilt from logs. This makes horizontal scaling trivial — just add replicas.
  </Accordion>

  <Accordion title="Session ID as location hint">
    The runner hash embedded in the session ID means any conductor can route any session request to the correct runner with a single map lookup. No Redis, no shared registry.
  </Accordion>

  <Accordion title="Sidecar abstraction">
    The runner communicates with the sidecar over HTTP. This means:

    * Different LLM providers require no changes to the Go runtime
    * Stubs replace real sidecars in tests
    * Future providers are new Node.js files, not Go changes
  </Accordion>

  <Accordion title="MCP as tool abstraction">
    Every session exposes an MCP endpoint. This unifies platform tools, external APIs, and future sandbox services behind a single interface — regardless of the LLM provider or SDK.
  </Accordion>

  <Accordion title="Profile immutability at runtime">
    Updating a profile definition never affects running sessions. Sessions are created from a profile snapshot; they outlive profile edits safely.
  </Accordion>
</AccordionGroup>