Architecture

The two-tier conductor/runner design that makes Orca horizontally scalable.

Overview

Orca is built on a two-tier architecture that separates concerns cleanly:

Tier	Component	Characteristics
Control Plane	Conductor	Stateless, horizontally scalable, HTTP API
Data Plane	Runner	Stateful, owns sessions, session-scoped MCP
Agent Execution	agent-worker	Node.js worker, one per runtime mode; inbound or sandbox-outbound transport

---

Conductor

The Conductor is a stateless Go binary that owns the HTTP API surface and routes work to runners.

Responsibilities

• Accept REST requests from the dashboard and API clients
• Maintain a run registry (in-memory + optional JSONL append-log)
• Delegate session creation to remote.Pool (round-robin to capable runner)
• Fan run events to SSE subscribers with replay and heartbeat
• Manage the MCP server catalog
• Seed default profiles to runners on startup

Key Properties

Because conductors are stateless, you can run any number of replicas behind a load balancer. Session routing never requires shared state — the session ID itself encodes which runner owns it.

Session ID format: sess-<runnerHash>-<8hex>

The runner hash is SHA256(RUNNER_BASE_URL)[:8]. Any conductor can extract the hash from a session ID and route directly to the owning runner without a shared registry.

---

Runner

The Runner is a stateful Go binary that owns all live agent state.

Responsibilities

• Store profiles and sessions in memory
• Build session-scoped toolkit views (intersection of profile tools list and registry)
• Resolve ${VAR} placeholders in MCP headers from its environment
• Dispatch run envelopes to agent-worker over the inbound sidecar API or a session-scoped outbound worker channel
• Expose a session-scoped MCP streamable HTTP endpoint per session
• Advertise capabilities and URL hash to the conductor via /runner/info

Session Lifecycle

---

agent-worker

The agent-worker is a Node.js process that wraps LLM provider SDKs and streams RunEvent objects back to the runner. It supports two HTTP transport modes without changing the runtime handlers or run envelope:

Transport	Activation	Behavior
Inbound server	Default	Listens on PORT; the runner calls POST /run and the worker returns an NDJSON response.
Outbound client	ORCA_RUNNER_URL and ORCA_WORKER_TOKEN are both set	Binds no port. A session-scoped worker long-polls the runner for commands, uploads run events as NDJSON, and uploads runtime state. ORCA_SESSION_ID is required.

Outbound mode is intended for workers inside per-session sandboxes that the runner cannot safely reach. Every runner call carries the session-scoped bearer token. Dropping the event upload cancels the active run, while transient poll or upload failures are retried with bounded exponential backoff.

After each outbound run, and when the runner sends export_state, the worker best-effort checkpoints conversation state to sandbox disk and uploads a durable copy to the runner. On restart it restores the disk copy before its first poll. The state directory defaults to /var/orca/state and can be changed with ORCA_STATE_DIR.

Mode Dispatch

The sidecar’s behavior is controlled entirely by the MODE environment variable:

MODE	SDK Used	Best For
claude	@anthropic-ai/claude-agent-sdk	Claude-specific features (native tool use, vision)
codex	@openai/codex-sdk	OpenAI Codex models
general	Vercel AI SDK	Multi-provider, model switching without redeployment
all	Dispatches by profile.runtime	Poly-sidecar — one process handles all runtimes

Runtime Configuration Isolation

The worker runtime is configured from the run envelope, not from the host agent CLI settings. Profile prompts, attached skills, session MCP, external MCP servers, and allowed tools are injected programmatically by Orca.

• claude runs the Claude Agent SDK with filesystem setting sources disabled, so host ~/.claude skills, plugins, and project settings are not discovered. Host built-in tools such as Bash, Read, Write, Edit, WebFetch, and subagent spawning are disabled by default, so executable work must route through the runner MCP sandbox tools.
• codex runs with an isolated CODEX_HOME and disables project-doc loading, so host ~/.codex config and repo AGENTS.md files are not inherited by agent runs. Codex runs in read-only sandbox mode by default; set AGENT_ORC_CODEX_HOME only when operators need to move that isolated Codex home directory.

Spawned Claude and Codex children receive a strict allowlisted environment instead of the full worker process.env. Model auth and the egress-guard variables are preserved, but worker-only storage credentials, database URLs, Redis URLs, and internal signing keys are not exposed to model-generated shell commands.

Set AGENT_WORKER_ALLOW_HOST_TOOLS=1 only as a local/debugging break-glass. It re-enables Claude host built-ins and restores Codex danger-full-access; unset or any other value keeps host tools disabled so execution fails closed unless it can use runner sandbox tools such as bash, run_skill_script, or VirtualFS execution.

Run Envelope

The runner sends this JSON body to the sidecar’s POST /run endpoint:

{
  "sessionId": "sess-a1b2c3d4-e5f6",
  "profile": {
    "name": "researcher",
    "runtime": "general",
    "systemPrompt": "You are a research assistant.",
    "tools": ["@default"],
    "model": "anthropic:claude-sonnet-4-6"
  },
  "subtask": {
    "title": "Research task",
    "prompt": "What is the capital of France?",
    "files": []
  },
  "sessionMcpUrl": "http://runner:7070/runner/sessions/sess-.../mcp",
  "mcpServers": [
    {
      "name": "my-api",
      "transport": "http",
      "url": "https://api.example.com/mcp",
      "headers": { "Authorization": "Bearer resolved-secret" }
    }
  ]
}

The sidecar responds with newline-delimited JSON RunEvent objects streamed as they are produced.

---

Data Flow: End-to-End

1. Client submits a run

   POST /api/runs { profile, title, prompt } arrives at the conductor.

2. Conductor routes to a runner

   remote.Pool round-robins across runners that advertise the required runtime capability. If the request omitted sessionId, the runner creates a fresh session and the conductor returns { runId, sessionId }.

3. Client opens SSE stream

   GET /api/runs/{runId}/stream — conductor registers subscriber and begins forwarding events.

4. Conductor dispatches to runner

   POST /runner/run with the run envelope over an NDJSON-streaming HTTP connection to the owning runner.

5. Runner dispatches to worker

   In inbound mode, the runner calls POST /run. In outbound mode, the session worker receives the same envelope by long-polling the runner. Profiles that opt in to @memory first have a --- CONTEXT FROM MEMORY --- block prepended to subtask.Prompt — see Memory Bank. The worker then connects to the LLM provider and begins streaming responses.

6. Tool calls loop through runner

   When the agent calls a tool, the sidecar invokes POST /runner/sessions/{id}/toolkit/invoke. The runner executes the tool and returns the result.

7. Events flow back

   The inbound response or outbound event upload carries RunEvent NDJSON → runner → conductor → SSE → all subscribers.

---

Capability Routing

Runners advertise their capabilities at startup via RUNNER_CAPABILITIES (default: claude,codex,general). The conductor’s remote.Pool only routes sessions to runners that support the requested profile.runtime.

---

Design Principles

Stateless control plane

Conductors hold no session state. The run registry is append-only (JSONL) and can be rebuilt from logs. This makes horizontal scaling trivial — just add replicas.

Session ID as location hint

The runner hash embedded in the session ID means any conductor can route any session request to the correct runner with a single map lookup. No Redis, no shared registry.

Sidecar abstraction

The runner communicates with the sidecar over HTTP. This means:

• Different LLM providers require no changes to the Go runtime
• Stubs replace real sidecars in tests
• Future providers are new Node.js files, not Go changes

MCP as tool abstraction

Every session exposes an MCP endpoint. This unifies platform tools, external APIs, and future sandbox services behind a single interface — regardless of the LLM provider or SDK.

Profile immutability at runtime

Updating a profile definition never affects running sessions. Sessions are created from a profile snapshot; they outlive profile edits safely.