SoAI Core runtime architecture

SoAI Core is one host runtime composed from explicit subsystems. The host process owns public protocols, authentication, configuration, persistence, scheduling, and lifecycle. Inference and provider implementations from .soaiplugin packages execute in dedicated worker processes and are represented in the host by proxies.

Composition and protocol ownership

SoAI application entrypoint
      |
      v
stage-zero edition preflight
      |
      v
Core or SoAI OS edition composition
      |
      v
application composition root
      |
      +-- API and WebUI transport
      +-- configuration and database
      +-- tasks, scheduling, and models
      +-- plugins and worker proxies
      +-- MCP and agent services
      `-- monitoring and background actors

The invoked launcher and artifact select the edition before application composition. Core and the appliance edition each start through their own launcher. The appliance integrates through typed Core contracts, while Core has no runtime path that discovers or loads appliance implementation.

Application composition is the only composition root. Foundation contracts depend only on the standard library and third-party packages. Each subsystem depends on those contracts and its own implementation, while cross-subsystem behavior is supplied through explicit protocols and dependencies. Plugin packages depend on the public plugin_sdk boundary, not on Core implementation modules.

Protocol ownership

Each public boundary applies its own authentication and access policy. Mutation endpoints validate schema and operation preconditions before changing state. The boundary converts internal failures into the safe response shape owned by that protocol; raw tracebacks, provider payloads, and local paths remain in diagnostics.

Request path and state ownership

client inference request
      |
      v
protocol parsing, authentication, and validation
      |
      v
model and capability resolution
      |
      v
admission, owner limits, queue, and scheduler
      |
      v
eligible plugin and model selection
      |
      v
host proxy == authenticated IPC == plugin worker
      |
      v
response or protocol-native stream

Model resolution can select a physical catalog model or expand a virtual model into eligible constituents. Admission applies queue, owner, plugin, disk, cancellation, and deduplication policy before dispatch. The scheduler reserves the selected runtime, and the plugin proxy translates the host call into authenticated IPC. The plugin object never becomes part of the host runtime.

A transient member failure can make another eligible virtual-model constituent available to the same routing plan. Automatic retry is limited to work whose contract is idempotent, deduplicated, or safely recoverable. Accepted durable work retains enough state to reach a truthful terminal outcome after process recovery.

State ownership

Agent execution

An agent turn alternates model inference with tool execution. SoAI supplies the tools available to the conversation, checks requested actions against their access policy, and waits for approval when interactive approval is required. Tool results are saved with turn progress and returned to the model so it can continue or produce a final response. The host owns this sequence and its cancellation state; inference plugins provide model responses without owning conversation permissions or tool execution.

Skill availability is captured when an ordinary chat input is accepted. The admitted snapshot keeps the selected content stable through queuing, steering, and recovery; later inputs use the current saved selection. Explicit requests activate instructions during turn preparation. Automatic activation uses skill_read and checks the remaining context budget before adding the complete instructions. Supporting text is read from the package, while actions remain subject to ordinary tool policy.

Lifecycle and failure recovery

Stage zero verifies the requested edition and handles dependency-light lifecycle commands before loading the full runtime. Bootstrap then reconciles configuration, opens and validates the database, constructs services, recovers accepted work, reconciles plugin packages and inherited processes, starts background actors, and publishes initial resource state. Readiness is announced only after essential startup and reconciliation complete.

Recovery follows each operation's contract. Saved conversation and domain progress can support reconciliation or continuation, while ordinary tasks left queued or running after a restart are marked failed with Server restarted and error code 503. Acceptance alone does not mean a task will run again.

An optional subsystem can publish degraded health without preventing unrelated capabilities from starting. An essential configuration, integrity, database, or recovery failure blocks readiness. Repair-plane operations remain available only when their own invariants hold; a running Uvicorn process alone is not evidence that the application is ready.

Controlled shutdown stops admission, resolves or cancels owned work according to its contract, closes interactive sessions, stops workers and background actors, flushes durable state, and releases process ownership. Cancellation, supersession, client disconnect, and deliberate shutdown are normal outcomes unless cleanup itself fails.

Failure isolation and recovery

A plugin worker crash removes that runtime from dispatch without importing the failure into the host process. The dispatch circuit breaker limits calls to a failing plugin; the Plugin Guardian owns health observation, quarantine, and bounded recovery. Compatibility, enabled state, worker state, quarantine, and circuit state remain separate so one condition is not misreported as another.

The worker process is an execution boundary rather than an operating-system sandbox, so plugin code retains the permissions of the account running SoAI. Security and integrity failures fail closed. Transient provider, backend, or optional-service failures are contained at their owning boundary and must not crash unrelated requests or subsystems.

Runtime ownership map

• The application entrypoint and edition composition own launcher selection and edition contracts.
• Application composition owns service construction; ordered startup owns readiness and recovery.
• The API boundary owns compatible and management routes and protocol projection.
• Orchestration services own routing, catalog behavior, admission, scheduling, and task state.
• The persistence boundary owns SQLite schema, serialization, transactions, and repositories.
• The plugin boundary owns package admission, catalog reconciliation, workers, proxies, health, and lifecycle.
• WebUI and MCP services own their respective state, transport, host, and server-mode integrations.

Related documentation

• Skills
• WebUI architecture
• Plugin architecture
• How SoAI works
• Virtual models, routing, and failover
• Resilience and self-healing
• HTTP API reference