SoAI basics
What is SoAI?
SoAI is a control plane for LLM inference that brings local engines and remote providers into one model pool. Applications connect through OpenAI-compatible or Anthropic-compatible APIs, while the SoAI WebUI manages models, plugins, routing, users, and system settings. See the SoAI overview for the product and routing layers.
What is the difference between SoAI Core and SoAI OS?
SoAI Core is the self-hosted application edition for a host you administer. SoAI OS is a dedicated Debian appliance containing Core plus host-management controls for storage, networking, drivers, package updates, and operating-system users. The installed artifact and launcher select the edition, Settings cannot convert one edition into the other, and application updates stay within the installed edition. See SoAI products and editions.
What backends does SoAI support?
SoAI ships 8 bundled plugins: External, Ollama, vLLM, Llama.cpp, CTranslate2, Llama.cpp Embeddings, Whisper, and Melo TTS. External connects to services that expose an OpenAI-compatible endpoint, including OpenAI, Groq, Together AI, Anthropic, OpenRouter, or your own deployment. Bundled plugins reference compares their roles and supported operations.
Does SoAI include models?
No. A fresh installation contains no models. SoAI setup in the first-run wizard can download a local model or connect a provider. For manual setup, install a compatible backend through Plugins, then download a model or register existing model files through Models. Remote provider models run at the provider and do not need a local download. See the Models catalog.
Can I use existing OpenAI client libraries?
Yes. Set the client base URL to SoAI, which defaults to http://localhost:5090/v1, and select an exact
model ID returned by the server. The compatible API is unavailable until the first administrator account exists,
then remains open while the API-key store has no records.
Once any API-key record exists, including a revoked or expired record, compatible requests require a valid
bearer token.
Only deleting every key record restores the open state. See
Authentication and API keys.
Can I use Claude Code?
Yes. Set ANTHROPIC_BASE_URL to the SoAI origin without /v1, select a SoAI model that
supports the capabilities Claude Code needs, and provide the SoAI API key through
ANTHROPIC_AUTH_TOKEN or ANTHROPIC_API_KEY. See
Using Claude Code with SoAI for the complete setup and TLS guidance.
How does the model list work?
GET /v1/models returns the models SoAI exposes through its compatible API, including virtual models. A
virtual model requires 2 or more unique catalog models and gives clients one model ID backed by load balancing or
ordered failover. GET /api/v1/models returns the richer management catalog used by the WebUI, including
plugin information, capabilities, and parameter state. See the HTTP API reference.
MCP, providers, and plugins
What is MCP, and which directions does SoAI support?
SoAI supports Model Context Protocol in two independent directions. In host mode, SoAI connects to external MCP
servers and makes their tools, resources, and prompts available to conversations. In server mode, authenticated
clients connect to /mcp and use the SoAI capabilities selected for exposure. Host connections use stdio
or Streamable HTTP, while server mode uses Streamable HTTP. MCP tokens are separate from SoAI API keys and do not
authenticate the WebUI or /v1 APIs. See MCP overview.
How do I add a remote API provider?
In the SoAI WebUI, open Models, select Add model, then select Add provider. Choose External under Plugin, enter the Provider name, OpenAI-compatible API URL, API key, and an optional Models filter, then select Add provider. The tab appears only when a provider-capable plugin is available and usable. SoAI requests the provider's model list and imports the accepted models into the catalog. Follow Adding your first backend for the complete local and remote workflows.
Can I write my own plugins?
Yes. A .soaiplugin is a ZIP package with a root __init__.py that defines one direct
Plugin class derived from plugin_sdk.BasePlugin. SoAI validates the package and its imports,
prepares a plugin-specific Python environment, and loads it through a dedicated
worker process. That process boundary is not an operating-system sandbox, so review third-party code before
installing it. See Creating a new plugin.
How do I use skills, and how are they different from plugins?
A skill supplies task instructions; a plugin connects an inference engine or provider. With
MCP usage (MCP_USE), open
Chat configuration > Skills, import a SKILL.md file or skill ZIP, enable it,
and select Save. Request it with $name or let the assistant choose it for a relevant
task. See Skills for usage and Creating a skill to write one.
Reliability and deployment
What happens if a backend fails?
For eligible non-provider-backed model requests, an open circuit breaker defers dispatch to the affected plugin. Provider-backed models do not use that dispatch circuit check. Plugin Guardian separately monitors runtime health, starts bounded recovery, and can quarantine a plugin after recovery is exhausted. See Resilience and self-healing, then follow Recover an unavailable plugin when manual action is required.
Does SoAI support multiple users?
Yes. SoAI provides multi-user authentication with per-user conversations, prompt templates, and preferences. SoAI
API keys carry only Public inference API access
(OPENAI_API) and can have quotas. In the WebUI, an
administrator with User administration
(USER_ADMIN) opens
Settings > Users to manage accounts, while an administrator with
ACL administration
(ACL_ADMIN) opens
Settings > ACL to manage role permissions. See
Administration overview.
What are the system requirements?
SoAI can run on a CPU-only host, although a compatible GPU is needed for reasonable local inference speed. Actual RAM, VRAM, storage, and compute requirements depend on the model, quantization, context length, plugin, and concurrency. Requirements lists the supported platform paths and minimum and recommended hardware tiers, while Capacity planning explains how to size a real workload.
Where does SoAI store its data?
SoAI keeps durable state under one system-data root by default, with separate locations for the database, managed files, downloaded models, plugin runtimes, logs, backups, Knowledge indexes, browser state, and WebUI media. Changing a configured path does not move existing data. See Data storage, retention, and relocation before moving or backing up any path.
How do I update SoAI?
In the WebUI, an account with Software updates
(SOFTWARE_UPDATE) opens the
Settings menu at the right of the header and selects Updates. Application updates
are separate from plugin-backend updates and, on SoAI OS, from host package updates. Automatic application
installation supports Core Linux x64 and ARM64,
macOS (experimental) Intel and Apple silicon, and
Windows x64. Linux and macOS use manifest-authenticated complete archives, while
Windows uses the current installer. An update cannot
change the installed edition. See Updating SoAI.
How is SoAI licensed?
SoAI Core is governed by the SoAI Source-Available License 1.0, while SoAI OS has a separate first-party license and purchase terms. Bundled plugins are MIT Components, and their managed backends and dependencies can carry additional terms. Read License and legal before relying on a right or deploying SoAI for an organization.