Smart Orchestrator for Artificial Intelligence

Your AI runs everywhere, SoAI brings it together

One endpoint in front of every model you run, and full command of the machine behind it. SoAI pools local engines and remote providers into one /v1 API, then runs the hardware, the agents, the automations, the knowledge base, and the metrics on that same box.

Run models where you decide: on your own CPU or GPU, through any cloud provider you connect, or across both.

Download SoAI Documentation

Built for the stack you already run

GPU hardware

  • NVIDIA
  • AMD
  • Intel

Clients

  • Claude Code
  • Codex
  • OpenCode
  • Qwen Code

Providers

  • OpenAI
  • Anthropic
  • OpenRouter
  • Qwen
  • Kimi
  • Z.ai

Platforms

  • Linux
  • Windows
  • macOS
  • Android

Plugins

Expand SoAI with plugins

Plugins connect SoAI to local inference engines and OpenAI-compatible providers. Eight ship in the box. SoAI offers the backend builds that fit the host's operating system, GPU, and drivers, and every eligible plugin gets a device selector, so you can pin each one to the cards it is allowed to use. Clone a plugin to run a second instance on its own GPUs or port, and build anything else as a .soaiplugin archive against the plugin SDK.

  • Llama.cpp GGUF models, with CPU, CUDA, ROCm, Vulkan and Metal builds.
  • vLLM Managed backend for GPTQ, AWQ, GGUF and transformer models.
  • Ollama Managed local backend, or a link to Ollama Cloud.
  • CTranslate2 Offline CPU translation using Helsinki-NLP models.
  • Embedding A dedicated GGUF embedding runtime for Knowledge.
  • Whisper Speech to text, with audio transcription and translation.
  • MeloTTS Text to speech, with a language pack per voice.
  • External Any OpenAI-compatible provider, imported as models.
Browse the Plugin Store

How it works

Work with SoAI

SoAI's web interface operates the whole system. You chat with models and run agents over your own files there, and in the same place you install plugins, download models, watch and control the GPUs, schedule automations, and manage who shares the server.

Use it on its own

SoAI's Chat page is a full agent runtime. Its Plan and Execute modes run a multi-step loop that calls a tool, reads what came back, and goes again, and while a turn runs you can approve or deny a gated call, answer a question the agent asks, or steer it with new instructions.

Working in SoAI directly, the agent has a shell on the host, a browser, your mail and calendar, and the knowledge base built from your own files. Any of them can run on demand or on a schedule you set. Because the runtime and the models share one control plane, the agent reaches all of it from the conversation you are already in.

There is no second service to run and no separate front end to install.

  • Agent loop

    Calls tools, reads results, goes again. Approvals, steering, subagents.

  • Compaction

    Summarizes a long thread mid-turn when it outgrows its token budget.

  • Attachments

    Files, images and audio go into a conversation and stay linked to it.

  • Uploads

    Upload files, scan with a camera, or point at items already in your workspace.

  • Prompts

    A saved prompt library, reusable across every conversation.

  • Files

    A file explorer over a per-conversation workspace: batch moves, transfers, search.

  • Global search

    One query returns chats, prompts, files, models, plugins, devices and settings.

  • Tools

    Shell, browser, mail, calendar and knowledge tools, on demand or scheduled.

Bring your own client

SoAI serves OpenAI-compatible routes and an Anthropic-compatible Messages surface. Clients that accept a custom base URL, key, and model identifier can use the matching protocol directly.

/v1/chat/completions, /v1/responses, /v1/models, /v1/embeddings, /v1/audio/* and /v1/images/* are all served on the same port, so transcription, speech and image tools land on the same key and the same routing rules as chat. Clients that make you pick a wire format can point at either the chat or the responses API.

Streaming works in each protocol's own event format, so a client that streams from OpenAI or Anthropic streams from SoAI unchanged. Responses are stored, so you can fetch one later, cancel it while it runs, delete it, or compact it when a thread gets long.

Connect coding agents to SoAI

  • Claude Code

    Three environment variables, then run claude. 

    export ANTHROPIC_BASE_URL="http://localhost:5090"
export ANTHROPIC_AUTH_TOKEN="soai-…"
export ANTHROPIC_MODEL="YOUR_MODEL_ID"

  • Codex CLI

    In ~/.codex/config.toml, with the SoAI key exported as SOAI_OPENAI_API_KEY. 

    model = "YOUR_MODEL_ID"
model_provider = "soai"

[model_providers.soai]
name = "SoAI"
base_url = "http://localhost:5090/v1"
env_key = "SOAI_OPENAI_API_KEY"
wire_api = "responses"

  • Opencode

    In opencode.json, with the SoAI key exported as SOAI_OPENAI_API_KEY. 

    {
  "provider": {
    "soai": {
      "npm": "@ai-sdk/openai-compatible",
      "options": {
        "baseURL": "http://localhost:5090/v1",
        "apiKey": "{env:SOAI_OPENAI_API_KEY}"
      },
      "models": { "YOUR_MODEL_ID": {} }
    }
  }
}

Call it from your code

Point your code at your own SoAI instead of a vendor endpoint and change the base URL and model name. The name resolves through the model catalog to whichever plugin serves it: a local Ollama, vLLM, or llama.cpp backend, a dedicated embedding or speech model, or a remote provider, and the same call your code already sends keeps working. 

curl http://localhost:5090/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "virtual/assistant",
    "messages": [{"role": "user", "content": "Hello"}],
    "stream": true
  }'

Where a request goes

Clients use one stable API even when routing changes. SoAI decides where each request runs, keeps every target healthy, and holds work in durable queues when something misbehaves. Public execution-error responses may identify the plugin that failed.

SoAI schedules capacity in layers, with a bounded queue per plugin and limits on how many backends and how much work one account can run at once. Identical non-streaming requests can share a single in-flight run, failed ones retry within bounds or reroute to another target, and the guardian quarantines a plugin whose recovery attempts run out. An operator can clear that isolation after validation succeeds. Queued work survives a restart, and the server stays usable through bursts of thousands of simultaneous requests.

Request sources such as apps and agents, messaging accounts, and automations reach the same SoAI process on port 5090. The control plane gates, plans, durably schedules, and executes each request against a direct local model, direct cloud model, or virtual model. Virtual models alone select a healthy local or cloud constituent. The interactive controls demonstrate bounded execution, load balancing, failover, cooldown, and quarantine. Model output returns over the exact selected route, passes through validation and delivery, and then follows the original path to its source. request sources Applications, SDKs, coding agents, and MCP clients send requests through SoAI's unified API. Apps & agents sdks · coding agents · mcp Connected Discord, WhatsApp, and Telegram accounts turn incoming messages into agent requests. Messaging accounts discord · whatsapp · telegram Schedules, triggers, and workflows start requests without an active user session. Automations schedules · triggers · workflows Durable tasks persist accepted inference work before scheduling so queued requests can be recovered and safely requeued after interruption. Durable tasks persist · recover · requeue API · Orchestrator Gate authenticates the caller, validates the request schema, and reserves the applicable quota before orchestration. 01 gate auth · schema · quotas Plan resolves request requirements and selects a healthy direct or virtual routing-pool candidate using availability, scoring, and failover policy. 02 plan requirements · candidates · failover Queue and schedule keeps accepted work durable, orders it by priority and fairness, and dispatches it within capacity limits. 03 queue & schedule durable priority · fairness · capacity Execute loads or reuses the selected runtime, acquires a bounded plugin slot, and receives inference output from the selected model before response validation. 04 execute load or reuse · plugin slot · inference Validate and deliver enforces the response contract and token limits, projects safe errors, and sends streaming or buffered output to its source. 05 validate & deliver frame contract · token limits · safe errors Guardian continuously health-checks backends, trips circuit breakers, and runs bounded recovery before isolating exhausted backends. guardian · always on health checks · circuit breakers bounded recovery attempts model destinations routing pool Virtual model is an optional routing-pool destination that selects a healthy local or cloud constituent using load balancing or ordered failover. Virtual model failover · local first Local models run on hardware or private endpoints controlled by the operator. Local models cpu · gpu · private APIs Cloud models are reached through connected provider and private API endpoints. Cloud models external providers Backends are isolated after bounded recovery is exhausted and return only after an operator clears quarantine and validation succeeds. Quarantined operator clear · validate
Explore the flow Hover, focus, or select any element to see its role in the request lifecycle and the live state driving that part of the flow. select any node to explore its role

SoAI Connect

Everything in sync, on every device

Open SoAI on your desktop, phone, or tablet and continue exactly where you left off, even while a run is still in progress. Conversations, live responses, drafts, attachments, and interface state stay synchronized across every device signed into the same account.

SoAI Connect is the Android client for your self-hosted SoAI server. It finds the correct SoAI port from the address you enter, remembers your connection and sign-in, handles local HTTPS certificates with explicit trust, and integrates camera, microphone, and file uploads with Android. With built-in Wake-on-LAN, you can power on your SoAI machine from Android before connecting.

Download SoAI Connect
The same SoAI session open on four devices at once: a laptop on the chat page, a second laptop on the hardware dashboard, a tablet on the prompt library, and a phone on the plugins page.

SoAI Bench

Benchmark, stress, and certify supported GPUs

SoAIBench runs synthetic OpenCL workloads on one selected GPU and measures compute, memory bandwidth, and dispatch latency on NVIDIA, AMD, and Intel cards. A certified run repeats the same workload five times while SoAI watches temperature and power, and it counts only if the card scored consistently across all five passes.

GPU leaderboard
# GPU model Median
1 NVIDIA GeForce RTX 5090 13,771
2 NVIDIA L40S 11,609
3 NVIDIA GeForce RTX 4090 10,753
4 AMD Radeon RX 7900 XTX 8,227
5 AMD Radeon PRO W7900 7,988
12 Intel Arc B580 3,238

Open the full leaderboard

SoAI OS

The host machine, inside the same control plane

SoAI OS is a bootable Debian 13 ISO. It comes up as an appliance: a KDE Plasma desktop, with SoAI running as a system service before the desktop appears. Its media is private, separate from the public SoAI Core artifacts.

SoAI OS administers the host it runs on, from the same authenticated session you use for models, with every host route gated behind an administrator.

Read about SoAI OS
Rackmount and tower server hardware.

Licensing

Free for personal use. Companies join the waitlist

SoAI Core is source-available under the SoAI Source-Available License 1.0. Personal use is free. Commercial licenses and organization evaluations are temporarily unavailable.

personal & homelab

Free

Use SoAI Core personally without a paid key. SoAI OS uses separate licensing and a private distribution channel.

Download SoAI License

companies & organizations

Waitlist only

Commercial licensing and organization evaluations are temporarily unavailable. Contact [email protected] for licensing information or to join the waitlist for future availability. Joining does not grant a license or guarantee an availability date.

Join the waitlist

Install

Install SoAI on your platform

See the installation guide for complete system requirements and release verification. Choose a platform below:

SoAI Core

Windows

Windows 10/11 · x64

Installer package

Download the setup application

Open the downloaded .exe, approve the unsigned-publisher warning, and finish setup in the native Windows wizard.

Download Installer

PowerShell

Install with one command

Set-ExecutionPolicy Bypass -Scope Process -Force; irm https://soai.to/install/windows | iex

Need more detail? Read the Windows installation steps.