Smart Orchestrator for Artificial Intelligence

Your AI runs everywhere, SoAI brings it together

SoAI is a self-hosted AI hub that brings local and cloud models together through one API and WebUI. Manage models, hardware, agents, automations, your knowledge base, and metrics from any device, wherever you are.

Run models where you decide: on your own CPU or GPU, through any cloud provider you connect, or across both.

Download SoAI Documentation

Built for the stack you already run

GPU hardware

  • NVIDIA
  • AMD
  • Intel

Clients

  • Claude Code
  • Codex
  • OpenCode
  • Qwen Code

Providers

  • OpenAI
  • Anthropic
  • OpenRouter
  • Qwen
  • Kimi
  • Z.ai

Platforms

  • Linux
  • Windows
  • macOS
  • Android

Plugins

Expand SoAI with plugins

Connect local engines and cloud providers with eight included plugins. Choose GPUs for supported plugins, run separate instances, or build your own with the plugin SDK.

  • Llama.cpp GGUF models on CPU, CUDA, ROCm, Vulkan, and Metal.
  • vLLM GPTQ, AWQ, GGUF, and transformer models.
  • Ollama Run Ollama locally or connect to Ollama Cloud.
  • CTranslate2 Offline translation with Helsinki-NLP models.
  • Embedding GGUF embeddings for your knowledge base.
  • Whisper Transcribe and translate audio recordings on your own hardware.
  • MeloTTS Text to speech with language-specific voices.
  • External Connect OpenAI-compatible providers as models.
Browse the Plugin Store

How it works

Work with SoAI

Chat, run agents, install plugins, and download models from your browser. Monitor GPUs, schedule tasks, and manage server access in the same interface.

Use it on its own

Run agents directly in Chat. Plan and Execute modes work through tasks using tools and their results. Approve calls that need permission, answer questions, or steer the agent as it works.

Give agents access to your files, browser, email, calendar, and server tools. Start tasks when you need them, or schedule them to run automatically.

  • Agent loop

    Works through tasks with tools, approvals, and subagents.

  • Compaction

    Summarizes long conversations to keep working within context limits.

  • Attachments

    Add files, images, and audio to conversations. Upload, scan with your camera, or use workspace files.

  • Prompts

    Save and reuse prompts across conversations.

  • Files

    Browse, move, transfer, and search conversation files.

  • Global search

    Find chats, prompts, files, models, plugins, devices, and settings.

Bring your own client

Connect clients that support OpenAI-compatible APIs or Anthropic Messages. Set your SoAI server address, API key, and model.

Both protocols support their native streaming formats. The Responses API also supports saved responses, cancellation, deletion, and conversation compaction.

Connect coding agents to SoAI

  • Claude Code

    Set these variables, then run claude. 

    export ANTHROPIC_BASE_URL="http://localhost:5090"
export ANTHROPIC_AUTH_TOKEN="soai-…"
export ANTHROPIC_MODEL="YOUR_MODEL_ID"

  • Codex CLI

    Add to ~/.codex/config.toml; export your key as SOAI_OPENAI_API_KEY. 

    model = "YOUR_MODEL_ID"
model_provider = "soai"

[model_providers.soai]
name = "SoAI"
base_url = "http://localhost:5090/v1"
env_key = "SOAI_OPENAI_API_KEY"
wire_api = "responses"

  • Opencode

    Add to opencode.json; export your key as SOAI_OPENAI_API_KEY. 

    {
  "provider": {
    "soai": {
      "npm": "@ai-sdk/openai-compatible",
      "options": {
        "baseURL": "http://localhost:5090/v1",
        "apiKey": "{env:SOAI_OPENAI_API_KEY}"
      },
      "models": { "YOUR_MODEL_ID": {} }
    }
  }
}

Call it from your code

Set your client’s base URL to your SoAI server and choose a model. SoAI routes the request to the matching local backend or cloud provider. 

curl http://localhost:5090/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "virtual/assistant",
    "messages": [{"role": "user", "content": "Hello"}],
    "stream": true
  }'

Where a request goes

Clients use one stable API while SoAI routes requests, monitors backend health, and queues waiting work. Capacity limits keep workloads under control, while bounded retries and failover help recover failed requests. Plugins that exhaust their recovery attempts are isolated.

Request sources such as apps and agents, messaging accounts, and automations reach the same SoAI process on port 5090. The control plane gates, plans, durably schedules, and executes each request against a direct local model, direct cloud model, or virtual model. Virtual models alone select a healthy local or cloud constituent. The interactive controls demonstrate bounded execution, load balancing, failover, cooldown, and quarantine. Model output returns over the exact selected route, passes through validation and delivery, and then follows the original path to its source. request sources Applications, SDKs, coding agents, and MCP clients send requests through SoAI's unified API. Apps & agents sdks · coding agents · mcp Connected Discord, WhatsApp, and Telegram accounts turn incoming messages into agent requests. Messaging accounts discord · whatsapp · telegram Schedules, triggers, and workflows start requests without an active user session. Automations schedules · triggers · workflows Durable tasks persist accepted inference work before scheduling so queued requests can be recovered and safely requeued after interruption. Durable tasks persist · recover · requeue API · Orchestrator Gate authenticates the caller, validates the request schema, and reserves the applicable quota before orchestration. 01 gate auth · schema · quotas Plan resolves request requirements and selects a healthy direct or virtual routing-pool candidate using availability, scoring, and failover policy. 02 plan requirements · candidates · failover Queue and schedule keeps accepted work durable, orders it by priority and fairness, and dispatches it within capacity limits. 03 queue & schedule durable priority · fairness · capacity Execute loads or reuses the selected runtime, acquires a bounded plugin slot, and receives inference output from the selected model before response validation. 04 execute load or reuse · plugin slot · inference Validate and deliver enforces the response contract and token limits, projects safe errors, and sends streaming or buffered output to its source. 05 validate & deliver frame contract · token limits · safe errors Guardian continuously health-checks backends, trips circuit breakers, and runs bounded recovery before isolating exhausted backends. guardian · always on health checks · circuit breakers bounded recovery attempts model destinations routing pool Virtual model is an optional routing-pool destination that selects a healthy local or cloud constituent using load balancing or ordered failover. Virtual model failover · local first Local models run on hardware or private endpoints controlled by the operator. Local models cpu · gpu · private APIs Cloud models are reached through connected provider and private API endpoints. Cloud models external providers Backends are isolated after bounded recovery is exhausted and return only after an operator clears quarantine and validation succeeds. Quarantined operator clear · validate
Explore the flow Hover, focus, or select an element to explore its role and current state.

Connect

Everything in sync, on every device

Continue conversations across desktop, phone, and tablet, even while a response is running. Drafts, attachments, and interface state stay synchronized across devices signed into your account.

SoAI Connect brings your server to Android, with saved sign-in, camera, microphone, and file uploads. Built-in Wake-on-LAN lets you power on your SoAI machine before connecting.

Download SoAI Connect
The same SoAI session open on four devices at once: a laptop on the chat page, a second laptop on the hardware dashboard, a tablet on the prompt library, and a phone on the plugins page.

SoAI Bench

Benchmark, stress, and certify supported GPUs

Test GPU performance and stability with SoAI Bench. Measure processing speed, memory performance, and responsiveness on supported NVIDIA, AMD, and Intel GPUs. Publish certified benchmark results to the public leaderboard to compare hardware and help choose your next upgrade.

GPU leaderboard
# GPU model Score
Loading scores…

See more scores...

SoAI OS

Manage your host from SoAI

Turn a computer into a dedicated AI machine. SoAI OS combines a Debian Linux desktop with SoAI and built-in controls for storage, networking, updates, and GPU drivers. Use the machine directly or manage it from another device, with your models and system controls in one place.

Read about SoAI OS
Rackmount and tower server hardware.

Licensing

Free for personal use. Companies join the waitlist

SoAI Core is source-available under the SoAI Source-Available License 1.0. Personal use is free. Commercial licenses and organization evaluations are temporarily unavailable.

personal & homelab

Free

Use SoAI Core personally without a paid key. SoAI OS uses separate licensing and a private distribution channel.

Download SoAI License

companies & organizations

Waitlist only

Commercial licensing and organization evaluations are temporarily unavailable. Contact info@soai.to for licensing information or to join the waitlist for future availability. Joining does not grant a license or guarantee an availability date.

Join the waitlist

Install

Install SoAI on your platform

Choose your platform. See the installation guide for requirements and release verification.

SoAI Core

Windows

Windows 11

Installer package

Download the setup application

Open the downloaded .exe and install SoAI.

Download Installer

PowerShell

Install with one command

Set-ExecutionPolicy Bypass -Scope Process -Force; irm https://soai.to/install/windows | iex

See Windows installation steps.