Inference engines, speech, translation, and remote providers for SoAI.
Offline machine translation on the CPU with CTranslate2 and Helsinki-NLP models, served through chat completions and completions.
A dedicated llama.cpp runtime for GGUF embedding models, used by Knowledge collections and the SoAI embeddings endpoint.
Connect SoAI to remote OpenAI-compatible APIs such as OpenAI, OpenRouter, Anthropic, Groq, and Together AI, and import the models they report.
Run GGUF models locally with llama.cpp, using the CPU, CUDA, ROCm, Vulkan, OpenVINO, SYCL, OpenCL, or Metal build that fits the host.
Local text to speech with MeloTTS. SoAI downloads its language packs as models and runs one speech request at a time.
Run Ollama as a managed local backend or connect it to Ollama Cloud, with chat, embeddings, vision, and tool calling where the model supports them.
Serve GPTQ, AWQ, GGUF, and transformer models with vLLM, on CPU builds or GPU variants for NVIDIA CUDA, AMD ROCm, and Intel XPU on Linux.
Local speech to text with OpenAI Whisper for audio transcription and translation, on CPU or accelerator builds that fit the host.