Llama.cpp
by SoAI · Bundled
Run GGUF models locally with llama.cpp, using the CPU, CUDA, ROCm, Vulkan, OpenVINO, SYCL, OpenCL, or Metal build that fits the host.
Llama.cpp is the bundled GGUF backend. It supports chat completions, completions, Responses, embeddings, vision, tool calling, structured output, and JSON Schema where the model supports them.
SoAI offers the backend variants that fit the host's operating system, GPU, and drivers, and it can search for and download models. On a host with NVIDIA, AMD, and Intel GPUs, clone the plugin to run separate instances with their own runtime variant and GPU binding. One instance cannot combine CUDA, ROCm, and XPU devices.
Version 1.2.1
Install from a file
In SoAI, open Plugins → Add plugin → Upload plugin and select the downloaded .soaiplugin file.
Install from a URL
Copy this URL, then open Plugins → Add plugin → Download plugin from URL in SoAI and paste it there.
https://soai.to/v1/llamacpp/1.2.1/24d3414a9ef0602e31f664e40bd763241cd16f643a82e076f7930b02ffebb0f9/llamacpp.soaiplugin
SHA-256 checksum
24d3414a9ef0602e31f664e40bd763241cd16f643a82e076f7930b02ffebb0f9
Previous versions 4
Version 1.2.0
Install from a file
In SoAI, open Plugins → Add plugin → Upload plugin and select the downloaded .soaiplugin file.
Install from a URL
Copy this URL, then open Plugins → Add plugin → Download plugin from URL in SoAI and paste it there.
https://soai.to/v1/llamacpp/1.2.0/f6e968b3c043c5450eaf832973125bd3cb9957faf640950924add02be7f9df02/llamacpp.soaiplugin
SHA-256 checksum
f6e968b3c043c5450eaf832973125bd3cb9957faf640950924add02be7f9df02
Version 1.1.1
Install from a file
In SoAI, open Plugins → Add plugin → Upload plugin and select the downloaded .soaiplugin file.
Install from a URL
Copy this URL, then open Plugins → Add plugin → Download plugin from URL in SoAI and paste it there.
https://soai.to/v1/llamacpp/1.1.1/3bceaa7d9cb6d93628a55a49885dd6f74c8522fc8678acbc53765fe87ddcdf55/llamacpp.soaiplugin
SHA-256 checksum
3bceaa7d9cb6d93628a55a49885dd6f74c8522fc8678acbc53765fe87ddcdf55
Version 1.1.0
Install from a file
In SoAI, open Plugins → Add plugin → Upload plugin and select the downloaded .soaiplugin file.
Install from a URL
Copy this URL, then open Plugins → Add plugin → Download plugin from URL in SoAI and paste it there.
https://soai.to/v1/llamacpp/1.1.0/dbe891d3b2fffcea76dffcc991d73c052508c92e23c34a947b9199360e4649f2/llamacpp.soaiplugin
SHA-256 checksum
dbe891d3b2fffcea76dffcc991d73c052508c92e23c34a947b9199360e4649f2
Version 1.0.1
Install from a file
In SoAI, open Plugins → Add plugin → Upload plugin and select the downloaded .soaiplugin file.
Install from a URL
Copy this URL, then open Plugins → Add plugin → Download plugin from URL in SoAI and paste it there.
https://soai.to/v1/llamacpp/1.0.1/8bebb974a76fc2b590caf07ea92cc1805af9404986bb3a78f4d1397cf9b3d1e5/llamacpp.soaiplugin
SHA-256 checksum
8bebb974a76fc2b590caf07ea92cc1805af9404986bb3a78f4d1397cf9b3d1e5