vLLM
by SoAI · Bundled
Serve GPTQ, AWQ, GGUF, and transformer models with vLLM, on CPU builds or GPU variants for NVIDIA CUDA, AMD ROCm, and Intel XPU on Linux.
vLLM is a managed local backend for GPTQ, AWQ, GGUF, and transformer models. It supports chat completions, completions, Responses, embeddings, vision, input audio, tool calling, structured output, and JSON Schema where the model supports them.
The bundled variants include CPU builds for Linux x64, Linux arm64, and Apple Silicon, plus NVIDIA CUDA on Linux, AMD ROCm on Linux x64, and an Intel XPU container on Linux x64. The Apple GPU variant and the Windows through WSL variant are currently unavailable. vLLM can also be cloned.
Version 1.1.3
Install from a file
In SoAI, open Plugins → Add plugin → Upload plugin and select the downloaded .soaiplugin file.
Install from a URL
Copy this URL, then open Plugins → Add plugin → Download plugin from URL in SoAI and paste it there.
https://soai.to/v1/vllm/1.1.3/605811a8d1f0a07edf809e88d68024c2abbf7bf6b47e1456ec32783187a75101/vllm.soaiplugin
SHA-256 checksum
605811a8d1f0a07edf809e88d68024c2abbf7bf6b47e1456ec32783187a75101
Previous versions 4
Version 1.1.2
Install from a file
In SoAI, open Plugins → Add plugin → Upload plugin and select the downloaded .soaiplugin file.
Install from a URL
Copy this URL, then open Plugins → Add plugin → Download plugin from URL in SoAI and paste it there.
https://soai.to/v1/vllm/1.1.2/e3a9cbcf0569e1a68313f10b24b7a46ab77921243ce81f5987e68a350e13381b/vllm.soaiplugin
SHA-256 checksum
e3a9cbcf0569e1a68313f10b24b7a46ab77921243ce81f5987e68a350e13381b
Version 1.1.1
Install from a file
In SoAI, open Plugins → Add plugin → Upload plugin and select the downloaded .soaiplugin file.
Install from a URL
Copy this URL, then open Plugins → Add plugin → Download plugin from URL in SoAI and paste it there.
https://soai.to/v1/vllm/1.1.1/b50e38f8db5fbb3be38425ca8cc0e49f7c38b14ffdce8e8a0e91fd1430021d2c/vllm.soaiplugin
SHA-256 checksum
b50e38f8db5fbb3be38425ca8cc0e49f7c38b14ffdce8e8a0e91fd1430021d2c
Version 1.1.0
Install from a file
In SoAI, open Plugins → Add plugin → Upload plugin and select the downloaded .soaiplugin file.
Install from a URL
Copy this URL, then open Plugins → Add plugin → Download plugin from URL in SoAI and paste it there.
https://soai.to/v1/vllm/1.1.0/518ff628b4643214022b70e9dbd5f8984d0b31307181966b62e7ed1efd93b93f/vllm.soaiplugin
SHA-256 checksum
518ff628b4643214022b70e9dbd5f8984d0b31307181966b62e7ed1efd93b93f
Version 1.0.1
Install from a file
In SoAI, open Plugins → Add plugin → Upload plugin and select the downloaded .soaiplugin file.
Install from a URL
Copy this URL, then open Plugins → Add plugin → Download plugin from URL in SoAI and paste it there.
https://soai.to/v1/vllm/1.0.1/a02501ea5e317a2934b069af044e9e52b1939909e35138c381a62adb7a88d3e6/vllm.soaiplugin
SHA-256 checksum
a02501ea5e317a2934b069af044e9e52b1939909e35138c381a62adb7a88d3e6