Host capacity is the physical boundary for model files, plugin runtimes, database traffic, request queues, and local inference. Choose a storage location for each growing data class, leave recovery headroom, and measure the workload on the host that will run it. Use the guidance below for host resources and filesystem placement. For scheduler controls, use Scheduler and runtime tuning.
Plan storage capacity
SYSTEM.PATHS.SYSTEM_DATA is the
root for most durable state. Models, managed plugin backends, plugin environments, files, the database, Knowledge
indexes, logs, and backups have separate child paths and can be placed on different volumes. Installed
.soaiplugin packages remain relative to the application base.
Use Data storage, retention, and relocation for the complete ownership map and safe relocation sequence. A configuration change does not move existing data.
Place data by access pattern
| Data class | Capacity characteristic | Placement concern |
|---|---|---|
| Primary database | Small relative to models, but latency-sensitive and continuously updated. | Use reliable local storage with correct ownership. Do not share one live database between instances. |
| Models and plugin runtimes | Usually the largest static or slowly changing data. | Prioritize capacity and read throughput. Keep enough temporary headroom for downloads, extraction, cloning, and updates. |
| Managed files and Knowledge indexes | Grows with user uploads and indexed content. | Plan both original files and derived index data; removing a collection is a data operation, not a filesystem shortcut. |
| Logs and metrics history | Continuous writes controlled by retention and sampling. | Keep diagnostic history long enough for operations, then tune collection only after measuring write pressure. |
| Backups | Can duplicate a large share of durable state. | Keep a verified copy off the host. Ensure source and destination have space for an in-progress backup. |
Keep disk admission headroom
SoAI reserves capacity before supported model and plugin downloads, uploads, backups, archive extraction, browser
downloads, and Knowledge writes. Durable inference admission also checks
MODELS.ROUTING.MIN_FREE_DISK_BYTES_FOR_ACCEPT,
which defaults to 268,435,456 bytes. A reserve rejection is a capacity signal, not a reason to bypass validation.
Monitor the exact filesystem that owns each configured path. Free space on the installation volume does not help a model store or backup destination mounted elsewhere. Account for filesystem overhead, temporary files, retained backups, database growth, and the old copy kept during a safe relocation.
Plan memory and compute capacity
Model weights are only part of memory demand. Context, key-value cache, backend workspace, batching, concurrent requests, plugin processes, Core, browser workers, and the operating system also consume RAM or VRAM. Measure the selected model, quantization, context size, and backend together. Do not infer capacity from parameter count or GPU memory alone.
A remote-provider model avoids local inference memory but still uses Core, network, queue, transcript, tool, and file-processing resources. A CPU backend shares system memory with the application and host. Leave enough headroom for failure handling and maintenance instead of planning around a peak that already causes swapping or allocation failures.
Separate compute bottlenecks
- GPU or CPU saturation during inference belongs to the selected model, backend, and concurrency.
- Slow model loading can be storage throughput, model size, backend initialization, or device allocation.
- A growing queue with idle hardware can indicate routing, provider, plugin, or per-owner limits.
- A responsive server with a stuttering page is client rendering; use Personal preferences.
In the SoAI WebUI, use Hardware and Metrics together. Hardware shows device, memory, temperature, power, storage, and network state; Metrics shows request, latency, token, queue, plugin, and model activity. Retained values after a disconnect are not current until the WebSocket supplies a fresh snapshot.
Maintain and measure the host
Updates and path moves can temporarily require both old and new copies. Recovery requires a readable database, matching encryption key, owned file data, and a verified backup outside the failure domain. A single large volume may be simple, but it is not an independent backup. Before increasing workload, verify that the host can still create a backup and restart cleanly under the expected disk and memory pressure.
Measure one change at a time
- Record free space, disk activity, memory, queue depth, model load time, and request latency.
- Repeat a representative workload with known model, prompt length, concurrency, and tool use.
- Change one placement or capacity variable.
- Repeat the workload and keep the change only when the result and recovery margin improve.