A virtual model puts several catalog models behind one stable name. Clients select that name in Chat or through the OpenAI-compatible API, and SoAI picks an eligible member for each request, which lets you spread load across models or fall back to another one without changing anything in the client. Use it when the members can genuinely serve the same workload, because a text model and an embedding-only model do not become interchangeable by sharing a name.
Create and route a virtual model
Every constituent must be an active, enabled catalog model whose plugin is installed, enabled, and not in an incompatible or error state, and a virtual model needs at least 2 of them with no duplicates. SoAI checks each one when you create or edit the alias and names both the model it rejected and the reason, so a rejection tells you what to fix. There is no upper limit on membership and no per-member weight or priority: for failover the configured order is the priority, and for load balancing the order is recalculated on every request.
Passing that check is not a promise that a member runs. At request time SoAI evaluates each one again and drops any that has been disabled or is no longer active, cannot satisfy the capabilities or modalities the request asks for, sits behind a plugin in a prohibitive state or an open circuit, or is still inside a cooldown from a recent failure. Only what survives that pass is ordered and tried.
Create a virtual model
- In the SoAI WebUI, open Models and select Virtual models.
- On Create virtual model, enter a Virtual model name. Letters, numbers, spaces, slash, underscore, period, and hyphen are accepted, and the name needs at least one letter or number.
- Choose Load balancing or Failover under Strategy.
- Search Available models and select at least 2 constituents.
- Select Create.
The name is the client-facing identity, so it cannot duplicate another virtual model or the name or alias of a local model. Once created, it appears in the catalog and in model pickers as soon as they refresh, and in the OpenAI-compatible model list beside ordinary models, where its advertised capabilities and modalities are the union of what its members support. A request that needs a capability only one member has still succeeds, because the runtime check drops the members that cannot serve it, but the advertised profile is broader than any single member.
API replies and streamed events keep the virtual model's name in their model field, including a
Responses object stored for later retrieval. Routing can select a different member without changing that
client-facing name.
Choose a routing strategy
Failover keeps the order you configured, skips the members that are not eligible, and starts with the first one that is. Choose it when one model is the one you actually want and the others exist to cover it.
Load balancing rebuilds the order for every request, scoring each eligible member on how much work is already
reserved against it, how much its plugin has active and queued, and whether choosing it would make a local backend
swap the model it currently holds. Members that score identically take turns when
MODELS.ROUTING.FAIR_TASK_ROTATION
is on, which spreads a steady stream of similar requests instead of sending all of them to the same leader; that
rotation is kept per virtual model and only breaks a tie at the front of the order.
When a member fails
Both strategies fail over. After an attempt fails for a reason worth retrying elsewhere, SoAI puts that member on a cooldown, excludes it from the rest of that request, and moves to the next member in the plan. Retry counts and backoff against a single target are the ordinary retry policy, which Resilience and self-healing describes; the virtual model adds a second target rather than replacing that behavior.
Failover has one hard boundary. It can only happen before the client has been given anything, so once a response has started streaming or delivery is otherwise under way, SoAI cannot move that request to another member and the failure reaches the client. A virtual model covers a member that fails to start or fails early, not one that dies mid-response.
When nothing is eligible, what happens depends on why. If the members are held out only by cooldowns, the request is deferred rather than failed and reports that the backends are on cooldown, and the same deferral applies to the whole alias for one cooldown window after a request through it fails terminally. If they are ineligible for a reason waiting cannot fix, such as none of them supporting a requested capability or modality, the request fails with a reason naming that requirement.
MODELS.ROUTING.HEALTH_CHECKS.VIRTUAL_MODEL_FAILOVER_COOLDOWN_SEC
sets both windows and defaults to 15 seconds. Change it after looking at task outcomes, plugin state, and queue
depth rather than before, because a long cooldown hides a backend that is flapping while a short one sends work
straight back into it. Scheduler and runtime tuning covers the rest of the
routing controls.
Manage, disable, and delete
Open Virtual models, then Manage virtual models. The edit action changes the strategy or the constituent list, while the name is fixed after creation because it is what clients send, so renaming means creating the replacement and deleting the original. Delete asks for confirmation and removes only the alias, never the models behind it.
To take one out of use without deleting it, turn off its toggle on the Models page. A disabled virtual model leaves the model pickers and the OpenAI-compatible model list and stops accepting requests, and its constituents keep working under their own names. The Edit model dialog and its Availability control belong to ordinary models, and a virtual model opens Edit virtual model instead.
A virtual model defined in
MODELS.ROUTING.VIRTUAL_MODELS
belongs to the base routing configuration and is read-only to both the API and the WebUI, so an attempt to edit,
enable, disable, or delete it is rejected and says why. Change it where it is defined instead, then follow the
reported restart or reload behavior.
Permissions
Viewing virtual models and routing configuration requires Routing read access
(MODEL_ROUTING_READ), which Standard
users hold by default. Creating, editing, enabling, disabling, or deleting one requires Model routing
(MODEL_ROUTING_ADMIN) and is restricted
to Administrators. Constituent models stay subject to their own model and plugin permissions.