Plugin SDK reference

plugin_sdk is the public Python boundary for a .soaiplugin package. It gives plugin authors the contracts, contexts, helpers, and error types needed to work with SoAI without importing private backend implementation modules.

SDK imports and Core contracts

import plugin_sdk
from plugin_sdk import BasePlugin, Config, JSONDict, ModelContext, PluginRuntimeServices, RequestContext

Import public names from plugin_sdk only. plugin_sdk resolves those names lazily, and dir(plugin_sdk) returns the public export list supported by the installed SoAI release.

Do not import a plugin_sdk.* submodule or a private package such as core, features, database, plugins, models, or orchestrator. Relative imports within the package are allowed. With PLUGINS.SECURITY.SAFETY_VALIDATION enabled, package safety validation rejects the prohibited imports.

Core contracts

BasePlugin stores the supplied configuration, event bus, metrics, model registry, plugin manager, runtime reporter, hardware and storage services, runtime flags, files helper, HTTP client, and installation path. Use the PluginRuntimeServices object passed at construction for available runtime capabilities rather than reaching into SoAI internals.

Capability and request helpers

The SDK exports OPENAI_FEATURE_MATRIX, OPENAI_CAPABILITY_CATEGORIES, OpenAIEndpoint, OpenAIImageFeature, OpenAIChatFeature, and OpenAIModality. Use them to keep manifest declarations and request handling aligned with the current capability taxonomy. It also exports helpers for compatible provider request bodies, provider headers, audio upload errors, structured-output errors, embedding batches, and prompt-token-count results.

handle_request receives a request payload, RequestContext, and ModelContext. It returns either a JSON response or an async byte stream. The response type must match the endpoint capability declared in the package manifest.

Privileged operations and failures

Use the guarded network helpers for outbound requests so offline and egress policy are applied consistently. Use the SDK path and archive helpers for files owned by the plugin. Use is_windows_reparse_point on a filesystem observation before treating a retained directory as ordinary staging. These helpers preserve the host's validation and resource-management rules.

Before a container backend reserves space for images or builds, call resolve_docker_storage to find the host storage that the selected Docker engine writes to. Reserve disk space against host_storage_path, and run later engine commands through docker_command so they reach the same endpoint. With Docker Desktop on Windows, that path is the folder that holds the engine's data disk; the disk's virtual size is not free space. Call revalidate_docker_storage with the admitted location before the first build or image change. Remote engines, engines in other virtual machines, and storage that cannot be verified fail with a typed cause such as backend_docker_storage_unmapped or backend_docker_storage_changed instead of a guessed path.

For bounded cleanup, deadline_after(seconds) returns a MonotonicDeadline whose remaining_seconds() method lets successive commands share one budget. Use run_idempotent_current_task_operation to finish an already owned cleanup task when cancellation repeats. An acknowledgement means cancellation was requested; the original backend operation remains active until cleanup finishes. Return success if installation has already committed and its cleanup completes. Raise IpcCancellationCleanupError when resource termination or cleanup cannot be verified, and retain the information needed to reconcile those resources on retry.

Errors and runtime outcomes

The SDK exports SoAIError, ValidationError, ConfigurationError, PluginConfigurationError, ExternalServiceError, OfflineModeError, ProcessError, and resource-exhaustion errors. Convert a problem into the appropriate typed result or exception at the operation boundary. Do not substitute an apparent success after a failed backend operation.

sanitize_public_diagnostic_text(value, max_length=400) redacts sensitive fragments and bounds a native diagnostic reason. It returns no text when the input cannot be safely retained. Preserve a recognized failure cause through a typed error's public message and code; keep unrecognized native text in diagnostic details and provide a safe public message with a trace reference.

For a native runtime check, interpreter_probe_budget(python_executable, ordinary_seconds=...) sizes one time budget for the interpreter and the files it loads, and run_native_probe(argv, budget=budget) runs a command within what remains of that budget. A result whose timed_out is set means the budget ran out. runtime_timeout_error(RuntimeVerificationTimeoutError, subject=..., allowance=budget.allowance) builds the typed timeout with a public message naming the subject, so a slow first check after installation is reported as a verification timeout rather than as an incomplete installation.

spawn_logged_process(..., output_observer=observer) calls observer(stream_name, line) for each decoded stdout or stderr line before log filtering, in the order each stream produced it. Keep the observer fast and non-blocking. To classify a startup failure from native output, pass a startup_failure callable to ServerObservationDependencies. When the process exits during startup, ServerObservation waits for the output to drain and then calls it; a returned exception becomes the startup failure instead of a generic exit report after process cleanup succeeds. A cleanup failure retains operational recovery and takes precedence over a contained compatibility failure. wait_for_process_output_drained gives the same bounded wait to a plugin that inspects output after an exit.

For an asynchronous executable check, call run_native_runtime_probe(plugin, argv, allowance=..., environment=..., cwd=...) with a NativeImageAllowance from native_image_allowance. The returned NativeRuntimeProbeResult includes captured output, exit status, whether the process started or timed out, and elapsed time. Timeout and cancellation wait for termination before releasing capture files. A NativeProbeTerminationError means termination could not be confirmed; retain the affected runtime for recovery instead of deleting or reusing it. classify_llama_server_probe(result, artifact_tag=..., binary_present=...) checks a server's reported version and build, with optional expected_version and expected_commit constraints.

Convert an operating-system file failure with build_filesystem_operation_error(exception, reference=...). Its public error distinguishes files in use, denied access, insufficient disk space, and other file-operation failures without exposing local paths. Keep captured process output and the original filesystem exception in diagnostics; display the typed error's public message and reference.

To inspect which GPUs a plugin's GPU_BINDING selects, pass the configured value to gpu_binding_selected_device_ids. Use the inventory and binding helpers described below to validate physical identities and build the accelerator environment before launch.

For an artifact known to require CUDA 13, call require_cuda_13_device_support(gpu_binding, gpu_inventory) before installation or launch when local GPU execution is required. It raises ValidationError when the effective selection contains only NVIDIA architectures known to be unsupported. A supported or unknown architecture in the selection, or conflicting or stale observations, prevents that verdict. Use the same inventory and configuration snapshots to build the launch environment. require_cpu_instruction_features(capabilities, required) checks an artifact's required instruction names against a complete CPU observation and reports the missing names. Incomplete evidence leaves native validation in place; neither helper substitutes a different device or backend.

A plugin reports a native compatibility failure by raising ValidationError with details["reason"] set to cuda_device_architecture_unsupported, native_cpu_features_unsupported, or openvino_tensor_shape_incompatible. SoAI returns the reason as the stable public error code with its own localized guidance and HTTP status 422. Leave public_message unset for these reasons so the host guidance is used.

Assess declared runtime requirements

assess_native_compatibility(requirements, capabilities, gpu_binding=..., gpu_inventory=...) returns a NativeCompatibilityAssessment. Its status is compatible when declared requirements are satisfied, incompatible when an observed fact proves a violation, or unknown when required observations or requirements are missing, stale or conflicting. The result includes structured reason codes. A compatible result describes those requirements; it does not replace artifact verification or a native readiness probe. require_native_compatibility(assessment) raises a public backend_runtime_incompatible error for a proven violation.

Keep artifact requirements in the plugin's release identity owner. A requirements object can declare driver with a runtime family and explicit minimum or maximum versions, cpu_features as instruction names or a mapping from normalized platform IDs to instruction lists, and device_architecture with observed architecture codes. Its supported field accepts a list or a mapping from normalized platform IDs to lists, so different artifacts can declare different support. Set complete only when the declared requirements are complete for the assessment. A platform missing from a requirements mapping is unknown; requirements for one architecture do not apply to another.

Backend options can include compatibility_requirements, an exact compatibility_artifact_tag, and compatibility_configuration, a mapping of configuration fields to the release values those requirements describe. Variant responses add optional compatibility status and reasons while retaining their existing fields. Use one frozen configuration and inventory snapshot for selection, then reassess before installation changes and launch. Preserve explicit selections and the existing applicability checks for CPU-only or remote execution. Unknown evidence must not become either a fabricated rejection or a claim of support.

A shared GPU observation can remain valid even when sampled_current is false. Architecture assessment accepts a cached observation only with the producer's inventory revision, observation and check timestamps, and an unexpired valid_until_ms. Stale, disconnected, partial, and invalid observations remain unknown. This does not make cached telemetry a new sample. The general require_native_compatibility helper continues to reject proven incompatibility only; an artifact's Auto-selection policy must separately require the evidence needed to establish support.

When a native probe fails, runtime_probe_failure_error(exit_status=..., phase=...) constructs a RuntimeVerificationFailedError with the observed exit status and safe public guidance. An illegal instruction is reported as such without inferring which processor feature caused it. Raw native output remains diagnostic-only. The timeout and process-cleanup contracts still apply independently.

Architecture requirements use execution_scope: "all_bound" by default. Use "runtime_default" for a runtime that executes on one default device, and pass its observed physical identity through runtime_device. Ordered UUID visibility can identify that default; numeric visibility and an all-visible binding need an observation from the runtime. Inventory order alone does not identify a CUDA ordinal. Driver family names are compared without case sensitivity, while conflicting versions remain unknown. For an all-visible initialization scope, assessment includes every observed device of the required vendor, even if a device is ineligible for model placement. Model offload settings do not prove that a native backend skips accelerator initialization.

Before spawning a prepared native runtime, obtain another current inventory and reassess the artifact. native_execution_devices_unchanged(initial_inventory, current_inventory, gpu_binding) checks that the local NVIDIA execution set and selected CUDA visibility tokens still match the prepared environment. It does not certify freshness or artifact support by itself. A changed or unproved identity requires a new launch attempt; do not launch the old environment using a compatibility result for different hardware.

System capabilities can retain usable CPU information when GPU discovery fails. A gpu_discovery outcome with status: "unavailable" includes a safe public error. Before admitting work that depends on GPU discovery, call require_gpu_discovery_capabilities(capabilities) to preserve that specific failure. CPU-only work can continue using the available CPU capabilities; missing GPU data does not establish compatibility with a GPU runtime. Carry the discovery outcome when projecting capabilities for an automatic selector, and validate it before making a hardware-dependent choice. Selections determined solely by the supported operating system and architecture do not require GPU discovery. Preserve the cause across process boundaries, including a topology change; do not replace failed discovery with a CPU-only roster.

Runtime admission can request physical inventory without optional tuning-capability enrichment through snapshot_accelerator_inventory(hardware, include_gpu_capabilities=False). This keeps optional control probes from delaying the observation used for compatibility checks. The default includes tuning capabilities; providers accept the optional include_gpu_capabilities argument. Freshness and physical identity still require validation before execution.

Validate configuration before saving

Override BasePlugin.validate_runtime_configuration(current_config, proposed_config, capabilities, gpu_inventory) to reject a proven incompatible execution selection before it is saved. The default method accepts the configuration. Treat the supplied mappings as read-only snapshots and raise a typed SDK error for a rejected selection. SoAI calls this hook on an already loaded plugin. An unloaded plugin can save configuration repairs without initializing its existing settings; configuration capability and GPU binding checks still apply, and runtime validation remains required before execution. Keep unrelated settings available when an installed runtime needs repair. Validation runs under plugin lifecycle and load ownership, before the configuration transaction acquires its persistence locks. If another writer changes the saved configuration during validation, the request returns HTTP 409 and requires a refresh before another save. Runtime application follows the save; a failure at that point reports that the configuration was saved.

Keep variant metadata in a static resource

BACKEND_VARIANT_OPTIONS accepts a literal option list or the exact expression plugin_sdk.load_backend_variant_options(__file__, "runtime_variants.json"). The filename must be a literal naming a root JSON file with a lowercase portable stem, and the file must contain an option list no larger than 1 MiB. Package inspection reads and validates it without executing plugin code, rejects duplicate JSON keys and applies the same option validation used for literal lists. The SDK reader also rejects symbolic links. Keep the resource in the package so worker loading and archive inspection read the same declarations.

Install and inspect a package in the SoAI WebUI

To test a package, sign in as an Administrator with Plugin administration (PLUGIN_ADMIN), open Plugins, select Add plugin, choose Upload plugin, drop the package on Drop a plugin here or select that area to choose it, then select Upload plugin. Follow progress in the dialog or the Tasks panel in the header. SoAI checks the archive, manifest, dependencies, and compatibility before it loads the worker.

Related documentation

• Creating a new plugin
• Plugin architecture
• Plugins overview
• Install and manage plugins
• Operational runbooks