Methodology, scoring, and certification

SoAI Bench is a synthetic OpenCL benchmark. It measures defined compute, memory, mixed, and dispatch-latency workloads on one selected GPU. It does not run an inference model, so its scores are not tokens per second and should not be used as a substitute for model-specific benchmarks.

Workload and scoring

A Standard pass runs ALU, general compute, dense matrix, memory, mixed compute and memory, and latency phases. The compute result divides completed operations across ALU, matrix, general compute, and mixed phases by their total measured duration. Memory throughput divides completed bytes by measured duration in the dedicated memory phase. The latency phase reports microseconds and dispatches per second and retains its diagnostic score separately.

The Standard overall score is calculated directly from 40 percent compute throughput and 60 percent memory throughput. Throughput inputs are clamped at zero and rounded to 6 decimal places before the weighted score is rounded to an integer. The latency score does not affect the overall score. This composite is a benchmark index with differently scaled input units; it is not normalized and does not predict tokens per second.

General compute completes each dispatch before submitting the next while preserving all 96 dispatches in a logical batch. Enqueue and completion overhead remain inside measured wall time. Calibration and warm-up are excluded from scoring, and a phase that does not complete cannot contribute a score.

Certified aggregation

The WebUI Standard flow completes at least 120 seconds of active workload warm-up followed by 5 measured passes. Compilation, process startup, result readback, and CPU validation do not count toward warm-up. Every measured phase runs completed workload batches for at least 1 second. Warm-up does not enter the final score. SoAI takes the median compute, memory, latency, ALU, and matrix result across measured passes, then computes the final weighted score. Duration and completed-work counts are summed across measured passes.

A certified result is leaderboard-eligible after all 5 measured passes provide valid positive workload, accounting, and numerical sample evidence. Throughput variation, signed throughput drift, and available temperature, power, utilization, and authoritative throttling observations are descriptive and do not alter the score or eligibility by an arbitrary threshold.

Each measured pass summarizes sensor readings acquired entirely within its workload phases. Power averages use the available readings, and missing sensors do not count as zero. Warm-up telemetry is aggregated separately. A reading that spans phase completion remains in the run's overall telemetry and safety checks but is excluded from pass statistics, so a pass can have no telemetry even when the run has readings. Saved results keep their recorded data.

Stress workload

Stress runs the mixed workload rather than the full Standard phase set. Its overall score uses 70 percent compute and 30 percent memory, with no latency component. The WebUI starts a 60-second quick stress run. Stress is meant to expose thermal or stability behavior and cannot produce a certified result.

Valid comparisons

A result records the selected device identity, runtime environment, settings snapshot, duration, sample count, pass details, available temperature and power telemetry, throttling state, and terminal reason. Hardware inventory and OpenCL inventory must resolve to the same selected GPU before work begins. One active run is allowed per GPU.

Make comparisons valid

  • Compare only results produced by the same profile and scoring method.
  • Keep GPU model, driver, OpenCL runtime, power and clock settings, cooling, and host load consistent.
  • Use certified Standard results for repeatable score comparisons.
  • Use model and backend benchmarks when the question is token throughput, time to first token, or inference latency.

A difference in SoAI Bench score identifies a difference in this synthetic workload under the recorded conditions. It does not by itself explain which model, quantization, backend, or prompt will perform better.

Related documentation