# Decision Runtime benchmark results, 2026-09-29

Environment: Apple M5 Max (18 cores: 6 performance + 12 efficiency), 48 GB RAM, macOS 26.6.2,
Python 3.11.4, runtime 0.3.0 (released wheel helixor_runtime-0.3.0-py3-none-any.whl with
helixor_constraint_network-1.0.1, pure-Python build), pydantic 2.13.5, fastapi 0.142.1,
starlette 1.7.0, uvicorn 0.54.0 (the versions pip resolved in a fresh virtual environment).
Load average (1 minute) 3.09 to 3.41 during the runs. Three runs per benchmark, five fresh
processes for cold start, run as described on the Reproduce page. The scripts ran unchanged.
Previous results for runtime 0.2.1: results-2026-09-28.md.

## bench_engine_time.py (engine-reported latency_us; 5,000 calls)
| Run | evaluations/s (wall clock) | mean us | p50 | p90 | p99 | max |
|---|---|---|---|---|---|---|
| 1 | 40,436 | 18.28 | 18.50 | 21.08 | 26.67 | 88.62 |
| 2 | 39,513 | 18.75 | 18.83 | 21.42 | 27.46 | 330.50 |
| 3 | 39,135 | 19.01 | 19.25 | 21.42 | 28.50 | 178.38 |

## bench_evaluate_e2e.py (call time; 100 warm-up + 10,000 calls)
| Run | p50 us | p90 | p99 | max | mean | ops/s | engine p50 / p99 |
|---|---|---|---|---|---|---|---|
| 1 | 24.75 | 27.12 | 35.54 | 91.42 | 24.53 | 40,768 | 18.54 / 27.33 |
| 2 | 24.29 | 26.46 | 32.50 | 73.33 | 23.85 | 41,921 | 18.29 / 24.96 |
| 3 | 24.96 | 29.12 | 36.46 | 693.92 | 25.37 | 39,413 | 18.67 / 28.00 |

## bench_cold_call.py (5 fresh processes)
import 111.35-128.50 ms; construct 0.56-0.62 ms; first evaluate 101.8-132.5 us; second evaluate 44.9-53.9 us;
whole process 0.14-0.16 s wall time (/usr/bin/time -p).

| Run | import ms | construct ms | first evaluate us | second evaluate us | process s |
|---|---|---|---|---|---|
| 1 | 128.50 | 0.62 | 111.9 | 49.0 | 0.16 |
| 2 | 113.61 | 0.56 | 112.6 | 49.1 | 0.14 |
| 3 | 113.79 | 0.56 | 110.2 | 45.5 | 0.15 |
| 4 | 116.10 | 0.59 | 132.5 | 53.9 | 0.15 |
| 5 | 111.35 | 0.60 | 101.8 | 44.9 | 0.14 |

## bench_streaming.py (2,048-byte text, 8-char fragments, 20 streams)
| Run | push p50 us | p90 | p99 | max | 2 KB stream p50 |
|---|---|---|---|---|---|
| 1 | 58.96 | 71.21 | 136.79 | 211.88 | 15.06 ms |
| 2 | 58.25 | 71.21 | 133.67 | 216.33 | 14.95 ms |
| 3 | 58.83 | 70.67 | 135.29 | 199.00 | 14.95 ms |

## bench_http.py (POST /v1/evaluate, loopback, one keep-alive connection, 50 warm-up + 1,000)
| Run | p50 us | p90 | p99 | max | req/s | server engine p50 us |
|---|---|---|---|---|---|---|
| 1 | 450.75 | 483.21 | 523.46 | 996.00 | 2,211 | 27.83 |
| 2 | 461.83 | 731.46 | 1157.46 | 7616.96 | 1,898 | 29.33 |
| 3 | 456.71 | 547.38 | 844.17 | 1071.71 | 2,090 | 29.75 |

Same script, same runtime wheel, in a second virtual environment pinned to the web stack of the
0.2.1 runs (fastapi 0.123.0, starlette 0.50.0, uvicorn 0.40.0), load average 1.89 to 1.97:

| Run | p50 us | p90 | p99 | max | req/s | server engine p50 us |
|---|---|---|---|---|---|---|
| 1 | 416.25 | 451.54 | 500.33 | 605.17 | 2,376 | 28.42 |
| 2 | 413.58 | 440.67 | 474.12 | 557.96 | 2,401 | 28.33 |
| 3 | 406.00 | 445.96 | 494.21 | 556.83 | 2,428 | 27.71 |

## Notes
- Import no longer loads the GPU solver's numeric libraries: with numpy installed in the
  environment, `import helixor_runtime` leaves numpy and torch out of sys.modules. Import is
  111-129 ms, still above the 100 ms cold-start target. `python -X importtime` attributes most of
  it to the runtime's own modules and the validation library.
- Engine time, call time and streaming are within run-to-run noise of the 0.2.1 figures.
- The HTTP round trip is higher than the 0.2.1 runs (340-360 us p50). About 40-50 us of the
  difference goes away with the older web stack; the rest was not attributed.