vllm-radiance vs radiance: the evidence side by side

The engine story depends on which column you read. In the closest workload family, vllm-radiance has shorter non-speculative wall times, while radiance has shorter MTP-4 wall times. Generation estimates and first-token latency tell a more mixed story. Here are the observations, their denominators and the differences that prevent a controlled engine verdict.

For each stack’s configurations and practical operating range, use the dedicated vllm-radiance and radiance pages.

The comparison contract

These are historical deployments on one Minisforum MS-01, 32 GB host RAM, with two R9700 32 GB GPUs. GPU A uses PCIe 4.0 ×8; GPU B uses ×4 through an adapter/redriver. Hardware details.

Sources: Y/Z sweep CSV, radiance campaign CSV, model identities. The recovered harness hash is recorded in the data dictionary. A shared serving name and harness do not prove identical weights, tokenization or cache behaviour.

The 19 October sweep included repeat non-speculative Swift-Quark cells around 373–439 s. That roughly 15% spread is evidence of variability in that setup, not a confidence interval or a universal threshold for the Y/Z and radiance comparisons. There are no matched repeat distributions here to establish statistical significance.

Dual-card wall time

Wall is the elapsed time for the whole cell: ten turns per agent, including waiting and prompt processing. Lower is better. All 16 text rows below are observations, not projected performance.

Relative change = (radiance / named vllm-radiance baseline − 1) × 100. Positive wall change means more time. Source cells: Y1–Y4 / Z1–Z4 in the sweep, R1–R4 / R6-mtp4-n1…n4 in the radiance grid.

Radiance MTP-4 also has shorter observed walls than radiance’s own non-speculative lane: 21.7%, 11.8%, 8.1% and 6.2% at n=1–4. At four agents it is 0.2% longer than historical vllm-radiance non-spec: 402.7 vs 401.8 s. The choice of baseline changes the headline.

The synthetic image-input cell is separate: 685.7 vs 651.9 s, radiance non-spec vs Y5, 5.2% longer. These are procedural screenshots, not the real coding fleets. Radiance’s 13 GiB-tier R9 records 402.0 s vs 402.7 s at 10 GiB; that 0.7 s gap does not establish a tier-speed benefit.

Latency and generation

Three metrics answer three different questions. TTFT measures the wait for first streamed content, including queueing. Mean turn TG averages completion tokens ÷ (request wall − TTFT) over valid turns. End-to-end output rate divides total successful output tokens by the whole cell wall, including prompt work and waits. Neither rate is raw kernel decode speed.

Both tables below compare the MTP-4 lanes: old Z1–Z4 vs new R6-n1…n4. Rates and averages are as exported; short outputs and streaming behaviour make TG estimates sensitive to accounting.

Sources: the same Y/Z sweep and radiance grid. The export columns are ttft_avg_s, turn_tg_avg / tg_avg_tokens_s, and decode_tokens_s. Output quantity is model-generated and can vary; higher aggregate output rate does not mean greater task productivity.

At n=1, radiance MTP-4 shows 25.7% lower mean turn TG, 91.6% higher end-to-end output rate and 11.1% shorter wall. At n=2, its mean TTFT is 88.4% higher. Those counterpoints belong beside the favourable wall-time result.

Within radiance at n=1, turning MTP-4 on changes mean turn TG 54.4 → 137.9 tok/s (+153.5%) and end-to-end output 19.61 → 24.79 tok/s (+26.4%). The previous write-up’s “+90%” mixed a historical non-speculative baseline into this within-engine explanation; it was not the MTP gain.

Deep bands and shallow bands

Selected non-speculative depth bands show lower radiance TTFT than historical Y cells:

This is a selection of bands, not an all-depth speedup. The historical n=2 shallow-band 0.0 is a no-sample marker, not zero latency, so no percentage is calculated from it. The released data does not isolate the mechanism behind these differences.

PP: leave the missing benchmark missing

No validated matched kernel PP measurement is available. The sweep’s pp_effective_ingest_tok_s uses prompt tokens ÷ TTFT and includes the effects of cache reuse. Monitor intake rates and bursty prompt-credit counters have different meanings again. Substituting those into a prefill leaderboard would manufacture a comparison the tests did not run. Definitions and formulas.

Single card: compare depth before latency

The 18 October single-card vllm-radiance profile uses MTP-4, FP8 KV and a 10 GiB tier; radiance TP1 uses MTP-4, FP8 KV and 13 GiB. The checkpoint representations and turn histories differ. At four agents, radiance has a 29.4% smaller growth cap, 88,913 vs 126,000 tokens.

Sources: original follow-up, historical config, radiance TP1. The lower radiance mean TTFT is 85.9%; maximum observed depth is 21.8% lower. These paired facts describe a different operating point, not an equal-depth speedup. Complete single-card report.

Memory: allocation is not throughput

The radiance TP2 pool grows 524,304 → 786,448 → 1,228,816 tokens as slots/context change. The original 524k figure was the four-slot target, not the physical two-card ceiling. The later pool represents +134.4% allocated capacity over that first boot. There is no sustained six-agent 200k grid. A host prefix cache stores completed conversation state; it does not add live GPU concurrency. Measured/projection ladder · capacity CSV.

Real tasks: delivery and clean exit

Radiance’s fleet produces 4/4 complete deliveries, 4/4 reported syntax passes, 2/4 clean deliveries: agents 0–1 time out at 3,600 seconds while agents 2–3 exit cleanly. Historical vllm-radiance dual produces 4/4 complete and clean deliveries in 30:10, with reported behaviour checks. Different budgets, context settings and validation scope prevent a coding-quality ranking. Radiance fleet CSV · historical fleet CSV · play the artefacts.

Evidence map

The result is a set of useful operating observations, with different strengths across wall time, first-token latency, output rate, context and completion. It does not establish a universal engine winner.

Home · vllm-radiance · radiance · Charts · Stuff for nerds · Disclosure