festbench-web/tests/Feature
PeterChrz ac941fdd27
fix: switch devlogic/sysops totals from geo-mean to token-weighted average
The headline totals were using a per-task geometric mean, which gave
disproportionate weight to short responses with low effective user_tps
(e.g. 2-token SoftBench answers at ~1.2 tok/s). This produced misleadingly
low Devlogic totals (20.5 for asus_210w) when the displayed suite speeds
were all in the 40-50 tok/s range.

Replace geo-mean accumulators (ln_sum/ln_n) with token-weighted
accumulators (rate_token_sum = sum(user_tps × tokens)). The headline total
is now rate_token_sum / total_tokens, matching the methodology already used
by each displayed suite row.

For ScenarioBench, the suite aggregate gen_tps (sum(gen_tokens)/sum(wall))
is weighted by the suite's total generated tokens, keeping the headline
total consistent with the displayed ScenarioBench row.

Results for the 210W bands:
  asus_210w:    devlogic 20.5 → 47.0, sysops 19.5 → 10.6
  sapphire_210w: devlogic 19.6 → 43.7, sysops 8.3 → 8.8
2026-10-05 00:00:19 -04:00
..
Auth Phase 3: results table home page, run detail with owner notes/visibility, my-runs page 2026-09-16 22:41:12 -04:00
Settings Initial commit: Laravel 12 livewire-starter-kit + flux-pro, fortify, spatie/laravel-data 2026-09-16 22:28:11 -04:00
BenchmarkRunTest.php v2 Phase C: suite-aware home table via all_results view, tailored run detail, per-suite tests 2026-09-18 14:49:09 -04:00
CliLoginTest.php v2.2: depthsweep suite (table, model, registry, run-detail branch), band column headers, GpuSeeder in DatabaseSeeder, device-flow + catalog + confirm-picker tests 2026-09-19 21:11:20 -04:00
DashboardTest.php Phase 3: results table home page, run detail with owner notes/visibility, my-runs page 2026-09-16 22:41:12 -04:00
ExampleTest.php Initial commit: Laravel 12 livewire-starter-kit + flux-pro, fortify, spatie/laravel-data 2026-09-16 22:28:11 -04:00
FestbenchImportTest.php v2 Phase C: suite-aware home table via all_results view, tailored run detail, per-suite tests 2026-09-18 14:49:09 -04:00
HomeSnapshotTest.php v2.1: session-grouped summary home page (GPU/model/quant/runner bands, per-suite pass/fail, avg + weighted tok/s), flat task table moved to /results, summary command + tables on schedule and after import 2026-09-18 20:12:30 -04:00
ResultsPageTest.php v2.1: session-grouped summary home page (GPU/model/quant/runner bands, per-suite pass/fail, avg + weighted tok/s), flat task table moved to /results, summary command + tables on schedule and after import 2026-09-18 20:12:30 -04:00
ServerConfigAndSummaryTest.php Rename Server Config to Inference Args in the UI 2026-09-21 08:02:34 -04:00
SessionSummaryTest.php fix: switch devlogic/sysops totals from geo-mean to token-weighted average 2026-10-05 00:00:19 -04:00