v2 Phase E: Postgres prod image (pdo_pgsql), entrypoint runs suites:refresh + snapshot, README v2

Dockerfile

@@ -27,7 +27,7 @@ RUN npm run build
 # Stage 3: production image — FrankenPHP serves the app (no nginx, no supervisor)
 FROM docker.io/dunglas/frankenphp:1.12-php8.5
 
-RUN install-php-extensions pdo_mysql intl zip opcache
+RUN install-php-extensions pdo_pgsql pgsql intl zip opcache
 
 WORKDIR /app
 

README.md

@@ -3,32 +3,72 @@ # festbench-web
 Web viewer for [FestBench](../llm_benchmark) results — LLM benchmark datapoints
 submitted by the `festbench` CLI, displayed in a fast, sortable table.
 
-Stack: Laravel 12 · Livewire 4 + Volt · Flux UI (Pro) · FrankenPHP · MySQL/MariaDB
+Stack: Laravel 12 · Livewire 4 + Volt · Flux UI (Pro) · FrankenPHP · PostgreSQL 17
 
-## Features
+## Architecture (v2)
 
-- **Home** — every datapoint festbench captures (per-task rows across all runs),
+**One table per bench suite**, monthly-partitioned on Postgres:
-  server-side sortable and filterable (suite, level, model, pass/fail, search).
+
-- **Run detail** (`/runs/{run_id}`) — full provenance: model identity, engine
+| Table | Suite-specific columns |
-  build, GPU, image digest, validators, suite versions, plus per-task metrics.
+|---|---|
-- **Ownership overlay** — festbench-submitted data is immutable; owners can add
+| `softbench_results` | section, tool_calls |
-  notes and toggle public/private visibility. Immutability is enforced in the
+| `infrabench_results` | kind (recall/constraint) |
-  model (`BenchmarkRun::OVERLAY_FIELDS`) and covered by tests.
+| `prosebench_results` | section, checks_passed/total, judge, prefill, prompt_tokens |
-- **My Runs** — list your own runs, flip visibility, see pass counts.
+| `psychbench_results` | section, checks_passed/total |
-- `festbench:import` — import a local festbench `benchmarks.db` (SQLite):
+| `scenario_runs` | scenario, outcome, turns, commands, submissions, rejected, t_pass, verdicts |
-  `php artisan festbench:import results/benchmarks.db --user=you@example.com`
+
+All share the core metric columns (ok, decode, user_tps, ttft, t_answer, wall,
+reasoning, visible, total, finish) plus a `payload JSONB` preserving the
+complete source row — future festbench fields need no migration.
+
+**Why:** each suite has a different shape; partitioning keeps appends on a hot
+partition; decommissioning a bench detaches whole partitions; and Postgres
+inlines the `all_results` UNION view so the combined home query uses each
+suite table's indexes directly.
+
+### Suite registry — decommissioning & adding benches
+
+`config/festbench.php` is the single source of truth: suite → model, `active`
+flag, `in_all_results` flag.
+
+- **Decommission a bench**: flip `active => false`, run
+  `php artisan festbench:suites:refresh`. The suite vanishes from the home
+  table and suite filter; historical data stays queryable and old run-detail
+  links keep rendering.
+- **Add a bench**: migration + model, registry entry, refresh.
+
+### Scale design (built lightly, ready for 10k submissions/day)
+
+- `all_results` view — combined home table; suite-filtered queries bypass it
+  and hit the concrete table's indexes.
+- `home_snapshot` — flat precomputed default view, refreshed every 5 minutes
+  (`festbench:snapshot`). At high traffic the request path never runs a UNION.
+- TTL staleness caching (60–300s), **never event-flush** — at high submission
+  rates event flushes would storm the cache; benchmark data a couple of minutes
+  old is correct.
+- Guest pages carry `Cache-Control: public, s-maxage=…, stale-while-revalidate`
+  so a CDN absorbs the bulk of views. Authenticated views are `private,
+  no-store` (they embed the user's private runs).
+- Queue-based CLI ingestion + Redis driver: documented as the required shape
+  for the future submit endpoint (auth session is the prerequisite).
+- Retention: everything is kept forever.
+- Monthly partitions exist for ±1 year at migration time; a scheduled
+  partition-creation command is the follow-up when real submission volume
+  approaches the horizon.
 
 ## Visibility rules
 
-Runs are public by default (matching the CLI's redaction philosophy). Guests see
+Runs are public by default. Guests see public runs only; authenticated users
-public runs only; authenticated users additionally see their own private runs.
+additionally see their own private runs. Festbench-submitted metrics are
+immutable — enforced by a model guard and covered by tests; owners can only
+add notes and toggle visibility.
 
 ## Local development
 
 Requirements: `frankenphp` binary on PATH, Node 20+, Docker/Podman.
 
 ```bash
-docker compose -f docker/develop.compose.yaml up -d   # MariaDB on 127.0.0.1:3306
+docker compose -f docker/develop.compose.yaml up -d   # Postgres 17 on 127.0.0.1:5432
 ./dev.sh                                              # frankenphp :8080 + vite HMR
 ```
 
@@ -42,49 +82,57 @@ ## Local development
 frankenphp php-cli artisan <command>
 ```
 
-The dev DB deliberately lives in a **container, reached over TCP** — the same
+### Importing festbench data
-topology as production, where MariaDB runs on a nearby host. Point `.env`
+
-`DB_HOST`/`DB_PORT` at wherever the DB actually lives.
+```bash
+# everything from a local benchmarks.db
+php artisan festbench:import results/benchmarks.db --user=you@example.com
+
+# one session, surgically, purging existing data first
+php artisan festbench:import results/benchmarks.db \
+  --user=you@example.com --label="deterministic sampling, repeat 1" --fresh
+```
+
+`--label` filters by the run's label; `--fresh` deletes all existing benchmark
+data first; rows route to per-suite tables via the registry; unknown suites
+keep their run but warn about skipped rows.
 
 ## Production (Coolify)
 
-Build pack: **Dockerfile**. The image is a single FrankenPHP process — no
+Build pack: **Dockerfile**. Single FrankenPHP process — no nginx, no php-fpm,
-nginx, no php-fpm, no supervisor.
+no supervisor.
 
-- `Dockerfile` — 3 stages: composer (Flux Pro auth as a BuildKit **secret**),
+- 3 stages: composer (Flux Pro auth as a BuildKit **secret**), node assets,
-  node assets, final `dunglas/frankenphp:1.12-php8.5` + `install-php-extensions
+  final `dunglas/frankenphp:1.12-php8.5` + `install-php-extensions pdo_pgsql
-  pdo_mysql intl zip opcache`.
+  pgsql intl zip opcache`.
-- `docker/entrypoint.sh` — retries `artisan migrate --force`, then `artisan
+- Entrypoint: `migrate --force` (retries) → `festbench:suites:refresh` →
-  optimize`, then `exec frankenphp run`.
+  `festbench:snapshot` → `frankenphp run`.
-- `docker/Caddyfile` — listens on `{$SERVER_NAME::8080}`; Coolify maps its
+- Caddyfile listens on `{$SERVER_NAME::8080}`; Coolify maps its port.
-  port to it. Static files served directly by Caddy, Laravel via `php_server`.
 
 Secrets/env to set in Coolify:
 
 | Variable | Notes |
 |---|---|
 | `APP_KEY` | generate once, keep stable |
-| `DB_HOST` / `DB_PORT` / `DB_DATABASE` / `DB_USERNAME` / `DB_PASSWORD` | the nearby MariaDB |
+| `DB_HOST` / `DB_PORT` / `DB_DATABASE` / `DB_USERNAME` / `DB_PASSWORD` | the nearby Postgres |
-| `SESSION_DRIVER=database`, `CACHE_STORE=database` | no redis needed |
+| `SESSION_DRIVER=database`, `CACHE_STORE=database` | no redis needed yet |
+| `FESTBENCH_HOME_CACHE_TTL` | seconds; default 300 |
 
-Flux Pro authentication: add a secret named `composer_auth`
+Flux Pro authentication: BuildKit secret `composer_auth` (contents = the
-(file contents = the project's `auth.json`) and configure Coolify to build with
+project's `auth.json`). The license key is never committed.
-`--secret id=composer_auth,src=…` (or `docker build --secret` in CI). The
-license key is never committed.
 
 Health check: `GET /up`.
 
 ## Submitting results (CLI integration — separate session)
 
-The schema (`benchmark_runs`, `run_results`) mirrors festbench's SQLite exactly,
+The schema mirrors festbench's output exactly; `festbench:import` is the
-promotes the sortable columns, and stores the **complete source row as JSON**
+reference implementation of the mapping. The authed ingestion endpoint for
-(`run_results.payload`) so new festbench fields require no migration. The
+`festbench --submit` is future work (queue-based, Redis-backed at volume).
-authed ingestion endpoint for `festbench --submit` is future work; the
-`festbench:import` command is the reference implementation of that mapping.
 
-## Deferred
+## Notes
 
-- `scenario_runs` (scenariobench's agentic metrics: turns, submissions, oracle
+- Scheduler must run in production for `festbench:snapshot` (Coolify's
-  verdicts) — intentionally not imported yet.
+  scheduled task or `schedule:work` in a sidecar when needed).
-- Worker mode (`FRANKENPHP_CONFIG="worker ..."`) — available as a one-env-var
+- Worker mode toggle: `FRANKENPHP_CONFIG="worker …"` — documented option, off
-  toggle when throughput demands it.
+  by default.
+- Tests run on in-memory SQLite (plain tables); partitioning is Postgres-only.

docker/entrypoint.sh

@@ -21,6 +21,8 @@ until php artisan migrate --force --no-ansi; do
   sleep 3
 done
 
+php artisan festbench:suites:refresh --no-ansi || true
+php artisan festbench:snapshot --no-ansi || true
 php artisan optimize --no-ansi || true
 
 exec frankenphp run --config /etc/caddy/Caddyfile