90 lines
3.7 KiB
Markdown
90 lines
3.7 KiB
Markdown
|
# festbench-web
|
||
|
|
|
||
|
|
Web viewer for [FestBench](../llm_benchmark) results — LLM benchmark datapoints
|
||
|
|
submitted by the `festbench` CLI, displayed in a fast, sortable table.
|
||
|
|
|
||
|
|
Stack: Laravel 12 · Livewire 4 + Volt · Flux UI (Pro) · FrankenPHP · MySQL/MariaDB
|
||
|
|
|
||
|
|
## Features
|
||
|
|
|
||
|
|
- **Home** — every datapoint festbench captures (per-task rows across all runs),
|
||
|
|
server-side sortable and filterable (suite, level, model, pass/fail, search).
|
||
|
|
- **Run detail** (`/runs/{run_id}`) — full provenance: model identity, engine
|
||
|
|
build, GPU, image digest, validators, suite versions, plus per-task metrics.
|
||
|
|
- **Ownership overlay** — festbench-submitted data is immutable; owners can add
|
||
|
|
notes and toggle public/private visibility. Immutability is enforced in the
|
||
|
|
model (`BenchmarkRun::OVERLAY_FIELDS`) and covered by tests.
|
||
|
|
- **My Runs** — list your own runs, flip visibility, see pass counts.
|
||
|
|
- `festbench:import` — import a local festbench `benchmarks.db` (SQLite):
|
||
|
|
`php artisan festbench:import results/benchmarks.db --user=you@example.com`
|
||
|
|
|
||
|
|
## Visibility rules
|
||
|
|
|
||
|
|
Runs are public by default (matching the CLI's redaction philosophy). Guests see
|
||
|
|
public runs only; authenticated users additionally see their own private runs.
|
||
|
|
|
||
|
|
## Local development
|
||
|
|
|
||
|
|
Requirements: `frankenphp` binary on PATH, Node 20+, Docker/Podman.
|
||
|
|
|
||
|
|
```bash
|
||
|
|
docker compose -f docker/develop.compose.yaml up -d # MariaDB on 127.0.0.1:3306
|
||
|
|
./dev.sh # frankenphp :8080 + vite HMR
|
||
|
|
```
|
||
|
|
|
||
|
|
Composer runs through frankenphp's embedded PHP (the host php.ini disables
|
||
|
|
most extensions):
|
||
|
|
|
||
|
|
```bash
|
||
|
|
frankenphp php-cli $(pwd)/vendor/bin/composer install
|
||
|
|
frankenphp php-cli $(pwd)/vendor/bin/pest
|
||
|
|
frankenphp php-cli $(pwd)/vendor/bin/pint
|
||
|
|
frankenphp php-cli artisan <command>
|
||
|
|
```
|
||
|
|
|
||
|
|
The dev DB deliberately lives in a **container, reached over TCP** — the same
|
||
|
|
topology as production, where MariaDB runs on a nearby host. Point `.env`
|
||
|
|
`DB_HOST`/`DB_PORT` at wherever the DB actually lives.
|
||
|
|
|
||
|
|
## Production (Coolify)
|
||
|
|
|
||
|
|
Build pack: **Dockerfile**. The image is a single FrankenPHP process — no
|
||
|
|
nginx, no php-fpm, no supervisor.
|
||
|
|
|
||
|
|
- `Dockerfile` — 3 stages: composer (Flux Pro auth as a BuildKit **secret**),
|
||
|
|
node assets, final `dunglas/frankenphp:1.12-php8.5` + `install-php-extensions
|
||
|
|
pdo_mysql intl zip opcache`.
|
||
|
|
- `docker/entrypoint.sh` — retries `artisan migrate --force`, then `artisan
|
||
|
|
optimize`, then `exec frankenphp run`.
|
||
|
|
- `docker/Caddyfile` — listens on `{$SERVER_NAME::8080}`; Coolify maps its
|
||
|
|
port to it. Static files served directly by Caddy, Laravel via `php_server`.
|
||
|
|
|
||
|
|
Secrets/env to set in Coolify:
|
||
|
|
|
||
|
|
| Variable | Notes |
|
||
|
|
|---|---|
|
||
|
|
| `APP_KEY` | generate once, keep stable |
|
||
|
|
| `DB_HOST` / `DB_PORT` / `DB_DATABASE` / `DB_USERNAME` / `DB_PASSWORD` | the nearby MariaDB |
|
||
|
|
| `SESSION_DRIVER=database`, `CACHE_STORE=database` | no redis needed |
|
||
|
|
|
||
|
|
Flux Pro authentication: add a secret named `composer_auth`
|
||
|
|
(file contents = the project's `auth.json`) and configure Coolify to build with
|
||
|
|
`--secret id=composer_auth,src=…` (or `docker build --secret` in CI). The
|
||
|
|
license key is never committed.
|
||
|
|
|
||
|
|
Health check: `GET /up`.
|
||
|
|
|
||
|
|
## Submitting results (CLI integration — separate session)
|
||
|
|
|
||
|
|
The schema (`benchmark_runs`, `run_results`) mirrors festbench's SQLite exactly,
|
||
|
|
promotes the sortable columns, and stores the **complete source row as JSON**
|
||
|
|
(`run_results.payload`) so new festbench fields require no migration. The
|
||
|
|
authed ingestion endpoint for `festbench --submit` is future work; the
|
||
|
|
`festbench:import` command is the reference implementation of that mapping.
|
||
|
|
|
||
|
|
## Deferred
|
||
|
|
|
||
|
|
- `scenario_runs` (scenariobench's agentic metrics: turns, submissions, oracle
|
||
|
|
verdicts) — intentionally not imported yet.
|
||
|
|
- Worker mode (`FRANKENPHP_CONFIG="worker ..."`) — available as a one-env-var
|
||
|
|
toggle when throughput demands it.
|