pch/festbench-llm
1
0
Fork 0
Code Issues Pull requests Projects Releases 6 Packages Wiki Activity Actions

6 releases 6 tags

RSS feed
  • v0.3.1 28b8e682a8

    v0.3.1 Stable

    pch released this 2026-09-12 03:36:57 +00:00 | 4 commits to main since this release

    Guided setup reworked, and four fixes found by using it.

    • the prerequisite check now tests the docker SOCKET, not just that the binary exists. Previously it reported 'container runtime ok' and then failed several prompts later with permission denied.
    • results are written to the working directory. They previously landed beside the binary, which breaks once it is installed to /usr/local/bin.
    • the guided run asks about the connection first and no longer asks for a model twice.
    • API keys are supported for endpoints that require one. Without this a keyed endpoint answers 401 and the suites scored it as model failures.
    • thinking levels are read from the model's own chat template rather than assumed.

    Requires only a container runtime. Linux x86_64.

    sha256sum -c SHA256SUMS
chmod +x festbench_0.3.1_linux_amd64
./festbench_0.3.1_linux_amd64 --check --endpoint <your endpoint>
    Downloads