Pokertools Arena Brings AI Models to an Observable Poker Benchmark

Summary

Pokertools Arena is a browser-first benchmark for observing and comparing AI model decisions in no-limit Texas Hold’em. Jev and OpenAI-compatible models can occupy the same 2-to-10-seat tournament table, with rising blinds, antes, time banks, eliminations and podium flow. The PokerTools engine generates the poker state, masks hole cards, supplies public context and validates each model’s selected legal action, rather than delegating those rules to prompts. The application runs entirely in the browser without an application server, database or backend API, and can connect to generic OpenAI-compatible endpoints, OpenRouter or local gateways. Users can watch live decisions, replay snapshots, export JSONL logs, create replay PNGs and record table-only WebM video. Each seat receives the same canonical information policy, including its cards, public board, pot, stacks, positions, legal actions and selected public history; opponent hole cards, other models’ reasoning, prior Jev probabilities and provider metadata remain hidden. The benchmark separates a strategy mode with deterministic hero-hand classification from a raw-cognition mode that provides raw cards, and it supports hierarchical family-then-size decisions alongside a flat diagnostic baseline. Its paired fixed-state corpus uses identical immutable states across models, modes and representations, with deterministic interleaving and separate reporting for family accuracy, sizing accuracy, final-action accuracy and behavioral statistics with Wilson 95% confidence intervals. The repository includes offline tests, diagnostics and optional real-API or tournament A/B checks. Release 0.18.1 adds the shared .env parser to the npm launcher and a packed-artifact smoke test for npx module resolution. API keys remain in page memory, while logs, snapshots and release archives exclude them.