How we check the coach
The engine evaluates the position. The coach has a different job: explain what changed and show it on the board. We test the moves, the explanations and the diagrams together.
Verification snapshot: , after the local Game Coach corrections. These are regression-test coverage counts, not an accuracy percentage or a measure of how much someone learns.
What the current checks cover
The frames include starting positions and subsequent moves, not 1,760 separate test games. These counts exclude additional comparison scenes under More analysis. A demonstration being available does not mean its explanation has earned a teaching-quality grade.
The primary-demonstration count is higher than in the 20 September snapshot (115) because the coach now names some events a few moves into the checked line, described below. An outcome at the end of a long line is still not promoted to an explanation of why the first move was worse. Useful full lines remain available as explicitly labelled comparisons, with step-by-step diagrams and comments.
What remains unresolved: in the same 194 cases, 72 do not have an isolated cause, and 1 is withheld because its evidence does not match. The coach must not turn either into a confident causal explanation. Where the evidence is compatible, comparison lines can still show useful ideas without claiming to prove that cause.
How we test it
- A fixed set, not fresh cherry-picking. We keep the same 50 public Lichess rapid games and 194 recorded turning points, selected from the losing side's most costly moves. Replaying their stored engine lines lets us compare code changes against the same evidence. This is a regression set, not a representative sample of every player or position.
- The exact board matters. The checks replay legal continuations and bind each teaching frame to its position. They test move arrows, named pieces, capture and recapture captions, and whether a claimed pin actually prevents the recapture being discussed.
- Counterexamples as well as successes. Separate advanced-teaching checks cover 20 positive examples and 18 negative examples, plus 420 local king-and-pawn table checks. These are focused engineering checks, not 458 independent assessments of coaching quality.
- A separate model review, with a dated scope. In the earlier 18 September review, GPT-6 Pro reviewed the specified repaired source and reproduced all 194 recorded stories and scene objects. It reported 52 focused tests without a failed check and closed those specified repair findings. These include repeated positions and boundary variations. That historical receipt is not approval of later source changes. This was independent AI-model review, not human chess-specialist certification.
What changed in recent updates
24 September. The coach now reads a few moves into the checked engine line. It can name a check followed by a capture, the winning sequence a better move had, a guard drawn away or taken, and a fork that the played move walks into. Each new capture claim needs the checked line to keep that material to its end.
Blue-panel upgrade. Teaching demonstrations now connect explanations to the exact board, with arrows and rings for the relevant relationships. Longer checked lines can be read in stages, with a separately labelled comparison where available. Additional bounded checks cover defender departure, overload and deflection, king-and-pawn endings, selected rook-ending techniques and pawn-race continuations.
The repair review also challenged false pin explanations, stale capture captions and mismatched move evidence. Closing those specific findings does not establish that every possible explanation is correct.
The blue-panel work adds bounded comparisons of shared development, different piece jobs, pawn exchanges and files, rook routes and queen retreat squares. It also distinguishes attacks from legal captures, binds move names and diagrams to their exact positions, and keeps full recorded continuations available in readable stages. These are specific teaching features, not evidence that every strategic motif is covered.
What happened to the old score?
An earlier version of this page reported 103 of 194 explained clearly on 8 September. That was an older grading result, not the current upgrade's score. The new verification does not supply a comparable fresh teaching-quality grade, so we are not converting demonstration counts or passed tests into one.
We also no longer present “zero wrong” as a current, product-wide claim. Passing a finite set of checks cannot establish that.
30 September evaluation. A separately frozen Game Coach candidate was assessed on 1,808 paired positions and failed the agreed factuality, lead, comparison and timing distribution gates. Later corrections and interface checks are development evidence; they do not turn that failed result into an unseen pass or establish the repaired build’s overall distributions.
What these checks cannot tell you
- Whether every mistake, good move or strategic idea in your games will receive a useful explanation. Some still will not.
- Whether an engine line covers every possible defence. A checked continuation is one line unless the displayed proof explicitly establishes more.
- Whether the app improves ratings, learning or subscriptions. We have not measured those outcomes here.
- Whether every phone and browser behaves identically. Earlier versions received representative iPhone Safari checks; this blue-panel upgrade has responsive-browser checks, not a fresh physical-phone certification.
- How this coach ranks against other products or how strong the underlying engine is. This page is not an engine-strength benchmark.
Check the explanation yourself
Game Coach and its coaching are unlimited and free, with no account needed. Open Chess Pangolin, choose Game Coach and import a game. Open a teaching demonstration or checked line when offered. Watch what changes on the board, and distinguish the played move from the comparison.
No language model writes the coach's explanations at runtime. They come from rules applied to the board and the available engine evidence. Language models have been used during development and review; that is a different role.