How it works
Frontier LLMs read the mathematical claims in ML papers, check each argument against its own assumptions and, where possible, propose a repair. Every report states the claim, what fails, the evidence, how much it matters and how it could be fixed.
Checking a claim
- Read the statement in context. Pin down the definitions, assumptions and paper version.
- Test the argument. Trace the disputed step and check every construction against the stated assumptions, with calculations or explicit counterexamples.
- Trace the consequences. Identify which later results depend on that step and what a correction must establish.
The approach follows the component and dependency method in Nguyen and Zhang’s slides and published workflow.
Severity
- No issues found
- No error identified in the checked claim. This includes concerns already resolved by the paper or its supplement.
- Minor / cosmetic
- Notation or presentation needs correction without changing the mathematical result.
- Repairable
- A proof gap or missing condition that a specific correction closes; the result holds once it is applied.
- Major
- A lemma or stated result fails as written, although the main theorem may survive. The report may still propose a corrected statement.
- Critical
- A central theoretical claim fails under its stated assumptions.
Each finding is graded by what the error breaks: notation, a fixable step, a stated result or the main theorem. Grades then went through an editorial consistency pass, and a paper takes the highest grade among its findings. Severity describes the checked claim only: a broken proof step does not make the theorem false, and a failed lemma does not overturn experiments. Labels such as Lemma error or Proof gap name the kind of problem. Papers with no issues are hidden in the index by default.
Coverage
Indexed does not mean every proof was checked: reviews examine selected claims. The earlier cohort holds 334 papers with at least 300 citations plus 2 reader-nominated papers; citation counts are dated Semantic Scholar snapshots. Paper links go to the primary source, and findings name the version checked.
Corrections and prior work
When a later version, author code or a public discussion addresses an issue, the report says what it changes: a formula, an argument or the scope of a statement.
Prior reports were searched for selected detailed notes only. Notes from the earlier collections were checked against PubPeer, Optimend, MathAudit, paperiswrong, the Black Spatula list and the materials for Bianchi et al. NeurIPS 2026 notes were checked against Pith, arXiv version histories and the authors’ repositories; OpenReview reviews could not be retrieved. Other reports were not searched systematically, although some cite earlier analyses. A listing is not the same error report, and a clean search does not establish priority or rule out an existing correction.
Suggest a correction with a source ↗