Pith. sign in

REVIEW 4 cited by

Compact Proofs of Model Performance via Mechanistic Interpretability

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2406.11779 v14 pith:JTVQGVMY submitted 2024-06-17 cs.LG cs.LO

Jason Gross , Rajashree Agrawal , Thomas Kwa , Euan Ong , Chun Hei Yip , Alex Gibson , Soufiane Noubir , Lawrence Chan This is my paper · ORCID
classification cs.LGcs.LO
keywords mechanisticmodelperformanceproofsinterpretabilityboundscompactfind
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We propose using mechanistic interpretability -- techniques for reverse engineering model weights into human-interpretable algorithms -- to derive and compactly prove formal guarantees on model performance. We prototype this approach by formally proving accuracy lower bounds for a small transformer trained on Max-of-K, validating proof transferability across 151 random seeds and four values of K. We create 102 different computer-assisted proof strategies and assess their length and tightness of bound on each of our models. Using quantitative metrics, we find that shorter proofs seem to require and provide more mechanistic understanding. Moreover, we find that more faithful mechanistic understanding leads to tighter performance bounds. We confirm these connections by qualitatively examining a subset of our proofs. Finally, we identify compounding structureless errors as a key challenge for using mechanistic interpretability to generate compact proofs on model performance.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Interpretability in Parameter Space: Minimizing Mechanistic Description Length with Attribution-based Parameter Decomposition

    cs.LG 2025-01 conditional novelty 7.0 of 10

    Attribution-based Parameter Decomposition splits a network's parameters into faithful, minimal, and simple components and recovers ground-truth mechanisms in toy models of superposition and compressed computation.

  2. Input Pathways Shape Few-Shot, Not Zero-Shot, Binding in Tiny Transformers: A Fully-Enumerable Study

    cs.LG 2026-07 accept novelty 6.0 of 10

    In information-matched tiny transformers, zero-shot compositional binding fails for every route, while few-shot efficiency is governed by input-pathway sharing and code readability.

  3. Modular addition without black-boxes: Compressing explanations of MLPs that compute numerical integration

    cs.LG 2024-12 conditional novelty 6.0 of 10

    A trained modular addition transformer's MLP layer is shown to compute its outputs by numerical integration, with a trig integral identity and linear-time error bounds.

  4. SATORI: Static Test Oracle Generation for REST APIs

    cs.SE 2025-08 unverdicted novelty 5.0 of 10

    SATORI statically infers REST API test oracles from OpenAPI specs via LLMs, reporting F1 74.3%, above AGORA+'s 69.3%, with 18 confirmed bugs; the supplied full text, however, is a different paper.

Pith tools