Computation and Language · arXiv:2610.01066 v1 · submitted 1 Oct 2026 · 73 references
Probe with Participation Trophies: Random-Reward RL as a Probe of LLM Capability
The paper says
“We connect the spurious-reward paradox to a model's reachability and propose random-reward reinforcement learning (RL) as a useful tool for the probing enterprise, addressing a decade-long debate over what probing performance actually reveals about a model.”
more of the abstract
There are two prevailing explanations for the surprising finding that even random rewards can improve the performance of large language models (LLMs): one attributes the gains to particular mechanisms within RL training; the other to data contamination. Our results motivate a different view: spurious-reward RL can probe a model's reachability, or what further training can attain from its current state under specified constraints, beyond what is reflected in its current performance. Two OLMo checkpoints with the same accuracy on synthetic arithmetic (3.5%), for example, reach 8.5% and 55% in their best runs under the same correctness-rewarded RL. Examining OLMo checkpoints across pre-training and mid-training reveals three distinct regimes of training response: early on, RL produces little improvement even when correct answers are rewarded; later in pre-training, rewarding correct answers becomes effective while random rewards remain weak; and, upon entering mid-training, even random rewards can produce large gains. A similar ordering appears in a number-masked supervised fine-tuning (SFT) analysis of these checkpoints, suggesting that the pattern is not specific to a particular RL mechanism. Moreover, RL with random rewards offers a distinctive perspective on what training can make an LLM do, since its reward signal supplies no information about which answers are correct. By asking what training can attain without correctness feedback, it addresses the label-leakage side of a central problem in decodability-based probing: whether a successful probe reveals the model's capabilities or learns the task itself.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 2 Oct 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. A serious, honestly-reported empirical study proposing random-reward RL as a reachability probe; the three-stage story is plausible but the load-bearing autodidactic claim currently rests on a lenient scorer that collapses under strict extraction. This line is wrong
2 major objections · 6 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 73 references say.
The paper says, at §5 / Fig. 5c; App. E.1, Table 7; App. D.1.2
The review did not quote this passage.
Pith says: Objection 1 of 2 · major
The three-stage pattern, and especially the autodidactic stage, is read off the primary lenient GSM extractor (Appendix D.1.2: a number after '####', an annotated equation, an answer phrase, or the last number in the response). Table 7 shows that all fifteen random-reward runs ending at or above 50% lenient accuracy fall below 50% under strict extraction, and the four headline P2 +5B runs (59.0–77.5% lenient) score 0.0% strict.
more of objection 1
The paper acknowledges this collapse in E.1 and offers one MMLU-condition example of genuine computation at lenient-correct/strict-0 ('3 * 8 = 24 ... 84 - 24 = 60', E.5), which rules out a purely format-driven reading; but no output-level evidence is shown for the P2 +5B runs, and the GT runs at these checkpoints are never rescored under strict, so whether the three-stage ordering survives a stricter scorer is unmeasured. Because the autodidactic stage is the paper's distinguishing claim, the manuscript should either provide output-level verification of genuine computation for the headline runs and a strict rescore of the GT runs, or re-state the autodidactic stage in strict-score terms.
The paper says, at §5, Eq. (7)
The review did not quote this passage.
Pith says: Objection 2 of 2 · major
The text states that 'we can derive the three stages from the ordering ΔGT ≥ Δrand proved in Section 4.' No such ordering is proved in Section 4: Theorem 4.1 establishes verifier-independence of the training trajectory, and Eq. (6) establishes Δrand ≤ Δreach by construction since random-reward RL is one allowed procedure.
more of objection 2
The stages of Eq. (7) are a classification of observed gain magnitudes whose '≈0' versus '>0' thresholds are never defined; they are an empirical description, not a consequence of a proved inequality, and the 'derivation' language should be corrected.
6 minor objections
The paper says, at Abstract; §3 and App. E.9.2, Table 30
Pith says: minor objection 1 of 6
The 462B versus 839B example (both 3.5% frozen, reaching 8.5% and 55% under GT) is presented in the abstract as motivation, but with four GT seeds per checkpoint the permutation test gives p = 0.125 (Table 30). The paper acknowledges this in E.9.2; the abstract and Figure 1 should carry a brief hedge given the small-sample basis.
The paper says, at Eqs. (3) and (6), §4
Pith says: minor objection 2 of 6
The paper measures only Δrand for a single fixed procedure, a lower bound on a reachability supremum that is never computed; interpretive phrasing such as 'probe a model's reachability' should consistently be cast as 'probe a lower bound on reachability under the random-reward procedure,' since the gap between Δrand and ReachB is unknown.
show 4 more
The paper says, at App. E.1, Table 7; App. D.1.2
Pith says: minor objection 3 of 6
Because the lenient extractor's final rule accepts 'the last number in the response,' a cheap and informative diagnostic would be a prompt-echo rate: how many lenient-correct answers coincide with a number appearing in the question. This would bound the contribution of the last-number rule to the autodidactic gains.
The paper says, at Fig. 6(b); App. E.4
Pith says: minor objection 4 of 6
Figure 6(b) overlays the RL-derived stage bands on the SFT panel although the SFT comparison is acknowledged as qualitative, with unmatched budgets, checkpoint coverage, and per-seed results unavailable; the bands may overstate the parallelism between the two procedures.
The paper says, at Table 2 / §5, Eq. (7)
Pith says: minor objection 5 of 6
The GT and Random conditions differ not only in reward but also in the reference coefficient (0 for GT, 0.01 for Random, Table 2). The paper discloses this in D.2, but the receptive-stage boundary (ΔGT > 0, Δrand ≈ 0) could be sensitive to β, and the main text near the stage definitions should restate the disclosure.
The paper says, at Author list; App. D.9, Table 33 caption
Pith says: minor objection 6 of 6
Minor presentation issues: the first author's name lacks an affiliation superscript in the author list, and 'no p value' in the Table 33 caption should be 'no p-value.'
What would settle it
Inspect the actual step-500 responses of the 15 random runs in Table 7 (especially the four P2 +5B runs) together with the matched GT runs at P2 +5B and P1 3,896B, and independently annotate whether each lenient-correct extraction is preceded by a genuine derivation (as in the MMLU example in E.5) or is a bare, prompt-echoed, or trailing number without working. If genuine derivations dominate, the concern is resolved and strict-0 reflects format collapse; if bare numbers dominate, the reachable-capability claim is not established. Reporting GT strict scores at the same checkpoints would additionally show whether the three-stage pattern survives a scorer-independent check.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 2 Oct 2026
- Reader holds up, with conditions
- Skeptic agrees
- Editor wrote the verdict above
- Referee major revision: 2 major objections, 6 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
Send it out to a serious referee, with one instruction: push on the measurement. The headline autodidactic gains are read off a lenient checker, and the paper's own Appendix E.1 shows all fifteen random-reward runs above 50% lenient fall below 50% strict, with the four P2 +5B headline runs at 0.0 strict. That is not a footnote. The three-stage reading depends entirely on that scorer, and the same scorer is fragile elsewhere: P1 3,896B jumps from 2.5% frozen under the primary rule to 41.5% under a prefix diagnostic (Table 22), and O2 from 15.5% to 53% under a first-block rule (Table 16). The authors never rescore the GT runs under strict, so we do not even know whether the whole dormant/receptive/autodidactic ordering is scorer-dependent. The missing experiment is simple: rescore everything under strict, and show actual outputs from those P2 +5B runs. If they compute correct answers while abandoning the requested format, the autodidactic claim survives and strict-0 is a format requirement; if they emit bare or echoed numbers, the reachable-math-capability claim is not established. To their real credit, the authors flag the strict collapse themselves in E.1, and the one MMLU-run example of genuine computation ('3 * 8 = 24 ... 84 - 24 = 60') cuts against a pure format-artifact reading. The lenient scorer also requires a numerically correct extracted answer, so random chance on a trailing number is unlikely. The residual risk is prompt-echoed or derivation-free numbers, which lenient extraction cannot distinguish. So this is conditional, not fatal. What is genuinely new: the offline shared-rollout experiment (Table 1), which holds rollouts and rewards fixed and varies only the learner, is a clean isolation of learner state from rollout quality that the cited spurious-reward accounts (Chen et al. 2025, Shao et al. 2026) do not provide. The training-input controls separating the math template from correctness content (E.5) are also well done. The three-stage taxonomy is useful even though it is task-specific and the autodidactic stage appears at only 3 of 12 mid-training checkpoints. Soft spots beyond the scorer: the framework is partly definitional. Reachability is a supremum that is never computed; Eq. (6) is a lower bound by construction, so the 'derivation' of three stages from delta_GT >= delta_rand (Section 5) is reading off a trichotomy, not a theorem. Seed counts are small (4 GT, 8 random), most pairwise contrasts are underpowered, and the P1-to-P2 checkpoints differ in data mixture and schedule as well as progress, which the authors acknowledge. Recommendation: peer review, conditional. Ask for strict-scorer robustness, output-level evidence from the headline runs, and either more power or a reframed claim. The work is 932 runs, all seeds reported, code shipped, and the methodological contribution will serve the interpretability audience regardless of how the central interpretation resolves.
The strongest objection
The three-stage pattern, and especially the autodidactic stage, is read off a lenient GSM extractor (Appendix D.1.2) that accepts a number after '####', an annotated equation, an answer phrase, or the last number in the response. Under the stricter standalone '#### <number>' rule, all 15 random runs ending above 50% lenient fall below 50% strict, and the four headline P2 +5B runs score 0.0 strict (Table 7). The concern is not confined to those runs: the 462B vs 839B equal-frozen-accuracy contrast (3.5% vs 3.5%, reaching 8.5% vs 55.0%) and the shared-rollout learner comparison both read off the same primary scorer, whose fragility the paper itself documents — P1 3,896B scores 2.5% under the primary rule but 41.5% under a prefix rule (Appendix E.7, Table 22), and O2 scores 15.5% vs 53.0% under a first-block diagnostic (Appendix E.3, Table 16). The authors acknowledge the strict collapse (E.1) and offer one MMLU-condition example of genuine computation at lenient-correct/strict-0 ('3 * 8 = 24 ... 84 - 24 = 60'), which cuts against a pure format-artifact reading. But no equivalent output evidence is shown for the P2 +5B headline runs, and the GT runs at these checkpoints are never rescored under strict, so it is unknown whether the entire three-stage ordering is scorer-dependent. Because the lenient scorer requires a numerically correct extracted answer, random chance on a trailing number is unlikely; the residual risk is that a run reproduces a number from the prompt or emits a plausible final number without derivation, which lenient extraction cannot distinguish from computed answers. This is the most load-bearing unresolved point: if the four high P2 +5B runs compute correctly while abandoning the requested format, the autodidactic capability claim survives and strict-0 is a format requirement; if they emit bare numbers, the central claim of reachable math capability without correctness feedback is not established.
The reader's weakest_assumption is the lenient-scorer measurement validity, and I agree this is the single most load-bearing unresolved point. The central claim has two pillars: Theorem 4.1 (no correctness information flows through the random reward channel, which is cleanly argued via the coupling in Appendix C.2) and the empirical three-stage pattern, whose autodidactic stage is the novel content. That empirical pillar collapses or survives depending on whether lenient-correct extractions are genuine computation. The paper is unusually transparent: it reports the strict collapse (Table 7), the extraction-sensitivity diagnostics (Tables 16 and 22), the underpowered shared-rollout tests (Table 21), and the P2 data-mixture confound (Appendix D.1.2). It also offers real independent support in a design sense: the shared-rollout learner contrast, the training-input controls, and the synthetic-data construction are genuine methodological contributions, and the MMLU example shows at least one case where lenient-correct output is real arithmetic. Those credits are why I would not escalate the verdict to REJECT: the concern is concrete and testable, and the paper's own example suggests the artifact reading may not hold. But the missing output-level evidence for the exact runs that bear the headline claim (P2 +5B, strict 0.0), combined with the demonstrated instability of the primary scorer across checkpoints, means CONDITIONAL remains the right verdict rather than ACCEPT. I considered the P2-data-mixture confound (mid-training includes TinyGSM-MIND and GSM8K) as an alternative headline concern, but the paper partially addresses it through task multiplicity (lookup, MBPP) and an explicit acknowledgment, whereas the scorer issue affects the very unit of measurement in every reported gain.
The referee's report
The claim is significant if it holds: it offers a correctness-free probe in the Hewitt–Liang tradition, connects the spurious-reward paradox to the probing debate, and, if the autodidactic stage is real, shows that mid-training checkpoints possess latent capability that training without correctness feedback can activate. The paper has genuine strengths: Theorem 4.1's pathwise coupling proof is clean; the shared-rollout design (Appendix E.7) isolates the learner's contribution beyond rollout quality; all seeds and nonsignificant results are reported with Holm corrections; and the authors disclose multiple scorer-sensitivity diagnostics (Tables 7, 16, 22) rather than concealing them. The central empirical claim, however, rests on a lenient answer extractor whose collapse under strict scoring the paper documents but does not resolve for the headline runs; that gap is the load-bearing issue for the paper's distinguishing contribution.
Pith's reading of the paper (a machine's summary, not the authors')
This paper claims that reinforcement learning with random rewards is a probe: because the reward carries no information about whether a response is correct, whatever a checkpoint gains from such training reveals what further training can attain without correctness feedback, a property the paper calls reachability. Applied across OLMo-2 pre-training and mid-training checkpoints on synthetic arithmetic, it finds three sequential regimes of training response: dormant (neither correct nor random rewards help), receptive (correct rewards work, random rewards stay weak), and autodidactic (random rewards alone produce large gains, up to 77.5% lenient accuracy from a 13.5% start). Two checkpoints with identical frozen accuracy (3.5%) reach 8.5% and 55% under the same correctness-rewarded protocol, which the paper reads as evidence that frozen accuracy understates training potential. If correct, a checkpoint's response to random-reward training reveals latent, not-yet-expressed capability and provides an answer to the long-standing label-leakage objection to probing.
What carries the argument. The central object is reachability, formalized as the highest expected task performance attainable from a fixed checkpoint within a training budget under a prespecified family of adaptation procedures, $\mathrm{Reach}_B(\theta_0) = \sup_{A\in\mathcal{A}} \mathbb{E}_U[J(A_B(\theta_0;U))]$, with random-reward reachability $\Delta_{\mathrm{rand}}(\theta_0)$ giving a lower bound because random-reward RL is one such procedure. Its load-bearing companion is the correctness-free family $\mathcal{A}_0$ of procedures whose training signals, selection decisions, and stopping rules are verifier-independent, together with Theorem 4.1, which couples two runs under different verifiers using identical random draws and shows their trajectories agree pathwise. The framework is anchored in predictive $V$-information: conditioning on the question already determines the answer in the Shannon sense, so the object of study is whether training makes that information computationally accessible.
Core claim. On its own terms, the paper establishes that random-reward RL supplies no correctness information through the reward channel, while still improving task performance in a way that depends on the starting checkpoint and the information in the prompt. Theorem 4.1 proves that when each reward is an independent fair coin flip, the mutual information between the reward and response correctness conditioned on training history and input is zero, and the whole training-trajectory law is invariant under changing the answer verifier; any gain under this protocol therefore occurs without the model being told which responses are correct. Empirically, the paper reports a three-stage developmental pattern across OLMo-2 checkpoints — dormant, receptive, autodidactic — observed not only under random-reward RL but also mirrored by number-masked supervised fine-tuning, and shows through a shared-rollout offline experiment that the same responses and rewards produce larger gains in the later checkpoint, isolating the learner state as a determinant of training response.
The premise it rests on. The load-bearing premise is that the lenient answer-extraction scorer measures real reachable mathematical capability: the paper itself reports that all fifteen high random-reward runs fall below 50% under strict extraction and that four score 0.0, so if the autodidactic gains are mostly shifts in output format or decoding rather than in computing correct answers, the central claim that mid-training checkpoints reach high reachable capability without correctness feedback is not established.
If the paper is right (Pith's inferences, not the authors')
A checkpoint's frozen accuracy can understate its training potential: two OLMo-2 checkpoints both scoring 3.5% reach 8.5% and 55% under the same correctness-rewarded RL, so identical current performance does not imply identical capacity to improve.
Post-training responsiveness develops in stages, so a model's receptiveness to further training depends on its training position; the same reward protocol that barely moves an early checkpoint lifts a mid-training one to high accuracy.
Task cues in the prompt matter even when rewards carry zero correctness information: with the math instruction retained, random-token prompts reach 68.5% terminal accuracy, while template-free random prompts stay between 14.5% and 23%.
The three-stage ordering is not specific to RL: number-masked supervised fine-tuning shows the same relative development, and code-generation and compositional-lookup tasks exhibit similar patterns with task-specific timing.
The theory entails the ordering $\Delta_{\mathrm{GT}} \ge \Delta_{\mathrm{rand}}$, and the paper claims OLMo-2 traverses the three regimes in sequence, with relatively distinct transitions.
If reachability tracks capability not expressed in frozen accuracy, a checkpoint scan of this kind could be used as a scheduling criterion — deciding when to stop pre-training and begin post-training — an operational use the paper does not itself propose.
The strict-versus-lenient extraction gap (all 15 high random-reward runs fall below 50% strict, four scoring 0.0) suggests testing whether autodidactic gains survive formatting-independent probes such as pass@k or equivalence-class consistency across equivalent question formulations; such tests would separate a genuine solving capability from a learned decoding habit.
Because the verifier-invariance theorem holds for any verifier-independent procedure, the same three-stage pattern should appear under other correctness-free objectives such as entropy or format-only training, offering a cheap robustness check on other model families that the paper leaves implicit.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
We connect the spurious-reward paradox to a model's reachability and propose random-reward reinforcement learning (RL) as a useful tool for the probing enterprise, addressing a decade-long debate over what probing performance actually reveals about a model. There are two prevailing explanations for the surprising finding that even random rewards can improve the performance of large language models (LLMs): one attributes the gains to particular mechanisms within RL training; the other to data contamination. Our results motivate a different view: spurious-reward RL can probe a model's reachability, or what further training can attain from its current state under specified constraints, beyond what is reflected in its current performance. Two OLMo checkpoints with the same accuracy on synthetic arithmetic (3.5%), for example, reach 8.5% and 55% in their best runs under the same correctness-rewarded RL. Examining OLMo checkpoints across pre-training and mid-training reveals three distinct regimes of training response: early on, RL produces little improvement even when correct answers are rewarded; later in pre-training, rewarding correct answers becomes effective while random rewards remain weak; and, upon entering mid-training, even random rewards can produce large gains. A similar ordering appears in a number-masked supervised fine-tuning (SFT) analysis of these checkpoints, suggesting that the pattern is not specific to a particular RL mechanism. Moreover, RL with random rewards offers a distinctive perspective on what training can make an LLM do, since its reward signal supplies no information about which answers are correct. By asking what training can attain without correctness feedback, it addresses the label-leakage side of a central problem in decodability-based probing: whether a successful probe reveals the model's capabilities or learns the task itself.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-02
The authors' main claim, as the classifier picked it out 0.97
“Our results motivate a different view: spurious-reward RL can probe a model's reachability, or what further training can attain from its current state under specified constraints, beyond what is reflected in its current performance.”
How the main claim is stated: asserted 0.54
What the abstract itself states, as probabilities: claims first 0.05 · states limitation 0.17 · human participants 0.03 · controlled comparison 0.74 · randomized assignment 0.07 · states quantitative result 0.98 · compares with prior results 0.85 · mentions code or data availability 0.03
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| We connect the spurious-reward paradox to a model's reachability and propose random-reward reinforcement learning (RL) as a useful tool for the probing enterprise, addressing a decade-long debate over what probing performance actually reveals about a model. | contribution 1.00 | 0.55 | 0.31 | asserted 0.92 |
| There are two prevailing explanations for the surprising finding that even random rewards can improve the performance of large language models (LLMs): one attributes the gains to particular mechanisms within RL training; the other to data contamination. | background 0.99 | 0.05 | 0.03 | asserted 0.96 |
| Our results motivate a different view: spurious-reward RL can probe a model's reachability, or what further training can attain from its current state under specified constraints, beyond what is reflected in its current performance. | interpretation 0.92 | 0.53 | 0.83 | asserted 0.54 |
| Two OLMo checkpoints with the same accuracy on synthetic arithmetic (3.5%), for example, reach 8.5% and 55% in their best runs under the same correctness-rewarded RL. | result 1.00 | 0.91 | 0.06 | asserted 1.00 |
| Examining OLMo checkpoints across pre-training and mid-training reveals three distinct regimes of training response: early on, RL produces little improvement even when correct answers are rewarded; later in pre-training, rewarding correct answers becomes effective while random rewards remain weak; and, upon entering mid-training, even random rewards can produce large gains. | result 1.00 | 0.91 | 0.11 | asserted 1.00 |
| A similar ordering appears in a number-masked supervised fine-tuning (SFT) analysis of these checkpoints, suggesting that the pattern is not specific to a particular RL mechanism. | result 0.50 | 0.86 | 0.04 | hedged possibility 0.95 |
| Moreover, RL with random rewards offers a distinctive perspective on what training can make an LLM do, since its reward signal supplies no information about which answers are correct. | interpretation 0.76 | 0.29 | 0.09 | asserted 0.97 |
| By asking what training can attain without correctness feedback, it addresses the label-leakage side of a central problem in decodability-based probing: whether a successful probe reveals the model's capabilities or learns the task itself. | interpretation 0.86 | 0.34 | 0.13 | asserted 0.97 |
What kind of paper the classifier reports this is: empirical study 0.61 · evidence from observation or experiment 0.95
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-02: 1 request, 6,590 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (12)
Showing 11 of the paper's 12 figures. Figures are the authors', reproduced from the paper.
References (73)
- [1] Shivam Agarwal, Zimin Zhang, Lifan Yuan, Jiawei Han, and Hao Peng. 2025. https://openreview.net/forum?id=UfFTBEsLgI&referrer= The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning . In Advances in Neural Information Processing Systems (NeurIPS) on Pith
- [2] Guillaume Alain and Yoshua Bengio. 2017. https://openreview.net/forum?id=ryF7rTqgl Understanding Intermediate Layers Using Linear Classifier Probes . In International Conference on Learning Representations (ICLR) on Pith
- [3] Yonatan Belinkov. 2022. https://aclanthology.org/2022.cl-1.7/ Probing Classifiers: Promises, Shortcomings, and Advances . Computational Linguistics, 48(1):207--219 on Pith
- [4] Nora Belrose, Igor Ostrovsky, Lev McKinney, Zach Furman, Logan Smith, Danny Halawi, Stella Biderman, and Jacob Steinhardt. 2025. https://arxiv.org/abs/2303.08112 Eliciting Latent Predictions from Transformers with the Tuned Lens . Preprint, arXiv:2303.08112 on PithPith's reviewarXiv
- [5] Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds , Alex Tamkin, Karina Nguyen... on Pith
- [6] Meng Cao, Lei Shu, Lei Yu, Yun Zhu, Nevan Wichers, Yinxiao Liu, and Lei Meng. 2024. https://aclanthology.org/2024.emnlp-main.515/ Enhancing Reinforcement Learning with Dense Rewards from Language Model Critic . In Conference on Empirical Methods in Natural Language Processing ... on Pith
- [7] Peter Chen, Xiaopeng Li, Ziniu Li, Wotao Yin, Xi Chen, and Tianyi Lin. 2025. https://openreview.net/forum?id=sE8DCSJTzd Exploration vs Exploitation: Rethinking RLVR through Clipping, Entropy, and Spurious Reward . In International Conference on Learning Representations (ICLR) on Pith
- [8] Xi Chen, Mingyu Jin, Jingcheng Niu, Yutong Yin, Jinman Zhao, Bangwei Guo, Dimitris N. Metaxas, Zhaoran Wang, Yutao Yue, and Gerald Penn. 2026. https://openreview.net/forum?id=3uC9teMlUt All Circuits Lead to Rome: Rethinking Functional Anisotropy in Circuit and Sheaf Discovery ... on Pith
- [9] Emily Cheng, Diego Doimo, Corentin Kervadec, Iuri Macocco, Lei Yu, Alessandro Laio, and Marco Baroni. 2024. https://openreview.net/forum?id=0fD3iIBhlV Emergence of a High-Dimensional Abstraction Phase in Language Transformers . In International Conference on Learning Represent... on Pith
- [10] Donovan Clay, Saket Gollapudi, Sankar Harilal, Min Jang, Jacob Morrison, Sewoong Oh, and Natasha Jaques. 2026. https://arxiv.org/abs/2608.24949 Demystifying Reinforcement Learning Post-Training of Language Models . Preprint, arXiv:2608.24949 on PithPith's reviewarXiv
- [11] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. https://arxiv.org/abs/2110.14168 Training Verifiers to Solve Math Word Problems . ... on PithPith's reviewarXiv
- [12] Alexis Conneau, German Kruszewski, Guillaume Lample, Lo \"i c Barrault, and Marco Baroni. 2018. https://www.aclweb.org/anthology/P18-1198 What You Can Cram into a Single \ &!\#* Vector: Probing Sentence Embeddings for Linguistic Properties . In Annual Meeting of the Associatio... on Pith
- [13] Yufei Cui, Yu Mao, Ziquan Liu, Qiao Li, Antoni B. Chan, Xue Liu, Tei-Wei Kuo, and Chun Jason Xue. 2021. https://arxiv.org/abs/2101.11353v2 Variational Nested Dropout on PithPith's reviewarXiv
- [14] Yanai Elazar, Shauli Ravfogel, Alon Jacovi, and Yoav Goldberg. 2021. https://aclanthology.org/2021.tacl-1.10/ Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals . Transactions of the Association for Computational Linguistics (TACL), 9:160--175 on Pith
- [15] Zitian Gao, Yilong Chen, Haoming Luo, Joey Zhou, and Bryan Dai. 2026. https://openreview.net/forum?id=0yUoCFSUaY&referrer= One-Shot Entropy Minimization for Language Model Reasoning . In International Conference on Machine Learning (ICML) on Pith
- [16] Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. https://aclanthology.org/2021.emnlp-main.446 Transformer Feed-Forward Layers Are Key-Value Memories . In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5484--5495 on Pith
- [17] Wes Gurnee, Nicholas Sofroniew, Adam Pearce, Mateusz Piotrowski, Isaac Kauvar, Runjin Chen, Anna Soligo, Paul Bogdan, Euan Ong, Rowan Wang, Ben Thompson, David Abrahams, Subhash Kantamneni, Emmanuel Ameisen, Joshua Batson, and Jack Lindsey. 2026. https://transformer-circuits.p... on Pith
- [18] Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. https://openreview.net/forum?id=d7KBjmI3GmQ Measuring Massive Multitask Language Understanding . In International Conference on Learning Representations (ICLR) on Pith
- [19] John Hewitt, Kawin Ethayarajh, Percy Liang, and Christopher Manning. 2021. https://aclanthology.org/2021.emnlp-main.122/ Conditional Probing: Measuring Usable Information beyond a Baseline . In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1626--1639 on Pith
- [20] John Hewitt and Percy Liang. 2019. https://aclanthology.org/D19-1275 Designing and Interpreting Probes with Control Tasks . In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2733--2743 on Pith
show 53 more
- [21] John Hewitt and Christopher D. Manning. 2019. https://aclanthology.org/N19-1419 A Structural Probe for Finding Syntax in Word Representations . In Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pages 4129--4138 on Pith
- [22] Yihuai Hong, Meng Cao, Dian Zhou, Lei Yu, and Zhijing Jin. 2025 a . https://aclanthology.org/2025.findings-acl.1111/ The Reasoning-Memorization Interplay in Language Models Is Mediated by a Single Direction . In Findings of the Association for Computational Linguistics: ACL (F... on Pith
- [23] Yihuai Hong, Lei Yu, Haiqin Yang, Shauli Ravfogel, and Mor Geva. 2025 b . https://aclanthology.org/2025.emnlp-main.985/ Intrinsic Test of Unlearning Using Parametric Knowledge Traces . In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 19513--19535 on Pith
- [24] Ziwei Ji, Lei Yu, Yeskendir Koishekenov, Yejin Bang, Anthony Hartshorn, Alan Schelten, Cheng Zhang, Pascale Fung, and Nicola Cancedda. 2025. https://aclanthology.org/2025.emnlp-main.187/ Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinations . In Conferenc... on Pith
- [25] Mingyu Jin, Yutong Yin, Jingcheng Niu, Qingcheng Zeng, Wujiang Xu, Mengnan Du, Wei Cheng, Zhaoran Wang, Tianlong Chen, and Dimitris N. Metaxas. 2026. https://arxiv.org/abs/2603.03415 Farther the Shift, Sparser the Representation: Analyzing OOD Mechanisms in LLMs . Preprint, ar... on PithPith's reviewarXiv
- [26] Mingyu Jin, Qinkai Yu, Jingyuan Huang, Qingcheng Zeng, Zhenting Wang, Wenyue Hua, Haiyan Zhao, Kai Mei, Yanda Meng, Kaize Ding, Fan Yang, Mengnan Du, and Yongfeng Zhang. 2025. https://aclanthology.org/2025.coling-main.37/ Exploring Concept Depth: How Large Language Models Acqu... on Pith
- [27] Mingyu Jin, Qinkai Yu, Dong Shu, Haiyan Zhao, Wenyue Hua, Yanda Meng, Yongfeng Zhang, and Mengnan Du. 2024. https://aclanthology.org/2024.findings-acl.108/ The Impact of Reasoning Step Length on Large Language Models . In Findings of the Association for Computational Linguisti... on Pith
- [28] Andrew Kyle Lampinen, Yuxuan Li, Eghbal Hosseini, Sangnie Bhardwaj, and Murray Shanahan. 2026. https://arxiv.org/abs/2601.20834 Linear Representations in Language Models Can Change Dramatically over a Conversation . Preprint, arXiv:2601.20834 on PitharXiv
- [29] Jin Hwa Lee, Thomas Jiralerspong, Lei Yu, Yoshua Bengio, and Emily Cheng. 2025. https://aclanthology.org/2025.acl-long.265/ Geometric Signatures of Compositionality Across a Language Model's Lifetime . In Annual Meeting of the Association for Computational Linguistics (ACL), p... on Pith
- [30] Yu Mao, Yufei Cui, Tei-Wei Kuo, and Chun Jason Xue. 2022 a . https://dl.acm.org/doi/10.1145/3503161.3548413 Accelerating General-Purpose Lossless Compression via Simple and Scalable Parameterization . In Proceedings of the ACM International Conference on Multimedia, pages 3205... on Pithdoi:10.1145/3503161.3548413
- [31] Yu Mao, Yufei Cui, Tei-Wei Kuo, and Chun Jason Xue. 2022 b . https://dl.acm.org/doi/10.1145/3485447.3511987 TRACE: A Fast Transformer-Based General-Purpose Lossless Compressor . In The Web Conference (WWW), pages 1829--1838. Association for Computing Machinery on Pithdoi:10.1145/3485447.3511987
- [32] Yu Mao, Jingzong Li, Yufei Cui, and Jason Chun Xue. 2023. https://ieeexplore.ieee.org/document/10247866 Faster and Stronger Lossless Compression with Optimized Autoregressive Framework . In ACM/IEEE Design Automation Conference (DAC), pages 1--6 on Pith
- [33] Yu Mao, Jingzong Li, Jun Wang, Hong Xu, Tei-Wei Kuo, Nan Guan, and Chun Jason Xue. 2025 a . https://ieeexplore.ieee.org/document/11132645 Easz: An Agile Transformer-Based Image Compression Framework for Resource-Constrained IoTs . In ACM/IEEE Design Automation Conference (DAC)... on Pith
- [34] Yu Mao, Yuyan Lin, Xue Liu, and Chun Jason Xue. 2025 b . https://openreview.net/forum?id=hQeFXhHcw6 LiteByte: Efficient and Fast-Adapting MLPs for Online Byte-Level Prediction . In ICML Workshop on Methods and Opportunities at Small Scale on Pith
- [35] Yu Mao, Holger Pirk, and Chun Jason Xue. 2025 c . https://arxiv.org/abs/2505.06297 Lossless Compression of Large Language Model-Generated Text via next-Token Prediction . Preprint, arXiv:2505.06297 on PithPith's reviewarXiv
- [36] Yu Mao, Jun Wang, Nan Guan, and Chun Jason Xue. 2025 d . https://www.computer.org/csdl/proceedings-article/cvpr/2025/436400ae342/299b8jFlbQQ WISE: A Framework for Gigapixel Whole-Slide-Image Lossless Compression . In Conference on Computer Vision and Pattern Recognition (CVPR)... on Pith
- [37] Yu Mao, Weilan Wang, Hongchao Du, Nan Guan, and Chun Jason Xue. 2024. https://arxiv.org/abs/2403.01384 On the Compressibility of Quantized Large Language Models . Preprint, arXiv:2403.01384 on PithPith's reviewarXiv
- [38] Kevin Meng, David Bau, Alex J. Andonian, and Yonatan Belinkov. 2022. https://openreview.net/forum?id=-h6WAS6eE4 Locating and Editing Factual Associations in GPT . In Advances in Neural Information Processing Systems (NeurIPS) on Pith
- [39] Jingcheng Niu, Subhabrata Dutta, Ahmed Elshabrawy, Harish Tayyar Madabushi, and Iryna Gurevych. 2025 a . https://openreview.net/forum?id=10QqO1tM1H Illusion or Algorithm? Investigating Memorization, Emergence, and Symbolic Processing in In-Context Learning . Transactions on Ma... on Pith
- [40] Jingcheng Niu, Andrew Liu, Zining Zhu, and Gerald Penn. 2024. https://openreview.net/forum?id=2HJRwwbV3G What Does the Knowledge Neuron Thesis Have to Do with Knowledge? In International Conference on Learning Representations (ICLR) on Pith
- [41] Jingcheng Niu, Wenjie Lu, Eric Corlett, and Gerald Penn. 2022 a . https://aclanthology.org/2022.blackboxnlp-1.27 Using Roark-Hollingshead Distance to Probe BERT's Syntactic Competence . In Proceedings of the BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks fo... on Pith
- [42] Jingcheng Niu, Wenjie Lu, and Gerald Penn. 2022 b . https://aclanthology.org/2022.coling-1.278 Does BERT Rediscover a Classical NLP Pipeline? In International Conference on Computational Linguistics (COLING), pages 3143--3153. International Committee on Computational Linguistics on Pith
- [43] Jingcheng Niu and Gerald Penn. 2020. https://aclanthology.org/2020.eval4nlp-1.11 Grammaticality and Language Modelling . In Proceedings of the Workshop on Evaluation and Comparison of NLP Systems, pages 110--119 on Pith
- [44] Jingcheng Niu, Xingdi Yuan, Tong Wang, Hamidreza Saghir, and Amir H. Abdi. 2025 b . https://aclanthology.org/2025.acl-long.791/ Llama See, Llama Do: A Mechanistic Perspective on Contextual Entrainment and Distraction in LLMs . In Annual Meeting of the Association for Computati... on Pith
- [45] Augustus Odena, Charles Sutton, David Martin Dohan, Ellen Jiang, Henryk Michalewski, Jacob Austin, Maarten Paul Bosma, Maxwell Nye, Michael Terry, and Quoc V. Le. 2021. Program Synthesis with Large Language Models . In N/a, page n/a on Pith
- [46] Team OLMo, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bhagia, Yuling Gu, Shengyi Huang, Matt Jordan, Nathan Lambert, Dustin Schwenk, Oyvind Tafjord, Taira Anderson, David Atkinson, Faeze Brahman, Christopher Clark, Pradeep Dasigi, Nouha Dziri, an... on PithPith's reviewarXiv
- [47] Hadas Orgad, Michael Toker, Zorik Gekhman, Roi Reichart, Idan Szpektor, Hadas Kotek, and Yonatan Belinkov. 2024. https://openreview.net/forum?id=KRnsX5Em3W LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations . In International Conference on Lea... on Pith
- [48] Kiho Park, Yo Joong Choe, and Victor Veitch. 2023. https://openreview.net/forum?id=T0PoOJg8cK The Linear Representation Hypothesis and the Geometry of Large Language Models . In Causal Representation Learning Workshop at NeurIPS on Pith
- [49] Tiago Pimentel and Ryan Cotterell. 2021. https://aclanthology.org/2021.emnlp-main.229 A Bayesian Framework for Information-Theoretic Probing . In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2869--2887 on Pith
- [50] Tiago Pimentel, Josef Valvoda, Rowan Hall Maudslay, Ran Zmigrod, Adina Williams, and Ryan Cotterell. 2020. https://aclanthology.org/2020.acl-main.420 Information-Theoretic Probing for Linguistic Structure . In Annual Meeting of the Association for Computational Linguistics (AC... on Pith
- [51] Abhilasha Ravichander, Yonatan Belinkov, and Eduard Hovy. 2021. https://aclanthology.org/2021.eacl-main.295/ Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance? In Conference of the European Chapter of the Association for Computational Linguistics (EACL)... on Pith
- [52] Rylan Schaeffer, Brando Miranda, and Sanmi Koyejo. 2023. https://openreview.net/forum?id=ITw9edRDlD Are Emergent Abilities of Large Language Models a Mirage? In Advances in Neural Information Processing Systems (NeurIPS) on Pith
- [53] Rulin Shao, Shuyue Stella Li, Rui Xin, Scott Geng, Yiping Wang, Sewoong Oh, Simon Shaolei Du, Nathan Lambert, Sewon Min, Ranjay Krishna, Yulia Tsvetkov, Hannaneh Hajishirzi, Pang Wei Koh, and Luke Zettlemoyer. 2026. https://openreview.net/forum?id=tqTNOpkP5j Spurious Rewards: ... on Pith
- [54] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. https://arxiv.org/abs/2402.03300 DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models . Preprint, arXiv:2402.03300 on PithPith's reviewarXiv
- [55] Erfan Shayegani, G. M. Shahariar, Sara Abdali, Lei Yu, Nael Abu-Ghazaleh , and Yue Dong. 2025. https://openreview.net/forum?id=HRkrWi3FWP Misaligned Roles, Misplaced Images: Structural Input Perturbations Expose Multimodal Alignment Blind Spots . In International Conference on... on Pith
- [56] Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019. https://aclanthology.org/P19-1452 BERT Rediscovers the Classical NLP Pipeline . In Annual Meeting of the Association for Computational Linguistics (ACL), pages 4593--4601 on Pith
- [57] Federico Tiblias, Irina Bigoulaeva, Jingcheng Niu, Simone Balloccu, and Iryna Gurevych. 2025. https://openreview.net/forum?id=vCKZ40YYPr Hypothesis-Driven Feature Manifold Analysis in LLMs via Supervised Multi-Dimensional Scaling . Transactions on Machine Learning Research (TMLR) on Pith
- [58] Elena Voita and Ivan Titov. 2020. https://aclanthology.org/2020.emnlp-main.14/ Information-Theoretic Probing with Minimum Description Length . In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 183--196 on Pith
- [59] Weilan Wang, Yu Mao, Tang Dongdong, Du Hongchao, Nan Guan, and Chun Jason Xue. 2024. https://aclanthology.org/2024.findings-emnlp.988/ When Compression Meets Model Compression: Memory-Efficient Double Compression for Large Language Models . In Findings of the Association for C... on Pith
- [60] Weilan Wang, Yu Mao, Dongdong Tang, Nan Guan, and Chun Jason Xue. 2026. https://ieeexplore.ieee.org/abstract/document/11460785 ECM: Enhancing Compressibility of Quantized Vision Encoder and LLM for Large Vision-Language Models . In IEEE International Conference on Acoustics, S... on Pith
- [61] Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022. https://openreview.net/forum?id=yzkSU5zd... on Pith
- [62] Jason Weston, Antoine Bordes, Sumit Chopra, Alexander M. Rush, Bart van Merri \"e nboer, Armand Joulin, and Tomas Mikolov. 2015. https://arxiv.org/abs/1502.05698 Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks . Preprint, arXiv:1502.05698 on PithPith's reviewarXiv
- [63] Mingqi Wu, Zhihao Zhang, Qiaole Dong, Zhiheng Xi, Jun Zhao, Senjie Jin, Xiaoran Fan, Yuhao Zhou, Huijie Lv, Ming Zhang, Yanwei Fu, Qin Liu, Songyang Zhang, and Qi Zhang. 2026. https://ojs.aaai.org/index.php/AAAI/article/view/40687 Reasoning or Memorization? Unreliable Results ... on Pith
- [64] Yilun Xu, Shengjia Zhao, Jiaming Song, Russell Stewart, and Stefano Ermon. 2019. https://openreview.net/forum?id=r1eBeyHFDH A Theory of Usable Information under Computational Constraints . In International Conference on Learning Representations (ICLR) on Pith
- [65] Lecheng Yan, Ruizhe Li, Guanhua Chen, Qing Li, Jiahui Geng, Wenxi Li, Longyue Wang, and Chenyang Lyu. 2026. https://openreview.net/forum?id=SGUSUm2491 Spurious Rewards Paradox: Mechanistically Understanding How RLVR Activates Memorization Shortcuts in LLMs . In International C... on Pith
- [66] Wannan Yang, Xinchi Qiu, Lei Yu, Yuchen Zhang, Aobo Yang, Narine Kokhlikyan, Nicola Cancedda, and Diego Garcia-Olano . 2025. https://openreview.net/forum?id=YM3RcI3q0E Hallucination Reduction with CASAL: Contrastive Activation Steering for Amortized Learning . In International... on Pith
- [67] Lei Yu, Meng Cao, Jackie CK Cheung, and Yue Dong. 2024 a . https://aclanthology.org/2024.findings-emnlp.466/ Mechanistic Understanding and Mitigation of Language Model Non-Factual Hallucinations . In Findings of the Association for Computational Linguistics: EMNLP (Findings of... on Pith
- [68] Lei Yu, Virginie Do, Karen Hambardzumyan, and Nicola Cancedda. 2024 b . https://openreview.net/forum?id=s5orchdb33 Robust LLM Safeguarding via Refusal Feature Adversarial Training . In International Conference on Learning Representations (ICLR) on Pith
- [69] Lei Yu, Jingcheng Niu, Zining Zhu, Xi Chen, and Gerald Penn. 2025. https://aclanthology.org/2025.emnlp-main.446 Sheaf Discovery with Joint Computation Graph Pruning and Flexible Granularity . In Conference on Empirical Methods in Natural Language Processing (EMNLP) on Pith
- [70] Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang. 2025. https://openreview.net/forum?id=4OsgYD7em5 Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? In Advances in Neural Information Pro... on Pith
- [71] Rosie Zhao, Alexandru Meterez, Sham M. Kakade, Cengiz Pehlevan, Samy Jelassi, and Eran Malach. 2025. https://openreview.net/forum?id=dp4KWuSDzj#discussion Echo Chamber: RL Post-Training Amplifies Behaviors Learned in Pretraining . In Conference on Language Modeling (COLM) on Pith
- [72] Zining Zhu and Frank Rudzicz. 2020. https://aclanthology.org/2020.emnlp-main.744 An Information Theoretic View on Selecting Linguistic Probes . In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9251--9262 on Pith
- [73] Zining Zhu, Soroosh Shahtalebi, and Frank Rudzicz. 2022. https://openreview.net/forum?id=BLXlV3MBLZ5 Predicting Fine-Tuning Performance with Probing . In Challenges & Perspectives in Creating Large Language Models on Pith
Cited by 1 Pith paper
Reviewed papers on Pith that cite this one. The full citation record
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment