Pith. sign in

REVIEW 3 major objections 5 minor 29 references

A transformer feed-forward layer's native residual write is a signed, task-conditioned steering signal: the same layer can push retrieval margins up or down depending on retrieval mode, and the write-gradient sign predicts the direction.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 09:43 UTC pith:3M55YMQ5

load-bearing objection Solid empirical taxonomy of signed FFN writes in retrieval; the s_l sign test is a consistency check rather than an independent mechanism test. the 3 major comments →

arxiv 2607.16254 v1 pith:3M55YMQ5 submitted 2026-06-27 cs.LG

More Than Memory: Task-Conditioned Signed FFN Writes in Long-Context Retrieval

Zhibo Yang This is my paper
classification cs.LG
keywords feed-forward networksresidual streammechanistic interpretabilitylong-context retrievaldirectional derivativesigned rolesretrieval marginattenuation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that the feed-forward networks (FFNs) of large language models are not only parametric memories but also directional actuators in the residual stream: in long-context retrieval, a layer's native write can push the model toward or away from the correct answer. By scaling each layer's own FFN write up and down without editing weights or injecting external vectors, the author finds signed response surfaces: some layers are suppressors (removing them helps, amplifying them hurts), some are amplifiers, and most layers switch role when retrieval changes from literal to semantic. The local directional derivative of retrieval margin along the native write, s_l = ΔF_l^T ∇M, separates the two monotone roles almost perfectly (34/35 suppressors negative, 18/18 amplifiers positive) and predicts attenuation damage on a natural-language QA probe with R²≈0.79. A sympathetic reader would care because it gives a compact diagnostic for which FFN layers to attenuate or replace in retrieval settings, and because it reframes FFN interpretability from storage to direction.

Core claim

On the paper's own terms, the discovery is that the vectors a feed-forward sublayer adds to the residual stream—its native writes—have causal, signed, layer-specific effects on retrieval. Scaling one layer's own write by a gain, without weight edits or external steering, produces response surfaces that are not uniform degradation: some layers are suppressors (removing the write helps, amplifying it hurts), some are amplifiers, and many are calibrated or weak. In controlled literal and semantic retrieval, the final layer is a suppressor in 7 of 8 model–suite pairs and 60% of layers switch role between retrieval modes. The local directional derivative s_l = ΔF_l^T ∇M separates the two monotone

What carries the argument

The central object is the native FFN residual write ΔF_l at each layer, probed by the gain-scaled forward pass h_{l+1} = u_l + αΔF_l and summarized by D_l(α) = M_base − M_l(α). The explanatory quantity is s_l = ΔF_l^T ∇M, the inner product of the native write with the retrieval-margin gradient—the local directional derivative of margin along the write. It anchors the taxonomy: negative s_l means the write points against retrieval (suppressor), positive means with retrieval (amplifier), and the zero-parameter predictor D_l(α)≈(1−α)s_l quantifies attenuation damage.

Load-bearing premise

The load-bearing premise is that the retrieval margin depends approximately linearly on the scaling gain between α=0 and α=2 for monotone layers, so endpoint signs and the local derivative s_l are measuring the same directional fact; if many layers are strongly nonlinear across that interval, the near-perfect sign separation would be an artifact of role definitions rather than an independent confirmation.

What would settle it

A central finite-difference check of the derivative at α=1 (for example, comparing margins at α=0.9 and 1.1) on every suppressor and amplifier layer, computed without invoking the first-order formula; if the sign of the measured derivative agrees with the endpoint-defined role at near-chance rates across a full layer grid, the claim that s_l anchors the monotone roles fails. Alternatively, one suppressor-classified layer with a U-shaped response (both endpoints hurt but a local maximum at α=1) would show the endpoint taxonomy can mislabel non-monotone layers.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Final-layer FFNs in controlled long-context retrieval are usually readout-adjacent suppressors: removing their write improves retrieval margins, so the output-adjacent FFN can inject information that competes with the retrieved answer rather than only storing facts.
  • An FFN layer's role is not a fixed property of what it stores: about 60% of layers flip between suppressor and amplifier roles when the retrieval mode shifts from literal to semantic.
  • The sign of s_l = ΔF_l^T ∇M gives a compact, parameter-free diagnostic for the two monotone roles and for attenuation damage (R²≈0.79 on natural-language QA), so write direction—not just write size—should be measured when evaluating FFN importance.
  • A held-out policy that attenuates the strongest suppressor layers improves retrieval margins over random and norm-matched layer sets, indicating suppressors are causally useful control points.
  • For architectures that decouple, sparsify, or replace FFN computation, the paper implies lookup accuracy alone is insufficient; the replacement write's direction relative to the retrieval gradient should also be checked.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the directional-derivative diagnostic holds across more architectures, a cheap post-hoc pass could flag individual FFN layers whose writes oppose the current task objective, without needing a full gain-scaling grid.
  • The same sign logic could extend beyond retrieval: whenever a readout margin or loss gradient is available, s_l may characterize whether a layer amplifies or suppresses that objective, making the method a general per-layer objective-alignment audit.
  • Because the paper leaves training dynamics open, one testable extension is to track s_l across checkpoints to see when suppressor and amplifier roles form and whether they are stable or reorganized during training.
  • The role-switch result suggests attention-focused explanations of literal-versus-semantic retrieval gaps are incomplete: some of the divergence is localized in FFN write direction, which could be validated by ablating the opposite-sign layers reported in the paper.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies whether native FFN residual writes in Transformer LMs have signed, task-conditioned causal effects on long-context retrieval. The intervention scales each layer's own FFN write by a gain α (0.25, ..., 2) without editing weights, measures retrieval-margin damage D_l(α), and classifies layers into calibrated, suppressor, amplifier, and weak/mixed roles from endpoint signs at α=0 and α=2. A local directional derivative s_l = ΔF_l^T ∇M is then used as a mechanism anchor: the paper reports that suppressors have s_l<0 in 34/35 cases and amplifiers have s_l>0 in 18/18 cases, that 60% of layers switch roles between literal and semantic retrieval, and that s_l predicts attenuation damage with raw R²≈0.79 on a safety-filtered LongBench retrieval-QA probe. The paper also reports threshold sweeps, leave-one-seed-out stability, norm/depth controls, and a held-out suppressor-attenuation policy check.

Significance. If the findings hold, they would sharpen the standard FFN-as-memory view by showing that the direction of the residual write is causally relevant and task-dependent in retrieval settings, and they would offer a compact post-hoc diagnostic for FFN replacement or decoupling studies. The study has notable strengths: direct native-scaling interventions, complete layer-gain grids with auditable prediction files, explicit robustness analyses (threshold sweep, seed LOO, norm controls), a zero-parameter first-order predictor, and a cross-protocol model (SmolLM2). However, the central mechanism test in §5.3 is not fully independent of the role definitions, and the transfer claim uses different metric emphases across protocols. These issues are fixable but require additional analysis.

major comments (3)
  1. [§5.3 and §3.3] The 34/35 and 18/18 sign separation is not an independent confirmation of the s_l mechanism. Roles are defined by endpoint signs at α=0 and α=2 (§3.2). Under the paper's own first-order relation D_l(α)≈(1−α)s_l, a suppressor (D(0)<0, D(2)>0) entails s_l<0 and an amplifier (D(0)>0, D(2)<0) entails s_l>0, provided the response is approximately linear over [0,2]. The paper validates the first-order predictor only for α∈{0.25,0.5,0.75} (§5.3, Table 5) and does not test curvature over the full interval containing α=2. The 61 calibrated layer-suite pairs in Table 4 show the response is not globally linear. Please add a direct curvature/lack-of-fit test (e.g., quadratic fit per layer, or a sign test restricted to layers with small curvature) or redefine roles from attenuation-only data and treat endpoints as a prediction.
  2. [§5.4, Table 6] The headline transfer R²=0.796/0.791 is computed on a post-hoc selected LongBench subset and is emphasized even though the same diagnostic has raw R²=0.074 (Qwen2.5 semantic, Table 5) and 0.139 (Pythia semantic). The paper explains the controlled-semantic gap by shifting to Pearson/MCC, but then uses raw R² on LongBench as evidence of quantitative transfer. To be comparable, report the same diagnostic set (raw R², Pearson, sign accuracy, MCC) for both protocols and specify whether the subset selection and R² metric were fixed before seeing the results. Otherwise the transfer claim is not apples-to-apples.
  3. [§5.4, Appendix D] The held-out policy sanity check selects top suppressor layers using the same single-layer margin-damage screen that defines the suppressor role. Attenuating those layers on a disjoint split is therefore a consistency check of the taxonomy, not an independent test of the s_l diagnostic. To support the diagnostic claim, select layers using s_l alone (or using attenuation data that excludes the endpoint definitions) and show that the held-out policy still outperforms controls.
minor comments (5)
  1. [Abstract] The phrase 'final FFN is a suppressor in 7 of 8 model-suite cases' is correct only when counting SmolLM2 semantic as calibrated; this is stated later, but the abstract could be clearer that the exception is a semantic-suite calibrated role.
  2. [§4] The retrieval-margin definition should specify token-level aggregation: is the margin the mean over gold tokens of per-token logit margins, or the margin computed from mean logits? This affects the interpretation and reproducibility of s_l.
  3. [§5.3, Table 5] Table 5 leaves literal Pearson/MCC entries as dashes. Please fill them or explicitly state why they are omitted, so readers can compare all diagnostics across protocols.
  4. [§5.4] The 'both-help' category appears in the LongBench transfer table but is not in the main controlled taxonomy Table 3. The relationship between 'both-help' and 'weak/mixed' should be stated explicitly.
  5. [§7] The 'Qwen3.5-9B-Base' model needs a public identifier or a detailed configuration (tokenizer, training data, availability) for reproducibility; as written it is ambiguous.

Circularity Check

0 steps flagged

No significant circularity: the s_l sign test is a parameter-free empirical check of the first-order model, not a restatement of the role definitions.

full rationale

The derivation chain is not circular. Roles are defined from forward-scaling endpoints D_l(0) and D_l(2), while s_l = ΔF_l^T ∇M is an independently computed backward-gradient quantity at α=1; no parameter is fitted to the reported R² or sign-separation results. The first-order relation D_l(α) ≈ (1−α)s_l is a stated approximation, and the 34/35 and 18/18 cross-tab is a genuine test of that approximation: endpoint signs do not logically determine the local derivative at α=1 for arbitrary nonlinear response surfaces, and indeed one suppressor exception and the low semantic R² values (e.g., 0.074 for Qwen2.5-7B) show the prediction can fail. The LongBench R²=0.796/0.791 is a zero-parameter prediction on a separate protocol, and the held-out policy check uses disjoint calibration/evaluation splits. There are no load-bearing self-citations or imported uniqueness theorems. The only real caveat is that the interpretability of the sign test depends on approximate linearity over α∈[0,2], which is a robustness/extrapolation concern, not circularity.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The central claims rest on the endpoint-role definitions, the margin proxy, the first-order expansion, and an unverifiable third model. No new physical entities are introduced.

free parameters (4)
  • weak-band threshold for endpoint roles = |D_l(α)|<0.1
    Role labels and role-switch counts use this pre-specified tolerance; the threshold sweep shows qualitative stability, but exact counts shift.
  • policy layer count k = 3 (and 5 in appendix)
    The held-out suppressor-attenuation policy attenuates the top k suppressor layers; k is chosen by hand, though controls use the same k.
  • policy attenuation gain α = 0.5
    The policy sanity check uses α=0.5; α=0 ablations are reported but not emphasized.
  • LongBench examples per subset = 100 per subset, seed 13
    The transfer probe samples 100 Qasper and 100 MultiFieldQA-en examples; the sampling procedure is not specified.
axioms (4)
  • standard math First-order Taylor expansion M_l(α)−M_l(1)≈(α−1)ΔF_l^T ∇M is valid enough at α∈[0,2]
    Invoked in §3.3 to derive D_pred=(1−α)s_l and again in §5.3; load-bearing for the mechanism claim.
  • domain assumption Logit margin of the gold answer is a valid retrieval-quality proxy
    All damage values and role labels are defined on this margin (§4), not on task accuracy.
  • domain assumption Endpoint responses at α=0 and α=2 summarize a layer's causal role
    The taxonomy in §3.2 classifies layers from only these two gains plus a weak band; intermediate response shape is not part of the label.
  • domain assumption Qwen3.5-9B-Base is a real, accessible Qwen-family checkpoint
    §4 describes it as a 32-layer sequential checkpoint but gives no public identifier; all Qwen3.5 results depend on it.

pith-pipeline@v1.3.0-alltime-deepseek · 17249 in / 16218 out tokens · 138119 ms · 2026-08-02T09:43:51.421964+00:00 · methodology

0 comments
read the original abstract

FFNs are often treated as parametric memories. In long-context retrieval, however, the sharper question is not only what they store, but whether their native residual writes push the current retrieval state toward or away from the correct answer. We test this by scaling the model's own FFN write one layer at a time, without editing weights or injecting external steering vectors. Across controlled literal and semantic retrieval suites, native FFN response surfaces are signed, layer-specific, and task-conditioned: the final FFN is a suppressor in 7 of 8 model-suite cases, and 60% of layers switch role between retrieval modes (95% CI [50%, 69%]). A local directional derivative along the native write separates the two monotone roles: suppressors have negative derivative in 34/35 cases, and amplifiers have positive derivative in 18/18 cases, so the roles are not reducible to write size. On a safety-filtered LongBench retrieval-QA probe, the same diagnostic predicts attenuation damage with raw R^2=0.796 on Qwen2.5-7B and 0.791 on Qwen3.5-9B; a held-out suppressor-attenuation policy improves retrieval margins over random and norm-matched controls. These results show that native FFN scaling exposes a signed, task-conditioned residual-write structure in retrieval, and that write-gradient alignment is a compact diagnostic for the two monotone roles.

Figures

Figures reproduced from arXiv: 2607.16254 by Zhibo Yang.

Figure 1
Figure 1. Figure 1: Native FFN residual flow without intervention. Attention first aggregates contextual evidence into the target-token residual state, the FFN computes a native write ∆Fl , and residual addition writes hl+1 = ul +∆Fl into the stream consumed by later attention layers and the output head. No gain scaling or damage metric is applied here. into a residual stream that every subsequent layer reads. In long-context… view at source ↗
Figure 2
Figure 2. Figure 2: FFN residual-write intervention. In a normal forward pass, attention aggregates contextual evidence into the target-token residual state, and the FFN writes a transformed vector ∆Fl back into the same residual stream that later attention reads. Native FFN scaling changes only the strength of that write, hl+1 ← ul + α∆Fl , and the readout margin gives Dl(α) = Mbase − Ml(α). 5 [PITH_FULL_IMAGE:figures/full_… view at source ↗
Figure 3
Figure 3. Figure 3: Signed-role diagnostic. Endpoint responses classify each layer by the signs of Dl(0) and Dl(2), yielding calibrated, amplifying, or suppressive roles. The backward diagnostic measures whether the same native write points with or against the retrieval objective: the forward write ∆Fl and retrieval gradient meet at the same residual node, and the inner product sl = ∆F ⊤ l ∇M is the local directional derivati… view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Final FFN layers are usually readout-adjacent suppressors. In the three main-protocol models, both literal and semantic final FFNs show the same qualitative pattern: α<1 helps and α>1 hurts. Qwen3.5-9B literal is the largest case (α=0 → −3.49, α=2 → +3.10). SmolLM2 semantic is calibrated, showing that the same taxonomy records protocol-sensitive final-layer roles. 5.3 Q3: Local directional derivative expla… view at source ↗
Figure 6
Figure 6. Figure 6: Same-layer FFN effects can flip between retrieval modes. Top opposite-sign examples at α=0.25. Blue bars are literal retrieval; red bars are semantic retrieval. Positive damage means the intervention hurt retrieval [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Suppressor/amplifier roles are predicted by the sign of sl = ∆F ⊤ l ∇M. Across the three full-backward main-protocol models plus SmolLM2 backward anchor layers, suppressor layers have sl<0 in 34/35 cases, and amplifier layers have sl>0 in 18/18 cases. Calibrated layers are shown as the non-monotone comparison class. The right panel illustrates that the same layer’s sl can flip between literal and semantic … view at source ↗
Figure 8
Figure 8. Figure 8: The safety-filtered LongBench retrieval-QA probe also has signed FFN response surfaces. Heatmaps show Dl(α) for Qwen2.5-7B and Qwen3.5-9B across every layer and gain on candidate-scored Qasper and MultiFieldQA-en examples. Signed structure persists under the natural QA probe, while the final-layer role can shift across protocols. endpoint threshold, Qwen2.5 has 3 calibrated layers, 8 suppressors, 0 amplifi… view at source ↗
Figure 9
Figure 9. Figure 9: Residual size is a partial signal, not a taxonomy explanation. (a) FFN relative norm overlaps across endpoint taxonomy types. (b) Leave-one-model-out taxonomy prediction from residual size/depth features. The classifier uses the three signed roles and omits weak/mixed layers; relative norm alone is below balanced chance, while adding depth and residual/cosine features remains weak. What the end-layer suppr… view at source ↗
Figure 10
Figure 10. Figure 10: Four-block attenuation exposes coarse depth roles. Each network is split into four contiguous FFN blocks. Cells show damage from scaling all FFN writes in one block to α=0.25 or α=0.5. Positive values mean block attenuation hurts retrieval; negative values mean it helps [PITH_FULL_IMAGE:figures/full_fig_p020_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

29 extracted references · 5 linked inside Pith

  1. [1]

    Proceedings of EMNLP , year =

    Mor Geva and Roei Schuster and Jonathan Berant and Omer Levy , title =. Proceedings of EMNLP , year =

  2. [2]

    Proceedings of EMNLP , year =

    Mor Geva and Avi Caciularu and Kevin Ro Wang and Yoav Goldberg , title =. Proceedings of EMNLP , year =

  3. [3]

    Proceedings of EMNLP , year =

    Mor Geva and Jasmijn Bastings and Katja Filippova and Amir Globerson , title =. Proceedings of EMNLP , year =. doi:10.18653/v1/2023.emnlp-main.751 , url =

    doi:10.18653/v1/2023.emnlp-main.751 2023

  4. [4]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Kevin Meng and David Bau and Alex Andonian and Yonatan Belinkov , title =. Advances in Neural Information Processing Systems (NeurIPS) , year =

  5. [5]

    Proceedings of ACL , year =

    Damai Dai and Li Dong and Yaru Hao and Zhifang Sui and Baobao Chang and Furu Wei , title =. Proceedings of ACL , year =

  6. [6]

    2021 , url =

    Nelson Elhage and Neel Nanda and Catherine Olsson and Tom Henighan and Nicholas Joseph and Ben Mann and Amanda Askell and Yuntao Bai and others , title =. 2021 , url =

    2021

  7. [7]

    ACM Computing Surveys , year =

    Song Wang and Yaochen Zhu and Haochen Liu and Zaiyi Zheng and Chen Chen and Jundong Li , title =. ACM Computing Surveys , year =

  8. [8]

    Rossi and Seunghyun Yoon and Hinrich Sch

    Ali Modarressi and Hanieh Deilamsalehy and Franck Dernoncourt and Trung Bui and Ryan A. Rossi and Seunghyun Yoon and Hinrich Sch. Proceedings of ICML , year =

  9. [9]

    Conference on Language Modeling (COLM) , year =

    Cheng-Ping Hsieh and Simeng Sun and Samuel Kriman and Shantanu Acharya and Dima Rekesh and Fei Jia and Yang Zhang and Boris Ginsburg , title =. Conference on Language Modeling (COLM) , year =

  10. [10]

    Liu and Kevin Lin and John Hewitt and Ashwin Paranjape and Michele Bevilacqua and Fabio Petroni and Percy Liang , title =

    Nelson F. Liu and Kevin Lin and John Hewitt and Ashwin Paranjape and Michele Bevilacqua and Fabio Petroni and Percy Liang , title =. Transactions of the Association for Computational Linguistics (TACL) , year =

  11. [11]

    Proceedings of ACL , year =

    Yushi Bai and Xin Lv and Jiajie Zhang and others , title =. Proceedings of ACL , year =

  12. [12]

    2024 , url =

    Xinrong Zhang and Yingfa Chen and Shengding Hu and others , title =. 2024 , url =

    2024

  13. [13]

    2023 , url =

    Greg Kamradt , title =. 2023 , url =

    2023

  14. [14]

    One Wide Feedforward Is All You Need , booktitle =

    Telmo Pessoa Pires and Ant. One Wide Feedforward Is All You Need , booktitle =. 2023 , url =

    2023

  15. [15]

    2024 , url =

    Xin Men and Mingyu Xu and Qingyu Zhang and Bingning Wang and Hongyu Lin and Yaojie Lu and Xianpei Han and Weipeng Chen , title =. 2024 , url =

    2024

  16. [16]

    Conference on Language Modeling (COLM) , year =

    Wataru Ikeda and Kazuki Yano and Ryosuke Takahashi and Jaesung Lee and Keigo Shibata and Jun Suzuki , title =. Conference on Language Modeling (COLM) , year =

  17. [17]

    Inference-Time Intervention: Eliciting Truthful Answers from a Language Model , booktitle =

    Kenneth Li and Oam Patel and Fernanda Vi. Inference-Time Intervention: Eliciting Truthful Answers from a Language Model , booktitle =. 2023 , url =

    2023

  18. [18]

    2024 , url =

    Nina Panickssery and Nick Gabrieli and Julian Schulz and Meg Tong and Evan Hubinger and Alexander Matt Turner , title =. 2024 , url =

    2024

  19. [19]

    Peters , title =

    Nishant Subramani and Nivedita Suresh and Matthew E. Peters , title =. Findings of ACL , year =

  20. [20]

    Vazquez and Ulisse Mini and Monte MacDiarmid , title =

    Alexander Matt Turner and Lisa Thiergart and Gavin Leech and David Udell and Juan J. Vazquez and Ulisse Mini and Monte MacDiarmid , title =. 2023 , url =

    2023

  21. [21]

    2025 , url =

    Tetiana Bas and Krystian Novak , title =. 2025 , url =

    2025

  22. [22]

    2024 , eprint =

    Vincent-Pierre Berges and Barlas Oguz and Daniel Haziza and Wen-tau Yih and Luke Zettlemoyer and Gargi Ghosh , title =. 2024 , eprint =. doi:10.48550/arXiv.2412.09764 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2412.09764 2024

  23. [23]

    2026 , eprint =

    Xin Cheng and Wangding Zeng and Damai Dai and Qinyu Chen and Bingxuan Wang and Zhenda Xie and Kezhao Huang and Xingkai Yu and Zhewen Hao and Yukun Li and Han Zhang and Huishuai Zhang and Dongyan Zhao and Wenfeng Liang , title =. 2026 , eprint =. doi:10.48550/arXiv.2601.07372 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2601.07372 2026

  24. [24]

    2026 , eprint =

    Ajay Jaiswal and Lauren Hannah and Han-Byul Kim and Duc Hoang and Arnav Kundu and Mehrdad Farajtabar and Minsik Cho , title =. 2026 , eprint =. doi:10.48550/arXiv.2602.00398 , url =

    doi:10.48550/arxiv.2602.00398 2026

  25. [25]

    2026 , eprint =

    Ranajoy Sadhukhan and Sheng Cao and Harry Dong and Changsheng Zhao and Attiano Purpura-Pontoniere and Yuandong Tian and Zechun Liu and Beidi Chen , title =. 2026 , eprint =. doi:10.48550/arXiv.2601.10639 , url =

    doi:10.48550/arxiv.2601.10639 2026

  26. [26]

    2024 , eprint =

    Xu Owen He , title =. 2024 , eprint =. doi:10.48550/arXiv.2407.04153 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2407.04153 2024

  27. [27]

    2023 , url =

    Stella Biderman and others , title =. 2023 , url =

    2023

  28. [28]

    doi:10.48550/arXiv.2412.15115 , url =

    2024 , eprint =. doi:10.48550/arXiv.2412.15115 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2412.15115 2024

  29. [29]

    2025 , eprint =

    Loubna Ben Allal and Anton Lozhkov and Elie Bakouch and Gabriel Martin Blazquez and Guilherme Penedo and Lewis Tunstall and others , title =. 2025 , eprint =. doi:10.48550/arXiv.2502.02737 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2502.02737 2025