Pith. sign in

REVIEW 4 major objections 6 minor 52 references

Grounding latent algorithm routing in transformer reasoning

T0 review · 4 major / 6 minor · reviewed 2026-07-31 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Dense transformers trained on mixed regimes learn internal route-like variables that pick which inductive-bias family fits each episode.

desk verdict Careful controlled diagnostic that dense transformers can learn route-like inductive-bias selection on a purpose-built benchmark; the circularity worry is real but the adaptive baselines and patching controls keep the result standing. read the letter →

arxiv 2607.24471 v1 pith:OG3JTU6N submitted 2026-07-27 cs.CL

Xiangbo Zhang , Xiaoxu Ma This is my paper
classification cs.CL
keywords in-contextlearninglatentalgorithmroutingtransformerreasoningROUTEBENCHactivationpatchinginductivebiasnuisanceinvariancemixtureofexperts
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a transformer can, from the support set alone, organize its answer around different families of solvers—shrinkage, sparsity, robustness, or locality—when the prompt syntax stays fixed. It defines latent algorithm routing by three checks: the preferred family must track the hidden data-generating regime, stay stable under surface rewrites, and flip under targeted middle-layer activation edits without wrecking answer quality. On ROUTEBENCH, dense models trained from scratch close most of the gap to an oracle that knows the true regime; a 306M model reaches route F1 84.1 and closes 80.9% of that gap, beating fixed solvers, global mixtures, and strong adaptive baselines. Linear probes recover route identity earlier than the answer, and matched activation patching selectively changes the best-fit family while retaining most answer quality. The claim is limited to this controlled setting: it is evidence that route-like internal variables can emerge in dense transformers, not a claim that pretrained language models already implement a universal router.

What carries the argument

Latent algorithm routing, grounded by three criteria—structural necessity, nuisance invariance, and causal editability—and measured by a behavioral proxy that assigns each episode the solver family whose output is closest to the model’s answer, plus mid-layer probes and matched activation patching on ROUTEBENCH’s ridge/lasso/Huber/kNN regime contrasts.

What would settle it

If mid-layer route probes collapsed under label permutation and matched target-route activation patches failed to raise target-family flips above random-source and wrong-route controls while preserving answers, the claim of functionally involved route variables would fail on this benchmark.

Watch

Extended reading notes

Core claim

Dense decoder-only transformers trained from scratch on ROUTEBENCH develop route-like internal variables: under fixed prompt form, solver-family preference tracks the latent regime, remains stable under nuisance perturbations, and can be selectively flipped by middle-layer activation interventions with high answer retention. A 306M model closes 80.9% of the oracle-routing gap with route F1 84.1, outperforming fixed representatives and stronger adaptive alternatives on routing and OOD metrics.

Load-bearing premise

That matching the model’s answer to the nearest fixed solver family, on a benchmark built so those families are clearly best in each regime, is enough to show the model is doing internal algorithm routing rather than continuous interpolation that happens to land near those solvers.

Editorial extensions

If this is right

  • Mixed-regime in-context training can induce internal selection among inductive-bias families without route labels or expert modules.
  • Route identity should be linearly readable earlier than the final answer and concentrated in middle residual states.
  • Targeted residual edits can change which solver family the model behaves like while keeping most answer quality.
  • Scale strengthens accuracy, route recovery, and nuisance robustness together on this diagnostic.
  • The same pattern should be tested before claiming routing in pretrained LMs or open-ended natural-language reasoning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the mechanism generalizes, training mixtures that deliberately pit opposing inductive biases against each other may be a practical way to grow internal routers without MoE hardware.
  • Closest-solver F1 may understate continuous mixtures; future diagnostics could score partial membership against soft oracle weights rather than hard family labels.
  • Natural-language and four-way robustness here suggest a next stress test: hold regimes fixed but move to pretrained checkpoints and free-form instructions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces "latent algorithm routing" as a diagnostic construct for in-context learning: a model is said to exhibit route-like behavior if its inferred solver-family preference (i) changes with the latent data-generating regime under fixed prompt serialization, (ii) is stable under nuisance perturbations, and (iii) is selectively editable by targeted activation interventions. The authors build ROUTEBENCH, a synthetic benchmark with three pairwise regime contrasts (sparse/dense, clean/heavy-tailed, global/local) over a four-member solver bank (ridge-, lasso-, Huber-, kNN-like representatives), and train dense decoder-only transformers (44M–612M) from scratch on mixed-regime next-token prediction without route labels. The 306M model closes 80.9% of the fixed-to-oracle NRMSE gap with route F1 84.1 against the oracle route; the effect persists under natural-language renderings, shuffling, paraphrase, and a unified four-way setting, and exceeds input-conditioned soft-mixture and unsupervised Gumbel-router baselines. Linear probes for the route peak in middle layers earlier than answer probes, and matched middle-layer activation patching flips the behavioral route toward the target family at 73.8% with 96.7% answer retention, with random-source, same-route, and wrong-route controls substantially less target-selective. Claims are explicitly scoped to the controlled benchmark.

Significance. If the results hold, this is a well-constructed controlled positive result on whether dense transformers can organize episode-level adaptation around inductive-bias families, and the diagnostic methodology is a genuine contribution independent of the specific finding. Particular strengths worth naming: the stylized theory (Theorems 1–2, Propositions 1–2) yields falsifiable predictions that are actually tested (e.g., route probes peaking earlier than answer probes, Fig. 3, consistent with Proposition 1); the patching analysis uses matched counterfactual sources with three non-trivial controls (Table 10) rather than a single intervention; the behavioral proxy is stress-tested with margin filtering (Table 8) and permutation/format/episode-ID probe controls (Table 9); seed stability is reported (Table 15); the Switch-MoE diagnostic (Table 11) is a nice explicit-vs-latent routing comparison; and code is publicly available. The authors are also commendably explicit that the evidence is triangulating rather than proof of a discrete symbolic router, and that nothing is claimed about pretrained LLMs. The main reservation, developed below, is that the three grounding criteria as constructed,

major comments (4)
  1. [§3, §5, Table 1, Figure 5] §3 (Grounding criteria) vs. §5/Table 1: the three criteria do not, individually or jointly, exclude the central alternative hypothesis the paper itself raises in §1 — continuous episode-conditioned interpolation. A model that smoothly adapts its effective regularization to episode statistics (e.g., a continuum between ridge-like and lasso-like behavior indexed by decodable regime statistics) would satisfy structural necessity (preference changes with z), nuisance invariance (regime statistics are format-invariant by construction), and causal editability (patching along a regime-statistic direction moves the output toward the target representative, since regimes are linearly separable by design and all three metrics are scored in the metric space of the same four representatives). The one experiment that bears on this, the input-conditioned mixture in Table 1, is confounded by capacity: t
  2. [§4.3, C.2, Tables 6 and 10] §4.3 and Appendix C.2: the 'answer retention' metric is load-bearing for the causal-editability criterion (it is what licenses the claim that route edits preserve answer quality, Table 6 and Table 10) but is not operationally defined. The text refers to 'the intervention evaluator's normalized answer-quality score divided by the corresponding original, unmodified model score' without specifying what the evaluator is: if it is inverse NRMSE on the same episodes, retention and ∆NRMSE should be redundant, yet the text insists they are 'not algebraically derived from one another'; if it is a learned or heuristic scorer, its construction must be described. As written, the retention column cannot be interpreted or reproduced, and the 96.7% figure anchors one of the three grounding criteria.
  3. [C.2, Table 8; cross-ref Tables 1, 6] Table 8 (margin-filtered controls): the headline route F1 (84.1) and patch-flip (73.8%) figures in Tables 1 and 6 are computed over all episodes, but Table 8 shows that on the bottom solver-margin quartile — the episodes where the behavioral proxy is ambiguous — route F1 falls to 66.3 and patch flip to 51.2, i.e., roughly half of the ambiguous episodes do not exhibit the claimed behavior. This is honestly reported in the appendix but is material to interpreting the main-text numbers: on a non-trivial fraction of episodes the proxy cannot distinguish 'model routes' from 'model output is merely nearest to one of four representatives by chance.' The main text should state the margin dependence alongside the headline figures and clarify whether Tables 1/6 numbers are all-episode or margin-filtered, so readers can gauge how much of the effect is carried by unambiguous episodes.
  4. [Tables 1, 2, 6, 15] Tables 1–2 and 6: no uncertainty estimates are given for the main comparisons. Seed variability is reported only for route F1 and consistency at three scales (Table 15), and not for NRMSE, gap-closed, probe AUC, or patch-flip metrics, and not at all for the baselines. Several headline comparisons hinge on moderate differences (306M vs 612M gap closure 80.9 vs 86.5; transformer vs Gumbel router F1 84.1 vs 77.9); without run-to-run variance on these quantities the strength of the scale-ordering claim in §5 and Figure 2 is hard to assess. Reporting multi-seed statistics for at least the 306M model and the two adaptive baselines would address this.
minor comments (6)
  1. [Tables 6, 9] Table 6 and Table 9 report fractional 'earliest 0.75 route layer' values (9.6, 7.8, 8.1, 10.4); the interpolation or thresholding procedure producing non-integer layer indices is not described.
  2. [§5, Table 1] The Gumbel router reaches route F1 77.9 with no route labels (Table 1) — itself arguably evidence for emergent discrete routing in an explicit architecture. A sentence of discussion on why the dense transformer should be expected to exceed an architecture with a built-in discrete bottleneck would sharpen the contribution.
  3. [§4.2, Appendix B] Training token budget, evaluation episode counts, and the capacity/training details of the input-conditioned mixture and Gumbel router baselines are not stated in the main text or Appendix B; these are needed to assess the baseline comparisons and to reproduce.
  4. [Table 12] Table 12 shows the 167M model's per-regime gap to oracle is uneven (0.158 vs 0.144 on sparse structure, but 0.206 vs 0.184 on heavy-tail noise); a brief comment on which regimes are hardest and why would aid interpretation.
  5. [Abstract, §1, §7] Typographical: 'throughlatent' (abstract and §1), 'ROUTEBENCHevaluates' (§7), 'aroute-then-solve' (§1); several LaTeX spacing artifacts in the reference list.
  6. [Figure 5] Figure 5 would be more informative with the oracle crossover point marked and a quantitative transition-sharpness measure, particularly if the interpolation-continuum analysis requested above is added.

Circularity Check

0 steps flagged · score 0.0 of 10

No algebraic or definitional circularity: theory bounds are assumption-driven, route labels are never trained on, and empirical metrics are measurements against an engineered diagnostic—not predictions forced by fitted inputs.

full rationale

The paper’s load-bearing chain does not reduce outputs to inputs by construction. Theorems 1–2 and Propositions 1–2 are elementary regret/mixture and linear-readout arguments under stated assumptions (regime-separated margins; full-column-rank maps; Lipschitz readout); their proofs do not fit parameters to transformer metrics or redefine the target quantity. ROUTEBENCH deliberately engineers regimes so Assumption 1 holds and fixed representatives are differentially optimal—that is experimental design, not a fitted-input-called-prediction. The behavioral proxy ρ̂_beh is explicitly a test-time nearest-solver readout, never a training target; oracle σ(z) is computed from latent-regime risk of the same bank, which makes route F1 and GapClosed diagnostic agreement scores rather than self-fulfilling predictions. Probe permutation/format controls (near-chance AUC) and matched patching controls (random/same/wrong-route) are independent checks, not renamings of the fit. There is no load-bearing self-citation uniqueness theorem, no ansatz smuggled from overlapping prior work, and no claim that specific F1/gap numbers are first-principles derivations. Shared use of the four-representative bank across proxy, oracle, and flip metrics raises construct-validity questions about continuous interpolation versus discrete routing, but that is not circularity under the analyzer’s criteria: nothing equates a claimed prediction to its defining equation or fitted parameter. Scope limitations are stated explicitly. Score 0; steps empty.

Assumptions & free parameters 5 free parameters · 6 assumptions · 3 invented entities

The central empirical claim rests on a constructed multi-regime risk separation, a four-prototype solver bank, a closest-solver behavioral proxy, and standard transformer training—not on fitted physical constants. Free choices are benchmark and training knobs. Axioms are mostly domain assumptions about when routing is necessary and what counts as evidence. The main invented construct is “latent algorithm routing” as a three-criterion behavioral-mechanistic object operationalized on ROUTEBENCH.

free parameters (5)
  • Model scale ladder (44M–612M width/depth/FF multipiers) = 44M, 89M, 167M, 306M, 612M
    Architecture sizes are design choices that drive the reported scaling of gap closure and route F1; not derived.
  • Support size and covariate dimension = m=32, d=24 (main)
    Main benchmark uses m=32 support points in R^24; shot-scaling table varies 8–64. These set task difficulty and route readability.
  • Sparse regime cardinality range ||w||_0 ∈ [2,4] = [2,4]
    Hand-chosen sparsity band that defines when lasso-like family should win.
  • Solver-bank hyperparameters (ridge/lasso/Huber/kNN representatives)
    Family representatives must be fixed to define oracle routes and ρ̂_beh; specific regularization/k are implementation choices (not fully numeric in main text).
  • Training token budget / LR schedule / contrast-balanced minibatching = AdamW, cosine decay, 2k warmup; equal token budget
    Shared optimization recipe across scales; affects whether route variables appear.
assumptions (6)
  • domain assumption Assumption 1: each latent regime has a unique optimal solver family separated by risk margin γ_z ≥ γ_min > 0.
    Load-bearing for Theorems 1–2 and for interpreting ROUTEBENCH as requiring routing (§3.1).
  • ad hoc to paper Behavioral route proxy ρ̂_beh(e) = arg min_k |f_θ(e) − A_k(e)| is a valid measurable stand-in for solver-family preference.
    Defined in §3; never observed in training; all route F1 and flip metrics depend on it.
  • domain assumption Ridge-, lasso-, Huber-, and kNN-like predictors adequately operationalize global shrinkage, sparsity, robustness, and locality families.
    §4.1 family definitions; oracle labels and gaps inherit this operationalization.
  • domain assumption Holding serialization fixed while varying latent episode statistics isolates inductive-bias routing from surface-template shortcuts.
    Core design claim in §3 and Figure 1; motivates nuisance tests.
  • standard math Standard facts: transformers can implement in-context algorithm-like computation; activation patching can test causal roles of internal states.
    Background from cited ICL and interpretability literature used as methodological prior.
  • ad hoc to paper Linear decodability / sufficient edit conditions (Props. 1–2) are the right signatures to expect if route variables live in intermediate residual states.
    Stylized sufficient conditions guiding probe peak and patching experiments, not full transformer proofs.
invented entities (3)
  • Latent algorithm routing (three grounding criteria)
    purpose: Define scientifically meaningful episode-conditioned solver-family selection distinct from template correlation or soft interpolation.
    Introduced in abstract/§3 as structural necessity + nuisance invariance + causal editability; evaluated only on ROUTEBENCH.
  • ROUTEBENCH
    purpose: Diagnostic benchmark with pairwise and four-way regimes favoring four inductive-bias families under fixed prompt form.
    New dataset/task suite; all main metrics are defined on it.
  • Behavioral route proxy ρ̂_beh and solver margin m_sol
    purpose: Infer preferred family without route labels and filter unambiguous episodes.
    Paper-defined readout used for F1, consistency, sensitivity, and intervention flips.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Grounding latent algorithm routing in transformer reasoning." pith.science (2026). https://pith.science/paper/OG3JTU6N

@misc{pith2026260724471,
  author       = {Pith},
  title        = {Pith review of: Grounding latent algorithm routing in transformer reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OG3JTU6N}},
  note         = {Machine review of arXiv:2607.24471}
}
read the original abstract

A central question in the in-context learning literature is whether transformers can organize episode-level adaptation around different inductive-bias families. We study this question in a controlled setting through latent algorithm routing: route-like behavior in which the solver-family preference changes with the latent data-generating regime while prompt form is held fixed, remains stable under nuisance perturbations, and is selectively influenced by targeted activation interventions without large losses in answer quality. We introduce ROUTEBENCH, a diagnostic benchmark whose regimes differentially favor global shrinkage, sparsity, robustness, and locality, operationalized by ridge-like, lasso-like, Huber-like, and kNN-like family representatives. Across dense decoder-only transformers trained from scratch at 44M-612M parameters, a 306M model closes 80.9 percent of the oracle-routing gap and achieves route F1 of 84.1. The effect remains substantial under natural-language renderings, shuffled supports, lexical paraphrases, and a unified four-way routing setting. Stronger adaptive alternatives, including an input-conditioned soft mixture and an unsupervised Gumbel router, narrow the gap but remain below the 306M and 612M models on route F1 and OOD performance. Probe controls and matched activation-patching controls further show that route-relevant internal directions are decodable and functionally involved in solver-family-consistent output behavior. These results provide controlled evidence that dense transformers trained on ROUTEBENCH can develop route-like internal variables, but they do not establish universal routing in pretrained language models or unrestricted natural-language reasoning.

Figures

Figures reproduced from arXiv: 2607.24471 by the authors.

Figure 1
Figure 1. Grounding latent algorithm routing. The serialized prompt format is held fixed [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Scaling trends across three grounding metrics. Left: oracle-gap closure. Middle: [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Layerwise route and answer probes. Route probes peak in middle layers and [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Matched-pair activation patching. Route-flip rates concentrate in middle-layer [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Continuous regime sweeps. Left: as sparsity increases, the 167M model increas [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]
Figure 6
Figure 6. Figure 6: Average route F1 across seeds for the 167M model. [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]
Figure 7
Figure 7. Figure 7: OOD breakdown by perturbation family. Left: nuisance-shift degradations. [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

52 extracted references · 2 canonical work pages

  1. [1]

    Transformers as

    Bai, Yu and Chen, Fan and Wang, Huan and Xiong, Caiming and Mei, Song , year = 2023, month = nov, url =. Transformers as. Thirty-Seventh

    2023

  2. [2]

    Understanding

    Bhattamishra, Satwik and Patel, Arkil and Blunsom, Phil and Kanade, Varun , year = 2023, month = oct, url =. Understanding. The

    2023

  3. [3]

    Brinkmann, Jannik and Sheshadri, Abhay and Levoso, Victor and Swoboda, Paul and Bartelt, Christian , editor =. A. Findings of the. doi:10.18653/v1/2024.findings-acl.242 , url =

    doi:10.18653/v1/2024.findings-acl.242 2024

  4. [4]

    Language

    Brown, Tom and Mann, Benjamin and Ryder, Nick and Subbiah, Melanie and Kaplan, Jared D and Dhariwal, Prafulla and Neelakantan, Arvind and Shyam, Pranav and Sastry, Girish and Askell, Amanda and Agarwal, Sandhini and. Language. Advances in

  5. [5]

    Chan, Stephanie C. Y. and Santoro, Adam and Lampinen, Andrew Kyle and Wang, Jane X. and Singh, Aaditya K. and Richemond, Pierre Harvey and McClelland, James and Hill, Felix , year = 2022, month = oct, url =. Data. Advances in

    2022

  6. [6]

    Findings of the

    Chatterjee, Anwoy and Renduchintala, H S V N S Kowndinya and Bhatia, Sumit and Chakraborty, Tanmoy , editor =. Findings of the. doi:10.18653/v1/2024.findings-emnlp.852 , url =

    doi:10.18653/v1/2024.findings-emnlp.852 2024

  7. [7]

    Conmy, Arthur and. Towards. Thirty-Seventh

  8. [8]

    IEEE Transactions on Information Theory , volume =

    Nearest Neighbor Pattern Classification , author =. IEEE Transactions on Information Theory , volume =. doi:10.1109/TIT.1967.1053964 , url =

    arXiv 1967

Show all 52 references
  1. [9]

    How to Think Step-by-Step:

    Dutta, Subhabrata and Singh, Joykirat and Chakrabarti, Soumen and Chakraborty, Tanmoy , year = 2024, month = apr, journal =. How to Think Step-by-Step:

    2024

  2. [10]

    Switch Transformers:

    Fedus, William and Zoph, Barret and Shazeer, Noam , year = 2022, journal =. Switch Transformers:

    2022

  3. [11]

    Gao, Hongfu and Zhang, Feipeng and Jiang, Wenyu and Shu, Jun and Zheng, Feng and Wei, Hongxin , year = 2024, month = nov, url =. On the. The

    2024

  4. [12]

    Garg, Shivam and Tsipras, Dimitris and Liang, Percy and Valiant, Gregory , year = 2022, month = oct, url =. What. Advances in

    2022

  5. [13]

    Inducing Causal Structure for Interpretable Neural Networks , booktitle =

    Geiger, Atticus and Wu, Zhengxuan and Lu, Hanson and Rozner, Josh and Kreiss, Elisa and Icard, Thomas and Goodman, Noah and Potts, Christopher , year = 2022, pages =. Inducing Causal Structure for Interpretable Neural Networks , booktitle =

    2022

  6. [14]

    doi:10.48550/arXiv.2304.05969 , url =

    Localizing. doi:10.48550/arXiv.2304.05969 , url =. arXiv , keywords =:2304.05969 , primaryclass =

    doi arXiv

  7. [15]

    Adaptive

    Graves, Alex , year = 2017, month = feb, number =. Adaptive. doi:10.48550/arXiv.1603.08983 , url =. arXiv , keywords =:1603.08983 , primaryclass =

    2017 doi arXiv

  8. [16]

    doi:10.48550/arXiv.2404.15255 , url =

    How to Use and Interpret Activation Patching , author =. doi:10.48550/arXiv.2404.15255 , url =. arXiv , keywords =:2404.15255 , primaryclass =

    doi arXiv

  9. [17]

    and Kennard, Robert W

    Hoerl, Arthur E. and Kennard, Robert W. , year = 2000, journal =. Ridge. doi:10.2307/1271436 , url =. 1271436 , eprinttype =

    2000 doi

  10. [18]

    Hong, Guan Zhe and Dikkala, Nishanth and Luo, Enming and Rashtchian, Cyrus and Wang, Xin and Panigrahy, Rina , year = 2025, month = oct, url =. A. The

    2025

  11. [19]

    , year = 1964, month = mar, journal =

    Huber, Peter J. , year = 1964, month = mar, journal =. Robust. doi:10.1214/aoms/1177703732 , url =

    1964

  12. [20]

    General-

    Kirsch, Louis and Harrison, James and. General-. doi:10.48550/arXiv.2212.04458 , url =. arXiv , keywords =:2212.04458 , primaryclass =

    doi arXiv

  13. [21]

    Transformers as

    Li, Yingcong and Ildiz, Muhammed Emrullah and Papailiopoulos, Dimitris and Oymak, Samet , year = 2023, pages =. Transformers as. Proceedings of the 40th

    2023

  14. [22]

    Locating and Editing Factual Associations in

    Meng, Kevin and Bau, David and Andonian, Alex and Belinkov, Yonatan , year = 2022, month = nov, series =. Locating and Editing Factual Associations in. Proceedings of the 36th

    2022

  15. [23]

    State of

    Mizrahi, Moran and Kaplan, Guy and Malkin, Dan and Dror, Rotem and Shahaf, Dafna and Stanovsky, Gabriel , year = 2024, journal =. State of. doi:10.1162/tacl_a_00681 , url =

    2024 doi

  16. [24]

    In-Context Learning and Induction Heads , author =

  17. [25]

    Steering

    Rimsky, Nina and Gabrieli, Nick and Schulz, Julian and Tong, Meg and Hubinger, Evan and Turner, Alexander , editor =. Steering. Proceedings of the 62nd. doi:10.18653/v1/2024.acl-long.828 , url =

    2024 doi

  18. [26]

    Quantifying

    Sclar, Melanie and Choi, Yejin and Tsvetkov, Yulia and Suhr, Alane , year = 2023, month = oct, url =. Quantifying. The

    2023

  19. [27]

    Outrageously

    Shazeer, Noam and Mirhoseini, Azalia and Maziarz, Krzysztof and Davis, Andy and Le, Quoc and Hinton, Geoffrey and Dean, Jeff , year = 2017, month = feb, url =. Outrageously. International

    2017

  20. [28]

    and Moskovitz, Ted and Hill, Felix and Chan, Stephanie C

    Singh, Aaditya K. and Moskovitz, Ted and Hill, Felix and Chan, Stephanie C. Y. and Saxe, Andrew M. , year = 2024, month = jul, series =. What Needs to Go Right for an Induction Head? A Mechanistic Study of in-Context Learning Circuits and Their Formation , shorttitle =. Procee...

    2024

  21. [29]

    Regression

    Tibshirani, Robert , year = 1996, journal =. Regression. 2346178 , eprinttype =

    1996

  22. [30]

    and Mini, Ulisse and MacDiarmid, Monte , year = 2024, month = oct, number =

    Turner, Alexander Matt and Thiergart, Lisa and Leech, Gavin and Udell, David and Vazquez, Juan J. and Mini, Ulisse and MacDiarmid, Monte , year = 2024, month = oct, number =. Steering. doi:10.48550/arXiv.2308.10248 , url =. arXiv , keywords =:2308.10248 , primaryclass =

    2024 doi arXiv

  23. [31]

    Understanding

    Venhoff, Constantin and Arcuschin, Iv. Understanding. Workshop on

  24. [32]

    Investigating Gender Bias in Language Models Using Causal Mediation Analysis , booktitle =

    Vig, Jesse and Gehrmann, Sebastian and Belinkov, Yonatan and Qian, Sharon and Nevo, Daniel and Singer, Yaron and Shieber, Stuart , year = 2020, month = dec, series =. Investigating Gender Bias in Language Models Using Causal Mediation Analysis , booktitle =

    2020

  25. [33]

    Transformers Learn In-Context by Gradient Descent , booktitle =

    Von Oswald, Johannes and Niklasson, Eyvind and Randazzo, Ettore and Sacramento, Jo. Transformers Learn In-Context by Gradient Descent , booktitle =

  26. [34]

    Uncovering Mesa-Optimization Algorithms in

    von Oswald, Johannes and Schlegel, Maximilian and Meulemans, Alexander and Kobayashi, Seijin and Niklasson, Eyvind and Zucchet, Nicolas and Scherrer, Nino and Miller, Nolan and Sandler, Mark and y Arcas, Blaise Ag. Uncovering Mesa-Optimization Algorithms in. doi:10.48550/arXiv...

    doi arXiv

  27. [35]

    Xie, Sang Michael and Raghunathan, Aditi and Liang, Percy and Ma, Tengyu , year = 2021, month = oct, url =. An. International

    2021

  28. [36]

    Zhang, Fred and Nanda, Neel , year = 2023, month = oct, url =. Towards. The

    2023

  29. [37]

    Zhou, Zhanke and Tao, Rong and Zhu, Jianing and Luo, Yiwen and Wang, Zengmao and Han, Bo , year = 2024, month = nov, url =. Can. The

    2024

  30. [38]

    and Wang, Zifan and Mallen, Alex and Basart, Steven and Koyejo, Sanmi and Song, Dawn and Fredrikson, Matt and Kolter, J

    Zou, Andy and Phan, Long and Chen, Sarah and Campbell, James and Guo, Phillip and Ren, Richard and Pan, Alexander and Yin, Xuwang and Mazeika, Mantas and Dombrowski, Ann-Kathrin and Goel, Shashwat and Li, Nathaniel and Byun, Michael J. and Wang, Zifan and Mallen, Alex and Basa...

    2025 doi arXiv

  31. [39]

    Not All Directions Matter: Towards Structured and Task-Aware Low-Rank Model Adaptation

    Xiao, Xi and Ma, Chenrui and Zhang, Yunbei and Liu, Chen and Wang, Zhuxuanzi and Li, Yanshu and Zhao, Lin and Hu, Guosheng and Wang, Tianyang and Xu, Hao. Not All Directions Matter: Towards Structured and Task-Aware Low-Rank Model Adaptation. Proceedings of the 64th Annual Mee...

    2026 doi

  32. [40]

    arXiv preprint arXiv:2604.07650 , year=

    How independent are large language models? A statistical framework for auditing behavioral entanglement and reweighting verifier ensembles , author=. arXiv preprint arXiv:2604.07650 , year=

    arXiv

  33. [41]

    2026 , eprint=

    SciTaRC: Benchmarking QA on Scientific Tabular Data that Requires Language Reasoning and Complex Computation , author=. 2026 , eprint=

    2026

  34. [42]

    Yanshu Li and Yi Cao and Hongyang He and Qisen Cheng and Xiang Fu and Xi Xiao and Tianyang Wang and Ruixiang Tang , booktitle=. M. 2025 , url=

    2025

  35. [43]

    Train Once, Reuse Everywhere: Generalizable Implicit

    Jiaqian Li and Yanshu Li and Ligong Han and Ruixiang Tang and Wenya Wang , booktitle=. Train Once, Reuse Everywhere: Generalizable Implicit. 2026 , url=

    2026

  36. [44]

    Causal Evidence Extraction and Triangulation in Crisis Reports using Large Language Models:

    Yuanjun Zhang and Mourad Oussalah , editor =. Causal Evidence Extraction and Triangulation in Crisis Reports using Large Language Models:. Findings of the Association for Computational Linguistics,. 2026 , url =

    2026

  37. [45]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    ReTrack: Evidence-Driven Dual-Stream Directional Anchor Calibration Network for Composed Video Retrieval , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  38. [46]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    HABIT: Chrono-Synergia Robust Progressive Learning Framework for Composed Image Retrieval , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  39. [47]

    Proceedings of the ACM International Conference on Multimedia , pages =

    OFFSET: Segmentation-based Focus Shift Revision for Composed Image Retrieval , author =. Proceedings of the ACM International Conference on Multimedia , pages =

  40. [48]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    INTENT: Invariance and Discrimination-aware Noise Mitigation for Robust Composed Image Retrieval , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  41. [49]

    ACM Transactions on Multimedia Computing, Communications and Applications , year=

    REFINE: Composed Video Retrieval via Shared and Differential Semantics Enhancement , author=. ACM Transactions on Multimedia Computing, Communications and Applications , year=

  42. [50]

    arXiv preprint arXiv:2603.26341 , year=

    Hint: Composed image retrieval with dual-path compositional contextualized network , author=. arXiv preprint arXiv:2603.26341 , year=

  43. [51]

    arXiv preprint arXiv:2603.29291 , year=

    MELT: Improve Composed Image Retrieval via the Modification Frequentation-Rarity Balance Network , author=. arXiv preprint arXiv:2603.29291 , year=

  44. [52]

    arXiv preprint arXiv:2604.01617 , year=

    STABLE: Efficient Hybrid Nearest Neighbor Search via Magnitude-Uniformity and Cardinality-Robustness , author=. arXiv preprint arXiv:2604.01617 , year=

Pith tools

Reviewed July 31, 2026 · model on record in the stance chip above.