Pith. sign in

REVIEW 5 major objections 6 minor 24 references

A verified 16–24% shift in attention toward function-token positions does not improve contextual retrieval in GPT-2, LLaMA-1B/3B, or OPT-1.3B, leading the authors to conclude that mean cross-positional attention degradation is largely descr

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 07:54 UTC pith:IYYWVLR7

load-bearing objection Strong descriptive negative result about attention-mass redistribution, but the headline claim that attention is descriptive rather than prescriptive is over-reached because the intervention is only verified at short distances. the 5 major comments →

arxiv 2607.20524 v1 pith:IYYWVLR7 submitted 2026-07-09 cs.AI

Attention Degradation, Function Token Anchoring, and the Limits of Attention-Based Intervention in Large Language Models

Sagar Dangal , Manoj Shakya This is my paper
classification cs.AI
keywords attention degradationfunction tokensrelay chainattention as explanationcausal interventionKV-cache evictionLLM interpretabilitypositional encoding
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks whether the widely observed decay of attention with token distance actually limits what a language model can retrieve from context, or is merely a byproduct of computations happening elsewhere. Across six coordinated experiments on GPT-2, LLaMA-3.2-1B/3B, OPT-1.3B, and distilgpt2, it documents a universal exponential-then-plateau degradation over 5–100 tokens, with rate and layer-wise entropy profiles that differ by architecture. Function-token anchoring—the idea that articles, prepositions, and punctuation act as structural relay points—holds in some architectures (notably OPT-1.3B) but reverses in RoPE-based LLaMA models. The central causal test, Relay-Aware Attention, verifiably increases attention mass at function-token positions by 16–24% yet produces null, negative, or net-zero behavioural effects; and degradation rate does not predict multi-fact retrieval accuracy. The paper’s conclusion matters because attention-score-based optimisations such as KV-cache eviction assume attention mass tracks causal importance, an assumption these results challenge.

Core claim

The central claim is that mean cross-positional attention degradation is largely descriptive rather than prescriptive: function tokens contribute to contextual processing through what their hidden states compute, not through the attention they receive. This is supported by Experiment 5, in which a bias added to attention logits towards function-token positions (Relay-Aware Attention) verifiably raises attention mass at those positions by 16–24% at short distance, yet yields null effects on GPT-2 and LLaMA-1B, preliminary harm on LLaMA-3B, and a distance-dependent mixed effect on OPT-1.3B that nets to approximately zero. Experiment 6 adds that degradation rate does not predict retrieval accur

What carries the argument

Relay-Aware Attention (RAA), a modification of scaled dot-product attention that adds a positive bias δ to the pre-softmax logits of function-token positions uniformly across layers and heads, is the central causal instrument: it verifiably redistributes attention mass (16–24% at distance 5) so that behavioural nulls cannot be blamed on a failed intervention. The companion machinery is the relay-chain hypothesis—function tokens as overlapping anchors of bounded contextual influence—and the multi-fact interference probe, which measures retrieval accuracy as the percentage of trials where a target fact beats distractor facts.

Load-bearing premise

The causal conclusion depends on RAA having actually redistributed attention at the long distances (40–90 tokens) where the behavioural nulls were measured, but the paper verifies redistribution only at distance 5; at long distances, softmax saturation or attention-sink absorption could have absorbed the bias, leaving the nulls ambiguous.

What would settle it

Measure attention mass at function-token positions at distances 40–90 under RAA with δ=2.0 in GPT-2 and LLaMA-1B; if the mass does not increase at those distances, the null behavioural results show intervention failure rather than causal irrelevance of attention.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • KV-cache eviction policies that select tokens by accumulated attention scores may discard tokens whose hidden states remain causally important, since attention mass and behavioural contribution dissociate.
  • Models with absolute positional encoding and greater depth should show stronger function-token dependence; RoPE-based models may be actively harmed by artificial emphasis on function tokens.
  • Strategic comma insertion at clause boundaries can reduce prediction degradation in the 40–80 token range for the tested absolute-encoding base models, but uniform punctuation injection is not a substitute.
  • Mean attention degradation rate is not a reliable predictor of multi-fact retrieval; model capacity, not degradation rate, is the dividing line.
  • Interpretability claims that use aggregate attention weights as evidence of contextual processing should be treated as descriptive correlates, not causal mechanisms, in autoregressive language models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the descriptive conclusion generalises, the promising intervention direction is representation-level—changing what function tokens compute (e.g., through activation patching or fine-tuning)—rather than attention-logit biasing.
  • The key untested condition is whether RAA actually moves attention at d=40–90; a reader wanting to resolve the ambiguity can measure attention mass at those distances under RAA, since softmax saturation or attention-sink absorption could otherwise explain the nulls.
  • The comma-insertion-versus-RAA contrast suggests a general principle: adding a new token creates a fresh computational node whose hidden state is shaped by the full forward pass, whereas reweighting attention to existing tokens does not alter what those positions have already computed.
  • The architecture-dependence implies a cross-linguistic prediction: languages that mark grammatical relations morphologically, rather than with free-standing function words, may lack the same relay-chain structure and should show different degradation and intervention profiles.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper studies short-range (5–100 token) attention degradation across GPT-2, LLaMA-3.2-1B/3B, OPT-1.3B, and distilgpt2. It reports: (i) an exponential-then-plateau degradation curve whose rate is inversely correlated with depth; (ii) architecture-dependent function-token anchoring, with OPT-1.3B showing preposition specificity, GPT-2 showing uniform non-specific dependence, and LLaMA showing long-distance reversal; (iii) a causal benefit of clause-boundary comma insertion in the 40–80 token range; (iv) a relay-chain coverage analysis; (v) Relay-Aware Attention (RAA), a logit bias toward function-token positions, which verifiably increases attention mass by 16–24% yet produces null, negative, or net-zero behavioral effects; and (vi) multi-fact retrieval probes showing degradation rate does not predict retrieval accuracy. The central claim is that mean cross-positional attention degradation is largely descriptive rather than prescriptive: function tokens matter through their hidden-state computations, not through attention mass. The paper explicitly limits its scope to base models below 3.5B parameters and English Wikipedia-style prose.

Significance. If the central negative result holds, it is a valuable contribution to the attention-as-explanation debate and carries practical implications for attention-score-based KV-cache eviction and for interpretability methodology. The paper's strengths include a coordinated multi-model design, a direct intervention that is verified to move attention mass (at least at short distances), honest and detailed limitations, and an explicit falsifiable negative result. The claim that mean attention is descriptive rather than prescriptive is not circular: RAA is not fit to the outcome, and the multi-fact probe is an independent test. However, the causal inference rests on the assumption that the RAA intervention actually redistributed attention at the distances where behavioral nulls are measured; this is verified only at d=5. The paper's own §6 concedes that softmax saturation or attention-sink absorption could have nullified the bias at d=40–90, which makes the key null evidence underdetermined. The other major concerns are low seed counts for LLaMA-3B RAA and the acknowledged instability of Experiment 2 cloze estimates.

major comments (5)
  1. [§3.8, §4.5, §6] The central causal conclusion depends on RAA being an effective intervention at the distances where nulls are observed. Attention redistribution is verified only at d=5 (16–24% increase), while the behavioral nulls for GPT-2 and LLaMA-1B are reported across all distances, including d=40–90. The paper itself concedes in §6 that 'softmax saturation or attention-sink absorption could in principle have absorbed the bias, making the null ambiguous at those distances.' If the bias was absorbed, these nulls show only that the intervention failed, not that attention mass is causally irrelevant. Please verify RAA-induced attention change at long distances for each model (e.g., report attention mass at function-token positions as a function of distance with and without RAA), or restrict the causal claim to distances where redistribution is confirmed.
  2. [§3.5, Table 3, §4.2] Experiment 2's inter-run instability is load-bearing for the architecture-dependent specificity claims. The text states that 'mean function-token replacement ∆p estimates vary by up to 2× across runs, suggesting results in the 5–100 sample range are sensitive to sample composition.' Yet Table 3 and §4.2 report specific architecture-dependent patterns (e.g., OPT preposition specificity at 16/21 distances) without quantifying run-to-run variability or providing confidence intervals. Please report per-seed/per-run variability, bootstrap intervals, or an alternative stability analysis so the reader can assess whether the claimed specificity patterns are robust to sample composition.
  3. [§3.8, Table 8, §4.5, §5.1] The LLaMA-3B RAA result, which is used to support the claim that function-token emphasis is 'actively harmful' for RoPE models, rests on only 2 seeds. The paper labels this as preliminary, which is commendable, but the conclusion in §5.1 goes beyond the evidence. A 2-seed result is not sufficient to support a cross-architecture claim; either provide additional seeds or substantially weaken the statement to a tentative observation that requires replication. This is especially important because the LLaMA-3B result is the only direct evidence of harm, as opposed to null or mixed effects.
  4. [§3.7, Table 7, §4.4] Experiment 4's relay-chain coverage uses effective radii derived from Experiment 2, but the paper notes that article and punctuation radii come from conditions 'showing weak or absent specificity in most models.' This makes the coverage figures for these categories largely a reflection of corpus density rather than model-specific anchoring. While this is acknowledged as a limitation, the coverage percentages are still presented as 'relay chain coverage' in the results and contribute to RQ4. Please reframe these numbers as corpus-density statistics, not as evidence for a functional relay chain, and distinguish clearly between categories with empirically grounded radii (prepositions) and those without.
  5. [§3.9, §4.6, Table 9] The distilgpt2 vs. GPT-2 comparison in Experiment 6 is uninformative because both models score below the ≈25% chance level at every distance, as the paper acknowledges. This floor effect means the paired t-test (p=0.41) cannot support the claim that degradation rate does not predict retrieval in this comparison. The stronger evidence comes from the LLaMA-1B vs. OPT-1.3B divergence, but this is a single cross-model comparison at one capacity tier. Please either redesign the probe to avoid floor effects for small models (e.g., fewer distractors, simpler facts) or explicitly limit the claim to the models and task difficulty where the metric is not at floor.
minor comments (6)
  1. [Abstract and §1] The phrase 'largely descriptive rather than prescriptive' is imprecise. Specify the scope: the 5–100 token range, base models below 3.5B parameters, English WikiText-style prose, and the specific intervention tested.
  2. [Equation (1), §3.4] Degradation is defined relative to Attn(5). Since absolute attention at d=5 is already low and varies across models, the percentage scale can exaggerate or obscure differences. Consider also reporting absolute attention values or a complementary metric.
  3. [Table 3] The abbreviation 'FN∆p' is used without definition in the table caption or preceding text. Define it explicitly in the caption.
  4. [§4.2] The sentence 'Weak positive correlations between distance and probability change (articles: r=0.51, p=0.02; punctuation: r=0.80, p<0.001)' should state the direction more clearly: positive correlation means less damage at longer distances, as is stated later. Consider rephrasing to avoid ambiguity.
  5. [Table 5, §4.3] The caption says '∆p = mean probability change for the function-token condition (negative = less damage than baseline).' However, for LLaMA-1B at d=70–80 the values are positive and described as 'marginal harm.' Please spell out the sign convention explicitly so the reader understands positive indicates harm.
  6. [§3.3] The punctuation category is listed as '( , . ; : ! ? )'. It would be clearer to include the space and comma explicitly, or provide a tokenizer-based description, since tokenization of punctuation can vary by model.

Circularity Check

0 steps flagged

No significant circularity: central claim rests on independent interventions and is not reduced to its inputs.

full rationale

The derivation chain is self-contained and non-circular. The central descriptive-vs-prescriptive conclusion rests on Experiments 5 and 6: RAA is an intervention that biases attention logits and is checked by a separate attention measurement (§3.8), while the multi-fact retrieval probe is an independent behavioral task compared with the measured degradation rates (§3.9, §4.6). The null behavioral effects are not fitted to the attention data, and no parameter is estimated from the outcome and then renamed a prediction. The only in-paper re-use of experimental output is Experiment 4's coverage calculation, where effective radii are taken from Experiment 2; however, the paper explicitly discloses that article and punctuation radii derive from conditions with weak or absent specificity and that the coverage figures primarily reflect corpus density rather than model-specific anchoring (§3.7, §6). The coverage result is therefore not used to validate or predict the same specificity effects. The stated limitations—RAA redistribution verified only at d=5, possible softmax saturation or attention-sink absorption at d=40–90, floor effects in the GPT-2 retrieval comparisons—are internal-validity and statistical-power concerns, not definitional circularity; they weaken the strength of the causal null but do not make the conclusion equivalent to its inputs. No self-citation is load-bearing, and the function-token categories are adopted from an external source and then tested rather than assumed.

Axiom & Free-Parameter Ledger

2 free parameters · 5 axioms · 0 invented entities

The central claim draws on standard attention mechanics plus domain assumptions about function-token categories, WikiText representativeness, and query-level attention as the unit of analysis. Two fitted quantities enter: the hand-selected RAA bias delta and effective radii derived from Experiment 2. No new physical entities or conserved quantities are introduced; the relay chain is an interpretive mechanism, not a new degree of freedom.

free parameters (2)
  • RAA bias delta = {0.0, 0.1, 0.5, 1.0, 2.0}
    Hand-selected uniform logit bias added to function-token positions across all layers and heads (Section 3.8). Results are reported at the strongest-effect delta; no principled selection or tuning curve is provided.
  • Effective radii for relay-chain coverage = articles 60 tokens; prepositions 25; punctuation 35
    Derived empirically from Experiment 2 results and then used to compute Experiment 4 coverage (Section 3.7). Coverage figures therefore partly encode the same data they claim to describe.
axioms (5)
  • domain assumption Function-token categories (articles, prepositions, punctuation) as defined by Zhang et al. [22] are meaningful units for analysis.
    Used throughout Experiments 2-5; categories are imposed by the authors and may not map onto causal structure in all tokenizers.
  • domain assumption Query-level attention weights, averaged across layers and heads, are a valid unit of analysis.
    Section 2.1/3.1; with GQA, query-level averaging may obscure head-level heterogeneity. The authors acknowledge this in Limitations.
  • domain assumption WikiText English encyclopedic prose is representative enough for degradation and coverage conclusions.
    Sections 3.2 and 5.7; the authors themselves note lower function-token density in conversational, telegraphic, or code-heavy text.
  • standard math Softmax mass conservation makes some distance-related attention decay arithmetically inevitable.
    Section 6 'Softmax normalisation artefact'; the paper uses this to support interpreting mean attention degradation as descriptive, which is consistent with standard attention mechanics.
  • ad hoc to paper RoPE encodes relative position such that function tokens are redundant as positional cues.
    Interpretive explanation in Sections 4.2 and 5.1 for LLaMA reversal effects; not directly measured and load-bearing for the architecture-dependence narrative.

pith-pipeline@v1.3.0-alltime-deepseek · 14448 in / 11985 out tokens · 112224 ms · 2026-08-02T07:54:00.083912+00:00 · methodology

0 comments
read the original abstract

Mean cross-positional attention degradation is widely reported in transformer interpretability, yet whether it causally limits contextual retrieval remains untested. We present six coordinated experiments across GPT-2, LLaMA-3.2-1B/3B, OPT-1.3B, and distilgpt2. We first characterise short-term (5-100 token) attention degradation, finding a universal exponential-then-plateau pattern whose rate is inversely correlated with depth, with distinct layer-wise entropy signatures per architecture. Function token anchoring proves architecture-dependent: OPT-1.3B (absolute positional encoding) shows distance-dependent preposition specificity, GPT-2 shows uniform non-specific dependence, and LLaMA (RoPE) shows reversal at long distances. Strategic comma insertion at clause boundaries causally reduces prediction degradation in the 40-80 token range, with the benefit tied to syntactic boundary alignment rather than token density. We then test the mechanism causally: Relay-Aware Attention (RAA), which biases attention logits toward function token positions, verifiably increases attention mass by 16-24% yet yields null effects on GPT-2 and LLaMA-1B, preliminary harm on LLaMA-3B, and a mixed effect on OPT-1.3B that nets to approximately zero. Multi-fact retrieval probes further show that degradation rate does not predict retrieval accuracy across models. We conclude that mean attention degradation is largely descriptive rather than prescriptive: function tokens contribute through what their hidden states compute, not through the attention they receive -- with implications for interpretability methodology and attention-score-based inference optimisations such as KV-cache eviction.

Figures

Figures reproduced from arXiv: 2607.20524 by Manoj Shakya, Sagar Dangal.

Figure 1
Figure 1. Figure 1: Attention degradation by token distance across architectures (data from Appendix A). All models [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: RAA effect on OPT-1.3B at δ = 2.0 (positive = improvement, negative = harm). Improvements at short distances are offset by harm at longer distances; the net effect is approximately zero. Dist. distilgpt2 GPT-2 LLaMA-1B OPT-1.3B d = 10 17.5% 14.5% 35.5% 36.0% d = 30 16.0% 14.5% 40.0% 46.0% d = 50 13.0% 15.5% 44.0% 67.0% d = 70 16.0% 17.0% 50.0% 76.0% d = 80 13.0% 10.0% 58.5% 84.0% [PITH_FULL_IMAGE:figures/… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 8 linked inside Pith

  1. [1]

    Is attention explanation? An introduction to the debate.arXiv preprint arXiv:2206.09775, 2022

    Adrien Bibal, Rémi Cardon, David Alfter, Rodrigo Wilkens, Xiaoou Wang, Thomas François, and Pierre Watrin. Is attention explanation? An introduction to the debate.arXiv preprint arXiv:2206.09775, 2022

    Pith/arXiv arXiv 2022

  2. [2]

    What does BERT look at? An analysis of BERT’s attention

    Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D Manning. What does BERT look at? An analysis of BERT’s attention. InProceedings of the 2019 ACL Workshop BlackboxNLP, pages 276–286, 2019

    2019

  3. [3]

    Do thinking tokens help or trap? Towards more efficient large reasoning models.arXiv preprint arXiv:2506.23840, 2025

    Bowen Ding, Yuhan Chen, Futing Wang, Lingfeng Ming, and Tao Lin. Do thinking tokens help or trap? Towards more efficient large reasoning models.arXiv preprint arXiv:2506.23840, 2025

    Pith/arXiv arXiv 2025

  4. [4]

    The Llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, et al. The Llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Pith/arXiv arXiv 2024

  5. [5]

    How do LLMs use their depth?arXiv preprint arXiv:2510.18871, 2025

    Arjun Gupta, Jimmy Yeung, Gopala Anumanchipalli, and Anna Ivanova. How do LLMs use their depth?arXiv preprint arXiv:2510.18871, 2025

    arXiv 2025

  6. [6]

    Context rot: How increasing input tokens impacts LLM performance

    Kevin Hong, Anton Troynikov, and James Huber. Context rot: How increasing input tokens impacts LLM performance. Technical report, Chroma Research, 2025

    2025

  7. [7]

    Attention is not Explanation

    Sarthak Jain and Byron C Wallace. Attention is not Explanation. InProceedings of NAACL-HLT 2019, pages 3543–3556, 2019

    2019

  8. [8]

    SnapKV: LLM knows what you are looking for before generation

    Yuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, and Deming Chen. SnapKV: LLM knows what you are looking for before generation. In Advances in Neural Information Processing Systems, volume 37, 2024

    2024

  9. [9]

    Lost in the middle: How language models use long contexts.Transactions of the Association for Computational Linguistics, 12:157–173, 2024

    Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts.Transactions of the Association for Computational Linguistics, 12:157–173, 2024

    2024

  10. [10]

    Locating and editing factual associa- tions in GPT

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associa- tions in GPT. InAdvances in Neural Information Processing Systems, volume 35, pages 17359–17372, 2022. 18

    2022

  11. [11]

    Pointer sentinel mixture models

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. InProceedings of the 5th ICLR, 2017

    2017

  12. [12]

    Train short, test long: Attention with linear biases enables input length extrapolation

    Ofir Press, Noah A Smith, and Mike Lewis. Train short, test long: Attention with linear biases enables input length extrapolation. InProceedings of the 10th ICLR, 2022

    2022

  13. [13]

    Demystifying reasoning dynamics with mutual information: Thinking tokens are information peaks in LLM reasoning.arXiv preprint arXiv:2506.02867, 2025

    Chen Qian, Dong Liu, Hao Wen, Zhen Bai, Yang Liu, and Jing Shao. Demystifying reasoning dynamics with mutual information: Thinking tokens are information peaks in LLM reasoning.arXiv preprint arXiv:2506.02867, 2025

    Pith/arXiv arXiv 2025

  14. [14]

    Language models are unsupervised multitask learners.OpenAI Blog, 1(8), 2019

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners.OpenAI Blog, 1(8), 2019

    2019

  15. [15]

    RoFormer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063, 2024

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. RoFormer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063, 2024

    2024

  16. [16]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. InAdvances in Neural Information Processing Systems, volume 30, pages 5998–6008, 2017

    2017

  17. [17]

    Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned

    Elena V oita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned. InProceedings of the 57th Annual Meeting of the ACL, pages 5797–5808, 2019

    2019

  18. [18]

    Attention is not not Explanation

    Sarah Wiegreffe and Yuval Pinter. Attention is not not Explanation. InProceedings of EMNLP-IJCNLP 2019, pages 11–20, 2019

    2019

  19. [19]

    Efficient streaming language models with attention sinks.arXiv preprint arXiv:2309.17453, 2023

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks.arXiv preprint arXiv:2309.17453, 2023

    Pith/arXiv arXiv 2023

  20. [20]

    IKOD: Mitigating visual attention degradation in large vision-language models.arXiv preprint arXiv:2508.03469, 2025

    Jian Yang, Chen Cui, Yi Zhou, Yang Chen, Peng Xia, Yue Wei, Tao Yu, Yue Huang, and Lin Wang. IKOD: Mitigating visual attention degradation in large vision-language models.arXiv preprint arXiv:2508.03469, 2025

    Pith/arXiv arXiv 2025

  21. [21]

    Reasoning with reinforced functional token tuning.arXiv preprint arXiv:2502.13389, 2025

    Kang Zhang, Qi Yao, Bo Lai, Jian Huang, Wei Fang, Dacheng Tao, Mingli Song, and Si Liu. Reasoning with reinforced functional token tuning.arXiv preprint arXiv:2502.13389, 2025

    Pith/arXiv arXiv 2025

  22. [22]

    Memory retrieval and consolidation in large language models through function tokens.arXiv preprint arXiv:2510.08203, 2025

    Shaohua Zhang, Yuan Lin, and Hang Li. Memory retrieval and consolidation in large language models through function tokens.arXiv preprint arXiv:2510.08203, 2025

    arXiv 2025

  23. [23]

    OPT: Open pre-trained transformer language models.arXiv preprint arXiv:2205.01068, 2022

    Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. OPT: Open pre-trained transformer language models.arXiv preprint arXiv:2205.01068, 2022

    Pith/arXiv arXiv 2022

  24. [24]

    H2O: Heavy-hitter oracle for efficient generative inference of large language models

    Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, Zhangyang Wang, and Beidi Chen. H2O: Heavy-hitter oracle for efficient generative inference of large language models. InAdvances in Neural Information Processing Systems, volume 36, 2023. 19

    2023