Pith. sign in

REVIEW 3 major objections 5 minor 48 references

EARN: Efficient Inference Acceleration for LLM-based Generative Recommendation by Register Tokens

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read EARN claims that adding learnable register tokens at both ends of the user prompt lets layers beyond the first quarter ignore the prompt entirely, delivering up to 3.79x speedup and 80.8% KV-cache reduction with accuracy that matches or…

desk verdict The speedup and cache-reduction engineering is credible and worth a referee's time, but the paper's motivating 'sparsity inversion' is computed backwards, so the story needs substantial rework. read the letter →

arxiv 2507.00715 v1 pith:V6HKLO4D submitted 2025-07-01 cs.IR

Chaoqun Yang , Xinyu Lin , Wenjie Wang , Yongqi Li , Teng Sun , Xianjing Han , Tat-Seng Chua This is my paper
classification cs.IR
keywords LLM-basedrecommendationinferenceaccelerationKVcachereductionregistertokensattentionsinkssparsitygenerative
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that LLM-based generative recommendation can be accelerated by exploiting two attention patterns specific to this task: early layers distribute attention densely while later layers are sparse, and attention sinks concentrate at both the head and the tail of the input sequence. On that basis, EARN inserts learnable register tokens at the two boundaries of the prompt, trains the first k layers to compress the user's interaction history into those tokens, and then prunes the prompt so that the remaining layers attend only to the registers and the generated output. If the claim is right, this yields up to a 3.79x wall-clock speedup and roughly 80% KV-cache memory reduction while preserving or slightly improving recommendation accuracy relative to full fine-tuning. A sympathetic reader would care because this directly addresses the deployment obstacle of high latency and memory cost that has kept LLM-based recommenders out of industrial settings.

What carries the argument

The load-bearing device is the pair of learnable register tokens: a prefix register inserted before the user prompt and a suffix register inserted after it. In training, the first k layers (k = 4 of 32 for Llama, k = 7 of 28 for Qwen) run full attention over prompt, registers, and target tokens, while layers l > k attend only to registers and newly generated tokens; the next-token-prediction loss therefore forces the registers to absorb what the later layers need. At inference, the prompt tokens are discarded at layer k, leaving only the prefix register, suffix register, and generated tokens in the attention and in the KV cache for all subsequent layers. This converts the bulk of the model's depth into register-focused computation, reducing attention complexity to roughly $\frac{k}{N}$ of the original FLOPs and cutting KV-cache memory by $\frac{(N-k)(L-r)}{NL}$.

What would settle it

Probe the suffix register's hidden state after layer k with a linear classifier trained to predict the held-out target item from that single token; if the probe performs at chance while a probe over the full-prompt hidden state is accurate, the register has not summarized the history and EARN's reported accuracy cannot come from the mechanism the paper describes.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the attention distribution in LLM-based generative recommendation is structured so that input prompt tokens become disposable after a few early layers. The authors identify a layer-wise attention sparsity inversion: unlike typical NLP tasks, early layers show dense attention (sparsity about 0.025 in Llama) and later layers show sparse attention (sparsity about 0.048), and a dual attention sinks phenomenon in which a large share of attention mass lands on head and tail tokens. Taking the tail sink as evidence that the final tokens can summarize the preceding interaction history, EARN places a learnable suffix register at the end of the prompt, a prefix register at the start as a task indicator, and trains the model so that for layers beyond k the only attended tokens are the two registers and the generated item IDs. The claim is that with the right training routine the registers carry the information the later layers need, so the prompt can be physically removed after layer k, shrinking the KV cache from roughly full prompt length to a few tokens in most layers and cutting FLOPs. The empirical case is that EARN beats or matches fine-tuning on Recall@10/20 and NDCG@10/20 across three datasets and two architectures, while achieving 2.71-3.79x speedup and 66.7-80.8% cache reduction.

Load-bearing premise

The whole method rests on the assumption that a few learnable tokens at the sequence boundary, after only the first quarter of the layers, capture all the user-history information the later layers need, so the prompt can be dropped entirely without hurting accuracy.

Editorial extensions

If this is right

  • With one prefix and one suffix register and k at one-fourth of the layers, EARN delivers 2.71–3.79x wall-clock speedup and 66.7%–80.8% KV-cache reduction across Beauty, Games, and MovieLens for both Llama and Qwen.
  • EARN's recommendation quality is reported as better than full fine-tuning in most comparisons (e.g., Beauty/Llama R@20 rises from 0.0225 to 0.0265), which implies that discarding the middle prompt in later layers does more than save compute, it also removes information the model does not need.
  • The acceleration improves with sequence length (up to 7x at 20K tokens on Llama) and with batch size, and EARN avoids the out-of-memory failures that fine-tuning hits at large batches.
  • The ablations show the suffix register is the critical component: removing it drops R@10 from about 0.017 to 0.004 on Llama k=7, while removing the prefix register costs much less, which pinpoints where the summarization burden lies.
  • Register training is indispensable: applying EARN's inference pruning to an ordinary fine-tuned model, without the restricted-attention training, reduces R@10 by about 72% on Llama k=7, so the method's gains are not a free lunch.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test this, but the dual-sink finding suggests the suffix register acts as a sufficient statistic of the user history; a direct probing experiment, training a linear classifier on the register's hidden state after layer k to predict the next item, would settle whether summarization is the actual mechanism.
  • One extension the authors leave implicit is adapting register count to sequence length: their grouped experiments show the gap between one and two suffix tokens narrows as prompts grow, hinting that very long histories may need more than one suffix register.
  • If the denoising effect is real (accuracy above fine-tuning on several datasets), then EARN could combine naturally with noisy or adversarial interaction filtering, and the register could serve as a compact, privacy-friendly user representation for downstream ranking.
  • The layer-sparsity inversion was measured on only three datasets and two LLMs; repeating the same measurements on other domains, such as code generation or multi-turn dialogue, would show whether the early-layer-only compression recipe transfers beyond recommendation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes EARN, an inference-acceleration method for LLM-based generative recommendation. EARN inserts learnable prefix and suffix register tokens at the boundaries of the input prompt, trains the first k layers with full attention so that the registers can summarize the user history, and at inference drops all prompt tokens after layer k so that the remaining layers attend only to registers and generated tokens. The authors motivate the design with two attention analyses—an alleged layer-wise sparsity inversion and a dual attention sinks pattern—and provide efficiency derivations for FLOPs, KV-cache size, and speedup. They evaluate EARN on Beauty, Games, and MovieLens-1M with LC-Rec and TIGER on Llama-7B and Qwen2.5-7B, reporting up to 3.79x speedup and 80.8% KV-cache reduction with accuracy at or above that of full finetuning.

Significance. If the empirical results hold, EARN is a practically useful contribution: it targets the prefilling-dominated latency regime of LLMRec, where cache-compression methods give limited end-to-end benefit, and it demonstrates a concrete mechanism—boundary register tokens trained in early layers—for pruning later-layer computations. The efficiency analysis in Section 3.3 is transparent and arithmetically sound, and the evaluation is unusually broad for the area: two LLMRec methods, two LLM architectures, three real-world datasets, plus additional HSTU and MMLU experiments. The code is publicly released. The main weaknesses are that the motivating sparsity analysis is internally inconsistent as written and that the 'better accuracy' claim rests on single-run differences of about 0.002–0.003 in R@10 without variance or significance information; these issues are local and fixable rather than fundamental.

major comments (3)
  1. [Appendix A.1, Eq. (14)] Equation (14) defines Sparsity = (1/n) Σ I(p_i > ε), which is the fraction of positions whose attention weight exceeds ε—that is a density measure, not a sparsity measure. The values in Table 4 then contradict the prose: for LLMRec, Sp_early 0.025 → Sp_later 0.048 means later layers have a higher fraction of non-negligible attention weights, i.e., they are less sparse, while Section A.1 and Section 1 say later layers are 'highly sparse' and redundant. Similarly, the NLP values 0.064 → 0.026 show sparsity increasing later, not decreasing. Because the claimed 'layer-wise attention sparsity inversion' is the stated motivation for pruning after layer k (Section 3.2 and Figure 3), the quantitative analysis as written does not support the conclusion that later layers are redundant. The metric must be corrected (e.g., define sparsity as the fraction of near-zero weights) or all interpretations reversed, and the thresholds (ε=0.05, T_h=3, T_t=n−3) should be justified with sensitivity checks.
  2. [Section 4.4.3 and Appendix A.3] The hyper-parameter recommendation states that the register layer depth should be one-fourth of the total layers, which would be k=8 for Llama-7B (32 layers), but the main results in Table 1 are obtained with k=4 on Llama (Appendix A.3), and Section 4.4.1 identifies the optimal range as k=4–7. The headline 3.79x speedup therefore corresponds to a configuration different from the recommended one. The ablation tables (Tables 2 and 3) use yet another setting (k=7 and 15 on Llama), so those numbers are not directly comparable to Table 1. Please align the recommended, reported, and ablation configurations, or explain why k=4 is the intended main-result setting despite the one-fourth heuristic.
  3. [Section 4.2, Table 1] The abstract and Section 4.2 claim that EARN achieves 'better accuracy' than the general finetuning approach, but every accuracy comparison is a single run with no seed variance, confidence intervals, or significance tests, and the R@10 differences are small (e.g., Beauty/Llama: 0.0167 vs 0.0145; MovieLens/Llama: 0.0259 vs 0.0247). With effect sizes of this magnitude, the accuracy-superiority claim is not yet supported as stated. Please report multiple seeds with means and standard deviations or conduct a significance test, and specify whether the claim refers to all metrics or only to selected ones.
minor comments (5)
  1. [Appendix A.3] The heading 'Experimential Details' should read 'Experimental Details'.
  2. [Section 1] The bullet list reports 'Sparsity: 0.06→0.03' and 'Sparsity: 0.01→0.07' using the same terminology as Eq. (14); once Eq. (14) is corrected, these arrows and the accompanying conclusions must be updated consistently.
  3. [Figure 2] Figure 2 shows attention distributions for a single head ('Head 0') of one model; it would help to state how representative this head is relative to the averaged numbers in Table 4.
  4. [Section 4.4.3] The phrase 'setting the register layer and using a single register token at one-fourth' is grammatically unclear; it should say 'setting the register layer depth to one-fourth of the total layers'.
  5. [Tables 1, 5, and 6] The unit of σ is GB in Tables 1 and 5 but MB in Table 6; please unify the units or state them explicitly in each table.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity: EARN's speedup and accuracy claims are arithmetic/empirical comparisons, not reductions to fitted inputs; the Eq. 14 'sparsity inversion' is an internal correctness flaw, and the paper's self-citations are non-load-bearing background.

full rationale

EARN's central claims are not circular. The efficiency numbers (Sec. 3.3, Eqs. 8-9, Table 1) are arithmetic consequences of the stated inference procedure (full attention for k layers, then only register + generated tokens), and the accuracy numbers are direct benchmark comparisons against Finetune and compression baselines, so no fitted parameter is renamed as a prediction. The register tokens are trained with next-token prediction and evaluated on held-out test users, making 'better accuracy than Finetune' an empirical result rather than an identity. The motivating attention analysis is not a reduction-to-input but it does contain a serious internal flaw that should be weighed: Eq. 14 defines 'Sparsity' as (1/n) sum of I(p_i > epsilon), which is a density, so the reported LLMRec pattern (Sp_early=0.025 < Sp_later=0.048) means the fraction of above-threshold attention weights increases in later layers, i.e., attention becomes less sparse; the paper's conclusion that later layers are 'highly sparse' and redundant reads the definition backward. Because the end-to-end speedup/accuracy claims do not derive from Eq. 14, this is a correctness risk in the motivation, not a circular derivation. The paper's self-citations ([16,21,22,23,36]) appear only as background or related-work pointers; none is invoked to forbid alternatives or to justify EARN's mechanism, so they are non-load-bearing. Hyperparameters k and n are tuned on the same datasets (Figs. 6-7) but are transparently reported as tuned, and the reported test numbers are not fitted quantities. Score 2 reflects only the non-load-bearing self-citations and the mild post-hoc character of the attention-threshold choices; no step of the claimed derivation is equivalent to its own input.

Assumptions & free parameters 4 free parameters · 5 assumptions · 2 invented entities

The central claim depends on k and n (tuned per model), on the correctness of the empirical attention claims (which use hand-set thresholds), and on the unproven assumption that register tokens capture user history in the first k layers. Register tokens are new learned entities with no independent evidence. No formal verification is provided.

free parameters (4)
  • register layer depth k = k=4 for Llama (32 layers), k=7 for Qwen2.5 (28 layers); Section 4.4.3 recommends N/4
    Determines how many early layers attend to prompts and controls the speedup/accuracy trade-off. Chosen per model from sensitivity analysis (Figure 6), not derived.
  • number of register tokens n = 1 prefix and 1 suffix in all main experiments
    Selected from ablation and sensitivity analysis (Figure 7); a single token was found optimal. Affects how much history can be summarized.
  • attention sparsity threshold epsilon = 0.05
    Sets the threshold in Eq. 14 for the sparsity metric. Chosen post hoc "according to the distribution characteristics of attention scores" and drives the layer-wise sparsity inversion claim.
  • sink head/tail window sizes T_h, T_t = T_h=3, T_t=n-3
    Chooses how many head and tail positions are counted as sinks in Eq. 15. Post hoc and directly shapes the dual attention sinks claim.
assumptions (5)
  • standard math A decoder-only transformer's FLOP and KV-cache formulas in Eqs. 8, 9, and 16-18 are accurate.
    Used in the efficiency analysis; standard FLOP counting.
  • domain assumption In LLMRec, the prefilling stage dominates latency and short decoding makes cache compression alone insufficient.
    Motivates reducing prompt processing; invoked in Sections 1 and 2.2.
  • domain assumption Attention sinks exist in LLMs and the BOS token can be replaced by learnable tokens; tail-position tokens can summarize preceding content.
    Borrowed from prior work [7, 30, 40] and used to justify prefix and suffix registers in Sections 3.1.1 and 3.1.2.
  • ad hoc to paper A learnable token can accumulate sufficient user-history information through the first k layers, so later layers can attend only to registers without meaningful accuracy loss.
    Central design premise of EARN (Section 3.2). Validated only empirically on three datasets and not derived.
  • ad hoc to paper Removing prompt token hidden states after layer k leaves residual connections, FFN, and layer normalization computations valid.
    The pipeline drops prompt vectors after k layers (Section 3.2.2) without re-normalizing; assumes layer statistics are not materially disrupted.
invented entities (2)
  • Prefix register token
    purpose: Learnable virtual token at sequence start; replaces BOS, intended to absorb task instruction and act as an attention sink.
    Introduced in Section 3.1.1. Evidence is only the in-paper ablation (w/o PR hurts slightly); no external falsifiable prediction beyond the reported benchmarks.
  • Suffix register token
    purpose: Learnable virtual token at sequence end; intended to summarize user interaction history.
    Introduced in Section 3.1.2. Evidence is the in-paper ablation (w/o SR collapses accuracy); no independent test outside the reported datasets.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EARN: Efficient Inference Acceleration for LLM-based Generative Recommendation by Register Tokens." pith.science (2026). https://pith.science/paper/V6HKLO4D

@misc{pith2026250700715,
  author       = {Pith},
  title        = {Pith review of: EARN: Efficient Inference Acceleration for LLM-based Generative Recommendation by Register Tokens},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V6HKLO4D}},
  note         = {Machine review of arXiv:2507.00715}
}
read the original abstract

Large Language Model-based generative recommendation (LLMRec) has achieved notable success, but it suffers from high inference latency due to massive computational overhead and memory pressure of KV Cache. Existing KV Cache reduction methods face critical limitations: cache compression offers marginal acceleration given recommendation tasks' short decoding steps, while prompt compression risks discarding vital interaction history. Through systematic analysis of attention patterns in LLMRec, we uncover two pivotal insights: 1) layer-wise attention sparsity inversion where early layers retain dense informative patterns while later layers exhibit high redundancy, and 2) dual attention sinks phenomenon where attention scores concentrate on both head and tail tokens of input sequences. Motivated by these insights, we propose EARN, an efficient inference framework that leverages the early layers to compress information into register tokens placed at the input sequence boundaries, then focuses solely on these tokens in the subsequent layers. Extensive experiments on three datasets, two LLMRec methods and two LLM architectures demonstrate EARN's superiority, achieving up to 3.79x speedup and 80.8% KV Cache reduction with better accuracy than the general finetuning approach. Our work bridges the efficiency-effectiveness gap in LLMRec, offering practical deployment advantages for industrial scenarios.

Figures

Figures reproduced from arXiv: 2507.00715 by the authors.

Figure 2
Figure 2. The attention distributions of different layers on [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Overview of the proposed EARN. During training, [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Efficiency under different batch sizes [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Efficiency under different sequence lengths. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 8
Figure 8. Figure 8: Effect of register token number 𝒏 under different prompt length. EARN’s performance across different prompt lengths. As shown in [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 7
Figure 7. Figure 7: Effect of register token number 𝒏. that the model can effectively capture the necessary information for recommendations without a substantial loss in inference efficiency. 3) Too deep: When the register layer depth is too deep (e.g., 𝑘 ≥ 13), the speedup decreases sign…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 26 canonical work pages

  1. [23]

    Xinyu Lin, Chaoqun Yang, Wenjie Wang, Yongqi Li, Cunxiao Du, Fuli Feng, See-Kiong Ng, and Tat-Seng Chua. 2025. Efficient inference for large language model-based generative recommendation. InThe 13th International Conference on Learning Representations

    work page 2025

  2. [1]

    Beidi Chen, Tri Dao, Eric Winsor, Zhao Song, Atri Rudra, and Christopher Ré

    work page

  3. [2]

    Guoxuan Chen, Han Shi, Jiawei Li, Yihang Gao, Xiaozhe Ren, Yimeng Chen, Xin Jiang, Zhenguo Li, Weiyang Liu, and Chao Huang. 2024. SepLLM: Accelerate large language models by compressing one segment into one separator.arXiv preprint arXiv:2412.12094(2024)

    arXiv 2024

  4. [3]

    Alexis Chevalier, Alexander Wettig, Anirudh Ajith, and Danqi Chen. 2023. Adapt- ing language models to compress contexts. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 3829–3846

    work page 2023

  5. [4]

    Yu Cui, Feng Liu, Pengbo Wang, Bohao Wang, Heng Tang, Yi Wan, Jun Wang, and Jiawei Chen. 2024. Distillation matters: empowering sequential recommenders to match the performance of large language models. InProceedings of the 18th ACM Conference on Recommender Systems. 507–517

    2024

  6. [5]

    Yichuan Deng, Zhao Song, Jing Xiong, and Chiwun Yang. 2024. How Sparse Attention Approximates Exact Attention? Your Attention is Naturally𝑛𝐶 -Sparse. arXiv preprint arXiv:2404.02690(2024)

    arXiv 2024

  7. [6]

    Xinyan Fan, Zheng Liu, Jianxun Lian, Wayne Xin Zhao, Xing Xie, and Ji-Rong Wen. 2021. Lighter and better: low-rank decomposed self-attention networks for next-item recommendation. InProceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1733–1737

    work page 2021

  8. [7]

    Xiangming Gu, Tianyu Pang, Chao Du, Qian Liu, Fengzhuo Zhang, Cunxiao Du, Ye Wang, and Min Lin. 2025. When attention sink emerges in language models: An empirical view. InThe 13th International Conference on Learning Representations

    work page 2025

Show all 48 references
  1. [8]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language under- standing. InThe 9th International Conference on Learning Representations

    2021

  2. [9]

    Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2023. LLMLingua: Compressing prompts for accelerated inference of large language models. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 13358–13376

    2023

  3. [10]

    Sein Kim, Hongseok Kang, Seungyoon Choi, Donghyun Kim, Minchul Yang, and Chanyoung Park. 2024. Large language models meet collaborative filtering: An efficient all-round LLM-based recommender system. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Da...

    2024

  4. [11]

    Lei Li, Yongfeng Zhang, and Li Chen. 2023. Prompt distillation for efficient LLM- based recommendation. InProceedings of the 32nd ACM International Conference on Information and Knowledge Management. 1348–1357

    2023

  5. [12]

    Lei Li, Yongfeng Zhang, Dugang Liu, and Li Chen. 2024. Large language models for generative recommendation: A survey and visionary discussions. InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING...

    2024

  6. [13]

    Pengxiang Li, Lu Yin, and Shiwei Liu. 2025. Mix-LN: Unleashing the power of deeper layers by combining Pre-LN and Post-LN, In The 13th International Conference on Learning Representations.arXiv preprint arXiv:2412.13795

    2025 arXiv

  7. [14]

    Yucheng Li, Bo Dong, Frank Guerin, and Chenghua Lin. 2023. Compressing context to enhance inference efficiency of large language models. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 6342–6353

    2023

  8. [15]

    Yuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, and Deming Chen. 2024. SnapKV: LLM knows what you are looking for before generation.Advances in Neural Informa- tion Processing Systems37 (2024), 22947–22970

    2024

  9. [16]

    Yongqi Li, Xinyu Lin, Wenjie Wang, Fuli Feng, Liang Pang, Wenjie Li, Liqiang Nie, Xiangnan He, and Tat-Seng Chua. 2024. A survey of generative search and recom- mendation in the era of large language models.arXiv preprint arXiv:2404.16924 (2024)

    2024 arXiv

  10. [17]

    Zongqian Li, Yinhong Liu, Yixuan Su, and Nigel Collier. 2024. Prompt compression for large language models: A survey. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volu...

    2024

  11. [18]

    Zongqian Li, Yixuan Su, and Nigel Collier. 2024. 500xCompressor: Generalized prompt compression for large language models.arXiv preprint arXiv:2408.03094 (2024)

    2024 arXiv

  12. [19]

    Jianghao Lin, Xinyi Dai, Rong Shan, Bo Chen, Ruiming Tang, Yong Yu, and Weinan Zhang. 2025. Large language models make sample-efficient recommender systems.Frontiers of Computer Science19, 4 (2025), 194328

    2025

  13. [20]

    Jianghao Lin, Xinyi Dai, Yunjia Xi, Weiwen Liu, Bo Chen, Hao Zhang, Yong Liu, Chuhan Wu, Xiangyang Li, Chenxu Zhu, et al . 2025. How can recommender systems benefit from large language models: A survey.ACM Transactions on Information Systems43, 2 (2025), 1–47

    2025

  14. [21]

    Xinyu Lin, Wenjie Wang, Yongqi Li, Fuli Feng, See-Kiong Ng, and Tat-Seng Chua

  15. [22]

    Xinyu Lin, Wenjie Wang, Yongqi Li, Shuo Yang, Fuli Feng, Yinwei Wei, and Tat-Seng Chua. 2024. Data-efficient fine-tuning for LLM-based recommendation. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. 365–374

    2024

  16. [24]

    Langming Liu, Liu Cai, Chi Zhang, Xiangyu Zhao, Jingtong Gao, Wanyu Wang, Yifu Lv, Wenqi Fan, Yiqi Wang, Ming He, et al. 2023. Linrec: Linear attention mechanism for long-term sequential recommender systems. InProceedings of the 46th International ACM SIGIR Conference on Resea...

    2023

  17. [25]

    Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. 2022. P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics...

    2022

  18. [26]

    Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. 2024. GPT understands, too.AI Open5 (2024), 208–215

    2024

  19. [27]

    Shi Luohe, Hongyi Zhang, Yao Yao, Zuchao Li, et al. 2024. Keep the cost down: A review on methods to optimize LLM’s KV-Cache consumption. InThe 1st Conference on Language Modeling (COLM)

    2024

  20. [28]

    Jesse Mu, Xiang Li, and Noah Goodman. 2023. Learning to compress prompts with gist tokens.Advances in Neural Information Processing Systems36 (2023), 19327–19352

    2023

  21. [29]

    Piotr Nawrot, Adrian Łańcucki, Marcin Chochowski, David Tarjan, and Edoardo M Ponti. 2024. Dynamic memory compression: retrofitting LLMs for accelerated inference. InThe 41st International Conference on Machine Learning. 37396–37412

    2024

  22. [30]

    Jianhui Pang, Fanghua Ye, Derek Wong, Xin He, Wanshun Chen, and Longyue Wang. 2024. Anchor-based large language models. InFindings of the Association for Computational Linguistics. 4958–4976

    2024

  23. [31]

    Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan Hulikal Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Tran, Jonah Samost, et al

  24. [32]

    Zhenmei Shi, Yifei Ming, Xuan-Phi Nguyen, Yingyu Liang, and Shafiq Joty. 2024. Discovering the gems in early layers: Accelerating long-context LLMs with 1000x input token reduction.arXiv preprint arXiv:2409.17422(2024)

    2024 arXiv

  25. [33]

    Wenqi Sun, Ruobing Xie, Junjie Zhang, Wayne Xin Zhao, Leyu Lin, and Ji-Rong Wen. 2024. Distillation is all you need for practically using different pre-trained recommendation models.arXiv preprint arXiv:2401.00797(2024)

    2024 arXiv

  26. [34]

    Qwen Team. 2024. Qwen2.5: A party of foundation models. https://qwenlm. github.io/blog/qwen2.5/

    2024

  27. [35]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971(2023)

    2023 arXiv

  28. [36]

    Wenjie Wang, Xinyu Lin, Fuli Feng, Xiangnan He, and Tat-Seng Chua. 2023. Generative recommendation: Towards next-generation recommender paradigm. arXiv preprint arXiv:2304.03516(2023)

    2023 arXiv

  29. [37]

    Haotian Wu, Yingpeng Du, Zhu Sun, Tianjun Wei, Jie Zhang, and Ong Yew Soon

  30. [38]

    Likang Wu, Zhi Zheng, Zhaopeng Qiu, Hao Wang, Hongchao Gu, Tingjia Shen, Chuan Qin, Chen Zhu, Hengshu Zhu, Qi Liu, et al . 2024. A survey on large language models for recommendation.World Wide Web27, 5 (2024), 60

    2024

  31. [39]

    Yunjia Xi, Hangyu Wang, Bo Chen, Jianghao Lin, Menghui Zhu, Weiwen Liu, Ruiming Tang, Weinan Zhang, and Yong Yu. 2025. Efficiency unleashed: Inference acceleration for LLM-based recommender systems with speculative decoding. arXiv preprint arXiv:2408.05676(2025)

    2025 arXiv

  32. [40]

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. 2024. Efficient streaming language models with attention sinks. InThe 12th International Conference on Learning Representations

    2024

  33. [41]

    A survey on efficient solutions of large language models for recommenda- tion.Authorea Preprints(2024)

    2024

  34. [42]

    Yuxin Zhang, Yuxuan Du, Gen Luo, Yunshan Zhong, Zhenyu Zhang, Shiwei Liu, and Rongrong Ji. 2024. CaM: Cache merging for memory-efficient LLMs inference. InThe 41st International Conference on Machine Learning. 58840–58850

    2024

  35. [43]

    Bowen Zheng, Yupeng Hou, Hongyu Lu, Yu Chen, Wayne Xin Zhao, Ming Chen, and Ji-Rong Wen. 2024. Adapting large language models by integrating collaborative semantics for recommendation. In2024 IEEE 40th International Conference on Data Engineering (ICDE). 1435–1448

    2024

  36. [44]

    Zixuan Zhou, Xuefei Ning, Ke Hong, Tianyu Fu, Jiaming Xu, Shiyao Li, Yuming Lou, Luning Wang, Zhihang Yuan, Xiuhong Li, et al. 2024. A survey on efficient inference for large language models.arXiv preprint arXiv:2404.14294(2024). EARN: Efficient Inference Acceleration for LLM-...

    2024 arXiv

  37. [45]

    Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhaojie Gong, Fangda Gu, Jiayuan He, et al . 2024. Actions speak louder than words: trillion-parameter sequential transducers for generative recommendations. InThe 41st International Conference on Machi...

    2024

  38. [2021]

    Scatterbrain: Unifying sparse and low-rank attention.Advances in Neural Information Processing Systems34 (2021), 17413–17426

    2021

  39. [2023]

    Recommender systems with generative retrieval.Advances in Neural Information Processing Systems36 (2023), 10299–10315

    2023

  40. [2024]

    InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

    Bridging items and language: A transition paradigm for large language model-based recommendation. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1816–1826

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.