Pith. sign in

REVIEW 3 major objections 5 minor 52 references

Multi-Grained Patch Training for Efficient LLM-based Recommendation

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Hierarchical patch compression lets LLM recommenders use fewer tokens and get better accuracy.

desk verdict PatchRec's token savings are real, but its accuracy gains over TALLRec are confounded by an extra training epoch; a careful revision should control for training budget and probe the pooling assumption. read the letter →

arxiv 2501.15087 v2 pith:XI63H4CT submitted 2025-01-25 cs.IR

Jiayi Liao , Ruobing Xie , Sihang Li , Xiang Wang , Xingwu Sun , Zhanhui Kang , Xiangnan He This is my paper · ORCID
classification cs.IR
keywords LLM-basedrecommendationsequentialcontextcompressionitempatchsessionefficientinferencelonguserhistoryhierarchical
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PatchRec claims that an LLM recommender can model long interaction histories without blowing up the prompt by replacing most of the text with averaged embeddings—“patches”—at two granularities: one vector per item, and one vector per short block of items. To make this work, the paper adds two training stages: a pre-training stage that pairs each compressed sequence with its uncompressed twin so the LLM learns to map patches back to titles, and a fine-tuning stage that compresses old interactions more aggressively than recent ones. On three benchmarks, this yields large efficiency gains: for the same truncated 100-item sequence, PatchRec uses only 7.34% of the LLM input tokens TALLRec needs while improving HR@20 by up to 32% on Goodreads, and at equal token counts it models 3.44 times more user behaviors with up to 13% higher HR@20 on MovieLens-1M. If these results hold, the main obstacle to lifelong user modeling in LLM recommenders—context length and quadratic attention cost—is substantially reduced.

What carries the argument

The load-bearing object is the hierarchical patch, constructed entirely in the LLM's embedding space. For item $j$, $\mathbf{z}^i_j = \frac{1}{|\mathcal{T}_j|}\sum_{t\in\mathcal{T}_j}\mathbf{x}_t$ averages the embedding vectors of the title's tokens; for a session of $L$ consecutive items, $\mathbf{z}^s_j = \frac{1}{|\mathcal{S}_j|}\sum_{t\in\mathcal{S}_j}\mathbf{z}^i_t$ averages those item patches. These vectors have the same dimensionality as ordinary token embeddings, so they fit into the prompt as pseudo-tokens. The mechanism that makes them usable is the two-stage training: Patch Pre-training augments each uncompressed sequence with a compressed twin and ramps the compression probability $p=\tau/T$ from 0 to 1, teaching the model the mapping from text spans to item patches; Patch Fine-tuning then fixes a time-aware granularity (PFT-I: recent $M$ items as text, everything earlier as item patches; PFT-S: latest group as text, second-latest as item patches, the rest as session patches), letting the model learn that old interactions carry less precise but still useful signal. The paper's argument is that this combination—not the averaging alone—is what lets the LLM interpret the patches and exploit long histories.

What would settle it

Train the identical two-stage pipeline but replace average pooling with a learned linear projection of the same token embeddings (or with attention pooling); if HR@20 does not change or improves, then the specific averaging identity is not the carrier of the reported gains. Alternatively, after Patch Pre-training, probe the LLM's hidden states at item-patch positions with a linear classifier trained to predict the item ID; if probe accuracy is near chance while end-task HR@20 is high, the accuracy gains are not explained by the patches encoding item identity.

Watch

Extended reading notes

Core claim

The paper's central claim is that a simple hierarchical average-pooling compression, if paired with the right training signal, is not merely a token-saving trick but an accuracy-improving representation for LLM-based sequential recommendation. An item patch is the mean of the token embeddings of that item's title (Eq. 2); a session patch is the mean of the item patches in a fixed-size block (Eq. 3). Because these patches live in the LLM's own embedding space, they can be fed to the model as if they were tokens, but the LLM does not natively know what they mean. PatchRec therefore first pre-trains with a schedule that gradually increases the probability of compressing each item, always showing a compressed and an uncompressed copy of the same sequence in the same batch, and then fine-tunes on sequences in which older interactions are compressed more (session patches), mid-age ones into item patches, and the newest interactions stay as raw text. The experiments claim that, with this recipe, PatchRec-I improves HR@20 over TALLRec by 9.04%/26.75%/3.45% at compression ratios 3.44/6.81/3.38 on MovieLens-1M, Goodreads, and MovieLens-100K, and PatchRec-S reaches compression ratios 9.23/13.62 while still beating TALLRec on the first two datasets. The paper interprets the gains as evidence that compression removes noise and that the two-stage training lets the LLM internalize both the patch vocabulary and the temporal importance of recent interactions.

Load-bearing premise

The whole result rests on the assumption that averaging token embeddings into an item patch, and then averaging item patches into a session patch, preserves enough of each item's identity and of the user's preference signal that fine-tuning can recover it; the paper never measures this information loss directly, and it only tests the assumption indirectly through end-task Hit Rate on three datasets.

Editorial extensions

If this is right

  • With the same token budget as TALLRec, PatchRec-I processes 100 items and still beats it on Hit Rate, with relative HR@20 gains of at least 7%, 20%, and 2% on MovieLens-1M, Goodreads, and MovieLens-100K.
  • At the same item count (100), PatchRec-I improves HR@20 over TALLRec by 9.04% on MovieLens-1M, 26.75% on Goodreads, and 3.45% on MovieLens-100K while using 3.44x, 6.81x, and 3.38x fewer tokens.
  • PatchRec-S shows the trade-off can be pushed much harder: at compression ratios of 9.23 and 13.62, it still beats TALLRec's HR@20 on MovieLens-1M and Goodreads.
  • For users with at least 500 interactions, PatchRec-I reaches a 25.11% HR@20 gain with only 12.99% of TALLRec's inference time, and PatchRec-S achieves a 20.31% gain with a 13.09-fold reduction in inference time, so the efficiency benefit grows with history length.
  • The ablation against dropout shows the gains are not a side effect of randomly deleting items; compression that keeps averaged content is what matters, not input dropout.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the paper does not run: apply the same embedding-space patch vocabulary to other long-input LLM tasks such as session-based search or dialogue history, since the compression itself is parameter-free and task-agnostic.
  • The gradual $p=\tau/T$ schedule in Patch Pre-training is a curriculum; one could make fine-tuning granularity adaptive per user or per sequence length, compressing more aggressively for users with long, noisy histories.
  • If average pooling really removes interaction noise, PatchRec should be more robust than TALLRec to injected noisy or irrelevant interactions in the history, which is a direct experiment absent from the paper.
  • The paper equates token count with compute; a direct FLOPs or wall-clock comparison on identical hardware at matched batch sizes would sharpen the efficiency claim, since patches may shift attention and caching behavior.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes PatchRec, a two-stage training method for LLM-based sequential recommendation. It compresses item titles into item patches via average pooling of token embeddings (Eq. 2) and further compresses groups of item patches into session patches (Eq. 3). In Patch Pre-training, the model sees both raw and randomly compressed versions of each sequence, with the compression probability increasing over training. In Patch Fine-tuning, recent items are kept as text while earlier items are represented as item patches (PFT-I) or session patches (PFT-S). Experiments on MovieLens-1M, Goodreads, and MovieLens-100K report that PatchRec reduces LLM input tokens by up to 13.62x while improving HR@20 over TALLRec by up to about 32%. The paper claims that PatchRec models longer user histories at lower inference cost without accuracy loss.

Significance. If the results hold, PatchRec is a practical and timely contribution to LLM-based recommendation, directly addressing the context-length bottleneck. The method is simple, the code is released, and the paper includes useful analyses such as the effect of sequence length and an ablation of the pre-training stage. However, the main accuracy claims are weakened by a training-budget confound, and the long-history advantages are not compared against existing LLM-based long-history recommenders. The representational assumption behind average pooling is also insufficiently examined. These issues are fixable but currently limit the confidence in the central claims.

major comments (3)
  1. [Section 4.1, Table 2] The comparison between PatchRec and TALLRec is not controlled for training budget. Section 4.1 states 'Each training stage consists of one epoch,' so PatchRec trains for two epochs (Patch Pre-training plus Patch Fine-tuning) while TALLRec, described as standard SFT, trains for one epoch. Moreover, during Patch Pre-training each batch contains both the raw and compressed versions of each sequence, so the model sees roughly two training examples per original interaction in that epoch and about three examples total per interaction. The reported HR@20 gains in Table 2 may therefore reflect additional training signal rather than the patch representation itself. Please add a control: TALLRec trained for the same number of epochs or gradient steps, or PatchRec with the pre-training stage removed but the total step count matched to TALLRec. Without this control, the headline improvement of up to 32% HR@20 is not established.
  2. [Section 3.1, Eqs. (2)-(3)] The paper assumes that average pooling of token embeddings into item patches and of item patches into session patches preserves enough item identity and preference signal for fine-tuning to recover. No analysis of information loss, no comparison with alternative pooling functions (e.g., max pooling, learned projections), and no probing of whether the LLM actually uses the patch representations is provided. This is load-bearing for the central claim that patch training lets LLMs interpret compressed embeddings. Please add an ablation of pooling choices and at least a basic analysis (e.g., cosine similarity of patch embeddings to original item embedding, or attention analysis) showing that patches retain item-specific information.
  3. [Section 4.6, Section 5.2] The paper claims that PatchRec models long user histories, but the experiments only compare against TALLRec and non-LLM baselines. The related work section cites existing LLM-based methods for long-term behavior modeling, namely ReLLA, PURE, and MemoryBank. None of these are included in the experiments, so the claim that PatchRec is capable of 'modeling practical historical behavior lengths' is not tested against the most relevant baselines. Please add comparisons to at least one such method, or explicitly restrict the claim to truncation-based LLM recommenders.
minor comments (5)
  1. [Table 2] The asterisk indicating significance (p-value << 0.05) is not accompanied by a description of the statistical test, the number of random seeds, or the variance across runs. Please report these details.
  2. [Section 4.5.2] The final sentence of Section 4.5.2 ends with 'ensuring that compression enhances.' which is incomplete. It should finish the thought, for example '...enhances recommendation performance.'
  3. [Section 3.2.2] In the first paragraph, 'Figue 5' should be 'Figure 5'.
  4. [Figure 6 caption] The caption contains 'MoviLens-100K', which should be 'MovieLens-100K'.
  5. [Section 4.6] The phrase 'with a even higher 13.09-fold reduction' should be 'with an even higher 13.09-fold reduction'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PatchRec's efficiency gains and accuracy comparisons are measured against external baselines, and the compressed embeddings are trained rather than defined by the reported outcomes.

full rationale

The central claims—token reduction ratios (7.34%, 3.44x, etc.) and HR@20 gains over TALLRec—are empirical measurements on MovieLens and Goodreads, not quantities fitted from or defined by the reported result. Eqs. (2) and (3) define item and session patches by average pooling of LLM token embeddings; this is an architectural choice, and the two-stage training (Patch Pre-training and Patch Fine-tuning) is a training procedure, not a derivation whose conclusion equals its premise. Table 3 ablates pre-training against no pre-training, and Table 4 compares against item dropout, so the improvement is not merely the act of dropping tokens. The M and L hyperparameters are hand-set and swept in Figure 7, which is parameter selection rather than fitted-input-as-prediction. The only related concern—that PatchRec trains for two epochs (Section 4.1: 'Each training stage consists of one epoch.') while TALLRec is trained for one—is a potential experimental confound about training budget, not circularity: it does not make any reported number equivalent by construction to an input. No load-bearing self-citation or imported uniqueness theorem appears; TALLRec and LLaRA citations are prior external methods, not justifications that force PatchRec's structure.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central method relies on a small set of unproven modeling assumptions about embedding semantics and LLM learnability. The free parameters M, L, and the compression schedule control the compression geometry and are chosen per dataset; the method itself does not derive them. No new physical or architectural entities are postulated.

free parameters (3)
  • M (number of most recent items kept as text in PFT-I) = 5 in main results; swept over 3, 5, 10, 20, 30, 50, 100 in Figure 7
    Controls the trade-off between retaining fine-grained text for recent items and achieving a higher compression ratio. The value is chosen per experiment without a principled rule.
  • L (session group size in PFT-S) = 5 for MovieLens-1M and Goodreads; 20 for MovieLens-100K and the long-history experiment
    Sets the boundary between item-level and session-level compression. It directly affects both compression ratio and recommendation accuracy.
  • p = tau/T compression schedule = linear from 0 to 1 over the pre-training stage
    The rate at which items are randomly replaced by patches during pre-training is chosen without ablation. The method depends on this curriculum for learning the patch-to-text correspondence.
assumptions (3)
  • domain assumption Average pooling of token embeddings yields a semantically useful item patch (Eq. 2).
    No information-theoretic or analytical justification is provided; all downstream results depend on this representation being learnable and preserving enough item identity.
  • domain assumption Showing compressed and uncompressed versions in one batch teaches the LLM the mapping between patches and text (Section 3.2.1).
    The paper provides an intuition and an ablation (Table 3), but no mechanistic evidence that the correspondence is learned rather than the patches simply acting as new tokens.
  • domain assumption Temporal decay of user interest can be modeled by compressing older interactions more heavily (Section 3.2.2, Eqs. 6 and 7).
    This is motivated by general sequential recommendation prior work, but the fixed stepwise decay by compression granularity is ad hoc and not compared to alternative decay patterns.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multi-Grained Patch Training for Efficient LLM-based Recommendation." pith.science (2026). https://pith.science/paper/XI63H4CT

@misc{pith2026250115087,
  author       = {Pith},
  title        = {Pith review of: Multi-Grained Patch Training for Efficient LLM-based Recommendation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XI63H4CT}},
  note         = {Machine review of arXiv:2501.15087}
}
read the original abstract

Large Language Models (LLMs) have emerged as a new paradigm for recommendation by converting interacted item history into language modeling. However, constrained by the limited context length of LLMs, existing approaches have to truncate item history in the prompt, focusing only on recent interactions and sacrificing the ability to model long-term history. To enable LLMs to model long histories, we pursue a concise embedding representation for items and sessions. In the LLM embedding space, we construct an item's embedding by aggregating its textual token embeddings; similarly, we construct a session's embedding by aggregating its item embeddings. While efficient, this way poses two challenges since it ignores the temporal significance of user interactions and LLMs do not natively interpret our custom embeddings. To overcome these, we propose PatchRec, a multi-grained patch training method consisting of two stages: (1) Patch Pre-training, which familiarizes LLMs with aggregated embeddings -- patches, and (2) Patch Fine-tuning, which enables LLMs to capture time-aware significance in interaction history. Extensive experiments show that PatchRec effectively models longer behavior histories with improved efficiency. This work facilitates the practical use of LLMs for modeling long behavior histories. Codes are available at https://github.com/ljy0ustc/PatchRec.

Figures

Figures reproduced from arXiv: 2501.15087 by the authors.

Figure 1
Figure 1. Most LLM4SR works consider the latest-𝐾 interac￾tions in the complete user interaction sequence as the trun￾cated historical item sequence fed to LLMs. accommodation of a significantly larger number of items within a fixed context window. However, the following two challenges remain: • Temporal significance of interactions: Such compressed repre￾sentations do not explicitly regard the temporal distinctions of releva… view at source ↗
Figure 4
Figure 4. Hierarchical compression. The textual tokens of an [PITH_FULL_IMAGE:figures/full_fig_p003_4.png] view at source ↗
Figure 5
Figure 5. Two-stage training framework of PatchRec. In the patch pre-training stage, we augment each uncompressed sequence [PITH_FULL_IMAGE:figures/full_fig_p004_5.png] view at source ↗
Figures from the paper (4 more)
Figure 6
Figure 6. Figure 6: Performance comparison between PatchRec-I with TALLRec with the same item numbers in interaction sequence. [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Performance comparison between PatchRec-I with TALLRec with comparable numbers of tokens in the interaction [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: The performance-efficiency trade-off of TALLRec, PatchRec-I, and PatchRec-S on Goodreads and MovieLens-100K [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: Performance (i.e., HR@20) and efficiency (i.e., in￾ference time) comparison of TALLRec, PatchRec-I, and PatchRec-S for long sequences (i.e., users with at least 500 be￾haviors) on MovieLens-1M. PatchRec enhances both perfor￾mance and efficiency in modeling long-term us…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 19 canonical work pages

  1. [1]

    Seunghwan Bang and Hwanjun Song. 2025. LLM-based User Profile Management for Recommender System. arXiv preprint arXiv:2502.14541 (2025)

    arXiv 2025

  2. [2]

    Keqin Bao, Jizhi Zhang, Wenjie Wang, Yang Zhang, Zhengyi Yang, Yancheng Luo, Fuli Feng, Xiangnan He, and Qi Tian. 2023. A Bi-Step Grounding Paradigm SIGIR ’25, July 13–18, 2025, Padua, Italy Jiayi Liao et al. for Large Language Models in Recommendation Systems. CoRR abs/2308.08434 (2023)

    arXiv 2023

  3. [3]

    Keqin Bao, Jizhi Zhang, Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He

  4. [4]

    Alexis Chevalier, Alexander Wettig, Anirudh Ajith, and Danqi Chen. 2023. Adapt- ing Language Models to Compress Contexts. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing . 3829–3846

    work page 2023

  5. [5]

    DeepSeek-AI. 2024. DeepSeek-V3 Technical Report. arXiv:2412.19437 [cs.CL] https://arxiv.org/abs/2412.19437

    arXiv 2024

  6. [6]

    Yiran Ding, Li Lyna Zhang, Chengruidong Zhang, Yuanyuan Xu, Ning Shang, Jiahang Xu, Fan Yang, and Mao Yang. 2024. LongRoPE: extending LLM con- text window beyond 2 million tokens. In Proceedings of the 41st International Conference on Machine Learning . 11091–11104

    work page 2024

  7. [7]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, and et al. 2024. The Llama 3 Herd of Models. CoRR abs/2407.21783 (2024)

    arXiv 2024

  8. [8]

    Yunfan Gao, Tao Sheng, Youlin Xiang, Yun Xiong, Haofen Wang, and Jiawei Zhang. 2023. Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System. CoRR abs/2303.14524 (2023)

    arXiv 2023

Show all 52 references
  1. [9]

    Tao Ge, Hu Jing, Lei Wang, Xun Wang, Si-Qing Chen, and Furu Wei. [n. d.]. In-context Autoencoder for Context Compression in a Large Language Model. In The Twelfth International Conference on Learning Representations

  2. [10]

    Shijie Geng, Shuchang Liu, Zuohui Fu, Yingqiang Ge, and Yongfeng Zhang. 2022. Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5). In RecSys. ACM, 299–315

    2022

  3. [11]

    Gemini Team Google. 2023. Gemini: A Family of Highly Capable Multimodal Models. CoRR abs/2312.11805 (2023)

    2023 arXiv

  4. [12]

    Albert Gu and Tri Dao. 2023. Mamba: Linear-Time Sequence Modeling with Selective State Spaces. arXiv preprint arXiv:2312.00752 (2023)

    2023 arXiv

  5. [13]

    Zhankui He, Zhouhang Xie, Rahul Jha, Harald Steck, Dawen Liang, Yesu Feng, Bodhisattwa Prasad Majumder, Nathan Kallus, and Julian McAuley. 2023. Large language models as zero-shot conversational recommenders. InProceedings of the 32nd ACM international conference on informatio...

    2023

  6. [14]

    Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk

  7. [15]

    Yupeng Hou, Shanlei Mu, Wayne Xin Zhao, Yaliang Li, Bolin Ding, and Ji-Rong Wen. 2022. Towards Universal Sequence Representation Learning for Recom- mender Systems. In KDD. ACM, 585–593

    2022

  8. [16]

    Yupeng Hou, Junjie Zhang, Zihan Lin, Hongyu Lu, Ruobing Xie, Julian McAuley, and Wayne Xin Zhao. 2024. Large language models are zero-shot rankers for recommender systems. In European Conference on Information Retrieval. Springer, 364–381

    2024

  9. [17]

    McAuley, and Wayne Xin Zhao

    Yupeng Hou, Junjie Zhang, Zihan Lin, Hongyu Lu, Ruobing Xie, Julian J. McAuley, and Wayne Xin Zhao. 2024. Large Language Models are Zero-Shot Rankers for Recommender Systems. In ECIR (2) (Lecture Notes in Computer Science, Vol. 14609). Springer, 364–381

    2024

  10. [18]

    Wenyue Hua, Shuyuan Xu, Yingqiang Ge, and Yongfeng Zhang. 2023. How to Index Item IDs for Recommendation Foundation Models. In SIGIR-AP. ACM, 195–204

    2023

  11. [19]

    Bowen Jin, Hansi Zeng, Guoyin Wang, Xiusi Chen, Tianxin Wei, Ruirui Li, Zhengyang Wang, Zheng Li, Yang Li, Hanqing Lu, Suhang Wang, Jiawei Han, and Xianfeng Tang. 2024. Language Models as Semantic Indexers. In ICML. OpenReview.net

    2024

  12. [20]

    Wang-Cheng Kang and Julian J. McAuley. 2018. Self-Attentive Sequential Rec- ommendation. In ICDM. IEEE Computer Society, 197–206

    2018

  13. [21]

    Jiacheng Li, Ming Wang, Jin Li, Jinmiao Fu, Xin Shen, Jingbo Shang, and Julian J. McAuley. 2023. Text Is All You Need: Learning Language Representations for Sequential Recommendation. In KDD. ACM, 1258–1267

    2023

  14. [22]

    Jiayi Liao, Xiangnan He, Ruobing Xie, Jiancan Wu, Yancheng Yuan, Xingwu Sun, Zhanhui Kang, and Xiang Wang. 2024. RosePO: Aligning LLM-based Recom- menders with Human Values. CoRR abs/2410.12519 (2024)

    2024 arXiv

  15. [23]

    Jiayi Liao, Sihang Li, Zhengyi Yang, Jiancan Wu, Yancheng Yuan, Xiang Wang, and Xiangnan He. 2024. LLaRA: Large Language-Recommendation Assistant. In SIGIR. ACM, 1785–1795

    2024

  16. [24]

    Jianghao Lin, Rong Shan, Chenxu Zhu, Kounianhua Du, Bo Chen, Shigang Quan, Ruiming Tang, Yong Yu, and Weinan Zhang. 2024. ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendation. In WWW. ACM, 3497–3508

    2024

  17. [25]

    Chengkai Liu, Jianghao Lin, Jianling Wang, Hanzhou Liu, and James Caverlee

  18. [26]

    Langming Liu, Liu Cai, Chi Zhang, Xiangyu Zhao, Jingtong Gao, Wanyu Wang, Yifu Lv, Wenqi Fan, Yiqi Wang, Ming He, Zitao Liu, and Qing Li. 2023. LinRec: Linear Attention Mechanism for Long-term Sequential Recommender Systems. In SIGIR. ACM, 289–299

    2023

  19. [27]

    Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics 12 (2024), 157–173

    2024

  20. [28]

    Jesse Mu, Xiang Li, and Noah Goodman. 2023. Learning to compress prompts with gist tokens. Advances in Neural Information Processing Systems 36 (2023), 19327–19352

    2023

  21. [29]

    OpenAI. 2023. GPT-4 Technical Report. CoRR abs/2303.08774 (2023)

    2023 arXiv

  22. [30]

    Jack W Rae, Anna Potapenko, Siddhant M Jayakumar, Chloe Hillier, and Tim- othy P Lillicrap. [n. d.]. Compressive Transformers for Long-Range Sequence Modelling. In International Conference on Learning Representations

  23. [31]

    Tran, Jonah Samost, Maciej Kula, Ed H

    Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan Hulikal Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Q. Tran, Jonah Samost, Maciej Kula, Ed H. Chi, and Mahesh Sathiamoorthy. 2023. Recommender Systems with Generative Retrieval. In NeurIPS

    2023

  24. [32]

    Chenze Shao, Fandong Meng, and Jie Zhou. 2024. Patch-Level Training for Large Language Models. arXiv:2407.12665 [cs.CL] https://arxiv.org/abs/2407.12665

    2024 arXiv

  25. [33]

    Jianlin Su, Murtadha H. M. Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. 2024. RoFormer: Enhanced transformer with Rotary Position Embedding. Neurocomputing 568 (2024), 127063

    2024

  26. [34]

    Jiaxi Tang and Ke Wang. 2018. Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding. In WSDM. ACM, 565–573

    2018

  27. [35]

    Sheng, and Mehmet A

    Shoujin Wang, Liang Hu, Yan Wang, Longbing Cao, Quan Z. Sheng, and Mehmet A. Orgun. 2019. Sequential Recommender Systems: Challenges, Progress and Prospects. In IJCAI. ijcai.org, 6332–6338

    2019

  28. [36]

    Xiaoqiang Wang, Suyuchen Wang, Yun Zhu, and Bang Liu. 2025. R 3 Mem: Bridging Memory Retention and Retrieval via Reversible Compression. arXiv preprint arXiv:2502.15957 (2025)

    2025 arXiv

  29. [37]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Cheng- peng Li, and et al. 2024. Qwen2 Technical Report. CoRR abs/2407.10671 (2024)

    2024 arXiv

  30. [38]

    Zhengyi Yang, Xiangnan He, Jizhi Zhang, Jiancan Wu, Xin Xin, Jiawei Chen, and Xiang Wang. 2023. A Generic Learning Framework for Sequential Recommenda- tion with Distribution Shifts. In SIGIR. ACM, 331–340

    2023

  31. [39]

    Zhengyi Yang, Jiancan Wu, Yanchen Luo, Jizhi Zhang, Yancheng Yuan, An Zhang, Xiang Wang, and Xiangnan He. 2023. Large Language Model Can Interpret Latent Space of Sequential Recommender. CoRR abs/2310.20487 (2023)

    2023 arXiv

  32. [40]

    Haochao Ying, Fuzhen Zhuang, Fuzheng Zhang, Yanchi Liu, Guandong Xu, Xing Xie, Hui Xiong, and Jian Wu. 2018. Sequential Recommender System based on Hierarchical Attention Networks. In IJCAI. ijcai.org, 3926–3932

    2018

  33. [41]

    Chanwoong Yoon, Taewhoo Lee, Hyeon Hwang, Minbyul Jeong, and Jaewoo Kang. 2024. CompAct: Compressing Retrieved Documents Actively for Question Answering. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 21424–21439

    2024

  34. [42]

    Zheng Yuan, Fajie Yuan, Yu Song, Youhua Li, Junchen Fu, Fei Yang, Yunzhu Pan, and Yongxin Ni. 2023. Where to Go Next for Recommender Systems? ID- vs. Modality-based Recommender Models Revisited. In SIGIR. ACM, 2639–2649

    2023

  35. [43]

    Zhenrui Yue, Sara Rabhi, Gabriel de Souza Pereira Moreira, Dong Wang, and Even Oldridge. 2023. LlamaRec: Two-Stage Recommendation using Large Language Models for Ranking. CoRR abs/2311.02089 (2023)

    2023 arXiv

  36. [44]

    Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, et al. 2020. Big bird: Transformers for longer sequences. Advances in neural information processing systems 33 (2020), 17283–17297

    2020

  37. [45]

    Junjie Zhang, Ruobing Xie, Yupeng Hou, Xin Zhao, Leyu Lin, and Ji-Rong Wen

  38. [46]

    Yang Zhang, Fuli Feng, Jizhi Zhang, Keqin Bao, Qifan Wang, and Xiangnan He

  39. [47]

    Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. 2024. Memo- rybank: Enhancing large language models with long-term memory. InProceedings of the AAAI Conference on Artificial Intelligence , Vol. 38. 19724–19731

    2024

  40. [49]

    ACM Transactions on Information Systems (2023)

    Recommendation as instruction following: A large language model em- powered recommendation approach. ACM Transactions on Information Systems (2023)

    2023

  41. [2016]

    In ICLR (Poster)

    Session-based Recommendations with Recurrent Neural Networks. In ICLR (Poster)

  42. [2023]

    In RecSys

    TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation. In RecSys. ACM, 1007–1014

  43. [2024]

    CoRR abs/2403.03900 (2024)

    Mamba4Rec: Towards Efficient Sequential Recommendation with Selective State Space Models. CoRR abs/2403.03900 (2024)

    2024 arXiv

  44. [2025]

    IEEE Trans

    CoLLM: Integrating Collaborative Embeddings Into Large Language Models for Recommendation. IEEE Trans. Knowl. Data Eng. 37, 5 (2025), 2329–2340

    2025

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.