REVIEW 3 major objections 4 minor 19 references
Large Memory Network for Recommendation
T0 review · 3 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read The paper claims that routing user behavior through a large shared, product-quantized memory block improves recommendation accuracy and can be deployed at million scale with negligible serving cost, with absolute gains shown on Douyin…
desk verdict An honest industry adaptation of product-key memory with real online gains, but the mechanism claims need an ablation that the paper doesn't provide. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the product-quantized memory block: a shared key matrix $M_K \in \mathbb{R}^{n\times d}$ decomposed as $M_K = M_{K-\text{row}} \times M_{K-\text{col}}$ with both factors in $\mathbb{R}^{\sqrt{n}\times d}$, plus a value matrix $M_V$ holding the stored information. The row/column decomposition reduces activation cost from $O(nd)$ to $O(\sqrt{n}d)$; a user-aware query MLP merges the item embedding with user features, top-K scores are selected by softmax, and values are combined by weighted pooling. Memory injection is carried by a Smooth-L1 loss $\text{Loss}_{\text{Memory}} = \text{SmoothL1}(M_O, M_Q)$ that pushes the memory output toward the query, trained jointly with the CTR loss.
What would settle it
Retrain LMN on the same dataset with the memory-injection term removed ($\alpha=0$) and with the memory values frozen at random initialization, keeping the query MLP and key tables trainable. If AUC and LogLoss remain close to the reported 0.7371 and 0.5986, the memory-storage claim is not what drives the gains; if performance drops back toward the MIMN baseline, the injection is doing the work.
Extended reading notes
Core claim
The central claim is that storing compressed user-history information in a global, large-scale memory block, rather than in per-user model weights, lets the recommender both perceive shared interests across similar users and retain long-term interests that fixed-length sequences forget. LMN extracts top-K memory values by a product-quantized two-axis lookup: memory keys are decomposed into row and column keys, so retrieving a value costs $O(\sqrt{n})$ instead of $O(n)$. A user-aware block conditions the memory query on the user embedding, and a Smooth-L1 loss between memory output and query injects user-item interaction information into the shared memory values during training. The paper argues that the resulting memory mechanism is lightweight enough to scale to million-slot tables in production, and that its global sharing is what explains the observed offline and online gains.
Load-bearing premise
The result depends on the assumption that the Smooth-L1 loss in Eq. (11) genuinely writes useful user-item information into the shared memory values; if that loss only acts as regularization, the reported gains could come from the extra query-MLP parameters and auxiliary loss rather than from the memory mechanism.
Editorial extensions
If this is right
- Larger memory tables are directly beneficial: varying the per-axis key count from 50 to 500 improves AUC from 0.7357 to 0.7380 and LogLoss from 0.5997 to 0.5978.
- LMN is a plug-and-play module: it can be attached to an existing CTR model and trained end-to-end with $\text{Loss}_{\text{LMN}} = \text{Loss}_{\text{CTR}} + \alpha \text{Loss}_{\text{Memory}}$.
- Because retrieval cost grows only as $\sqrt{n}$, the memory can be scaled to million-slot capacity with limited compute growth.
- A global shared table makes different users' behavior mutually influence retrieval, providing the spatial generalization that per-user sequence models lack.
Reading between the lines
- Not stated in the paper: an ablation with the memory-injection coefficient $\alpha=0$ and frozen memory values would separate genuine memorization from the added capacity of the query MLP and auxiliary loss.
- Not stated in the paper: the temporal-memory claim implies that users with longer histories should benefit most; a subgroup analysis by history length would test this directly.
- Not stated in the paper: the same shared memory could memorize outputs of other modules, such as feature-cross or compressor MLPs, which the paper's conclusion mentions only as future direction.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Large Memory Network (LMN), a CTR-prediction module that compresses user behavioral sequences into a global, product-quantized memory table of keys and values. During training, top-K memory slots are retrieved with user-aware queries and a Smooth-L1 auxiliary loss injects user-item information into the memory values. The authors report offline gains on a four-week Douyin E-Commerce Search dataset (AUC +4.68% over MIMN) and an online A/B test showing +0.87% order/user and +0.72% order/search improvements, together with a GPU-sharded memory parameter server deployment. The central claim is that the shared memory mechanism, rather than added model capacity, drives these gains and enables spatial perception and temporal memorization.
Significance. If the central claim holds, LMN would be a practically valuable way to increase model capacity in industrial recommenders without proportional compute cost, and the described deployment architecture is a real engineering contribution. The paper also provides a scaling experiment and an online A/B test, which are useful de-risking signals. However, the manuscript does not currently isolate the memory mechanism from added parameters and auxiliary regularization, and the reported significance levels are not backed by confidence intervals or variance information. The central mechanism-level claim is therefore not yet supported by the evidence presented.
major comments (3)
- [2.2.3, Eq. (11); Section 3.2] The only mechanism that writes information into the memory values is the Smooth-L1 loss between M_O and M_Q, where M_Q is computed from the current item x and user features u. This loss does not directly force M_O to contain information from other positions in the user's sequence; it can be satisfied by a codebook of per-item query reconstructions. Because Table 1 includes no ablation that disables the injection loss (e.g., alpha=0) or that replaces learned memory values with random/fixed values while holding the query MLP capacity constant, the reported AUC/LogLoss gains cannot be attributed specifically to spatial or temporal memory content rather than to the additional query MLP parameters and the auxiliary loss's regularizing effect.
- [Table 2; Section 3.1] The scaling experiment varies sqrt(n) from 50 to 500, which simultaneously increases the number of memory keys and memory values and therefore the parameter count. Without controlling for total parameters or measuring memory-slot utilization, the monotone improvement in AUC is equally consistent with a conventional capacity-scaling effect and does not substantiate a memory-specific scaling law. In addition, Section 3.1 states that the number of memory keys is set to 300 while Table 2 labels rows by sqrt(n), leaving the actual value of n and the relation between n and sqrt(n) ambiguous; please define both and report the resulting parameter counts for each row.
- [3.3, Table 3; Table 1] The online A/B test reports p=0 for the two commercial metrics, and Table 1 reports p<0.05, but neither result is accompanied by confidence intervals, variance estimates, the number of experimental units analyzed, or the exact significance test used. With more than 160 million users in the online test, even a negligible effect can be statistically significant, and a p-value of exactly 0 is not a meaningful report for a continuous metric. Please provide effect sizes with confidence intervals and describe the statistical procedure; otherwise the claim of 'substantial significance' is not verifiable.
minor comments (4)
- [Eq. (3) and Eq. (9)] The 'cross operation' in Eq. (3) and the broadcast operation in Eq. (9) are not formally defined; please specify exactly how M_K is constructed from the row and column keys and how S_row and S_col are combined into the two-dimensional score matrix.
- [2.2.4, Eq. (12)] The memory loss weight alpha is never given a value; please report the chosen value and, ideally, a small sensitivity analysis.
- [3.3] The online latency cost is reported only as a 0.38% increment; please include the absolute serving latency before and after deployment and the measurement methodology.
- [3.1] The dataset and code are proprietary, and preprocessing and hyperparameter details are minimal; adding more details (sequence length used, embedding initialization, training configuration) would improve reproducibility.
Circularity Check
No load-bearing circularity; the only self-citation is background material and the auxiliary memory loss is not the evaluation target.
full rationale
The paper's central claim is that the shared, product-quantized memory block improves recommendation quality, as measured by AUC/LogLoss on held-out data and by online order/user and order/search metrics. These metrics are external to the model's training objectives, so the evaluation does not reduce to the model's inputs by construction. The memory injection loss in Eq. 11, SmoothL1(M_O, M_Q), is an auxiliary training loss that aligns the memory output with the user-aware query; it is not a prediction target or an evaluation metric. The reported gains are computed on the last week of a four-week log split (Section 3.1) and on a 20% online A/B test (Section 3.3), independent of the training-time memory loss. The scaling experiment in Table 2 varies the number of memory keys, which is a capacity change rather than a fitted-parameter-then-predicted design. The only self-citation is reference [3] (Zheng Chai et al., 2022), cited in the Introduction for the general statement that user behavior sequences depict user interests; it is background and does not support the memory mechanism itself. The skeptical concern that Eq. 11 may cause the memory values to reconstruct query-like features rather than store cross-user or long-term history is a legitimate attribution/ablation question, but it is not a circular derivation: the model could be improved by added capacity or regularization without the memory storing sequence history, yet the paper's claims would then be empirically false rather than true-by-construction. Therefore no circular step meets the evidentiary bar of exhibiting a specific reduction of a claimed prediction to its own inputs.
Assumptions & free parameters
free parameters (4)
- Memory key count (sqrt(n)) =
300 in main experiments, scaled to 500
- Top-K number of activated memory slots =
not reported
- Memory loss weight alpha =
not reported
- Embedding and memory dimension d =
not reported
assumptions (3)
- domain assumption All users share one global memory block, and a user-aware query is enough to make that shared memory useful across users.
- domain assumption The Smooth-L1 memory loss in Eq. 11 writes compressed user-item behavior into memory values.
- domain assumption The product-quantized additive row/column scores in Eqs. 7-9 approximate the full memory attention well enough for top-K selection.
Cite this review
Pith. "Pith review of Large Memory Network for Recommendation." pith.science (2026). https://pith.science/paper/7JBNNYLD
@misc{pith2026250205558,
author = {Pith},
title = {Pith review of: Large Memory Network for Recommendation},
year = {2026},
howpublished = {\url{https://pith.science/paper/7JBNNYLD}},
note = {Machine review of arXiv:2502.05558}
}
read the original abstract
Modeling user behavior sequences in recommender systems is essential for understanding user preferences over time, enabling personalized and accurate recommendations for improving user retention and enhancing business values. Despite its significance, there are two challenges for current sequential modeling approaches. From the spatial dimension, it is difficult to mutually perceive similar users' interests for a generalized intention understanding; from the temporal dimension, current methods are generally prone to forgetting long-term interests due to the fixed-length input sequence. In this paper, we present Large Memory Network (LMN), providing a novel idea by compressing and storing user history behavior information in a large-scale memory block. With the elaborated online deployment strategy, the memory block can be easily scaled up to million-scale in the industry. Extensive offline comparison experiments, memory scaling up experiments, and online A/B test on Douyin E-Commerce Search (ECS) are performed, validating the superior performance of LMN. Currently, LMN has been fully deployed in Douyin ECS, serving millions of users each day.
Figures
Reference graph
Works this paper leans on
-
[3]
Zheng Chai, Zhihong Chen, Chenliang Li, Rong Xiao, Houyi Li, Jiawei Wu, Jingxu Chen, and Haihong Tang. 2022. User-aware multi-interest learning for candidate matching in recommenders. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval . 1326–1335
2022
-
[1]
Weijie Bian, Kailun Wu, Lejian Ren, Qi Pi, Yujing Zhang, Can Xiao, Xiang-Rong Sheng, Yong-Nan Zhu, Zhangming Chan, Na Mou, et al . 2022. CAN: feature co-action network for click-through rate prediction. InProceedings of the fifteenth ACM international conference on web search and data mining . 57–65
work page 2022
-
[2]
Yue Cao, Xiaojiang Zhou, Jiaqi Feng, Peihao Huang, Yao Xiao, Dayao Chen, and Sheng Chen. 2022. Sampling is all you need on modeling long-term user behaviors for CTR prediction. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management . 2974–2983
2022
-
[4]
Jianxin Chang, Chenbin Zhang, Zhiyi Fu, Xiaoxue Zang, Lin Guan, Jing Lu, Yiqun Hui, Dewei Leng, Yanan Niu, Yang Song, et al. 2023. TWIN: TWo-stage interest network for lifelong user behavior modeling in CTR prediction at kuaishou. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 3785–3794
2023
-
[5]
Gabriel de Souza Pereira Moreira, Sara Rabhi, Jeong Min Lee, Ronay Ak, and Even Oldridge. 2021. Transformers4rec: Bridging the gap between nlp and sequential/session-based recommendation. In Proceedings of the 15th ACM con- ference on recommender systems . 143–153
work page 2021
-
[6]
R Girshick. 2015. Fast R-CNN. In Proceedings of the IEEE International Conference on Computer Vision (ICCV)
work page 2015
-
[7]
Xingzhuo Guo, Junwei Pan, Ximei Wang, Baixu Chen, Jie Jiang, and Mingsheng Long. 2024. On the Embedding Collapse when Scaling up Recommendation Models. In Forty-first International Conference on Machine Learning
work page 2024
-
[8]
Herve Jegou, Matthijs Douze, and Cordelia Schmid. 2010. Product quantization for nearest neighbor search. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 1 (2010), 117–128
2010
Show all 19 references
-
[9]
Guillaume Lample, Alexandre Sablayrolles, Marc’Aurelio Ranzato, Ludovic De- noyer, and Hervé Jégou. 2019. Large memory layers with product keys. Advances in Neural Information Processing Systems 32 (2019)
2019
-
[10]
Qi Liu, Xuyang Hou, Haoran Jin, Zhe Wang, Defu Lian, Tan Qu, Jia Cheng, Jun Lei, et al. 2023. Deep Group Interest Modeling of Full Lifelong User Behaviors for CTR Prediction. arXiv preprint arXiv:2311.10764 (2023)
2023 arXiv
-
[11]
Alexander Miller, Adam Fisch, Jesse Dodge, Amir-Hossein Karimi, Antoine Bor- des, and Jason Weston. 2016. Key-Value Memory Networks for Directly Reading Documents. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing
2016
-
[12]
Qi Pi, Weijie Bian, Guorui Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Practice on long sequential user behavior modeling for click-through rate prediction. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . 2671–2679
2019
-
[13]
Qi Pi, Guorui Zhou, Yujing Zhang, Zhe Wang, Lejian Ren, Ying Fan, Xiaoqiang Zhu, and Kun Gai. 2020. Search-based user interest modeling with lifelong sequential behavior data for click-through rate prediction. In Proceedings of the 29th ACM International Conference on Informat...
2020
-
[14]
Ling Yan, Wu-Jun Li, Gui-Rong Xue, and Dingyi Han. 2014. Coupled group lasso for web-scale ctr prediction in display advertising. In International conference on machine learning. PMLR, 802–810
2014
-
[15]
Wenhui Yu, Chao Feng, Yanze Zhang, Lantao Hu, Peng Jiang, and Han Li. 2024. IFA: Interaction Fidelity Attention for Entire Lifelong Behaviour Sequence Mod- eling. arXiv preprint arXiv:2406.09742 (2024)
2024 arXiv
-
[16]
Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhao- jie Gong, Fangda Gu, Jiayuan He, et al. [n. d.]. Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations. In International conference on machine learning
-
[17]
Buyun Zhang, Liang Luo, Yuxin Chen, Jade Nie, Xi Liu, Shen Li, Yanli Zhao, Yuchen Hao, Yantao Yao, Ellie Dingqiao Wen, et al. [n. d.]. Wukong: Towards a Scaling Law for Large-Scale Recommendation. In Forty-first International Confer- ence on Machine Learning
-
[18]
Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Deep interest evolution network for click-through rate prediction. In Proceedings of the AAAI conference on artificial intelligence , Vol. 33. 5941–5948
2019
-
[19]
Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining ...
2018
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.