REVIEW 3 major objections 6 minor 34 references
An Industrial-Scale Sequential Recommender for LinkedIn Feed Ranking
T0 review · 3 major / 6 minor · reviewed 2026-08-02 · deepseek-v4-flash
Pith's one-line read A transformer that ranks LinkedIn's Feed by interleaving posts and member actions beat the production DCNv2 ranker, lifting time spent by 2.10% in online A/B tests.
desk verdict Credible industrial deployment study with real engineering value; the online lift is under-reported and position debiasing needs a closer look, but this deserves a serious referee. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the interleaved sequence of post and action tokens fed into a decoder-only transformer with a causal attention mask: each historical post is represented by compact feature embeddings, and multi-hot action vectors are projected into the same dimension, so the sequence becomes [post1, action1, post2, action2, ...]. This transformer encodes the member's evolving interests; its output is concatenated with late-fused context features (candidate popularity, dwell-time buckets, viewer–actor affinity) and passed to an MMoE head for multi-task prediction. Position debiasing is handled by learning per-position logit offsets for the top-60 feed positions and scoring candidates at
What would settle it
Run an online A/B test where the same trained Feed SR model is served with the scoring position set to 1 and then to 10 instead of 5. If the position-debiasing is correct, the time-spent lift relative to the production ranker should remain roughly constant across these scoring positions; if the measured lift shifts significantly with scoring position, the reported +2.10% gain is not a clean measure of ranking quality.
Extended reading notes
Core claim
Feed SR is a sequential recommender that encodes a member's recent 1000 impressions as an interleaved sequence of post representations and action representations, processes it with a causal transformer using rotary positional embeddings, then late-fuses candidate-conditional numeric features and passes the result through a multi-gate mixture-of-experts head to predict dwell, like, comment, and share actions. Compared to the existing production DCNv2-based ranker, Feed SR improves time spent by +2.10% overall and up to +2.38% for daily active users in online A/B tests, while using a substantially reduced feature set. The model has been serving the majority of LinkedIn Feed traffic for over th
Load-bearing premise
The load-bearing premise is that the learned position logit offsets plus scoring at position 5 fully remove position bias, so the reported +2.10% time-spent lift reflects genuine relevance gains rather than a calibration artifact.
Editorial extensions
If this is right
- Sequential models can replace many manually engineered history transforms in feed ranking, cutting the feature set to about 20% while improving engagement.
- Late fusion of candidate-conditional features is nearly as accurate as early fusion but reduces training time by about 12%, and also simplifies online serving by reducing history storage and feature-fetch overhead.
- Scoring all candidates in a single forward pass with a shared context (via a custom attention mask and CUDA kernel) makes transformer inference practical at production scale, giving an 80x speedup on the transformer forward pass.
- Increasing sequence length is the most consistent scaling axis for this model, suggesting that longer interaction histories are the most reliable path to further quality gains.
- Learned absolute position embeddings cause score instability due to highly skewed sequence-length distributions, while RoPE keeps predicted scores stable and improves offline AUC.
Reading between the lines
- If the position-debiasing heuristic (learned offsets plus scoring at position 5) is genuinely robust, the same fixed-position scoring trick could be transferable to other feed-ranking systems that suffer from position bias, without needing online position-aware serving.
- The paper's success with a 20% feature set suggests that many cross-features in traditional DLRMs may be redundant when a transformer can learn temporal interaction patterns directly from behavioral sequences.
- The failure of the LLM-based ranker, despite promising offline results, points to a practical lesson: token efficiency in encoding user history is a decisive factor for industrial sequential recommenders, since text-based history encoding became prohibitively expensive.
- The scaling-law result, where sequence length scales more reliably than depth or embedding dimension, could motivate future work on history-length extension (e.g., using more than 1000 impressions or multi-year histories) rather than purely increasing model depth.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents Feed Sequential Recommender (Feed SR), a transformer-based sequential ranking model that replaces the DCNv2-based production ranker for LinkedIn Feed. The model interleaves post and action tokens, uses RoPE, late-fused context features, an MMoE head, IPW plus learned position logit offsets, and a range of training/inference optimizations. Offline ablations report AUC improvements across head architectures, features, loss weighting, and scaling behaviour. Online A/B results in Table 5 claim +2.10% time spent overall, with the largest gains for active members and neutral for new members. Section 9 states Feed SR is now the majority member experience. The abstract additionally claims +3.52% like/comment/reshares, which does not appear in Section 7.
Significance. If the online result is a clean measurement, the paper is a valuable industrial case study showing that a transformer sequential model using roughly 20% of production features can outperform a DCNv2 ranker at LinkedIn scale. The paper's strengths include detailed serving optimizations (SRMIS kernel, shared context batching), compute-matched HSTU comparisons, scaling-law analyses along multiple axes, and honest deployment lessons. The main risk is that the headline lift is a single point estimate with no confidence interval, and the position-debiasing mechanism is asserted rather than validated. These issues are addressable with additional analysis, so the paper merits major revision rather than rejection.
major comments (3)
- [Section 7 / Abstract] The abstract claims +3.52% like/comment/reshares (LCR), but Table 5 in Section 7 reports only time-spent metrics; no LCR result appears anywhere in the results section. If LCR was measured, include it with a confidence interval; if not, the abstract overstates the findings. Additionally, the +2.10% time-spent figure is a single point estimate with no CI, significance level, or segment-level variance. Given that Section 7 breaks out DAU/WAU/MAU/New, reporting standard errors or confidence intervals would materially strengthen the central claim.
- [§4.6.1, Table 5] The validity of the online lift rests on the assertion that scoring at position 5 with IPW and learned position logit offsets removes position bias. The paper gives no diagnostics: no propensity model calibration or coverage check, no analysis of possible double-counting between IPW and explicit position logits, and no stratified online results by served position. Since at serving the position-5 logit offset is constant, it cannot correct position bias already learned from historical exposure; only IPW can, and that is not validated. A stratified analysis of A/B logs by served position, or an offline holdout evaluation with position randomized, would directly test whether the +2.10% reflects relevance rather than positional exposure.
- [§4.7] In-session leakage is mitigated by randomizing item order within sessions, but during serving the model receives chronological order. This train/serve skew is acknowledged but its effect on the reported online gain is not quantified. The paper states that both randomization and attention masking resolve overfitting; if both work, reporting offline AUC for both and an online A/B for the masking variant (or a justification for why randomization does not hurt) would address the concern that the deployed model is trained on a distribution that does not match serving.
minor comments (6)
- [§4.2.2] '1017 FLOPs' should read '10^17 FLOPs' (the superscript is missing).
- [§5.1] 'LLama' should be 'LLaMA'.
- [§8.3] 'from and infrastructure perspective' is a typo; should be 'from a modeling and an infrastructure perspective'.
- [Appendix D] Figures 11 and 12 have y-axis values omitted 'due to legal reasons'; this limits interpretability. Consider plotting normalized axes or explaining the omission in the text.
- [§4.6.2] The claim that early online experiments show gains beyond those reported is unsupported by any numbers. Either include the results or remove the claim.
- [Eq. (1)] The feature transforms f_j are not specified as learned or fixed. Clarify which transforms are learned embeddings, log transforms, or identity mappings.
Circularity Check
No significant circularity: the central claim is an online A/B comparison, and the self-citations are contextual rather than load-bearing.
full rationale
The paper's headline result is an empirical online A/B test (Section 7, Table 5: +2.10% time spent) against the existing production ranker, so the claim is measured rather than derived from a fitted parameter that is then renamed as a prediction. The position-debiasing procedure in Section 4.6.1 (IPW plus learned position logit offsets, with online scoring at position 5) is a training-time calibration device; the paper explicitly says it used this to reuse the production objective weights without further tuning, and it does not generate the reported lift by construction. The self-citations—[3] (existing LiRank/DCNv2 tower), [12] (TransAct multi-item scoring prototype), [19] (retrieval), [20] (post embeddings)—supply contextual infrastructure and are not used to force the architecture, define the success metric, or forbid alternatives. Comparisons against HSTU, TransAct, and LLM-Ranker are run in-paper. No equation or fitted value reduces to its own input, so there is no specific circular step to quote.
Assumptions & free parameters
free parameters (7)
- Maximum sequence length T =
1000
- Online scoring position =
5
- Timestamp loss half-life =
60 days
- Position loss half-life =
sequence length
- Negative downsampling retention and loss weight =
0.1 retention; weight 10
- Inverse propensity scores =
per-position, from offline data
- Scaling-law slope =
0.0093 Long Dwell AUC per order-of-magnitude FLOPs
assumptions (4)
- domain assumption Engagement actions (click, like, comment, share, dwell) are valid proxies for Feed relevance, and optimizing them improves member experience.
- domain assumption Evaluation AUC on logged impressions predicts online member value.
- domain assumption Randomizing item order within sessions during training does not materially change the learned user-interest signal while removing leakage.
- domain assumption Position logit offsets and scoring at position 5 remove position bias.
Cite this review
Pith. "Pith review of An Industrial-Scale Sequential Recommender for LinkedIn Feed Ranking." pith.science (2026). https://pith.science/paper/RMZCQENA
@misc{pith2026260212354,
author = {Pith},
title = {Pith review of: An Industrial-Scale Sequential Recommender for LinkedIn Feed Ranking},
year = {2026},
howpublished = {\url{https://pith.science/paper/RMZCQENA}},
note = {Machine review of arXiv:2602.12354}
}
read the original abstract
LinkedIn Feed enables professionals worldwide to discover relevant content, build connections, and share knowledge at scale. We present Feed Sequential Recommender (Feed SR), a transformer-based sequential ranking model for LinkedIn Feed that replaces a DCNv2-based ranker and meets strict production constraints. We detail the modeling choices, training techniques, and serving optimizations that enable deployment at a scale of 1.2 billion members. Feed SR has been serving the majority of LinkedIn's Feed traffic for over three months and shows significant improvements in member engagement (+2.10% time spent, +3.52% like, comments, or reshares) in online A/B tests compared to the existing production model. We also describe our deployment experience with alternative sequential and LLM-based ranking architectures and why Feed SR provided the best combination of online metrics and production efficiency.
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[30]
Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhao- jie Gong, Fangda Gu, Michael He, et al. 2024. Actions speak louder than words: Trillion-parameter sequential transducers for generative recommendations.arXiv preprint arXiv:2402.17152(2024)
arXiv 2024
-
[1]
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. Layer normaliza- tion.arXiv preprint arXiv:1607.06450(2016)
arXiv 2016
-
[2]
Thomas Bachlechner, Bodhisattwa Prasad Majumder, Henry Mao, Gary Cottrell, and Julian McAuley. 2021. Rezero is all you need: Fast convergence at large depth. InUncertainty in Artificial Intelligence. PMLR, 1352–1361
2021
-
[3]
Fedor Borisyuk, Mingzhou Zhou, Qingquan Song, Siyu Zhu, Birjodh Tiwana, Ganesh Parameswaran, Siddharth Dangi, Lars Hertel, Qiang Charles Xiao, Xi- aochen Hou, et al . 2024. LiRank: Industrial Large Scale Ranking Models at LinkedIn. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 4804–4815
2024
-
[4]
Adam Casson. 2023. Transformer FLOPs. https://adamcasson.com/posts/ transformer-flops. Accessed: 2026-01-28
2023
-
[5]
Yekun Chai, Shuo Jin, and Xinwen Hou. 2020. Highway transformer: Self-gating enhanced self-attentive networks.arXiv preprint arXiv:2004.08178(2020)
arXiv 2020
-
[6]
Qiwei Chen, Huan Zhao, Wei Li, Pipei Huang, and Wenwu Ou. 2019. Behavior sequence transformer for e-commerce recommendation in alibaba. InProceedings of the 1st international workshop on deep learning practice for high-dimensional sparse data. 1–4
2019
-
[7]
Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al
Show all 34 references
-
[8]
Fu, Stefano Ermon, Atri Rudra, and Christopher Ré
Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. 2022. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness. InAdvances in Neural Information Processing Systems (NeurIPS). https://arxiv. org/abs/2205.14135
2022 arXiv
-
[9]
Jaixin Deng, Shiyao Wang, Kuo Cai Li, Qigen Ren, Qigen Hu, Weifeng Ding, Qiang Luo, and Guorui Zhou. 2025. OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignment.arXiv preprint arXiv:2502.18596(2025). https://arxiv.org/abs/2502.18596
2025 arXiv
-
[10]
Swapnil Ghike and Shubham Gupta. 2016. FollowFeed: Linkedin’s feed made faster and smarter. https://www.linkedin.com/blog/engineering/feed/followfeed- linkedin-s-feed-made-faster-and-smarter
2016
-
[11]
Ruidong Han, Bin Yin, Shangyu Chen, He Jiang, Fei Jiang, Xiang Li, Chi Ma, Mincong Huang, Xiaoguang Li, Chunzhen Jing, Yueming Han, Menglei Zhou, Lei Yu, Chuan Liu, and Wei Lin. 2025. MTGR: Industrial-Scale Genera- tive Recommendation Framework in Meituan. InProceedings of the...
2025 arXiv
-
[12]
Lars Hertel, Neil Daftary, Fedor Borisyuk, Aman Gupta, and Rahul Mazumder
-
[13]
Dmitry Ivanov, Dennis Van Der Staay Ichen, Taylor Colin, Liu King, Fen Will, Kindi Rahal, Anirudh Sudarshan, and Shahin Safai. 2022. TorchRec: A PyTorch Domain Library for Recommendation Systems. InProceedings of the 16th ACM Conference on Recommender Systems (RecSys)
2022
-
[14]
Wang-Cheng Kang and Julian McAuley. 2018. Self-Attentive Sequential Recom- mendation. InProceedings of the IEEE International Conference on Data Mining (ICDM). https://arxiv.org/abs/1808.09781
2018 arXiv
-
[15]
Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed H. Chi. 2018. Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture- of-Experts. InProceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD)
2018
-
[16]
Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed H Chi. 2018. Modeling task relationships in multi-task learning with multi-gate mixture-of- experts. InProceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 1930–1939
2018
-
[17]
Azzolini, et al
Maxim Naumov, Deepak Mudigere, Hao-Jun Michael Shi, Jianyu Huang, Narayanan Sundaraman, Jongsoo Park, Xiaodong Wang, Udit Gupta, Carole-Jean Wu, Alisson G. Azzolini, et al. 2019. Deep Learning Recommendation Model for Personalization and Recommendation Systems.arXiv preprint a...
2019 arXiv
-
[18]
Qiji Pei, Changhua Lv, Chao Li, Junfeng Ge, and Wenwu Ou. 2021. End-to-end User Behavior Retrieval in Click-through Rate Prediction Model. InProceedings of the 28th International Conference on Machine Learning (ICML)
2021
-
[19]
Sudarshan Srinivasa Ramanujam, Antonio Alonso, Saurabh Kataria, Siddharth Dangi, Akhilesh Gupta, Birjodh Singh Tiwana, Manas Somaiya, Luke Simon, David Byrne, Sojeong Ha, et al. 2025. Large Scale Retrieval for the LinkedIn Feed using Causal Language Models.arXiv preprint arXiv...
2025
-
[20]
Sudarshan Srinivasa Ramanujam, Akanksha Bindal, Yu Jiang, Timothy J Hazen, David Golland, Fengyu Zhang, Daqi Sun, Wanning Li, Birjodh Singh Tiwana, Siddharth Dangi, et al. 2025. LinkedIn Post Embeddings: Industrial Scale Em- bedding Generation and Usage across LinkedIn. InProc...
2025
-
[21]
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. 2024. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing568 (2024), 127063
2024
-
[22]
Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential Recommendation with Bidirectional En- coder Representations from Transformer. InProceedings of the 28th ACM Inter- national Conference on Information and Knowledge Managemen...
2019 arXiv
-
[23]
Hugo Touvron, Matthieu Cord, Alexandre Sablayrolles, Gabriel Synnaeve, and Hervé Jégou. 2021. Going deeper with image transformers. InProceedings of the IEEE/CVF international conference on computer vision. 32–42
2021
-
[24]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need.Advances in neural information processing systems30 (2017)
2017
-
[25]
Ruoxi Wang, Bin Fu, Gang Fu, and Ming Wang. 2021. DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems. InProceedings of the Web Conference (WWW)
2021
-
[26]
Ruoxi Wang, Rakesh Shivanna, Derek Cheng, Sagar Jain, Dong Lin, Lichan Hong, and Ed Chi. 2021. Dcn v2: Improved deep & cross network and practical lessons for web-scale learning to rank systems. InProceedings of the web conference 2021. 1785–1797
2021
-
[27]
Xue Xia, Pong Eksombatchai, Nikil Pancha, Dhruvil Deven Badani, Po-Wei Wang, Neng Gu, Saurabh Vishwas Joshi, Nazanin Farahpour, Zhiyuan Zhang, and An- drew Zhai. 2023. Transact: Transformer-based realtime user action model for recommendation at pinterest. InProceedings of the ...
2023
-
[28]
Ruibin Xiong, Yunchang Yang, Di He, Kai Zheng, Shuxin Zheng, Chen Xing, Huishuai Zhang, Yanyan Lan, Liwei Wang, and Tieyan Liu. 2020. On layer normalization in the transformer architecture. InInternational conference on machine learning. PMLR, 10524–10533
2020
-
[29]
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 technical report.arXiv preprint arXiv:2505.09388(2025)
2025 arXiv
-
[31]
Zhe Zhao, Lichan Hong, Li Wei, Jilin Chen, Aniruddh Nath, Shawn Andrews, Aditee Kumthekar, Maheswaran Sathiamoorthy, Xinyang Yi, and Ed Chi. 2019. Recommending what video to watch next: a multitask ranking system. InPro- ceedings of the 13th ACM conference on recommender syste...
2019
-
[32]
Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. InProceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. ...
2018
-
[2016]
InProceedings of the 1st Workshop on Deep Learning for Recommender Systems (DLRS)
Wide & Deep Learning for Recommender Systems. InProceedings of the 1st Workshop on Deep Learning for Recommender Systems (DLRS)
-
[2024]
Efficient user history modeling with amortized inference for deep learning recommendation models.arXiv preprint arXiv:2412.06924(2024)
2024 arXiv
Reviewed August 2, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.