Pith. sign in

REVIEW 5 major objections 6 minor 46 references

The paper argues that user representation for recommendations should be a managed fleet of heterogeneous specialist embedding models, each matched to a distinct signal family, and that with redundancy control and logging-free evaluation thi

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-07-31 23:15 UTC pith:7GRFZS5G

load-bearing objection Fleet-of-specialists is a real system, but the paper never proves the fleet beats a single generalist — referee it, but demand the missing comparison. the 5 major comments →

arxiv 2607.24015 v1 pith:7GRFZS5G submitted 2026-07-27 cs.IR

Mosaic: A Fleet of User Embedding Specialists for Recommendation at Meta

John Zhiyuan Zheng , Xian Sun , Xiangyang Mou , Yujunrong Ma , Christina You , Michael Jiayuan He , Hrishikesh Paranjape , Aakarsha Agarwal
show 1 more author
Hong Li
This is my paper
classification cs.IR
keywords user embeddingsrecommendation systemsmulti-task learningembedding fleetredundancy lossevaluation methodologyindustrial recommender systemssequential models
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Most industrial user-representation systems build one large generalist model or a shared backbone. This paper takes the opposite route: it manages user representation as a fleet of four architecturally diverse specialists, each focused on one facet of user behavior. To keep the fleet additive rather than redundant, it introduces MRM composite labels and a cosine redundancy loss. It also proposes a logging-free evaluation framework so new embeddings can be judged in hours instead of days. If true, the result is a reusable embedding library whose whole is stronger than any single model, with clear gains on downstream ranking and engagement metrics.

Core claim

Mosaic claims that splitting user representation across a heterogeneous fleet — memorization-driven hash-table models, dense-heavy counter-interaction models, sequential HSTU-based models, and CoTrain models that receive end-to-end downstream gradients — produces embeddings that consistently improve downstream rankers. The paper further claims two mechanisms make the fleet work: MRM constructs composite multi-task labels (e.g., the Cartesian product of like and comment) to supervise joint user intent, and CRL adds an auxiliary loss that pushes each new embedding toward subspaces orthogonal to existing ones. Combined with a hybrid CPU/GPU serving stack and CoEval logging-free evaluation, the

What carries the argument

The fleet decomposition itself is the central object: four model families are each matched to a different inductive bias (sparse memorization, dense counter interactions, temporal sequences, downstream-task alignment). Two mechanisms carry the argument: MRM (Multi-task Relations Mining) builds composite cross-task labels from correlated engagement tasks to expose joint intent structure; CRL (Cosine Redundancy Loss) penalizes cosine similarity between a new embedding and all existing fleet embeddings, maximizing marginal information. CoEval pairs a frozen user tower with a copy of the downstream ranker to measure ΔNE without feature logging.

Load-bearing premise

The load-bearing premise is an unreported prototype result in Section 3.2 stating that a fleet of smaller specialists produced stronger embedding quality than a large generalist model with matched data and compute; no setup, metrics, or numbers are provided, and if a matched-compute generalist actually matches or beats the fleet, the core architectural bet has no empirical foundation.

What would settle it

Reproduce the matched-compute comparison with equal total parameters and training data: train a single generalist whose total capacity equals the sum of the four specialists, and compare downstream NE on the same surfaces. If the generalist matches or beats the fleet's NE, the central claim fails. Additionally, ablate CRL by setting λ=0 and rerunning; if downstream NE does not degrade, the redundancy-loss mechanism is not load-bearing. Finally, for a new specialist, compute both CoEval ΔNE and full feature-logging ΔNE; if the ranking of candidates diverges, the logging-free evaluation framewor

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Downstream rankers can consume compact upstream embeddings without enlarging their own serving graph, amortizing heavy user-side modeling across many consumers.
  • Adding new specialists improves downstream NE only if their marginal information is protected; the paper shows the fleet's contribution breakdown with dense-heavy specialists contributing the largest share (−0.22% NE on one surface).
  • CoEval compresses the embedding evaluation cycle from days to under half the time (3–5× iteration velocity), with alignment to full feature-logging evaluations.
  • Sequential specialist scaling shows longer sequences and larger expert/embedding dimensions improve NE at measurable QPS cost, giving a tuning trade-off curve.
  • The design is reusable across surfaces: offline gains appear consistently on all six surfaces tested, and online A/B tests show topline lifts of +0.10% to +0.28%.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the fleet advantage is real, the binding constraint on adding specialists shifts from model capacity to redundancy management; the marginal value per new compute dollar could be tracked and optimized via the CRL coefficient λ.
  • The CoEval idea could generalize beyond user embeddings to any frozen upstream feature module, potentially eliminating feature-logging pipelines for evaluation across the industry.
  • A matched-compute comparison between a single generalist and the full fleet is the missing decisive experiment; the paper's prototype claim is currently anecdotal, and until it is reported, the architectural bet rests on indirect evidence.
  • The paper mentions integrity and cold-start tasks as beneficiaries, but reports no results on them; testing fleet embeddings on cold-start cohorts and safety tasks is a natural next step.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. Mosaic is an industrial user-embedding platform that replaces a single generalist user model with a 'fleet' of four specialist families — memorization-driven, dense-heavy, sequential, and CoTrain — each trained on different signal types and inductive biases. The paper contributes MRM (multi-task composite labels) and CRL (cosine redundancy loss) to reduce overlap among embeddings, a hybrid CPU/GPU online/offline serving stack, and two logging-free evaluation methods (CoEval and User Tower Zero-Out). The reported results show offline NE improvements on several Meta surfaces, feature-importance rankings, sequential-model scaling ablations, and online A/B lifts. The central claim is that a fleet of specialists outperforms a matched single generalist model and that the redundancy-control and evaluation components make the fleet sustainable at scale.

Significance. If the fleet-vs-generalist comparison were properly substantiated, this would be an important industrial contribution: it is a concrete counterpoint to the dominant single-backbone/foundation-model trend and it provides engineering mechanisms (MRM, CRL, CoEval, hybrid serving) that other large-scale recommender systems could reuse. The paper's strengths are its detailed system description, realistic treatment of serving constraints (Table 3), and the breadth of the deployment evidence across six surfaces and multiple specialist families (Tables 4 and 5). The sequential scaling study (Table 6) is a useful operational trade-off analysis. However, the paper's headline architectural bet is currently supported only by an unreported prototype result, and the statistical claims are not backed by confidence intervals or significance tests.

major comments (5)
  1. [§3.2 and §1 (Contributions)] The load-bearing claim that a fleet of smaller specialists beats a large generalist is supported only by the sentence: 'In a prototype comparison with matched data and compute, a fleet of smaller specialists produced stronger embedding quality than a large generalist model.' No setup, metrics, data, compute budget, or results are given. All offline and online experiments in §5 compare the full Mosaic fleet against a baseline without Mosaic embeddings; they do not test the fleet against a matched-compute single generalist. If a single generalist trained on the same cross-surface data and compute budget matches or outperforms the fleet, the paper's primary design principle is unjustified. Please either report the prototype experiment in full or substantially weaken the contribution claims.
  2. [§5.2, Table 4; §5.3, Table 7] The abstract, §5.2, and §5.3 repeatedly state that improvements are 'statistically significant' and 'consistent,' but no confidence intervals, standard errors, p-values, or numbers of A/B runs are provided. Many offline deltas are very small (e.g., Surface 2 Task A EvalΔNE = -0.03%), and without uncertainty estimates it is impossible to verify significance. At minimum, report confidence intervals or significance tests for each ΔNE and each online topline lift; if significance was established through internal Meta tooling, give the relevant details or state clearly that the intervals are unavailable.
  3. [§4.1 (MRM and CRL)] MRM and CRL are presented as central techniques for maximizing marginal information, but no ablation isolates their contribution. CRL directly enforces cosine orthogonality to existing embeddings, so the statement that a new specialist 'adds unique signal' is partly true by construction; the open question is whether this orthogonality translates into downstream NE gains beyond simply adding more features. MRM's composite labels are described as 'one of our most effective techniques,' yet no experiment compares training with and without MRM labels, nor with and without CRL. Please add controlled ablations, or clearly label these as design choices without causal claims.
  4. [§4.2 (CoEval and User Tower Zero-Out)] CoEval and User Tower Zero-Out are claimed as key contributions and the paper states that 'validation experiments against the full feature logging pipeline show that the CoEval produces accurate evaluations aligned with downstream NE,' but no such validation results are reported. The 3–5× iteration-velocity improvement appears in the introduction and conclusion, while §4.2 says only 'less than half of its original time' (i.e., >2×), which is inconsistent. Please provide the validation data (e.g., correlation or error of CoEval ΔNE vs. logged-feature ΔNE) and reconcile the velocity numbers.
  5. [§5.2, Table 5] The specialist-family ablation reports only aggregated EvalΔNE values with no standard errors and no description of how the aggregation across embeddings is computed. Since the number of embeddings per family differs widely (8 vs. 20 vs. 6 vs. 3), the per-embedding contribution is not recoverable from the table. Please clarify whether the ΔNE is cumulative, the order in which families are added, and how variance is handled.
minor comments (6)
  1. [§5.2, Figure 3] FI percentiles are shown without any confidence intervals or run-to-run variance; 'consistently rank in the top 2%' would be stronger with error bars or a stability statement.
  2. [§5.2, Table 6] QPS deltas are reported relative to an implicit baseline (512 seq length), but no absolute QPS or latency values are given; please state the baseline configuration explicitly.
  3. [§3.2, Table 2] The qualitative tradeoff table is useful as design rationale, but it should be labeled as such rather than as an empirical result, since no direct comparison is reported.
  4. [§4.2.2] User Tower Zero-Out is described but no result using it is presented. If it was used in the paper's evaluations, say where; otherwise, present it as a proposed method with a validation placeholder.
  5. [Abstract/Introduction/Conclusion] The '3–5×' iteration-velocity claim appears only in the abstract/contribution list and conclusion, while the body states a 2× improvement; please reconcile these numbers or cite the supporting section.
  6. [Throughout] There are minor typographical issues, e.g., 'Mosaicprovides' in the conclusion and inconsistent italics for 'Mosaic' in a few places. A careful copy edit is recommended.

Circularity Check

0 steps flagged

No significant circularity: downstream NE is an external metric; the unreported fleet-vs-generalist prototype is a support gap, not a circular reduction.

full rationale

Mosaic's headline result is measured as downstream ranker NE and online A/B lifts, which are external to the upstream training objectives; no equation in the paper defines these metrics in terms of the Mosaic training loss. MRM constructs composite labels from existing task labels, but uses them as new supervision signals rather than as fitted targets that are then reported as predictions. CRL (Eq. 2) directly enforces cosine orthogonality to existing embeddings, but the paper's central evaluation is downstream NE, so any claim that orthogonality 'maximizes incremental information' is an interpretation that downstream experiments could falsify; it is not a reduction. CoEval's Eq. (3) defines ΔNE as the embedding's contribution by construction, yet the paper presents CoEval as a validation method checked against the feature-logging pipeline, not as a derivation of the headline gains. Self-citations (e.g., HSTU [38]) are reused architectures and are not load-bearing proofs of Mosaic's claims. The real weakness is Section 3.2's unreported prototype comparison ('a fleet of smaller specialists produced stronger embedding quality than a large generalist model') with no setup or numbers; that is an omitted-evidence gap, not circularity, because the paper does not attempt to derive that conclusion from its own definitions. Nothing in the manuscript reduces a prediction to its inputs by construction.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The central empirical claims rely on modeling and evaluation assumptions described above; no free-parameter values are reported. The paper introduces named systems and techniques (Mosaic, MRM, CRL, CoEval, Zero-Out) but no new physical or mathematical entities with independent falsifiable handles outside the paper.

free parameters (5)
  • CRL weight lambda and warm-up schedule = not reported
    Section 4.1.2 defines the total objective as L = L_main + lambda * L_red and states lambda is tuned with a warm-up schedule. No value or sensitivity analysis is given.
  • MRM correlation threshold tau = not reported
    Section 4.1.1 clusters tasks when Spearman rho >= tau; tau is a hand-set threshold with no reported value or sensitivity study.
  • Time-spent bucketization calibration = not reported
    Section 4.1.1 bucketizes continuous labels 'calibrated to balance the per-bucket distribution'; the number and width of buckets are fitted preprocessing choices.
  • Top-K FI dense features for memorization specialist = not reported
    Section 3.3.1 says the memorization specialist consumes 'a small set of top FI dense features'; K is unspecified.
  • Stochastic-length sequence truncation distribution = serving-time distribution (not specified)
    Section 3.3.3 trains with lengths sampled from the serving-time distribution; the exact distribution is not defined, and this choice materially affects the sequential specialist.
axioms (5)
  • domain assumption Downstream ranker NE is the authoritative offline measure of embedding quality.
    Sections 3.1 and 5.1 treat downstream NE as the primary offline metric. The authors note upstream NE can diverge from downstream impact, but take downstream NE itself as faithful to user engagement without public validation.
  • domain assumption CoEval's frozen-tower evaluation reproduces the results of the full feature-logging pipeline.
    Section 4.2.1 claims 'Validation experiments against the full feature logging pipeline show that the CoEval produces accurate evaluations aligned with downstream NE,' but no numbers, baselines, or error ranges are reported.
  • ad hoc to paper Joint/cross-task composite labels encode user intents not captured by marginal task labels.
    Section 4.1.1 asserts that the joint label distribution carries information beyond per-task marginals and uses this to construct MRM supervision. No experiment isolates this effect.
  • ad hoc to paper Cosine orthogonality to existing embeddings increases incremental user knowledge for downstream tasks.
    Section 4.1.2 minimizes average cosine similarity between new and existing embeddings, equating orthogonality with new information. No ablation isolates CRL's contribution to downstream NE.
  • domain assumption A user tower trained with a training-only architecture can be served standalone without quality loss.
    Section 3.1 states that only the user tower is deployed at inference time and the training-only architecture is discarded, but no evaluation of standalone degradation is reported.

pith-pipeline@v1.3.0-alltime-deepseek · 14597 in / 15772 out tokens · 152251 ms · 2026-07-31T23:15:21.748683+00:00 · methodology

0 comments
read the original abstract

User representation is one of the highest-leverage modeling problems in industrial recommendation systems: a single advancement in how users are encoded can propagate across retrieval, ranking, and integrity tasks at platform scale. Prior industrial user representation work builds either a single user model that emits one or more embedding vectors or a shared backbone with task-specific adaptation. In this paper, we present Mosaic, a foundational user modeling platform that employs a fleet of specialists to learn user embeddings. The fleet comprises four architecturally diverse model families - memorization-driven, dense-heavy, sequential-based, and CoTrain models - each focusing on a distinct facet of user behavior. We developed MRM (Multi-task Relations Mining) and CRL (Cosine Redundancy Loss) techniques to maximize the marginal information contribution of each new specialist. We also introduce CoEval and User Tower Zero-Out, new logging-free embedding evaluation framework that improves development velocity while preserving downstream-aligned accuracy. Our hybrid CPU/GPU, online-and-offline serving stack allows each specialist to choose the adequate serving strategy to meet the freshness, latency, and computational requirements. Mosaic delivers consistent and significant offline NE improvements in addition to online gains.

Figures

Figures reproduced from arXiv: 2607.24015 by Aakarsha Agarwal, Christina You, Hong Li, Hrishikesh Paranjape, John Zhiyuan Zheng, Michael Jiayuan He, Xiangyang Mou, Xian Sun, Yujunrong Ma.

Figure 1
Figure 1. Figure 1: High-level Mosaic system overview. 3.3 Specialist Architectures The fleet comprises four specialist families, each matched to a dis￾tinct objective: memorization-driven (sparse features), dense-heavy (aggregated counters), sequential (action histories), and CoTrain (downstream engagement) [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Model architecture of the Mosaic fleet, comprising a training-only architecture (left) and four user-tower specialist families (right). For simplicity, we draw all embeddings pointing to one training-only block. In practice, each Mosaic specialist has its own distinct training-only architecture. to the lengths it will see in production rather than only the max￾imum training length. Second, timestamp incons… view at source ↗
Figure 3
Figure 3. Figure 3: The FI breakdown of Mosaic among specialist cate￾gories. Specialist family ablation. We also conducted an ablation study to break down the NE contribution across specialist families. Ta￾ble 5 illustrates the breakdown on Surface 1: dense heavy special￾ists, with ∼20 embeddings deployed, contribute the largest share at −0.22% NE, followed by CoTrain (−0.15%) and HSTU-CInt sequen￾tial (−0.12%) models. Featur… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

46 extracted references · 1 canonical work pages

  1. [1]

    Adrien Bardes, Jean Ponce, and Yann LeCun. 2022. VICReg: Variance-Invariance- Covariance Regularization for Self-Supervised Learning. InProc. ICLR. doi:10. 48550/arXiv.2105.04906

    Pith/arXiv arXiv doi:10.48550/arxiv.2105.04906 2022

  2. [2]

    Yukuo Cen, Jianwei Zhang, Xu Zou, Chang Zhou, Hongxia Yang, and Jie Tang

  3. [5]

    Jianxin Chang, Chenbin Zhang, Yiqun Hui, Dewei Leng, Yanan Niu, Yang Song, and Kun Gai. 2023. PEPNet: Parameter and Embedding Personalized Network for Infusing with Personalized Prior Information. InProc. KDD. doi:10.1145/3580305. 3599884

    doi:10.1145/3580305 2023

  4. [6]

    Xiangyi Chen, Kousik Rajesh, Matthew Lawhon, Zelun Wang, Hanyu Li, Haomiao Li, Saurabh Vishwas Joshi, Pong Eksombatchai, Jaewon Yang, Yi-Ping Hsu, Jiajing Xu, and Charles Rosenberg. 2025. PinFM: Foundation Model for User Activity Sequences at a Billion-scale Visual Discovery Platform. InProc. RecSys. doi:10. 1145/3705328.3748050

    arXiv 2025

  5. [7]

    Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, Rohan Anil, Zakaria Haque, Lichan Hong, Vihan Jain, Xiaobing Liu, and Hemal Shah

  6. [8]

    Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep Neural Networks for YouTube Recommendations. InProc. RecSys. 191–198. doi:10.1145/2959100. 2959190

    doi:10.1145/2959100 2016

  7. [9]

    Jiaxin Deng, Shiyao Wang, Kuo Cai, Lejian Ren, Qigen Hu, Weifeng Ding, Qiang Luo, and Guorui Zhou. 2025. OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignment.arXiv preprint arXiv:2502.18965(2025). doi:10.48550/arXiv.2502.18965

    Pith/arXiv arXiv doi:10.48550/arxiv.2502.18965 2025

  8. [10]

    Zhibo Fan, Hongtao Lin, Haoyu Chen, Bowen Deng, Hedi Xia, Yuke Yan, and James Li. 2025. Synergizing Implicit and Explicit User Interests: A Multi- Embedding Retrieval Framework at Pinterest. InProc. KDD. doi:10.1145/3711896. 3737265

    doi:10.1145/3711896 2025

  9. [11]

    Lin Guan, Jia-Qi Yang, Zhishan Zhao, Beichuan Zhang, Bo Sun, Xuanyuan Luo, Jinan Ni, Xiaowen Li, Yuhang Qi, Zhifang Fan, Hangyu Wang, Qiwei Chen, Yi Cheng, Feng Zhang, and Xiao Yang. 2026. Make It Long, Keep It Fast: End-to-End 10k-Sequence Modeling at Billion Scale on Douyin Recommendation. InProc. WWW. doi:10.1145/3774904.3792811

    arXiv 2026

  10. [12]

    Jie Hu, Li Shen, Samuel Albanie, Gang Sun, and Enhua Wu. 2020. Squeeze- and-Excitation Networks.IEEE Transactions on Pattern Analysis and Machine Intelligence42, 8 (2020), 2011–2023. doi:10.1109/TPAMI.2019.2913372

    arXiv 2020

  11. [13]

    Wang-Cheng Kang and Julian McAuley. 2018. Self-Attentive Sequential Recom- mendation. InProc. ICDM. 197–206. doi:10.1109/ICDM.2018.00035

    arXiv 2018

  12. [14]

    Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix Factorization Tech- niques for Recommender Systems.Computer42, 8 (2009), 30–37. doi:10.1109/ MC.2009.263

    2009

  13. [15]

    Chao Li, Zhiyuan Liu, Mengmeng Wu, Yuchi Xu, Huan Zhao, Pipei Huang, Guoliang Kang, Qiwei Chen, Wei Li, and Dik Lun Lee. 2019. Multi-Interest Network with Dynamic Routing for Recommendation at Tmall. InProc. CIKM. 2615–2623. doi:10.1145/3357384.3357814

    arXiv 2019

  14. [16]

    Dai Li, Kevin Course, Wei Li, Hongwei Li, Jie Hua, Yiqi Chen, Zhao Zhu, Rui Jian, Xuan Cao, Bi Xue, Yu Shi, Jing Qian, Kai Ren, Matt Ma, Qunshu Zhang, and Rui Li. 2025. Realizing Scaling Laws in Recommender Systems: A Foundation-Expert Paradigm for Hyperscale Model Deployment.arXiv preprint arXiv:2508.02929 (2025). doi:10.48550/arXiv.2508.02929

    Pith/arXiv arXiv doi:10.48550/arxiv.2508.02929 2025

  15. [17]

    Xiangru Lian, Binhang Yuan, Xuefeng Zhu, Yulong Wang, Yongjun He, Honghuan Wu, Lei Sun, Haodong Lyu, Chengjun Liu, Xing Dong, Yiqiao Liao, Mingnan Luo, Congfei Zhang, Jingru Xie, Haonan Li, Lei Chen, Renjie Huang, Jianying Lin, Chengchun Shu, Xuezhong Qiu, Zhishan Liu, Dongying Kong, Lei Yuan, Hai Yu, Sen Yang, Ce Zhang, and Ji Liu. 2022. Persia: An Open,...

    arXiv 2022

  16. [18]

    Zhuoran Liu, Leqi Zou, Xuan Zou, Caihua Wang, Biao Zhang, Da Tang, Bolin Zhu, Yijie Zhu, Peng Wu, Ke Wang, and Youlong Cheng. 2022. Monolith: Real Time Recommendation System With Collisionless Embedding Table.arXiv preprint arXiv:2209.07663(2022). doi:10.48550/arXiv.2209.07663

    Pith/arXiv arXiv doi:10.48550/arxiv.2209.07663 2022

  17. [19]

    Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed H. Chi. 2018. Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of- Experts. InProc. KDD. 1930–1939. doi:10.1145/3219819.3220007

    arXiv 2018

  18. [20]

    Maxim Naumov, Dheevatsa Mudigere, Hao-Jun Michael Shi, Jianyu Huang, Narayanan Sundaraman, Jongsoo Park, Xiaodong Wang, Udit Gupta, Carole-Jean Wu, Alisson G. Azzolini, Dmytro Dzhulgakov, Andrey Mallevich, Ilia Cherni- avskii, Yinghai Lu, Raghuraman Krishnamoorthi, Ansha Yu, Volodymyr Kon- dratenko, Stephanie Pereira, Xianjie Chen, Wenlin Chen, Vijay Rao,...

    Pith/arXiv arXiv doi:10.48550/arxiv.1906.00091 2019

  19. [21]

    Aditya Pal, Chantat Eksombatchai, Yitong Zhou, Bo Zhao, Charles Rosenberg, and Jure Leskovec. 2020. PinnerSage: Multi-Modal User Embedding Framework for Recommendations at Pinterest. InProc. KDD. 2311–2320. doi:10.1145/3394486. 3403280

    doi:10.1145/3394486 2020

  20. [22]

    Nikil Pancha, Andrew Zhai, Jure Leskovec, and Charles Rosenberg. 2022. Pinner- Former: Sequence Modeling for User Representation at Pinterest. InProc. KDD. 3702–3712. doi:10.1145/3534678.3539156

    arXiv 2022

  21. [23]

    Qi Pi, Guorui Zhou, Yujing Zhang, Zhe Wang, Lejian Ren, Ying Fan, Xiaoqiang Zhu, and Kun Gai. 2020. Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction. InProc. CIKM. 2685–2692. doi:10.1145/3340531.3412744

    arXiv 2020

  22. [24]

    Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Q

    Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan H. Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Q. Tran, Jonah Samost, Maciej Kula, Ed H. Chi, and Maheswaran Sathiamoorthy. 2023. Recommender Systems with Generative Retrieval. InProc. NeurIPS. doi:10.48550/arXiv.2305.05065

    Pith/arXiv arXiv doi:10.48550/arxiv.2305.05065 2023

  23. [25]

    Noam Shazeer, Zhenzhong Lan, Youlong Cheng, Nan Ding, and Le Hou. 2020. Talking-Heads Attention.arXiv preprint arXiv:2003.02436(2020). doi:10.48550/ arXiv.2003.02436

    Pith/arXiv arXiv doi:10.48550/arxiv.2003.02436 2020

  24. [26]

    Xiang-Rong Sheng, Liqin Zhao, Guorui Zhou, Xinyao Ding, Binding Dai, Qiang Luo, Siran Yang, Jingshan Lv, Chi Zhang, Hongbo Deng, and Xiaoqiang Zhu. 2021. One Model to Serve All: Star Topology Adaptive Recommender for Multi-Domain CTR Prediction. InProc. CIKM. 4104–4113. doi:10.1145/3459637.3481941

    arXiv 2021

  25. [27]

    Zihua Si, Lin Guan, ZhongXiang Sun, Xiaoxue Zang, Jing Lu, Yiqun Hui, Xingchao Cao, Zeyu Yang, Yichen Zheng, Dewei Leng, Kai Zheng, Chenbin Zhang, Yanan Niu, Yang Song, and Kun Gai. 2024. TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou. InProc. CIKM. doi:10.1145/3627673.3680030

    arXiv 2024

  26. [28]

    Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang

  27. [29]

    Hongyan Tang, Junning Liu, Ming Zhao, and Xudong Gong. 2020. Progres- sive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations. InProceedings of the 14th ACM Conference on Recommender Systems (RecSys). 269–278. doi:10.1145/3383313.3412236

    arXiv 2020

  28. [30]

    Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang. 2017. Deep & Cross Network for Ad Click Predictions. InProc. ADKDD Workshop at KDD. 1–7. doi:10.1145/ 3124749.3124754

    arXiv 2017

  29. [31]

    Xu Wang, Jiangxia Cao, Zhiyi Fu, Kun Gai, and Guorui Zhou. 2025. HoME: Hierarchy of Multi-Gate Experts for Multi-Task Learning at Kuaishou. InProc. KDD. doi:10.1145/3690624.3709416

    arXiv 2025

  30. [32]

    Xue Xia, Pong Eksombatchai, Nikil Pancha, Dhruvil Deven Badani, Po-Wei Wang, Neng Gu, Saurabh Vishwas Joshi, Nazanin Farahpour, Zhiyuan Zhang, and An- drew Zhai. 2023. TransAct: Transformer-based Realtime User Action Model for Recommendation at Pinterest. InProc. KDD. doi:10.1145/3580305.3599918

    arXiv 2023

  31. [33]

    Xue Xia, Saurabh Vishwas Joshi, Kousik Rajesh, Kangnan Li, Yangyi Lu, Nikil Pancha, Dhruvil Deven Badani, Jiajing Xu, and Pong Eksombatchai. 2025. Trans- Act V2: Lifelong User Action Sequence Modeling on Pinterest Recommendation. InProc. CIKM. doi:10.1145/3746252.3761433

    arXiv 2025

  32. [34]

    Xu Xie, Fei Sun, Zhaoyang Liu, Shiwen Wu, Jinyang Gao, Jiandong Zhang, Bolin Ding, and Bin Cui. 2022. Contrastive Learning for Sequential Recommendation. RecSys ’26, September 27-October 02, 2026, Minneapolis, MN, USA Zhiyuan Zheng et al. InProc. ICDE. 1259–1273. doi:10.1109/ICDE53745.2022.00099

    arXiv 2022

  33. [35]

    Bencheng Yan, Shilei Liu, Zhiyuan Zeng, Zihao Wang, Yizhen Zhang, Yujin Yuan, Langming Liu, Jiaqi Liu, Di Wang, Wenbo Su, Pengjie Wang, Jian Xu, and Bo Zheng. 2026. Unlocking Scaling Law in Industrial Recommendation Systems with a Three-step Paradigm based Large User Model. InProceedings of the 19th ACM International Conference on Web Search and Data Mini...

    arXiv 2026

  34. [36]

    Xinyang Yi, Ji Yang, Lichan Hong, Derek Zhiyuan Cheng, Lukasz Heldt, Aditee Kumthekar, Zhe Zhao, Li Wei, and Ed H. Chi. 2019. Sampling-Bias-Corrected Neural Modeling for Large Corpus Item Recommendations. InProc. RecSys. 269–

    2019

  35. [37]

    Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. 2021. Bar- low Twins: Self-Supervised Learning via Redundancy Reduction. InProc. ICML, Vol. 139. 12310–12320. doi:10.48550/arXiv.2103.03230

    Pith/arXiv arXiv doi:10.48550/arxiv.2103.03230 2021

  36. [38]

    Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhaojie Gong, Fangda Gu, Jiayuan He, Yinghai Lu, and Yu Shi. 2024. Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations. InProc. ICML, Vol. 235. doi:10.48550/arXiv.2402.17152

    Pith/arXiv arXiv doi:10.48550/arxiv.2402.17152 2024

  37. [39]

    Buyun Zhang, Liang Luo, Yuxin Chen, Jade Nie, Xi Liu, Daifeng Guo, Yanli Zhao, Shen Li, Yuchen Hao, Yantao Yao, Guna Lakshminarayanan, Ellie Dingqiao Wen, Jongsoo Park, Maxim Naumov, and Wenlin Chen. 2024. Wukong: Towards a Scaling Law for Large-Scale Recommendation. InProc. ICML, Vol. 235. doi:10. 48550/arXiv.2403.02545

    Pith/arXiv arXiv doi:10.48550/arxiv.2403.02545 2024

  38. [40]

    Guorui Zhou, Hengrui Hu, Hongtao Cheng, Huanjie Wang, Jiaxin Deng, Jinghao Zhang, Kuo Cai, Lejian Ren, Lu Ren, Liao Yu, Pengfei Zheng, Qiang Luo, Qianqian Wang, Qigen Hu, Rui Huang, Ruiming Tang, Shiyao Wang, Shujie Yang, Tao Wu, Wuchao Li, Xinchen Luo, Xingmei Wang, Yi Su, Yunfan Wu, Zexuan Cheng, Zhanyu Liu, Zixing Zhang, Bin Zhang, Boxuan Wang, Chaoyi ...

    Pith/arXiv arXiv doi:10.48550/arxiv.2508.20900 2025

  39. [41]

    Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Deep Interest Evolution Network for Click-Through Rate Prediction. InProc. AAAI, Vol. 33. 5941–5948. doi:10.1609/aaai.v33i01.33015941

    doi:10.1609/aaai.v33i01.33015941 2019

  40. [42]

    Guorui Zhou, Xiaoqiang Zhu, Chengru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep Interest Network for Click-Through Rate Prediction. InProc. KDD. 1059–1068. doi:10.1145/3219819. 3219823

    doi:10.1145/3219819 2018

  41. [43]

    Kun Zhou, Hui Wang, Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen. 2020. S3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization. InProc. CIKM. 1893–1902. doi:10.1145/3340531.3411954

    arXiv 2020

  42. [44]

    Jie Zhu, Zhifang Fan, Xiaoxie Zhu, Yuchen Jiang, Hangyu Wang, Xintian Han, Haoran Ding, Xinmin Wang, Wenlin Zhao, Zhen Gong, Huizhi Yang, Zheng Chai, Zhe Chen, Yuchao Zheng, Qiwei Chen, Feng Zhang, Xun Zhou, Peng Xu, Xiao Yang, Di Wu, and Zuotao Liu. 2025. RankMixer: Scaling Up Ranking Models in Industrial Recommenders. InProc. CIKM. doi:10.1145/3746252.3761507

    arXiv 2025

  43. [277]

    doi:10.1145/3298689.3346996

    arXiv

  44. [2016]

    Wide & Deep Learning for Recommender Systems. InProc. DLRS Workshop at RecSys. 7–10. doi:10.1145/2988450.2988454

    arXiv

  45. [2019]

    BERT4Rec: Sequential Recommendation with Bidirectional Encoder Rep- resentations from Transformer. InProc. CIKM. 1441–1450. doi:10.1145/3357384. 3357895

    work page doi:10.1145/3357384

  46. [2020]

    Controllable Multi-Interest Framework for Recommendation. InProc. KDD. 2942–2951. doi:10.1145/3394486.3403344

    arXiv