Pith. sign in

REVIEW 3 major objections 7 minor 39 references

Self-supervised sequential user models work on the open web’s fragmented, short histories and lift live RTB metrics.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A self-supervised Transformer on short open-web browsing sequences improves production CTR and win-rate models and delivers +2.13% live CTR under RTB latency and privacy constraints.

T0 review reviewed 2026-07-31 challenge →

load-bearing objection Solid industrial deployment paper: sequential UFMs work under open-web RTB constraints, with a live A/B that actually anchors the claim. the 3 major comments →

arxiv 2607.28019 v1 pith:APU4JYB5 submitted 2026-07-30 cs.LG

Building a User Foundation Model for the Open Web

Solal Vernier , Ivan Can Arisoy , Merwan Barlier , Bla\v{z} \v{S}krlj This is my paper
classification cs.LG
keywords User Foundation ModelsSequential ModelingSelf-Supervised LearningOpen-Web RTBLLM as OptimizerCTR PredictionReal-Time BiddingRecommender Systems
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most industrial user foundation models assume logged-in users with long, stable histories. Open-web real-time bidding does not: identity is fragmented, many requests have no history, and what exists is short and disjointed, so production systems usually collapse history into counters and recency buckets. This paper shows that a Transformer encoder pre-trained with masked language modeling plus a sequence-level contrastive loss on browsing triplets, then fine-tuned on clicks, still produces a useful user representation under those constraints. The same embedding improves the production CTR ranker, a second ranker architecture, and a bid win-rate model, and a seven-day live A/B test records higher CTR and lower effective cost per click. The practical claim is that sequential self-supervision is a viable, ranker-agnostic improvement axis even when user identity is non-persistent and history is sparse.

Core claim

A user foundation model pre-trained on open-web browsing sequences with masked language modeling and a temporally disjoint sequence-level contrastive objective, then fine-tuned on click prediction, yields embeddings that improve multiple production RTB models—including in the history-absent regime—and translate offline RIG gains into live CTR and eCPC lifts under sub-millisecond serving constraints.

What carries the argument

The encoder’s two-view pre-training: each user history is cut at a random point into past and future subsequences of interaction triplets (publisher, advertiser, event type), aligned by NT-Xent on pooled [CLS] embeddings while MLM reconstructs masked tokens inside each view; the fine-tuned pooled embedding is then fed through a small adapter into downstream rankers.

Load-bearing premise

That the single encoder configuration picked by maximizing noisy proxy scores over about a hundred search trials, with one training run per reported cell, is not mainly a winner’s-curse artifact of that selection process.

What would settle it

A multi-seed retrain of the selected encoder versus the pre-NAS baseline, plus a matched-budget search that proposes from the same lifter catalog without the LLM, that fails to recover comparable offline RIG and live CTR/eCPC lifts on held-out production traffic.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Sequential user pre-training becomes a decoupled improvement axis for open-web RTB rankers that already consume hand-crafted features.
  • The same click-fine-tuned embedding can be dropped into structurally different tasks such as bid win-rate without task-specific fine-tuning.
  • History-absent and opt-out traffic can still benefit by encoding only the current request triplet.
  • Reviewable code-level training edits searched with an LLM-in-the-loop can produce a deployable encoder under industrial latency budgets.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If short, privacy-bounded sessions already carry transferable sequential signal, other identity-fragmented domains (cookieless ads, cross-device attribution) may gain from the same temporally disjoint contrastive recipe.
  • Broadening the token vocabulary beyond publisher–advertiser–event to price or creative context is a direct next ablation the results motivate but do not run.
  • Serving the encoder in a separate DMP path suggests foundation-style user models can be productized as shared features across many bid-path models rather than baked into one ranker.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper presents a user foundation model (UFM) for open-web RTB, where identity is fragmented, histories are short/bounded, and a large share of traffic has no history. A bidirectional Transformer is pre-trained with masked language modeling on interaction triplets (publisher, advertiser, event type) plus a sequence-level NT-Xent loss over temporally disjoint past/future partitions of each user history, then fine-tuned on impression-level click labels. The pooled [CLS] embedding is integrated via a small adapter into production rankers. An LLM-in-the-loop search over a curated catalog of literature-anchored code edits (“lifters”) produces the deployed encoder. Offline, the NAS-lifted representation yields +1.354% RIG on the production GDCN CTR ranker, +0.894% on DCN2, and +1.197% on a bid win-rate model (click-only fine-tune), including +0.99% RIG in the length-1 (history-absent) bucket. A 7-day full-traffic A/B test reports +2.13% CTR and −1.13% eCPC (80% CIs excluding zero), with analogous gains on visit metrics. The authors explicitly scope the NAS claim as an existence proof and flag single-seed/winner’s-curse caveats in §5.5.

Significance. If the result holds, the paper establishes sequential self-supervised user modeling as a viable, ranker-agnostic improvement axis under open-web RTB constraints that prior UFMs (ShopperBERT, PinnerFormer, PinFM, ExFM, etc.) do not address. The combination of (i) multi-architecture and cross-task offline lifts, (ii) an explicit history-length breakdown including the opt-out-dominated length-1 regime, (iii) a skip-pretrain ablation, and (iv) a production live A/B with CIs excluding zero is stronger evidence than is typical for industrial sequential-recommendation systems papers. The temporally disjoint contrastive construction and the serving topology (DMP micro-batch, length-1 fallback) are concrete engineering contributions. The LLM-as-optimizer workflow is secondary and correctly scoped; the primary contribution is demonstrated open-web UFM deployability under C1/C2.

major comments (3)
  1. [§5.1, Tables 3–5; §5.5] §5.1–5.2 and Tables 3–5 report offline RIG lifts from single training runs per cell (§5.5). While the 7-day A/B anchors the central viability claim, the offline numbers (including the NAS vs baseline gap of +1.065% → +1.354% and the cross-task +1.197%) remain point estimates subject to the winner’s-curse selection over ~100 end-to-end trials noted in §5.5. A multi-seed retrain of the committed UFM-NAS configuration (and of UFM-Base) on the production-aligned offline setup, with mean±std RIG, would make the offline claims load-bearing rather than confirmatory anecdotes. This is a bounded request; it does not require re-running the full NAS.
  2. [§5.1, Table 3; §3.2] The paper does not report a strong non-foundation sequential baseline under the same integration protocol (e.g., a from-scratch BERT4Rec/SASRec-style encoder fine-tuned only on clicks, or mean-pooled triplet embeddings without MLM+NT-Xent). Table 2’s skip-pretrain ablation shows pre-training matters for the proxy, but it does not isolate whether the deployed lift over GDCN comes from sequential structure per se versus the specific pre-train recipe. One controlled sequential baseline in the Table 3 protocol would substantially strengthen the claim that self-supervised pre-training—not merely adding a short-history encoder—is the operative axis.
  3. [§5.1, Table 4, Figure 3; constraints C1–C2] §3.1 and §5.1 emphasize gains in both history-absent and history-present regimes, and Figure 3 / Table 4 break RIG down by event count. The manuscript never states the traffic mass (or impression share) in the length-1 bucket versus ≥2, nor the opt-in rate. Without that mixture weight, the practical importance of the +0.99% length-1 lift versus the +1.62% history-present lift cannot be assessed, and the claim that the model “produces stable, useful output across both regimes” (C2) remains only partially quantified. Please add the empirical mixture (and, if possible, RIG contribution weighted by traffic share).
minor comments (7)
  1. [§4.2.1, Figure 2] §4 and Figure 2: the NAS trajectory is informative, but the lifter catalog size (~150), the ten “genome” categories, and the acceptance rule (proxy RIG lift then confirmatory production re-eval) would benefit from a short appendix table listing the committed lifters in order with one-line rationales, so the trajectory is fully replayable as claimed.
  2. [§3.1.4, Eq. (1)] Eq. (1): clarify whether Δt is computed per subsequence after the random cut (as the text implies) and whether padding positions receive a distinguished time value or inherit the [CLS] zero. Also state the units after log1p scaling more explicitly in the equation caption.
  3. [§5.4, Table 6] §5.4 / Table 6: 80% CIs are stated as the internal standard; for a general RecSys audience, briefly note why 80% rather than 95% and whether the same qualitative conclusion holds at 95% if available.
  4. [§3.4] §3.4: “at least two non-linear layers” for the adapter is an important empirical finding; a one-row ablation (linear vs 2-layer vs 3-layer) in an appendix would make it citable rather than parenthetical.
  5. [§2] Related work: Abacus [2] is the closest open-web/display-ads sequential SSL baseline; a slightly sharper contrast on backbone (GRU vs Transformer), contrastive design, and integration topology in one paragraph would help readers place the contribution.
  6. [§3.1.2] Minor clarity: L=49 is “16 events × 3 tokens + [CLS]”; confirm whether truncation keeps the most recent events (standard) and state this explicitly in §3.1.2.
  7. [Table 4; Abstract; §1] Typos/style: “opt-out dominated” in Table 4 is slightly ambiguous (length-1 includes cold-start opt-in as well as opt-out); consider “no prior history.” Abstract and §1 repeat the same RIG/A/B numbers almost verbatim—tighten one of them.

Circularity Check

1 steps flagged

No load-bearing circularity: empirical UFM gains are confirmed by separate production offline eval and live A/B, not forced by pre-train objectives or self-citation.

specific steps
  1. fitted input called prediction [§4.2.2 Table 2; caveat §5.5 (iii)]
    "The supervised-stage selection likewise consulted proxy scores on the test set; since its candidates shared one pre-trained backbone and differed only in fine-tuning recipe and evaluation-head configuration, the scope for selection-induced optimism is narrow. ... Table 2: NAS fine-tuning and proxy results on the test set. ... NAS-lifted ... Test RIG 0.0547"

    Stage-2 NAS used test-set proxy AUC/RIG in selection, then Table 2 reports test RIG/AUC lifts for the selected config. Those particular table numbers are partly selection-conditioned rather than fully held-out predictions. This is minor nested-evaluation leakage on a secondary NAS proxy table only; production GDCN/DCN2/win-rate offline lifts and the live A/B use separate evaluation paths and are not forced by this fit.

full rationale

This is an industrial systems/empirical paper. The pre-training losses (MLM + temporally-disjoint NT-Xent) and click fine-tuning do not algebraically determine downstream production RIG, win-rate RIG, or live CTR/eCPC; those are measured on held-out production traffic and a 7-day A/B. Citations to BERT4Rec, CL4SRec, DuoRec, GDCN, etc. are external recipe anchors, not uniqueness theorems that force the result. The only mild evaluation circularity is that supervised-stage NAS selection consulted proxy scores on the test set (§5.5) while Table 2 also reports those test metrics—ordinary selection leakage on a secondary NAS table, already scoped by the authors as point estimates / existence proof, and not load-bearing for the central open-web UFM viability claim, which rests on disjoint production offline lifts and the live A/B. No self-definitional reduction, no ansatz smuggled as theorem, no renaming of a known law.

Axiom & Free-Parameter Ledger

8 free parameters · 7 axioms · 2 invented entities

Load-bearing content is empirical and engineering-heavy. The claim rests on standard SSL/Transformer assumptions, domain facts about open-web RTB traffic, and many recipe hyperparameters selected by NAS rather than theory. No new physical entities; ‘lifters’ and the UFM encoder are engineered artifacts. Free parameters are the usual trained weights plus search-chosen scalars (mask rate, temperature, depth/width, etc.).

free parameters (8)
  • MLM mask ratio ρ = 0.30
    Set to 0.30 by empirical choice, higher than BERT’s 0.15; directly affects pre-training signal.
  • NT-Xent temperature τ = 0.04
    Contrastive sharpness parameter committed by NAS; changes representation geometry.
  • Embedding dimension / encoder width = 48 (NAS-lifted)
    NAS-committed model size (reported 48-dim embeddings in figure context) selected to maximize proxy metrics.
  • Max sequence length L = 49 tokens / 16 events
    Hard cap of 49 tokens (16 events × 3 + CLS), truncating real histories; shapes what the encoder can see.
  • Contrastive mini-batch size N = 1024 histories
    Number of histories per batch sets the negative set size for NT-Xent; NAS moved batch size (2048→1024).
  • Pre-training epochs and learning rate schedule = 6 epochs (NAS-lifted); LR swept (e.g. 0.03/0.02)
    NAS-selected training horizon (6 epochs vs baseline 10) and LR values; determine the committed checkpoint.
  • Adapter depth/width and output dimension = ≥2 nonlinear layers (exact dims production-specific)
    Integration module required ≥2 nonlinear layers empirically; architecture chosen to make the embedding usable by GDCN.
  • Time-embedding affine parameters α_d, β_d = learned
    Learned per-dimension scale/shift on log delay; part of the fitted encoder.
axioms (7)
  • domain assumption Bidirectional Transformer + MLM is a valid pretext for user-sequence representation (BERT/BERT4Rec lineage).
    Invoked throughout §3.2; not re-proved, taken from NLP/recsys prior work.
  • ad hoc to paper Temporally disjoint past/future partitions of one user history form valid contrastive positive pairs for open-web fragments.
    §3.1.1; differs from crop/mask/reorder or dropout positives in CL4SRec/DuoRec; central inductive bias for this domain.
  • domain assumption Publisher/advertiser/event-type triplets plus log-delay time embedding adequately tokenize open-web RTB behavior.
    §3.1.3–3.1.4; vocabulary and features chosen from industrial CTR structure.
  • domain assumption Pooled [CLS] embedding after click fine-tuning is a task-transferable user feature for other heads (win-rate) and rankers.
    §3.3–3.4 and §5.2; transfer is empirical, not guaranteed.
  • domain assumption Sub-ms DMP micro-batch serving and 16-event/7-day history cap preserve enough signal for production gains.
    Constraint C1 and §5.3 serving design; necessary for the deployment claim.
  • domain assumption Relative Information Gain vs a constant base-rate predictor is a meaningful offline proxy for production ranking quality.
    §4.1.1 definition used as primary selection and reporting metric.
  • standard math User-level randomization (and request-level for opt-out) yields unbiased A/B estimates of CTR/eCPC effects.
    Standard experiment design assumption in §5.4.
invented entities (2)
  • Lifter (reviewable literature-anchored code-level training-pipeline edit) no independent evidence
    purpose: Atomic search unit for LLM-in-the-loop NAS over the UFM training pipeline (~150 catalog entries).
    Named engineering abstraction in §4; not a physical entity. independent_evidence false as a scientific object—only justified by downstream proxy/production metrics in this work.
  • Teads open-web User Foundation Model (UFM-NAS encoder) no independent evidence
    purpose: Production user-sequence encoder producing [CLS] embeddings for DSP rankers under C1/C2.
    The deployed artifact; evidence is internal offline+A/B only, no public checkpoint.

reviewed 2026-07-31 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Building a User Foundation Model for the Open Web." pith.science (2026). https://pith.science/paper/APU4JYB5

@misc{pith2026260728019,
  author       = {Pith},
  title        = {Pith review of: Building a User Foundation Model for the Open Web},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/APU4JYB5}},
  note         = {Machine review of arXiv:2607.28019}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

User foundation models have demonstrated strong results in e-commerce and social recommendation, but most industrial deployments assume environments where user identity is stable and persistent. Open-web real-time bidding (RTB) operates on a structurally different data distribution: user identity is fragmented and non-persistent across browsing sessions, and the availability of browsing history depends on user privacy choices. Consequently, a significant portion of traffic carries no historical data, and available records often consist of relatively short, disjointed sessions. As a result, historical signals in this domain are typically represented as aggregated counters and recency buckets, leaving the sequential structure unexploited. To address this limitation, we present a user foundation model that applies self-supervised learning on user browsing histories and show that the learned representation improves multiple downstream production tasks, demonstrating the viability of this approach on the open web. We pre-train a Transformer encoder with masked language modeling and a sequence-level contrastive objective, then fine-tune it on the click prediction task. We optimize the encoder's pre-training pipeline with an LLM-in-the-loop search over a curated catalog of reviewable, code-level edits (lifters), instantiating the LLM-as-optimizer paradigm in an industrial setting. The same encoder representation yields +1.197% RIG on the production bid win-rate model and +1.354% RIG on the production CTR ranker; a 7-day live A/B test confirms +2.13% CTR, -1.13% eCPC (80% CI excluding zero on both metrics).

Figures

Figures reproduced from arXiv: 2607.28019 by Bla\v{z} \v{S}krlj, Ivan Can Arisoy, Merwan Barlier, Solal Vernier.

Figure 1
Figure 1. Figure 1: End-to-end pipeline. (1) Self-supervised pre-training: in the input token sequence, each triplet consists of P (publisher [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Pre-training NAS trajectory and committed lifters. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: RIG broken down by number of events in the user [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

39 extracted references · 12 linked inside Pith

  1. [1]

    James Bergstra and Yoshua Bengio. 2012. Random Search for Hyper-Parameter Optimization.Journal of Machine Learning Research13, 2 (2012), 281–305

    2012

  2. [2]

    Sullivan Castro, Artem Betlei, Thomas Di Martino, and Nadir El Manouzi. 2026. Abacus: Self-Supervised Event Counting-Aligned Distributional Pretraining for Sequential User Modeling. InProceedings of the Nineteenth ACM International Conference on Web Search and Data Mining (WSDM). doi:10.1145/3773966.3779391 arXiv:2512.16581

    arXiv 2026

  3. [3]

    Junyi Chen, Lu Chi, Bingyue Peng, and Zehuan Yuan. 2024. HLLM: Enhancing Sequential Recommendations via Hierarchical Large Language Models for Item and User Modeling. arXiv:2409.12740 [cs.IR]

    Pith/arXiv arXiv 2024

  4. [4]

    Qiwei Chen, Huan Zhao, Wei Li, Pipei Huang, and Wenwu Ou. 2019. Behavior Se- quence Transformer for E-commerce Recommendation in Alibaba. InProceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data (DLP-KDD). Article 12

    2019

  5. [5]

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A Simple Framework for Contrastive Learning of Visual Representations. In International Conference on Machine Learning (ICML). 1597–1607

    2020

  6. [6]

    Xiangyi Chen, Kousik Rajesh, Matthew Lawhon, Zelun Wang, Hanyu Li, Haomiao Li, Saurabh Vishwas Joshi, Pong Eksombatchai, Jaewon Yang, Yi-Ping Hsu, Jiajing Xu, and Charles Rosenberg. 2025. PinFM: Foundation Model for User Activity Sequences at a Billion-Scale Visual Discovery Platform. InProceedings of the 19th ACM Conference on Recommender Systems (RecSys...

    arXiv 2025

  7. [7]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT). 4171–4186

    2019

  8. [8]

    Chrisantha Fernando, Dylan Banarse, Henryk Michalewski, Simon Osindero, and Tim Rocktäschel. 2023. Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution. arXiv:2309.16797

    Pith/arXiv arXiv 2023

  9. [9]

    Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. SimCSE: Simple Contrastive Learning of Sentence Embeddings. InProceedings of the 2021 Conference on Em- pirical Methods in Natural Language Processing (EMNLP). 6894–6910

    2021

  10. [10]

    Qingyan Guo, Rui Wang, Junliang Guo, Bei Li, Kaitao Song, Xu Tan, Guoqing Liu, Jiang Bian, and Yujiu Yang. 2024. Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers. InInternational Conference on Learning Representations (ICLR). arXiv:2309.08532

    Pith/arXiv arXiv 2024

  11. [11]

    Wang-Cheng Kang and Julian McAuley. 2018. Self-Attentive Sequential Recom- mendation. InIEEE International Conference on Data Mining (ICDM). 197–206

    2018

  12. [12]

    Seyed Mehran Kazemi, Rishab Goel, Sepehr Eghbali, Janahan Ramanan, Jaspreet Sahota, Sanjay Thakur, Stella Wu, Cathal Smyth, Pascal Poupart, and Mar- cus Brubaker. 2019. Time2Vec: Learning a Vector Representation of Time. arXiv:1907.05321 [cs.LG]

    Pith/arXiv arXiv 2019

  13. [13]

    Byunghyun Kim and Jae-Gil Lee. 2024. Continuous-Time Linear Positional Embedding for Irregular Time Series Forecasting. arXiv:2409.20092 [cs.LG]

    Pith/arXiv arXiv 2024

  14. [14]

    Jiacheng Li, Yujie Wang, and Julian McAuley. 2020. Time Interval Aware Self- Attention for Sequential Recommendation. InProceedings of the Thirteenth ACM International Conference on Web Search and Data Mining (WSDM). 322–330

    2020

  15. [15]

    Liam Li and Ameet Talwalkar. 2019. Random Search and Reproducibility for Neural Architecture Search. InProceedings of the 35th Conference on Uncertainty in Artificial Intelligence (UAI). 367–377

    2019

  16. [16]

    Mingfu Liang, Xi Liu, Rong Jin, Boyang Liu, Qiuling Suo, Qinghai Zhou, Song Zhou, Laming Chen, Hua Zheng, Zhiyuan Li, et al. 2025. External Large Foun- dation Model: How to Efficiently Serve Trillions of Parameters for Online Ads Recommendation. InCompanion Proceedings of the ACM on Web Conference 2025 (WWW Industry Track). arXiv:2502.17494

    Pith/arXiv arXiv 2025

  17. [17]

    Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2024. Eureka: Human-Level Reward Design via Coding Large Language Models. InInternational Conference on Learning Representations (ICLR). arXiv:2310.12931

    Pith/arXiv arXiv 2024

  18. [18]

    Nelson, Herbie Bradley, Adam Gaier, Arash Moradi, Amy K

    Elliot Meyerson, Mark J. Nelson, Herbie Bradley, Adam Gaier, Arash Moradi, Amy K. Hoover, and Joel Lehman. 2024. Language Model Crossover: Variation through Few-Shot Prompting. InACM Transactions on Evolutionary Learning and Optimization. arXiv:2302.12170

    arXiv 2024

  19. [19]

    Alexander Novikov, Ngân V ˜u, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, et al . 2025. AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery. arXiv:2506.13131 [cs.AI] Google DeepMind technical report

    Pith/arXiv arXiv 2025

  20. [20]

    Nikil Pancha, Andrew Zhai, Jure Leskovec, and Charles Rosenberg. 2022. Pinner- Former: Sequence Modeling for User Representation at Pinterest. InProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD). 3702–3712

    2022

  21. [21]

    Ruihong Qiu, Zi Huang, Hongzhi Yin, and Zijian Wang. 2022. Contrastive Learn- ing for Representation Degeneration Problem in Sequential Recommendation. InProceedings of the Fifteenth ACM International Conference on Web Search and Data Mining (WSDM). 813–823

    2022

  22. [22]

    Pawan Kumar, Emilien Dupont, Francisco J

    Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M. Pawan Kumar, Emilien Dupont, Francisco J. R. Ruiz, Jordan S. Ellenberg, Pengming Wang, Omar Fawzi, Pushmeet Kohli, and Alhussein Fawzi

  23. [23]

    Kyuyong Shin, Hanock Kwak, Kyung-Min Kim, Minkyu Kim, Young-Jin Park, Jisu Jeong, and Seungjae Jung. 2021. One4all User Representation for Recommender Systems in E-commerce. arXiv:2106.00573 [cs.IR] ShopperBERT

    Pith/arXiv arXiv 2021

  24. [24]

    Xin Song, Zhilin Guan, Ruidong Han, et al . 2026. MTFM: A Scalable and Alignment-Free Foundation Model for Industrial Recommendation in Meituan. arXiv:2602.11235 [cs.IR]

    arXiv 2026

  25. [25]

    Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang

  26. [26]

    Aäron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation Learning with Contrastive Predictive Coding. arXiv:1807.03748 [cs.LG]

    Pith/arXiv arXiv 2018

  27. [27]

    Gomez, Łukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is All You Need. InAdvances in Neural Information Processing Systems (NeurIPS). 5998– 6008

    2017

  28. [28]

    Blaž Škrlj, Yonatan Karni, Grega Gašperšič, Blaž Mramor, Yulia Stolin, Martin Jakomin, Jasna Urbančič, Yuval Dishi, Natalia Silberstein, Ophir Friedler, and Assaf Klein. 2025. DCN 2: Interplay of Implicit Collision Weights and Explicit Cross Layers for Large-Scale Recommendation. arXiv:2506.21624 [cs.LG]

    Pith/arXiv arXiv 2025

  29. [29]

    Chenyang Wang, Weizhi Ma, Chong Chen, Min Zhang, Yiqun Liu, and Shaoping Ma. 2023. Sequential Recommendation with Multiple Contrast Signals.ACM Transactions on Information Systems (TOIS)41, 1 (2023), Article 11

    2023

  30. [30]

    Fangye Wang, Hansu Gu, Dongsheng Li, Tun Lu, Peng Zhang, and Ning Gu. 2023. Towards Deeper, Lighter and Interpretable Cross Network for CTR Prediction. InProceedings of the 32nd ACM International Conference on Information and Knowledge Management (CIKM). ACM, 2628–2637. doi:10.1145/3583780.3615089 arXiv:2311.04635

    arXiv 2023

  31. [31]

    Xu Xie, Fei Sun, Zhaoyang Liu, Shiwen Wu, Jinyang Gao, Bolin Ding, and Bin Cui. 2022. Contrastive Learning for Sequential Recommendation. InIEEE 38th International Conference on Data Engineering (ICDE). 1259–1273

    2022

  32. [32]

    Da Xu, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, and Kannan Achan

  33. [33]

    Bei Yang, Jie Gu, Ke Liu, Xiaoxiao Xu, Renjun Xu, Qinghui Sun, and Hong Liu. 2023. Empowering General-Purpose User Representation with Full-Life- Cycle Behavior Modeling. InProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD). doi:10.1145/3580305.3599331 arXiv:2110.11337

    arXiv 2023

  34. [34]

    Le, Denny Zhou, and Xinyun Chen

    Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, and Xinyun Chen. 2024. Large Language Models as Optimizers. InInternational Conference on Learning Representations (ICLR). arXiv:2309.03409

    Pith/arXiv arXiv 2024

  35. [35]

    Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep Interest Network for Click- Through Rate Prediction. InProceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD). 1059–1068

    2018

  36. [36]

    Kun Zhou, Hui Wang, Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen. 2020. S 3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization. InPro- ceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM). 1893–1902

    2020

  37. [2019]

    InProceedings of the 28th ACM International Conference on Information and Knowledge Management (CIKM)

    BERT4Rec: Sequential Recommendation with Bidirectional Encoder Rep- resentations from Transformer. InProceedings of the 28th ACM International Conference on Information and Knowledge Management (CIKM). 1441–1450

  38. [2020]

    InInternational Conference on Learning Representations (ICLR)

    Inductive Representation Learning on Temporal Graphs. InInternational Conference on Learning Representations (ICLR)

  39. [2024]

    InNature, Vol

    Mathematical Discoveries from Program Search with Large Language Models. InNature, Vol. 625. 468–475

This paper was first reviewed by grok-4.5 on July 31, 2026.