Pith. sign in

REVIEW 5 major objections 6 minor 2 cited by

PinFM: Foundation Model for User Activity Sequences at a Billion-scale Visual Discovery Platform

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read PinFM is a foundation model for user activity sequences: a 20-billion-parameter transformer, pretrained on years of user actions and fine-tuned into downstream ranking models, that improves engagement metrics and cold-start item…

desk verdict The deployed system and online A/B lifts are credible, but the abstract's 20% cold-start claim is unsupported by Table 2 and needs to be reframed before this is publishable. read the letter →

arxiv 2507.12704 v3 pith:TALHUMC4 submitted 2025-07-17 cs.LG cs.IR

Xiangyi Chen , Kousik Rajesh , Matthew Lawhon , Zelun Wang , Hanyu Li , Haomiao Li , Saurabh Vishwas Joshi , Pong Eksombatchai
show 4 more authors
Jaewon Yang Yi-Ping Hsu Jiajing Xu Charles Rosenberg
This is my paper
classification cs.LGcs.IR
keywords recommendersystemsuseractivitysequencesfoundationmodelpretrainingandfine-tuningsequentialrecommendationcold-startitemscross-attentiontransformerembeddingquantization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PinFM is a foundation model for user activity sequences: a transformer with over 20 billion parameters, pretrained on roughly two years of user actions and then fine-tuned into downstream ranking models. The paper's central claim is that this pretrain-then-finetune recipe works for industrial recommender systems, where a single large sequence model can be reused across multiple applications and surfaces. The authors report that adding PinFM to the Home Feed and Related Items ranking models improves engagement in online A/B tests, including a 5.70% lift in Saves on fresh items on the Home Feed. They also report that engineering the input sequence, such as appending candidate item features, lets the model learn interactions between user history and candidate items, increasing engagement with new items. If those results hold, they show that a foundation-model approach can scale to billion-user traffic while staying within recommender latency and cost budgets.

What carries the argument

The load-bearing machinery is threefold. The first is the Deduplicated Cross-Attention Transformer (DCAT), which computes the transformer's context pass once per user, stores per-layer key/value caches, and then reuses those caches to score many candidates via cross-attention instead of running full self-attention for every candidate; the paper reports a 600% serving-throughput improvement over FlashAttention-based self-attention. The second is input sequence engineering for fine-tuning: the candidate item is appended to the user-action sequence, optionally with a GraphSAGE candidate embedding summed into the candidate's representation and a learnable token before it (the PinFM-GraphSAGE-LT variant), so the transformer can learn user-candidate interactions through attention. The third is a set of cold-start and efficiency mechanisms: candidate item id randomization during fine-tuning, item-age dependent dropout on module outputs for fresh items, and post-training int4 quantization of the billion-row ID embedding tables, which cuts the embedding table to 31.25% of its original size with statistically neutral online performance. Pretraining losses, an infoNCE next-token loss, a multi-token loss, and a future-token loss, shape the pretrained representations, while a reduced learning rate and optional sequence losses keep the pretrained knowledge intact during fine-tuning.

What would settle it

Train the same downstream ranking models with identical compute and fine-tuning budgets, replacing PinFM's pretrained sequence encoder with a randomly initialized transformer of the same architecture and size; if the HIT@3 and online engagement lifts persist at the same magnitude, the gains are attributable to capacity and fine-tuning rather than to pretrained user-sequence knowledge. A second check is to evaluate the fine-tuned models on a public next-item prediction benchmark: if PinFM underperforms smaller baselines there, the reported gains are surface-specific rather than a property of the pretrained representation.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a large transformer pretrained with next-token and multi-token prediction objectives over user activity sequences can be coupled to existing ranking models across different applications and improve their engagement metrics. PinFM processes a user's action history into a representation; the candidate item is inserted into the input sequence so that cross-attention can contextualize the history against that candidate. The paper reports offline HIT@3 lifts (for example, a 3.76% relative improvement in Home Feed Save HIT@3 for the best input variant) and statistically significant online A/B gains: Home Feed sitewide Saves +1.20%, Home Feed Fresh Saves +5.70%, and Related Items sitewide Saves +0.72%. It further claims that the cold-start techniques, candidate item randomization and item-age dependent dropout, turn an offline Fresh Saves deficit of about -4% into a +17% lift, and that the Deduplicated Cross-Attention Transformer plus quantized embeddings keep the added serving cost neutral. The paper therefore asserts an end-to-end paradigm: one pretrained sequence model, shared across applications, can be fine-tuned per surface, handle items unseen at pretraining time, and run online at billion-user scale.

Load-bearing premise

The entire evaluation rests on the assumption that the HIT@3 metric (whether the top three items in a same-time group received the corresponding user action) faithfully reflects recommendation quality and that the "without PinFM" baselines were trained as hard as the PinFM-augmented models; if the baselines are undertrained or the metric is gameable, the reported relative lifts would not establish the central claim.

Editorial extensions

If this is right

  • A single pretrained sequence model can be amortized across multiple recommender surfaces, avoiding the cost of training a large sequence model separately for each application.
  • Candidate items can be folded into the user sequence, so the same foundation model learns user-candidate interactions by input construction rather than by a separate interaction module.
  • Fresh-item performance can be recovered with candidate-id randomization and age-dependent dropout, which the paper demonstrates by moving Home Feed Fresh Saves offline from negative to strongly positive and online to +5.70%.
  • The DCAT reuse pattern (one user context scored against many candidates) plus quantized embeddings makes a 20-billion-parameter model servable within recommender latency budgets, supporting deployment to over half a billion users.
  • Fine-tuning is required: without it the lifts vanish, which the paper reads as evidence that pretraining alone does not transfer directly to ranking objectives.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension would be to apply the DCAT caching trick to any ranking stage where the user context is shared across a large candidate list, such as retrieval reranking, where the same ratio of one user sequence to many candidates holds and the same KV-reuse pattern could cut cost.
  • The cold-start recipe suggests a general principle: randomizing the identity embedding during training and regularizing the module's output for fresh items decouples overall gains from fresh-item losses; this is testable on other id-based ranking models beyond the two surfaces reported.
  • Because the reported lifts are relative to internal baselines and evaluated with HIT@3, the paper does not establish that PinFM would beat smaller sequence models on public benchmarks; testing the same fine-tuning recipe on a public sequential-recommendation dataset would clarify how much of the gain comes from pretraining scale versus platform-specific features.
  • The input-sequence ablations imply that the model's understanding of a candidate depends on where and how the candidate appears in the sequence; varying the position and number of candidate tokens, rather than only appending one, is a dimension the paper leaves unexplored.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper presents PinFM, a pretrained transformer-based foundation model over user activity sequences at Pinterest. The model is pretrained with contrastive next-token, multi-token, and future-token losses on two years of user actions, then fine-tuned into the Home Feed and Related Items ranking models. The authors introduce the Deduplicated Cross-Attention Transformer (DCAT) to share user-sequence computation across candidates, int4 post-training quantization for the billion-scale embedding table, and several cold-start mechanisms (candidate ID randomization, item-age dependent dropout, GraphSAGE embeddings, and a learnable token). Offline experiments report relative HIT@3 lifts across input-sequence variants, pretraining/fine-tuning losses, positive-action sets, vocabulary sizes, and cold-start techniques; online A/B tests report engagement lifts on Home Feed and Related Items. The paper claims a 20% increase in engagement with new items and states that PinFM is deployed to serve more than half a billion users.

Significance. If the results hold, PinFM is a substantial industrial contribution: it demonstrates a viable pretrain-and-fine-tune paradigm for user activity sequences across multiple recommender surfaces, and the engineering innovations (DCAT, int4 quantization, KV-cache sharing) address real serving constraints for a model with a 20-billion-parameter embedding table. The paper reports several negative or neutral results honestly, such as the I2I Fresh Saves decline in Table 7 and the small int4 quantization drop in Section 4.2. The main weaknesses are statistical reporting and an overclaimed cold-start headline; these are fixable and do not invalidate the core engineering/empirical contribution.

major comments (5)
  1. [Abstract and Section 5.1, Table 2] The abstract's claim that PinFM leads to 'a 20% increase in engagement with new items' is not supported by the reported data. Table 2 reports relative HIT@3 lifts against a 'w/o PinFM' baseline: the final cold-start variant is +17.72% on HF 28d Save and +12.01% on HF 7d. The 'more than 20% boost' in the text is the arithmetic gap between two PinFM variants (-4.40% for PinFM-cs-none and +17.72% for PinFM-cs-CIR-IDD-GSLT), not a measured 20% increase in user engagement. The online Fresh Saves lift for Home Feed in Table 7 is +5.70%, not 20%. Please rephrase the claim as an offline relative HIT@3 gain or supply online engagement evidence that actually supports a 20% effect.
  2. [Section 5.1, Tables 1-6] All offline comparisons report only relative HIT@3 lifts, with no raw hit rates, confidence intervals, standard errors, or significance tests. HIT@3 is a custom metric, and the description of how recommendation groups are formed and how ties are handled is underspecified. Because the headline conclusions (e.g., the 3.76% HF Save lift in Table 1 and the cold-start gains in Table 2) rest entirely on this metric, the paper should report raw values, evaluation sample sizes, and some measure of variability, ideally across repeated runs or held-out time periods.
  3. [Section 5.2, Table 7] The statement that all online results are statistically significant at the 95% confidence level is not substantiated. No p-values, confidence intervals, sample sizes, or multiple-comparison corrections are given, and the I2I Fresh Saves entry is -0.82% while still being asserted as significant. Please describe the A/B setup (randomization unit, experiment duration, metric definitions) and report uncertainty for each lift, including negative results.
  4. [Section 5.1, Tables 1 and 2] The 'w/o PinFM' baselines are not described in terms of model capacity, training budget, hyperparameter tuning, or feature set. If the baseline ranking models were not trained to a comparable level of effort, all reported relative lifts would be inflated. Please document the baseline training configuration and, where possible, report the performance of the same downstream model with an equally sized sequence encoder trained only on downstream data.
  5. [Sections 3.2 and 5.1, Table 2] The cold-start techniques (CIR rate of 10%, IDD dropout probabilities of 0.7 and 0.5, age thresholds of 7 and 28 days, and the GSLT variant) appear to have been selected after observing the negative fresh-item effect of PinFM-cs-none, and no held-out validation or repeated evaluation is reported. This creates a selection-on-the-evaluation-metric risk for the Table 2 gains. Please validate the chosen settings on a time-based holdout or report stability across repeated experiments.
minor comments (6)
  1. [Section 3.2, Table 2] The variant name 'PinFM-cs-CPR' in the IDD definition appears to be a typo for 'PinFM-cs-CIR'.
  2. [Section 3.3] The reported 1.4% drop from eliminating causal masking is not tied to any table or uncertainty estimate; please attach it to the relevant ablation or provide the supporting numbers.
  3. [Section 4.1] The throughput improvements (600% serving, 200% training) lack measurement context; specify hardware, batch sizes, sequence lengths, and the baseline attention implementation against which the gains are computed.
  4. [Abstract and Section 4.2] The '20B+ parameters' claim should clarify that the embedding tables contribute roughly 20 billion parameters and that the transformer layers are a separate, smaller component.
  5. [Figure 3] The y-axis labels and units for Save and Hide lift are missing, and the '0 iteration' baseline should be explicitly defined in the caption.
  6. [Section 5.2] The sentence 'we observed increase an in feed diversity' contains a typo, and the diversity metric is not defined; please add a precise definition.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PinFM's claims are empirical offline and online comparisons, not derivations whose outputs equal their inputs; the abstract's '20%' cold-start phrasing overstates Table 2 but is an evidence-reporting issue, not a circular reduction.

full rationale

The paper's central claims are empirical rather than derivational. There is no equation in which a reported metric equals an input, a fitted parameter, or a self-citation by construction: HIT@3 is defined independently of PinFM's parameters, and the reported lifts are differences against a 'w/o PinFM' baseline (Tables 1, 2, 3) or statistically significant online A/B results (Table 7). The cold-start techniques (CIR, IDD, GSLT) are evaluated on the same HIT@3 metric that motivated them, which creates a selection or optimism concern, but the paper explicitly reports both the negative result of PinFM-cs-none on fresh items (HF 28d Save -4.40%) and the positive result of the full variant (+17.72%), so the contrast is an empirical finding rather than a fitted parameter renamed as a prediction. The abstract's phrase 'leading to a 20% increase in engagement with new items' overstates what Table 2 demonstrates: the 'more than 20% boost' is the relative HIT@3 gap between two PinFM variants, while the online Fresh Saves lift is +5.70%; this is a correctness/evidence-strength issue, not a circularity. The only self-citation used to justify a design choice, [8] for ID-based embeddings, is accompanied by independent infrastructure arguments and is an externally published result, so it is not load-bearing for the main evaluation. Lack of baseline training-effort detail and confidence intervals for Table 2 is a reproducibility concern, not a circular reduction. No circular step can be exhibited, so the score is 0.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard deep learning machinery plus domain assumptions about transferability of user sequences and metric validity. The free parameters above are hand-chosen or learned hyperparameters that the reported results depend on. No new physical entities or theoretical constructs are introduced.

free parameters (7)
  • infoNCE temperature tau = learnable, small initial value
    Controls sharpness of the contrastive loss in Eq. (2); learned during pretraining and affects all downstream fine-tuning.
  • sequence length L = a few hundred; fixed to 256 in DCAT serving
    Truncation length for pretraining sequences; chosen for compute/latency tradeoff (Sec 3.1, Sec 4.1).
  • future-token window L' = not specified
    Window length for multi-token loss L_mtl; hand-chosen and subsampled to reduce computation (Sec 3.1).
  • CIR randomization rate = 10%
    Fraction of candidate item IDs randomized during fine-tuning to simulate cold start (Sec 3.2).
  • item-age dependent dropout probabilities = p=0.7 for items <7 days, p=0.5 for 7-28 days
    Dropout added to PinFM outputs for fresh items; thresholds and probabilities chosen ad hoc (Sec 5.1).
  • item-age thresholds = 7 days and 28 days
    Define fresh items for IDD; selected to fit the cold-start evaluation buckets (Sec 5.1).
  • embedding vocabulary size = 160M rows (8 sub-tables x 80M rows x 32 dims)
    Chosen after ablations showing gains at larger vocabularies (Table 6); contributes about 20B parameters.
assumptions (5)
  • domain assumption User activity sequences are a transferable signal across recommender applications.
    The pretrain-finetune premise; stated in Sec 1, 3.1.
  • domain assumption Predicting future positively engaged items (next-token and windowed losses) is a useful pretraining objective for downstream ranking.
    Motivates L_ntl, L_mtl, L_ftl; validated only by downstream metrics (Sec 3.1).
  • domain assumption HIT@3 is a faithful proxy for recommendation quality.
    All offline results use this custom metric; its definition and validity are given in Sec 5.1.
  • domain assumption The deduplication operator Psi is exact and invertible, so de-duplicated sequence encoding produces the same context for all candidates.
    DCAT relies on this to reuse KV cache; Eq. (3)-(4) in Sec 4.1.
  • standard math Standard transformer/attention mechanics and infoNCE loss are sound.
    Unproved background results relied on throughout Sec 3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PinFM: Foundation Model for User Activity Sequences at a Billion-scale Visual Discovery Platform." pith.science (2026). https://pith.science/paper/TALHUMC4

@misc{pith2026250712704,
  author       = {Pith},
  title        = {Pith review of: PinFM: Foundation Model for User Activity Sequences at a Billion-scale Visual Discovery Platform},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TALHUMC4}},
  note         = {Machine review of arXiv:2507.12704}
}
read the original abstract

User activity sequences have emerged as one of the most important signals in recommender systems. We present a foundational model, PinFM, for understanding user activity sequences across multiple applications at a billion-scale visual discovery platform. We pretrain a transformer model with 20B+ parameters using extensive user activity data, then fine-tune it for specific applications, efficiently coupling it with existing models. While this pretraining-and-fine-tuning approach has been popular in other domains, such as Vision and NLP, its application in industrial recommender systems presents numerous challenges. The foundational model must be scalable enough to score millions of items every second while meeting tight cost and latency constraints imposed by these systems. Additionally, it should capture the interactions between user activities and other features and handle new items that were not present during the pretraining stage. We developed innovative techniques to address these challenges. Our infrastructure and algorithmic optimizations, such as the Deduplicated Cross-Attention Transformer (DCAT), improved our throughput by 600% on Pinterest internal data. We demonstrate that PinFM can learn interactions between user sequences and candidate items by altering input sequences, leading to a 20% increase in engagement with new items. PinFM is now deployed to help improve the experience of more than half a billion users across various applications.

Figures

Figures reproduced from arXiv: 2507.12704 by the authors.

Figure 1
Figure 1. Training and fine-tuning of PinFM losses L𝑛𝑡𝑙 and L𝑚𝑡𝑙 put approximately equal weights on predict￾ing future tokens for each input sequence length, but we want the model to be more predictive especially when the sequence length is close to the downstream input sequence length 𝐿𝑑 . To achieve this, we add another loss to predict a future window of positive tokens on H𝐿𝑑 , we term this the Future-Token Loss (L𝑓 𝑡𝑙). L… view at source ↗
Figure 2
Figure 2. shows the serving infrastructure design for PinFM. Notably, we separate our large ID embedding table from the dense model and serve it on a CPU cluster. This allows us to scale our embedding table and model size without having to share the limited GPU memory available during inference. The inference router is responsible for fetching the relevant ID embeddings for the user sequence and [PITH_FULL_IMAGE:figures/full… view at source ↗
Figure 3
Figure 3. Save and Hide vs. number of pretraining iterations [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Clinical Audit Logs as Multi-Axial Traces of Care Delivery

    cs.CY 2026-07 conditional novelty 5.0 of 10

    EHR audit log entries should be read as multi-axial events — clinician, patient, team, and process orderings at once — motivating shared foundation-model representations and a cross-axis benchmark.

  2. Large Foundation Model for Ads Recommendation

    cs.LG 2025-08 conditional novelty 5.0 of 10

    Tencent's LFM4Ads transfers user, item, and user-item cross representations from a pre-trained foundation model into downstream ad models via feature, module, and model-level mechanisms, reporting a 2.45% platform-wid...

Reference graph

Works this paper leans on

34 extracted references · 11 canonical work pages · cited by 2 Pith papers

  1. [1]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin...

    work page 2020

  2. [3]

    Zeyu Cui, Jianxin Ma, Chang Zhou, Jingren Zhou, and Hongxia Yang. 2022. M6-Rec: Generative Pretrained Language Models are Open-Ended Recom- mender Systems. CoRR abs/2205.08084 (2022). doi:10.48550/ARXIV.2205.08084 arXiv:2205.08084

    arXiv doi:10.48550/arxiv.2205.08084 2022

  3. [4]

    Tri Dao. 2023. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning. arXiv:2307.08691 [cs.LG] https://arxiv.org/abs/2307.08691

    arXiv 2023

  4. [5]

    Hamed Firooz, Maziar Sanjabi, Adrian Englhardt, Aman Gupta, Ben Levine, Dre Olgiati, Gungor Polatkan, Iuliia Melnychuk, Karthik Ramgopal, Kirill Tala- nine, Kutta Srinivasan, Luke Simon, Natesh Sivasubramoniapillai, Necip Fazil Ayan, Qingquan Song, Samira Sriram, Souvik Ghosh, Tao Song, Vignesh Kotha- palli, Xiaoling Zhai, Ya Xu, Yu Wang, and Yun Dai. 202...

    work page doi:10.48550/arxiv.2501.16450 2025

  5. [6]

    Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk

  6. [7]

    Yupeng Hou, Shanlei Mu, Wayne Xin Zhao, Yaliang Li, Bolin Ding, and Ji-Rong Wen. 2022. Towards Universal Sequence Representation Learning for Recom- mender Systems. In KDD ’22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, August 14 - 18, 2022 , Aidong Zhang and Huzefa Rangwala (Eds.). ACM, 585–593. doi:10.114...

    arXiv 2022

  7. [8]

    Yi-Ping Hsu, Po-Wei Wang, Chantat Eksombatchai, and Jiajing Xu. 2024. Taming the One-Epoch Phenomenon in Online Recommendation System by Two-stage Contrastive ID Pre-training. In Proceedings of the 18th ACM Conference on Rec- ommender Systems. 838–840

    work page 2024

  8. [9]

    Chengkai Huang, Tong Yu, Kaige Xie, Shuai Zhang, Lina Yao, and Julian J. McAuley. 2024. Foundation Models for Recommender Systems: A Survey and New Perspectives. CoRR abs/2402.11143 (2024). doi:10.48550/ARXIV.2402.11143 arXiv:2402.11143

    arXiv doi:10.48550/arxiv.2402.11143 2024

Show all 34 references
  1. [10]

    Lei Huang, Hao Guo, Linzhi Peng, Long Zhang, Xiaoteng Wang, Daoyuan Wang, Shichao Wang, Jinpeng Wang, Lei Wang, and Sheng Chen. 2025. SessionRec: Next Session Prediction Paradigm For Generative Sequential Recommendation. arXiv:2502.10157 [cs.IR] https://arxiv.org/abs/2502.10157

    2025 arXiv

  2. [11]

    Dmytro Ivchenko, Dennis Van Der Staay, Colin Taylor, Xing Liu, Will Feng, Rahul Kindi, Anirudh Sudarshan, and Shahin Sefati. 2022. TorchRec: a PyTorch Domain Library for Recommendation Systems. In Proceedings of the 16th ACM Conference on Recommender Systems (Seattle, WA, USA)...

    2022 doi

  3. [12]

    Wang-Cheng Kang and Julian J. McAuley. 2018. Self-Attentive Sequential Rec- ommendation. In IEEE International Conference on Data Mining, ICDM 2018, Singapore, November 17-20, 2018 . IEEE Computer Society, 197–206. doi:10.1109/ ICDM.2018.00035

    2018

  4. [13]

    Nikhil Khani, Li Wei, Aniruddh Nath, Shawn Andrews, Shuo Yang, Yang Liu, Pendo Abbo, Maciej Kula, Jarrod Kahn, Zhe Zhao, Lichan Hong, and Ed H. Chi

  5. [14]

    Daya Khudia, Jianyu Huang, Protonu Basu, Summer Deng, Haixin Liu, Jongsoo Park, and Mikhail Smelyanskiy. 2021. FBGEMM: Enabling High-Performance Low-Precision Deep Learning Inference. arXiv preprint arXiv:2101.05615 (2021)

    2021 arXiv

  6. [15]

    Jiacheng Li, Ming Wang, Jin Li, Jinmiao Fu, Xin Shen, Jingbo Shang, and Julian J. McAuley. 2023. Text Is All You Need: Learning Language Representations for Sequential Recommendation. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ...

    2023

  7. [16]

    Mingfu Liang, Xi Liu, Rong Jin, Boyang Liu, Qiuling Suo, Qinghai Zhou, Song Zhou, Laming Chen, Hua Zheng, Zhiyuan Li, Shali Jiang, Jiyan Yang, Xiaozhen Xia, Fan Yang, Yasmine Badr, Ellie Wen, Shuyu Xu, Hansey Chen, Zhengyu Zhang, Jade Nie, Chunzhi Yang, Zhichen Zeng, Weilin Zh...

    2025 doi arXiv

  8. [17]

    Zihan Liu, Yupeng Hou, and Julian J. McAuley. 2024. Multi-Behavior Generative Recommendation. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, CIKM 2024, Boise, ID, USA, October 21-25, 2024 , Edoardo Serra and Francesca Spezzano ...

    2024

  9. [18]

    Maxim Naumov, Dheevatsa Mudigere, Hao-Jun Michael Shi, Jianyu Huang, Narayanan Sundaraman, Jongsoo Park, Xiaodong Wang, Udit Gupta, Carole-Jean Wu, Alisson G. Azzolini, Dmytro Dzhulgakov, Andrey Mallevich, Ilia Cherni- avskii, Yinghai Lu, Raghuraman Krishnamoorthi, Ansha Yu, V...

    2019 arXiv

  10. [19]

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018)

    2018 arXiv

  11. [20]

    Nikil Pancha, Andrew Zhai, Jure Leskovec, and Charles Rosenberg. 2022. Pin- nerFormer: Sequence Modeling for User Representation at Pinterest. In KDD ’22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, August 14 - 18, 2022 , Aidong ...

    2022

  12. [21]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9

    2019

  13. [22]

    Tran, Jonah Samost, Maciej Kula, Ed H

    Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan Hulikal Kesha- van, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Q. Tran, Jonah Samost, Maciej Kula, Ed H. Chi, and Mahesh Sathiamoorthy. 2023. Rec- ommender Systems with Generative Retrieval. In Advances in Neural I...

    2023

  14. [23]

    Steffen Rendle. 2010. Factorization Machines. In 2010 IEEE International Confer- ence on Data Mining . 995–1000. doi:10.1109/ICDM.2010.127

    2010 doi

  15. [24]

    Zihua Si, Lin Guan, Zhongxiang Sun, Xiaoxue Zang, Jing Lu, Yiqun Hui, Xingchao Cao, Zeyu Yang, Yichen Zheng, Dewei Leng, Kai Zheng, Chenbin Zhang, Yanan Niu, Yang Song, and Kun Gai. 2024. TWIN V2: Scaling Ultra-Long User Be- havior Sequence Modeling for Enhanced CTR Prediction...

    2024 doi arXiv

  16. [25]

    Ruoxi Wang, Rakesh Shivanna, Derek Zhiyuan Cheng, Sagar Jain, Dong Lin, Lichan Hong, and Ed H. Chi. 2021. DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems. InWWW ’21: The Web Conference 2021, Virtual Event / Ljubljana, Slovenia...

    2021

  17. [26]

    Chuhan Wu, Fangzhao Wu, Tao Qi, Jianxun Lian, Yongfeng Huang, and Xing Xie. 2020. PTUM: Pre-training User Model from Unlabeled User Behaviors via Self-supervision. In Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020 (Find...

    2020 doi

  18. [27]

    Xue Xia, Pong Eksombatchai, Nikil Pancha, Dhruvil Deven Badani, Po-Wei Wang, Neng Gu, Saurabh Vishwas Joshi, Nazanin Farahpour, Zhiyuan Zhang, and Andrew Zhai. 2023. TransAct: Transformer-based Realtime User Action Model for Recommendation at Pinterest. In Proceedings of the 2...

    2023

  19. [28]

    Xue Xia, Pong Eksombatchai, Nikil Pancha, Dhruvil Deven Badani, Po-Wei Wang, Neng Gu, Saurabh Vishwas Joshi, Nazanin Farahpour, Zhiyuan Zhang, and Andrew Zhai. 2023. Transact: Transformer-based realtime user action model for recommendation at pinterest. In Proceedings of the 2...

    2023

  20. [29]

    Ruibin Xiong, Yunchang Yang, Di He, Kai Zheng, Shuxin Zheng, Chen Xing, Huishuai Zhang, Yanyan Lan, Liwei Wang, and Tieyan Liu. 2020. On layer normalization in the transformer architecture. In International conference on machine learning. PMLR, 10524–10533

    2020

  21. [31]

    Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhaojie Gong, Fangda Gu, Jiayuan He, Yinghai Lu, and Yu Shi. 2024. Ac- tions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations. In Forty-first Internation...

    2024

  22. [32]

    Gaowei Zhang, Yupeng Hou, Hongyu Lu, Yu Chen, Wayne Xin Zhao, and Ji- Rong Wen. 2024. Scaling Law of Large Sequential Recommendation Models. In Proceedings of the 18th ACM Conference on Recommender Systems, RecSys 2024, Bari, Italy, October 14-18, 2024 , Tommaso Di Noia, Pasqu...

    2024

  23. [33]

    Guorui Zhou, Xiaoqiang Zhu, Chengru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep Interest Network for Click-Through Rate Prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining...

    2018

  24. [1068]

    doi:10.1145/3219819.3219823

  25. [2016]

    In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.)

    Session-based Recommendations with Recurrent Neural Networks. In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1511.06939

    2016 arXiv

  26. [2024]

    Bridging the Gap: Unpacking the Hidden Challenges in Knowledge Distil- lation for Online Ranking Systems. In Proceedings of the 18th ACM Conference on Recommender Systems, RecSys 2024, Bari, Italy, October 14-18, 2024 , Tommaso Di Noia, Pasquale Lops, Thorsten Joachims, Katrie...

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.