Pith. sign in

REVIEW 4 major objections 4 minor 31 references

A three-stage pipeline with a generative reinforcement-learning alignment stage closes the gap between multimodal pretraining and ranking objectives, delivering a statistically significant 0.55% increase in order volume at Taobao Shangou.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 11:30 UTC pith:JMEIADC4

load-bearing objection A credible industrial systems paper with a real A/B lift, but the abstract's AUC numbers are off by 100x and the paper never isolates its headline Stage 2 GRPO alignment in the ranking experiments. the 4 major comments →

arxiv 2607.29213 v1 pith:JMEIADC4 submitted 2026-07-31 cs.IR cs.LG

GALA: Generative Aligned Learning for Adaptive Multimodal Representation in the Taobao Shangou Recommender System

Jiping Liu , Zhongmin Zhang , Zisen Sang , Zhijia Fang , Tao Ouyang , Ma Jiang , Shaopeng Liang , Zeyang Hou
show 2 more authors
Guodong Cao Jia Jia
This is my paper
classification cs.IR cs.LG
keywords multimodal recommendationgenerative reinforcement learningGRPOadaptive gatingfood deliveryrepresentation alignmentembedding fusiononline A/B test
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that the gap between content pretraining and behavior-driven ranking in multimodal recommendation can be closed by inserting a generative reinforcement-learning alignment stage. Multimodal shop embeddings are first pretrained on query-image-text triplets from search logs, then refined by GRPO with a binary conversion reward from next-shop prediction, and finally fused with ID embeddings through adaptive gating with an auxiliary loss. The authors argue this is deployable under frozen-embedding serving constraints and report consistent offline AUC/PCOC improvements plus a statistically significant +0.55% order-volume increase in an online A/B test.

Core claim

At the center is a three-stage pipeline. Stage 1 aligns image, text, and query embeddings using contrastive triplets mined from search-to-purchase logs. Stage 2 treats next-shop prediction as a generative task: a language-model decoder takes a fusion prompt of spatiotemporal context, user history, and candidate shops, and reinforcement learning (GRPO) with a binary reward for predicting the purchased shop updates both the decoder and the multimodal embeddings. Stage 3 freezes those embeddings and integrates them with ID embeddings through an adaptive gating mechanism whose auxiliary loss prevents the multimodal pathway from collapsing under ID-dominated training. If correct, the claim is tha

What carries the argument

The load-bearing mechanism is the intermediate generative RL alignment stage: a fusion prompt embeds user history and candidate shops via Stage 1's fused multimodal embeddings; a decoder predicts a bounded index token for the next purchased shop; GRPO with a group-normalized binary reward (correct index) updates the decoder, the index embedding matrix, and the shop embeddings. This is followed by adaptive gating (a sigmoid over the ID hidden state) plus a stop-gradient auxiliary loss on the multimodal branch, which preserves multimodal contribution under long-term ID-dominant training.

Load-bearing premise

The binary correctness reward for next-shop prediction must be a reliable enough signal that updating the multimodal embeddings on it improves ranking once those embeddings are frozen; if this reward overfits to candidate-set artifacts or retrieval biases, the claimed gains would not be attributable to the proposed alignment mechanism.

What would settle it

A reader could rerun the offline experiments without Stage 2 (GALA T5 without GRPO) and compare next-day ranking AUC/PCOC, or construct a counterfactual where the candidate set is replaced with random negatives; if the gains vanish or reverse, the GRPO reward is exploiting retrieval artifacts rather than user intent.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If correct, content embeddings can be behaviorally refined without per-request encoder updates, preserving millisecond-level serving latency.
  • The compact 128-dimensional embeddings match or beat much higher-dimensional generic embedders, lowering memory and inference cost.
  • Long-tail shops automatically receive higher multimodal gate weights, improving coverage and exposure (0.5% shop exposure width, with larger gains during peak dining hours).
  • The staged pipeline offers a deployable template for other two-stage retrieval-and-ranking systems that rely on frozen offline embeddings.
  • The auxiliary-loss-plus-gating design suggests a general way to keep weak but valuable signals alive when strong ID signals dominate training.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The binary next-shop reward is only a proxy for order value; a utility-weighted reward could plausibly produce larger business gains, though the paper deliberately avoids its hyperparameter sensitivity.
  • The same alignment recipe likely transfers to other long-tail marketplaces—e-commerce, local services, or content platforms—where content is sparse and freshness matters more than raw ID history.
  • A reader could test whether the gate collapse is really the dominant failure mode by varying λ and measuring how much of the AUC gain comes from the auxiliary loss versus the gating itself.
  • The Stage 2 reward could overfit to candidate-set artifacts; evaluating with random-negative candidate sets would separate genuine intent alignment from retrieval bias.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper presents GALA, a three-stage multimodal representation learning pipeline for the Taobao Shangou food-delivery recommender system. Stage 1 performs domain-adaptive query–image–text triplet pretraining; Stage 2 uses GRPO (Group Relative Policy Optimization) on a next-shop prediction task to refine the multimodal embeddings with conversion-based rewards; Stage 3 fuses the frozen multimodal embeddings with ID embeddings via an adaptive gating network and a hybrid loss. The manuscript claims consistent offline gains over SOTA methods and a statistically significant +0.55% online order-volume lift (95% CI [0.342%, 0.756%], p < 0.01) from a deployed A/B test.

Significance. If the central claims hold, GALA is a practically valuable template for aligning frozen multimodal embeddings with downstream ranking objectives under production latency constraints. The paper's strengths include a deployed system with a large-scale online A/B test reported with day-level confidence intervals, detailed hyperparameter reporting, and a clear articulation of the frozen-embedding serving constraint that motivates the three-stage design. The proposed Stage 3 adaptive gating with auxiliary loss is also well-motivated and supported by the stratified analysis in Table IV. However, the claimed novelty and contribution rest on Stage 2, the generative RL alignment stage, and the manuscript does not currently provide sufficient evidence that Stage 2 rather than Stage 1 or Stage 3 is responsible for the ranking and online gains. The abstract's AUC figures also do not match the offline tables, which is a serious reporting inconsistency.

major comments (4)
  1. [Abstract and Section V.E.2 / Table III] The abstract states 'consistent offline gains of +0.12/+0.20 AUC', but Table III (Overall row) reports CTR-AUC gains of 0.7240→0.7263 (+0.0023) and CVR-AUC gains of 0.8156→0.8193 (+0.0037). Section V.E.2 itself describes these as '+0.0023' and '+0.0037'. The abstract numbers are roughly fifty times larger than the reported values and are not supported by any table. This is a load-bearing inconsistency: the central quantitative claim of offline improvement is misstated, and a reader cannot tell which numbers are correct. The abstract must be corrected to match Table III, or additional evidence for the larger gains must be provided.
  2. [Table III and Section V.E.2] The paper's central innovation is Stage 2 (generative RL alignment), but no ablation isolating Stage 2's contribution to ranking is presented. Table III includes GALA-gme emb, GALA-image emb, and GALA-text emb under the adaptive fusion framework, but there is no row using the Stage 1 embeddings (e.g., GALA-T5) without GRPO combined with the Stage 3 module. Table II shows Stage 2 improves retrieval Recall@K (0.877 vs. 0.860 for GALA-T5), but retrieval quality does not necessarily translate to ranking gains. Consequently, the offline AUC gains in Table III and the online +0.55% order-volume lift in Section V.G cannot be causally attributed to Stage 2; they could be driven entirely by Stage 1 alignment and Stage 3 adaptive gating. This omission is load-bearing for the paper's main claim and should be addressed with an explicit no-GRPO ranking ablation.
  3. [Section IV.C (Eq. 9) and Section V.E.1] The GRPO reward in Eq. (9) is binary correctness of the predicted index against the logged purchased shop within a retrieved candidate set C. The candidate set is described as returned by the retrieval module, and in production it is natural for C to be ordered by retrieval score (as implied by Fig. 5). If so, the ground-truth indices are concentrated at low positions, and the LLM decoder can learn a positional prior or exploit the learnable index embedding matrix E_idx without improving the underlying multimodal embeddings e(·). The paper does not report the index distribution, candidate-order randomization, or controls for position bias. Moreover, because the full 7B decoder is trained and e(·) is only one of the trainable components, there is no direct evidence that gradients actually update e(·) in a way that improves semantic quality. Without such evidence, the retrieval gains attri
  4. [Tables II–IV and Section V.E] All offline performance numbers are reported as point estimates with no confidence intervals, standard errors, or significance tests. This is especially problematic in Table III, where several differences are very small: e.g., GALA-gme emb and LUM both report CTR-AUC 0.7251, and GALA-image emb and LUM differ by only 0.0001. Table IV also reports stratified AUC values without variance. Given that the headline offline gains are on the order of 0.002–0.004 AUC units, uncertainty quantification is essential to establish that the differences are not noise. The online A/B test provides one credible significance statement, but the offline evaluation, which is used to support the method's general claims, lacks this discipline.
minor comments (4)
  1. [Section V.A] The text says 'we provide more details on the dataset schema, construction/filtering, rules and temporal train/validation/test splits in Table I,' but Table I is a high-level summary with no explicit schema (e.g., no feature names, types, or validation split rules). Please either expand Table I or adjust the wording.
  2. [Section IV.B, Eq. (1)] The notation 'i+' in Eq. (1) is a superscript plus sign, not an index; this is nonstandard and could be confused with the negative samples i_j. Consider writing i^+ or clarifying the positive sample notation.
  3. [Section V.F] The hardware description 'PPU610' is ambiguous; please specify the accelerator type and the number of machines used for the latency benchmark.
  4. [General] The 'AI-GENERATED CONTENT ACKNOWLEDGEMENT' section is formatted as a single run-on word. This is a minor formatting issue, but it is inconsistent with the rest of the manuscript's style.

Circularity Check

0 steps flagged

No significant circularity: GALA's stages are trained and evaluated on distinct held-out signals, and its central claims rest on offline temporal splits plus a randomized online A/B test rather than on self-referential definitions or fitted parameters renamed as predictions.

full rationale

The paper's central claim is an empirical engineering claim: a three-stage pipeline improves retrieval and ranking metrics at Taobao Shangou. No derivation step reduces to its own input by construction. Stage 1 uses search-to-purchase triplets for contrastive pretraining; Stage 2 uses user behavior sequences with a conversion-based binary reward (Eq. 9) to update the multimodal embeddings e(·); Stage 3 freezes those embeddings and trains an adaptive gating fusion with a hybrid loss. The offline evaluations use explicitly held-out next-day test splits (Table I: 'Test: next-day logs'), and the online claim is supported by a randomized traffic-split A/B test with day-level statistical significance (Section V.G). Thus, the fact that the reward and the downstream ranking objective both originate from the same platform's conversion logs is expected supervised learning, not a reduction of the prediction to the fit. The self-citations ([3]–[5]) appear only as background references for classical ID-based ranking models and are not load-bearing for the proposed mechanism; no uniqueness theorem or ansatz is imported from the authors' prior work. The reviewer concern that Stage 2's contribution is not isolated in Table III is a missing-ablation / attribution limitation, not a circularity: it does not make any reported result equivalent to its input by construction. Accordingly, the circularity score is 0.

Axiom & Free-Parameter Ledger

8 free parameters · 5 axioms · 0 invented entities

The paper introduces no new physical or conceptual entities. Its central claim rests on domain assumptions about log-derived supervision and on standard optimizers; several hyperparameters are hand-set or tuned on internal data. The SFT warm-up via a teacher LLM is a notable dependency that is not independently validated.

free parameters (8)
  • Contrastive temperature τ = 2.0
    Fixed in Stage 1 loss Eq. (1); the paper does not report sensitivity.
  • Loss weights w1, w2, w3 = 0.3 / 0.3 / 0.4
    Hand-set weights in Eq. (2); no ablation across weights.
  • GRPO clipping ε = 0.2
    Set in Section V.C for Eq. (10); standard PPO clip but hand-chosen.
  • KL regularization β = 0.001
    Set in Section V.C for Eq. (10) to control policy drift.
  • Auxiliary loss weight λ = 0.01
    Key hyperparameter in Eq. (16); Table V shows sensitivity, so the chosen value is tuned on internal validation.
  • Stage 1 learning rate = 2e-5
    Set in Section V.C; not justified beyond standard practice.
  • Stage 2 learning rate = 5e-5
    Set in Section V.C; not justified beyond standard practice.
  • SFT rationales dataset size = 1M
    Size of teacher-generated reasoning samples in Section IV.C.5; chosen by the authors.
axioms (5)
  • domain assumption Search-to-purchase triplets (query, image, text) are reliable positives for semantic alignment; in-batch non-purchased shops are representative negatives.
    Introduced in Section IV.B; if the mined positives are noisy or negatives are too easy, Stage 1 embeddings won't capture intent.
  • domain assumption Binary conversion reward is a sufficient target for aligning embeddings with ranking; utility-weighted rewards are rejected as noisy.
    Section IV.C.7 Eq. (9); the entire Stage 2 contribution depends on this reward being informative.
  • standard math GRPO with clipping and KL penalty is a valid and stable policy-gradient optimizer for this autoregressive setup.
    Section IV.C.7 Eq. (10); assumed standard, but no convergence analysis is given.
  • domain assumption Frozen-embedding offline-computation/KV-lookup serving is the correct production constraint, making end-to-end training infeasible.
    Section I and V.B; the whole design and baseline selection depend on this constraint.
  • domain assumption The teacher LLM (Qwen-72B) generates rationales that are faithful to the ground-truth decision and useful for SFT warm-up.
    Section IV.C.5; no quality control of the 1M synthesized rationales is described.

pith-pipeline@v1.3.0-daily-deepseek · 19079 in / 14632 out tokens · 141236 ms · 2026-08-03T11:30:08.992916+00:00 · methodology

0 comments
read the original abstract

Modern recommender systems in food delivery increasingly leverage multimodal signals, including images, text, and user interaction histories, to enhance user experience, yet effective fusion of these heterogeneous modalities remains challenging, hindering both the joint modeling of multimodal signals and adaptation to evolving user intent. In mainstream two-stage approaches, the separation between content-semantic pretraining of image-text encoders and behavior-driven ranking models limits alignment between semantic understanding and user behavior patterns. To address these issues, we present GALA, a three-stage pipeline whose core innovation lies in an intermediate "generative RL alignment" stage that constructs multimodal pretraining data from user behavior and refines it via conversion-based rewards, effectively bridging the pretraining-fine-tuning gap to align with downstream objectives. GALA comprises three stages: first, behavior-aware triplet pretraining on query-image-text pairs from search logs to early capture user intent and content preferences; second, a novel intermediate stage that refines multimodal embeddings through reward-driven optimization (GRPO) to dynamically align them with user behavior and bridge the pretraining-fine-tuning gap; and finally, integration of multimodal and ID embeddings via adaptive gating with a hybrid loss, preserving multimodal contributions under long-term ID-dominant training. GALA has been deployed in the production environment at Taobao Shangou, serving over 200 million daily active users. Compared with state-of-the-art (SOTA) methods, it delivers consistent offline gains of +0.12/+0.20 AUC along with better PCOC metrics. Large-scale online A/B tests further report a 0.55 percent increase in order volume, confirming GALA's effectiveness at industrial scale and its robustness across diverse demand patterns.

Figures

Figures reproduced from arXiv: 2607.29213 by Guodong Cao, Jia Jia, Jiping Liu, Ma Jiang, Shaopeng Liang, Tao Ouyang, Zeyang Hou, Zhijia Fang, Zhongmin Zhang, Zisen Sang.

Figure 1
Figure 1. Figure 1: Taobao Shangou Recommender System Architecture. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the Three-Stage Framework GALA. the contributions of ID-based and multimodal representations. An auxiliary supervision term regulates the gate weights to preserve multimodal effectiveness under long-term ID￾dominant training, mitigating performance degradation. Together, these stages enable GALA to unify multimodal pretraining, behavior-driven alignment, and adaptive fusion in a unified three-s… view at source ↗
Figure 3
Figure 3. Figure 3: Triplet Data for Domain-Adaptive Cross-Modal Align [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Pretraining Stage of GALA. for each shop (based on transaction logs, with non-primary category items excluded). Each (query, image, text) triplet is thus obtained from these high-confidence interactions, as illustrated in [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Post-Training Stage of GALA. fused representation, denoted as F usion, combines these mul￾timodal features to produce a comprehensive shop embedding. C. Generative User Behavior Alignment Stage 1 produces semantically aligned multimodal shop embeddings. As shown in [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Ranking Model Training Stage of GALA. dedicated interaction layers, analogous to ID-based feature interactions. For instance, [21] realized multimodal feature interaction by calculating the similarity between user multi￾modal sequences and target shop multimodal features; [10] performed bucket processing on similarity scores to enhance the generalization ability of multimodal recommendation. In addition, a… view at source ↗
Figure 7
Figure 7. Figure 7: Shop distribution and coverage. (a) Shops are bucketed by their occurrence counts in two months of training data; the shop ratio per bucket shows a pronounced long-tail distribution. (b) Coverage of next-day (t+1) candidate shops by training data aggregated over the past k days (t−1 to t−60); coverage increases with more history and plateaus at ∼ 80%. • GALA-gme emb: uses frozen multimodal embeddings ex￾tr… view at source ↗
Figure 8
Figure 8. Figure 8: t-SNE visualization of multimodal feature distributions [PITH_FULL_IMAGE:figures/full_fig_p010_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Visualization of model attention under different query [PITH_FULL_IMAGE:figures/full_fig_p010_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Comparison of attention maps under different methods for the given query ”qia luo noodles”. The visualization shows [PITH_FULL_IMAGE:figures/full_fig_p011_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Contributions of Multimodal/ID Gate Weights Across [PITH_FULL_IMAGE:figures/full_fig_p011_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Offline Training and Offline Deployment of GALA. [PITH_FULL_IMAGE:figures/full_fig_p012_12.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

31 extracted references · 7 linked inside Pith

  1. [1]

    Deep interest evolution network for click-through rate prediction,

    G. Zhou, N. Mou, Y . Fan, Q. Pi, W. Bian, C. Zhou, X. Zhu, and K. Gai, “Deep interest evolution network for click-through rate prediction,” inProceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificia...

    doi:10.1609/aaai.v 2019

  2. [2]

    Click prediction learning for effective advertising,

    S. Patil, K. Raut, P. Palsodkar, T. Singh, Y . Dubey, and R. Umate, “Click prediction learning for effective advertising,” in2022 International Conference on Emerging Trends in Engineering and Medical Sciences (ICETEMS), 2022, pp. 283–288

    2022

  3. [3]

    Spatiotemporal-enhanced network for click-through rate prediction in location-based services,

    S. Lin, Y . Yu, X. Ji, T. Zhou, H. He, Z. Sang, J. Jia, G. Cao, and N. Hu, “Spatiotemporal-enhanced network for click-through rate prediction in location-based services,”arXiv preprint arXiv:2209.09427, 2022

    Pith/arXiv arXiv 2022

  4. [4]

    Cspm: A contrastive spatiotemporal preference model for ctr prediction in on-demand food delivery services,

    G. Jiang, X. Li, R. Jing, R. Zhao, X. Ni, G. Cao, and N. Hu, “Cspm: A contrastive spatiotemporal preference model for ctr prediction in on-demand food delivery services,” inProceedings of the 32nd ACM International Conference on Information and Knowledge Management, ser. CIKM ’23. New York, NY , USA: Association for Computing Machinery, 2023, p. 3978–3982...

    arXiv 2023

  5. [5]

    Pre-train and fine-tune: Recommenders as large models,

    Z. Jiang, C. Chen, H. Feng, Y . Yang, J. Liu, J. Zhang, J. Jia, and N. Hu, “Pre-train and fine-tune: Recommenders as large models,” inCompanion Proceedings of the ACM on Web Conference 2025, ser. WWW ’25. New York, NY , USA: Association for Computing Machinery, 2025, p. 267–276. [Online]. Available: https://doi.org/10.1145/3701716.3715255

    arXiv 2025

  6. [6]

    Multi- modality is all you need for transferable recommender systems,

    Y . Li, H. Du, Y . Ni, P. Zhao, Q. Guo, F. Yuan, and X. Zhou, “Multi- modality is all you need for transferable recommender systems,” in2024 IEEE 40th International Conference on Data Engineering (ICDE), 2024, pp. 5008–5021

    2024

  7. [7]

    End-to-end training of multimodal model and ranking model,

    X. Deng, L. Xu, X. Li, J. Yu, E. Xue, Z. Wang, D. Zhang, Z. Liu, G. Zhou, Y . Song, N. Mou, S. Jiang, and H. Li, “End-to-end training of multimodal model and ranking model,” 2024. [Online]. Available: https://arxiv.org/abs/2404.06078

    Pith/arXiv arXiv 2024

  8. [8]

    Lemur: Large scale end-to-end multimodal recommendation,

    X. Han, H. Chen, Q. Lin, J. Gao, X. Ren, L. Zhu, Z. Ye, S. Wu, X. Xie, X. Gan, B. Wei, P. Xu, Z. Wang, Y . Zheng, J. Lin, D. Wu, and J. Ge, “Lemur: Large scale end-to-end multimodal recommendation,”

  9. [9]

    Alignrec: Aligning and training in multimodal recommendations,

    Y . Liu, K. Zhang, X. Ren, Y . Huang, J. Jin, Y . Qin, R. Su, R. Xu, Y . Yu, and W. Zhang, “Alignrec: Aligning and training in multimodal recommendations,” inProceedings of the 33rd ACM International Conference on Information and Knowledge Management, CIKM 2024, Boise, ID, USA, October 21-25, 2024, E. Serra and F. Spezzano, Eds. ACM, 2024, pp. 1503–1512. ...

    arXiv 2024

  10. [10]

    Enhancing taobao display advertising with multimodal representations: Challenges, approaches and insights,

    X. Sheng, F. Yang, L. Gong, B. Wang, Z. Chan, Y . Zhang, Y . Cheng, Y . Zhu, T. Ge, H. Zhu, Y . Jiang, J. Xu, and B. Zheng, “Enhancing taobao display advertising with multimodal representations: Challenges, approaches and insights,” inProceedings of the 33rd ACM International Conference on Information and Knowledge Management, CIKM 2024, Boise, ID, USA, O...

    arXiv 2024

  11. [11]

    Must: An effective and scalable framework for multimodal search of target modality,

    M. Wang, X. Ke, X. Xu, L. Chen, Y . Gao, P. Huang, and R. Zhu, “Must: An effective and scalable framework for multimodal search of target modality,” in2024 IEEE 40th International Conference on Data Engineering (ICDE), 2024, pp. 4747–4759

    2024

  12. [12]

    Alleviating the inconsistency of multimodal data in cross-modal retrieval,

    T. Li, X. Yang, Y . Ke, B. Wang, Y . Liu, and J. Xu, “Alleviating the inconsistency of multimodal data in cross-modal retrieval,” in2024 IEEE 40th International Conference on Data Engineering (ICDE), 2024, pp. 4643–4656

    2024

  13. [13]

    Neural discrete representation learning,

    A. van den Oord, O. Vinyals, and K. Kavukcuoglu, “Neural discrete representation learning,” inProceedings of the 31st International Con- ference on Neural Information Processing Systems, ser. NIPS’17. Red Hook, NY , USA: Curran Associates Inc., 2017, p. 6309–6318

    2017

  14. [14]

    Mm-rec: Multimodal news recommendation,

    C. Wu, F. Wu, T. Qi, and Y . Huang, “Mm-rec: Multimodal news recommendation,”CoRR, vol. abs/2104.07407, 2021. [Online]. Available: https://arxiv.org/abs/2104.07407

    Pith/arXiv arXiv 2021

  15. [15]

    Recommender systems with generative retrieval,

    S. Rajput, N. Mehta, A. Singh, R. H. Keshavan, T. Vu, L. Heldt, L. Hong, Y . Tay, V . Q. Tran, J. Samost, M. Kula, E. H. Chi, and M. Sathiamoorthy, “Recommender systems with generative retrieval,” inAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, Dec...

    2023

  16. [16]

    HLLM: enhancing sequential recommendations via hierarchical large language models for item and user modeling,

    J. Chen, L. Chi, B. Peng, and Z. Yuan, “HLLM: enhancing sequential recommendations via hierarchical large language models for item and user modeling,”CoRR, vol. abs/2409.12740, 2024. [Online]. Available: https://doi.org/10.48550/arXiv.2409.12740

    Pith/arXiv arXiv doi:10.48550/arxiv.2409.12740 2024

  17. [17]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning transferable visual models from natural language supervision,” inProceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, ser. Proceedings of Machine L...

    2021

  18. [18]

    Chinese CLIP: contrastive vision-language pretraining in chinese,

    A. Yang, J. Pan, J. Lin, R. Men, Y . Zhang, J. Zhou, and C. Zhou, “Chinese CLIP: contrastive vision-language pretraining in chinese,”CoRR, vol. abs/2211.01335, 2022. [Online]. Available: https://doi.org/10.48550/arXiv.2211.01335

    Pith/arXiv arXiv doi:10.48550/arxiv.2211.01335 2022

  19. [19]

    Align before fuse: Vision and language representation learning with momentum distillation,

    J. Li, R. R. Selvaraju, A. Gotmare, S. R. Joty, C. Xiong, and S. C. Hoi, “Align before fuse: Vision and language representation learning with momentum distillation,” inAdvances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, M. Ranzato, A. Beygelzimer...

    arXiv 2021

  20. [20]

    GME: improving universal multimodal retrieval by multimodal llms,

    X. Zhang, Y . Zhang, W. Xie, M. Li, Z. Dai, D. Long, P. Xie, M. Zhang, W. Li, and M. Zhang, “GME: improving universal multimodal retrieval by multimodal llms,”CoRR, vol. abs/2412.16855, 2024. [Online]. Available: https://doi.org/10.48550/arXiv.2412.16855

    Pith/arXiv arXiv doi:10.48550/arxiv.2412.16855 2024

  21. [21]

    Unlocking scaling law in industrial recommendation systems with a three-step paradigm based large user model,

    B. Yan, S. Liu, Z. Zeng, Z. Wang, Y . Zhang, Y . Yuan, L. Liu, J. Liu, D. Wang, W. Su, P. Wang, J. Xu, and B. Zheng, “Unlocking scaling law in industrial recommendation systems with a three-step paradigm based large user model,”CoRR, vol. abs/2502.08309, 2025. [Online]. Available: https://doi.org/10.48550/arXiv.2502.08309

    doi:10.48550/arxiv.2502.08309 2025

  22. [22]

    Adapting large language models by integrating collaborative semantics for recommendation,

    B. Zheng, Y . Hou, H. Lu, Y . Chen, W. X. Zhao, M. Chen, and J. Wen, “Adapting large language models by integrating collaborative semantics for recommendation,” in40th IEEE International Conference on Data Engineering, ICDE 2024, Utrecht, The Netherlands, May 13-16, 2024. IEEE, 2024, pp. 1435–1448. [Online]. Available: https://doi.org/10.1109/ICDE60146.2024.00118

    arXiv 2024

  23. [23]

    Momentum contrast for unsupervised visual representation learning,

    K. He, H. Fan, Y . Wu, S. Xie, and R. B. Girshick, “Momentum contrast for unsupervised visual representation learning,” in2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020. Computer Vision Foundation / IEEE, 2020, pp. 9726–9735. [Online]. Available: https://doi.org/10.1109/CVPR42600.2020.00975

    arXiv 2020

  24. [24]

    Representation learning with contrastive predictive coding,

    A. van den Oord, Y . Li, and O. Vinyals, “Representation learning with contrastive predictive coding,”CoRR, vol. abs/1807.03748, 2018. [Online]. Available: http://arxiv.org/abs/1807.03748

    Pith/arXiv arXiv 2018

  25. [25]

    Notellm-2: Multimodal large representation models for recommendation,

    C. Zhang, H. Zhang, S. Wu, D. Wu, T. Xu, X. Zhao, Y . Gao, Y . Hu, and E. Chen, “Notellm-2: Multimodal large representation models for recommendation,” inProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, V .1, KDD 2025, Toronto, ON, Canada, August 3-7, 2025, Y . Sun, F. Chierichetti, H. W. Lauw, C. Perlich, W. H. Tok, a...

    2025

  26. [26]

    Exploring the limits of transfer learning with a unified text-to-text transformer,

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y . Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,”J. Mach. Learn. Res., vol. 21, pp. 140:1–140:67, 2020. [Online]. Available: https: //jmlr.org/papers/v21/20-074.html

    2020

  27. [27]

    The use of the area under the ROC curve in the evaluation of machine learning algorithms,

    A. P. Bradley, “The use of the area under the ROC curve in the evaluation of machine learning algorithms,”Pattern Recognit., vol. 30, no. 7, pp. 1145–1159, 1997. [Online]. Available: https: //doi.org/10.1016/S0031-3203(96)00142-2

    doi:10.1016/s0031-3203(96)00142-2 1997

  28. [28]

    Practical lessons from predicting clicks on ads at facebook,

    X. He, J. Pan, O. Jin, T. Xu, B. Liu, T. Xu, Y . Shi, A. Atallah, R. Herbrich, S. Bowers, and J. Q. n. Candela, “Practical lessons from predicting clicks on ads at facebook,” inProceedings of the Eighth International Workshop on Data Mining for Online Advertising, ser. ADKDD’14. New York, NY , USA: Association for Computing Machinery, 2014, p. 1–9. [Onlin...

    arXiv 2014

  29. [2023]

    Available: http://papers.nips.cc/paper\ files/paper/202 3/hash/20dcab0f14046a5c6b02b61da9f13229-Abstract-Conference.html

    [Online]. Available: http://papers.nips.cc/paper\ files/paper/202 3/hash/20dcab0f14046a5c6b02b61da9f13229-Abstract-Conference.html

  30. [2025]

    Available: https://arxiv.org/abs/2511.10962

    [Online]. Available: https://arxiv.org/abs/2511.10962

    arXiv

  31. [2826]

    Available: https://doi.org/10.1145/3690624.3709440

    [Online]. Available: https://doi.org/10.1145/3690624.3709440

    arXiv