Pith. sign in

REVIEW 3 major objections 5 minor 53 references

Transferable and Forecastable User Targeting Foundation Model

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single model selects targeted users from a one-sentence demand across security, marketing, and recommendation domains.

desk verdict Interesting forecastable user-targeting idea, but the missing train/test temporal split makes the headline zero-shot results look like memorization rather than transfer. read the letter →

arxiv 2412.12468 v2 pith:P2U6X4I4 submitted 2024-12-17 cs.LG cs.AI

Bin Dou , Baokun Wang , Yun Zhu , Xiaotong Lin , Yike Xu , Xiaorui Huang , Yang Chen , Yun Liu
show 6 more authors
Shaoshuai Han Yongchao Liu Tianyi Zhang Yu Cheng Weiqiang Wang Chuntao Hong
This is my paper · ORCID
classification cs.LGcs.AI
keywords usertargetingunderstandingself-supervisedpre-trainingmulti-modalcontrastivelearningfoundationmodelforecastability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that one pre-trained foundation model can replace the per-scenario classifiers normally used for user targeting. Given a single sentence describing the desired audience—for example, "Please select potential 3C buyers"—the model ranks candidates by the similarity between the sentence embedding and a user embedding built from payment history, mini-program usage, search text, and account attributes. The model’s forecastability is engineered through temporal separation: user representations come only from past behavior, while the text description used as a training label is generated from future behavior. The authors report that this model outperforms LLM-based and embedding-based baselines when used directly with no additional training, and also when given just a handful of seed users, across security, marketing, and recommendation scenarios on the Alipay platform, and that it has been deployed in production. If true, non-expert marketers could write one sentence and get a ranked list of users without building a new model for each campaign.

What carries the argument

The central object is the temporal separation built into every training pair: user representation from the past window $t_1$ and text description from the future window $t_2$. The model has a user encoder that consumes behavioral sequences, tabular account features, and search text; a text encoder (a fine-tuned large language model) that consumes the templated future description; and an attention-based fusion module that combines the user encoders’ outputs. These are aligned with a contrastive loss that maximizes cosine similarity between matching user-text pairs and minimizes it for in-batch negatives. Two self-supervised objectives stabilize the user encoder before alignment: a contrastive predictive coding objective that predicts future behavioral embeddings from past context, and a cyclic regularization that keeps a user’s representation consistent across weeks; tabular features are pre-trained with masked-token and replaced-token detection. The machinery’s job is to make the user embedding encode only what is true now, while the text embedding encodes what will happen next, so that at inference a one-sentence demand about a future behavior can be matched to users who are currently on track for it.

What would settle it

Retrain the model after deleting from the 500M-pair pre-training corpus every user who appears in the five test scenarios, including that user’s future-text descriptions; if zero-shot accuracy on those held-out users drops to near chance, the reported gains came from memorizing pre-training users rather than from forecastable transfer.

Watch

Extended reading notes

Core claim

The central claim is that a single user-targeting foundation model, pre-trained with a two-stage contrastive objective on heterogeneous user data, transfers across security, marketing, and recommendation scenarios and is forecastable. The forecastability comes from the data construction: for each user, the model’s input is a representation of history—bill payments, mini-program usage, search text, and tabular account features—up to a time $t_1$, while the paired text description is generated from the user’s behavior in a future window $t_2$. After pre-training, the model maps a one-sentence demand such as “Please select potential 3C buyers” into the same embedding space as user representations, and users are ranked by cosine similarity; a few-shot variant tunes a small prompt with a triplet loss that includes hard-negative users. The authors report that this model beats LLM-based and user-embedding baselines in accuracy, precision, and recall on five real-world benchmark scenarios and on additional application scenarios, and that it has been deployed on the Alipay platform since 2024.

Load-bearing premise

The results establish transfer and forecasting only if the users and time windows used to build the pre-training text labels are kept disjoint from the users and labels in the test benchmarks, so the model cannot simply memorize a user’s future behavior.

Editorial extensions

If this is right

  • Marketers can write a one-sentence demand and get a ranked user list without building scenario-specific classifiers or gathering labels.
  • New targeting scenarios can reuse the same pretrained model with zero additional training, provided the required user data modalities are available.
  • With only ten seed users (five positive, five negative), the few-shot prompt-tuning variant improves over zero-shot performance, so low-resource domains still get a usable targeting model.
  • Because the model’s training pairs use future text for labels and past data for representations, it can anticipate which currently visible user states lead to a described future behavior.
  • The model’s production deployment has already produced documented lifts in click-through rate for specific audiences.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The two-stage recipe—self-supervised user representation first, text alignment second—could be lifted to any domain where the unit being targeted has a past state and a describable future state, such as churn prediction, credit risk, or subscription renewal.
  • If the forecastability claim is driven by the $t_1/t_2$ gap, then varying that gap should change the trade-off between prediction horizon and accuracy; a natural stress test is to shorten and lengthen the pre-training future window and measure zero-shot performance on the same benchmarks.
  • The hard-negative prompt-tuning scheme is a general way to inject business constraints into a contrastive user model, and could be reused in any setting where false positives are costly, not just user targeting.
  • A testable extension of the one-sentence interface is to use the aligned embedding space for explanation as well as ranking, retrieving the nearest text neighbors that justify a user’s inclusion.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes FOUND, a two-stage pre-trained user targeting foundation model for Alipay. In the first stage, heterogeneous user data (behavioral sequences, tabular features, search text) are encoded with self-supervised objectives; in the second stage, the fused user representation is aligned via contrastive learning with a templated text description generated from the user's future behavior. At inference, the model supports zero-shot user targeting from a one-sentence demand and few-shot targeting via prompt tuning with seed users. The authors report results on five Alipay benchmarks spanning security, marketing, and recommendation, claiming consistent improvements over LLM-based and user-embedding baselines, and state that the model has been deployed on Alipay since 2024.

Significance. If the claims are supported, this would be a practically significant contribution: a single industrial user-targeting model that transfers across domains and can be queried with a natural-language demand. The paper has real strengths: it uses a large in-house dataset, describes a two-stage pretraining design that addresses training stability, and provides deployment evidence. The central forecastability claim, however, depends critically on a temporal and user-level separation between the pretraining corpus and the test benchmarks, and the manuscript does not establish that separation. The reported empirical gains therefore cannot be interpreted as evidence of forecasting or transfer as written. The paper also contains an unexplained contradictory ablation result and reports no variance or significance measures.

major comments (3)
  1. [Section 5.1.1 and Appendix A.2] The central claim that FOUND is 'forecastable' requires that the 500M D_train user-text pairs are disjoint from the D_test labels in both users and time. The manuscript never states this. Eq. (4) and Eq. (14) show that each pretraining text is generated from future behavior Q_{i,t2}, while the user representation is built from history up to t1, and Appendix A.2 fixes these windows to the previous seven weeks and the following two weeks. D_test labels are defined as whether a user 'conducts the operations in the domain' (Section 5.1.1). If, for any test user, the label period overlaps with the two-week future window already used to generate that user's pretraining text, then the contrastive objective has already encoded the label in the text side of a pair, and zero-shot retrieval becomes a lookup over memorized future profiles rather than a forecast. Please specify the exact calendar periods for D_train and D_test, report the fraction of users and time windows that overlap, and present results for a split in which all D_test users and label periods are excluded from D_train.
  2. [Table 5, Section 5.3.1] The ablation table contradicts the component-attribution story: on D_test #5, the 'w/o Tabular' row reports 93.5 accuracy, which is 9.2 points higher than the Full model's 84.3. This is inconsistent with the claim that all data modalities contribute to performance. The authors need to explain this result or correct the table; without an explanation, the ablation does not support the conclusion that the full multi-source model is necessary.
  3. [Tables 2-5] All experimental comparisons are reported as single accuracy/precision/recall numbers without error bars, confidence intervals, or significance tests. Since the claimed improvements over baselines are sometimes a few points (e.g., 1-2 accuracy points in several D_test rows), the reader cannot assess whether the differences are reliable or within noise. Please report variance across at least a few random seeds or a paired significance test.
minor comments (5)
  1. [Appendix A.1] The text says 'λ_MLM is set at 0.6 and λ_MLM is 0.4'; the second coefficient should presumably be λ_RTD. Please fix the notation.
  2. [Table 1] The introduction and dataset description refer to 13 test scenarios, but Table 1 lists only five. Please clarify whether the remaining eight are omitted or whether the table is representative.
  3. [Section 5.2.1] The analysis says the model outperforms baselines by '10%/15%/10%' on Accuracy/Precision/Recall, but the actual margins in Table 2 vary considerably by scenario (e.g., roughly 3-11 accuracy points). Please state the aggregate or per-scenario calculation precisely.
  4. [Section 4.4.1] There is a typo in 'the seconda stage'; it should read 'the second stage'.
  5. [Figure 1] The figure caption contains a fragment ('it' pre-trained in a two strategy') and the notation e(V) in the figure is not defined in the caption. Please make the caption self-contained.

Circularity Check

1 steps flagged · score 6.0 of 10

Zero/few-shot “forecast” may reduce to retrieving pretraining text: D_test labels and D_train future-text targets are never shown to be temporally or user-level disjoint.

  1. fitted input called prediction [Sec. 4.1 (Eq. 4), Sec. 4.3.2 (Eqs. 14–15), Sec. 5.1.1, App. A.2]
    "“each user’s language description is built based on the future behaviors mentioned before” (Sec. 4.1); “D_train contains around 500 million pieces of user information (user-text pairs), with each piece organized according to Section 4.1” and “label l_i ranges from {0,1}, denoting whether the user conducts the operations in the domain” (Sec. 5.1.1); “The embedding of the user information and future text are generated from the previous-seven and next-two time windows, respectively” (App. A.2)."

    Eq. (14) turns the future-behavior text Q_i,t2 into the text embedding e^q_i, and Eq. (15) trains the user encoder to map the history V_i,t1 to that text. The D_test label l_i is exactly whether the user conducts the domain operation—the same operation templated into Q_i,t2 in Sec. 4.1. The paper never states that D_test users or label windows are disjoint from D_train's future 2-week windows. If they are not, the zero-shot retrieval can be solved by retrieving the memorized future profile: the claimed forecast reduces to a lookup over the pretraining target, and the reported gains measure memorization rather than transfer. Only an explicit temporal/user split would break this equivalence, and none is described.

full rationale

The paper's architecture is otherwise a standard CLIP-style contrastive pipeline; no load-bearing self-citation chain or imported uniqueness theorem appears. Reference [44] is related work by overlapping authors but is not used to justify a premise. The single circularity threat is the forecastability evaluation: pre-training text is generated from future behavior, and test labels are future domain operations, with no stated temporal or user separation between the 500M-pair D_train and the 13 D_test scenarios. Because Eq. (15) trains the model to map history to that future text, any overlap makes the test labels a function of the pre-training target, so the claimed zero/few-shot gains would reduce to memorized retrieval. This is a conditional but concrete reduction from the paper's own equations; a clean held-out time split would resolve it. The ablation anomaly (“w/o Tabular” 93.5 vs Full 84.3 on D_test #5) is a coherence concern, not circularity. On the evidence in the manuscript, the central “forecastable transfer” claim is not independently established as written, warranting a score of 6 rather than 0–2.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim depends on the untested assumption that benchmark labels are temporally disjoint from the future-behavior text used in pre-training, plus hand-set hyperparameters that are not ablated. No new entities are introduced.

free parameters (5)
  • Temporal window split = 7 weeks history / 2 weeks future for user-text alignment
    Chosen by hand; determines how much future information the text label contains and directly affects the forecastability claim.
  • Cyclic regularization coefficient lambda_cyc = 0.1
    Hand-set; controls strength of KL regularization in behavioral sequence pre-training.
  • Tabular loss weights lambda_MLM and lambda_RTD = 0.6 and 0.4
    Hand-set; balance MLM and RTD objectives in tabular encoder pretraining.
  • Triplet margin alpha = not reported
    Hand-set; margin in few-shot triplet loss, exact value not disclosed.
  • Embedding dimension and number of cross-attention layers = 1024-dim; 5 cross-attention layers
    Architectural choices, not justified by experiments.
assumptions (4)
  • domain assumption Future behavior text descriptions are a valid and sufficient supervision signal for user targeting.
    The entire forecastability claim rests on the assumption that templated text of next-period purchases/clicks can be predicted from prior behavior and is what a marketer's demand sentence encodes.
  • ad hoc to paper The test benchmark labels are not contaminated by pre-training user-text pairs.
    The paper does not state a temporal or user-level split; this assumption is necessary for the reported zero-shot gains to reflect generalization rather than memorization.
  • domain assumption Query rewriting maps arbitrary one-sentence demands to the template distribution without changing the intended target.
    Zero-shot transfer depends on the rewritten query being semantically equivalent to the original demand and distributionally close to training text.
  • standard math Standard contrastive learning and transformer architectures behave as expected at the reported scale.
    The method uses InfoNCE, MLM, RTD, cross-attention, and LoRA without new theoretical guarantees.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Transferable and Forecastable User Targeting Foundation Model." pith.science (2026). https://pith.science/paper/P2U6X4I4

@misc{pith2026241212468,
  author       = {Pith},
  title        = {Pith review of: Transferable and Forecastable User Targeting Foundation Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P2U6X4I4}},
  note         = {Machine review of arXiv:2412.12468}
}
read the original abstract

User targeting, the process of selecting targeted users from a pool of candidates for non-expert marketers, has garnered substantial attention with the advancements in digital marketing. However, existing user targeting methods encounter two significant challenges: (i) Poor cross-domain and cross-scenario transferability and generalization, and (ii) Insufficient forecastability in real-world applications. These limitations hinder their applicability across diverse industrial scenarios. In this work, we propose FOUND, an industrial-grade, transferable, and forecastable user targeting foundation model. To enhance cross-domain transferability, our framework integrates heterogeneous multi-scenario user data, aligning them with one-sentence targeting demand inputs through contrastive pre-training. For improved forecastability, the text description of each user is derived based on anticipated future behaviors, while user representations are constructed from historical information. Experimental results demonstrate that our approach significantly outperforms existing baselines in cross-domain, real-world user targeting scenarios, showcasing the superior capabilities of FOUND. Moreover, our method has been successfully deployed on the Alipay platform and is widely utilized across various scenarios.

Figures

Figures reproduced from arXiv: 2412.12468 by the authors.

Figure 1
Figure 1. Pre-training and inference for user targeting foundation model. We propose an industrial framework for language [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Self-supervised pre-training tasks for user behav [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Few-shot User Targeting. The embeddings of both [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: t-SNE visualization of user representations. Every [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: The architecture of our tabular encoder. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 20 canonical work pages

  1. [1]

    Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al. 2023. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.04023 (2023)

    arXiv 2023

  2. [2]

    Vance W Berger and YanYan Zhou. 2014. Kolmogorov–smirnov test: Overview. Wiley statsref: Statistics reference online (2014)

    work page 2014

  3. [3]

    Andrew P Bradley. 1997. The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern recognition 30, 7 (1997), 1145–1159

    1997

  4. [4]

    William Brannon, Suyash Fulay, Hang Jiang, Wonjune Kang, Brandon Roy, Jad Kabbara, and Deb Roy. 2023. Congrat: Self-supervised contrastive pretraining for joint graph and text embeddings. arXiv preprint arXiv:2305.14321 (2023)

    arXiv 2023

  5. [5]

    Dave Chaffey and Fiona Ellis-Chadwick. 2019. Digital marketing. Pearson uk

    work page 2019

  6. [6]

    Chun-Fu Richard Chen, Quanfu Fan, and Rameswar Panda. 2021. Crossvit: Cross- attention multi-scale vision transformer for image classification. In Proceedings of the IEEE/CVF international conference on computer vision . 357–366

    work page 2021

  7. [7]

    Kyunghyun Cho. 2014. Learning phrase representations using RNN encoder- decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014)

    arXiv 2014

  8. [8]

    Chilin Fu, Weichang Wu, Xiaolu Zhang, Jun Hu, Jing Wang, and Jun Zhou

Show all 53 references
  1. [9]

    Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang, Jiadai Sun, Jiajie Zhang, Jiale Cheng, Jiayi Gui, Jie Tang, Jing Zhang, Juanzi Li, Lei Zhao, Lindong Wu, Lucen Zhong, Mingdao Liu, Minlie H...

  2. [10]

    Jie Gu, Feng Wang, Qinghui Sun, Zhiquan Ye, Xiaoxiao Xu, Jingmin Chen, and Jun Zhang. 2021. Exploiting behavioral consistence for universal user representation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 4063–4071

    2021

  3. [11]

    Alexander Hermans, Lucas Beyer, and Bastian Leibe. 2017. In defense of the triplet loss for person re-identification. arXiv preprint arXiv:1703.07737 (2017)

    2017 arXiv

  4. [12]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021)

    2021 arXiv

  5. [13]

    Xin Huang, Ashish Khetan, Milan Cvitkovic, and Zohar Karnin. 2020. Tabtrans- former: Tabular data modeling using contextual embeddings. arXiv preprint arXiv:2012.06678 (2020)

    2020 arXiv

  6. [14]

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. 2021. Scaling up visual and vision- language representation learning with noisy text supervision. In International conference on machine learning . PMLR, 4904–4916

    2021

  7. [15]

    Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proc. of AACL

    2019

  8. [16]

    Hyeyoung Ko, Suyeon Lee, Yoonseo Park, and Anna Choi. 2022. A survey of recommendation systems: recommendation models, techniques, and application fields. Electronics 11, 1 (2022), 141

    2022

  9. [17]

    Z Lan. 2019. Albert: A lite bert for self-supervised learning of language represen- tations. arXiv preprint arXiv:1909.11942 (2019)

    2019 arXiv

  10. [18]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning . PMLR, 19730–19742

    2023

  11. [19]

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International conference on machine learning . PMLR, 12888–12900

    2022

  12. [20]

    Zicheng Li, Shoushan Li, and Guodong Zhou. 2022. Pre-trained token-replaced detection model as few-shot learner. arXiv preprint arXiv:2203.03235 (2022)

    2022 arXiv

  13. [21]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2024. Visual instruc- tion tuning. Advances in neural information processing systems 36 (2024)

    2024

  14. [22]

    Kang Liu, Feng Xue, Dan Guo, Peijie Sun, Shengsheng Qian, and Richang Hong

  15. [23]

    Zhuang Liu, Yunpu Ma, Matthias Schubert, Yuanxin Ouyang, and Zhang Xiong

  16. [24]

    Carlo Milana and Arvind Ashta. 2021. Artificial intelligence techniques in finance and financial markets: a survey of the literature. Strategic Change 30, 3 (2021), 189–209

    2021

  17. [25]

    IEEE Transactions on Multimedia 25 (2023), 9343–9355

    Multimodal graph contrastive learning for multimedia-based recommenda- tion. IEEE Transactions on Multimedia 25 (2023), 9343–9355

    2023

  18. [26]

    Narjes Nikzad-Khasmakhi, MA Balafar, and M Reza Feizi-Derakhshi. 2019. The state-of-the-art in expert recommendation systems. Engineering Applications of Artificial Intelligence 82 (2019), 126–147

    2019

  19. [27]

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018)

    2018 arXiv

  20. [28]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems 35...

    2022

  21. [29]

    Yabo Ni, Dan Ou, Shichen Liu, Xiang Li, Wenwu Ou, Anxiang Zeng, and Luo Si

  22. [30]

    Kyuyong Shin, Hanock Kwak, Kyung-Min Kim, Minkyu Kim, Young-Jin Park, Jisu Jeong, and Seungjae Jung. 2021. One4all user representation for recommender systems in e-commerce. arXiv preprint arXiv:2106.00573 (2021)

    2021 arXiv

  23. [31]

    Duncan Simester, Artem Timoshenko, and Spyros I Zoumpoulis. 2020. Targeting prospective customers: Robustness of machine-learning methods to typical data challenges. Management Science 66, 6 (2020), 2495–2522

    2020

  24. [32]

    Kai Sun, Yifan Ethan Xu, Hanwen Zha, Yue Liu, and Xin Luna Dong. 2023. Head- to-tail: How knowledgeable are large language models (llm)? AKA will llms replace knowledge graphs? arXiv preprint arXiv:2308.10168 (2023)

    2023 arXiv

  25. [33]

    A Vaswani. 2017. Attention is all you need. Advances in Neural Information Processing Systems (2017)

    2017

  26. [34]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

    2021

  27. [35]

    Xiao Wang, Guangyao Chen, Guangwu Qian, Pengcheng Gao, Xiao-Yong Wei, Yaowei Wang, Yonghong Tian, and Wen Gao. 2023. Large-scale multi-modal pre-trained models: A comprehensive survey. Machine Intelligence Research 20, 4 (2023), 447–482

    2023

  28. [36]

    Wei Wei, Chao Huang, Lianghao Xia, and Chuxu Zhang. 2023. Multi-modal self-supervised learning for recommendation. In Proceedings of the ACM Web Conference 2023. 790–800

    2023

  29. [37]

    Zhihao Wen and Yuan Fang. 2023. Augmenting low-resource text classification with graph-grounded pre-training and prompting. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. 506–516

    2023

  30. [38]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Cheng- peng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671 (2024)

    2024 arXiv

  31. [39]

    Junjie Wang, Dan Yang, Binbin Hu, Yue Shen, Wen Zhang, and Jinjie Gu. 2024. Know Your Needs Better: Towards Structured Understanding of Marketer De- mands with Analogical Reasoning Augmented LLMs. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data...

    2024

  32. [40]

    Junqi Zhang, Bing Bai, Ye Lin, Jian Liang, Kun Bai, and Fei Wang. 2020. General- purpose user embeddings based on mobile app usage. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . 2831–2840

    2020

  33. [41]

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. 2022. Conditional prompt learning for vision-language models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 16816–16825

    2022

  34. [42]

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. 2022. Learning to prompt for vision-language models. International Journal of Computer Vision 130, 9 (2022), 2337–2348

    2022

  35. [43]

    Beier Zhu, Yulei Niu, Yucheng Han, Yue Wu, and Hanwang Zhang. 2023. Prompt- aligned gradient for prompt tuning. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 15659–15669

    2023

  36. [44]

    Fajie Yuan, Xiangnan He, Alexandros Karatzoglou, and Liguang Zhang. 2020. Parameter-efficient transfer from sequential behaviors for user modeling and recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retri...

    2020

  37. [49]

    #=(𝑇$,𝑇%,…𝑇&)Continuous Features𝑇!'(#𝜖ℛ!'(# Transformer×𝑁Multi-Head AttentionAdd & Norm Add & NormFeed Forward MLP Mean Pooling 𝑓!

    Yun Zhu, Haizhou Shi, Xiaotang Wang, Yongchao Liu, Yaoke Wang, Boci Peng, Chuntao Hong, and Siliang Tang. 2024. GraphCLIP: Enhancing Transferabil- ity in Graph Foundation Models for Text-Attributed Graphs. arXiv preprint arXiv:2410.10329 (2024). WWW Companion ’25, April 28-May...

    2024 arXiv

  38. [50]

    Though showing fantastic capability at Alipay platform, our model’s generalizability to other platforms or industrial scenarios is not fully tested

  39. [51]

    Therefore, our future work will focus on the following:

    The training cost remains high though our two-stage strategy releases the requirement compared to the from-scratch strategy. Therefore, our future work will focus on the following:

  40. [52]

    Enhancing the model transferability across platforms. We plan to achieve this by enhancing the model’s ability for processing multi- modal user features (such as images, videos, knowledge graph, etc) and improving the model scalability for more data

  41. [53]

    Reducing the time consumption for the foundation model pre-training

  42. [2018]

    In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

    Perceive your users in depth: Learning universal user representations from multiple e-commerce tasks. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . 596–605

  43. [2022]

    In Proceedings of the 2022 International Conference on Multimedia Retrieval

    Multi-modal contrastive pre-training for recommendation. In Proceedings of the 2022 International Conference on Multimedia Retrieval . 99–108

    2022

  44. [2023]

    In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management

    Robust user behavioral sequence representation via multi-scale stochastic distribution prediction. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management . 4567–4573

  45. [2024]

    arXiv:2406.12793

    ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools. arXiv:2406.12793

    arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.