{"id":"2db8b86d-ea11-41ae-b778-3e34d3cc5337","arxiv_id":"2412.12468","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"FOUND aligns user history embeddings with text descriptions of future behavior, enabling zero-shot and few-shot user targeting from a single sentence on Alipay.","lead":"FOUND is a pre-trained model that matches a one-sentence description of a target audience to a user's historical payment, browsing, and search data. The authors report large gains over existing methods on Alipay targeting scenarios and say the model is running in production.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The pre-training corpus and test labels are never shown to be temporally or user-level disjoint; because pre-training text is generated from future behavior, the zero/few-shot results may reflect memorization, not forecastable transfer.","rationale":"The reader's weakest_assumption is precisely the condition that must hold for the forecastability claim: D_train user-text pairs and D_test labels must be generated from disjoint users or disjoint time periods. The paper never states this, and its own Sec. 4.1/Appendix A.2 make the overlap scenario concrete. The architecture is plausible and the pre-training is substantial, but without a temporal/user split the comparison in Tables 2 and 3 cannot distinguish transfer from memorization of future behavior. This validates the reader's REJECT verdict; no separate adjustment is needed. The ablation inconsistency in Table 5 is a further sign that the empirical narrative needs correction, but it is secondary to the missing split.","tokens_in":16168,"tokens_out":6128,"duration_ms":58611,"concrete_test":"Have the authors recompute Tables 2 and 3 under a strict split: remove every user appearing in any D_test scenario from the 500M pre-training corpus, and require that the label period for each test scenario begins after the end of the latest 2-week future-text window used to build any D_train pair; if this design is already in place, publish the cutoff dates and user-intersection counts. If zero-shot and few-shot gains over ARALLM collapse under this split, the reported accuracy is memorization rather than transfer.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that FOUND forecasts and transfers: a model trained on (history up to t1, text generated from t2) should select users for a new one-sentence demand. That requires that the 500M D_train pairs (Sec. 5.1.1) do not contain the same user's future behavior as the one encoded in the D_test labels. Sec. 4.1 constructs each text Q_i,t2 from future behaviors, and Appendix A.2 fixes the alignment window to previous 7 weeks / next 2 weeks, but no sentence in Sec. 5.1.1 states when D_test users or label periods sit relative to this corpus. If a test user's label is defined by behavior in the same 2-week window used to build that user's pre-training text, then Eq. (14) has already encoded the answer in the text side of a contrastive pair; zero-shot retrieval becomes a lookup over memorized future profiles, and the claimed 'forecastability' is not measured. The issue is compounded by the lack of error bars or released artifacts, and by an unexplained ablation: in Table 5, 'w/o Tabular' beats Full by 9.2 accuracy on D_test #5 (93.5 vs 84.3), contradicting the paper's component-attribution story. Neither is proof that the method is wrong, but together they make the headline empirical claim unsupported as written.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FOUND, a two-stage pre-trained user targeting foundation model for Alipay. In the first stage, heterogeneous user data (behavioral sequences, tabular features, search text) are encoded with self-supervised objectives; in the second stage, the fused user representation is aligned via contrastive learning with a templated text description generated from the user's future behavior. At inference, the model supports zero-shot user targeting from a one-sentence demand and few-shot targeting via prompt tuning with seed users. The authors report results on five Alipay benchmarks spanning security, marketing, and recommendation, claiming consistent improvements over LLM-based and user-embedding baselines, and state that the model has been deployed on Alipay since 2024.","tokens_in":16414,"tokens_out":3492,"duration_ms":33012,"significance":"If the claims are supported, this would be a practically significant contribution: a single industrial user-targeting model that transfers across domains and can be queried with a natural-language demand. The paper has real strengths: it uses a large in-house dataset, describes a two-stage pretraining design that addresses training stability, and provides deployment evidence. The central forecastability claim, however, depends critically on a temporal and user-level separation between the pretraining corpus and the test benchmarks, and the manuscript does not establish that separation. The reported empirical gains therefore cannot be interpreted as evidence of forecasting or transfer as written. The paper also contains an unexplained contradictory ablation result and reports no variance or significance measures.","major_comments":[{"comment":"The central claim that FOUND is 'forecastable' requires that the 500M D_train user-text pairs are disjoint from the D_test labels in both users and time. The manuscript never states this. Eq. (4) and Eq. (14) show that each pretraining text is generated from future behavior Q_{i,t2}, while the user representation is built from history up to t1, and Appendix A.2 fixes these windows to the previous seven weeks and the following two weeks. D_test labels are defined as whether a user 'conducts the operations in the domain' (Section 5.1.1). If, for any test user, the label period overlaps with the two-week future window already used to generate that user's pretraining text, then the contrastive objective has already encoded the label in the text side of a pair, and zero-shot retrieval becomes a lookup over memorized future profiles rather than a forecast. Please specify the exact calendar periods for D_train and D_test, report the fraction of users and time windows that overlap, and present results for a split in which all D_test users and label periods are excluded from D_train.","section":"Section 5.1.1 and Appendix A.2"},{"comment":"The ablation table contradicts the component-attribution story: on D_test #5, the 'w/o Tabular' row reports 93.5 accuracy, which is 9.2 points higher than the Full model's 84.3. This is inconsistent with the claim that all data modalities contribute to performance. The authors need to explain this result or correct the table; without an explanation, the ablation does not support the conclusion that the full multi-source model is necessary.","section":"Table 5, Section 5.3.1"},{"comment":"All experimental comparisons are reported as single accuracy/precision/recall numbers without error bars, confidence intervals, or significance tests. Since the claimed improvements over baselines are sometimes a few points (e.g., 1-2 accuracy points in several D_test rows), the reader cannot assess whether the differences are reliable or within noise. Please report variance across at least a few random seeds or a paired significance test.","section":"Tables 2-5"}],"minor_comments":[{"comment":"The text says 'λ_MLM is set at 0.6 and λ_MLM is 0.4'; the second coefficient should presumably be λ_RTD. Please fix the notation.","section":"Appendix A.1"},{"comment":"The introduction and dataset description refer to 13 test scenarios, but Table 1 lists only five. Please clarify whether the remaining eight are omitted or whether the table is representative.","section":"Table 1"},{"comment":"The analysis says the model outperforms baselines by '10%/15%/10%' on Accuracy/Precision/Recall, but the actual margins in Table 2 vary considerably by scenario (e.g., roughly 3-11 accuracy points). Please state the aggregate or per-scenario calculation precisely.","section":"Section 5.2.1"},{"comment":"There is a typo in 'the seconda stage'; it should read 'the second stage'.","section":"Section 4.4.1"},{"comment":"The figure caption contains a fragment ('it' pre-trained in a two strategy') and the notation e(V) in the figure is not defined in the caption. Please make the caption self-contained.","section":"Figure 1"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, here's my take on 2412.12468.\n\nThe interesting bit is the forecastable user-text pairing: build the supervision text from future behavior while the user representation comes from history, then align them with contrastive learning. That is a genuinely sensible way to make a targeting model that can answer a one-sentence demand without retraining, and the two-stage pretraining is a reasonable way to tame the heterogeneous Alipay data. On the engineering side this is substantial work, and the deployed CTR numbers, if true, suggest it has real value in production.\n\nThe problem is that the paper never establishes that the zero-shot results measure transfer rather than memorization. The pretraining corpus is 500M pairs built from future behavior in a two-week window, and the test labels are the same kind of future-behavior labels. Nowhere in Section 5.1.1 or in the appendix is it stated that the test users and label periods are disjoint from the pretraining pairs. With that missing, the accuracy numbers in Tables 2 and 3 are exactly what you would get if the model had already seen the user's future text during pretraining. That is a load-bearing gap, and it is not a minor omission.\n\nThere are also smaller red flags. The ablation in Table 5 shows 'w/o Tabular' beating the full model on D_test #5 by 9 accuracy points; that contradicts the authors' story that all modalities help. The claimed improvements in the text ('10%/15%/10%') do not match the arithmetic in Table 2. There are no error bars, and no data or code are released.\n\nI do want to be fair: the architecture is plausible, the components are well chosen, and the writing is clear. The future-text supervision idea deserves credit. But as written, the central claim of forecastable transfer is unsupported. The right fix is to state exactly how train and test are separated in time and by user, and to rerun the ablation after correcting whatever caused that row.\n\nWho is this for? Practitioners who want a blueprint for an industrial user targeting model, and researchers who want a concrete example of the memorization-vs-transfer pitfall. It deserves a serious referee, but the referee should focus tightly on the data split and the internal consistency of the tables. If I were handling it, I would send it to review under the expectation of heavy revision or rejection as currently presented.","headline":"Interesting forecastable user-targeting idea, but the missing train/test temporal split makes the headline zero-shot results look like memorization rather than transfer.","tokens_in":17052,"tokens_out":4794,"would_cite":false,"duration_ms":38943,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single model selects targeted users from a one-sentence demand across security, marketing, and recommendation domains.","keywords":["user targeting","user understanding","self-supervised pre-training","multi-modal pre-training","contrastive learning","foundation model","forecastability"],"falsifier":"Retrain the model after deleting from the 500M-pair pre-training corpus every user who appears in the five test scenarios, including that user’s future-text descriptions; if zero-shot accuracy on those held-out users drops to near chance, the reported gains came from memorizing pre-training users rather than from forecastable transfer.","tokens_in":15909,"feed_emoji":"🎯","tokens_out":12019,"duration_ms":90509,"temperature":0.7,"pith_summary":"This paper argues that one pre-trained foundation model can replace the per-scenario classifiers normally used for user targeting. Given a single sentence describing the desired audience—for example, \"Please select potential 3C buyers\"—the model ranks candidates by the similarity between the sentence embedding and a user embedding built from payment history, mini-program usage, search text, and account attributes. The model’s forecastability is engineered through temporal separation: user representations come only from past behavior, while the text description used as a training label is generated from future behavior. The authors report that this model outperforms LLM-based and embedding-based baselines when used directly with no additional training, and also when given just a handful of seed users, across security, marketing, and recommendation scenarios on the Alipay platform, and that it has been deployed in production. If true, non-expert marketers could write one sentence and get a ranked list of users without building a new model for each campaign.","feed_headline":"A single AI model targets users from one sentence across three domains","feed_subtitle":"Trained on 500M user-history/text pairs, it can pick an audience from one sentence without per-scenario retraining.","key_machinery":"The central object is the temporal separation built into every training pair: user representation from the past window $t_1$ and text description from the future window $t_2$. The model has a user encoder that consumes behavioral sequences, tabular account features, and search text; a text encoder (a fine-tuned large language model) that consumes the templated future description; and an attention-based fusion module that combines the user encoders’ outputs. These are aligned with a contrastive loss that maximizes cosine similarity between matching user-text pairs and minimizes it for in-batch negatives. Two self-supervised objectives stabilize the user encoder before alignment: a contrastive predictive coding objective that predicts future behavioral embeddings from past context, and a cyclic regularization that keeps a user’s representation consistent across weeks; tabular features are pre-trained with masked-token and replaced-token detection. The machinery’s job is to make the user embedding encode only what is true now, while the text embedding encodes what will happen next, so that at inference a one-sentence demand about a future behavior can be matched to users who are currently on track for it.","core_discovery":"The central claim is that a single user-targeting foundation model, pre-trained with a two-stage contrastive objective on heterogeneous user data, transfers across security, marketing, and recommendation scenarios and is forecastable. The forecastability comes from the data construction: for each user, the model’s input is a representation of history—bill payments, mini-program usage, search text, and tabular account features—up to a time $t_1$, while the paired text description is generated from the user’s behavior in a future window $t_2$. After pre-training, the model maps a one-sentence demand such as “Please select potential 3C buyers” into the same embedding space as user representations, and users are ranked by cosine similarity; a few-shot variant tunes a small prompt with a triplet loss that includes hard-negative users. The authors report that this model beats LLM-based and user-embedding baselines in accuracy, precision, and recall on five real-world benchmark scenarios and on additional application scenarios, and that it has been deployed on the Alipay platform since 2024.","pith_inferences":["The two-stage recipe—self-supervised user representation first, text alignment second—could be lifted to any domain where the unit being targeted has a past state and a describable future state, such as churn prediction, credit risk, or subscription renewal.","If the forecastability claim is driven by the $t_1/t_2$ gap, then varying that gap should change the trade-off between prediction horizon and accuracy; a natural stress test is to shorten and lengthen the pre-training future window and measure zero-shot performance on the same benchmarks.","The hard-negative prompt-tuning scheme is a general way to inject business constraints into a contrastive user model, and could be reused in any setting where false positives are costly, not just user targeting.","A testable extension of the one-sentence interface is to use the aligned embedding space for explanation as well as ranking, retrieving the nearest text neighbors that justify a user’s inclusion."],"forward_implications":["Marketers can write a one-sentence demand and get a ranked user list without building scenario-specific classifiers or gathering labels.","New targeting scenarios can reuse the same pretrained model with zero additional training, provided the required user data modalities are available.","With only ten seed users (five positive, five negative), the few-shot prompt-tuning variant improves over zero-shot performance, so low-resource domains still get a usable targeting model.","Because the model’s training pairs use future text for labels and past data for representations, it can anticipate which currently visible user states lead to a described future behavior.","The model’s production deployment has already produced documented lifts in click-through rate for specific audiences."],"supporting_citations":[{"why":"Supplies the contrastive image-text alignment template that the user-text alignment stage adapts to user data.","marker":"[29]"},{"why":"Supplies the contrastive predictive coding objective that makes the user encoder forecast future behavioral embeddings.","marker":"[27]"},{"why":"Supplies the column embedding and masked/replaced-token pre-training used for tabular user features.","marker":"[13]"},{"why":"The LLM-based one-sentence targeting baseline that the paper compares against in zero-shot and few-shot tests.","marker":"[34]"},{"why":"A general-purpose user embedding baseline used for representation evaluation and few-shot targeting comparison.","marker":"[30]"},{"why":"A behavioral-sequence self-supervised baseline for user representation evaluation.","marker":"[8]"}],"fun_headline_variants":["One sentence drives user targeting across domains","Forecastable, transferable: user targeting from a prompt","Alipay's FOUND: target users with a single sentence","Cross-domain user targeting from natural language","User targeting foundation model: one sentence, any scenario"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The results establish transfer and forecasting only if the users and time windows used to build the pre-training text labels are kept disjoint from the users and labels in the test benchmarks, so the model cannot simply memorize a user’s future behavior.","fun_headline_variants_meta":{"raw":{"variants":["One sentence drives user targeting across domains","Forecastable, transferable: user targeting from a prompt","Alipay's FOUND: target users with a single sentence","Cross-domain user targeting from natural language","User targeting foundation model: one sentence, any scenario"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000293,"raw_usage":{"total_tokens":1708,"prompt_tokens":950,"completion_tokens":758,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":566,"completion_tokens_details":{"reasoning_tokens":683}},"tokens_in":566,"tokens_out":758,"duration_ms":6748,"temperature":1.0,"reasoning_tokens":683,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:02:49.062187+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the model after deleting from the 500M-pair pre-training corpus every user who appears in the five test scenarios, including that user’s future-text descriptions; if zero-shot accuracy on those held-out users drops to near chance, the reported gains came from memorizing pre-training users rather than from forecastable transfer.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the contrastive image-text alignment template that the user-text alignment stage adapts to user data."}],"review_version":1}