{"id":"cfec4cc0-181b-4379-ae29-fdcc8e4c21a6","arxiv_id":"2509.01184","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"MARS aligns text and image features of items, then augments sparse user sequences with retrieved sequences from similar heavy users, improving CTR prediction offline and online.","lead":"MARS is a system that improves click-through rate prediction for users with little history by pulling in behavior sequences from similar heavy users, using aligned image and text features to decide similarity. It shows a deployed recommender method tackling the cold-start problem at scale, with reported gains on Kuaishou's platform.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Offline Table 2 gains may be leakage: leave-one-out split is unspecified and retrieved high-active sequences can contain the target user's test item.","rationale":"The reader identified the cosine-similarity transfer assumption as the weakest point. I agree that assumption is untested, but it is partially supported by the baselines: random augmentation (Ran, Ran-S, RepPad) hurts or helps little, while retrieval-based augmentation helps, which suggests the retrieved sequences carry some signal beyond length. The more serious unaddressed risk is temporal leakage in the offline evaluation. The paper says only 'leave-one-out strategy' without specifying timestamps or a cutoff for the retrieved high-active histories. In standard leave-one-out, the held-out interaction is each user's last interaction; if the high-active user's history includes interactions after that time, the augmented training/input sequence can contain the target user's future label. Because the deployment section (§3.5) describes real-time logs but the offline protocol does not document temporal constraints, this is an open possibility. A strict temporal re-run would settle it: if the gains vanish, the central claim is not supported by Table 2; if they persist, the concern is cleared. The reader's verdict is already CONDITIONAL, and my check adds a concrete condition rather than changing the overall disposition, so the verdict remains CONDITIONAL/UNCHANGED. I would not move to REJECT without running the check, because the online A/B results, though small, are a separate source of evidence and the leakage is not yet confirmed.","tokens_in":15167,"tokens_out":9374,"duration_ms":118803,"concrete_test":"Re-run the Table 2 protocol with a strict temporal split: sort each dataset by timestamp, hold out each user's last interaction as test (leave-one-out), and construct every augmented sequence for a target low-active user at time t using only high-active-user interactions with timestamp < t; also compute the item-user similarity in Eq. 16 using target-user embeddings built only from interactions before t. If MARS's AUC advantage over the best trainable baseline (CL4SRec/CoSeRec/BASRec) does not persist, the Table 2 gains are leakage artifacts. MovieLens-1M and Amazon reviews both carry timestamps, so this is directly runnable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in Table 2 is that semantic retrieval augmentation improves CTR. The offline evaluation uses only 'the leave-one-out strategy' (§4.1.1), with no statement that retrieval is restricted to interactions before the target user's test timestamp. In §3.4, Eq. 18 prepends the entire filtered history of the most similar high-active user; in §3.5, high-active histories are built from interaction logs. If a high-active user's history contains the low-active user's held-out test item, then at evaluation time the true label item appears directly in the augmented input sequence. Since high-active users are the top 30% by sequence length, their histories are long and include many popular/overlapping items, so such collisions are plausible. The reported AUC advantage could then reflect the model recognizing the test item in its input rather than any genuine transfer of interest. This is more load-bearing than the cosine-similarity transfer assumption: a leak alone can explain the full observed gain, whereas the semantic-transfer assumption is at least indirectly supported by the random-augmentation baselines.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"MARS proposes a two-stage framework for CTR prediction under sparse user behavior. In stage 1, a Stein-kernel-based alignment module projects text and image features into a shared multimodal space and forms user embeddings by average pooling (Eq. 13). In stage 2, for each low-active user, the most similar high-active user is retrieved by cosine similarity (Eq. 15), their behavior sequence is filtered by item-user similarity (Eq. 16-17), and the filtered sequence is prepended to the low-active user's history (Eq. 18). The paper reports offline AUC gains over heuristic and trainable baselines on ML-1M, Amazon-Beauty, and Amazon-Toys (Table 2), ablation results (Table 3), and online A/B lifts at Kuaishou (Tables 4-5). The authors claim the method is deployed and code is provided.","tokens_in":15424,"tokens_out":4349,"duration_ms":51034,"significance":"If the results are valid, MARS offers a practically interesting approach to data sparsity in CTR by exploiting multimodal item semantics and cross-user sequence transfer, with the notable strength of a clear two-stage design, extensive ablation, and public code. The offline improvements over strong baselines and the positive online A/B results would support the claim of industrial applicability. However, the load-bearing offline evaluation may be compromised by label leakage due to an unspecified leave-one-out split, and the alignment loss is not fully specified. These issues must be resolved before the empirical claims can be accepted.","major_comments":[{"comment":"The leave-one-out split is described without any temporal constraint. The paper states 'adopt the leave-one-out strategy' but does not specify that retrieval for a target user excludes interactions after that user's test timestamp. Since high-active user histories are built from interaction logs (§3.5.1), a high-active user's sequence can contain the low-active user's held-out test item. Equation (18) directly prepends this sequence, so at evaluation time the true label item may appear in the augmented input. This can fully explain the offline AUC gains in Table 2, independent of any semantic transfer. Please specify the split precisely and re-run experiments with a strict 'no future interactions' retrieval constraint.","section":"§4.1.1 and §3.4, Eq. (18)"},{"comment":"The entropy regularizer is not defined. Equation (7) introduces ∇_e log q(e) as the 'approximate Stein score function' but does not give the estimator, the kernel, or the derivation. Consequently, the claim in the abstract of an 'unbiased semantic space' is unsubstantiated and the alignment module is not reproducible. Please provide the explicit Stein score estimator (e.g., spectral Stein gradient estimator) and the exact RBF kernel used, and justify the unbiasedness claim with a supporting derivation or reference.","section":"§3.2, Eq. (7)"},{"comment":"Asterisks indicate statistically significant improvements over the best trainable baseline (t-test, p<0.05), but no variance, standard deviations, or number of runs are reported. The t-test appears to be applied to point estimates only. Given the small AUC differences (e.g., ML-1M: 0.7813 vs 0.7793 for BASRec; Toys: 0.5901 vs 0.5894 for CL4SRec), reporting mean±std over multiple seeds and a paired-test setup is essential to support the significance claims.","section":"Table 2"},{"comment":"The online A/B test description is too thin to assess. The paper reports relative lifts but no p-values, confidence intervals, sample sizes, control/allocation details, or exact metric definitions. The claim that these are 'statistically significant improvements' is not verifiable from the provided information. Please add standard A/B reporting details, or explicitly state that the lifts are observed effects without significance testing.","section":"§4.5, Tables 4-5"}],"minor_comments":[{"comment":"The 'leave-one-out strategy' is ambiguous. Specify which interaction is held out for train/validation/test (e.g., last interaction per user) and how the split handles users with very short histories.","section":"§4.1.1"},{"comment":"The attention equation would be clearer if Q, K, V were explicitly defined as L(e)W_Q, L(e)W_K, L(e)W_V. As written, the softmax argument and multiplication with V are understandable but notationally inconsistent with the surrounding text.","section":"§3.1, Eq. (4)"},{"comment":"Average pooling over historical items does not specify how padding or variable-length sequences are handled. A masking note would prevent ambiguity.","section":"§3.3, Eq. (13)"},{"comment":"The 'Sparsity' column is not defined. State the formula (e.g., 1 - #Interactions/(#User*#Item)) for clarity.","section":"Table 1"},{"comment":"References [25] and [26] appear to describe the same work (SIM / Search-based User Interest Modeling). Please consolidate to avoid duplication.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The leakage concern in the offline evaluation is the most serious issue; if the leave-one-out split is not temporally constrained, the central empirical claim could be entirely an artifact of label leakage. The current manuscript does not rule this out. The online A/B reporting is also too thin to independently verify, but this is often the norm for industry papers. I recommend major revision with a specific request to either clarify the temporal split or re-run the experiments under a strict no-future-interactions constraint, and to fully specify Eq. (7)."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I read the MARS paper with interest, and the stress-test note you passed along is on target. The core idea—retrieve behavior sequences from high-active users using multimodal embeddings, then prepend a filtered version to low-active users' histories—is a reasonable extension of ReDA and the long line of retrieval-based augmentation. The careful ablations (w/o txt, w/o img, w/o stein) and the reported online A/B lifts (small but real) indicate the authors have built something that works in production. The code link is a plus, even though it is not anonymous. So there is genuine substance here.\n\nThe soft spots are not trivial. The offline protocol is the biggest one. Section 4.1.1 says only 'leave-one-out strategy' without specifying that the retrieved high-active sequences exclude the target user's held-out test item. As written, nothing prevents the high-active user's history from containing that exact test item, and the filtering step (Eq. 16-17) would then select it, because the target user clicked it and thus its embedding should be similar to the user's pooled representation. If that happens, the augmented input contains the label item, and the AUC gains in Table 2 could reflect pattern recognition instead of transfer. This is not a minor implementation detail; it is load-bearing. The online results are not subject to this leakage, and they show only 0.05-0.7% lift, so the method may still do something useful, but the offline numbers as reported are not convincing until this is ruled out.\n\nOther issues are secondary but worth noting. The 'unbiased' claim in the abstract is never defined or proved. The 'Stein kernel' label overstates Eq. (5), which is a standard RBF kernel. The Stein score estimator in Eq. (7) is under-specified—they never say how the score is computed. No error bars or standard deviations are reported despite significance asterisks. And the GitHub link is not anonymous, which is a minor violation of double-blind review.\n\nWho is this for? People working on CTR prediction or data augmentation in recommender systems, especially those facing the low-active user sparsity problem. The deployment details are useful, and the online A/B framework is a good example of industrial validation. But the paper needs a serious referee who can dig into the offline split, and the authors need to re-run experiments with an explicit temporal split or at least exclude the target item from retrieval. It deserves peer review, not desk rejection, because the problem is important and the idea has merit. I just would not cite the offline AUC numbers until the leakage question is answered.","headline":"MARS has a sensible idea and real online evidence, but the offline evaluation as written does not rule out target-item leakage, so the headline gain is not yet trustworthy.","tokens_in":15925,"tokens_out":2877,"would_cite":false,"duration_ms":34234,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that sparse click histories can be augmented with behavior sequences borrowed from similar high-active users, matched in a semantic space where item text and image features are aligned.","keywords":["CTR prediction","data augmentation","modality alignment","retrieval augmentation","low-active users","multimodal user embeddings","Stein kernel","behavior sequence"],"falsifier":"Run MARS side by side with a control that performs the same retrieval and filtering but replaces the borrowed sequence with a random sequence of the same length drawn from the same high-active user pool; if the control matches MARS's AUC, the semantic content of the borrowed items is not what drives the gain. A second, sharper check is to invert the alignment objective during pretraining (push text and image embeddings apart) and see whether retrieval-augmented AUC drops below the unaugmented base model; if it does not, alignment quality is irrelevant to the claimed effect.","tokens_in":15079,"feed_emoji":"📈","tokens_out":9297,"duration_ms":105527,"temperature":0.7,"pith_summary":"This paper tries to establish that click-through rate prediction for users with sparse interaction histories can be improved by lending them the behavior sequences of similar high-active users—provided the similarity is measured in a semantic space in which item text and image signals have been aligned. The proposed framework, MARS, works in two stages: a Stein-kernel-based alignment step projects text and image features into a shared, entropy-regularized space and fuses them with item-ID embeddings, producing average-pooled user embeddings; a retrieval step then finds the most similar high-active user, filters that user's history by item-level cosine similarity, and prepends the filtered sequence to the low-active user's history. Offline, MARS reports higher AUC than heuristic and trainable augmentation baselines on MovieLens-1M, Amazon Beauty, and Amazon Toys, with the largest relative gain on the sparsest dataset. Online, deployed on a short-video recommendation platform, it reports consistent lifts in usage-time metrics and markedly larger lifts for the low-activity segment. If correct, this means that content semantics, not just collaborative signals, can safely transfer interest across users and that sparse users can be modeled better without training extra augmentation parameters.","feed_headline":"Borrowed histories lift click prediction for sparse users","feed_subtitle":"Aligning text and image signals lets similar active users' filtered clicks fill sparse histories.","key_machinery":"The carrying machinery is a two-stage retrieval pipeline governed by a single semantic space. Stage one builds that space: text and image features are projected to a common dimension, a Stein-kernel alignment loss maximizes the RBF similarity between the two modalities while an approximate-Stein-score entropy term spreads the embeddings out, and cross-attention fuses text, image, and item-ID signals into one item embedding; a user is the average of those embeddings over her history. Stage two spends the space: for each low-active user, the high-active user with the highest cosine similarity between normalized user embeddings is retrieved, each borrowed item is re-checked against the target u","core_discovery":"The central discovery is that sequences from high-active users can be transferred to low-active users without any additional trainable augmentation parameters, as long as retrieval is performed in a semantically aligned multimodal space. MARS first aligns text and image item features using a Stein-kernel objective: an RBF kernel similarity between modalities is maximized while an entropy regularizer derived from the Stein score function prevents representation collapse; cross-attention then fuses text, image, and item-ID embeddings, and each user is represented by the average of their fused item embeddings. At retrieval time, a low-active user is matched to the high-active user with the larg","pith_inferences":["A testable implication the paper leaves open is that average pooling is not the only or best user aggregator; replacing it with attention-based pooling over fused item embeddings could sharpen retrieval and further lift CTR.","Because borrowed items are prepended without source markers, the downstream model must infer on its own that the prefix belongs to another user; adding a position or source feature could tell us whether the model relies on the borrowed prefix per se or just on longer sequences.","The framework's success suggests the aligned user embeddings could serve other cold-start tasks—candidate retrieval, follow prediction, or cross-scenario transfer—where interaction sparsity is equally the bottleneck; the paper notes the embeddings are reusable but does not quantify those uses.","The largest risk is silent mode collapse or modality dominance in the aligned space; an offline diagnostic that tracks the ratio of intra-modal to cross-modal nearest-neighbor distances under varying alignment weights would provide an early warning independent of AUC."],"forward_implications":["Data sparsity in CTR prediction can be attacked without training extra augmentation parameters: once the aligned embedding space is built, augmentation is a nearest-neighbor lookup plus a cosine-similarity filter.","Semantic consistency of borrowed items, not model complexity, is what drives gains: heuristic random or repeated padding mostly hurts, while filtered retrieval helps.","Both text and image signals plus explicit cross-modal alignment are jointly necessary; ablations removing any one component push AUC below the base model.","The best way to pack borrowed history depends on data sparsity: a single most-similar user sequence with filtering is safest on dense data, while combining several similar users helps in sparse, diverse catalogs.","Low-activity users are the main beneficiaries in production; the reported lifts on usage time and 7-day engagement metrics are much larger for that segment than for the average population."],"supporting_citations":[{"why":"Defines the DIN base CTR model whose behavior sequence MARS augments and against which gains are measured.","marker":"[45]"},{"why":"Supplies the pretrained vision encoder used to extract image features for items.","marker":"[1]"},{"why":"Supplies the pretrained language model used to encode textual item descriptions.","marker":"[35]"},{"why":"Provides the Stein gradient estimator used to build the entropy regularizer that prevents embedding collapse during alignment.","marker":"[19]"},{"why":"Provides the entropy-bottleneck idea referenced for the approximate Stein-score entropy regularizer.","marker":"[39]"},{"why":"Defines the CL4SRec contrastive augmentation baseline and the crop/mask/reorder operators MARS is compared against.","marker":"[41]"},{"why":"Defines the CMRSI and CoSeRec augmentation baselines using item-correlation substitute and insert operations.","marker":"[21]"},{"why":"Supplies BASRec, the balanced mixup augmentation baseline MARS outperforms among trainable methods.","marker":"[12]"},{"why":"Supplies the RepPad repeated-padding heuristic baseline used in comparisons and padding-strategy analysis.","marker":"[10]"}],"fun_headline_variants":["Multimodal alignment enriches sparse click histories","Borrow sequences from active users via aligned signals","Align text–image to transfer user behavior sequences","Sparse click paths filled by cross-modal retrieval"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The load-bearing premise is that cosine similarity between average-pooled, CTR-trained multimodal user embeddings is a faithful measure of interest compatibility, so that a high-active user's filtered history genuinely helps a low-active user rather than injecting noise.","fun_headline_variants_meta":{"raw":{"variants":["Multimodal alignment enriches sparse click histories","Borrow sequences from active users via aligned signals","Align text–image to transfer user behavior sequences","Sparse click paths filled by cross-modal retrieval"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00022,"raw_usage":{"total_tokens":1300,"prompt_tokens":778,"completion_tokens":522,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":522,"completion_tokens_details":{"reasoning_tokens":463}},"tokens_in":522,"tokens_out":522,"duration_ms":6358,"temperature":1.0,"reasoning_tokens":463,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T12:47:50.117846+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run MARS side by side with a control that performs the same retrieval and filtering but replaces the borrowed sequence with a random sequence of the same length drawn from the same high-active user pool; if the control matches MARS's AUC, the semantic content of the borrowed items is not what drives the gain. A second, sharper check is to invert the alignment objective during pretraining (push text and image embeddings apart) and see whether retrieval-augmented AUC drops below the unaugmented base model; if it does not, alignment quality is irrelevant to the claimed effect.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the DIN base CTR model whose behavior sequence MARS augments and against which gains are measured."},{"cited_title":"MVEB: Self-Supervised Learning with Multi-View Entropy Bottleneck","cited_arxiv_id":"2403.19078","evidence_quote":"Provides the entropy-bottleneck idea referenced for the approximate Stein-score entropy regularizer."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the CL4SRec contrastive augmentation baseline and the crop/mask/reorder operators MARS is compared against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the RepPad repeated-padding heuristic baseline used in comparisons and padding-strategy analysis."}],"review_version":1}