{"id":"495e4c30-46d0-4721-952e-865b223b3247","arxiv_id":"2607.17473","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"RAMP splits ad prediction into personalized and non-personalized towers, then uses a distillation-style loss to transfer knowledge so non-personalized predictions stay accurate when user features are absent.","lead":"RAMP is a training method that keeps click-through and conversion prediction accurate when user-linked features (age, gender, IDs) are removed, by training two separate prediction paths and aligning their outputs. It reports consistent gains on ad datasets when personalized features are missing, while remaining competitive when they are available.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"As described, the public-benchmark masking protocol may let models exploit train/test overlap between personalized and non-personalized versions of the same users, so the claimed gains may not transfer to real non-consented traffic; IndustryAd's small gain is the only non-leaky evidence.","rationale":"The reader's CONDITIONAL verdict is appropriate and I do not propose changing it. My concern sharpens the reader's weakest assumption: it is not just that non-consenting users may behave differently, but that the masking protocol as described creates a specific train/test overlap—the same samples or users appear in both the personalized and masked non-personalized sets. This is a concrete experimental-design issue that can be tested directly with a user-disjoint split. The method itself is plausible: the dual-tower design and prediction alignment are reasonable, and the IndustryAd result provides some positive evidence under realistic conditions. However, the headline claim is broader than the clean evidence. The paper's own caveats (Sections 4.1.2 and 5) support this reading. I did not find an internal mathematical contradiction or a reason to reject outright. The condition should be that the public-benchmark results are confirmed under a split with no user overlap between personalized training data and non-personalized evaluation data, or that the abstract is revised to reflect the smaller, more honest natural-traffic gain.","tokens_in":20136,"tokens_out":9217,"duration_ms":83020,"concrete_test":"Re-run the public-benchmark comparisons under a user-disjoint consent split. Partition users (e.g., by device_ip on Avazu, userid on TaobaoAd, user_id on CriteoPrivateAd) into a consented group, whose full features are used during training, and a non-consented group, whose samples are only ever seen with personalized features removed and never have a corresponding all-feature version in either training or validation. Compare RAMP against the best baseline and PP on the non-consented test users' AUC, with standard deviations and significance tests. If RAMP's advantage falls below the 0.1% 'meaningful' threshold or shrinks to the level of the IndustryAd result (+0.14 AUC), the public-benchmark gains are an artifact of the masking protocol, and the abstract's broad claim should be narrowed to the natural-traffic setting.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that RAMP consistently outperforms state-of-the-art methods 'when personalized features are missing.' The public-benchmark evidence for this claim rests on the masking protocol in Section 4.1.2. As written, a 'pure non-personalized dataset' is created by masking personalized fields and then combined with the original personalized dataset into a mixed training/evaluation set. This means each non-personalized example is a masked version of an example (or of a user) that also appears with full personalized features in the personalized portion of the data. During training, the model can learn a mapping from the remaining non-personalized features to the label either from the identical sample's fully-featured version or from other samples of the same user. At test time, non-personalized samples from those same users are then predicted using that learned mapping. Real non-consented traffic consists of users for whom no such personalized record exists, so this overlap is a form of evaluation leakage relative to the deployment scenario. The paper itself acknowledges the proxy limitation in Section 4.1.2 and Section 5, but the numerical evidence shows how much this matters: on the synthetic CriteoPrivateAd benchmark RAMP gains +0.87 AUC over the best baseline, while on the natural-traffic IndustryAd dataset—where no such overlap exists—the gain is only +0.14 AUC. The abstract's broad claim is therefore not yet supported outside the masking protocol.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RAMP, a model-agnostic training framework for CTR/CVR prediction when personalized features are unavailable at inference. RAMP combines (i) a dual-tower 'personalized pathway' whose two towers are trained with output masking so that one specializes in personalized traffic and the other in non-personalized traffic; (ii) a separate non-personalized pathway trained only on non-personalized features; and (iii) a logit-level prediction-alignment loss that distills knowledge from the personalized pathway into the non-personalized side. At inference, the non-personalized tower is intended to serve traffic without personalized features. Experiments on Avazu, TaobaoAd, CriteoPrivateAd, and a private IndustryAd dataset report consistent non-personalized AUC gains over strong baselines (from +0.10 to +0.87 percentage points), competitive personalized AUC, and ablations showing the contributions of the dual-tower and alignment components. The code is publicly available.","tokens_in":20559,"tokens_out":8167,"duration_ms":80542,"significance":"If the results hold, RAMP is a practically useful, model-agnostic technique for ad ranking under consent-driven feature restrictions, with no extra inference-time cost. The paper's strengths include public code, evaluation on four datasets (including a natural-traffic industrial dataset), multiple backbone integrations, and a clear ablation of the two pathway components. The central idea—aligning a non-personalized predictor with a personalized predictor at the logit level—is simple and plausible, and the reported gains are consistent across datasets. However, the strongest public-benchmark evidence relies on a synthetic masking protocol whose validity for real non-consented traffic is the main open question. The natural-traffic IndustryAd gain is much smaller, and several reporting issues (missing error bars, a missing appendix for an A/B claim, and incomplete feature-category details) currently prevent full confidence in the quantitative claims.","major_comments":[{"comment":"The public-benchmark evidence is obtained by masking personalized fields of the same samples/users that also appear in personalized form in the training data. This evaluates robustness to feature absence under a matching population, not generalization to real non-consented traffic, where users may have different behavior. The paper acknowledges this in Section 4.1.2 and Section 5, but the abstract's claim 'when personalized features are missing' is broader. The natural-traffic IndustryAd result is much smaller (+0.11 AUC over the best baseline vs +0.87 on CriteoPrivateAd). Please either temper the abstract to 'under synthetic feature-masking and one natural-traffic deployment' or provide additional analysis (e.g., stratified results by user overlap or a covariate-shift experiment) to support the broader claim.","section":"Section 4.1.2 / Abstract"},{"comment":"The validity of the masking simulation depends entirely on which fields are labeled personalized vs non-personalized, but Table 1 lists only example field names. For Avazu, if a quasi-identifier such as device_id is retained in the non-personalized set, the 'non-personalized' input can still carry user/device identity and the reported gains would be inflated. Provide a complete list of personalized and non-personalized fields for all datasets and justify the categorization, especially for Avazu and TaobaoAd where device-level identifiers exist.","section":"Section 4.1.2 / Table 1"},{"comment":"The paper states that all RAMP improvements are statistically significant at p<1e-3, but no standard deviations, confidence intervals, or test details are reported. Several gains are 0.10–0.22 percentage points, which may or may not be significant across 5 seeds. Report mean and variance (or per-seed values) for all tables, state the exact test procedure, and clarify whether the comparisons are paired over seeds.","section":"Section 4.2 / Table 2"},{"comment":"The claim that 'RAMP was A/B tested in industry production for CVR prediction, the total advertiser value (TAV) improved over 3% (See Appendix??)' points to a missing appendix. This is an externally validated claim that cannot be checked. Either include the appendix with experimental details (duration, traffic split, metric definition, significance) or remove the sentence.","section":"Section 4.2"},{"comment":"There is a direct contradiction about inference. Section 4.1.4 says 'during validation and testing, only the foundational dual-tower component is used ... The non-personalized pathway is thus not required during inference,' while Section 4.1.6 says 'when personalized features are unavailable, only the non-personalized pathway is activated.' This ambiguity affects the architecture description and the privacy-compliance claims. Clarify which component serves non-personalized traffic at inference.","section":"Section 4.1.4 vs Section 4.1.6"}],"minor_comments":[{"comment":"The phrase 'The training cost of the method is introduced in Appendix??' has a dangling appendix reference. Either add the appendix or remove the pointer.","section":"Section 4.1.4"},{"comment":"The x-axis label 'Distance Loss Weight' would be clearer as 'β'; the caption should state which backbone is used in each panel, as it currently only mentions 'RAMP-FCN' and 'RAMP-PNN' in the body text.","section":"Figure 3"},{"comment":"The sentence 'The proposed framework is evaluated in three configurations based on the backbone model: PP ... and RAMP' lists only two configurations. Please rephrase.","section":"Section 4.1.4"},{"comment":"The t-SNE visualization does not specify which embeddings are plotted—those from the personalized pathway, the non-personalized pathway, or a concatenation. Since the non-personalized pathway is not used at inference, the claim of 'feature separation' needs a precise definition of what is being visualized.","section":"Section 4.5"},{"comment":"References [16] and [17] are duplicates of the same Hinton et al. work; consolidate.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The core method is sound and the empirical story is promising, but the manuscript currently overclaims relative to the evidence: the synthetic masking protocol is the main source of the large gains, the natural-traffic evidence is thin, and the statistical and appendix claims are not verifiable. The missing A/B appendix and the incomplete field-level categorization should be addressed before publication. I would be willing to review a revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper you should know about: RAMP, a training-time framework for CTR/CVR when personalized features are unavailable at inference. What's genuinely new is the combination — a dual-tower personalized pathway with output masking, a separate non-personalized pathway, and logit-level alignment between them. Each piece is known; the package is not. The paper does solid work: three public datasets plus an industrial one, multiple backbones, comparisons against KD/PFD/HAPFD, ablations on the alignment weight and on gradually dropping personalized features, and code released. The gains on non-personalized AUC are consistent and the KD comparison is fair. That is real credit. The soft spots are proportionate. The public-benchmark protocol mixes masked and original versions of the same users in training and test, which lets the model learn a mapping from the masked to the full version of effectively the same examples. Real non-consented traffic doesn't have that correspondence. The authors flag the masking limitation in Section 4.1.2, but the numbers show how much it matters: RAMP gains +0.87 AUC on synthetic CriteoPrivateAd, yet only +0.14 on the natural IndustryAd data. The abstract's 'consistently outperforms' claim is therefore only supported under the synthetic protocol, not yet for actual consent-restricted deployment. Two smaller issues: no error bars in any table (despite five seeds, they only claim t-test significance), and the A/B TAV lift over 3% points to a missing appendix — unverifiable as printed. The private IndustryAd is fine as supplementary evidence, but it carries weight the paper asks it to carry alone on the naturalistic front. If you're working on feature-availability robustness or privileged-feature distillation, this is worth reading for the architecture and the negative results on feature-level alignment. The central idea is plausible, but the headline needs hardening with a cleaner evaluation, e.g., a held-out cohort of users whose personalized features are never seen in training. That would also give the industrial claim a stronger base. Yes, send it to peer review — it deserves a serious referee, and the concerns are fixable. I would cite the architecture with a caveat, and bring it to reading group maybe; the methodology discussion on leakage is valuable for anyone doing masked-feature evaluation.","headline":"Useful architecture with honest caveats, but the headline robustness claim leans on a masking protocol that overstates real-world transfer; the natural-traffic evidence is much weaker.","tokens_in":654,"tokens_out":1676,"would_cite":true,"duration_ms":23922,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"When personalized features are stripped away, RAMP keeps ad CTR/CVR accuracy near its full-feature level, outperforming best baselines by 0.10–0.87% AUC.","keywords":["click-through rate prediction","conversion rate prediction","non-personalized recommendation","privacy constraints","feature masking","knowledge distillation","dual-tower architecture","online advertising"],"falsifier":"Log real traffic with true consent labels, run RAMP and the best baseline on the non-consenting subset only, and compare non-personalized AUC; if RAMP's advantage disappears, reverses, or falls below roughly 0.1% AUC, the transfer effect demonstrated under synthetic masking would not carry over to actual deployment.","tokens_in":1408,"feed_emoji":"🎯","tokens_out":1269,"duration_ms":60349,"temperature":0.7,"pith_summary":"Online ad systems predict clicks and conversions using user-linked signals such as IDs, device attributes, and behavior history, but privacy regulations increasingly make those signals unavailable for some or all traffic. The paper argues that accuracy under this constraint does not have to collapse: it proposes RAMP, a training scheme that separates the learning signal for personalized and non-personalized traffic and then aligns them. On four datasets spanning CTR and CVR prediction, RAMP improves non-personalized AUC by 0.10% to 0.87% over the best baseline, with all gains statistically significant, and it stays competitive when full features are available. The practical point is that a serving system can keep using a single low-latency model while a training-time auxiliary pathway transfers knowledge from rich, consented data to the restricted setting. The catch is that the public-benchmark experiments simulate missing features by masking, and the paper openly notes this may not capture all real differences of non-consenting traffic.","feed_headline":"RAMP recovers ad accuracy when personalized features vanish","feed_subtitle":"Training with dual towers and logit alignment keeps CTR/CVR prediction accurate when consent blocks user-linked features.","key_machinery":"The load-bearing mechanism is the masking-and-alignment pathway pair: a personalized pathway containing two parallel towers with shared embeddings but independent parameters, whose outputs are gated by a per-sample mask so that tower A only receives supervision on personalized traffic and tower B only on non-personalized traffic; a separate non-personalized pathway that consumes non-personalized features only; and a distance loss computed as the mean absolute difference between the two pathways' logits on non-personalized samples, formulated through a cross-entropy-style objective to align their final predictions. The key design choice is logit-level rather than feature-level alignment, whic","core_discovery":"RAMP's central claim is that a model can be trained to serve accurately when personalized features are absent by using a dual-tower personalized pathway with output masking—tower A learns on personalized traffic, tower B on non-personalized traffic using the same inputs—plus a separate non-personalized pathway trained only on non-personalized features, while a distillation-inspired L1 logit-level alignment loss pulls the two pathways' predictions together. During training only, the non-personalized pathway absorbs knowledge from the richer personalized pathway; at inference it is discarded and the dual-tower component alone makes predictions, choosing the appropriate tower according to featu","pith_inferences":["If masking faithfully mimics consent-constrained traffic, the method implies that platforms could offer near-personalized ad quality on non-consenting traffic without storing user-linked data—a direct improvement in the privacy–quality trade-off.","The success of logit-level over feature-level alignment suggests a transferable design principle: when two input regimes differ, align final predictions rather than internal representations; this is testable in other missing-feature prediction problems.","The mild degradation under progressive feature removal hints that RAMP could support gradual tightening of consent policies rather than only binary all-or-nothing feature availability.","The smaller gain on the one natural-traffic dataset compared with the synthetic-masking benchmarks suggests real-world improvements may be quieter than headline numbers, pointing to the value of testing on naturally non-consenting traffic."],"forward_implications":["RAMP is model-agnostic: it improves non-personalized prediction when plugged into each of the evaluated backbone models.","The gains span task types and scales: CTR on two public datasets and CVR on one public and one industrial dataset, with the largest AUC gain on the largest dataset.","Inference is unchanged: only the dual-tower component is needed at serving time, so the privacy-constrained regime adds no extra latency or model complexity at runtime.","Even with heavy personalized-feature loss during training, non-personalized AUC degrades only slightly, suggesting graceful handling of partial feature availability.","An industry A/B test reported in the paper shows total advertiser value improving by more than 3% with RAMP."],"fun_headline_variants":["RAMP masks and aligns to keep ads accurate sans personal data","Ad targeting without personal data? RAMP says yes via alignment","Dual-tower masking: RAMP preserves ad accuracy under privacy limits","When personal features vanish, RAMP's alignment pathway saves CTR/CVR"],"cache_read_input_tokens":22272,"weakest_assumption_plain":"The paper's headline result rests on assuming that removing personalized features from otherwise ordinary traffic is a faithful stand-in for traffic from users who actually withheld consent; the authors themselves state that masking 'may not fully capture all characteristics of real non-consented traffic.'","fun_headline_variants_meta":{"raw":{"variants":["RAMP masks and aligns to keep ads accurate sans personal data","Ad targeting without personal data? RAMP says yes via alignment","Dual-tower masking: RAMP preserves ad accuracy under privacy limits","When personal features vanish, RAMP's alignment pathway saves CTR/CVR"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000185,"raw_usage":{"total_tokens":1185,"prompt_tokens":796,"completion_tokens":389,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":540,"completion_tokens_details":{"reasoning_tokens":314}},"tokens_in":540,"tokens_out":389,"duration_ms":4765,"temperature":1.0,"reasoning_tokens":314,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T17:51:11.370176+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Log real traffic with true consent labels, run RAMP and the best baseline on the non-consenting subset only, and compare non-personalized AUC; if RAMP's advantage disappears, reverses, or falls below roughly 0.1% AUC, the transfer effect demonstrated under synthetic masking would not carry over to actual deployment.","supporting_citations":[],"review_version":1}