{"id":"08d24065-df54-450e-b722-bfc960777bf6","arxiv_id":"2509.03187","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A plug-in contrastive loss that enforces monotonicity between numerical features and recommender outputs, via counterfactual sample synthesis, improves AUC, GAUC, and monotonicity.","lead":"The paper adds a training-time rule to recommender models: larger numerical feature values (like a video's click count) must lead to larger predicted scores. By synthesizing modified 'counterfactual' examples and adding a contrastive ranking loss, the authors report large accuracy gains on public and industrial datasets plus a live deployment.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Fixed increasing-monotonicity assumption for all numerical features (Sec. 4.1) is unvalidated; if any feature is non-monotonic or inverted, the pairwise and label-augmentation losses (Eqs. 7–9) inject mis-specified supervision, undermining the claimed general effectiveness.","rationale":"The reader identified the monotonicity-direction assumption as the weakest point; I agree that this is the most load-bearing concern because it underpins both the loss construction and the interpretability claim, and the circular Mono_rate metric cannot validate it. The proposed test—partial dependence on a baseline model—directly checks the assumption. Since the paper's empirical results are consistent with the assumption holding on the tested datasets, but the paper does not justify it or study robustness, the appropriate verdict remains CONDITIONAL pending this validation.","tokens_in":11382,"tokens_out":8842,"duration_ms":96930,"concrete_test":"For each numerical feature in the industrial dataset and for KuaiRand's numerical features, train a baseline DNN (without CCSS) on the training set, then compute partial dependence plots of the predicted score with respect to each feature on a held-out sample, varying the feature while holding others at their observed values. If any feature's partial dependence is non-monotonic (e.g., decreases over part of its range or changes sign), the fixed increasing assumption is violated. A follow-up experiment: re-run DNN+CCSS with that feature excluded, or with the per-feature direction learned from the sign of the Spearman correlation; if AUC drops significantly relative to Table 5, the assumption is load-bearing; if not, the method is robust to the violation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—that CCSS improves AUC/GAUC across all baselines and datasets—depends on the assumption in Section 4.1 that for every numerical feature 'the larger the input value, the larger the output result should be.' This monotonic-increasing direction is wired into the sample synthesis (Section 3.2.2, Table 2), the pairwise hinge losses (Section 3.3.1, Eqs. 7–8), and the label assignment for factual samples (Eq. 9, where a factual sample for a positive original is labeled positive). The authors do not verify that each feature in KuaiRand or the industrial dataset actually satisfies this condition. If a feature has a non-monotonic or inverted relationship with the target, then whenever that feature is selected for perturbation, the pairwise losses push in the wrong direction and the synthetic factual sample receives an incorrect label, actively corrupting learning on those samples. Because the sampling probability is proportional to Shapley importance (Section 3.2.1), features with the largest, possibly incorrect, influence are corrupted most often. The reported gains (e.g., DNN +30% RelaImpr in Table 5) could be driven by the majority of features being monotonic while the method silently degrades others; the framework would not then be general or model-agnostic. The Mono_rate metric (Eq. 13) evaluates exactly the (F,O) and (C,O) pairs that the loss optimizes, so the interpretability improvements (Table 5) are circular and provide no independent evidence that the monotonicity direction is correct.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CCSS (Contrastive learning with Counterfactual Samples Synthesizing), a model-agnostic framework for imposing monotonicity between neural network outputs and numerical features in recommender systems. For each training instance, CCSS synthesizes a counterfactual sample and a factual sample by perturbing one numerical feature (chosen with probability proportional to Shapley importance) to the center of an adjacent bucket, assigns the factual sample the original label, and adds pairwise hinge losses to rank the original, factual, and counterfactual samples consistently with the assumed monotonic direction. The authors evaluate CCSS on KuaiRand-Pure and a large industrial dataset with five backbones (DNN, Wide&Deep, PNN, DCN, DeepFM), reporting consistent AUC/GAUC improvements and large Mono_rate gains, plus a 3.93% collect_rate improvement in an online A/B test.","tokens_in":11832,"tokens_out":3338,"duration_ms":37613,"significance":"The problem is practically important: numerical features are ubiquitous in CTR prediction, and enforcing sensible monotonic behavior can improve both effectiveness and interpretability. The proposed method is simple, model-agnostic, and demonstrates consistent offline gains across several architectures, which is a meaningful empirical result. The online deployment result is also valuable. However, the interpretability claim is currently supported only by Mono_rate, a metric that is essentially the training objective's own pairwise accuracy, and the method rests on a fixed and unvalidated monotonicity-direction assumption. If these weaknesses are addressed with independent evaluation and per-feature validation, the framework would be a solid contribution to industrial recommender systems.","major_comments":[{"comment":"The Mono_rate metric is circular with respect to the training objective. Eq. (13) defines Mono_rate over exactly the (F,O) and (C,O) pairs that the pairwise hinge losses in Eqs. (7)–(8) and the augmented pointwise loss in Eq. (9) explicitly optimize. Thus the large Mono_rate gains in Table 5 are a direct measure of how well the model fits the training-time ranking constraints, not an independent evaluation of interpretability. To support the interpretability claim, please provide a separate evaluation, e.g., monotonicity checks over all numerical feature values (not only adjacent-bucket neighbors), or human/domain validation that the predicted scores respect the expected monotonic relationships for arbitrary feature perturbations.","section":"§3.3.1, §4.4.1, Eq. (13) vs Eqs. (7)–(9)"},{"comment":"The method assumes a fixed monotonicity direction: 'the larger the input value, the larger the output result should be' for all numerical features in both datasets (Sec. 4.1). This direction is wired into sample synthesis (Table 2), the pairwise losses (Eqs. 7–8), and the label assignment for factual samples (Eq. 9). No evidence is given that every feature satisfies this monotone-increasing assumption. If any feature has an inverted or non-monotonic relationship, the losses provide incorrect supervision for all samples where that feature is perturbed, and since features are sampled with probability proportional to Shapley importance, the damage is concentrated on the most influential features. Please validate the monotonicity direction per feature, or extend the method to infer or adapt the direction from data, and report sensitivity to this assumption.","section":"§4.1, §3.2.2, Table 2"},{"comment":"Table 7 appears to mislabel the full model as 'DNN'. The first row reports AUC=0.786 and GAUC=0.732, which match the DNN+CCSS values in Table 5 (0.7860/0.7320), whereas the actual DNN baseline in Table 5 is 0.7200/0.6560. As a result, the ablation does not compare the ablated variants against the true baseline, and the phrase 'confirm the advantages of our proposed random strategy' is not supported by the table as presented. Please correct the row labels and include the actual DNN baseline.","section":"§4.5, Table 7"}],"minor_comments":[{"comment":"The sentence is broken: '...removing phrases, which alThe generated counterfactual samples...' — likely a missing continuation or formatting error.","section":"§2.3"},{"comment":"The text says 'For all the three datasets' but only two datasets are used. Please correct to 'two datasets'.","section":"§4.3.1"},{"comment":"The Shapley value computation is not described. Please specify how feature importance q_i is calculated for numerical features (e.g., which background distribution, which model output, and how many feature coalitions are sampled).","section":"§3.2.1"},{"comment":"The ablation variants have inconsistent naming: 'CCSS(Only Data Augmentation)' in the text corresponds to 'DNN(Only Factual Pointwise loss)' in Table 7. Please unify the terminology.","section":"§4.5 and §4.3.2"},{"comment":"References [2] and [3] are the same paper (Cheng et al., Wide & Deep). Please deduplicate.","section":"References"},{"comment":"The hyperparameter α is set to 1.0 in all experiments, but Figure 3 shows sensitivity. Please state whether α=1.0 was chosen based on a validation set for each backbone or uniformly.","section":"§4.3.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is from an industrial lab and reports a deployed system, which is notable. The main technical concern is not the method's effectiveness but the validity of the interpretability evaluation: Mono_rate is essentially the training objective's accuracy, so the interpretability claim is not independently evidenced. The monotonicity-direction assumption is also a potential correctness risk, especially for a 'general' framework. These are fixable with additional validation and a more careful evaluation design, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a read. CCSS is a simple idea: for each training sample, pick a numerical feature according to Shapley importance, synthesize a factual sample by moving to the right bucket center and a counterfactual by moving left (for increasing monotonicity), then add pairwise hinge losses so the model scores factual > original > counterfactual. It's genuinely model-agnostic — they test DNN, Wide&Deep, PNN, DCN, DeepFM — and the reported AUC/GAUC gains are large and consistent across backbones, including a 3.93% collect_rate gain online. That's a useful contribution, especially for industrial teams that want monotonicity without architectural constraints.\n\nThe real soft spot is the evaluation of interpretability. Mono_rate counts the fraction of (F,O) and (C,O) pairs correctly ordered on the test set — exactly the quantity the pairwise losses optimize. So the 'interpretability improvement' is partly circular; it tells you the model learned the training objective, not that the monotonic direction is semantically correct. Independent evidence is missing. Relatedly, Section 4.1 asserts 'the larger the input value, the larger the output result' for all numerical features in both datasets, but doesn't verify this. If a feature were inverted or non-monotonic, the loss would push in the wrong direction. The paper's Table 2 shows the decreasing case, so they know how to handle it, but they don't report which features use which direction. Given the consistent AUC gains, the monotonicity assumption is probably mostly right, but it's an assumption.\n\nOther issues are minor: the ablation table seems mislabeled (the 'DNN' row matches DNN+CCSS in Table 5), the Shapley computation isn't specified, and the online test reports only a single number with no significance or duration. None of these undercut the main result, but they need tightening.\n\nWho's this for? People building deep CTR models with numerical features who want a cheap regularizer that encodes prior knowledge. The paper is from a strong industrial group and the method is easy to implement. It deserves a serious referee; the evaluation needs work, but the idea is sound.","headline":"A practical plug-in regularizer for monotonic numerical features in recommenders, with large reported gains; the interpretability metric is circular and the fixed monotonicity direction is unvalidated, but the core idea is solid.","tokens_in":12265,"tokens_out":3352,"would_cite":true,"duration_ms":33053,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CCSS enforces known monotonicity between numerical features and recommender scores by contrasting counterfactual samples; five model architectures plus an online A/B test report gains.","keywords":["recommender systems","numerical features","monotonicity","counterfactual samples","contrastive learning","CTR prediction","interpretability","Shapley value"],"falsifier":"Take a dataset with a numerical feature whose true effect is non-monotonic, such as a recommendation score that rises then falls with video length, or deliberately invert one feature's relationship with the label so the ground-truth effect is decreasing. Train the same backbone with and without CCSS assuming increasing monotonicity for that feature: if the monotonicity assumption is load-bearing, CCSS should lower AUC or GAUC and worsen Mono_rate on the violating feature relative to the backbone alone, whereas generic data augmentation would not produce that reversal.","tokens_in":11320,"feed_emoji":"📈","tokens_out":14333,"duration_ms":140514,"temperature":0.7,"pith_summary":"The paper sets out to prove a simple prior can be taught to any recommender: if a numerical feature such as likes, watch time, or click count is higher, the model's predicted score should be higher too. It argues that this monotonicity is central to interpretability and that enforcing it explicitly, rather than leaving it to chance, also improves prediction accuracy. The proposed framework, CCSS, synthesizes counterfactual and factual versions of each training sample by shifting one numerical feature in the expected direction or the opposite, then trains the network to rank the three samples accordingly. On a public benchmark and a 1.29-billion-sample industrial dataset, every tested backbone improves in AUC, GAUC, and a monotonicity-rate metric, and an online A/B test reports a 3.93% collect-rate gain. The load-bearing assumption is that each feature's monotonic direction is known and fixed.","feed_headline":"Contrasting counterfactual samples lifts AUC across five architectures","feed_subtitle":"Counterfactual training makes predictions respect known feature monotonicity, improving interpretability and accuracy.","key_machinery":"The central mechanism is pairwise contrastive ranking over locally synthesized samples. Given an original training sample, the Counterfactual Sample Synthesizer picks one numerical feature with probability proportional to its Shapley-value importance and moves that feature to the center of the neighboring discretization bucket: toward the expected direction for a factual sample (assigned the original label) and away from it for a counterfactual sample (unlabeled). Hinge losses then enforce, for positive labels under increasing monotonicity, that the factual sample scores higher than the original and the original higher than the counterfactual; the order reverses for negative labels or decrea","core_discovery":"The central claim is that monotonicity between numerical features and the neural network output is a learnable, model-agnostic objective rather than a network-architecture constraint. CCSS does this in two stages: during training, for each sample it selects one numerical feature with probability proportional to its Shapley-value feature importance, then moves that feature to the center of an adjacent discretization bucket, producing a counterfactual sample (moved opposite to the expected effect) and a factual sample (moved in the expected direction). Hinge pairwise losses enforce the ranking factual greater than original greater than counterfactual for monotonic-increasing features, with the","pith_inferences":["If the gains come from enforcing monotonicity rather than generic data augmentation, the same recipe should transfer to other ranking targets such as likes, follows, or dwell-time prediction whenever the numerical features have a defensible monotonic direction; this is a testable claim beyond the paper's collect-rate and CTR evidence.","The fixed global monotonicity direction is the framework's boundary: features with non-monotonic effects, such as engagement that peaks at mid-range values, would need per-feature direction learning or segment-wise monotonic constraints. A synthetic experiment with a known inverted feature would cleanly separate the monotonicity mechanism from generic augmentation.","Shapley-value importance sampling adds computational cost; a cheaper surrogate, such as approximate feature attribution or learned importance, could make CCSS practical in higher-throughput settings while preserving the contrastive ranking signal."],"forward_implications":["On both datasets, every backbone improves when CCSS is added; the paper reports relative-to-chance (RelaImpr) gains of at least 6.0% in AUC and 4.5% in GAUC.","Mono_rate, the fraction of factual/original/counterfactual triples ranked in the expected order, rises sharply for the top seven important numerical features, indicating improved interpretability of ranking decisions.","Ablation results show that removing any component (factual pointwise augmentation, factual contrastive loss, counterfactual contrastive loss) or replacing Shapley-weighted feature selection with equal-probability selection lowers AUC and GAUC relative to the full method.","Online A/B testing on a collect prediction model shows a 3.93% collect-rate gain when CCSS is added to an already optimized DCN model, suggesting the offline gains transfer to live traffic.","Because CCSS synthesizes samples locally for each training instance and only adds losses on those synthesized inputs, it can be added to an existing CTR or engagement model without changing the model's architecture."],"supporting_citations":[{"why":"Supplies the counterfactual-sample-synthesizing idea that CCSS adapts from visual question answering to recommendation.","marker":"[1]"},{"why":"Defines the Wide & Deep backbone used as one baseline and as an example of dense numerical-feature input.","marker":"[3]"},{"why":"Provides the AUC evaluation metric used as the main offline effectiveness measure.","marker":"[6]"},{"why":"Gives Shapley value feature importance, which CCSS uses to set the probability of disturbing each numerical feature.","marker":"[7]"},{"why":"Motivates numerical-feature embedding learning via discretization; CCSS perturbs within those learned buckets.","marker":"[8]"},{"why":"Defines the DeepFM backbone used as a baseline.","marker":"[9]"},{"why":"Defines the PNN backbone used as a baseline.","marker":"[14]"},{"why":"Provides Shapley effects and global sensitivity analysis theory supporting the feature-importance sampling step.","marker":"[18]"},{"why":"Defines the DCN backbone and supplies the RelaImpr relative-improvement metric used in comparisons.","marker":"[20]"},{"why":"Introduces the GAUC intra-user ranking metric used for offline and online relevance evaluation.","marker":"[22]"}],"fun_headline_variants":["Model-agnostic counterfactual contrast lifts recommender AUC","Contrasting counterfactual samples enforces feature monotonicity","Two-stage counterfactual training sharpens recommender predictions","Counterfactual contrastive learning teaches monotonicity to any model","Shapley-weighted counterfactuals boost recommender AUC and interpretability"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The framework assumes the monotonic direction of every numerical feature is known in advance and fixed—in all experiments, larger input values should give larger predictions—and that moving a feature in that direction keeps the original label valid; if a feature is inverted or non-monotonic, the contrastive supervision pushes the model in the wrong direction.","fun_headline_variants_meta":{"raw":{"variants":["Model-agnostic counterfactual contrast lifts recommender AUC","Contrasting counterfactual samples enforces feature monotonicity","Two-stage counterfactual training sharpens recommender predictions","Counterfactual contrastive learning teaches monotonicity to any model","Shapley-weighted counterfactuals boost recommender AUC and interpretability"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000615,"raw_usage":{"total_tokens":2646,"prompt_tokens":645,"completion_tokens":2001,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":389,"completion_tokens_details":{"reasoning_tokens":1912}},"tokens_in":389,"tokens_out":2001,"duration_ms":13949,"temperature":1.0,"reasoning_tokens":1912,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T11:05:33.252031+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a dataset with a numerical feature whose true effect is non-monotonic, such as a recommendation score that rises then falls with video length, or deliberately invert one feature's relationship with the label so the ground-truth effect is decreasing. Train the same backbone with and without CCSS assuming increasing monotonicity for that feature: if the monotonicity assumption is load-bearing, CCSS should lower AUC or GAUC and worsen Mono_rate on the violating feature relative to the backbone alone, whereas generic data augmentation would not produce that reversal.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the counterfactual-sample-synthesizing idea that CCSS adapts from visual question answering to recommendation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives Shapley value feature importance, which CCSS uses to set the probability of disturbing each numerical feature."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Motivates numerical-feature embedding learning via discretization; CCSS perturbs within those learned buckets."},{"cited_title":"In 2016 IEEE 16th International Conference on Data Mining (ICDM)","cited_arxiv_id":null,"evidence_quote":"Provides Shapley effects and global sensitivity analysis theory supporting the feature-importance sampling step."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the DCN backbone and supplies the RelaImpr relative-improvement metric used in comparisons."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the GAUC intra-user ranking metric used for offline and online relevance evaluation."}],"review_version":1}