{"id":"add8d54e-cdfa-4ba8-9c93-a1e5cfad8791","arxiv_id":"2502.05561","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"DMI adds dimension-level Gaussian noise to multi-interest user vectors and denoises them with cross-attention over pruned history items, improving candidate matching on three datasets and in an online test.","lead":"This paper proposes DMI, a recommender-system module that cleans up multi-interest user vectors by adding controlled noise and then denoising them with attention over the most relevant past items. If the reported gains hold, candidate matching can become more accurate and more diverse, which matters for large-scale recommendation platforms.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The item-pruning mechanism is never tested against random pruning, so the claim that attention-based pruning removes interest-irrelevant attributes is unsupported.","rationale":"The reader's weakest assumption is precisely that the attention weights used in Eq. 12 are calibrated enough for pruning to remove interest-irrelevant items. My stress-test identifies that this assumption is not merely unproven; it is untestable from the reported experiments because no random-pruning or attention-scrambling control is included. The DMI-IP ablation (Table 3) only compares 'all items' versus 'top-K attention items'; it cannot rule out a pure context-length effect. This is the most load-bearing concern because it targets the core causal story of the paper: without the pruning mechanism actually identifying relevant items, the 'dimension-level refinement' framing collapses into generic denoising with a shorter context, and the reported gains may have a trivial explanation. This is a correctness risk rather than a stylistic issue, and it can be fixed with a small set of controlled experiments. Other issues noted by the reader—missing code, placeholder platform, and sparse online statistics—are important but do not directly test the mechanism. My concrete test would settle the concern. If the test shows top-K attention pruning is no better than random, the paper's central contribution statement would need to be substantially weakened, but the empirical pipeline might still be publishable as a diffusion-based refinement method with a different justification. Therefore the appropriate verdict remains CONDITIONAL, not REJECT: the concern is addressable and does not currently rise to outright rejection of the empirical results.","tokens_in":18839,"tokens_out":4901,"duration_ms":52918,"concrete_test":"Run DMI on Amazon Book and Beauty using three pruning conditions with the same γ: (1) top-K attention as in Eq. 12; (2) K items sampled uniformly at random from H; (3) bottom-K attention (lowest scores). Also run a condition where attention weights are randomly permuted across items before TopK selection, preserving the score distribution but breaking the item-score correspondence. Compare Recall@50 and NDCG@50 across conditions. If (1) significantly exceeds (2) and (4) (e.g., >1% relative improvement, with overlapping standard errors assessed), the pruning strategy is genuinely exploiting attention calibration. If (2) or (4) matches (1), the pruning benefit is not attributable to selecting interest-relevant items and the paper's central mechanism claim must be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central explanatory mechanism of DMI is that the item pruning strategy (Eq. 12) selects the top-K historical items by attention weights to supply 'interest-relevant collaborative information' to the cross-attention denoiser (Eq. 11). This is the component that supposedly enables dimension-level refinement: pruning is what removes the irrelevant attributes identified in Section 1. The only ablation offered is DMI-IP (Table 3), which removes pruning entirely and uses all historical items. That comparison cannot distinguish between two very different explanations: (a) pruning removes low-attention, interest-irrelevant items, improving reconstruction; or (b) pruning merely shortens the cross-attention context, acting as a regularizer or reducing attention dilution. The paper's own hyper-parameter analysis in Section 4.5.3 shows that as the pruning ratio γ decreases, performance initially declines and then improves. This non-monotonic behavior is compatible with a mechanism that is not simply 'drop the irrelevant tail' and raises the possibility that the optimal pruning set is not the one defined by the extractor's attention weights. The attention weights themselves come from the same base multi-interest extractor that the paper argues produces coarse-grained representations containing irrelevant information (Section 1), so using those weights as the ground truth for relevance is a real risk. If random pruning of the same size were to perform as well as attention-based pruning, then the claimed mechanism would be unsupported; DMI would still be a working empirical pipeline, but its stated novelty—interest-relevant pruning at the dimension level—would not be established. Since code is not released, this missing control is the single largest unverified link in the causal chain between the proposed design and the reported gains.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DMI (Diffusion Multi-Interest model), a plug-in diffusion module that refines coarse-grained user interest vectors in multi-interest candidate matching. A base multi-interest extractor (ComiRec-SA combined with REMI) produces K interest vectors by attentively aggregating historical item embeddings. DMI adds dimension-level Gaussian noise to the selected interest vector, then reconstructs a denoised vector with a Transformer cross-attention module conditioned on a diffusion-step embedding and on a pruned set of historical items selected by their attention weights. The final user representation is a convex combination of the refined and original vectors, and training is joint over a sampled-softmax recommendation loss and a Euclidean reconstruction loss. Experiments on Amazon Book, Amazon Beauty, and Gowalla report consistent improvements over baselines, ablations support the key components, and an online A/B test reports engagement gains.","tokens_in":19199,"tokens_out":8544,"duration_ms":86379,"significance":"If the results hold, DMI is a practically useful and relatively simple enhancement for multi-interest matching: it shows that a diffusion-style denoiser, guided by interest-specific collaborative signals, can improve retrieval quality over a strong REMI baseline, with consistent gains across three datasets and a reported real-world deployment. The paper's strengths include the breadth of offline evaluation, the inclusion of ablation and hyper-parameter studies, and the explicit commitment to code release upon acceptance. The main weaknesses are that the central pruning mechanism is not tested against random pruning, several equations and tables are internally inconsistent or underspecified, and the online A/B test lacks the statistical detail needed to assess the claimed improvement.","major_comments":[{"comment":"The scaling factor s in Eq. (8) is never defined, yet it multiplies the entire linear noise schedule and directly controls the forward process. Please define s, state its value in the implementation settings, and clarify whether it is a tunable hyper-parameter. In addition, Eq. (9) writes p_theta(hat_v_{t-1} | hat_v_t) = N(hat_v_t; mu_theta, ...), but the distribution should be over hat_v_{t-1}, and Algorithm 1 step 5 says 'given v0 and T via q(v_T|v0)' although t was sampled uniformly in step 4; these need correction.","section":"3.2, Eq. (8)"},{"comment":"There are internal numerical inconsistencies that undermine confidence in the reported results. For Gowalla with the default K=4, Table 5 reports Recall@50=0.2731 and NDCG@50=0.2308, whereas Table 2 reports Recall@50=0.2735 and NDCG@50=0.2263 for the same setting. In Table 3, the Gowalla DMI-IP recall is printed as '26.93' rather than a fraction, and several 'Improv.' percentages (e.g., DMI-GD on Book and DMI-IP on Gowalla) do not match the values in the preceding columns. Please provide corrected tables and explain any differences due to seeds or settings.","section":"4.5.2, Table 5; 4.3, Table 3"},{"comment":"The claim that the item pruning strategy removes interest-irrelevant items is not tested against random pruning. The ablation DMI-IP removes pruning entirely, but that comparison conflates two effects: the selection of items by attention weights and the reduction of cross-attention context length. The analysis in Section 4.5.3 shows a non-monotonic relationship between the pruning ratio gamma and performance, which is not obviously explained by 'drop the irrelevant tail.' Please add a control that prunes the same number of items randomly, or by the lowest attention weights, to establish that attention-based selection itself is responsible for the observed improvement.","section":"3.2, 4.3, 4.5.3"},{"comment":"The DMI-GD ablation changes two design choices simultaneously: it removes the stop-gradient between the diffusion module and the multi-interest extractor, and it also drops the fusion operation in Eq. (13), using the denoised vector directly as the final interest representation. The reported convergence failure cannot therefore be attributed specifically to the stop-gradient. Please run separate ablations for (i) keeping fusion but removing stop-gradient and (ii) removing fusion but keeping stop-gradient, so the two effects can be disentangled.","section":"4.3, DMI-GD"},{"comment":"The online A/B test is reported only as relative percentage changes over one week, with no number of users, fraction of traffic, confidence intervals, p-values, or variance information. The statement that 'even a 0.2% improvement in user engagement is considered statistically significant' is unsupported without details of the significance testing procedure. Please report the experimental setup and standard statistical quantities for the online test.","section":"4.6, Table 6"},{"comment":"The reconstruction loss L_dm trains the denoiser to reproduce v0, which is the coarse-grained interest vector produced by the same extractor that DMI is supposed to refine. Thus the diffusion objective by itself does not define a 'clean' interest vector independent of the coarse extractor; any removal of irrelevant attributes must come from the recommendation loss L_S through the fusion in Eq. (13). Please clarify the identification of the clean target and provide evidence that the diffusion objective contributes beyond L_S, for example by including a lambda=0 ablation or by comparing v0 with an alternative target constructed from the positive item.","section":"3.3, Eq. (18)"}],"minor_comments":[{"comment":"There are repeated typographical errors, including 'denosing' for 'denoising,' 'precess' for 'process,' 'RALATED WORKS' in the related-work heading, and 'DDRM' in Section 4.5.3 where DMI is meant. These should be cleaned up.","section":"Throughout"},{"comment":"The denoising module is described in the text as taking the noisy embedding v_t, but Eq. (11) uses hat_v_t. Please standardize the notation so that the training-time and inference-time inputs to the Transformer are clearly defined.","section":"Eq. (11)"},{"comment":"The column header for Div(all) has an upward arrow, but the reported DMI value is lower than the REMI value and the text describes this as a slight reduction. Please correct the arrow or the interpretation.","section":"Table 4"},{"comment":"Since the paper positions DMI against existing plug-in diffusion recommenders such as DiffRec and the plug-in embedding-denoising work in [28,29], it would be helpful to state explicitly why these are not included in the offline comparison, or to add them as baselines.","section":"5.2, Table 2"}],"recommendation":"major_revision","confidential_remarks":"The paper appears to be an industry-led submission with a placeholder for the company name; that is acceptable for anonymized review. The main concern beyond the technical issues is that the paper's headline mechanism (attention-based pruning removes interest-irrelevant attributes) is asserted rather than directly evidenced, and the numerical inconsistencies in Tables 3 and 5 should be resolved before this is ready for a top venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth engaging with. The core idea—apply a diffusion denoiser directly to the interest vectors at the dimension level, guided by pruned collaborative items—is genuinely new in the multi-interest literature. The offline results are consistent across three datasets and the ablations support the main components. But the claimed mechanism for item pruning is not actually tested, and the tables have enough typos that I'd want a cleaned version before relying on the numbers.\n\nDMI does something none of the cited diffusion recommenders do: it refines the multi-interest extractor's own vectors rather than generating items or denoising interaction matrices. The cross-attention denoiser with step embeddings is reasonable. The gradient stopping to keep the extractor stable is a sensible design. Offline gains of 5–18% against strong baselines like REMI are meaningful. The online result, if real, is decent (0.59% engagement).\n\nFirst and biggest soft spot: the item pruning strategy is never compared against random pruning or fixed truncation. The ablation DMI-IP removes pruning entirely, which confounds \"pruning helps\" with \"attention-based pruning specifically helps.\" The paper's own hyperparameter analysis shows non-monotonic behavior in gamma, and Section 4.5.3 even concedes that \"the primary noise is not caused by items with the lowest attention scores.\" That is exactly the stress-test concern: the mechanism may just be context shortening or regularization, not interest-aware selection. A random-pruning control is directly addressable, and its absence is the single biggest gap.\n\nSecond: precision issues. Eq. 8 has an undefined scaling factor s. Table 3 has a Gowalla Recall value of \"26.93\" for DMI-IP that is clearly a typo (should be 0.2693). Table 5's Gowalla NDCG@50 for K=4 (0.2308) contradicts Table 2 (0.2263). The online A/B section says \"0.2% improvement is considered statistically significant\" without reporting p-values, confidence intervals, or the number of users. The code-release promise is conditioned on acceptance, which is not ideal.\n\nThird: the self-referential nature—the denoiser reconstructs the extractor's own v0, and the final representation mixes the reconstruction with the original v. This means the \"refinement\" is bounded by how much the extractor's attention weights are actually informative. That is not circular in a fatal way, but it does undercut the claim of fixing coarse-grained errors.\n\nBottom line: this is for people working on multi-interest retrieval or diffusion recommenders, especially in industry. A serious referee could get a lot out of the pruning-control experiment, statistical details, and cleaned tables. I'd send it to peer review with the expectation of heavy revision, not desk reject.","headline":"DMI has a real, if incremental, idea—dimension-level diffusion refinement for multi-interest vectors—and honest offline experiments, but the pruning mechanism is under-tested and the paper has several numeric inconsistencies.","tokens_in":19760,"tokens_out":2614,"would_cite":false,"duration_ms":24453,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that coarse multi-interest vectors can be refined into more personalized representations by adding dimension-level Gaussian noise and denoising with cross-attention over pruned historical items, and reports gains on…","keywords":["multi-interest recommendation","diffusion model","candidate matching","interest refinement","item pruning","cross-attention denoising","representation learning","recommender systems"],"falsifier":"Run DMI with the TopK pruning indices in Eq. 12 replaced by random subsets of historical items of the same size; if Recall@50 does not fall outside the reported variance, the attention-based pruning is not the mechanism producing the gain. A second check is to perturb the base attention weights used for pruning and observe whether DMI's advantage over its no-pruning variant shrinks, as the paper's calibration assumption predicts.","tokens_in":18676,"feed_emoji":"🎯","tokens_out":10314,"duration_ms":91843,"temperature":0.7,"pith_summary":"Multi-interest recommenders distill a user's history into several interest vectors by attention-weighting whole items, so each vector tends to carry attributes that are irrelevant to the user's actual interest. This paper proposes DMI, which treats those coarse vectors as noisy signals: it adds small dimension-level Gaussian noise, then iteratively reconstructs a cleaner vector with a cross-attention denoiser that attends only to the historical items the base extractor judged most relevant. The authors report that DMI outperforms every compared baseline on the Book, Beauty, and Gowalla benchmarks, and that replacing only the multi-interest model in a live mainfeed recommender raised user engagement by 0.59 percent over one week. The paper argues this makes diffusion-based denoising a practical plug-in for the candidate matching stage rather than a redesign of the recommender.","feed_headline":"Dimension-level diffusion denoising lifts recall by up to 18%","feed_subtitle":"Scheduled noise plus item pruning gives sharper interest vectors and beats all baselines online and offline.","key_machinery":"The load-bearing machinery is the denoising module $f_\\theta$: a cross-attention transformer that reconstructs a clean interest vector from a noisy one, conditioned on a pruned set of historical items. The forward process adds scheduled isotropic Gaussian noise at the dimension level of the interest vector $v_0$ (Eqs. 6-8), and the reverse process uses the transformer with the noisy vector as query and $\\text{concat}(e_t, C)$ as keys and values, where $C$ selects the TopK historical item embeddings by the extractor's attention weights (Eq. 12). The item-pruning step does the conceptual work: it decides which collaborative information is interest-relevant before attention is applied, so the reconstruction is guided by targeted context rather than the full noisy history. A gradient stop and the fusion rule in Eq. 13 keep the base extractor trained by the recommendation loss while the diffusion loss refines representations.","core_discovery":"The central claim is that the residual error left by item-level interest extraction lives partly in individual dimensions of the interest vector, and that a diffusion loop can remove it. Starting from an attention-aggregated vector $v_0$, DMI samples noisy states $v_t = \\sqrt{\\bar\\alpha_t}\\,v_0 + \\sqrt{1-\\bar\\alpha_t}\\,\\varepsilon$ using a deliberately small noise schedule, then trains a denoising module $f_\\theta$ to predict $v_0$ from $v_t$ via cross-attention over the pruned historical items $C = H[\\,:\\, \\text{TopK}_{\\gamma n}(\\mathbf{a})]$. The final user representation is a weighted fusion $Z_u = \\eta \\tilde v_0 + (1-\\eta) v$, and gradients from the reconstruction loss are stopped at the extractor so the diffusion module cannot trivialize the extractor's job. The paper reports consistent Recall, Hit Rate, and NDCG gains on three public datasets and a 0.59 percent engagement lift in an online A/B test, with ablations showing that removing the diffusion loop, the cross-attention denoiser, the item pruning, or the gradient stop each degrades performance.","pith_inferences":["A testable extension the paper does not run is to mount the same dimension-level diffusion module on other attention-based multi-interest extractors; the mechanism predicts gains whenever the base attention weights carry signal, not only with REMI/ComiRec-style attention.","The pruning-ratio analysis hints that the optimal selection ratio depends on how much noise the base extractor injects; comparing sensitivity curves across extractors of different quality would directly test the paper's noise-as-irrelevant-attributes interpretation.","If interest noise concentrates in particular dimensions, an anisotropic or dimension-dependent noise schedule could beat the isotropic Gaussian schedule used here; this follows naturally from the paper's dimension-level framing but is not tested in it."],"forward_implications":["Across Book, Beauty, and Gowalla, DMI improves Recall@20 by 12.8 to 16.2 percent over its strongest baseline, with similar gains in Hit Rate and NDCG, indicating the refinement transfers across domains and sparsity levels.","Ablations attribute the gain to all four design choices: the diffusion loop, the cross-attention denoiser over an MLP, the pruning of low-attention items, and the gradient stop that protects the extractor; removing any one degrades Recall@50 by up to roughly 77 percent.","Category-level analysis shows retrieved items become more concentrated in categories aligned with each interest while diversity among hit items increases, implying the refinement changes which categories are retrieved, not just the ranking within them.","In an online A/B test on a large mainfeed recommender, replacing only the multi-interest model with DMI raised engagement by 0.59 percent, category clicks by 0.31 percent, favorites by 1.09 percent, and comments by 0.64 percent, while DDIM kept inference affordable."],"supporting_citations":[{"why":"supplies the self-attentive multi-interest extraction formulation whose attention weights and interest vectors DMI builds on.","marker":"[1]"},{"why":"supplies REMI, the default base multi-interest model and the strongest offline baseline; DMI is combined with REMI by default.","marker":"[25]"},{"why":"supplies the DiffRec formulation of diffusion denoising for recommender embeddings and the linear noise schedule DMI adapts to interest vectors.","marker":"[29]"},{"why":"supplies the plug-in diffusion denoising idea with collaborative information guidance, the direct precedent DMI extends with pruning and dimension-level refinement.","marker":"[28]"},{"why":"supplies DDIM, used to cut inference steps during the online deployment.","marker":"[23]"},{"why":"supplies the reparameterization trick used to sample noisy interest vectors at arbitrary diffusion steps.","marker":"[22]"},{"why":"supplies MIND, the first multi-interest candidate matching model and a baseline that frames the task.","marker":"[11]"},{"why":"supplies Re4, a strong multi-interest baseline that DMI compares against and must outperform.","marker":"[27]"}],"fun_headline_variants":["Diffusion refines interest vectors dimension by dimension","Denoising interest dimensions boosts recall in recommenders","Diffusion model prunes noise from user interest dimensions","Refining multi-interest via diffusion: 18% recall gain","DMI: dimension-level diffusion for sharper user interests"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the base extractor's attention weights are calibrated well enough that pruning the lowest-attention historical items removes interest-irrelevant noise while preserving the collaborative context the denoiser needs; if those weights are miscalibrated, the pruned context inherits the very coarse-grained error DMI is meant to fix.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion refines interest vectors dimension by dimension","Denoising interest dimensions boosts recall in recommenders","Diffusion model prunes noise from user interest dimensions","Refining multi-interest via diffusion: 18% recall gain","DMI: dimension-level diffusion for sharper user interests"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000207,"raw_usage":{"total_tokens":1424,"prompt_tokens":991,"completion_tokens":433,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":607,"completion_tokens_details":{"reasoning_tokens":356}},"tokens_in":607,"tokens_out":433,"duration_ms":4615,"temperature":1.0,"reasoning_tokens":356,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T18:50:15.218292+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run DMI with the TopK pruning indices in Eq. 12 replaced by random subsets of historical items of the same size; if Recall@50 does not fall outside the reported variance, the attention-based pruning is not the mechanism producing the gain. A second check is to perturb the base attention weights used for pruning and observe whether DMI's advantage over its no-pruning variant shrinks, as the paper's calibration assumption predicts.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies REMI, the default base multi-interest model and the strongest offline baseline; DMI is combined with REMI by default."},{"cited_title":"Denoising Diffusion Recommender Model","cited_arxiv_id":"2401.06982","evidence_quote":"supplies the plug-in diffusion denoising idea with collaborative information guidance, the direct precedent DMI extends with pruning and dimension-level refinement."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies DDIM, used to cut inference steps during the online deployment."}],"review_version":1}