{"id":"dc2c7d41-b2dc-4f14-a0e5-a05959f90706","arxiv_id":"2606.05621","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A recommender-denoising method that trains a noise recognizer on LLM-simulated user misbehaviors instead of relying on heuristic rules.","lead":"This paper proposes a new way to clean noisy user-click data in recommender systems: an LLM agent simulates common ways users click things they don't truly want, then a separate model is trained to spot such fake clicks in real data. On three benchmarks the method beats existing denoising approaches, but its key assumption—that simulated noise matches real noise—is never directly validated.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Synthetic noise is generated only from unobserved candidate items (i∉I_u), while real noise to be detected lies inside the observed interaction set; the recognizer is trained on a different domain than it is applied to, and the paper provides no direct validation that it transfers.","rationale":"The reader's weakest assumption was simulation fidelity: that LLM-generated noise approximates real noise. I agree partially but sharpen the concern to a structural domain mismatch. Even a perfect generative model of noise behavior would produce unobserved candidate pairs, while the target set N consists of observed interactions. The recognizer never sees positive examples from the deployment distribution, so the burden is on direct transfer evidence. This is not merely a missing comparison; it is a train/serve skew in the core supervised-learning formulation. End-to-end recommendation improvements (Table 2) and robustness to injected noise (RQ4) do not establish that real noisy interactions are identified. The paper should remain CONDITIONAL, but the acceptance condition should explicitly require a target-domain recognition evaluation with ground-truth labels (or a carefully justified proxy) plus error bars and code. My read does not change the reader's verdict, hence UNCHANGED, but it identifies a more precise and more decisive condition than 'simulation fidelity' alone.","tokens_in":17617,"tokens_out":9537,"duration_ms":107866,"concrete_test":"Construct a target-domain noise benchmark from MovieLens-1M: use the original star ratings to define a held-out set of plausible noisy observed interactions (e.g., interactions whose true rating is 1 or 2) and clean observed interactions (ratings 4 or 5), restricted to users not in the simulated subset Ũ. Run the trained ANCHOR recognizer on these observed interactions and report AUC / P@K. As a control, retrain the recognizer with the same architecture but with positive noise examples drawn from observed low-rated interactions (i.e., in-domain positives) and compare. If ANCHOR's AUC on the held-out observed-noise set is near chance, or substantially below the in-domain-trained control, the disjoint training domain is the cause and the central claim fails. This check directly tests whether the recognizer recognizes real noise rather than only synthetic exposure items.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that a recognizer trained on synthetic noise identifies real noise in D. But by construction (Section 4.1.2), every synthetic noisy interaction satisfies i∈C_u and i∉I_u, so the positive training class Ñ is disjoint from the observed interaction set D. The recognizer's training set (Eq. 14) labels original interactions of simulated users as clean (label 0) and these unobserved candidate items as noisy (label 1). At deployment it scores original observed interactions (D) and labels N⊆D as noise (Section 3). Thus positive examples at inference come from a domain never seen in training: observed interactions that are noisy. The classifier can exploit shortcuts such as history membership, item degree, or exposure rank rather than any real noise mechanism. The paper calls Ñ 'designed to approximate' N, but N⊆D while Ñ∩D=∅; no distributional or ground-truth comparison is provided. End-to-end gains in Table 2 cannot isolate recognition of real noise: they might come from semantic regularization, hard-negative effects of boundary-adjacent noise (Section 4.2), or generic filtering of hard-to-fit examples. RQ4 injects artificial noise and measures downstream accuracy, not whether the recognizer finds real noisy interactions. This is the load-bearing weakness in the Creation-Recognition paradigm.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a new paradigm for recommendation denoising: instead of inferring noise from unlabeled data via heuristics, it proactively creates labeled noisy interactions using LLM-based user simulators, then trains a supervised noise recognizer on those labels and applies it to the original interaction matrix. This paradigm is instantiated in ANCHOR, which simulates five types of out-of-preference noise (misclick, curiosity, caption bias, popularity bias, position bias) using a recommender-in-the-loop agentic framework, augments them with boundary-adjacent hard noise via an iterative creator-recognizer refinement, and trains a semantic-collaborative recognizer (LightGCN embeddings plus text encodings plus an MLP classifier). The recognizer is then used to identify and remove noisy interactions from the real data, and the cleaned data are used to train downstream recommender backbones. Experiments on DBbook2014, Book-Crossing, and MovieLens-1M with GMF and LightGCN backbones report consistent improvements over five denoising baselines, with ablations and hyperparameter analyses.","tokens_in":18074,"tokens_out":4356,"duration_ms":50087,"significance":"The proposed Creation-Recognition paradigm is conceptually appealing and, if the synthetic noise recognizer genuinely transfers to real noise, would constitute a meaningful shift from heuristic denoising to supervised denoising. The paper is clearly written, the framework is modular, and the Appendix provides detailed prompts that make the pipeline reproducible. The empirical gains in Table 2 are consistent across datasets and backbones, and the ablation study shows that each component contributes. However, the central claim that the trained recognizer identifies real noisy interactions is not directly validated. The synthetic noise is generated from candidate items that the user has never interacted with, while at deployment the recognizer is asked to flag observed interactions. This domain mismatch is the load-bearing issue of the paper and is currently only asserted, not demonstrated. Without direct evidence of transfer, the reported downstream improvements could be explained by other mechanisms, such as hard-negative sampling or semantic regularization.","major_comments":[{"comment":"The synthetic positive class is structurally disjoint from the inference domain. The generated noisy items are required to satisfy i∈C_u and i∉I_u (§4.1.2), and the recognizer training set D_disc labels these as 'noisy' (1) while labeling the original interactions of sampled users as 'clean' (0). At deployment, however, the recognizer scores observed interactions in D, i.e., items that do belong to I_u. Thus the classifier never sees a positive example from the target domain during training; it can learn a trivial shortcut such as 'not in user history ⇒ noisy' that has no discriminative power on D. The paper states that Ñ is 'designed to approximate' N (Section 3), but provides no distributional comparison and no ground-truth evaluation. Consequently, the gains in Table 2 cannot be attributed to identifying real noise; they may stem from hard-negative effects or regularization. Please ad","section":"4.1.2 / Eq. (14)"},{"comment":"The robustness experiment does not measure whether the recognizer actually identifies the injected noisy interactions. The experiment injects adversarial noise at 5–20% and reports recommendation metrics, not detection metrics. The stated RQ4 is 'How effectively does ANCHOR identify and remove noisy interactions?' but no identification accuracy is reported. Moreover, if the injected noise is generated in the same way as the training noise, the evaluation would be circular and would overstate transfer to real noise. Please report direct detection results against the injected noise with known labels (AUC, recall@k, false-positive rate), and ideally use held-out noise generated by genuinely different mechanisms.","section":"5.5 / RQ4"},{"comment":"The recognizer training labels all original interactions of simulated users as clean (label 0). This assumption contradicts the paper's own premise that real implicit feedback D contains noise (Section 3). Any real noisy interaction belonging to a simulated user is systematically mislabeled as clean, which could train the recognizer to suppress exactly the signal it is meant to detect at deployment. The paper does not quantify or discuss this bias. Please address this, e.g., by using a conservative clean set, contrasting with a variant that leaves out ambiguous interactions, or analyzing sensitivity to this labeling assumption.","section":"4.3 / Eq. (14)"},{"comment":"The 'w/o Gen' ablation replaces the generated out-of-preference noise with random negative sampling. This shows that having any synthetic positives is helpful, but it does not test whether the five simulation mechanisms approximate real noise; any positive/negative contrast would likely improve the recognizer in a purely architectural sense. This is a useful sanity check, but it does not support the central transfer claim. The paper needs a separate experiment that varies the realism of the synthetic noise (e.g., comparing behavior-grounded simulation against random negatives while holding the recognizer architecture constant) and evaluates recognition on real or realistically injected noise.","section":"5.3 / Table 3"}],"minor_comments":[{"comment":"The text refers to 'w/o ABR' but the table and the earlier bullet list use 'w/o IBR'. Please make the terminology consistent.","section":"Section 5.3, text after Table 3"},{"comment":"The hyperparameter sensitivity analysis does not describe how the hyperparameters were selected or on which split. If the optimal values (|Ñ_u|=4, α=0.2, T=4) were chosen by looking at the test metrics reported in Figure 3, the reported performance may be optimistic. Please state the validation procedure.","section":"Section 5.4"},{"comment":"The sentence 'Figures 4 reports the results' has a grammatical error. Also, the two panels of Figure 4 are not clearly labeled; please clarify which dataset is shown in which panel.","section":"Section 5.5"},{"comment":"The paper reports point estimates without standard deviations or significance tests. Given that the improvements over LLaRD are sometimes small (e.g., Table 2, LightGCN on MovieLens-1M: 0.2015 vs. 0.1903 R@10), please report multiple runs and statistical significance.","section":"Section 5.2"},{"comment":"The mechanism by which boundary-adjacent noise is used downstream is described only at a high level ('boundary-sensitive supervisory signals' and 'hard negative signals'). It is not clear whether these detected interactions are removed from the training set, added as negatives, or used in some other way in the final recommender training. Please make the experimental protocol explicit.","section":"Section 4.2"},{"comment":"The sampled user subset size |Ũ|=1,000 is fixed, but no sensitivity analysis is reported for this important parameter. Please include it or justify the choice.","section":"Section 5.4"}],"recommendation":"major_revision","confidential_remarks":"The paper's central premise—that synthetic noise created from unobserved candidate items can train a recognizer that detects noise inside the observed interaction set—is interesting but currently unsupported. The authors need to provide direct evidence of transfer, such as detection metrics on known noisy interactions or a distributional comparison between synthetic and real noise. Without that, the end-to-end gains could be explained by less interesting mechanisms. I would encourage the editor to request such evidence before acceptance, as the framework is otherwise well-motivated and clearly presented."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's central claim—that ANCHOR identifies real noise via a recognizer trained on LLM-generated synthetic noise—is never directly tested, and the stress-test note lands cleanly: synthetic noise is drawn from candidate items the user never interacted with (i∉I_u), while the recognizer is deployed on observed interactions (D). The recognizer never sees a positive example from that deployment domain during training. It learns to separate original observed interactions (labeled clean) from unobserved candidates (labeled noisy), so at inference it can realistically exploit shortcuts like item popularity or a simple 'has the user clicked this before' rule. The paper calls Ñ 'designed to approximate' N but provides no distributional comparison, no ground-truth noise labels, and no user study. The injection experiments (RQ4) only show that removing some fraction of data helps downstream metrics; they don't show the removed interactions are actually noise.\n\nThat said, the paper does real work. The Creation-Recognition paradigm is a fresh framing for denoising recommendation, and the idea of using LLM-as-User simulation to generate noisy labels is a meaningful departure from heuristic loss-based or LLM-knowledge-injection baselines. The framework is coherent, the five noise mechanisms are reasonable, and the experiments are broad: three datasets, two backbones, ablations, and hyperparameter sensitivity. The reported gains over LLaRD and DCF are substantial, and the ablations show each component contributes. If the transfer problem were solved, this would be a useful recipe.\n\nOther soft spots are secondary but real: no error bars or repeated runs, hyperparameters seem tuned on the same data, and no code is released. The recognizer also shares a LightGCN pretrained on the same interaction matrix it later cleans, which invites leakage questions. These are fixable; the domain mismatch is not.\n\nWho should read this? Anyone working on LLM-based simulation for recommendation will want to know the idea, but they should treat the empirical claims as upper bounds until the recognizer is shown to identify noise in real interaction data. This deserves a serious referee — the novelty and the quantity of experiments justify engagement — but the referee should ask for direct noise-recognition evaluation, an analysis of the train/deployment domain gap, and code before the claims are accepted.\n\nMy bottom line: send it out for review, but be prepared for a revision requirement on the central mechanism.","headline":"The Creation-Recognition idea is genuinely new, but the paper never shows that a recognizer trained on synthetic unobserved interactions can detect real noise inside observed interactions—that gap is load-bearing.","tokens_in":18457,"tokens_out":1777,"would_cite":false,"duration_ms":22199,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that creating labeled noise through LLM-simulated user behavior lets a learned recognizer filter real noise from implicit feedback better than heuristic denoising methods.","keywords":["recommendation denoising","implicit feedback","LLM-as-user simulation","noise generation","noise recognition","agent-based simulation","supervised denoising","recommender systems"],"falsifier":"Compute the recognizer's predictions on a held-out set of real interactions whose noisy or clean status is known (for example, from a logged user survey, a known misclick log, or a manually labeled audit) and measure precision and recall; if the recognizer performs near chance or systematically mislabels hard examples, the synthetic-to-real transfer fails. A cheaper probe is to compare the feature distributions of synthetic noise versus real flagged interactions; a large divergence would undermine the approximation claim.","tokens_in":17551,"feed_emoji":"🤖","tokens_out":3986,"duration_ms":39569,"temperature":0.7,"pith_summary":"The paper tries to establish that recommendation denoising—deciding which clicks and purchases reflect true taste and which are noise—can be turned from an unsupervised guessing problem into a supervised learning problem. It proposes to create labeled noise by having LLM-powered agents simulate real-world non-preference behaviors such as misclicks, curiosity clicks, caption-driven clicks, popularity-driven clicks, and position-driven clicks, then train a recognizer on those labels and apply it to real interaction data. On three book and movie datasets with two recommendation backbones, the paper reports that this creation-recognition approach consistently outperforms existing denoising methods, with the largest gains coming from the simulated-noise generation itself.","feed_headline":"Make fake noise to find real noise in recommendations","feed_subtitle":"By generating labeled noisy clicks with LLM agents, it turns denoising into a supervised task and reports better recommendations.","key_machinery":"The load-bearing object is the synthetic noise set, produced by five LLM-prompted behavior simulators (misclick, curiosity, caption bias, popularity bias, position bias) within a recommender-in-the-loop interaction environment, plus a creator-recognizer iterative refinement loop for boundary-adjacent noise. This labeled set converts noise identification into binary classification: a recognizer trained with cross-entropy plus a BPR auxiliary loss, then thresholded to delete estimated noisy interactions. The work it does is to replace heuristic assumptions about what noise looks like with direct supervision on simulated examples.","core_discovery":"The central claim is that the best way to predict noise is to create it. Instead of inferring which interactions are noisy from heuristics like high training loss, ANCHOR constructs a synthetic noisy interaction set by simulating user behavior with LLM agents in a recommender-in-the-loop setting, then uses this set to train a parametric noise recognizer that combines collaborative ID embeddings and semantic text embeddings. The recognizer is applied to real data to filter noise, and boundary-adjacent hard noise generated by a creator-recognizer feedback loop is used to sharpen the decision boundary. The paper reports consistent improvements over five baselines on DBbook2014, Book-Crossing, a","pith_inferences":["A direct test of the core premise would be measuring whether the recognizer's flagged noise matches ground-truth noise from logs with known misclicks or manually labeled audits; the paper does not provide such a test.","The recognizer's confidence threshold is a free parameter; operational gains likely depend on choosing it well, and the paper does not report how the threshold should be selected in practice.","Because the recognizer trains only on simulated users' original interactions plus synthetic noise, its transfer to unseen users rests on the assumption that collaborative and semantic features generalize beyond the simulated population.","The five noise mechanisms are author-chosen; real-world implicit feedback may contain other noise sources (bots, multi-account behavior, temporal drift) that would require additional simulation mechanisms."],"forward_implications":["If synthetic noise approximates real noise, denoising becomes a transferable supervised task, and the trained recognizer can be reused without re-prompting LLMs at deployment.","Boundary-adjacent noise can serve as hard-negative or boundary-sensitive signals for downstream recommenders, not just as data to be removed.","Because the noise simulation is extensible through new prompts, new noise types can be added as they are discovered without redesigning the loss function or heuristics.","The reported gains on two different backbone models suggest the approach is model-agnostic and could wrap other recommender architectures."],"fun_headline_variants":["How to find real noise: create it first","LLM agents generate fake clicks to teach noise detection","Making noise to remove noise in recommendations","From heuristics to supervision: denoising by simulation","Turning noise detection into a supervised task via agentic generation"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The whole approach hangs on the assumption that the LLM-simulated noisy interactions are close enough in distribution to real-world noise that a recognizer trained on them will flag real noise; the paper gives no direct evidence for this equivalence.","fun_headline_variants_meta":{"raw":{"variants":["How to find real noise: create it first","LLM agents generate fake clicks to teach noise detection","Making noise to remove noise in recommendations","From heuristics to supervision: denoising by simulation","Turning noise detection into a supervised task via agentic generation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000194,"raw_usage":{"total_tokens":1224,"prompt_tokens":814,"completion_tokens":410,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":558,"completion_tokens_details":{"reasoning_tokens":336}},"tokens_in":558,"tokens_out":410,"duration_ms":4091,"temperature":1.0,"reasoning_tokens":336,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T04:47:51.634563+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the recognizer's predictions on a held-out set of real interactions whose noisy or clean status is known (for example, from a logged user survey, a known misclick log, or a manually labeled audit) and measure precision and recall; if the recognizer performs near chance or systematically mislabels hard examples, the synthetic-to-real transfer fails. A cheaper probe is to compare the feature distributions of synthetic noise versus real flagged interactions; a large divergence would undermine the approximation claim.","supporting_citations":[],"review_version":2}