{"id":"cad4101b-f923-4a5d-8b15-1e9e64b1eadd","arxiv_id":"2411.12989","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Inserting short consecutive item sequences into user interaction histories lets a data owner detect whether a sequential recommender was trained on the protected dataset.","lead":"A team from NTU and UQ proposes a way to hide a secret item sequence inside recommender training data so that models trained on that data can be recognized later. The method preserves recommendation quality in tests on five models and three datasets, but its robustness to attacks is only partial.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Receptive-field generalization is asserted rather than measured; if NARM and contrastive models lack the assumed local attention structure, the RF-based filler uniqueness and DWRS-U insertion rationale are not grounded.","rationale":"The reader's weakest assumption identifies the same gap: the RF model is measured on a subset of architectures and datasets and then applied broadly. I agree this is the most load-bearing soft spot. However, I do not believe it warrants changing the CONDITIONAL verdict to a stronger rejection. The paper's core empirical observation—that inserting a short consecutive item sequence into up to 1% of training histories makes watermarked models rank the target item far above oracle models—is supported by consistent results across five models and three datasets, including models for which the RF theory is questionable. Even if the RF mechanism is not the true cause, the watermarking phenomenon appears robust in the reported settings. The concern is therefore about the explanation and the claimed methodological advantage over random insertion, not about the basic validity result. The concrete test I propose would settle whether the RF-based uniqueness constraint and insertion-position strategy are actually responsible for the reported gains. Since the paper already needs additional validation for its core mechanism, the appropriate verdict remains CONDITIONAL: accept the empirical watermarking result as plausible, but require the RF assumption to be verified where it is applied and require variance reporting to show that DWRS-D's advantage over baseline is significant.","tokens_in":20192,"tokens_out":11550,"duration_ms":129355,"concrete_test":"For each model not used in Fig. 3 (NARM, CL4SRec, DuoRec) and for the Steam dataset, train oracle models and compute the same attention/contribution heatmaps over held-out sequences, then estimate the RF size for each target position as, e.g., the minimal window around the target containing 95% of attention mass. Re-run DWRS-D on Steam and Beauty using the measured RF sizes. If NARM's effective RF is the full sequence, or if DWRS-D no longer consistently beats DWRS-D-base under the measured RF, the RF-based mechanism is not supported by the experiments.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanism of DWRS is the Receptive Field (RF) assumption formalized in Eq. 6, namely that a target item's embedding aggregates only items in a small local window. This is inferred from attention heatmaps of SASRec and Bert4Rec on ML-1M and Beauty (Fig. 3), but the method is then applied to NARM, CL4SRec, DuoRec, and Steam without verifying that those models/datasets have the same local structure. NARM is an RNN encoder-decoder; Eq. 6 is a transformer attention approximation and does not describe GRU hidden-state computation. For Beauty and Steam the paper sets RF to the entire sequence in Sec. 5.4.1, which is not a local receptive field at all and turns Algorithm 1's filler-uniqueness constraint into a global item-disjointness constraint. If the true RF for these cases is global or very different from the assumed one, the filler-uniqueness constraint has no stated justification, and DWRS-U's strategy of inserting before the most unpopular subsequence loses its theoretical basis. The empirical advantage of DWRS-D over random insertion (DWRS-D-base) in Fig. 4 is modest, and no error bars are reported; the reported advantage could be noise rather than evidence for the RF mechanism. This does not necessarily falsify the raw empirical claim that repeated watermark triples are memorized, but it does undermine the paper's proposed explanation and the claimed role of RF in achieving that memorization.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces DWRS, a data watermarking method for sequential recommender systems. It defines a watermark as a consecutive item sequence inserted into user interaction histories, with two variants: DWRS-D for dataset-level watermarking and DWRS-U for individual-user watermarking. The method uses a Receptive Field (RF) notion, derived from attention heatmaps of transformers, to guide insertion positions and to enforce a filler-item uniqueness constraint. Experiments on ML-1M, Amazon Beauty, and Steam across five models (SASRec, Bert4Rec, NARM, CL4SRec, DuoRec) report high watermark validity (e.g., Recall/NDCG near 1.0 for DWRS-D), preserved model utility relative to oracle models, near-zero watermark validity on oracle models, and robustness against finetuning, distillation, and sequential rule mining attacks. The paper also includes a comparison of DWRS-D against a random-insertion baseline (DWRS-D-base) and a position ablation for DWRS-U.","tokens_in":20549,"tokens_out":6469,"duration_ms":58579,"significance":"If the reported effectiveness is reliable, DWRS provides a practical black-box-verifiable method for data owners to claim unauthorized training on their datasets, and it is, per the paper's claim, the first data-watermarking method specifically for recommender systems. The paper includes several good practices: a public code repository, a random-insertion baseline to isolate the contribution of the RF-guided placement, oracle-model comparisons that establish discriminability, and full utility tables in the appendix. The main limitations are the absence of error bars or significance tests for the primary comparisons and the unverified generalization of the receptive-field assumption to all tested architectures and datasets. These issues are local in the sense that they can be addressed with additional measurements and reporting, but they currently leave the theoretical explanation for the method's advantage not fully supported.","major_comments":[{"comment":"The receptive-field model is inferred from attention heatmaps of SASRec and Bert4Rec on ML-1M and Beauty (Fig. 3), but DWRS-D is evaluated on NARM (an RNN encoder-decoder), the contrastive models CL4SRec and DuoRec, and on Steam without verifying that these models/datasets exhibit the same local RF structure. For Beauty and Steam, §5.4.1 sets RF to the entire sequence, which is not a local window and turns Algorithm 1's filler-uniqueness constraint into a global item-disjointness condition. The proposed explanation for why DWRS-D outperforms the random-insertion baseline is therefore not grounded for these cases. I ask the authors to either measure the RF (e.g., via attention or gradient-based attribution) for each model-dataset pair used in the evaluation and set the RF accordingly, or show that watermark validity is insensitive to the chosen RF size over a reasonable range. Without this, the load-bearing claim that RF-guided placement is the mechanism behind the reported advantage is unsupported.","section":"§3.2.1, Eq. (6); §5.4.1"},{"comment":"The advantage of DWRS-D over the random-insertion baseline DWRS-D-base is the primary evidence for the benefit of the RF-guided placement, but no error bars or statistical significance tests are reported; the paper states only that results are averaged over five runs. In several panels (e.g., NDCG@k for DuoRec, and Recall@k for DuoRec in Fig. 8), the DWRS-D and DWRS-D-base curves are visually indistinguishable and the differences are on the order of 10^-3. The claim that DWRS-D outperforms DWRS-D-base therefore cannot be distinguished from noise as presented. Please report standard deviations or confidence intervals for all primary watermark-validity comparisons and for the hyperparameter studies in Figs. 5 and 7.","section":"§5.4.2, Fig. 4 and Fig. 8"},{"comment":"The choice of inserting the watermark before the most unpopular subsequence (BUS) is a central component of DWRS-U, but its superiority is demonstrated on a single model-dataset pair (SASRec on ML-1M) with no error bars. The heuristic that unpopular items after the target provide additional overfitting capacity is not directly verified; the reader cannot tell whether the BUS advantage is a general property of the method or an artifact of that particular dataset. To support the claimed advantage over the other positions, the authors should report the position ablation on at least one additional model-dataset pair, with variance information.","section":"§4.2.2, §5.8, Fig. 7b"}],"minor_comments":[{"comment":"The condition `RF(pos) ∪ I_filler != ∅` appears to be a typo; the textual description indicates that the intent is to resample until the receptive field contains no previously seen filler items, i.e., `RF(pos) ∩ I_filler == ∅`. As written, the union condition is always true whenever RF(pos) is non-empty, which would prevent the loop from terminating as intended.","section":"Algorithm 1, line 8"},{"comment":"The text says \"Taking ML-1M on Bert4Rec for an example\" but Table 3 is titled \"ML-1M on SASRec\". Please correct this inconsistency.","section":"§5.5.2 and Table 3"},{"comment":"The statement \"we have empirically verified that, under the default setting of our experiment, by setting l > 30 and using unpopular items as watermark items, the watermark is almost guaranteed to be successful with Recall@10 > 0.9\" is not supported by any experiment reported in the paper; please include the verification or revise the statement to avoid an unsupported claim.","section":"§4.2.2"},{"comment":"The definition of RF as \"the range where all items within it have attention values much greater than zero\" lacks a quantitative criterion; please specify how the RF size (e.g., five items before and after the target for ML-1M) is derived from the heatmaps in Fig. 3.","section":"§3.2.1"},{"comment":"The captions for Figures 4 and 8 are identical, and neither figure labels the y-axis; please differentiate the captions and add metric labels to the axes.","section":"Figures 4 and 8"}],"recommendation":"major_revision","confidential_remarks":"The paper is clearly written and addresses a timely problem. My main concern is that the central comparison (DWRS-D vs. random insertion) lacks variance reporting, and the receptive-field assumption is extrapolated to models and datasets where it has not been verified. Both are fixable: error bars and a small set of RF verification experiments would substantially strengthen the paper. There is also an apparent typo in Algorithm 1 that should be corrected. I would be supportive of acceptance after a revision that addresses these points."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Honest take: this is the first paper I know to define data watermarking for sequential recommender systems. The problem is well-posed, the two settings (dataset vs. user) are distinct and sensible, and the evaluation protocol—querying the model with the watermark body and checking the ranking of the response—is practical for black-box ownership claims. The experiments are broad: five architectures, three datasets, and oracle comparisons near zero, which is exactly the right way to establish discriminability. Model utility is essentially unchanged. Code is provided.\n\nThe soft spots are concentrated in the mechanism and the statistical support. The receptive-field assumption, Eq. (6), is inferred from attention heatmaps of SASRec and Bert4Rec on ML-1M and Beauty, then applied to NARM (an RNN encoder-decoder), CL4SRec and DuoRec (contrastive models), and to Steam, without checking whether those models/datasets have the same local structure. Worse, for Beauty and Steam the RF is set to the entire sequence in Sec. 5.4.1, which is not a local receptive field and undermines the filler-uniqueness justification. So the paper's central explanation for why DWRS-D works is not established for a majority of the evaluated settings. The empirical advantage over random insertion (Fig. 4) is modest and reported without error bars or significance tests, so part of that advantage could be noise. This does not falsify the raw claim that repeated inserted triples are memorized—that is plausible and partly supported by the data—but it does mean the RF-based insertion heuristic and DWRS-U's unpopular-subsequence strategy rest on shaky ground.\n\nThe robustness claim against distillation is also stronger than the evidence. Table 4 shows watermark validity at R@10 dropping from ~99.7% to ~29.5%. The paper's argument that a nonzero recall still gives evidence is reasonable, but calling that 'robust' is generous. The attack evaluation is thin: one finetuning curve (SASRec on ML-1M), one distillation table, rule mining only on ML-1M.\n\nNone of this is fatal. The problem definition is valuable, the experiments are reproducible (code provided), and the concerns are addressable: report variances, select RF sizes on validation data, verify the RF assumption for non-transformer models, and characterize attack results more honestly. I'd send it to review; a serious referee can push on the mechanism. The paper is mostly useful to people working on recommender-system security and data licensing; it won't reshape the field, but it defines a problem and gives a plausible baseline.","headline":"A genuinely new problem and a plausible empirical method, but the receptive-field mechanism is over-generalized and the statistical support is thin; worth refereeing.","tokens_in":21030,"tokens_out":2715,"would_cite":true,"duration_ms":57368,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Inserting a short consecutive item sequence into a small fraction of user histories watermarks a sequential recommender so that data owners can prove unauthorized training with black-box queries.","keywords":["data watermarking","sequential recommendation","dataset ownership","user watermarking","receptive field","backdoor trigger","black-box verification","model utility"],"falsifier":"Train NARM, CL4SRec, and DuoRec on Steam with the DWRS-D watermark inserted under the five-item receptive field and compare watermark Recall@10 against random-position insertion; if the gap disappears on any architecture, or if sensitivity analysis shows the target embedding depends on items far outside the assumed window, the receptive-field mechanism is not what carries the watermark.","tokens_in":19976,"feed_emoji":"🏷️","tokens_out":6576,"duration_ms":59883,"temperature":0.7,"pith_summary":"This paper tries to establish that a data owner can watermark a sequential recommender system by inserting a short run of consecutive items into user interaction histories, and then prove unauthorized training by black-box queries: append the first part of the watermark to benign sequences and check whether the final watermark item ranks at the top. The proposed method, DWRS, has two variants: DWRS-D for watermarking an entire dataset and DWRS-U for protecting a single user's data. Across five sequential recommender architectures and three datasets, a model trained on watermarked data ranks the watermark response at or near the top, while oracle models trained on clean data rank it at essentially zero. The paper argues this gives a practical copyright-claim mechanism that preserves recommendation utility and survives finetuning, distillation, and sequential rule mining.","feed_headline":"A 3-item watermark in 1% of data can prove recommenders used it","feed_subtitle":"Black-box queries reveal the watermark item ranked top on trained models and near zero on clean ones.","key_machinery":"The load-bearing object is the Receptive Field (RF) of a target item: the contiguous window of surrounding items whose attention values are much greater than zero. The paper formalizes the mechanism as $x_i \\approx \\sum_{j \\in \\text{RF}} \\alpha_{ij} V_j$, so the target item's embedding is a weighted sum of value vectors in its local window. The RF justifies both design choices: the watermark body must lie inside the target's RF, hence the watermark items must be consecutive, and DWRS-D enforces a filler-uniqueness constraint so that no non-watermark item inside any RF appears twice across watermarked sequences, preventing the model from learning spurious correlations with the response. For user watermarking, the same local-aggregation idea justifies inserting the watermark before the most unpopular subsequence, which supplies additional learnable dependency.","core_discovery":"The central claim is that a watermark can be defined as a sequence of consecutive items $S_{\\text{wm}} = \\{i_1^{\\text{wm}}, \\dots, i_l^{\\text{wm}}\\}$, where the first $l-1$ items form the watermark body $x_{\\text{wm}}$ and the last item is the target response $y_{\\text{wm}}$. After inserting this sequence into a small fraction of user histories, a trained sequential recommender memorizes the pattern because, by the receptive-field assumption, the embedding of the target item aggregates mainly nearby items. Detection is black-box: the owner appends $x_{\\text{wm}}$ to benign user sequences and checks whether $y_{\\text{wm}}$ is ranked at the top. The paper's experiments report watermark validity above 0.7 on Bert4Rec and above 0.9 on SASRec, NARM, CL4SRec, and DuoRec with a length-3 watermark inserted into 1% of users, while oracle models give near-zero validity and model utility stays close to the clean-trained oracle.","pith_inferences":["We infer that the method's edge over random insertion would show up mainly under the receptive-field assumption; on models or datasets where attention is not locally concentrated, the filler-uniqueness step may not matter, so a comparison with random insertion on NARM, CL4SRec, or DuoRec on Steam would isolate whether RF guidance is the active ingredient.","An attacker who preprocesses sequences by dropping rare items, truncating histories, or reordering interactions before training could weaken the watermark, since every design choice relies on the exact consecutive placement of the trigger; this scenario is not tested in the paper.","The same consecutive-pattern idea could transfer to other sequence models, such as large-language-model-based recommenders, but longer-range attention would likely require longer watermarks and raise the detection risk from pattern mining.","The filler-uniqueness constraint caps how many users can be watermarked on small datasets; if a dataset cannot supply enough filler-disjoint positions, the 1% ratio cannot be reached and the utility-preservation guarantee may need re-evaluation."],"forward_implications":["A data owner who suspects a model was trained on their data can test it by querying with the two-item prefix of a length-3 watermark; a top-ranked response is evidence of unauthorized training, since clean models score near zero.","A watermark length of 3 is enough for dataset watermarking; length 2 fails, so a single prefix item does not create a memorizable pattern.","Inserting into 1% of user histories is sufficient, and increasing the ratio raises validity with diminishing returns.","The watermark survives finetuning, distillation, and sequential rule mining, although finetuning and distillation substantially lower both watermark validity and model utility.","A single user can be protected with a longer watermark of 10-20 items placed before the most unpopular part of their history, but effectiveness varies across models, with Bert4Rec being the hardest case."],"supporting_citations":[{"why":"Supplies SASRec, one of the two attention architectures whose heatmaps define the receptive-field assumption and a primary testbed.","marker":"[11]"},{"why":"Supplies Bert4Rec, the bi-directional attention model used to infer receptive fields and the hardest-to-train testbed.","marker":"[24]"},{"why":"Supplies NARM, the RNN-based sequential recommender used to test whether the receptive-field insertion transfers beyond attention models.","marker":"[12]"},{"why":"Supplies CL4SRec, the contrastive-learning sequential recommender used in the evaluations.","marker":"[29]"},{"why":"Supplies DuoRec, the contrastive-learning recommender used in the evaluations.","marker":"[19]"},{"why":"Supplies the black-box data-free model extraction method used for the distillation attack, as well as the dataset preprocessing protocol.","marker":"[36]"},{"why":"Supplies the 1% injection ratio as the standard threshold for adversarial attacks on recommenders, which DWRS-D adopts.","marker":"[25]"},{"why":"Supplies ERMiner sequential rule mining, the attack used to test whether the watermark pattern can be detected or removed.","marker":"[4]"}],"fun_headline_variants":["1% of users hold a 3-item watermark that exposes any recommender trained on them","Black-box probe: hidden 3-item sequence reveals recommenders trained on watermarked data","Data watermarking for recommenders: a 1% insertion leaves a detectable trace in models","Watermark recommender data with a 3-item sequence in 1% of users—detect it black-box","A 3-item watermark in 1% of user data proves recommender training via black-box query"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's advantage over random insertion rests on the assumption that every trained recommender builds an item's representation mainly from a small local window of surrounding items, an assumption the paper verifies only through attention heatmaps of SASRec and Bert4Rec on two datasets.","fun_headline_variants_meta":{"raw":{"variants":["1% of users hold a 3-item watermark that exposes any recommender trained on them","Black-box probe: hidden 3-item sequence reveals recommenders trained on watermarked data","Data watermarking for recommenders: a 1% insertion leaves a detectable trace in models","Watermark recommender data with a 3-item sequence in 1% of users—detect it black-box","A 3-item watermark in 1% of user data proves recommender training via black-box query"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001846,"raw_usage":{"total_tokens":7250,"prompt_tokens":937,"completion_tokens":6313,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":553,"completion_tokens_details":{"reasoning_tokens":6191}},"tokens_in":553,"tokens_out":6313,"duration_ms":44713,"temperature":1.0,"reasoning_tokens":6191,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:58:14.416398+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train NARM, CL4SRec, and DuoRec on Steam with the DWRS-D watermark inserted under the five-item receptive field and compare watermark Recall@10 against random-position insertion; if the gap disappears on any architecture, or if sensitivity analysis shows the target embedding depends on items far outside the assumed window, the receptive-field mechanism is not what carries the watermark.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the 1% injection ratio as the standard threshold for adversarial attacks on recommenders, which DWRS-D adopts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies ERMiner sequential rule mining, the attack used to test whether the watermark pattern can be detected or removed."}],"review_version":1}