{"id":"e666be86-9623-4273-876e-6a7f7f2ff855","arxiv_id":"2508.01074","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":3,"one_line_summary":"A teacher-student distillation method that carries task knowledge through an out-of-distribution dataset evades all tested dataset-ownership verification methods, according to the authors' experiments.","lead":"This computer vision paper presents a method for training a model on protected data without leaving the traces that dataset-ownership detectors look for. It matters because the authors say their attack defeats all eleven detectors they tested, a result that would remove the main technical protection for proprietary training data.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Universal 'all eleven DOV methods' claim rests on test-bed fairness; abstract does not show detectors at standard operating points or held-out methods.","rationale":"The reader's verdict is UNVERDICTED because the supplied full text is corrupted and only the abstract is readable. My stress-test pass reaches the same practical conclusion: the central claim cannot currently be verified. I identify one load-bearing concern that is more specific than 'unverifiable': the abstract commits to a universal comparative statement across eleven DOV methods, and the integrity of that comparison depends on whether the detectors were evaluated at standard operating points and whether any were held out. This matches the second of the reader's two weakest assumptions, but the reader also emphasized the separability of task knowledge from identifier traces; I focus on test-bed fairness because it is the most direct support for the 'all eleven' universal claim and is checkable once the clean text or code is available. I do not allege any internal inconsistency or bad faith, and the framework could well survive this check. Since neither the reader nor I can audit the methods, the appropriate verdict remains UNVERDICTED rather than ACCEPT, CONDITIONAL, or REJECT.","tokens_in":14163,"tokens_out":3767,"duration_ms":49998,"concrete_test":"Obtain the clean source (or ask authors for code) and rerun the evasion pipeline against at least two DOV methods that were not used during framework development, using each detector's published code, default architecture, and chosen false-positive operating point. Also recompute the reported evasion rates for all eleven methods at a common FPR (e.g., 1%) rather than at method-specific thresholds. If a held-out detector exceeds its FPR bound on the evasive student, or if reported rates drop below the claimed level at the common operating point, the 'eliminates all copyright identifiers' claim is not established as stated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline claim is a universal negative: the evasive student 'eliminates all copyright identifiers' across eleven DOV methods and outperforms nine prior attacks. For that claim to hold, the eleven methods must be a fair, representative test bed. The abstract gives no assurance that each DOV method is run at its published operating point (e.g., a fixed false-positive rate), that detector hyperparameters and underlying models are not inadvertently tuned to the student architecture, or that any detector was held out during development of the framework. If the authors selected thresholds or architectures after seeing detector outputs, 'all eleven' may demonstrate overfitting to the chosen suite rather than a general evasion capability. This is the weakest load-bearing premise. Separately, the supplied full text is mojibake and includes an unrelated arXiv header, so the experimental protocol, ablations, and error bars cannot currently be audited; the universal claim therefore cannot be accepted on the abstract alone. No internal inconsistency is alleged; the concern is about external validity and reproducibility of the comparative claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a unified evasion framework against Dataset Ownership Verification (DOV). A teacher model trained on a protected dataset transfers 'task-relevant yet identifier-independent' knowledge to a surrogate student via an out-of-distribution (OOD) dataset as an intermediary. The OOD transfer set is curated using vision-language and large language models, and 'selective transfer' is used to balance generalization accuracy against evasion of DOV detectors. The abstract claims that across diverse datasets and eleven DOV methods, the approach 'simultaneously eliminates all copyright identifiers' and outperforms nine existing evasion attacks in both generalization and effectiveness with moderate computational overhead.","tokens_in":14263,"tokens_out":2962,"duration_ms":39063,"significance":"If the claims hold, this is a significant negative result for the DOV literature: it would demonstrate a practical, unified evasion that defeats a broad suite of published detectors and that scales across datasets. The use of external, published DOV methods as evaluation benchmarks is a strength over self-defined metrics, and the teacher–student transfer via a VLM/LLM-curated OOD intermediary is a creative and timely idea. However, the significance is conditional on the empirical claims being credible, and the manuscript as supplied does not currently permit that assessment: the full text is unreadable mojibake, the abstract provides no run counts, error bars, per-detector breakdowns, or negative cases, and the universal 'eliminates all copyright identifiers' claim is a strong universal negative that requires unusually high-quality evidence.","major_comments":[{"comment":"The central empirical claim—'eliminates all copyright identifiers' across eleven DOV methods—is a universal negative, but the abstract reports no per-detector results, no false-positive-rate settings, no run counts, and no error bars. The full text supplied to the referee is corrupted mojibake, so the experimental protocol, tables, and ablations cannot be audited. This is load-bearing because the paper's contribution is precisely an empirical evasion claim; without specified operating points and variance estimates, the universal wording is unsupported.","section":"Abstract / Full text"},{"comment":"The 'all eleven DOV methods' claim requires that each detector be evaluated at its published operating point (e.g., a fixed false-positive rate) with its original hyperparameters and architectures, and that the evasion framework was not tuned to the specific eleven detectors. The manuscript gives no indication that a detector was held out during development, nor that thresholds were chosen before seeing detector outputs. If the selective-transfer trade-off weight or curation threshold was selected after observing detector results, the reported success may reflect overfitting to this particular suite rather than a general evasion capability. The authors should evaluate at standard operating points, report AUC/TPR@FPR, and include a held-out or adaptive detector analysis.","section":"Abstract (test-bed fairness)"},{"comment":"The 'selective transfer' and OOD curation involve several free parameters—trade-off weight, transfer set size, curation threshold, and OOD gallery choice—that are not analyzed for sensitivity. The claim of outperforming nine prior attacks 'in both generalization and effectiveness' depends on the chosen operating point. A Pareto-style analysis or a sweep over these parameters is needed to show that the reported point is not an isolated cherry-pick and to characterize the generalization–evasion trade-off.","section":"Framework description (selective transfer)"},{"comment":"The framework assumes that 'task-relevant yet identifier-independent' knowledge can be separated from the statistical traces that DOV detectors rely on. If those signals are entangled with task features, the student must either retain detectable traces or lose accuracy. The manuscript provides no direct evidence for this separability beyond the claimed results, which are currently unauditable. An ablation on datasets where detectors are known to rely on task-correlated statistics, or a theoretical argument about why the OOD intermediary removes the trace, is needed to make the premise credible.","section":"Framework premise (knowledge/trace separability)"}],"minor_comments":[{"comment":"The supplied full text is corrupted (mojibake), making it impossible to verify equation numbers, table values, and figure captions; please resubmit a clean, readable version.","section":"Full text"},{"comment":"The full text includes an unrelated arXiv header ('arXiv:2508.01066v3 [quant-ph] 30 Jan 2026') that should be removed.","section":"Full text"},{"comment":"The phrase 'eliminates all copyright identifiers' is ambiguous: it could mean zero detections at any threshold, or detections below a chosen false-positive rate. Please state the precise detection metric and the threshold for each DOV method.","section":"Abstract"},{"comment":"The claim of 'moderate computational overhead' should be quantified (e.g., wall-clock time relative to standard training, GPU-hours, or inference cost).","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The supplied full text is unreadable due to mojibake, which prevents any substantive check of the experiments. Before further review, please confirm with the authors that the submitted PDF or source was correctly converted. If the corrupted text is a pipeline artifact, the editor should obtain a clean version. Separately, the universal 'all eleven' claim will likely require additional experiments with held-out detectors and sensitivity analysis; the current abstract alone would not justify acceptance in a serious journal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe thing to know: this paper could be an important negative result for dataset ownership verification, but I cannot vouch for it. The copy I have is a corrupted extraction—mojibake plus an unrelated quant-ph header—so the entire methods, tables, and ablations are unreadable. The abstract alone is not enough to accept the universal claim that one framework eliminates all eleven DOV methods and beats nine prior attacks.\n\nWhat is genuinely new, on paper: the idea of transferring knowledge from a teacher trained on the copyright dataset to a surrogate student through an OOD intermediary, with an LLM/VLM selecting the most informative subset and selective transfer to balance generalization and evasion. That is a coherent and plausible way to break the correlation between task knowledge and the traces DOV detectors look for. If it works, it is the strongest evasion result in the subfield. The problem is real and the paper is attacking it in a direct, comprehensive way.\n\nThe soft spots are mostly about evidence, not logic. The headline claim is a universal negative with no error bars, no run counts, and no negative cases in the abstract. The test bed covers eleven DOV methods, but there is no indication that each detector is run at its published operating point, that their hyperparameters are not adapted to this specific attack, or that any method was held out during development. That is the classic recipe for overfitting to a detector suite. I would be careful about the word 'eliminates all,' because it reads as stronger than what a fixed suite of eleven methods can establish. The separability premise—that identifier-independent domain knowledge can be transferred while identifier traces are left behind—is plausible but not proven by the abstract; if those signals are entangled, the trade-off between evasion and accuracy will be real.\n\nWho gets value: anyone working on data provenance, dataset licensing, or model watermarking. The paper deserves a serious referee if a clean version can be obtained; the claims are important and the framework is worth examination. I would not cite it on the strength of this abstract, and I would not bring it to reading group until the full text is recoverable.\n\nRecommendation: engage with it, but demand the full text, the code, and the detector configurations. Desk rejection would be wrong given the potential; acceptance without inspection would also be wrong.","headline":"Potentially significant evasion framework, but the only available copy is unreadable, so the universal claims rest on the abstract alone.","tokens_in":14858,"tokens_out":2963,"would_cite":false,"duration_ms":34275,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a teacher trained on a protected dataset can pass only task-relevant, identifier-free knowledge through an out-of-distribution intermediary, erasing all copyright traces across eleven dataset-ownership verification…","keywords":["dataset ownership verification","evasion attack","teacher-student transfer","out-of-distribution data","vision-language models","large language models","data provenance","copyright protection"],"falsifier":"A concrete check: construct a protected dataset where the watermark or identifier is deliberately correlated with task labels, train the proposed student through the OOD gallery, and run the eleven DOV detectors with hyperparameters tuned to this setting; if any detector flags the student at a rate approaching that for a directly trained model, the universality of the evasion claim fails.","tokens_in":13902,"feed_emoji":"🕵️","tokens_out":4811,"duration_ms":54615,"temperature":0.7,"pith_summary":"This paper argues that dataset-ownership verification (DOV) methods—which try to detect whether a protected dataset was used to train a model—are far weaker than their evaluations suggest. The authors claim that a simple two-stage recipe can defeat them: train a teacher on the copyrighted data, then transfer only the knowledge the teacher has that is useful for the task but carries no dataset identifier, using an out-of-distribution dataset as a bridge. They report that across eleven DOV methods and nine prior evasion baselines, their framework removes every copyright identifier while keeping task accuracy competitive, at moderate extra compute. If true, this means current post-hoc provenance checks cannot reliably certify that a model has not been trained on stolen data.","feed_headline":"New evasion scheme defeats all 11 data-ownership detectors tested","feed_subtitle":"Teacher-student transfer through an out-of-distribution set erases copyright identifiers while keeping task accuracy, the authors report.","key_machinery":"The load-bearing object is the teacher–student transfer through an out-of-distribution (OOD) intermediate set. The teacher is trained on the protected dataset; the student is trained not on that data but on curated OOD images, using the teacher's task-oriented knowledge as guidance. Because the OOD set carries none of the dataset's identifier signals, the student has no trace of them to leave. The curation step uses vision-language and large language models to select the most informative and reliable subsets, and selective transfer decides how much task knowledge to pass, controlling the accuracy-versus-evasion trade-off.","core_discovery":"The central claim, stated on the paper's own terms, is that dataset-ownership verification is evadable in a unified way. Previous evasion attacks were oversimplistic, so DOV seemed safe; the paper introduces a framework in which a teacher model first learns from the copyright dataset, then a student model is trained on an out-of-distribution gallery curated by vision-language and large language models, receiving only task-relevant, identifier-independent knowledge from the teacher. The curated transfer subset is chosen for informativeness and reliability, and knowledge is transferred selectively to balance generalization against evasion. Experiments spanning eleven DOV methods show the student model eliminates all copyright identifiers and beats nine state-of-the-art evasion attacks on both generalization and effectiveness, with moderate computational overhead. The paper presents this as a proof of concept exposing key vulnerabilities in current DOV methods.","pith_inferences":["A direct extension would test the same two-stage transfer against training-data attribution and membership-inference methods, since those also rely on traces left in weights by the training data; if the separation assumption holds there, the attack would generalize beyond DOV.","The framework's dependence on LLM and VLM curation suggests a testable scaling prediction: as curation budget or model quality drops, evasion success should decline, and at some threshold a detector should start to catch the student.","If DOV detectors are meant to serve as forensic evidence, the paper's result implies they need a fundamentally different signal, such as oracle queries to the original data or monitoring of the training process, rather than inspecting final weights."],"forward_implications":["Current DOV evaluations that include only earlier, oversimplified evasion attacks will overstate how safe a model is from containing stolen data.","A stolen-data model can be made undetectable to all eleven tested DOV methods while keeping competitive task accuracy, so ownership verification alone is not enough to certify training data.","The selective-transfer mechanism gives an attacker a tunable trade-off between task accuracy and evasion effectiveness, so an attacker can choose how much utility to sacrifice.","The eleven methods being evaded all rely on traces that do not survive the teacher–student OOD transfer, which is evidence that DOV designs need new kinds of identifiers."],"supporting_citations":[],"fun_headline_variants":["Unified evasion beats all 11 dataset-ownership detectors","One framework evades every data-ownership detector tested","All 11 provenance checks fail against new transfer attack","Teacher-student trick wipes out dataset-ownership markers","Evasion beats 11 detectors: one transfer, zero traces"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument assumes that the identifier signals DOV methods rely on can be cleanly separated from the task knowledge a model needs, so a student trained only on the second kind through an OOD bridge loses nothing it needs while keeping none of the detectable traces.","fun_headline_variants_meta":{"raw":{"variants":["Unified evasion beats all 11 dataset-ownership detectors","One framework evades every data-ownership detector tested","All 11 provenance checks fail against new transfer attack","Teacher-student trick wipes out dataset-ownership markers","Evasion beats 11 detectors: one transfer, zero traces"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000697,"raw_usage":{"total_tokens":3162,"prompt_tokens":971,"completion_tokens":2191,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":587,"completion_tokens_details":{"reasoning_tokens":2111}},"tokens_in":587,"tokens_out":2191,"duration_ms":19748,"temperature":1.0,"reasoning_tokens":2111,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T05:51:32.627574+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete check: construct a protected dataset where the watermark or identifier is deliberately correlated with task labels, train the proposed student through the OOD gallery, and run the eleven DOV detectors with hyperparameters tuned to this setting; if any detector flags the student at a rate approaching that for a directly trained model, the universality of the evasion claim fails.","supporting_citations":[],"review_version":1}