{"id":"a76376a9-af1b-48c4-b789-71b7d8a0f671","arxiv_id":"2506.13563","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"A website fingerprinting attack classifier that, given a few known-bad test points, finds and suppresses backdoor-poisoned training data via influence scores and Fisher-information-guided parameter updates.","lead":"This paper adds a machine-unlearning step to a website fingerprinting classifier so it can auto-detect and forget backdoor-poisoned training samples, recovering roughly 80-87% accuracy on poisoned inputs where undefended classifiers collapse. A practical reason to read it: it proposes few-shot detection of poisoned points and parameter-level forgetting that avoids full retraining, relevant to any deep learning model trained on outsourced data.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 2's 'P ← update' is never defined, so the reported detoxification cannot be tied to any concrete parameter-suppression mechanism; the Section 4a influence-score separation is likewise unvalidated.","rationale":"The reader's REJECT verdict is well-founded, but my most load-bearing concern differs from the reader's stated weakest_assumption. The reader emphasizes the influence-score separation assumption in Section 4a; I agree that is a genuine risk and is untested, but the more immediate blocker is Algorithm 2's undefined update rule. Without an explicit update equation, the reported empirical results cannot be attributed to any concrete unlearning mechanism, and no independent assessment of the method is possible. The reader did list this gap in the rationale ('Algorithm 2's update is not defined'), so there is partial agreement. My proposed check directly forces the missing specification and a reproducibility test. This supports the existing REJECT verdict and does not change it.","tokens_in":15731,"tokens_out":7183,"duration_ms":79347,"concrete_test":"Obtain the exact mathematical expression for Algorithm 2's 'P ← update' (including the FIM estimator, K, and threshold), implement it on tor_100w_2500tr with 3000 poisoned points, and rerun Table 1. If the 87.03% poisoned-dataset accuracy cannot be reproduced under any reasonable reading of Section 4c, the claimed parameter-suppression mechanism is not defined well enough to support the central claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that targeted parameter adjustment removes the backdoor. The only description of that adjustment is Section 4c's verbal statement that Top-K parameters important to D_fo but not D_re are 'suppressed in proportion to the contribution value gap', and Algorithm 2 line 15 is literally 'P ← update'. No update equation, FIM estimator, learning rate, K value, or suppression schedule is provided. Consequently, the reported 87.03%/83.80% poisoned-dataset accuracies, the clean-dataset retention, and the ablations in Tables 3 and 4 cannot be checked or compared: any implementation choice, such as additive versus multiplicative suppression, how the FIM diagonal is estimated, or how K is chosen, could produce different results. This is not merely a presentation issue; it is an omitted specification of the mechanism the article claims. A related soft spot is that Eq. 4 assumes augmentation amplifies influence-score changes specifically for poisoned points, but no detection-level precision, recall, or ROC is reported, so even the input to Algorithm 2 is unverified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes an unlearning-enhanced website fingerprinting (WF) attack that aims to remove backdoor poisoning from a WF classifier. The method first uses influence scores of training points on a small set of known poisoned test points, combined with traffic-trace augmentation, to identify poisoned training samples; it then uses a Fisher Information Matrix to select and suppress parameters that are important for the forgotten (poisoned) set but not for the retained (clean) set. Experiments on closed-world and open-world Tor datasets report poisoned-dataset accuracy of 87.03% and 83.80%, respectively, with test accuracy around 79-83% and a 2-3x runtime speedup compared with DF, TF, and AWF baselines, whose poisoned/test accuracy is below 10% (or about 40% for AWF under open-world). Ablation experiments on parameter selection and parameter suppression are also reported.","tokens_in":15900,"tokens_out":5409,"duration_ms":56705,"significance":"The problem is timely and the high-level idea is reasonable: few-shot identification of poisoned training points via influence scores, followed by parameter-level unlearning via FIM sensitivity, could be a useful contribution to WF robustness. The paper uses public WF datasets and reports large empirical gains over undefended WF classifiers, and the runtime advantage is a plausible practical benefit. However, as written, the central mechanism is under-specified and the evaluation omits key controls, so the contribution is not established at the level required for publication.","major_comments":[{"comment":"The central unlearning step is not specified: line 15 of Algorithm 2 is literally \"P <- update\". The paper's claimed detoxification depends on how the selected Top-K parameters are suppressed, but no update equation, FIM estimator, learning rate, K value, or suppression schedule is given. Without this specification, Tables 1-4 cannot be reproduced, and the claim that parameter adjustment removes the backdoor (Eq. 3) is not checkable. Please provide the full parameter update rule, the exact FIM computation (diagonal or full, empirical samples, damping), the K values used, and the suppression schedule.","section":"Section 4c, Algorithm 2"},{"comment":"The detection signal is not validated. The paper asserts that clean points' influence scores are stable under trace augmentation while poisoned points' scores are \"significantly amplified\", but no detection-level precision, recall, ROC, or distribution comparison is reported. In addition, the influence-score computation is not detailed (e.g., how the Hessian inverse is estimated for the WF classifier), and Eq. (4) is a ratio while the text describes a difference; the definition of S and the handling of negative or near-zero influence scores are not given. Because D_fo is the input to the unlearning module, this omission is load-bearing.","section":"Section 4a, Eq. (4)"},{"comment":"The comparison set includes only undefended WF classifiers (DF, TF, AWF). To support the claim that the proposed parameter-based unlearning is effective, the evaluation needs backdoor-removal baselines such as fine-tuning on D_re, retraining from scratch on D_re, selective synaptic dampening, SCRUB, or BadT. Without such baselines, the results show only that poisoning harms undefended WF models, not that the proposed unlearning is competitive with or better than existing unlearning methods.","section":"Section 5a-5d (Tables 1-4)"},{"comment":"The reported numbers are inconsistent regarding time and lack variance. Table 1 reports \"Ours Time/s = 297.53\" in the closed-world setting, while Table 3 reports \"Baseline = 10.60s\" for the same closed-world setting (with a different poisoning count), and the text does not state whether Table 3 reports only the unlearning module or the full pipeline. Also, the statement that each result is the average of three random seeds is not supported by any standard deviation or confidence interval. Please clarify the measurement scope and report dispersion metrics.","section":"Section 5b/5c, Tables 1 and 3"},{"comment":"The detection threshold is described as \"experimentally determined\" and K as \"set according to user requirements\". If these hyperparameters are selected on the same poisoned test set and trigger distribution used for evaluation, the headline accuracies are conditional on that fitting. Please describe the hyperparameter selection procedure (e.g., a separate validation set) and provide a sensitivity analysis over the threshold and K; otherwise the reported numbers cannot be interpreted as predictive performance.","section":"Sections 4a and 5b"},{"comment":"The method assumes the defender has a set D_ab of 300 known poisoned test points. The paper does not explain how the WF analyst obtains such labeled poisoned points in practice, nor why these points are representative of the backdoor trigger distribution. This assumption is strong and should be justified or relaxed in the threat model; without it, the \"few-shot detection\" claim is only partially supported.","section":"Section 4a and Section 5 (poisoning setup)"}],"minor_comments":[{"comment":"The manuscript contains numerous typos and duplicated references (e.g., [3] and [22], [4] and [34]); a careful proofreading pass is needed.","section":"Throughout"},{"comment":"The text says the difference in influence scores is amplified, but Eq. (4) is a ratio; please align the notation and define all symbols in the equation.","section":"Section 4a, Eq. (4)"},{"comment":"The manuscript contains an extraneous header \"南京航空航天大学本科毕业设计（论文）\" (undergraduate thesis template text) that should be removed.","section":"Pages 23-25"},{"comment":"The figure captions are garbled (e.g., Figure 1 contains repeated phrases) and the figures are not described in enough detail in the body text; please revise.","section":"Figures 1 and 2"},{"comment":"The statement that K \"can be set according to user requirements\" gives no guidance on selecting K; the experiments should report the K values used in Tables 1-4.","section":"Section 4c"}],"recommendation":"reject","confidential_remarks":"The manuscript appears to be an unedited thesis draft, with template artifacts and duplicated references. My recommendation is driven primarily by the unspecified unlearning update mechanism and the absence of both detection-level evaluation and unlearning baselines; these are substantive gaps, not merely presentation issues."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new thing here is the combination: influence functions to find poisoned training points from a few known-bad test points, trace augmentation to amplify the signal, and then Fisher-information-based selective parameter suppression to forget the backdoor without retraining. I have not seen that exact pipeline in the WF literature, and the idea is sensible. The paper also uses public datasets, compares against DF/TF/AWF, and includes ablations. Those are real strengths. The reported numbers are large enough that, if the method works as described, it is useful to WF practitioners. But the manuscript as written cannot support the headline claims. Algorithm 2 line 15 is literally 'P <- update'. There is no update equation, no FIM estimator, no learning rate, no K, no suppression schedule. The influence-score computation is also not detailed. So the 87.03% and 83.80% poisoned-dataset accuracies are unreproducible; any of a dozen implementation choices could produce different results. The Section 4a claim that trace augmentation amplifies influence-score differences specifically for poisoned points is plausible but unvalidated, with no precision, recall, or ROC for the detector itself. The detection threshold and Top-K are fitted, and the evaluation uses the same trigger distribution used for poisoning, so the numbers are conditional on those choices. There are also no unlearning baselines (BadT, SCRUB, EU-k, CF-k are cited but not compared), no variance reporting, and the abstract's 'complete removal' is contradicted by the 80-87% accuracy on poisoned data, which is good relative to baselines but not complete removal. The unrelated Nanjing University of Aeronautics and Astronautics header is a minor hygiene issue, but it signals an unpolished draft. On balance: the central idea is not circular and the authors seem to understand what they are trying to do, but the paper is a sketch, not a verifiable report. The soft spots are load-bearing, not cosmetic. If a revision spelled out the algorithms and shipped code, the result might be a solid contribution. As is, I would not cite it and would not trust the numbers. Still, I would not desk-reject it outright; the problem is real and the approach is novel enough to deserve expert referee time, with the expectation of major revision.","headline":"A plausible new combination of influence-function detection and FIM-based unlearning for WF backdoors, but the core mechanisms are specified just enough to see that they are not specified at all.","tokens_in":690,"tokens_out":976,"would_cite":false,"duration_ms":22941,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A Website Fingerprinting classifier can locate its own poisoned training points from a few known-bad test points and erase the backdoor by selectively adjusting parameters, without retraining.","keywords":["Website Fingerprinting","Backdoor Poisoning","Machine Unlearning","Influence Functions","Fisher Information Matrix","Tor anonymity","Backdoor Detection","Parameter-based Unlearning"],"falsifier":"Train a poisoned WF model using the paper's own procedure so the poisoned training points are known by construction, then compute the Eq. (4) influence-score difference for every training point against the 300 known-bad test points. If the clean and poisoned difference distributions overlap so heavily that no threshold reproduces the paper's detection accuracy---for instance, if the ROC curve area is far from 1---then the detector is not the source of the reported poisoned-data accuracy.","tokens_in":15508,"feed_emoji":"🛡️","tokens_out":8347,"duration_ms":80091,"temperature":0.7,"pith_summary":"The paper is trying to show that a Website Fingerprinting classifier that has been backdoor-poisoned can repair itself using machine unlearning rather than retraining. Detection starts from a small number of known-bad test points, computes each training point's influence on those test points, and amplifies the signal by re-running the same influence calculation after random trace augmentation. The paper argues that poisoned training points show a much larger influence-score change than clean ones under this augmentation, and that this locates the poisoned data. The poisoned data is then forgotten by adjusting only the model parameters that matter most to it, which the experiments claim restores around 80 percent accuracy on trigger-bearing traffic while keeping clean accuracy near 95 percent and cutting runtime 2-3 times.","feed_headline":"Poisoned Tor traffic classifiers restored to ~87% accuracy","feed_subtitle":"Influence scores locate poisoned training points; targeted parameter tweaks erase the backdoor 2-3x faster than retraining.","key_machinery":"The carrying mechanism is an influence-score ratio used as a poisoning probe. For each training point, the method computes its influence contribution $S((x_i,y_i)^{tr}, D_{ab})$ to a small set of known-bad test points, then recomputes it after randomly augmenting those test points through insertion, splitting, merging, and reversal of Tor trace units with rematched labels. The detection statistic is $S((x_i,y_i)^{tr}, D_{ab}) / S((x_i,y_i)^{tr}, D'_{ab}) - 1$, which the paper claims is stable for clean points and amplified for points with backdoor triggers. The forgetting step uses the Fisher Information Matrix, the negative expected Hessian of the model's log-likelihood, as a per-parameter importance measure over the retained and forgotten subsets; the Top-K parameters that are most important to the forgotten set and least important to the retained set are then suppressed in proportion to that importance gap.","core_discovery":"The central claim is that backdoor poisoning of a WF classifier can be undone without retraining by combining two cheap signals. First, the difference in influence scores before and after trace augmentation separates poisoned from clean training points, so the poisoned subset can be recovered automatically from a few hundred known-bad test points. Second, the Fisher Information Matrix ranks how important each model parameter is to the poisoned subset versus the clean subset; suppressing only the Top-K parameters that are highly important to poisoned data and unimportant to clean data removes the backdoor's effect. On the closed-world dataset the paper reports 87.03% poisoned-data accuracy and 96.26% clean-data accuracy; on the open-world dataset it reports 83.80% and 94.82%, compared with under 10% poisoned-data accuracy for the DF and TF baselines, plus a 2-3x runtime speedup.","pith_inferences":["A natural extension is to run the same influence-score probe before training, using the detected points to scrub backdoor triggers from a dataset rather than to repair a model; nothing in the paper's mechanism restricts it to the post-training setting.","An adaptive poisoner who knows this defense could try to make triggers invariant under insert/split/merge/flip augmentation, which would close the influence-score gap; the paper does not test such an adversary.","The reported 'few-shot' regime uses 300 known-bad test points; how detection degrades as that number shrinks is not measured, so the practical promise of the method depends on an untested scaling curve."],"forward_implications":["A poisoned WF classifier can be returned to useful operation: roughly 87% accuracy on trigger-bearing traffic in the closed-world setting and 84% in the open-world setting, with clean-data accuracy around 95%.","The repair requires only a few hundred known-bad test points and no full audit of the training set, so the method is deployable in the outsourced-training scenario that motivates it.","Selective parameter adjustment instead of retraining gives a 2-3x runtime speedup across the whole WF pipeline.","The ablation shows the parameter-selection step is the load-bearing part: without it, open-world poisoned accuracy collapses to about 26%, and clean accuracy drops by roughly 20 points."],"supporting_citations":[{"why":"Defines the Deep Fingerprinting convolutional baseline whose poisoned accuracy the method must lift from under 10%.","marker":"[3]"},{"why":"Provides the Triplet Fingerprinting baseline and the n-shot learning setting the paper contrasts with its few-shot detection.","marker":"[5]"},{"why":"Supplies the Automated WF baseline and the public Tor WF datasets used in the closed- and open-world experiments.","marker":"[6]"},{"why":"Contributes the influence-function calculation that produces the per-training-point scores used for poisoned-point detection.","marker":"[49]"},{"why":"Contributes the Fisher Information Matrix as the per-parameter importance measure used to select parameters for suppression.","marker":"[8]"},{"why":"Earlier selective-forgetting method that injects FIM-scaled noise; the paper's Top-K adjustment is presented as a faster alternative.","marker":"[48]"},{"why":"Establishes the backdoor-neural-network threat model: trigger-bearing inputs flip predictions while clean accuracy stays high.","marker":"[35]"},{"why":"Shows how to embed backdoor triggers in Tor website fingerprints, defining the poisoning attack used to build the poisoned datasets.","marker":"[36]"},{"why":"Supplies the memorization premise that some parameters specialize to a few memorized examples, justifying Top-K parameter suppression.","marker":"[44]"}],"fun_headline_variants":["Unlearning scours backdoor-poisoned Tor traces","Influence scores expose and erase Tor backdoors","Backdoor-proof WF: unlearning beats retraining","Unlearning drops backdoor accuracy to 87% in Tor"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that poisoned training points produce much larger influence-score changes than clean points under trace augmentation; if triggers are augmentation-robust or clean points are equally volatile, the threshold detector stops separating the two classes and the whole unlearning pipeline collapses.","fun_headline_variants_meta":{"raw":{"variants":["Unlearning scours backdoor-poisoned Tor traces","Influence scores expose and erase Tor backdoors","Backdoor-proof WF: unlearning beats retraining","Unlearning drops backdoor accuracy to 87% in Tor"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000477,"raw_usage":{"total_tokens":2391,"prompt_tokens":997,"completion_tokens":1394,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":613,"completion_tokens_details":{"reasoning_tokens":1329}},"tokens_in":613,"tokens_out":1394,"duration_ms":9675,"temperature":1.0,"reasoning_tokens":1329,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:58:12.646244+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a poisoned WF model using the paper's own procedure so the poisoned training points are known by construction, then compute the Eq. (4) influence-score difference for every training point against the 300 known-bad test points. If the clean and poisoned difference distributions overlap so heavily that no threshold reproduces the paper's detection accuracy---for instance, if the ROC curve area is far from 1---then the detector is not the source of the reported poisoned-data accuracy.","supporting_citations":[{"cited_title":"Sirinam, N","cited_arxiv_id":null,"evidence_quote":"Provides the Triplet Fingerprinting baseline and the n-shot learning setting the paper contrasts with its few-shot detection."},{"cited_title":"Automated Website Fingerprinting through Deep Learning[C]","cited_arxiv_id":null,"evidence_quote":"Supplies the Automated WF baseline and the public Tor WF datasets used in the closed- and open-world experiments."},{"cited_title":"Understanding Black -box Predictions via Influence Functions","cited_arxiv_id":null,"evidence_quote":"Contributes the influence-function calculation that produces the per-training-point scores used for poisoned-point detection."},{"cited_title":"A.; Milan, K.; Quan, J.; Ramalho, T.; Grabska -Barwinska, A.; et al","cited_arxiv_id":null,"evidence_quote":"Contributes the Fisher Information Matrix as the per-parameter importance measure used to select parameters for suppression."},{"cited_title":"Eternal sunshine of the spotless net: Selective forgetting in deep networks","cited_arxiv_id":null,"evidence_quote":"Earlier selective-forgetting method that injects FIM-scaled noise; the paper's Top-K adjustment is presented as a faster alternative."},{"cited_title":"BadNets: Identifying Vulnerabilities in the Ma chine Learning Model Supply Chain[C]","cited_arxiv_id":null,"evidence_quote":"Establishes the backdoor-neural-network threat model: trigger-bearing inputs flip predictions while clean accuracy stays high."},{"cited_title":"Red Pill and Blue Pill: Controllable Website Fingerprinting De fense via Dynamic Backdoor Learning[C]","cited_arxiv_id":null,"evidence_quote":"Shows how to embed backdoor triggers in Tor website fingerprints, defining the poisoning attack used to build the poisoned datasets."}],"review_version":2}