{"id":"e02fa61a-1480-4391-82e6-95b8ea415b71","arxiv_id":"2505.04015","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A post-training defense that linearizes and merges fully connected layers reduces Trojan attack success rate on Vision Transformers while compressing the model.","lead":"MergeGuard is a post-training defense that makes parts of a neural network linear, then merges those layers, which shrinks the model and weakens hidden backdoor triggers. It reports lower attack success rates than fine-tuning on CIFAR-10 models while keeping test accuracy mostly intact.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"MergeGuard's central claim depends on the untested assumption that backdoors concentrate in terminal fully connected layers; Section IV-A selects layers based on a self-cited conjecture and Section V admits layer-selection algorithms are future work.","rationale":"The reader's weakest assumption correctly identifies the terminal-layer concentration premise, and the manuscript text supports that concern: Section IV-A explicitly bases layer selection on a self-cited conjecture, and Section V lists automatic layer selection as future work. This is genuinely load-bearing because the central claim is that MergeGuard is architecture-agnostic and removes trojans while preserving accuracy; if the layer choice is arbitrary, the observed ASR reduction could be a side effect of aggressive compression rather than the proposed mechanism. The empirical core remains credible and the conditional verdict is appropriate, so no change to the reader's verdict is needed. A layer-position ablation is the direct check because it isolates whether the method works because it targets terminal layers or because it applies enough perturbation anywhere.","tokens_in":9486,"tokens_out":8037,"duration_ms":82981,"concrete_test":"Run MergeGuard with identical hyperparameters but regularize the first four ViT blocks (or the first three PreAct-ResNet18 layers) instead of the last ones, plus a variant regularizing all fully connected layers, across the five attack types with at least three seeds each. If ASR stays comparably low with clean accuracy within about one point, the terminal-layer assumption is unnecessary; if ASR remains high or accuracy collapses, the defense's effectiveness depends on an unvalidated layer-selection choice, and the paper should scope its claims accordingly.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing assumption is that Trojan-related neurons are concentrated in the last few fully connected layers, so merging exactly those layers removes the backdoor while preserving accuracy. In Section IV-A, the authors apply regularization to the last three layers of PreAct-ResNet18 and the last four of the Vision Transformer, 'presuming the trojan's presence in the terminal layers based on the conjectures of previous works [8]'; reference [8] is the authors' own prior work, and no ablation varies this choice. The paper's own conclusion lists 'developing algorithms for the selection of MergeGuard layers' as future work, confirming that layer selection is a heuristic rather than a validated component. If the premise fails for attacks whose trigger is encoded in earlier or attention layers, MergeGuard may simply be degrading the model rather than removing backdoor-related neurons, which would invalidate the architecture-agnostic claim and weaken the 'maintains model accuracy' claim. The tables also report no error bars or alternative layer sets, so the observed ASR drop cannot be cleanly attributed to the terminal-layer hypothesis versus generic perturbation or pruning effects. This makes the defense's core mechanism under-specified at the exact point where its generality rests.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"MergeGuard is a post-training defense against Trojan/backdoor attacks. It adds a regularization term to a fine-tuning objective that drives the negative slope of PReLU (and similar) activations toward 1, allowing consecutive fully connected layers to be merged into a single equivalent linear layer; the authors argue that this removes layers containing backdoor-related neurons. The method is evaluated on CIFAR-10 against TrojanNN, WaNet, BadNet, Blended, and SIG on PreAct-ResNet18 and ViT-base-16, and compared with fine-tuning, Fine-Pruning, FTSAM, and Neural Cleanse. The paper reports lower average ASR, faster runtime, and a 15% parameter reduction on the ViT, and claims that MergeGuard maintains model accuracy and generalizability.","tokens_in":9782,"tokens_out":5612,"duration_ms":58174,"significance":"If the claims hold, MergeGuard would be a practical, architecture-agnostic mitigation that doubles as a compression method, with particular value for Vision Transformers where existing defenses struggle. The paper's strengths include an open-source implementation, evaluation against five attacks on two architectures, and a clearly stated threat model. However, the central claims are currently conditional: the key probabilistic bound in Eq. (4) is asserted without proof, the layer-selection heuristic rests on a self-cited conjecture, the ViT results in Table II show nontrivial accuracy drops, and no error bars or ablations are reported. These gaps prevent the paper from supporting its strong conclusions as written.","major_comments":[{"comment":"Equation (4) is asserted with the phrase 'It can be shown' but no proof is given. This bound is load-bearing because it justifies the (1 - alpha)^2 regularizer used throughout the method. Please provide a complete derivation, specify whether the input X is treated as a vector or scalar in the absolute value, state the exact conditions on W1, W2, b1, b2 under which C is finite, and clarify whether the bound holds uniformly during training as alpha changes. Without this proof, the theoretical grounding of MergeGuard is incomplete.","section":"Section III-D, Eq. (4)"},{"comment":"The core mechanism assumes that backdoor-related neurons are concentrated in the terminal fully connected layers, but the supporting citation [8] is an earlier work from the same research group, and no ablation varies the set of regularized layers. The paper's own conclusion lists 'developing algorithms for the selection of MergeGuard layers' as future work, confirming that this choice is a heuristic. Please add an ablation that, for example, regularizes earlier layers, a random layer subset, or the full network, and report ASR and test accuracy for each choice. Without such an experiment, the observed ASR reduction cannot be cleanly attributed to terminal-layer trojan removal rather than to generic perturbation or compression effects.","section":"Section IV-A"},{"comment":"The abstract claims MergeGuard 'maintains model accuracy,' but Table II shows ViT test accuracy drops from 98.03 to 92.70 for TrojanNN, from 98.16 to 92.71 for Blended, and from 95.85 to 93.55 for BadNet, i.e., drops of 1.5 to 5.4 points relative to the trojaned baseline. In addition, for TrojanNN and BadNet the post-MergeGuard ASR (7.39% and 11.38%) is higher than after Neural Cleanse (0.72% and 0.27%), so the claim of consistently outperforming existing defenses is too strong as stated. Please revise the claims, quantify the accuracy trade-off, and report error bars over multiple runs.","section":"Table II and Abstract"},{"comment":"No standard deviations, confidence intervals, or repeated-run results are reported for any defense. Given that the differences between methods are sometimes small and that fine-tuning on only 5% of the clean dataset will have nontrivial variance, single runs do not support the comparative conclusions drawn in the text. Please report mean and standard deviation over at least three random seeds, and disclose the grid-searched learning rates and other hyperparameters for each defense in the main text or supplement.","section":"Tables I and II, Section IV-B"}],"minor_comments":[{"comment":"There is a typo: 'exeptional' should be 'exceptional'.","section":"Introduction, Section I"},{"comment":"The compression ratio formula omits bias terms and assumes a particular parameter count; please state explicitly whether biases are counted and give the exact expression used to compute the 15% parameter reduction reported in Section IV.","section":"Section III-E, Eq. (6)"},{"comment":"The notation for the ELU variant is confusing: beta is introduced as the ELU parameter and later alpha controls the linear/nonlinear balance, but the relationship between alpha, beta, and the original ELU is not specified. Please clarify how alpha is optimized and how this variant connects to the proof of Eq. (4).","section":"Section III-G"},{"comment":"The expansion 'Feature Squeezing and Model Augmentation (FTSAM)' appears to be incorrect; FTSAM in reference [12] stands for fine-tuning with sharpness-aware minimization. Please correct the terminology.","section":"Section IV-B, Figure 3"},{"comment":"The phrase 'architecture-agnostic' is used repeatedly, but only two architectures are evaluated; please either test at least one additional architecture family (e.g., a transformer beyond ViT-base or a different CNN) or qualify the claim as applying to the tested architectures.","section":"Section IV-A"}],"recommendation":"major_revision","confidential_remarks":"The manuscript fits the journal's scope and the proposed defense is interesting, but the central claims are currently under-supported: the theory is missing a proof, the layer-selection premise is validated only by a self-cited conjecture, and the accuracy-maintenance claim is contradicted by the ViT table. The paper can likely be repaired with an added proof, an ablation over layer choices, multi-seed results, and revised claims, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The useful core of this paper is a real empirical finding: linearizing and merging the last few fully connected layers of a trojaned Vision Transformer drops attack success rate from 90+% to single digits across five backdoor attacks, while fine-tuning, FTSAM, and Neural Cleanse each fail on at least one attack. That is a meaningful result for a model class where defenses often don't transfer, and the speed advantage (10–17x over FP/NC) makes it practical. The compression effect (15% parameter reduction) is a bonus.\n\nThe method itself is simple: add a regularizer that pushes PReLU slopes toward 1, then merge adjacent linear layers. The novelty isn't the compression trick; it's applying it as Trojan mitigation and showing it works on ViTs. The paper is reasonably honest about the components being known.\n\nThe main weaknesses are the following. (1) The abstract says the method “simultaneously improves model generalizability and performance.” Table II shows the opposite on ViTs—accuracy drops by 1.5 to 5.4 points depending on attack. That claim needs to be corrected. (2) Layer selection is the load-bearing assumption. They regularize the last three/four layers based on a conjecture from their own prior work [8], and the conclusion admits layer-selection algorithms are future work. No ablation tests other layer sets, so we don't know whether the ASR drop comes from removing terminal layers specifically or just from compressing any part of the network. Without that, the stated mechanism (“removing layers containing backdoor-related neurons”) is unverified. (3) There are no error bars or multiple seeds; this is a single-run comparison, which is thin for a security claim. (4) The bound in Eq. (4) is asserted without proof. It's not central to the empirical story, but it should either be proved or explicitly labeled as a conjecture.\n\nNone of these are fatal. The core result is that a simple post-processing regularizer consistently reduces ASR on ViTs where other defenses fail. That is worth publishing after revision. The paper also provides code, which helps reproducibility.\n\nWho this is for: researchers working on backdoor defenses, especially for transformer architectures. It's not a full solution; it's one more tool, and a cheap one. The unvalidated layer-selection heuristic is a clear gap for follow-up work.\n\nI'd send this to peer review, expecting the authors to add layer-selection ablations, error bars, and correct the abstract. The idea deserves referee time.","headline":"A promising post-training backdoor defense for ViTs that consistently lowers ASR, but the paper overstates accuracy preservation and leans on an untested layer-selection hypothesis.","tokens_in":10240,"tokens_out":2922,"would_cite":true,"duration_ms":28753,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MergeGuard removes trojan backdoors from trained models by linearizing and merging fully connected layers, preserving accuracy while sharply cutting attack success rate.","keywords":["trojan attacks","backdoor defense","model compression","layer merging","Vision Transformer","PReLU linearization","post-training defense","attack success rate"],"falsifier":"Train or obtain a trojaned model whose backdoor is planted in early layers (for example, by poisoning the patch embedding or first encoder blocks of a Vision Transformer, or by an attack that places the trigger pattern in a mid-network residual path), run MergeGuard under the same protocol, and measure attack success rate. If ASR stays high while accuracy is preserved, the terminal-layer premise fails. A simpler version: apply MergeGuard to different layer subsets of the same trojaned model and check whether ASR reduction changes substantially, which would show whether the fixed last-layer choice is essential.","tokens_in":9315,"feed_emoji":"🛡️","tokens_out":8681,"duration_ms":78716,"temperature":0.7,"pith_summary":"MergeGuard is a post-training defense against trojan (backdoor) attacks that works by nudging activation functions toward linearity and then fusing adjacent fully connected layers into a single equivalent layer. The paper's key claim is that this fusion deletes terminal-layer neurons that encode the trigger, so the model becomes clean without retraining from scratch. On CIFAR-10, the method keeps Vision Transformer test accuracy near the trojaned baseline (93.2% vs. 94.7%) while lowering average attack success rate from 93.5% to 5.2%, and it outperforms standard fine-tuning, Fine-Pruning, FTSAM, and Neural Cleanse, several of which leave ASR above 25% on transformers. Because the same procedure also reduces parameter count by up to 15% and runs much faster, the paper argues that compression and trojan mitigation can be achieved in one pass.","feed_headline":"Merging layers cuts trojan attack success to 5% on transformers","feed_subtitle":"Post-training layer fusion keeps model accuracy and outperforms fine-tuning on vision transformers.","key_machinery":"The load-bearing object is the mergeable layer: two consecutive fully connected (or convolutional) layers separated by a PReLU activation $\\mathrm{PReLU}_\\alpha(x)=\\max(0,x)+\\alpha\\min(0,x)$. As $\\alpha$ approaches 1, the PReLU becomes the identity, and the composition collapses to one equivalent linear layer, so the network depth drops by one with zero additional error if $\\alpha=1$ exactly. The regularizer $(1-\\alpha)^2$ added to the cross-entropy loss during a brief fine-tuning on clean data is what drives this collapse, and the paper proves that for any $\\delta\\in[0,1]$, the probability that $|Y_{\\mathrm{linear}}-Y_\\alpha|^2$ exceeds $C(1-\\alpha)^2$ is below $\\delta$, with $C$ depending on the largest singular value of $W_2W_1$. In the experiments, regularization is applied only to the last three layers of PreAct-ResNet18 and the last four layers of the Vision Transformer, reflecting the paper's working conjecture that trojan-related neurons concentrate in terminal layers.","core_discovery":"MergeGuard's central claim is that trojan mitigation can be achieved by post-training compression of the final fully connected blocks. Starting from a trojaned model and a small clean dataset, the method fine-tunes with a regularizer that penalizes $(1-\\alpha)^2$ for PReLU-family activations, driving the slope $\\alpha$ toward 1 so that the activation approximates the identity. Two adjacent linear layers then collapse into a single layer with weight $W_2 W_1$ and bias $W_2 b_1 + b_2$, removing the intermediate layer and the backdoor-related neurons it contains. The paper derives a probabilistic bound showing the error of this linearization is controlled by $(1-\\alpha)^2$, and it evaluates the procedure on PreAct-ResNet18 and ViT-base-16 against five attacks (TrojanNN, WaNet, BadNet, Blended, SIG), reporting that on Vision Transformers MergeGuard keeps accuracy essentially intact while reducing average attack success rate from 93.5% to 5.2%—a result no compared method achieves. The paper further claims that the same mechanism yields a 15% parameter reduction and 14% MAC reduction on ViT, and that the defense is model-agnostic.","pith_inferences":["If terminal-layer concentration is not universal, MergeGuard could be combined with a weight-signature scan to select which layers to linearize; the same regularizer could then serve as a per-layer trojan probe, since layers that resist linearization are likely to carry trigger-specific nonlinear structure.","Because the regularizer strength is fixed at 1 in all experiments, there is likely a trade-off frontier between ASR reduction and accuracy; sweeping the coefficient could let practitioners tune the defense for a given deployment risk.","The layer-merging operation changes the model's function class, so it may also alter robustness to adversarial perturbations or out-of-distribution inputs; measuring those before and after could reveal side effects beyond trojan removal.","The compression ratio formula shows bottleneck MLPs can expand rather than compress, so on architectures with narrow hidden layers MergeGuard would need to be applied to wider blocks only—a constraint the paper flags but does not turn into a design rule."],"forward_implications":["Transformer models acquired from untrusted sources can be cleansed with a short fine-tuning on a small clean dataset, without per-class trigger reverse engineering.","The mitigation comes with compression: on ViT-base-16, parameter count drops 15% and MACs drop 14% with negligible accuracy loss.","Because MergeGuard is orders of magnitude faster than Fine-Pruning and Neural Cleanse, it is practical for large models where those methods are prohibitive.","The method generalizes to convolutional architectures such as PreAct-ResNet18, though the compression ratio can be negative in bottleneck-shaped MLPs, so layer choice matters.","Existing cleansing methods should not be assumed to transfer from CNNs to vision transformers; the paper's comparison documents that standard fine-tuning and FTSAM often leave ASR near 100% on ViT."],"supporting_citations":[{"why":"Supplies the conjecture that trojan-related neurons are concentrated in the terminal layers, motivating the choice of which layers to regularize.","marker":"[8]"},{"why":"Baseline defense whose analysis of why vanilla fine-tuning fails motivates MergeGuard's weight-perturbing approach; also compared in the experiments.","marker":"[12]"},{"why":"Defines BadNets, a data-poisoning attack used as one of the five evaluation attacks.","marker":"[18]"},{"why":"Defines TrojanNN, a trigger-reverse-engineering and retraining attack used in the evaluation.","marker":"[20]"},{"why":"Defines WaNet, a warping-based backdoor attack used in the evaluation.","marker":"[21]"},{"why":"Defines Blended, a blending-based backdoor attack used in the evaluation.","marker":"[19]"},{"why":"Defines SIG, a label-poisoning-free signal backdoor used in the evaluation.","marker":"[22]"},{"why":"Fine-Pruning, a compared defense combining pruning and fine-tuning, whose compute cost motivates MergeGuard's efficiency claim.","marker":"[24]"},{"why":"Neural Cleanse, a compared defense whose per-class trigger reverse engineering is computationally expensive, motivating MergeGuard's speed advantage.","marker":"[13]"},{"why":"Supplies the benchmark's predefined attack and training parameters used for consistent CIFAR-10 experiments.","marker":"[11]"}],"fun_headline_variants":["Layer merging cuts trojan attack success to 5.2% on ViT","MergeGuard collapses layers, backdoor success drops from 93.5% to 5.2%","Post-training layer fusion keeps accuracy, drops trojan success to 5%","Linearizing layers cuts backdoor success, compute on ViT","Merging layers keeps accuracy, drops backdoors to 5% and compute by 14%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that trojan-related neurons are concentrated in the terminal fully connected layers, so compressing only the last few layers removes the trigger; the paper relies on this conjecture without ablating other layer choices.","fun_headline_variants_meta":{"raw":{"variants":["Layer merging cuts trojan attack success to 5.2% on ViT","MergeGuard collapses layers, backdoor success drops from 93.5% to 5.2%","Post-training layer fusion keeps accuracy, drops trojan success to 5%","Linearizing layers cuts backdoor success, compute on ViT","Merging layers keeps accuracy, drops backdoors to 5% and compute by 14%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001002,"raw_usage":{"total_tokens":4210,"prompt_tokens":890,"completion_tokens":3320,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":506,"completion_tokens_details":{"reasoning_tokens":3210}},"tokens_in":506,"tokens_out":3320,"duration_ms":23427,"temperature":1.0,"reasoning_tokens":3210,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:39:39.509335+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train or obtain a trojaned model whose backdoor is planted in early layers (for example, by poisoning the patch embedding or first encoder blocks of a Vision Transformer, or by an attack that places the trigger pattern in a mid-network residual path), run MergeGuard under the same protocol, and measure attack success rate. If ASR stays high while accuracy is preserved, the terminal-layer premise fails. A simpler version: apply MergeGuard to different layer subsets of the same trojaned model and check whether ASR reduction changes substantially, which would show whether the fixed last-layer choice is essential.","supporting_citations":[{"cited_title":"Trojan signatures in dnn weights,","cited_arxiv_id":null,"evidence_quote":"Supplies the conjecture that trojan-related neurons are concentrated in the terminal layers, motivating the choice of which layers to regularize."},{"cited_title":"Enhancing fine- tuning based backdoor defense with sharpness-aware minimization,","cited_arxiv_id":null,"evidence_quote":"Baseline defense whose analysis of why vanilla fine-tuning fails motivates MergeGuard's weight-perturbing approach; also compared in the experiments."},{"cited_title":"Trojaning attack on neural networks,","cited_arxiv_id":null,"evidence_quote":"Defines TrojanNN, a trigger-reverse-engineering and retraining attack used in the evaluation."},{"cited_title":"A new backdoor attack in cnns by training set corruption without label poisoning,","cited_arxiv_id":null,"evidence_quote":"Defines SIG, a label-poisoning-free signal backdoor used in the evaluation."},{"cited_title":"Fine-pruning: Defending against backdooring attacks on deep neural networks,","cited_arxiv_id":null,"evidence_quote":"Fine-Pruning, a compared defense combining pruning and fine-tuning, whose compute cost motivates MergeGuard's efficiency claim."},{"cited_title":"Neural cleanse: Identifying and mitigating backdoor attacks in neural networks,","cited_arxiv_id":null,"evidence_quote":"Neural Cleanse, a compared defense whose per-class trigger reverse engineering is computationally expensive, motivating MergeGuard's speed advantage."},{"cited_title":"Backdoorbench: A comprehensive benchmark of backdoor learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the benchmark's predefined attack and training parameters used for consistent CIFAR-10 experiments."}],"review_version":1}