{"id":"4216daeb-1250-44b9-925f-beea3ec3dd46","arxiv_id":"2608.03681","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"KeepAD prunes ViT tokens at two depths with coverage and anomaly-aware selection, retaining under 20% of tokens while keeping industrial and medical zero-shot anomaly detection AUROC within about 2.7 points of dense inference.","lead":"KeepAD is a method that removes most image tokens from a vision transformer during anomaly detection while trying to keep the few tokens that show defects. It reports sub-20% token retention with under 2.7 AUROC point average loss and up to 7.9x throughput gain over CLIP baselines.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central claim is conditional on frozen L12 anomaly prototypes and hand-set gate constants remaining predictive on every target; the paper's own medical-transfer ablation shows these components weaken under domain shift, so sub-20% retention with ≤2.7 pp average loss is not yet established per…","rationale":"The reader's weakest-assumption analysis already identifies the load-bearing point: frozen synthetic-anomaly-trained prototypes and inference-only calibrations must transfer across the cross-dataset protocol. The present stress-test pass agrees and sharpens the concern by pointing to the joint dependence of the L12 budget and block selection on the same prototype affinities: a miscalibration corrupts both how many tokens survive and which blocks survive. The paper's own ablations are honest and unusually thorough—DTR/CMR are defined on physical survivors, owner-recovery is explicitly separated from token retention, seed stability is reported, and the medical-suite component transfer in Appendix F.2 does show that margins shrink under domain shift. That transparency is why the concern is not a fatal flaw: the central claim may survive, but only if the prototype-transfer assumption holds on each target. The reader's CONDITIONAL verdict already captures exactly this state, so no verdict adjustment is needed. The specific check proposed here—ablating the L12 anomaly-prototype term at matched budget across all medical targets—would settle whether the assumption is load-bearing or whether the visual salience branch alone would carry the headline numbers.","tokens_in":32623,"tokens_out":9566,"duration_ms":93952,"concrete_test":"Run the released prune70 pipeline on all seven medical targets with the L12 anomaly-prototype term removed (set alpha_i^12 = 0 in Eq. 3), with the evidence controller re-calibrated on the source split to match the original mean L12 keep, and compare per-dataset CMR and P-AUROC to the full method and to the dense no-pruning row. If on any target the full method fails to beat the prototype-free variant in CMR, or its P-AUROC loss from dense exceeds 2.7 points, the central claim is target-dependent and should be reported per dataset rather than as a 13-benchmark average.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline operating points are produced by a two-stage router whose deep stage is the actual risk gate: at L12, frozen normal/anomaly prototypes from a Stage-1 head trained on source-domain synthetic collage-and-crop anomalies supply both the block-selection score r_i^12 (Eq. 3) and the evidence statistic E(I) that sets the image budget K_tar^12 (Eq. 4). If these affinities are miscalibrated on a target, budget and selection fail together: the controller spends tokens on high-affinity normal regions while dropping the defect block, and CMR rises before P-AUROC reflects it, because nearest-survivor recovery copies responses from nearby survivors (Eq. 6, Appendix B). The paper reports suite-average medical transfer (Table 19) and its own Appendix F.2 shows component margins shrink under transfer: replacing local coverage with global top-k changes P-AUROC/CMR by -8.3/+6.8, and removing self-distillation by -1.4/+3.4; the two cross-dataset directions in Table 4 already show direction-specific failures for fixed budgets and single-cue scorers. The abstract's 'within 2.7 pp' average also conceals per-dataset drops of 6.8 I-AUROC on BrainMRI and 3.9 P-AUROC on Endo at prune70. The central claim therefore rests on the untested assumption that the frozen L12 prototype affinity remains predictive on every target, not just on the two transfer directions where the checkpoint and scalar calibrations (rho_min/rho_max, kappa, c, Tb, q(I) gate) were fixed. That assumption is plausible but not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes KeepAD, a two-stage token pruning framework for zero-shot anomaly detection built on a frozen CLIP ViT-L/14@336px detector. Stage 1 trains a dense anomaly detector with per-layer normal/anomaly prototypes; Stage 2 freezes it and trains lightweight selectors at layers 8 and 12. At L8, selection is coverage-preserving over non-overlapping 2x2 blocks with a deterministic rescue set; at L12, block-level selection uses a fusion of visual salience and frozen-prototype affinity under an image-adaptive budget. Dense-to-sparse self-distillation supervises the shallow router during training. Localization is recovered by nearest-survivor assignment on the original patch grid, while image-level scores aggregate only physical survivors. Experiments across six industrial and seven medical benchmarks report final token retention below 20% at the prune70 operating point, average I-AUROC/P-AUROC degradation within 2.7 percentage points, and up to 7.9x throughput over Bayes-PFL, the strongest CLIP-based baseline.","tokens_in":32935,"tokens_out":10339,"duration_ms":99020,"significance":"If the empirical claims hold, KeepAD is a useful and well-posed contribution to efficient zero-shot anomaly detection. The paper identifies an asymmetric pruning risk and an early-pruning paradox, and translates them into a concrete two-stage router with physical-survivor tracking. The evaluation is unusually thorough for this area: matched-budget controls for the adaptive budget (Table 5), physical defection-preservation metrics DTR/CMR, seed stability (Table 18), backbone generality (Table 9), comparison with generic token reducers in the same frozen detector (Table 6), and a medical-suite transfer study (Table 19) are all present, and the code is released. The main value is in demonstrating that anomaly-aware, coverage-preserving routing can largely preserve detection and localization while substantially reducing compute. The principal caveat is that some load-bearing components, especially the L12 controller constants and the frozen prototypes, are hand-tuned and only partially stress-tested under domain transfer.","major_comments":[{"comment":"The two cross-dataset directions use different Stage-2 checkpoints (epoch 7 for MVTec-to-VisA, epoch 10 for VisA-to-MVTec), and Appendix A.6 states that the Stage-2 data configuration uses the source-domain test split. Please clarify the checkpoint selection criterion and the exact split used for selector training. If the checkpoint epoch was chosen using target-domain test performance, or if the source-domain test split (with its real anomaly labels) was used for training, the reported numbers do not constitute a clean cross-dataset zero-shot evaluation. Please specify the selection rule, and if necessary rerun with a fixed epoch and with selector training restricted to the source-domain train split.","section":"§4.1 and Appendix A.6"},{"comment":"The deep-stage router and its image-adaptive budget rely on constants rho_min, rho_max, c, T_b, kappa, on the frozen L12 prototypes from Stage 1, and on the independently calibrated sparse-mid gate q(I). These are listed in Table 7, but no sensitivity analysis is reported for them. The medical-transfer ablation in Table 19 shows that component conclusions do shift under domain transfer (replacing local coverage with global top-k changes P-AUROC/CMR by -8.3/+6.8; removing self-distillation by -1.4/+3.4). Please add a sensitivity study over the controller and gate constants on source validation and on the medical suite, and state explicitly whether the constants were fixed before any target-domain results were inspected.","section":"§3.2, Eq. (4), Appendix A.4 and F.2"},{"comment":"Appendix B correctly warns that post-recovery P-AUROC cannot verify defect-token retention and that the 2x2 block coverage floor does not guarantee defect survival. However, the main benchmark tables report only I-AUROC and P-AUROC; DTR and CMR are reported only for the two transfer directions and as a medical-suite average. Since the central localization claim depends on physical defect preservation before recovery, please report per-benchmark DTR and CMR for the main operating points, and make clear in the abstract and results that pixel-level AUROC is measured after nearest-survivor recovery rather than on physical survivors alone.","section":"Appendix B and Tables 1/2/3/19"},{"comment":"The abstract's 'within 2.7 percentage points' is an average degradation across benchmarks and could be read as a per-dataset guarantee. The actual per-dataset drops at prune70 include -6.8 I-AUROC on BrainMRI and -3.9 P-AUROC on Endo. Please state the per-dataset ranges or add a clear qualification that individual datasets can degrade substantially more than the average.","section":"Abstract and Table 2"}],"minor_comments":[{"comment":"The '7.9x speedup over the strongest CLIP-based baseline' is a system-level comparison: it comes from comparing KeepAD at prune70 with Bayes-PFL, while the same pruning gives only about 1.48x over the paper's own dense detector. This is a legitimate comparison, but the abstract should state the baseline-specific nature of the 7.9x number.","section":"§1 and Table 1"},{"comment":"Figure 2 reports 'AUC' values, but the caption does not define what is being measured. The appendix later clarifies that these are token-level salience AUCs of a transplanted L8 scorer; the caption should be updated accordingly.","section":"Figure 2 and Appendix C.2"},{"comment":"Several medical datasets report only I-AUROC or only P-AUROC (for example, OCT17 and CVC-ClinicDB). Adding a column note or an em-dash convention would help readers understand the metric coverage.","section":"Table 2"},{"comment":"The notation TopCell and TopScore is used without an explicit definition; please state that TopCell(min(16,R8), ...) selects the highest-scoring remaining token per coarse 8x8 cell and TopScore selects by global score, as described in the surrounding text.","section":"Appendix A.3, Eq. (10)"}],"recommendation":"major_revision","confidential_remarks":"The most important issue is the data and checkpoint selection protocol. If the source-domain test split is used for Stage-2 training, or if the two different checkpoint epochs were selected on target test performance, the paper's central cross-dataset claims would need re-evaluation under a stricter protocol. I would ask the authors to clarify these choices and, if needed, rerun the main experiments with a source-train-only training set and a fixed epoch before considering acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"KeepAD is a genuinely useful systems paper. What's new: it prunes tokens inside the ViT for zero-shot anomaly detection, not after dense encoding like VMAD, and it introduces a coverage floor plus deterministic rescue to address the asymmetric risk of deleting the only defect evidence. The physical-survivor metrics DTR/CMR are a good idea—post-recovery P-AUROC can hide complete misses. The experiments are unusually careful: matched-budget controls, block-geometry and rescue decompositions, seed stability, and an honest Appendix B that admits coverage does not guarantee defect retention. The adaptive budget beats shuffled and fixed budgets at equal compute, which is the right control. The main claims—sub-20% retention with average I/P-AUROC loss within 2.7 points—are supported on the benchmarks tested.\n\nSoft spots. The average hides per-dataset drops of 6.8 I-AUROC on BrainMRI and 3.9 P-AUROC on Endo at prune70. That is not fatal, but the abstract's phrasing invites over-reading. The operating points depend on hand-set controller constants (rho_min/max, c, Tb, kappa, gate temperature) and the released checkpoints use epoch 7 for one transfer direction and epoch 10 for the other. That is a small flag: if the authors had a single fixed recipe, that should be easy to state. Baseline accuracies are taken from cited papers rather than rerun; speeds are rerun, so the accuracy comparison is not fully apples-to-apples. The stress-test concern about frozen L12 prototypes failing under domain shift is plausible but not demonstrated—the medical suite transfer shows component margins shrink, but the full method's average loss is still small (-0.2/-1.8). The mechanism is not proven to generalize to every target, but the paper does not overclaim it; it reports benchmark results, not a guarantee.\n\nWho this is for: anyone working on efficient anomaly detection or token pruning for dense prediction. The DTR/CMR metrics alone are worth citing. It deserves a serious referee; the empirical work is reproducible (code released) and the ablations set a high bar. I would send it to review and ask for a single fixed checkpoint rule, a sensitivity sweep on the controller constants, and per-dataset breakdowns in the abstract or at least a more cautious summary. Verdict: accept with revisions if the numbers hold under unified baselines.","headline":"Solid empirical paper on defect-preserving token pruning for zero-shot AD; averages hide per-dataset drops and some hand-tuning, but the ablations and physical-survivor metrics make it worth serious review.","tokens_in":33550,"tokens_out":3421,"would_cite":true,"duration_ms":28914,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"KeepAD prunes the vision transformer down to fewer than 20% of tokens while holding zero-shot anomaly detection accuracy within 2.7 AUROC points across industrial and medical benchmarks.","keywords":["zero-shot anomaly detection","token pruning","vision transformer","defect localization","efficient inference","self-distillation","image-adaptive budget","CLIP"],"falsifier":"A concrete check: on a new benchmark of small defects, stratify the complete-miss rate by ground-truth defect size. If defects spanning 1-4 tokens are missed in more than the roughly 2.7% of images the paper reports for its smallest-defect stratum at the final budget, the coverage floor is not preserving the smallest evidence.","tokens_in":32351,"feed_emoji":"🔍","tokens_out":9954,"duration_ms":81953,"temperature":0.7,"pith_summary":"The paper claims that token pruning inside a frozen vision transformer can be made safe for zero-shot anomaly detection by treating token selection as high-recall, defect-preserving routing rather than a generic efficiency trick. KeepAD keeps fewer than 20% of image tokens at inference and still holds average image-level and pixel-level AUROC degradation within 2.7 percentage points across six industrial and seven medical benchmarks, while reaching up to 7.9x throughput over the strongest CLIP-based baseline. This matters because anomaly evidence is scarce: most computation is spent on normal regions, but pruning can accidentally delete the only tokens that reveal a defect. The resolution is a two-stage router that is conservative early, aggressive late, and supervised by a dense-to-sparse self-distillation signal that costs nothing at inference.","feed_headline":"Prune 80% of tokens, keep anomaly detection within 2.7 points","feed_subtitle":"A two-stage router spares rare defect tokens from pruning, cutting compute up to 7.9x in zero-shot anomaly detection.","key_machinery":"The central object is the two-stage progressive router inserted at layers 8 and 12 of the frozen backbone. At L8 the selector computes a prototype-free visual score, then applies coverage-preserving selection over non-overlapping $2\\times2$ patch blocks (one representative per non-empty block) followed by a deterministic rescue of up to 16 diversity candidates and remaining high-score tokens; at L12 it combines the visual score with the sigmoid of the difference between cosine similarities to frozen abnormal and normal prototypes, standardizes the fused risks per image, and chooses whole $2\\times2$ blocks under a budget controlled by the dispersion and upper-tail of those risks. The selectors are trained with a hard-topology, soft-gradient construction plus dense-to-sparse self-distillation, in which an unpruned teacher pass supplies detached deeper-layer anomaly responses that align the early routing; at inference only a single hard-pruned forward pass remains, and original-grid coordinates of survivors support nearest-survivor dense recovery.","core_discovery":"The central claim is that token pruning for zero-shot anomaly detection should be formulated as a risk-aware routing problem, not a classification-preserving compression problem. KeepAD does this with two stages inside a frozen CLIP ViT-L/14 backbone: at layer 8, before anomaly semantics are reliable, it uses coverage-preserving selection over local $2\\times2$ patch blocks plus a deterministic rescue of high-score and spatially diverse tokens; at layer 12, it fuses visual salience with the difference between affinities to frozen normal and abnormal prototypes, then selects whole blocks under an image-adaptive budget that spends more compute on images with sharp or heavy-tailed anomaly evidence. Dense localization is recovered by mapping each dropped position to its nearest surviving token on the original grid, while image-level scores use only physical survivors so recovered positions cannot fake anomaly evidence. On six industrial and seven medical zero-shot anomaly detection benchmarks, the paper reports retention below 20%, average I-AUROC/P-AUROC degradation within 2.7 points, and up to 7.9x throughput over the strongest CLIP-based baseline.","pith_inferences":["An implication the paper leaves implicit is that the coverage-plus-rescue design could transfer to other sparse-evidence tasks, such as medical lesion detection or video anomaly detection, where missing a small region is far costlier than wasting compute.","The paper's physical-survivor metrics DTR and CMR are a useful template for the field: post-recovery pixel AUROC can stay high even when every defect token was pruned, so any pruning method claiming to preserve dense predictions should report survivor-based recall, not just recovered maps.","A natural extension the authors do not pursue is to make the L8 keep fraction image-adaptive as well, since it is currently a user-chosen operating point (30-70%) rather than a learned decision.","The depth analysis suggests token separability is non-monotonic, so the L8/L12 breakpoints are a trade-off; one could test whether the same two-stage idea should place pruning layers per backbone rather than fixing them globally."],"forward_implications":["At the most aggressive operating point, the model retains 14-18% of tokens and reaches 41.1 img/s, a 7.9x throughput gain over the strongest CLIP-based baseline with only 1.9-2.2 average AUROC loss on industrial transfer.","Ablations show that L8 pruning is the main efficiency lever: removing it costs 6.5-8.3 img/s, more than removing L12 pruning (2.7-3.2 img/s), while the L21 early exit adds 0.9-3.3 img/s with no AUROC change.","Replacing L8 coverage with global top-k increases complete misses from below 1% to 4.0-6.3% and costs 8.9-10.7 P-AUROC points, so the coverage floor is what makes aggressive early pruning safe.","The image-adaptive budget works per image: shuffling the same budget multiset across images raises the complete-miss rate by at least 4 points, so the gain comes from matching compute to evidence rather than from aggregate compute.","Dense-to-sparse self-distillation is the key training signal for the shallow router: without it, complete-miss rate jumps from below 1% to 4.3-6.9% and I-AUROC drops by 2.7-4.9 points."],"supporting_citations":[{"why":"Defines the cross-dataset zero-shot anomaly detection protocol and the CLIP-based accuracy baseline KeepAD is measured against.","marker":"Zhou et al. 2024"},{"why":"Supplies the strongest CLIP-based baseline; the 7.9x speedup claim compares throughput with this method at comparable accuracy.","marker":"Qu et al. 2025"},{"why":"Supplies the strongest ViT-based baseline; KeepAD reports a 1.7x throughput gain over it.","marker":"Hou et al. 2026"},{"why":"Provides the primary industrial benchmark (MVTec-AD) used for source-to-target transfer evaluation.","marker":"Bergmann et al. 2019"},{"why":"Provides the VisA dataset, whose trained checkpoint is transferred to all other industrial and medical targets.","marker":"Zou et al. 2022"},{"why":"Token merging baseline (ToMe) used in the same-frozen-detector comparison showing generic reduction trails anomaly-aware routing.","marker":"Bolya et al. 2023"},{"why":"Adaptive token sampling baseline (ATS) used in the same-frozen-detector comparison.","marker":"Fayyaz et al. 2022"},{"why":"Token reorganization baseline (EViT) used in the same-frozen-detector comparison.","marker":"Liang et al. 2022"},{"why":"Training-free attention-graph pruning baseline (Zero-TPrune) used in the same-frozen-detector comparison.","marker":"Wang, Dedhia, and Jha 2024"}],"fun_headline_variants":["Prune 80% of tokens, keep anomaly detection within 2.7 points","Defect-preserving token pruning: 7.9x faster, ≤2.7-point AUROC drop","Keep the needle, prune the haystack: 20% tokens, 2.7-point cost","80% token cut, 2.7-point AUROC loss, 7.9x speedup"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The result depends on the frozen normal and abnormal prototypes, trained on synthetic collage-and-crop defects from source-domain data only, transferring to unseen industrial and medical targets well enough that the inference-time calibrations keep working without target-domain tuning.","fun_headline_variants_meta":{"raw":{"variants":["Prune 80% of tokens, keep anomaly detection within 2.7 points","Defect-preserving token pruning: 7.9x faster, ≤2.7-point AUROC drop","Keep the needle, prune the haystack: 20% tokens, 2.7-point cost","80% token cut, 2.7-point AUROC loss, 7.9x speedup"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001733,"raw_usage":{"total_tokens":6912,"prompt_tokens":1068,"completion_tokens":5844,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":684,"completion_tokens_details":{"reasoning_tokens":5741}},"tokens_in":684,"tokens_out":5844,"duration_ms":38777,"temperature":1.0,"reasoning_tokens":5741,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:16:41.961176+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete check: on a new benchmark of small defects, stratify the complete-miss rate by ground-truth defect size. If defects spanning 1-4 tokens are missed in more than the roughly 2.7% of images the paper reports for its smallest-defect stratum at the final budget, the coverage floor is not preserving the smallest evidence.","supporting_citations":[],"review_version":3}