{"id":"7fecbb2a-4c56-493c-8218-ae23551271ab","arxiv_id":"2608.00264","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A differentiable soft-pruning framework (SAPER) and a Laplacian-based head visualization reduce ViT attention-head counts up to 24-fold with 4-11 point accuracy drops on ImageNet-1K.","lead":"The authors propose SAPER, a method that prunes attention heads in vision transformers using a differentiable soft top-k selection, plus a spectral visualization (Laplace maps) of attention-head behavior. Tests on DINOv2 show large FLOPs reductions at modest accuracy cost compared with the RAPTOR baseline.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 2's RAPTOR FLOPs exceed the unpruned model (61.86 vs 46.33 GFLOPs), so the claimed 'outperforms RAPTOR in FLOPs reduction' is unsupported until the measurement protocol is explained.","rationale":"The reader's weakest_assumption identifies exactly the load-bearing concern: Table 2's RAPTOR FLOPs are not plausibly comparable to SAPER's. I agree. The central claim, as flagged in the reader's strongest_claim, is the accuracy-efficiency advantage over RAPTOR. That advantage is quantified almost entirely by the FLOPs columns in Table 2. The RAPTOR numbers are internally inconsistent with the unpruned DINOv2 baselines: a compressed/recurrent model should not use more compute than the full model, and the flat values across k=24,36,48 (61.86 GFLOPs for ViT-B) suggest a measurement artifact or a different convention. This is a correctness risk, not merely a matter of framing. The proposed concrete test — recomputing RAPTOR FLOPs with the same tool and input resolution — would settle whether the comparison is valid. If the numbers turn out to be wrong, the paper's headline claim over RAPTOR collapses; if they are correct under a clearly stated protocol (e.g., full recurrent unrolling to preserve depth), the authors need to say so explicitly, because as written the table is misleading. A secondary weakness, also noted by the reader, is that the 'interpretability-guided' label is loose: Section 3.3's LapSum selector uses only learned scores, with no input from the spectral clustering or Laplace maps of Sections 3.1–3.2. That does not directly undermine the efficiency claim, but it weakens the paper's framing as 'interpretability-guided pruning.' Since the reader's verdict is CONDITIONAL and my concern is the same one, I recommend keeping the verdict unchanged: conditional acceptance pending clarification and verification of the FLOPs numbers.","tokens_in":30140,"tokens_out":11210,"duration_ms":105681,"concrete_test":"Using the official RAPTOR code (or, if unavailable, the architecture described in RAPTOR), instantiate the ViT-B model with B=2 blocks (24 heads) and count single-image inference FLOPs at 224×224 with the same FLOPs-counting tool used for SAPER (e.g., fvcore or thop). Also count the unpruned DINOv2 ViT-B with the same tool. If RAPTOR's count is below 46.33 GFLOPs, or differs from 61.86 by more than a few percent, Table 2's numbers are not comparable to SAPER's, and the headline 'outperforms RAPTOR in FLOPs reduction' must be re-evaluated. Repeat the same check for ViT-S k=12 (16.15 vs. 12.25 GFLOPs).","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central efficiency claim, stated in the abstract and Section 4.2, is that SAPER 'outperforms the competitive RAPTOR baseline in FLOPs reduction.' This rests entirely on Table 2's FLOPs columns. Those numbers are not credible as inference costs of pruned models. For ViT-S, RAPTOR is listed at 16.15 GFLOPs for all reported head counts (12, 24, 36, 48), while the unpruned DINOv2 ViT-S is 12.25 GFLOPs. For ViT-B, RAPTOR is listed at 61.86 GFLOPs for k=24, 36, and 48, while the unpruned ViT-B is 46.33 GFLOPs. A pruned/recurrent model that removes blocks should not cost strictly more FLOPs than the full model; moreover, the identical values across different k strongly suggest the FLOPs are not recomputed for the actual pruned architecture but taken from a different measurement convention (e.g., full recurrent unrolling to preserve depth, training FLOPs, or a different input resolution). The paper explicitly draws the headline comparison: 'SAPER reduces GFLOPs by roughly a factor of two at matching head counts (e.g., 32.17 vs. 61.86 GFLOPs for ViT-B at k=24)'. If 61.86 is not a comparable single-inference FLOPs count, this sentence and the abstract's claim are unsupported. The issue is not a minor methodological detail; it is the quantitative basis for the claimed advantage over RAPTOR and is internally inconsistent with the unpruned baseline.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SAPER, a differentiable attention-head pruning framework for DINOv2 vision transformers, together with two interpretability analyses: semantic clustering of attention heads based on Pearson correlation of per-image maximal CLS-attention values, and 'Laplace maps' that visualize attention heads through eigenvectors of the normalized Laplacian of the attention graph. The authors claim that SAPER achieves a favorable accuracy-efficiency trade-off, outperforming the RAPTOR baseline in FLOPs reduction while retaining competitive classification accuracy on ImageNet-1K and CIFAR-100. The paper reports experiments for ViT-S and ViT-B, comparing fine-tuning and knowledge-distillation modes against RAPTOR across head budgets k=6,12,24,36,48. The central quantitative claim is that SAPER reduces FLOPs by roughly a factor of two relative to RAPTOR at matching head counts (e.g., 32.17 vs. 61.86 GFLOPs for ViT-B at k=24).","tokens_in":30566,"tokens_out":6682,"duration_ms":61374,"significance":"If the efficiency numbers are correct, the paper offers a flexible alternative to block-level recurrent pruning, with detailed per-head budgets and an interpretability-oriented analysis of attention-head structure. The manuscript is transparent in reporting averages over multiple runs and extensive hyperparameter search details in the appendix. The spectral visualization of attention heads is a plausible novel contribution. However, the headline comparison with RAPTOR rests entirely on FLOPs values in Table 2 that are internally inconsistent: RAPTOR's reported pruned FLOPs exceed the unpruned DINOv2 baseline. This is a load-bearing issue for the abstract's central claim. In addition, the claimed connection between the interpretability analyses and the actual pruning mechanism is not demonstrated. These concerns make the current version unsuitable without substantial revision.","major_comments":[{"comment":"The central claim that SAPER 'outperforms RAPTOR in FLOPs reduction' is unsupported because the RAPTOR FLOPs numbers in Table 2 are not credible under any consistent inference protocol. For ViT-B, the unpruned DINOv2 baseline is listed as 46.33 GFLOPs, but RAPTOR at k=24,36,48 is listed as 61.86 GFLOPs; for ViT-S, RAPTOR is listed as 16.15 GFLOPs at all k, while the unpruned model is 12.25 GFLOPs. A pruned model should not use more compute than the full model, and identical values across different k strongly suggest the numbers were not recomputed for the pruned architectures. The paper's own comparison — '32.17 vs. 61.86 GFLOPs for ViT-B at k=24' — and the abstract's efficiency claim depend on this number. The authors must either provide a reproducible FLOPs measurement protocol under identical conditions (input size, model definition, unrolling, inclusion of overhead) or withdraw the a","section":"§4.2, Table 2"},{"comment":"The normalized Laplacian of an attention map is asserted to be a symmetric positive semi-definite matrix, but the attention map is a directed weighted graph (row-stochastic after softmax, or a logit matrix before softmax). For a directed graph, the standard normalized Laplacian is not symmetric PSD, so eigenvectors and the Fiedler vector are not generally well-defined real vectors. The paper does not describe any symmetrization step (e.g., L=(A+A^T)/2 or degree-normalized symmetrization). This needs to be fixed or the Laplace-map construction needs to be redefined, especially since the authors call the Fiedler vector the 'real-valued solution to the minimum cut problem,' which itself is only a relaxation.","section":"§3.2"},{"comment":"The paper's title and Section 2.4 describe Laplace maps as 'a signature that also serves as a pruning criterion,' and the introduction states that interpretability insights are leveraged by SAPER. However, the differentiable selection mechanism in Section 3.3 uses LapSum on a score vector r (Eq. 2), and the paper never defines r in terms of the spectral or clustering features. The scores appear to be learnable parameters optimized end-to-end. As written, the interpretability analyses of Sections 3.1–3.2 are not functionally connected to the pruning algorithm; they are exploratory studies. The authors should either specify how the spectral/semantic features enter the selection score, or temper the 'interpretability-guided' framing and the claim that Laplace maps serve as a pruning criterion.","section":"§3.3, Eq. (2)"},{"comment":"The comparison with RAPTOR is also complicated by evaluation-protocol differences. For ImageNet-1K, RAPTOR accuracy is quoted from the original paper, while SAPER uses either a linear probe (fine-tune mode) or a kNN classifier (distillation mode). The authors do not state whether these protocols match RAPTOR's evaluation. At k=24 on ViT-B ImageNet-1K, RAPTOR reports 81.2 and SAPER distilled reports 80.58, so the headline is not that SAPER is more accurate, but that it uses fewer FLOPs. Given that the FLOPs values are questionable, the overall accuracy-efficiency advantage over RAPTOR is currently not demonstrated. A standardized comparison, ideally re-running RAPTOR under the same measurement protocol, is needed.","section":"§4.2, Table 1"}],"minor_comments":[{"comment":"The captions use 'DB=' and 'Sil=' without defining them. Please spell out Davies-Bouldin index and Silhouette score in the caption or text.","section":"Figure 2"},{"comment":"The text refers to 'layers L0-L7 and L8-L12'; a 12-layer ViT has layers L0 through L11, so the second interval should be L8-L11. The same issue appears in the appendix discussions of 'final two layers' and 'final three layers.'","section":"§4.2"},{"comment":"Several entries contain the typographical artifact '✗−✗' (e.g., RAPTOR24 ViT-S ImageNet-1K). This should be cleaned to a single '✗' or an explicit N/A.","section":"Table 1"},{"comment":"The statement that the Fiedler vector 'represents the real-valued solution to the minimum cut problem' is imprecise. It is the solution to a spectral relaxation of the normalized cut, not the exact solution. Please rephrase.","section":"§3.2"},{"comment":"The hyperparameter tables are extensive, but the paper does not provide code or a repository link. Given the volume of tuning details, a public implementation would substantially improve reproducibility.","section":"Appendix E"}],"recommendation":"major_revision","confidential_remarks":"The FLOPs comparability issue in Table 2 is the most serious concern. If the RAPTOR FLOPs numbers cannot be reproduced under a consistent protocol, the paper's primary quantitative contribution collapses to 'SAPER reduces FLOPs relative to the unpruned model,' which is less novel. The second major concern is that the interpretability analysis does not actually drive the pruning selection; this affects the paper's framing more than its empirical results. I would encourage the editor to ask for a revised version that either recomputes RAPTOR FLOPs and accuracy under controlled settings or explicitly restricts the claims to comparisons where the measurement protocol is documented."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuine contributions are the Laplace maps--an RGB visualization from the Fiedler and next eigenvectors of a head's attention graph--and the head-level semantic clustering showing functionally similar heads at different depths, a nice refinement of RAPTOR's layer-level blocks. The pruning part is a straightforward application of LapSum soft top-k to head selection; that is fine, but it means the 'interpretability-guided' label is doing more work than the method does. The spectral scores do not feed the selector; the selector learns importance scores from the downstream loss. The maps and clustering are motivation, not the pruning signal. That disconnect should be stated plainly.\n\nThe bigger problem is Table 2. The central quantitative claim is that SAPER outperforms RAPTOR in FLOPs reduction. But the RAPTOR FLOPs numbers are not credible as written. For ViT-S, RAPTOR is listed at 16.15 GFLOPs for k=12, 24, 36, and 48, identical, and above the unpruned 12.25. For ViT-B, 61.86 for k=24, 36, and 48, above the unpruned 46.33. A pruned model should not cost more than the full model. Identical values across k also suggest these numbers were not computed on the actual pruned architectures. Until the authors explain the measurement protocol, the abstract's 'outperforms RAPTOR in FLOPs reduction' is unsupported.\n\nSmaller issues: the normalized Laplacian is asserted to be symmetric positive semi-definite, but attention maps are directed; the paper does not describe the symmetrization. The semantic clustering uses Pearson correlation of max-attention patches over 50k validation images, which is fine, but the cluster quality metrics (silhouette ~0.08-0.13) are weak. And there is no code or data release mentioned, which for an empirical paper would help.\n\nWhat is solid: the SAPER accuracy numbers are measured, not fitted; the experiments are extensive, with hyperparameter search and multiple seeds; and the paper is honest about where RAPTOR cannot be compared. The method probably works as a flexible head-pruning tool. But the headline comparison needs fixing, or the claim needs to be softened.\n\nThis paper deserves a serious referee. The issues are addressable, and the empirical study is substantial. I would send it out, with a request that the authors clarify FLOPs measurement and either connect the spectral scores to the selector or drop the 'interpretability-guided' framing. For my own work, I would not cite it yet.","headline":"A useful pruning recipe with a nice visualization, but the headline efficiency claim rests on FLOPs numbers for RAPTOR that don't add up.","tokens_in":828,"tokens_out":2681,"would_cite":false,"duration_ms":46790,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SAPER prunes attention heads one by one, beating block pruning on FLOPs","keywords":["attention head pruning","vision transformers","DINOv2","spectral clustering","Laplace maps","LapSum soft top-k","interpretability","FLOPs reduction"],"falsifier":"Compute FLOPs for a DINOv2 ViT-B pruned to 24 heads using a standard FLOPs counter at the paper's input resolution, for both SAPER's selected heads and RAPTOR's recurrent block configuration. If RAPTOR's true FLOPs is below SAPER's (or even below the unpruned model's 46.33 GFLOPs), the paper's efficiency-advantage claim over RAPTOR is false. Alternatively, reproduce the 61.86 GFLOPs figure for RAPTOR's 24-head configuration; if it cannot be reproduced, the baseline comparison is not legitimate.","tokens_in":1286,"feed_emoji":"✂️","tokens_out":1980,"duration_ms":58057,"temperature":0.7,"pith_summary":"The paper claims that the attention heads of a DINOv2 vision transformer are not uniformly redundant but fall into functional clusters that repeat across layers, so heads can be removed one at a time rather than in contiguous blocks. To exploit this, it introduces SAPER, a differentiable pruning method that learns a soft top-k mask over all heads and then fine-tunes or distills the surviving subset. On ImageNet-1K and CIFAR-100, SAPER keeps classification accuracy within a few points of the unpruned model while cutting FLOPs well below the unpruned model and below the block-recurrent RAPTOR baseline at matched head counts. A sympathetic reader would care because it suggests a practical path to smaller vision foundation models guided by what each head actually computes.","feed_headline":"One-by-one head pruning cuts ViT FLOPs below block pruning","feed_subtitle":"A spectral signature reveals which attention heads are redundant, letting you keep just the ones that matter.","key_machinery":"The key machinery is threefold: Laplace maps, which render the Fiedler vector and next two eigenvectors of each head's normalized attention-graph Laplacian as an RGB image; spectral clustering of head activation statistics (Pearson correlation on maximum-attention features) to expose cross-depth functional redundancy; and the LapSum soft top-k selector, a differentiable relaxation of hard top-k that lets a continuous budget k be annealed to pick exactly k heads. The first two justify which heads are redundant; the third translates that into a trainable pruning decision.","core_discovery":"The central discovery is that the normalized-Laplacian spectrum of an attention map gives a label-free signature of a head's function, and that these signatures cluster into a small number of groups that recur at different depths of the network. Building on this, SAPER uses the LapSum soft top-k operator to select any prespecified number of heads in an end-to-end differentiable way. With as few as 6 of 144 heads in DINOv2 ViT-B, distilled SAPER retains 72.97% ImageNet-1K top-1 accuracy (vs. 84.14% unpruned) at 30.05 GFLOPs; at 24 heads it reaches 80.58% at 32.17 GFLOPs, roughly half the FLOPs of the RAPTOR baseline at the same head count.","pith_inferences":["If the spectral signature transfers across architectures and datasets, the same clustering could pre-initialize pruning for new ViT variants without extra labeled data.","The clustering may serve as a testable proxy for functional redundancy: ablating heads from the same cluster one at a time should cause similar accuracy drops, which could be verified empirically.","Token-level pruning, which the paper mentions as future work, could be combined with head pruning to yield further FLOPs savings if the spectral lens extends to token importance.","The FLOPs advantage over RAPTOR at matched head counts depends on counting conventions; if RAPTOR's figures are measured with unrolled recurrent depth or a different input resolution, the gap may be overstated or reversed."],"forward_implications":["A user can choose any desired head count between 1 and the full model, enabling a smooth accuracy-compute trade-off rather than block-multiple jumps.","Pruning to 24 heads cuts GFLOPs from 46.33 to 32.17 in ViT-B while losing under 4 points of ImageNet-1K accuracy in distilled mode.","Interpretability guides the selection: surviving heads are spread across early-layer 'convolutional' roles and late-layer object-level roles, so the pruned model keeps a diversity of functions.","Attention heads can stay frozen during fine-tuning or distillation, which reduces training cost and worked better than adjusting them.","The head-level clustering confirms and refines the layer-level block structure of ViTs, showing that functional recurrence happens within individual heads across depth."],"fun_headline_variants":["Spectral shortcuts: prune ViT heads by their attention signatures","Soft top-k prunes ViT heads, halves FLOPs vs baseline","Find redundant ViT heads via Laplacian spectra, then prune","SAPER: differentiable head pruning with spectral clustering"],"cache_read_input_tokens":32256,"weakest_assumption_plain":"The main quantitative claim that SAPER outperforms RAPTOR in FLOPs reduction assumes the FLOPs numbers for RAPTOR in Table 2 were computed under the same counting conventions and input settings as SAPER's; in particular, RAPTOR's pruned ViT-B at 24 heads is listed at 61.86 GFLOPs, which is higher than the unpruned model's 46.33 GFLOPs, so the comparison is only meaningful if the two sets of numbers are commensurable.","fun_headline_variants_meta":{"raw":{"variants":["Spectral shortcuts: prune ViT heads by their attention signatures","Soft top-k prunes ViT heads, halves FLOPs vs baseline","Find redundant ViT heads via Laplacian spectra, then prune","SAPER: differentiable head pruning with spectral clustering"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000546,"raw_usage":{"total_tokens":2426,"prompt_tokens":700,"completion_tokens":1726,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":444,"completion_tokens_details":{"reasoning_tokens":1653}},"tokens_in":444,"tokens_out":1726,"duration_ms":11794,"temperature":1.0,"reasoning_tokens":1653,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T00:51:49.356279+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute FLOPs for a DINOv2 ViT-B pruned to 24 heads using a standard FLOPs counter at the paper's input resolution, for both SAPER's selected heads and RAPTOR's recurrent block configuration. If RAPTOR's true FLOPs is below SAPER's (or even below the unpruned model's 46.33 GFLOPs), the paper's efficiency-advantage claim over RAPTOR is false. Alternatively, reproduce the 61.86 GFLOPs figure for RAPTOR's 24-head configuration; if it cannot be reproduced, the baseline comparison is not legitimate.","supporting_citations":[],"review_version":1}