Pith. sign in

REVIEW 4 major objections 5 minor 29 references

When Does Pruning Benefit Vision Representations?

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read Sparsity can lift interpretability, object discovery, and human alignment at certain pruning levels, but the sweet spots shift by architecture.

desk verdict A useful but fragile empirical map of pruning effects across CNNs and ViTs; the headline sweet-spot results are not yet supported by the reported single-run evidence. read the letter →

arxiv 2507.01722 v3 pith:RRB3LDHC submitted 2025-07-02 cs.CV

classification cs.CV
keywords pruninginterpretabilityobjectdiscoveryhumanalignmentvisiontransformersResNetattributionmethodssweetspots
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Neural network pruning is usually justified by efficiency, but this paper asks whether removing weights can also improve what a vision model shows and how it behaves. Using iterative magnitude pruning with learning rate rewinding on ResNet and Vision Transformer backbones, the authors trace three properties as sparsity grows: the quality of attribution maps (interpretability), unsupervised object localization, and accuracy on human-vision-style distortion benchmarks. They find 'sweet spots' — sparsity levels at which pruned models beat their dense counterparts on these properties — for CNNs and one transformer, while other transformer configurations degrade monotonically. The paper's overall claim is that pruning's effect on representations is not uniformly harmful or helpful; it depends on architecture, model size, and the property being measured.

What carries the argument

The central machinery is sparsity as a controlled intervention: models are iteratively pruned globally by lowest weight magnitude and retrained with learning rate rewinding (the LRR variant of iterative magnitude pruning). The effects are measured with three instruments: RMA and RRA, which score how much explanation mass or rank falls inside ground-truth masks; LOST's CorLoc for unsupervised object discovery; and the Geirhos et al. distortion-benchmark accuracy as a proxy for human alignment. The 'sweet spot' concept is the load-bearing idea: a narrow sparsity interval where the pruned model simultaneously beats the dense baseline on accuracy and on at least one representation-quality metric.

What would settle it

Collect human psychophysics judgments on the same 17 distortion datasets used in the paper: if human response patterns do not correlate with pruned-model accuracy trajectories across sparsity, the claimed human-alignment sweet spots would not be about alignment. A simpler check: rerun the identical pipeline with a different pruning schedule (e.g., one-shot magnitude pruning instead of iterative LRR) and see whether the reported sweet spots — ResNet18 at 15%, ResNet50 at 25.8%, ViT-B-32 at 40/60/90% — persist.

Watch

Extended reading notes

Core claim

The central claim is that pruning can benefit vision representations beyond efficiency: at certain sparsity levels, sparse models exhibit higher interpretability, better unsupervised object discovery, and stronger alignment with human perception than the dense baseline. These benefits are not universal: ResNet18 shows a sweet spot around 15% sparsity for interpretability and ResNet50 around 25.8%; ResNet18's object discovery peaks at 85% sparsity; ViT-B-32 shows object-discovery sweet spots at 40%, 60%, and 90%, then collapses beyond 95%; and smaller models (ResNet18, ViT-B-32) degrade less than their larger counterparts on distortion benchmarks. The paper also reports an architectural split: for CNNs, interpretability metrics and accuracy are negatively correlated as sparsity grows, while for ViTs they tend to decline together.

Load-bearing premise

The human-alignment conclusions assume that accuracy on the Geirhos distortion datasets is a valid measure of how closely a model matches human perception; the paper never compares model outputs to actual human judgments.

Editorial extensions

If this is right

  • If sweet spots replicate, practitioners can prune a network once, at a prescribed sparsity, and get a model that is smaller, no less accurate, and more explainable than the dense original.
  • The architectural split implies that pruning decisions for CNNs and ViTs should be made differently: for ResNets, sparsity can clean up attribution maps even as accuracy slips; for ViTs, interpretability tends to fall with accuracy.
  • The object-discovery sweet spots suggest sparse representations discard redundant features that confuse unsupervised localization, making pruning a free boost for LOST-style methods.
  • Smaller models' stability under pruning supports the view that compression and human alignment can go together, motivating pruning as a regularizer for shape bias.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's human-alignment proxy (distortion accuracy) leaves open whether sparse models actually match human judgments; a direct psychophysics comparison would test whether the sweet spots are perceptual or merely distributional.
  • The sweet spots' dependence on architecture and size suggests that pruning recipes should be tuned per model rather than transferred; a testable extension is to search for sweet spots with a sparsity sweep on a new architecture and see if they predict out-of-distribution robustness.
  • The contrast between LRR iterative pruning and earlier 'sweet spot' work on VGG suggests the phenomenon may be technique-dependent; re-running with fine-tuning or lottery-ticket rewinding would delimit when pruning helps.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper investigates how iterative magnitude pruning with learning-rate rewinding affects three properties of ImageNet-trained vision models (ResNet18, ResNet50, ViT-B-32, SwinV2): interpretability measured by RMA/RRA on attribution maps, unsupervised object discovery measured by a CorLoc-style score on VOC2012, and human alignment measured by accuracy on the Geirhos et al. distortion benchmark. The central claim is that architecture-dependent 'sweet spots' exist at which pruned models outperform dense baselines on interpretability, object discovery, and distortion robustness, and that smaller models preserve human alignment better than larger ones.

Significance. If the central claims are supported, the paper would provide a useful empirical map of when pruning helps rather than hurts vision representations, and the released code would facilitate replication. However, the current evidence is fragile: all results are single-run point estimates, sweet spots are identified post hoc without variance estimates or statistical tests, and some reported sparsity coordinates are inconsistent with the stated pruning schedule. The human-alignment conclusions also rest on a proxy that is not validated against human judgments.

major comments (4)
  1. [§4.1–4.2, Algorithm 1] The sparsity values at which sweet spots are reported are not consistent with the pruning schedule described in the paper. Algorithm 1 and Section 4.1 set k=20% global magnitude pruning; under the standard 'prune 20% of the remaining weights' reading the reachable sparsities are 0, 20, 36, 48.8, 59.0, 67.2, ... percent, and under '20% of the original weights' they are 0, 20, 40, 60, 80, ... percent. Neither reading produces the reported ResNet18 sweet spot at 15% sparsity or the ResNet50 sweet spot at 25.8% sparsity (Section 4.2, Figure 3), and the ViT-B-32 object-discovery sweet spots at 40%, 60%, and 90% (Figure 5) are ambiguous without a precise schedule. The authors should state the exact pruning schedule, correct the reported coordinates, and confirm that every claimed peak corresponds to a checkpoint that was actually evaluated.
  2. [§4.2, Figures 3–6] All reported results are single-curve point estimates with no error bars and no repeated runs. Sweet spots are identified post hoc across many sparsity levels, metrics, and datasets; without variance estimates, a peak selected in this way can be a sampling artifact. The authors should report means and standard deviations over at least three seeds for the main figures and, ideally, a statistical comparison showing that the peak value exceeds the dense baseline.
  3. [§3.3, Human alignment (RQ3)] The RQ3 conclusion that smaller pruned models 'better preserve human-alignment' is based on accuracy on the Geirhos et al. distortion datasets, but the paper never compares model outputs to actual human judgments. Equating distortion robustness with human perceptual alignment is a proxy that should be justified explicitly, or the conclusions should be rephrased in terms of robustness to distortions rather than human alignment.
  4. [§3.3, Eq. (3)] Equation (3) defines the object-discovery score as |LOST∩GT|/|LOST∪GT|, which is a per-image pixel intersection-over-union, but the text states that this is the CorLoc objective introduced by LOST. In the LOST benchmark and in standard object-localization evaluation, CorLoc is the fraction of images whose predicted box has IoU above a threshold (typically 0.5) with the ground-truth box. The authors should clarify the exact evaluation protocol and, if pixel-level IoU was used, explain how it relates to the standard CorLoc metric.
minor comments (5)
  1. [§3.2, RRA definition] The informal definition of P_topK uses 'until the k-th pixel' while the metric is denoted RRA with K as the mask size; the capitalization should be made consistent (K vs. k).
  2. [Algorithm 1] The pseudocode's loop step says 'Return to step 2,' but after retraining with learning-rate rewinding the next operation should be pruning (step 3), not training to completion from scratch; the loop control should be corrected.
  3. [§4.1, pruning amount] The sentence 'we prune k = 20% of the model's weights' is ambiguous: it should state whether 20% refers to the current remaining weights or the original weight count, since this determines the reachable sparsity levels.
  4. [§4.2, Figure 2 caption] The phrase 'contrastive scenario' in the caption of Figure 2 is unclear; the intended contrast between the two columns should be described more explicitly.
  5. [Throughout] There are minor typographical errors, such as 'Intepretability' at the start of Section 4.2 and inconsistent formatting of author names in the related-work discussion (e.g., 'Von Rad et al.').

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical benchmark study with no fitted parameters, no derivation chain, and no load-bearing self-citations.

full rationale

The paper's claims are all direct measurements on externally defined benchmarks and publicly available methods. RQ1 interprets RMA/RRA computed against VOC2012 ground-truth masks using standard attribution methods (IG, GradCAM, G-GradCAM); RQ2 uses the LOST CorLoc score on VOC2012; RQ3 uses the Geirhos et al. distortion benchmark. None of these metrics is defined in terms of the paper's conclusions, and no parameter is fitted to the target results. The 'sweet spots' are stated as observed maxima in the plotted curves, not as outputs of an optimization or fitting procedure. The only self-citation (Renzulli and Grangetto [19]) appears in a background sentence about resource-constrained deployment and is not load-bearing. The human-alignment measure is a surrogate (accuracy on distorted-image sets rather than direct human judgments), which is a validity concern, not a circularity: the benchmark is independent, pre-existing, and not constructed from the paper's own outputs. The limitations section appropriately notes the restricted model/pruning/explanation scope. No circular step can be quoted because none exists.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new constructs; it relies on external benchmarks and hand-chosen pruning hyperparameters. The main assumptions concern the validity of the evaluation metrics as proxies for interpretability, object discovery, and human alignment.

free parameters (1)
  • Pruning step size k = 20% of weights per iteration
    Chosen by hand; defines the sparsity grid used to detect sweet spots, so the reported sweet-spot locations depend on this arbitrary choice.
assumptions (4)
  • domain assumption VOC2012 pixel masks are a valid ground truth for quantifying interpretability via RMA and RRA
    Section 3.2 uses RMA/RRA against VOC segmentation masks to score attribution quality; if masks do not reflect explainability, the RQ1 conclusions do not follow.
  • domain assumption Accuracy on Geirhos et al. distortion datasets is a valid proxy for human alignment
    Section 3.3, RQ3, equates maintained accuracy on these distortions with human alignment without using human perceptual judgments.
  • domain assumption LOST applied to pruned supervised models yields valid object discovery with the unspecified feature layer choice
    Section 3.3, Object discovery, CorLoc results depend on seed selection from model features, but the extracted layer is not specified.
  • domain assumption Magnitude-based LRR pruning is representative of pruning effects in general
    Section 3.1 and the limitations section state that only one pruning technique is used; generalization to other pruning methods is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of When Does Pruning Benefit Vision Representations?." pith.science (2026). https://pith.science/paper/RRB3LDHC

@misc{pith2026250701722,
  author       = {Pith},
  title        = {Pith review of: When Does Pruning Benefit Vision Representations?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RRB3LDHC}},
  note         = {Machine review of arXiv:2507.01722}
}
read the original abstract

Pruning is widely used to reduce the complexity of deep learning models, but its effects on interpretability and representation learning remain poorly understood. This paper investigates how pruning influences vision models across three key dimensions: (i) interpretability, (ii) unsupervised object discovery, and (iii) alignment with human perception. We first analyze different vision network architectures to examine how varying sparsity levels affect feature attribution interpretability methods. Additionally, we explore whether pruning promotes more succinct and structured representations, potentially improving unsupervised object discovery by discarding redundant information while preserving essential features. Finally, we assess whether pruning enhances the alignment between model representations and human perception, investigating whether sparser models focus on more discriminative features similarly to humans. Our findings also reveal the presence of sweet spots, where sparse models exhibit higher interpretability, downstream generalization and human alignment. However, these spots highly depend on the network architectures and their size in terms of trainable parameters. Our results suggest a complex interplay between these three dimensions, highlighting the importance of investigating when and how pruning benefits vision representations.

Figures

Figures reproduced from arXiv: 2507.01722 by the authors.

Figure 1
Figure 1. We first train and prune a model on a source task at various pruning levels. Then, we assess how pruning affects interpretability (RQ1) and performance on down￾stream tasks such as OD (RQ2) and HA (RQ3). G-GradCAM), qualitatively and quantitatively (RQ1). We further benchmark their generalization on downstream tasks such as OD (RQ2) and HA (RQ3). In the next sections, we detail each step of our pipeline, including t… view at source ↗
Figure 2
Figure 2. Qualitative evaluation of ResNet50 on two input examples throughout pruning with corresponding model sparsity S and predicted class. scenario where attribution quality can degrade even when classification accuracy is maintained. On the right, both model predictions and explanations degrade with increased pruning. At S = 0, the unpruned ResNet50 predicts a visually similar class (Sturgeon), highlighting semantically … view at source ↗
Figure 3
Figure 3. RMA and RRA performances for ResNet18, ResNet50 and Swin. tation masks as ground truth for measuring attribution quality. The evaluation metrics reveal varying and often declining performances across architectures. For CNNs (top-left and top-right plots of [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: ViT-B-32 RMA and RRA performances. 0 20 40 60 80 100 Sparsity 0 10 20 30 40 CorLoc resnet18 resnet50 vit_b_32 [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: ResNet18, ResNet50, ViT-B-32 performances on CORLOC test. Object discovery (RQ2) [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Models accuracies on the HA task [6] (top-left Swin, top-right ViT-B-32, bottom-left ResNet50 and bottom-right ResNet18). chameleon, and Komodo dragon instead of tench, all animals sharing a similar scaled skin texture despite having different shapes and contexts. 5 Co…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

29 extracted references · 23 canonical work pages

  1. [26]

    Less is More: The Influence of Pruning on the Explainability of CNNs

    Weber, D., Merkle, F., Schöttle, P., Schlögl, S.: Less is more: The influence of pruning on the explainability of cnns. arXiv preprint arXiv:2302.08878 (2023)

  2. [1]

    Interpreting Convolutional Neural Networks Through Compression

    Abbasi-Asl, R., Yu, B.: Interpreting convolutional neural networks through com- pression. arXiv preprint arXiv:1711.02329 (2017)

  3. [2]

    Information Fusion81, 14–40 (2022)

    Arras, L., Osman, A., Samek, W.: Clevr-xai: A benchmark dataset for the ground truth evaluation of neural network explanations. Information Fusion81, 14–40 (2022)

  4. [3]

    In: 2009 IEEE Conference on Computer Vision and Pattern Recognition

    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. pp. 248–255 (2009)

  5. [4]

    International Journal of Computer Vision111(1), 98–136 (Jan 2015)

    Everingham, M., Eslami, S.M.A., Van Gool, L., Williams, C.K.I., Winn, J., Zisser- man, A.: The pascal visual object classes challenge: A retrospective. International Journal of Computer Vision111(1), 98–136 (Jan 2015)

  6. [5]

    In: International Conference on Learning Representations (2018)

    Frankle, J., Carbin, M.: The lottery ticket hypothesis: Finding sparse, trainable neural networks. In: International Conference on Learning Representations (2018)

  7. [6]

    In: Advances in Neural Information Processing Systems 34 (2021)

    Geirhos, R., Narayanappa, K., Mitzkus, B., Thieringer, T., Bethge, M., Wichmann, F.A., Brendel, W.: Partial success in closing the gap between human and machine vision. In: Advances in Neural Information Processing Systems 34 (2021)

  8. [7]

    In: International Conference on Learning Representations (2019)

    Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F.A., Brendel, W.: Imagenet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In: International Conference on Learning Representations (2019)

Show all 29 references
  1. [8]

    arXiv preprint arXiv:2307.13698 (2023)

    Ghosh, S., Batmanghelich, K.: Exploring the lottery ticket hypothesis with ex- plainability methods: Insights into sparse network performance. arXiv preprint arXiv:2307.13698 (2023)

  2. [9]

    Hooker, S., Courville, A., Dauphin, Y., Frome, A.: Selective brain damage: Mea- suring the disparate impact of model pruning (2019)

  3. [10]

    In: Euro- pean Conference on Computer Vision

    Jing, T., Xia, H., Tian, R., Ding, H., Luo, X., Domeyer, J., Sherony, R., Ding, Z.: Inaction: Interpretable action decision making for autonomous driving. In: Euro- pean Conference on Computer Vision. pp. 370–387. Springer (2022)

  4. [11]

    A series of books in psychology, Worth Publishers (2009)

    Kolb, B., Whishaw, I.: Fundamentals of Human Neuropsychology. A series of books in psychology, Worth Publishers (2009)

  5. [12]

    Liao, Z., Quétu, V., Nguyen, V.T., Tartaglione, E.: Can unstructured pruning reduce the depth in deep neural networks? In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops. pp. 1402–1406 (October 2023)

  6. [13]

    Proceedings of the Na- tional Academy of Sciences116(44), 22071–22080 (2019) 12 E

    Murdoch, W.J., Singh, C., Kumbier, K., Abbasi-Asl, R., Yu, B.: Definitions, meth- ods, and applications in interpretable machine learning. Proceedings of the Na- tional Academy of Sciences116(44), 22071–22080 (2019) 12 E. Cassano et al

  7. [14]

    arXiv preprint arXiv:2409.19727 (2024)

    vonRad,J.,Seuffert,F.:Investigatingtheeffectofnetworkpruningonperformance and interpretability. arXiv preprint arXiv:2409.19727 (2024)

  8. [15]

    In: International conference on machine learning

    Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F., Ben- gio, Y., Courville, A.: On the spectral bias of neural networks. In: International conference on machine learning. pp. 5301–5310. PMLR (2019)

  9. [16]

    In: Van Pelt, J., Corner, M., Uylings, H., Lopes Da Silva, F

    Rakic, P., Bourgeois, J.P., Goldman-Rakic, P.S.: Synaptic development of the cere- bral cortex: implications for learning, memory, and mental illness. In: Van Pelt, J., Corner, M., Uylings, H., Lopes Da Silva, F. (eds.) The Self-Organizing Brain: From Growth Cones to Functiona...

  10. [17]

    Neural computation29(9), 2352–2449 (2017)

    Rawat, W., Wang, Z.: Deep convolutional neural networks for image classification: A comprehensive review. Neural computation29(9), 2352–2449 (2017)

  11. [18]

    In: International Conference on Learning Representations (2020)

    Renda, A., Frankle, J., Carbin, M.: Comparing rewinding and fine-tuning in neural network pruning. In: International Conference on Learning Representations (2020)

  12. [19]

    In: 2022 IEEE International Conference on Image Processing (ICIP)

    Renzulli, R., Grangetto, M.: Towards efficient capsule networks. In: 2022 IEEE International Conference on Image Processing (ICIP). pp. 2801–2805 (2022)

  13. [20]

    In: Proceedings of the IEEE international conference on computer vision

    Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad- cam: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision. pp. 618–626 (2017)

  14. [21]

    In: BMVC 2021-32nd British Machine Vision Conference (2021)

    Siméoni, O., Puy, G., Vo, H.V., Roburin, S., Gidaris, S., Bursuc, A., Pérez, P., Marlet, R., Ponce, J.: Localizing objects with self-supervised transformers and no labels. In: BMVC 2021-32nd British Machine Vision Conference (2021)

  15. [22]

    In: ICLR (workshop track) (2015)

    Springenberg, J., Dosovitskiy, A., Brox, T., Riedmiller, M.: Striving for simplicity: The all convolutional net. In: ICLR (workshop track) (2015)

  16. [23]

    (eds.) Advances in Neural Information Processing Systems

    Sundaram, S., Fu, S., Muttenthaler, L., Tamir, N., Chai, L., Kornblith, S., Darrell, T., Isola, P.: When does perceptual alignment benefit vision representations? In: Globerson, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J., Zhang, C. (eds.) Advances in Neural...

  17. [24]

    In: International conference on machine learning

    Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: International conference on machine learning. pp. 3319–3328. PMLR (2017)

  18. [25]

    IEEE Journal of Biomedical and Health Informatics (2023)

    Wang, S., Du, X., Liu, G., Xing, H., Jiao, Z., Yan, J., Liu, Y., Lv, H., Xia, Y.: An interpretable data-driven medical knowledge discovery pipeline based on artificial intelligence. IEEE Journal of Biomedical and Health Informatics (2023)

  19. [27]

    In: Meila, M., Zhang, T

    Wong, E., Santurkar, S., Madry, A.: Leveraging sparse linear layers for debug- gable deep networks. In: Meila, M., Zhang, T. (eds.) Proceedings of the 38th In- ternational Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 139, pp. 11205–11216. PMLR...

  20. [28]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Ye, L., Rochan, M., Liu, Z., Wang, Y.: Cross-modal self-attention network for referring image segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 10502–10511 (2019)

  21. [29]

    Pattern Recognition115, 107899 (2021)

    Yeom, S.K., Seegerer, P., Lapuschkin, S., Binder, A., Wiedemann, S., Müller, K.R., Samek, W.: Pruning by explaining: A novel criterion for deep neural network prun- ing. Pattern Recognition115, 107899 (2021)

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.