REVIEW 3 major objections 6 minor 47 references
ITP: Instance-Aware Test Pruning for Out-of-Distribution Detection
T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read Pruning overconfident weights at test time improves OOD detection.
desk verdict Solid, practical OOD pruning method with honest ablations, but the universal improvement claim fails on Places365 and the SOTA claim leans on appendix-only baselines. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is the class-specific parameter contribution distribution. For a weight $W_{ij}$ in the last fully connected layer, its contribution to class $j$ for input $x$ is $W_{ij} h_i(x)$; the paper estimates mean $\mu_{ij}$ and $\sigma_{ij}$ over class-$j$ training data, assuming approximate Gaussianity. Coarse pruning (CRP) zeroes weights whose mean contribution falls in the lowest $p$ percentile, while fine-grained test pruning (FTP) zeroes weights whose standardized contribution $(c - \mu)/\sigma$ exceeds $\lambda$, a right-tailed Z-score test. The final OOD score is the negative energy of the logits computed with the element-wise product of both masks, which adaptively suppresses both noise and overconfidence per sample.
What would settle it
Take a trained classifier and its class-specific contribution statistics, then compute the two sides of Equation (14) on a fresh OOD benchmark. If the summed contributions of the pruned parameters are not larger for OOD than for ID inputs, the claimed mechanism for CRP is false; alternatively, show that setting those same parameters to zero raises FPR95 compared with keeping them.
Extended reading notes
Core claim
The central discovery is that OOD detection can be improved by pruning, at test time, parameters that are overconfident with respect to the in-distribution contribution distribution, not only by pruning parameters that are under-contributing. The paper defines each last-layer weight's contribution to a class output as its multiplicative effect on that class's logit, estimates a per-class Gaussian model of these contributions from ID data, and uses two masks: a fixed coarse mask that drops the lowest-contribution weights and an instance-specific fine mask that drops weights whose contribution exceeds the mean by more than a threshold lambda in standardized units. The pruned logits yield OOD scores that separate better, especially on CIFAR-100, where ITP lowers FPR95 from 68.45 (Energy baseline) to 35.03, and on ImageNet where average FPR95 drops to 22.51.
Load-bearing premise
The load-bearing premise is the inequality in Remark 1: parameters with the lowest average contribution to in-distribution predictions contribute more to the logits of OOD inputs than to those of ID inputs.
Editorial extensions
If this is right
- ITP improves OOD detection on top of energy, MSP, ODIN, GradNorm, and MLS scores without retraining.
- With only 30 images per class used to estimate distributions, ITP retains nearly full performance while preprocessing overhead drops to a few minutes.
- ITP is complementary to activation rectification; ITP combined with ReAct beats strong sparsification baselines on CIFAR-100 and ImageNet.
- The method adds negligible test-time cost because pruning is a masked matrix multiplication and requires no backpropagation.
Reading between the lines
- Because FTP is instance-dependent, it could be reused for selective prediction and calibrated rejection in tasks beyond OOD detection, such as detecting misclassified or ambiguous inputs.
- The Gaussian assumption in contribution distributions could be relaxed to heavier-tailed families; the Z-score test would then be replaced by a quantile-based rule, potentially improving robustness to outliers.
- It may be possible to extend the same two-stage pruning to earlier layers or to neuron-level dimensions, where redundant and overconfident signals also reside.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes ITP, a post-hoc OOD detection method that prunes last-layer weights. It first estimates class-specific parameter contribution distributions from ID data (assuming Gaussianity). CRP prunes parameters with lowest average contribution; FTP prunes, per test sample, parameters whose contribution Z-score exceeds a threshold λ. The OOD score is negative energy of the pruned logits. Experiments on CIFAR (DenseNet-101) and ImageNet (ResNet-50) show average FPR95 improvements over baselines, with ablations confirming both components.
Significance. If the claims hold, ITP is a useful addition to the post-hoc OOD detection toolbox: it is simple, training-free, has low preprocessing overhead, and is compatible with existing scores and ReAct. The derivation of the contribution formula (Eq. 5) is correct, and the ablations are informative. However, the central mechanism is justified only by an unproven inequality, and the method does not improve over the energy baseline on Places365 in the CIFAR experiments, so the claim that ITP 'improves OOD detection' needs to be scoped.
major comments (3)
- [§3.4, Remark 1, Eq. (14)] The inequality in Eq. (14) is asserted without proof and is the sole justification for the benefit of CRP. It is an empirical claim about OOD inputs that cannot be guaranteed from ID statistics alone. The paper's own per-dataset results (Tables 7 and 8) show that this mechanism fails on Places365: ITP has FPR95 45.75 vs Energy 39.40 on CIFAR-10, and 82.39 vs 79.20 on CIFAR-100. Reporting only averages in Tables 1 and 2 conceals these systematic failures. The authors should either prove Eq. (14) under stated conditions, validate it per OOD dataset, or explicitly scope the method's benefit to OOD types where the inequality holds.
- [§3.3, Eqs. (9)-(10)] The 'right-tailed Z-score test' is not a statistical hypothesis test: it is a deterministic threshold on a single observation using estimated moments, with λ selected on a validation set. No p-value, significance level, or error-rate control is provided, and the Gaussianity assumption is only checked visually (Appendix B). This framing overstates the method's statistical grounding; the authors should either replace it with a heuristic threshold description or provide proper statistical justification.
- [Appendix E and Table 4] The hyperparameters p and λ are selected on Gaussian-noise images. The paper does not justify that Gaussian noise is a valid proxy for the OOD distributions used in evaluation, and the results are highly sensitive to these choices (Table 4 shows ImageNet FPR95 ranging from 22.51 to 73.19 across the grid). The authors should justify the validation strategy or analyze sensitivity to the choice of validation OOD set, since the reported improvements depend on tuned hyperparameters.
minor comments (6)
- [Tables 7 and 8] The per-dataset failures on Places365 are not discussed in the main text; the authors should mention them and provide a possible explanation, as they bear on the scope of the method's effectiveness.
- [§3.3, after Eq. (9)] The phrase 'single-sample hypothesis testing task' is misleading; the procedure is better described as a fixed threshold rule on a Z-score.
- [Tables 9 and 10] There is a typo in the table titles: 'benckmarks' should be 'benchmarks'.
- [Figure 1 and Appendix B] The caption mentions 'pre-ReLU activations' but does not explain why these are used for visualization; please define this in the main text or figure caption.
- [Appendix E] Please specify the size of the Gaussian-noise validation images and whether they are preprocessed (resized/cropped) in the same way as ID images.
- [§4.3, ablation discussion] The explanation for FTP's smaller improvement on ResNet-50 ('larger feature space ... noise dominates') is speculative; consider providing supporting analysis or tempering the claim.
Circularity Check
No significant circularity: ITP's OOD scores are computed from pruning masks built on ID training data only, and the test OOD sets are not used in fitting.
full rationale
The claimed derivation chain is self-contained. CRP masks are defined by thresholding the mean contribution mu_ij estimated from class-j training data (Eqs. 6-8); FTP masks are defined by a right-tailed Z-score test whose mean and std are also from ID training data (Eqs. 9-10). The final OOD score is the negative energy of the pruned logits (Eqs. 3 and 12) and is evaluated on held-out OOD datasets that are not used when constructing the masks. Hyperparameters p and lambda are selected on a Gaussian-noise validation set (Appendix E); this is model selection on a proxy OOD distribution, not fitting to the test OOD sets, so the average FPR95/AUROC numbers remain an independent evaluation. The theoretical Remarks are heuristic: Remark 1's inequality (Eq. 14) is an asserted empirical property of redundant parameters, not a consequence of the definitions; Remark 2's inequality (Eq. 15) is essentially a restatement of the pruning rule and is not used to force the experimental outcome. The paper's self-citations (Yang et al. 2021/2023a/2023b/2024) appear only as general related-work references and are not load-bearing. The fact that the central assumption can fail on a particular OOD set (e.g., Places365, where ITP is worse than Energy on CIFAR-10/100 in Tables 7-8) concerns correctness/robustness, not circularity.
Assumptions & free parameters
free parameters (2)
- p (CRP pruning percentile) =
p=10 (CIFAR-10), p=20 (CIFAR-100), p=30 (ImageNet-1k)
- lambda (FTP Z-score threshold) =
lambda=2.2 (CIFAR-10), lambda=1.6 (CIFAR-100), lambda=1.5 (ImageNet-1k)
assumptions (4)
- domain assumption Parameter contributions of last-layer weights approximately follow Gaussian distributions per class
- domain assumption Redundant parameters (lowest average contribution on ID) contribute more to ID-class logits for OOD inputs than for ID inputs (Eq. 14)
- ad hoc to paper Gaussian-noise images are a valid proxy for OOD data for hyperparameter selection
- standard math Energy score is a reliable OOD scoring function
Cite this review
Pith. "Pith review of ITP: Instance-Aware Test Pruning for Out-of-Distribution Detection." pith.science (2026). https://pith.science/paper/UETDOFAB
@misc{pith2026241212566,
author = {Pith},
title = {Pith review of: ITP: Instance-Aware Test Pruning for Out-of-Distribution Detection},
year = {2026},
howpublished = {\url{https://pith.science/paper/UETDOFAB}},
note = {Machine review of arXiv:2412.12566}
}
read the original abstract
Out-of-distribution (OOD) detection is crucial for ensuring the reliable deployment of deep models in real-world scenarios. Recently, from the perspective of over-parameterization, a series of methods leveraging weight sparsification techniques have shown promising performance. These methods typically focus on selecting important parameters for in-distribution (ID) data to reduce the negative impact of redundant parameters on OOD detection. However, we empirically find that these selected parameters may behave overconfidently toward OOD data and hurt OOD detection. To address this issue, we propose a simple yet effective post-hoc method called Instance-aware Test Pruning (ITP), which performs OOD detection by considering both coarse-grained and fine-grained levels of parameter pruning. Specifically, ITP first estimates the class-specific parameter contribution distribution by exploring the ID data. By using the contribution distribution, ITP conducts coarse-grained pruning to eliminate redundant parameters. More importantly, ITP further adopts a fine-grained test pruning process based on the right-tailed Z-score test, which can adaptively remove instance-level overconfident parameters. Finally, ITP derives OOD scores from the pruned model to achieve more reliable predictions. Extensive experiments on widely adopted benchmarks verify the effectiveness of ITP, demonstrating its competitive performance.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al
Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F. L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al. 2023. Gpt-4 technical report. CoRR, abs/2303.08774
arXiv 2023
-
[2]
Ahn, Y. H.; Park, G.; and Kim, S. T. 2023. LINe: Out-of-Distribution Detection by Leveraging Important Neurons. In CVPR, pages 19852--19862
work page 2023
-
[3]
L.; Li, X.; He, W.; Gou, L.; and Ren, L
Behpour, S.; Doan, T. L.; Li, X.; He, W.; Gou, L.; and Ren, L. 2023. GradOrth: A Simple yet Efficient Out-of-Distribution Detection with Orthogonal Projection of Gradients. In NeurIPS
work page 2023
-
[4]
Bendale, A.; and Boult, T. E. 2016. Towards Open Set Deep Networks. In CVPR, pages 1563--1572
work page 2016
-
[5]
Chen, C.; Fu, Z.; Liu, K.; Chen, Z.; Tao, M.; and Ye, J. 2023. Optimal Parameter and Neuron Pruning for Out-of-Distribution Detection. In NeurIPS
work page 2023
-
[6]
Cimpoi, M.; Maji, S.; Kokkinos, I.; Mohamed, S.; and Vedaldi, A. 2014. Describing Textures in the Wild. In CVPR, pages 3606--3613
work page 2014
-
[7]
Djurisic, A.; Bozanic, N.; Ashok, A.; and Liu, R. 2023. Extremely Simple Activation Shaping for Out-of-Distribution Detection. In ICLR
work page 2023
-
[8]
Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; Uszkoreit, J.; and Houlsby, N. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In ICLR
2021
Show all 47 references
-
[9]
Geiger, A.; Lenz, P.; and Urtasun, R. 2012. Are we ready for autonomous driving? The KITTI vision benchmark suite. In CVPR, pages 3354--3361
2012
-
[10]
S.; Sun, Y.; and Li, Y
Ghosal, S. S.; Sun, Y.; and Li, Y. 2024. How to Overcome Curse-of-Dimensionality for Out-of-Distribution Detection? In AAAI, pages 19849--19857
2024
-
[11]
He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016. Deep Residual Learning for Image Recognition. In CVPR, pages 770--778
2016
-
[12]
He, R.; Yuan, Y.; Han, Z.; Wang, F.; Su, W.; Yin, Y.; Liu, T.; and Gong, Y. 2024. Exploring Channel-Aware Typical Features for Out-of-Distribution Detection. In AAAI, pages 12402--12410
2024
-
[13]
Hendrycks, D.; Basart, S.; Mazeika, M.; Zou, A.; Kwon, J.; Mostajabi, M.; Steinhardt, J.; and Song, D. 2022. Scaling Out-of-Distribution Detection for Real-World Settings. In ICML, volume 162, pages 8759--8773
2022
-
[14]
Hendrycks, D.; and Gimpel, K. 2017. A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks. In ICLR
2017
-
[15]
V.; Aodha, O
Horn, G. V.; Aodha, O. M.; Song, Y.; Cui, Y.; Sun, C.; Shepard, A.; Adam, H.; Perona, P.; and Belongie, S. J. 2018. The INaturalist Species Classification and Detection Dataset. In CVPR, pages 8769--8778
2018
-
[16]
Huang, G.; Liu, Z.; van der Maaten, L.; and Weinberger, K. Q. 2017. Densely Connected Convolutional Networks. In CVPR, pages 2261--2269
2017
-
[17]
Huang, H.; Li, Z.; Wang, L.; Chen, S.; Zhou, X.; and Dong, B. 2021. Feature Space Singularity for Out-of-Distribution Detection. In AAAI, volume 2808
2021
-
[18]
Huang, R.; Geng, A.; and Li, Y. 2021. On the Importance of Gradients for Detecting Distributional Shifts in the Wild. In NeurIPS, pages 677--689
2021
-
[19]
Kong, H.; and Li, H. 2022. BFAct: Out-of-Distribution Detection with Butterworth Filter Rectified Activations. In Sun, F.; Cangelosi, A.; Zhang, J.; Yu, Y.; Liu, H.; and Fang, B., eds., ICCSIP, volume 1787, pages 115--129
2022
-
[20]
Krizhevsky, A. 2009. Learning Multiple Layers of Features from Tiny Images
2009
-
[21]
Lee, K.; Lee, K.; Lee, H.; and Shin, J. 2018. A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks. In NeurIPS, pages 7167--7177
2018
-
[22]
Liang, S.; Li, Y.; and Srikant, R. 2018. Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks. In ICLR
2018
-
[23]
E.; Setio, A
Litjens, G.; Kooi, T.; Bejnordi, B. E.; Setio, A. A. A.; Ciompi, F.; Ghafoorian, M.; Van Der Laak, J. A.; Van Ginneken, B.; and S \'a nchez, C. I. 2017. A survey on deep learning in medical image analysis. Medical image analysis, 42: pages 60--88
2017
-
[24]
D.; and Li, Y
Liu, W.; Wang, X.; Owens, J. D.; and Li, Y. 2020. Energy-based Out-of-distribution Detection. In NeurIPS
2020
-
[25]
Malinin, A.; and Gales, M. J. F. 2018. Predictive Uncertainty Estimation via Prior Networks. In NeurIPS, pages 7047--7058
2018
-
[26]
H.; and V \' a zquez, D
Monteiro, J.; Rodr \' guez, P.; No \" e l, P.; Laradji, I. H.; and V \' a zquez, D. 2023. Constraining Representations Yields Models That Know What They Don't Know. In ICLR
2023
-
[27]
Netzer, Y.; Wang, T.; Coates, A.; Bissacco, A.; Wu, B.; and Ng, A. 2011. Reading Digits in Natural Images with Unsupervised Feature Learning
2011
-
[28]
Radford, A.; Narasimhan, K.; Salimans, T.; Sutskever, I.; et al. 2018. Improving language understanding by generative pre-training
2018
-
[29]
S.; et al
Shapley, L. S.; et al. 1953. A value for n-person games. Classics in game theory
1953
-
[30]
Sun, Y.; Guo, C.; and Li, Y. 2021. ReAct: Out-of-distribution Detection With Rectified Activations. In NeurIPS, pages 144--157
2021
-
[31]
Sun, Y.; and Li, Y. 2022. DICE: Leveraging Sparsification for Out-of-Distribution Detection. In ECCV, volume 13684, pages 691--708
2022
-
[32]
Sun, Y.; Ming, Y.; Zhu, X.; and Li, Y. 2022. Out-of-Distribution Detection with Deep Nearest Neighbors. In ICML, volume 162, pages 20827--20840
2022
-
[33]
Wang, H.; Li, Z.; Feng, L.; and Zhang, W. 2022. ViM: Out-Of-Distribution with Virtual-logit Matching. In CVPR, pages 4911--4920
2022
-
[34]
A.; Oliva, A.; and Torralba, A
Xiao, J.; Hays, J.; Ehinger, K. A.; Oliva, A.; and Torralba, A. 2010. SUN database: Large-scale scene recognition from abbey to zoo. In CVPR, pages 3485--3492
2010
-
[35]
Xu, M.; Lian, Z.; Liu, B.; and Tao, J. 2023. VRA: Variational Rectified Activation for Out-of-distribution Detection. In NeurIPS
2023
-
[36]
A.; Zhang, Y.; Finkelstein, A.; Kulkarni, S
Xu, P.; Ehinger, K. A.; Zhang, Y.; Finkelstein, A.; Kulkarni, S. R.; and Xiao, J. 2015. Turkergaze: Crowdsourcing saliency with webcam based eye tracking. CoRR, abs/1504.06755
2015 arXiv
-
[37]
Yang, J.; Wang, P.; Zou, D.; Zhou, Z.; Ding, K.; Peng, W.; Wang, H.; Chen, G.; Li, B.; Sun, Y.; Du, X.; Zhou, K.; Zhang, W.; Hendrycks, D.; Li, Y.; and Liu, Z. 2022. OpenOOD: Benchmarking Generalized Out-of-Distribution Detection. In NeurIPS
2022
-
[38]
Yang, Y.; Huang, Y.; Guo, W.; Xu, B.; and Xia, D. 2023 a . Towards Global Video Scene Segmentation with Context-Aware Transformer. In AAAI, 3206--3213
2023
-
[39]
Yang, Y.; Jiang, N.; Xu, Y.; and Zhan, D. 2024. Robust Semi-Supervised Learning by Wisely Leveraging Open-Set Data. TPAMI, 46(12): 8334--8347
2024
-
[40]
Yang, Y.; Zhang, C.; Xu, Y.; Yu, D.; Zhan, D.; and Yang, J. 2021. Rethinking Label-Wise Cross-Modal Retrieval from A Semantic Sharing Perspective. In IJCAI, 3300--3306
2021
-
[41]
Yang, Y.; Zhang, Y.; Song, X.; and Xu, Y. 2023 b . Not All Out-of-Distribution Data Are Harmful to Open-Set Active Learning. In Oh, A.; Naumann, T.; Globerson, A.; Saenko, K.; Hardt, M.; and Levine, S., eds., NeurIPS
2023
-
[42]
Yu, F.; Zhang, Y.; Song, S.; Seff, A.; and Xiao, J. 2015. LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop. CoRR, abs/1506.03365
2015 arXiv
-
[43]
Zhang, J.; Yang, J.; Wang, P.; Wang, H.; Lin, Y.; Zhang, H.; Sun, Y.; Du, X.; Zhou, K.; Zhang, W.; Li, Y.; Liu, Z.; Chen, Y.; and Li, H. 2023. OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detection. CoRR, abs/2306.09301
2023 arXiv
-
[44]
Zhou, B.; Lapedriza, \` A .; Khosla, A.; Oliva, A.; and Torralba, A. 2018. Places: A 10 Million Image Database for Scene Recognition. TPAMI
2018
-
[45]
Zhu, Y.; Chen, Y.; Xie, C.; Li, X.; Zhang, R.; Xue, H.; Tian, X.; Zheng, B.; and Chen, Y. 2022. Boosting Out-of-distribution Detection with Typical Features. In NeurIPS
2022
-
[46]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...
-
[47]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.