{"id":"4e8caad2-191b-42b1-a83c-9419da803f15","arxiv_id":"2412.12566","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"ITP prunes redundant weights plus per-instance overconfident weights in a network's last layer to improve out-of-distribution detection scores.","lead":"This paper introduces ITP, a post-hoc method that prunes a neural network's last-layer weights at test time to improve out-of-distribution detection. It removes both redundant weights and weights that behave overconfidently on a given input, and reports gains on CIFAR and ImageNet benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"CRP/FTP improvement hinges on an unproven assumption that pruned parameters contribute more for OOD than ID; per-dataset results (Places365) show the assumption can fail.","rationale":"The reader identified Eq. (14) as the weakest assumption; I agree but believe the concern is more acute and broader than the reader stated. Eq. (14) is not a harmless missing proof: it is the mechanism by which CRP is supposed to widen the gap, and an analogous assumption is required for FTP. Without it, pruning could just as easily shrink the gap. The per-dataset tables in the appendix contain direct counterexamples (Places365), which the main text does not discuss. This makes the central claim 'improves OOD detection' over-broad; a conditional verdict is appropriate. The concrete check is feasible because the code is released and the per-dataset metrics are already reported. Verdict remains CONDITIONAL, so no change to the reader's verdict is needed.","tokens_in":18776,"tokens_out":7917,"duration_ms":72150,"concrete_test":"Using the released code, compute for each OOD test dataset (a) the left and right sides of Eq. (14) for CRP, and (b) the mean normalized contribution removed by FTP on ID vs OOD samples. Check whether the CRP inequality holds per dataset and whether FTP removes more normalized contribution on OOD than ID. If the inequality is violated on Places365 or other datasets where ITP underperforms, the theoretical justification is incomplete and the central claim should be revised to state the condition under which ITP helps.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanism of ITP is a distributional inequality: the parameters selected for pruning (low-average contribution in CRP, high instance Z-score in FTP) must contribute more to ID-class logits for OOD inputs than for ID inputs. Equation (14) states this for CRP, and the same assumption underlies FTP's right-tail test, but no proof is provided. The concern is load-bearing because if the inequality fails, CRP/FTP removes information that is informative for ID or fails to reduce OOD energy, so the OOD score no longer separates ID from OOD. The paper's own per-dataset results show this failure mode: on Places365, ITP is worse than the energy baseline on both CIFAR-10 (FPR95 45.75 vs 39.40) and CIFAR-100 (82.39 vs 79.20), though the averages in Tables 1-2 are improved. Reporting only averages hides systematic per-dataset violations of the assumption. Thus the claim 'ITP improves OOD detection' should be scoped to OOD types where the inequality empirically holds.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ITP, a post-hoc OOD detection method that prunes last-layer weights. It first estimates class-specific parameter contribution distributions from ID data (assuming Gaussianity). CRP prunes parameters with lowest average contribution; FTP prunes, per test sample, parameters whose contribution Z-score exceeds a threshold λ. The OOD score is negative energy of the pruned logits. Experiments on CIFAR (DenseNet-101) and ImageNet (ResNet-50) show average FPR95 improvements over baselines, with ablations confirming both components.","tokens_in":18985,"tokens_out":5014,"duration_ms":42511,"significance":"If the claims hold, ITP is a useful addition to the post-hoc OOD detection toolbox: it is simple, training-free, has low preprocessing overhead, and is compatible with existing scores and ReAct. The derivation of the contribution formula (Eq. 5) is correct, and the ablations are informative. However, the central mechanism is justified only by an unproven inequality, and the method does not improve over the energy baseline on Places365 in the CIFAR experiments, so the claim that ITP 'improves OOD detection' needs to be scoped.","major_comments":[{"comment":"The inequality in Eq. (14) is asserted without proof and is the sole justification for the benefit of CRP. It is an empirical claim about OOD inputs that cannot be guaranteed from ID statistics alone. The paper's own per-dataset results (Tables 7 and 8) show that this mechanism fails on Places365: ITP has FPR95 45.75 vs Energy 39.40 on CIFAR-10, and 82.39 vs 79.20 on CIFAR-100. Reporting only averages in Tables 1 and 2 conceals these systematic failures. The authors should either prove Eq. (14) under stated conditions, validate it per OOD dataset, or explicitly scope the method's benefit to OOD types where the inequality holds.","section":"§3.4, Remark 1, Eq. (14)"},{"comment":"The 'right-tailed Z-score test' is not a statistical hypothesis test: it is a deterministic threshold on a single observation using estimated moments, with λ selected on a validation set. No p-value, significance level, or error-rate control is provided, and the Gaussianity assumption is only checked visually (Appendix B). This framing overstates the method's statistical grounding; the authors should either replace it with a heuristic threshold description or provide proper statistical justification.","section":"§3.3, Eqs. (9)-(10)"},{"comment":"The hyperparameters p and λ are selected on Gaussian-noise images. The paper does not justify that Gaussian noise is a valid proxy for the OOD distributions used in evaluation, and the results are highly sensitive to these choices (Table 4 shows ImageNet FPR95 ranging from 22.51 to 73.19 across the grid). The authors should justify the validation strategy or analyze sensitivity to the choice of validation OOD set, since the reported improvements depend on tuned hyperparameters.","section":"Appendix E and Table 4"}],"minor_comments":[{"comment":"The per-dataset failures on Places365 are not discussed in the main text; the authors should mention them and provide a possible explanation, as they bear on the scope of the method's effectiveness.","section":"Tables 7 and 8"},{"comment":"The phrase 'single-sample hypothesis testing task' is misleading; the procedure is better described as a fixed threshold rule on a Z-score.","section":"§3.3, after Eq. (9)"},{"comment":"There is a typo in the table titles: 'benckmarks' should be 'benchmarks'.","section":"Tables 9 and 10"},{"comment":"The caption mentions 'pre-ReLU activations' but does not explain why these are used for visualization; please define this in the main text or figure caption.","section":"Figure 1 and Appendix B"},{"comment":"Please specify the size of the Gaussian-noise validation images and whether they are preprocessed (resized/cropped) in the same way as ID images.","section":"Appendix E"},{"comment":"The explanation for FTP's smaller improvement on ResNet-50 ('larger feature space ... noise dominates') is speculative; consider providing supporting analysis or tempering the claim.","section":"§4.3, ablation discussion"}],"recommendation":"major_revision","confidential_remarks":"The paper is technically sound in its empirical execution and the code is available, but the central theoretical justification (Eq. 14) is an unproven empirical assumption, and the per-dataset results show that the method can underperform the energy baseline. This is not a rejectable flaw because the average results are strong and the method is post-hoc and efficient, but the authors must seriously revise the scoping and the statistical framing. I would lean toward major revision rather than minor because the load-bearing claim about CRP's mechanism is unsupported and the paper's own data contradict an unrestricted 'improves OOD detection' statement."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: ITP is a genuine incremental contribution—combining a fixed redundancy mask with an instance-specific Z-score mask for test-time pruning—and the empirical work is mostly careful. But the central theoretical justification is an unproven inequality, and the headline numbers hide systematic failures on Places365. The SOTA claim is too strong given KNN and ASH only appear in the appendix.\n\nWhat's new: DICE and LINe prune weights that matter for ID; ITP adds a second step that zeroes out weights whose per-instance contribution is far above the class-specific mean, i.e. parameters that are overconfident on that particular input. That's a real twist. The contribution formula is clean and the ablation shows both CRP and FTP help, especially on CIFAR-100. The preprocessing overhead is genuinely low and the code is public. The paper also shows the method is compatible with other OOD scores, which is useful.\n\nSoft spots, in order of severity:\n\n1. The theory. Remark 1 rests on Eq. 14, the assertion that pruned redundant parameters contribute more to ID-class logits for OOD than for ID. That's asserted, not proven, and it is load-bearing: if the inequality fails, CRP could remove useful information. The paper says it is 'empirically validated' in Figure 3, but Figure 3 shows two cherry-picked ImageNet OOD sets, not the failure cases.\n\n2. The per-dataset results contradict the universal claim. On CIFAR-10 and CIFAR-100 with Places365 as OOD, ITP is worse than plain Energy (45.75 vs 39.40 FPR95, and 82.39 vs 79.20). The averages in Tables 1-2 are pulled up by big wins on SVHN, Textures, etc. This tells you the assumption behind FTP/CRP doesn't hold for every OOD type. The paper never mentions this.\n\n3. SOTA claim. In the main tables the strongest baselines KNN and ASH are absent; in the appendix ASH-S beats ITP on CIFAR-10 FPR95 (15.05 vs 16.72) and ties on ImageNet. The claim 'state-of-the-art' needs qualification.\n\n4. Hyperparameters p and lambda are chosen on a Gaussian-noise validation set. That's a reasonable model-selection trick, but it's a tuned threshold, not a principled statistical test as the 'right-tailed Z-score test' framing implies.\n\nWho it's for: researchers working on post-hoc OOD detection, especially on sparsification-based methods. They'll find the idea useful and reproducible.\n\nRecommendation: definitely send to peer review. It deserves referees, but the authors should be asked to address the per-dataset breakdown, include KNN/ASH in main tables, and either prove or soften Eq. 14.","headline":"Solid, practical OOD pruning method with honest ablations, but the universal improvement claim fails on Places365 and the SOTA claim leans on appendix-only baselines.","tokens_in":19525,"tokens_out":3186,"would_cite":true,"duration_ms":25638,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Pruning overconfident weights at test time improves OOD detection.","keywords":["out-of-distribution detection","test-time pruning","weight sparsification","Z-score test","parameter contribution distribution","post-hoc method","overconfidence","energy score"],"falsifier":"Take a trained classifier and its class-specific contribution statistics, then compute the two sides of Equation (14) on a fresh OOD benchmark. If the summed contributions of the pruned parameters are not larger for OOD than for ID inputs, the claimed mechanism for CRP is false; alternatively, show that setting those same parameters to zero raises FPR95 compared with keeping them.","tokens_in":18546,"feed_emoji":"✂️","tokens_out":4818,"duration_ms":38520,"temperature":0.7,"pith_summary":"The paper claims that a post-hoc, training-free pruning scheme can substantially improve out-of-distribution detection in pre-trained image classifiers. Its diagnosis is that parameters which are important for in-distribution predictions can still behave overconfidently on OOD inputs, inflating the model's ID confidence. To fix this, ITP first removes redundant last-layer weights using class-specific contribution statistics, then, for each test sample, prunes weights whose contribution is an outlier high according to a right-tailed Z-score test. The result is a wider separation between ID and OOD energy scores, reported as large reductions in FPR95 on CIFAR and ImageNet benchmarks. The significance is that better OOD rejection is achieved without retraining or access to OOD data.","feed_headline":"Test-time pruning of overconfident weights lifts OOD detection","feed_subtitle":"New post-hoc method cuts CIFAR-100 FPR95 from 68.45 to 35.03 percent without retraining.","key_machinery":"The machinery is the class-specific parameter contribution distribution. For a weight $W_{ij}$ in the last fully connected layer, its contribution to class $j$ for input $x$ is $W_{ij} h_i(x)$; the paper estimates mean $\\mu_{ij}$ and $\\sigma_{ij}$ over class-$j$ training data, assuming approximate Gaussianity. Coarse pruning (CRP) zeroes weights whose mean contribution falls in the lowest $p$ percentile, while fine-grained test pruning (FTP) zeroes weights whose standardized contribution $(c - \\mu)/\\sigma$ exceeds $\\lambda$, a right-tailed Z-score test. The final OOD score is the negative energy of the logits computed with the element-wise product of both masks, which adaptively suppresses both noise and overconfidence per sample.","core_discovery":"The central discovery is that OOD detection can be improved by pruning, at test time, parameters that are overconfident with respect to the in-distribution contribution distribution, not only by pruning parameters that are under-contributing. The paper defines each last-layer weight's contribution to a class output as its multiplicative effect on that class's logit, estimates a per-class Gaussian model of these contributions from ID data, and uses two masks: a fixed coarse mask that drops the lowest-contribution weights and an instance-specific fine mask that drops weights whose contribution exceeds the mean by more than a threshold lambda in standardized units. The pruned logits yield OOD scores that separate better, especially on CIFAR-100, where ITP lowers FPR95 from 68.45 (Energy baseline) to 35.03, and on ImageNet where average FPR95 drops to 22.51.","pith_inferences":["Because FTP is instance-dependent, it could be reused for selective prediction and calibrated rejection in tasks beyond OOD detection, such as detecting misclassified or ambiguous inputs.","The Gaussian assumption in contribution distributions could be relaxed to heavier-tailed families; the Z-score test would then be replaced by a quantile-based rule, potentially improving robustness to outliers.","It may be possible to extend the same two-stage pruning to earlier layers or to neuron-level dimensions, where redundant and overconfident signals also reside."],"forward_implications":["ITP improves OOD detection on top of energy, MSP, ODIN, GradNorm, and MLS scores without retraining.","With only 30 images per class used to estimate distributions, ITP retains nearly full performance while preprocessing overhead drops to a few minutes.","ITP is complementary to activation rectification; ITP combined with ReAct beats strong sparsification baselines on CIFAR-100 and ImageNet.","The method adds negligible test-time cost because pruning is a masked matrix multiplication and requires no backpropagation."],"supporting_citations":[{"why":"Supplies the coarse-grained sparsification baseline that ITP's CRP builds on.","marker":"(Sun and Li 2022)"},{"why":"Provides the energy scoring function that ITP uses and the main baseline it improves.","marker":"(Liu et al. 2020)"},{"why":"Offers the activation rectification method that ITP combines with for further gains.","marker":"(Sun, Guo, and Li 2021)"},{"why":"Defines a competitive sparsification baseline using Shapley values and activated features.","marker":"(Ahn, Park, and Kim 2023)"},{"why":"Supplies a parameter and neuron pruning baseline against which ITP is compared.","marker":"(Chen et al. 2023)"},{"why":"Provides a channel-aware typical-set activation baseline that ITP outperforms.","marker":"(He et al. 2024)"},{"why":"Gives the OpenOOD v1.5 benchmark used for standardized evaluation in the appendix.","marker":"(Zhang et al. 2023)"}],"fun_headline_variants":["Instance-aware test pruning trims overconfident weights for OOD","Post-hoc pruning of overconfident parameters improves OOD detection","Adaptive test pruning removes overconfident weights for OOD","Prune overconfident weights at test time to lift OOD accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is the inequality in Remark 1: parameters with the lowest average contribution to in-distribution predictions contribute more to the logits of OOD inputs than to those of ID inputs.","fun_headline_variants_meta":{"raw":{"variants":["Instance-aware test pruning trims overconfident weights for OOD","Post-hoc pruning of overconfident parameters improves OOD detection","Adaptive test pruning removes overconfident weights for OOD","Prune overconfident weights at test time to lift OOD accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000546,"raw_usage":{"total_tokens":2614,"prompt_tokens":952,"completion_tokens":1662,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":568,"completion_tokens_details":{"reasoning_tokens":1588}},"tokens_in":568,"tokens_out":1662,"duration_ms":12623,"temperature":1.0,"reasoning_tokens":1588,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:56:55.051643+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a trained classifier and its class-specific contribution statistics, then compute the two sides of Equation (14) on a fresh OOD benchmark. If the summed contributions of the pruned parameters are not larger for OOD than for ID inputs, the claimed mechanism for CRP is false; alternatively, show that setting those same parameters to zero raises FPR95 compared with keeping them.","supporting_citations":[{"cited_title":"D.; and Li, Y","cited_arxiv_id":null,"evidence_quote":"Provides the energy scoring function that ITP uses and the main baseline it improves."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Offers the activation rectification method that ITP combines with for further gains."},{"cited_title":"H.; Park, G.; and Kim, S","cited_arxiv_id":null,"evidence_quote":"Defines a competitive sparsification baseline using Shapley values and activated features."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies a parameter and neuron pruning baseline against which ITP is compared."}],"review_version":1}