{"id":"9d6d4df0-52a9-4dbf-9ad6-be888343ad33","arxiv_id":"1908.07235","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A learned classifier over k-nearest-neighbor statistics in a pretrained network's representation space predicts misclassifications and detects out-of-distribution images without training on out-of-distribution examples.","lead":"Neural networks can give high-confidence wrong answers on images unlike anything in their training set. This paper proposes a helper model that looks at how close a new image's internal representation is to stored training images, plus the original model's confidence, to flag likely mistakes and out-of-distribution inputs.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"NUC's reported gains may come from its softmax-confidence input; the paper runs no confidence-only ablation, so the density-estimation contribution to the central claim is unisolated.","rationale":"The reader's weakest assumption emphasizes OOD transfer of the in-distribution error proxy, but also explicitly notes that 'the learned component's contribution beyond softmax confidence is not isolated in the tables.' I agree with the latter point and consider it the more decisive issue: it affects even the in-distribution claim, and it can be settled directly with an ablation that the paper should have run. The OOD transfer worry is real but is partially mitigated by the fact that NUC is tested on three OOD datasets with reasonable results; the missing confidence-only control, by contrast, leaves open the possibility that the reported gains are simply a nonlinear calibration of softmax confidence. The small margins in Tables 1 and 2 (e.g., 0.002 AUROC on CIFAR-10) make this concern acute. This is an addressable omission, not a demonstrated failure, so the reader's CONDITIONAL verdict remains appropriate; no change is needed.","tokens_in":9043,"tokens_out":8069,"duration_ms":87013,"concrete_test":"Train NUC variants with the same architecture, training procedure, and data as Algorithm 1, but with inputs restricted to (a) s(y_i) only and (b) s(y_i) plus the Eq. 3 agreement statistic at k=10. Evaluate both variants on the same test sets used in Tables 1 and 2 (ILSVRC2012 validation, ImageNet-V2, ImageNet-C, ImageNet-unk, SVHN, CIFAR-10) and report AUROC with bootstrap 95% confidence intervals. If the confidence-only variant matches full NUC within +/-0.005 AUROC on every set, the neighborhood statistics contribute no measurable benefit and the central claim fails; if a clear margin remains, the density information is doing real work.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline result (Tables 1 and 2) is that NUC beats softmax confidence, calibrated softmax, k-NN density statistics, and Mahalanobis distance. However, NUC's input, defined in Eq. 4, includes s(y_i), the original model's softmax confidence, alongside the neighborhood terms. Figure 3 is the only ablation of this input: the variant 'NUC (-confidence)' is strongly k-dependent and, on the validation set, appears to fall below the plain softmax baseline (0.844 in Table 1) for most values of k shown. No model is trained with s(y_i) alone, or with s(y_i) plus only the cheap agreement statistic Eq. 3, so the positive gap between NUC and Softmax/Softmax† could be due to NUC being a nonlinear, trainable re-scoring of the confidence value rather than to density estimation in representation space. The temperature-scaled Softmax† baseline does not settle this, because temperature scaling is monotone, whereas g_theta can implement arbitrary nonlinear calibration. Since the central claim is specifically that neighborhood/density statistics improve uncertainty prediction, this missing control is load-bearing: if a confidence-only version matches full NUC, the method's stated mechanism and its advantage over the softmax baselines are not supported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes the Neighborhood Uncertainty Classifier (NUC), a post-hoc uncertainty estimator for a pretrained classifier. NUC computes k-nearest-neighbor statistics in the penultimate representation space of the base model (Eqs. 1-3), concatenates them with the base model's softmax confidence s(y_i) (Eq. 4), and trains a small permutation-invariant network g_theta to predict whether the base model's prediction is correct. Training uses only in-distribution data and the binary cross-entropy loss of Eq. 5, via Algorithm 1. The trained NUC score is then evaluated for in-distribution misclassification detection (Table 1) and out-of-distribution detection (Table 2) on ImageNet-scale data, using Inception-ResNet-v2 as the base model. The paper reports that NUC outperforms softmax confidence, temperature-scaled softmax, the k-NN statistics of Eqs. 1-3, and Mahalanobis distance on most of the tested settings.","tokens_in":9295,"tokens_out":4032,"duration_ms":43825,"significance":"If the reported results hold, NUC would be a practically useful and conceptually simple method: it requires no out-of-distribution training data, works with an arbitrary pretrained classifier, and is evaluated on a realistic large-scale setting rather than only on MNIST/CIFAR. The use of a fixed pretrained checkpoint, public datasets, and a clearly specified training procedure (Algorithm 1) is a strength, and the paper's central hypothesis--that neighborhood statistics in representation space carry signal about model error--is well motivated and partially supported by Figure 1. However, the current evidence does not yet isolate the contribution of the neighborhood statistics from the softmax-confidence input, and two evaluation choices (restricting OOD positives to correctly classified in-distribution images, and selecting k on the same validation set used for scoring) weaken the strength of the empirical claims. These issues are fixable with additional experiments, but they are load-bearing for the paper's central claim that density estimation in representation space is what drives the improvement.","major_comments":[{"comment":"The paper does not run the key ablation needed to support its central claim. Eq. (4) defines NUC's input as the neighborhood statistics together with s(y_i), the base model's softmax confidence. Figure 3 shows that removing the confidence input makes performance strongly dependent on k, but it does not show the converse: a model trained with s(y_i) alone, or with s(y_i) plus only the cheap agreement statistic Eq. (3). Since g_theta is a nonlinear function, a confidence-only version could reproduce or exceed the reported NUC results in Tables 1 and 2 simply by recalibrating the softmax output. Without this control, the stated advantage over the Softmax and Softmax† baselines cannot be attributed to density estimation in representation space. I request an ablation comparing (i) g_theta trained on s(y_i) alone, (ii) s(y_i) plus Eq. (3), (iii) s(y_i) plus all neighborhood statistics, and (iv) the current full NUC, with the same training setup and k.","section":"Section 3, Eq. (4) and Figure 3"},{"comment":"The out-of-distribution evaluation restricts the in-distribution positive set to 'correctly classified ILSVRC2012 validation set images.' This makes the OOD detection task easier and does not match the deployment setting, where the detector must separate OOD inputs from all in-distribution inputs, including misclassified ones. Restricting positives in this way can favor methods that are good at flagging errors rather than methods that genuinely detect distributional shift, and it changes the AUROC/AUPR numbers for all methods. The authors should rerun the OOD experiments using the full ILSVRC2012 validation set as the in-distribution set, or at minimum report both variants and justify the restriction.","section":"Table 2 caption and Section 4.2"},{"comment":"No error bars, confidence intervals, or statistical significance tests are reported. Several headline differences are small (e.g., NUC AUROC 0.862 vs. Softmax 0.848 on ILSVRC2012 validation in Table 1, and 0.846 vs. 0.835 for Mahalanobis on ImageNet-unknown in Table 2), and with a single evaluation of a deterministic pipeline it is unclear whether these differences are meaningful. I ask the authors to report bootstrap confidence intervals over the test set or repeated training runs, particularly for the ImageNet-unknown and Imagenet-C results.","section":"Section 4 and Tables 1-2"},{"comment":"The hyperparameter k is selected using the same ILSVRC2012 validation set that is then used for scoring in Table 1. Figure 3 plots validation AUROC as a function of k, and the text states 'we set k = 10' immediately afterward. This is a mild form of validation-set overfitting for NUC, and it is not clear whether the baselines receive an analogous tuning (Softmax† is calibrated on the validation set, but the k-NN and Mahalanobis baselines are not tuned in the same way). The authors should either use a separate split for hyperparameter selection or report the sensitivity of the Table 1 results across a range of k.","section":"Section 4, Figure 3 and Table 1"}],"minor_comments":[{"comment":"Equations (1) and (2) are called kernel density estimates, but they are sums of distances rather than sums of normalized kernel evaluations; this should be clarified or the terminology changed. Similarly, Eq. (3) is an agreement count, not a conditional probability density, and the notation P(x_i | \\hat y_i) is misleading.","section":"Section 2, Eqs. (1)-(3)"},{"comment":"The term 'Imagenetunk.' appears in both the text and Table 2; this appears to be a typo and should be 'ImageNet-unk' or similar.","section":"Section 4.2 and Table 2"},{"comment":"The statement that epistemic uncertainty 'cannot be reduced even as the size of the training set increases' is imprecise under standard definitions; epistemic uncertainty is usually reducible with more data. The intended point appears to be that a fixed pretrained model cannot reduce its own epistemic uncertainty, but the current wording is confusing.","section":"Section 1, last paragraph of Introduction"},{"comment":"Algorithm 1 iterates over the 'training set' and queries nearest neighbors from a set A that is also described as containing the training set representations. The text says the exact point is excluded, but Algorithm 1 does not explicitly state this exclusion in the pseudocode; adding it would prevent ambiguity.","section":"Section 3, Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The missing confidence-only ablation is the central technical risk: the paper's stated contribution is density estimation in representation space, but the current experiments do not rule out the possibility that NUC is simply a nonlinear recalibration of softmax confidence. The OOD evaluation's restriction to correctly classified in-distribution positives is also a substantive concern that should be addressed before publication. If the requested ablations and evaluation fixes confirm the current ranking, the paper would make a solid contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a clean, honest paper about adding uncertainty scores to a pretrained ImageNet classifier by feeding k-NN statistics from representation space, plus the model's softmax confidence, into a small permutation-invariant net trained on in-distribution mistakes. The method is simple, the related work is fairly cited, and the ImageNet-scale evaluation covers several OOD sets. If the reported margins hold, it's a useful post-hoc tool. But the paper's central mechanism is not actually isolated: since NUC's input includes the softmax confidence, and Figure 3 shows the no-confidence variant often sits around or below the plain softmax baseline, the gap in Tables 1 and 2 could be driven mostly by a learned nonlinear rescoring of confidence, not by density estimation. There is no model trained with confidence alone, which would have settled this.\n\nWhat is genuinely new: the specific learned aggregation over k-NN distance/label-agreement statistics with a permutation-invariant network, applied to a single pretrained model at ImageNet scale, trained only on in-distribution errors. The authors are upfront that distance-based predecessors exist, and they include several relevant baselines. The qualitative check in Figure 1 supports the underlying assumption. I don't see circularity: no OOD labels are used in training.\n\nWeak spots, in order of importance. First, the missing confidence-only ablation is load-bearing. The k-NN features may still help, but the evidence as presented doesn't show it; Figure 3 suggests they might not on validation. Second, the gains over softmax are small (0.862 vs 0.848 AUROC in-distribution; 0.976 vs 0.975 on CIFAR-10 OOD), and there are no error bars or significance tests, so I wouldn't bank on the exact ordering. Third, k is selected on the validation set, which is also the evaluation set for the in-distribution task; mild, but worth a sensitivity note or nested selection. Fourth, two directly relevant baselines, Deep k-NN and ODIN, are cited in related work but not included.\n\nOverall: this is a solid, honest contribution that deserves a serious referee, but the empirical claim needs a confidence-only ablation and a bit more statistical care before I'd trust the headline margins. I'd point a practitioner to it as a cheap post-hoc uncertainty option, and I'd read the paper myself if revising the section on OOD detection. For peer review, yes: send it out, with the ablation as the main requested revision.","headline":"Useful post-hoc uncertainty method with honest related work, but the reported gains over softmax confidence are small and may come from the confidence input itself; the missing confidence-only ablation is the key issue.","tokens_in":9824,"tokens_out":2417,"would_cite":false,"duration_ms":25130,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A small network reading the neighborhood of a test point in representation space outperforms softmax confidence at predicting model mistakes and out-of-distribution inputs.","keywords":["uncertainty estimation","out-of-distribution detection","representation space density","k-nearest neighbors","misclassification prediction","ImageNet","Neighborhood Uncertainty Classifier","softmax confidence"],"falsifier":"Train NUC on ILSVRC2012 errors, then evaluate it on out-of-distribution points crafted to sit close to training representations in the last layer, such as small adversarial perturbations. If NUC's AUROC for OOD detection falls to or below the softmax baseline, the transfer of the learned density-to-error mapping would be refuted.","tokens_in":8833,"feed_emoji":"🎯","tokens_out":8119,"duration_ms":68371,"temperature":0.7,"pith_summary":"This paper introduces the Neighborhood Uncertainty Classifier (NUC), a post-hoc model that predicts when a pretrained image classifier will be wrong. It gathers the $k$ nearest training-set neighbors of a test image in the classifier's last-layer representation space, summarizes them by distances and label agreements, adds the softmax confidence, and trains a small permutation-invariant network to forecast mistakes using only in-distribution examples. The central claim is that this density-based score transfers to out-of-distribution detection without ever training on out-of-distribution data. On ImageNet-scale benchmarks, NUC reports higher AUROC than softmax confidence, calibrated softmax, k-NN density statistics, and Mahalanobis distance for both misclassification prediction and out-of-distribution detection. The paper concludes that local density in representation space is a practical source of uncertainty for existing deep models.","feed_headline":"Neighborhood density beats softmax at spotting wrong predictions","feed_subtitle":"A small network trained only on in-distribution mistakes also spots out-of-distribution images at ImageNet scale.","key_machinery":"The load-bearing object is the set of $k$ nearest neighbors of a test representation inside a stored index of all training-set representations at the final hidden layer. The paper defines three summary statistics from this set: unconditional neighbor distance (Eq. 1), class-conditional neighbor distance (Eq. 2), and neighbor label agreement (Eq. 3). The actual mechanism is a two-layer permutation-invariant neural network, built with a Deep Sets-style aggregation step (Eq. 6), that consumes per-neighbor distances and label agreement indicators together with the original softmax confidence and outputs the probability of a mistake. This network is trained by minimizing binary cross-entropy on in-distribution correct/incorrect labels, so the neighbor statistics act as a learned local density estimate rather than a fixed kernel density.","core_discovery":"The paper's central discovery is that the correctness of a pretrained classifier's prediction on a new input can be predicted more accurately from the local density of training representations around that input than from the classifier's own confidence. It demonstrates on ILSVRC2012 that incorrectly classified validation images lie farther from their nearest training-set neighbors and show lower neighbor label agreement (Fig. 1). NUC is then trained as a binary classifier that maps neighbor distances, neighbor label agreements, and the softmax score to the probability of a mistake; as a direct corollary, the same trained NUC ranks truly out-of-distribution images as uncertain without having seen any such image. In the reported experiments, NUC achieves AUROC 0.862 versus 0.848 for softmax on in-distribution misclassification detection and 0.846 versus 0.835 for Mahalanobis on unknown ImageNet classes, and it also edges out baselines on SVHN and CIFAR-10.","pith_inferences":["The paper does not isolate the contribution of the learned neighborhood statistics from the softmax signal; a direct ablation would train NUC with only the softmax score as input, and the AUROC difference would quantify how much density adds beyond confidence.","The transfer assumption could be tested with graded distribution shifts, such as increasing corruption severity in ImageNet-C, to see whether NUC's margin over baselines shrinks as the representation geometry departs from the training distribution.","An extension the paper leaves implicit is replacing the full training-set index with a small set of learned prototype representations, which would make the method feasible when storing all representations is impractical.","The method's behavior under adversarial perturbations is untested; because adversarial inputs are designed to lie near the training manifold in representation space, they are the stress case for the claim that distance correlates with distributional distance."],"forward_implications":["NUC can be attached to an already trained classifier without retraining it, requiring only a stored index of training representations and a single epoch of training for the small uncertainty network.","Because it is trained on in-distribution errors, the method applies to any real-world model whose training error is not zero, which is typical of regularized state-of-the-art networks.","The reported gains over calibrated softmax on ImageNet-V2 and ImageNet-C suggest that local representation density captures mistakes that confidence calibration does not.","The stability of the full model across values of $k$ in Fig. 3 means a practitioner does not need to tune $k$ precisely when the softmax confidence is included.","On out-of-distribution detection, NUC's edge over Mahalanobis distance on ImageNet unknown classes indicates that a learned combination of distance and label agreement is more flexible than a single class-conditional Gaussian."],"supporting_citations":[{"why":"Supplies the pretrained Inception-ResNet-v2 model whose final-layer representation space is used throughout.","marker":"[30]"},{"why":"Provides the ImageNet ILSVRC2012 training set used to build the representation index and the validation set for evaluating misclassification detection.","marker":"[27]"},{"why":"Defines the softmax-confidence baseline and the AUROC/AUPR metrics used for all comparisons.","marker":"[11]"},{"why":"Provides the temperature-calibration method for the calibrated softmax baseline (marked †).","marker":"[7]"},{"why":"Supplies the Mahalanobis distance baseline that NUC is compared against for out-of-distribution detection.","marker":"[17]"},{"why":"Provides ImageNet-V2, an alternative validation set used to test in-distribution uncertainty under subtle distribution shift.","marker":"[26]"},{"why":"Provides ImageNet-C, corrupted ImageNet images used to test robustness of uncertainty estimates.","marker":"[10]"},{"why":"Supplies the permutation-invariant aggregation construction used for the NUC network.","marker":"[32]"}],"fun_headline_variants":["Density in representation space beats softmax for uncertainty","Neighbor density predicts wrong predictions and OOD data","Representation-space density spotlights model mistakes","Local density in representation space flags errors and OOD","Predict model errors with representation-space density"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the relationship between local density in representation space and model error, learned from in-distribution mistakes, stays valid for inputs far outside the training distribution; the paper supports it only with a qualitative picture of the validation set.","fun_headline_variants_meta":{"raw":{"variants":["Density in representation space beats softmax for uncertainty","Neighbor density predicts wrong predictions and OOD data","Representation-space density spotlights model mistakes","Local density in representation space flags errors and OOD","Predict model errors with representation-space density"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000706,"raw_usage":{"total_tokens":3134,"prompt_tokens":847,"completion_tokens":2287,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":463,"completion_tokens_details":{"reasoning_tokens":2216}},"tokens_in":463,"tokens_out":2287,"duration_ms":17275,"temperature":1.0,"reasoning_tokens":2216,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:21:23.871765+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train NUC on ILSVRC2012 errors, then evaluate it on out-of-distribution points crafted to sit close to training representations in the last layer, such as small adversarial perturbations. If NUC's AUROC for OOD detection falls to or below the softmax baseline, the transfer of the learned density-to-error mapping would be refuted.","supporting_citations":[{"cited_title":"Inception-v4, Inception- ResNet and the Impact of Residual Connections on Learning","cited_arxiv_id":null,"evidence_quote":"Supplies the pretrained Inception-ResNet-v2 model whose final-layer representation space is used throughout."}],"review_version":1}