Pith. sign in

REVIEW 3 major objections 3 minor 20 references

A Framework for Uncertainty Quantification Based on Nearest Neighbors Across Layers

T0 review · 3 major / 3 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Nearest neighbors across layers outperform softmax at error detection

desk verdict A simple layer-wise kNN uncertainty idea, undermined by an overclaim in the abstract and a missing final-layer control. read the letter →

arxiv 2506.19895 v1 pith:KQP3IC6V submitted 2025-06-24 cs.LG cs.AI

Miguel N. Font , José L. Jorro-Aragoneses , Carlos M. Alaíz This is my paper
classification cs.LGcs.AI
keywords uncertaintyquantificationmisclassificationdetectionnearestneighborslayer-wiseanalysisactivationspaceconfidenceestimationdeeplearningpost-hocmethods
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the way a query's nearest-neighbor class distribution shifts between the layers of a trained neural network carries a usable signal about whether the final prediction is right. The framework stores every training activation vector, retrieves the $k$ closest training cases for each layer, and derives two metrics from their class labels: Decision Change (how often the dominant neighborhood class flips between layers) and Layer Uncertainty (the entropy of the neighborhood class mix at each layer). On MNIST and CIFAR-10 with a small CNN, a logistic regression on these layer-wise features detects misclassifications and estimates confidence at least as well as, and on the harder CIFAR-10 task better than, the standard softmax confidence baseline, and further improves when the features are added to softmax. The procedure is post-hoc, needs no retraining or architecture changes, and leaves an audit trail of retrieved training cases behind each uncertainty score. If the claim holds, any already-trained classifier can be fitted with a low-cost layer-wise error flag without touching the model.

What carries the argument

The central object is the layer-wise nearest-neighbor retrieval in activation space, organized in a Training Activation Repository and a Prediction Behavior Analysis Table. For each layer $\ell$, the framework stores the activation vectors of all training examples; for a query it retrieves the $k$ closest examples under the Bray–Curtis distance. Decision Change counts how often the modal class of the retrieved neighborhood changes between consecutive layers, using $\hat{y}_\ell = \mathrm{mode}\{y_i : i \in \mathcal{N}_\ell\}$ and flipping when $\hat{y}_\ell \neq \hat{y}_{\ell-1}$. Layer Uncertainty is the Shannon entropy of neighborhood class proportions at each layer, $H_\ell = -\sum_c p_\ell(c) \log p_\ell(c)$. These two layer-wise feature vectors, not the activations themselves, are what a logistic regression consumes to predict whether the network's answer on the query is correct.

What would settle it

Train the same logistic misclassification detector on the same CNN and datasets, but compute Decision Change and Layer Uncertainty features from randomly permuted layer indices of the retrieved neighborhoods; if the AUROC against softmax is unchanged, the layer-wise progression is not the carrier of the signal. Alternatively, if a final-layer-only $k$-NN feature set matches the full layer-wise AUROC, the across-layer dynamics add nothing.

Watch

Extended reading notes

Core claim

The paper's central claim is that the trajectory of nearest-neighbor class distributions through the layers of a trained network is a usable uncertainty signal. For every layer, the framework stores the activation vectors of all training points; for a new query it retrieves the $k$ nearest cases in each layer under Bray–Curtis distance and records their true labels. From this Prediction Behavior Analysis Table it computes Decision Change, the number of times the modal class of the retrieved neighborhood changes between consecutive layers, and Layer Uncertainty, the Shannon entropy of the neighborhood class distribution at each layer. The authors report that, combined, these layer-wise features discriminate correct from incorrect predictions more accurately than the final softmax probability on CIFAR-10, and that adding them to softmax improves error detection on MNIST, indicating the internal retrieval dynamics carry information not present in the final output.

Load-bearing premise

The framework assumes that the class-label mix of the $k$ nearest training examples in each intermediate layer's activation space, measured with Bray–Curtis distance, is informative about whether the model's final prediction is correct, and that this information is not already fully contained in the softmax probability.

Editorial extensions

If this is right

  • A deployed network's confidence can be corrected without retraining, because DC and LU are computed post-hoc and can be concatenated onto the softmax score.
  • Misclassification detectors built this way should transfer across prediction domains, since they rely on the trained network's own activations rather than on external error models or labels from other tasks.
  • The retrieved-neighbor table behind each uncertainty score makes the estimate inspectable, letting a user separate cases where the neighborhood is mislabeled from cases where the model itself is poorly learned.
  • On harder classification regimes with higher error rates, the combined metrics show a larger gain over softmax, suggesting the method pays off exactly when naive confidence is least reliable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same retrieval table could be recycled for out-of-distribution detection and for active learning, because examples whose neighborhood composition is unstable across layers are precisely the ones a labeler should examine; the paper lists out-of-distribution detection as future work but not active learning.
  • Bray–Curtis distance is one instantiation of the framework's underlying bet that activation-space neighborhoods are semantically coherent; swapping in cosine or Euclidean distance would test whether the metric, rather than the retrieval concept, carries the reported gains.
  • Because the experiments cover two small datasets and one two-convolution CNN, a natural extension is to check whether the layer-wise signal survives in transformer architectures, where layer definitions and activation magnitudes differ substantially; the paper itself flags RNNs and transformers as future directions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper proposes a post-hoc, model-agnostic uncertainty quantification framework for neural networks. For each layer of a trained network, the framework retrieves the k nearest training samples in that layer's activation space (using Bray-Curtis distance) and constructs a Prediction Behavior Analysis Table. From this table it defines two metrics: Decision Change (DC), the number of times the dominant class among the k retrieved labels changes between consecutive layers, and Layer Uncertainty (LU), the Shannon entropy of the class distribution among the k retrieved labels in each layer. These metrics are evaluated on a small CNN trained on MNIST and CIFAR-10, with logistic regression models fitted on 80% of the test set to predict whether a network prediction is correct, and evaluated on the remaining 20%. The paper reports AUROC, AUPR+, and AUPR- for softmax confidence, DC, LU, DC+LU, and SM+DC+LU. The central claim is that the layer-wise metrics, especially combined, improve misclassification detection and confidence estimation beyond the softmax baseline, particularly on the harder CIFAR-10 task.

Significance. If the central claim held in its strongest form, the framework would be a useful post-hoc UQ tool: it requires no retraining or architecture changes, it produces inspectable nearest-neighbor evidence, and the authors make code and results publicly available. The layer-wise perspective is a plausible and interesting direction, and the interpretability argument (users can inspect the retrieved cases) is a genuine strength. However, as presented, the evidence does not isolate the layer-wise mechanism, and the abstract's unqualified claim of 'outperforming softmax-based confidence' is contradicted by the paper's own MNIST results. The paper would be strengthened materially by adding the missing control baselines and by qualifying the performance claims to match the reported numbers.

major comments (3)
  1. [Abstract; Section 4.3; Table 2] The abstract claims the proposed metrics are 'outperforming softmax-based confidence' without qualification, but Table 2 shows the opposite on MNIST for the error-detection metric AUPR-: softmax reaches 33.96 while DC+LU reaches 26.76, and even SM+DC+LU reaches only 27.89. Section 4.3 states that combining DC and LU improves AUROC and AUPR for detecting errors on MNIST, which is not true for AUPR-. The claims should be restricted to the metrics and datasets where they hold, or the paper should explain why AUPR- is not the relevant criterion.
  2. [Section 4.3; Table 2] The experiment lacks a final-layer-only k-nearest-neighbor baseline. LU computed at the final layer is exactly the entropy of the class distribution among the k nearest training samples in the penultimate or pre-softmax embedding, and a kNN entropy baseline at the final layer would isolate whether the 'across layers' component contributes anything beyond known feature-space kNN uncertainty signals. Without this control, the observed gains over softmax on CIFAR-10 could be attributable to neighborhood entropy at the final layer alone, leaving the paper's specific novelty untested. This is a load-bearing gap for the central claim and should be addressed with an experiment.
  3. [Section 4.3; Table 2] The comparison is based on a single model and a single train/test split, with no standard deviations, repeated runs, or statistical significance tests. On CIFAR-10, the differences between LU and DC+LU are small (AUROC 72.30 vs. 73.49, AUPR- 56.37 vs. 57.56), and it is not clear whether these differences are stable across seeds or architectures. The paper should either report multiple seeds with variance or temper the conclusion that combining the metrics 'significantly enhance[s]' misclassification detection.
minor comments (3)
  1. [Table 1] In the DC panel for MNIST, the last row reads '10.94CIF AR-10', which appears to be a formatting error where the next dataset label is concatenated with the preceding number; the intended value and label boundaries should be restored.
  2. [Table 2 caption] The caption says the optimal number of neighbors is selected 'over the train set based on the results in Table 1', but Table 1 reports validation-set results; the wording should be clarified to say that validation performance was used to choose k.
  3. [Section 3.2] The choice of Bray-Curtis distance is motivated only by the framework's configurability; since the manuscript emphasizes a layer-wise analysis, a sentence justifying why Bray-Curtis is appropriate for activation vectors (e.g., non-negativity under ReLU) would help readers assess the robustness of the results.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DC and LU are defined from training activations and evaluated on a held-out split against an external softmax baseline, so the central claim has independent empirical content.

full rationale

The proposed metrics (Decision Change and Layer Uncertainty) are computed from the class labels of the k nearest training cases in each layer's activation space, with no fitting to test correctness labels. The evaluation fits logistic regression on correctness labels using these metrics as features, then measures AUROC and AUPR on a 20% held-out portion of the test set, comparing against a softmax baseline and a dummy classifier. This is standard supervised evaluation of a post-hoc score, not a reduction of the prediction to the fitted input. The optimal k is selected on a validation split of the same test-derived dataset, which is hyperparameter tuning and does not define the metrics in terms of the target. No load-bearing self-citation or imported uniqueness theorem appears; references are external benchmarks. The absence of a final-layer-only kNN control is a missing ablation and a threat to the novelty of the across-layers contribution, but it is an experimental gap, not circularity. Therefore the paper's derivation chain is not circular; score 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central empirical claim rests on one free hyperparameter k, a distance choice, and the assumption that neighbor label distributions in activation space are informative about correctness. No new physical or mathematical entities are introduced; the method reuses existing activations and training labels.

free parameters (2)
  • Neighborhood size k = k=3 for DC on MNIST, k=20 for LU on MNIST, k=20 for both on CIFAR-10
    Selected by validation AUROC/AUPR on a subset of the logistic regression training split; results in Table 1 show strong dependence on k.
  • Logistic regression weights for combining metrics = Not reported
    Used to combine DC and LU (and softmax) into a confidence score; fitted on 80% of test-set correctness labels, with no coefficients or regularization reported.
assumptions (4)
  • domain assumption True class labels in the Training Activation Repository are correct and are a valid proxy for the local class distribution.
    The framework infers dominant class and entropy from stored training labels; label noise or retrieval errors would directly distort DC and LU. Invoked in Sections 3.1 through 3.3.
  • ad hoc to paper Bray-Curtis distance is an appropriate similarity measure for activation vectors.
    Adopted without comparative justification or normalization discussion; alternative distances are acknowledged but untested in Section 3.2.
  • domain assumption Nearest neighbors in intermediate activation spaces reflect the model's decision dynamics and are predictive of final correctness.
    This is the load-bearing empirical premise; no theoretical argument is given, and the paper tests it on only two small CNN setups in Sections 3.3 and 4.
  • domain assumption The single 80/20 split of the test set provides unbiased estimates of misclassification detection quality.
    Logistic regression evaluators are fitted on test predictions and evaluated on a single 20% holdout; no repeated resampling or confidence intervals are used in Section 4.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Framework for Uncertainty Quantification Based on Nearest Neighbors Across Layers." pith.science (2026). https://pith.science/paper/KQP3IC6V

@misc{pith2026250619895,
  author       = {Pith},
  title        = {Pith review of: A Framework for Uncertainty Quantification Based on Nearest Neighbors Across Layers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KQP3IC6V}},
  note         = {Machine review of arXiv:2506.19895}
}
read the original abstract

Neural Networks have high accuracy in solving problems where it is difficult to detect patterns or create a logical model. However, these algorithms sometimes return wrong solutions, which become problematic in high-risk domains like medical diagnosis or autonomous driving. One strategy to detect and mitigate these errors is the measurement of the uncertainty over neural network decisions. In this paper, we present a novel post-hoc framework for measuring the uncertainty of a decision based on retrieved training cases that have a similar activation vector to the query for each layer. Based on these retrieved cases, we propose two new metrics: Decision Change and Layer Uncertainty, which capture changes in nearest-neighbor class distributions across layers. We evaluated our approach in a classification model for two datasets: CIFAR-10 and MNIST. The results show that these metrics enhance uncertainty estimation, especially in challenging classification tasks, outperforming softmax-based confidence.

Figures

Figures reproduced from arXiv: 2506.19895 by the authors.

Figure 1
Figure 1. Overview of the proposed framework. The extraction of relevant cases is detailed in Subsections 3.1 and 3.2, while the definition and computation of uncertainty metrics is explained in Subsection 3.3. 3.1 Training samples processing This stage constructs a structured representation that enables the comparison of similar behaviors between a new prediction and the samples used to train the NN model. The framework proc… view at source ↗
Figure 2
Figure 2. illustrates how this procedure works with a prediction example for both MNIST and CIFAR-10 datasets. On the left side, the new image to be classified is shown. In the central part, the five training samples retrieved from the TAR for layers 0 and 5 of each dataset are displayed; as explained above, these are the instances with the most similar activation vectors to the prediction. On the right side, the PBAT generat… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

20 extracted references · 7 canonical work pages

  1. [1]

    Information Fusion76, 243–297 (2021)

    Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U.R., Makarenkov, V., Nahavandi, S.: A review of uncertainty quantification in deep learning: Techniques, applica- tions and challenges. Information Fusion76, 243–297 (2021). https://doi.org/ 10.1016/j.inffus.2021.05.008

    doi:10.1016/j.inffus.2021.05.008 2021

  2. [2]

    Heliyon 4(11), e00938 (2018).https://doi.org/10.1016/j.heliyon.2018.e00938

    Abiodun, O.I., Jantan, A., Omolara, A.E., Dada, K.V., Mohamed, N.A., Arshad, H.: State-of-the-art in artificial neural network applications: A survey. Heliyon 4(11), e00938 (2018).https://doi.org/10.1016/j.heliyon.2018.e00938

    work page doi:10.1016/j.heliyon.2018.e00938 2018

  3. [3]

    van Amersfoort, J., Smith, L., Teh, Y.W., Gal, Y.: Uncertainty estimation using a single deep deterministic neural network (2020).https://doi.org/10.48550/ arXiv.2003.02037

    work page Pith review arXiv doi:10.48550/arxiv.2003.02037 2020

  4. [4]

    IEEE Signal Processing Magazine29(6), 141–142 (2012).https://doi

    Deng, L.: The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine29(6), 141–142 (2012).https://doi. org/10.1109/MSP.2012.2211477

    arXiv 2012

  5. [5]

    Pattern Recognition35(10), 2279–2301 (2002).https://doi

    Egmont-Petersen, M., de Ridder, D., Handels, H.: Image processing with neural networks—a review. Pattern Recognition35(10), 2279–2301 (2002).https://doi. org/10.1016/S0031-3203(01)00178-9

    work page doi:10.1016/s0031-3203(01)00178-9 2002

  6. [6]

    Pattern Recognition Letters27(8), 861–874 (2006)

    Fawcett, T.: An introduction to ROC analysis. Pattern Recognition Letters27(8), 861–874 (2006). https://doi.org/10.1016/j.patrec.2005.10.010

    doi:10.1016/j.patrec.2005.10.010 2006

  7. [7]

    Artificial Intelligence Review 56(1), 1513–1589 (2023).https://doi.org/10.1007/s10462-023-10562-9

    Gawlikowski, J., Tassi, C.R.N., Ali, M., Lee, J., Humt, M., Feng, J., Kruspe, A., Triebel, R., Jung, P., Roscher, R., Shahzad, M., Yang, W., Bamler, R., Zhu, X.X.: A survey of uncertainty in deep neural networks. Artificial Intelligence Review 56(1), 1513–1589 (2023).https://doi.org/10.1007/s10462-023-10562-9

    doi:10.1007/s10462-023-10562-9 2023

  8. [8]

    Hendrycks, D., Gimpel, K.: A Baseline for Detecting Misclassified and Out-of- Distribution Examples in Neural Networks (2018).https://doi.org/10.48550/ arXiv.1610.02136 12 M. N. Font et al

    arXiv doi:10.48550/arxiv.1610.02136 2018

Show all 20 references
  1. [9]

    Applied Soft Computing 149, 111027 (2023).https://doi.org/10.1016/j.asoc.2023.111027

    Kabir, H.D., Mondal, S.K., Khanam, S., Khosravi, A., Rahman, S., Qazani, M.R.C., Alizadehsani, R., Asadi, H., Mohamed, S., Nahavandi, S., Acharya, U.R.: Uncertainty aware neural network from similarity and sensitivity. Applied Soft Computing 149, 111027 (2023).https://doi.org/...

    2023

  2. [10]

    Kendall, A., Gal, Y.: What uncertainties do we need in bayesian deep learning for computer vision? (2017).https://doi.org/10.48550/arXiv.1703.04977

    2017 doi arXiv

  3. [11]

    https://doi.org/10.1016/j.strusafe.2008.06

    Kiureghian, A.D., Ditlevsen, O.: Aleatory or epistemic? Does it matter? Structural Safety 31(2), 105–112 (2009). https://doi.org/10.1016/j.strusafe.2008.06. 020

    2009 doi

  4. [12]

    Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images (2009), https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf

    2009

  5. [13]

    Lee, K., Lee, K., Lee, H., Shin, J.: A simple unified framework for detecting out-of- distribution samples and adversarial attacks (2018).https://doi.org/10.48550/ arXiv.1807.03888

    2018 doi arXiv

  6. [14]

    Liang, S., Li, Y., Srikant, R.: Principled detection of out-of-distribution examples in neural networks (2017).https://doi.org/10.48550/arXiv.1706.02690

    2017 doi arXiv

  7. [15]

    Liu, J.Z., Lin, Z., Padhy, S., Tran, D., Bedrax Weiss, T., Lakshminarayanan, B.: Simple and principled uncertainty estimation with deterministic deep learning via distance awareness (2020).https://doi.org/10.48550/arXiv.2006.10108

    2020 doi arXiv

  8. [16]

    In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Mukhoti, J., Kirsch, A., van Amersfoort, J., Torr, P.H., Gal, Y.: Deep Deter- ministic Uncertainty: A New Simple Baseline . In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 24384–24394. IEEE Computer Society (2023).https://doi.org/10.1109/CVP...

    2023

  9. [17]

    Ramalho, T., Miranda, M.: Density Estimation in Representation Space to Predict Model Uncertainty (2020).https://doi.org/10.1007/978-3-030-62144-5_7

    2020 doi

  10. [18]

    PloS one10, e0118432 (2015)

    Saito, T., Rehmsmeier, M.: The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PloS one10, e0118432 (2015). https://doi.org/10.1371/journal.pone.0118432

    2015 doi

  11. [19]

    npj Computational Materials9(1) (2023)

    Tan, A.R., Urata, S., Goldman, S., Dietschreit, J.C.B., Gómez-Bombarelli, R.: Single-model uncertainty quantification in neural network potentials does not con- sistently outperform model ensembles. npj Computational Materials9(1) (2023). https://doi.org/10.1038/s41524-023-01180-8

    2023 doi

  12. [20]

    48550/arXiv.2207.07235

    Thiagarajan, J.J., Anirudh, R., Narayanaswamy, V., Bremer, P.T.: Single model uncertainty estimation via stochastic data centering (2022).https://doi.org/10. 48550/arXiv.2207.07235

    2022 doi arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.