{"id":"301c6e28-75c7-4ecf-b12c-92a7a2d394ba","arxiv_id":"2502.00456","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Distance from a softmax prediction to the mean softmax vector of correct examples can serve as a confidence score for flagging low-confidence and out-of-distribution predictions, based on MNIST and CIFAR-10 experiments.","lead":"This paper proposes a simple way to make a neural network say 'I don't know': measure how far a prediction's confidence vector sits from the average vector of correctly classified examples for each class, and flag predictions that are too far away. It reports that on MNIST and CIFAR-10 this distance tracks accuracy, but the method is tested without comparisons to existing uncertainty or out-of-distribution detection baselines.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central abstention claim is only evaluated on the training data used to derive centroids and thresholds; the reported accuracy-retention trend may be an in-sample artifact.","rationale":"The reader's weakest assumption concerns cluster compactness and unimodality. That is a real concern, but the more load-bearing issue for the paper's central claim is that the reported evidence is in-sample. The paper derives centroids and thresholds on training data and then reports accuracy and retention on the same training data; Figure 3 is explicitly labeled as training results and Table 1 uses training-set sizes. This makes the central claimed relationship between distance and confidence appear stronger than it is: at small thresholds, the method is essentially removing training points that are far from the fitted centroids, and those points disproportionately include known training errors. A held-out test evaluation could still support the claim, and the method is simple and plausible, so the current descriptor should remain conditional rather than being rejected outright. The reader's rationale did note that the threshold is fit to training errors, which overlaps with this concern, but did not identify the in-sample evaluation as the central weakness, so agreement is partial.","tokens_in":13336,"tokens_out":4746,"duration_ms":51075,"concrete_test":"Run a strict split evaluation: train the CNN and ViT as in the paper; compute per-class centroids from training correct predictions (Algorithm 1) and per-class thresholds from training incorrect predictions (Algorithm 2). Then evaluate on the MNIST and CIFAR-10 test sets, reporting accuracy and retention at the Algorithm 2 thresholds and across a threshold sweep. Include a baseline that rejects the same fraction of examples using maximum softmax probability. If the accuracy-retention gain disappears on held-out data, or if the method is no better than the max-softmax baseline at matched retention, the central claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that distance to a per-class mean softmax centroid can decide when to defer a prediction to a human operator. The evidence offered is Figure 3 and Table 1. However, Figure 3's caption states that the training dataset results are shown, and Table 1's totals are 60,000 MNIST and 50,000 CIFAR-10 examples, i.e., the training sets, not the held-out test sets described in Section 4. The same training data is used to form the correct-prediction centroids (Algorithm 1), to define the threshold from the minimum incorrect-prediction distance (Algorithm 2), and to measure accuracy and retention under thresholding. Since the reject rule removes points far from centroids, and misclassified training points are known to be over-represented at large distances, the monotone increase in accuracy as the threshold decreases in Figure 3 is partly a statement about the fitting set, not a predictive statement about deployment. Moreover, Algorithm 2's threshold is never actually reported or tested: the experiments sweep arbitrary thresholds from 0.8 down to 0.05. Without a held-out evaluation at the proposed thresholds, there is no direct evidence that the abstention rule will improve accuracy or catch errors on new data.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a confidence/abstention mechanism for neural network classifiers with softmax outputs. For each class, a centroid is computed as the mean softmax vector of correct training predictions (Algorithm 1), and a per-class threshold is defined as the minimum distance of any incorrect training prediction to that centroid (Algorithm 2). At deployment, predictions whose distance to the predicted-class centroid exceeds the threshold are meant to be labelled 'unknown' and deferred. The authors evaluate the approach on MNIST with a CNN and on CIFAR-10 with a ViT, varying distance thresholds and reporting accuracy, retention, and exclusion rates for in-distribution and 'MNISTified' out-of-distribution datasets. The central claim is that Euclidean distance to a mean softmax centroid is a suitable confidence proxy and can determine when automated predictions should be deferred to human operators.","tokens_in":13558,"tokens_out":4114,"duration_ms":41984,"significance":"If the central claim were established on held-out data, the method would be attractive: it is computationally lightweight, requires no retraining or network modification, and the reported trends are qualitatively consistent across two architectures. The paper also makes a useful exploratory contribution by relating softmax geometry to model accuracy and by testing out-of-distribution data through the MNISTified experiments. However, the significance is currently limited by the absence of held-out evaluation, the circular definition of the 'conservative' threshold, the lack of baselines, and the absence of error bars. The reported accuracy changes are small, and the main experimental figure is explicitly based on training data, so the paper does not yet substantiate its deployment-oriented claims.","major_comments":[{"comment":"The evaluation supporting the main accuracy–retention claim is performed on the training data: Figure 3's caption states that the training dataset results are shown, and Table 1's totals (60,000 for MNIST and 50,000 for CIFAR-10) are the training-set sizes. Because the centroids (Algorithm 1) and thresholds are derived from the same correct/incorrect predictions used to compute accuracy and retention, the monotone improvement in accuracy as the threshold decreases is an in-sample statement and may not transfer to deployment. Please report the same analyses on the held-out test sets (10,000 examples each) and include error bars or confidence intervals from multiple runs or bootstrap resamples.","section":"§4, Figure 3 and Table 1"},{"comment":"The proposed 'conservative' threshold is defined as the minimum distance of incorrect training predictions to the class centroid; 100% accuracy on the training set under this threshold holds by construction, and the paper provides no evidence that this threshold yields any particular accuracy on unseen data. The experiments do not actually use Algorithm 2: Figure 3 and Table 1 sweep thresholds from 0.8 to 0.05, so the threshold rule that the paper proposes is never evaluated. Please evaluate the Algorithm 2 threshold on the held-out test set and report the resulting accuracy and retention; if the threshold is intended only as a heuristic, state this explicitly and justify the range used.","section":"§1, §3, Algorithm 2"},{"comment":"No baselines or statistical error bars are reported. The claim that Euclidean distance to a mean softmax centroid is a suitable confidence proxy needs comparison with standard alternatives—such as maximum softmax probability, predictive entropy, or temperature-scaled softmax—to show that the geometric distance adds information beyond what is already available from the softmax output. The reported accuracy changes are small (MNIST 98.0% to 99.0%, CIFAR-10 99.4% to 99.7%), and without intervals it is unclear whether the effect is reliable or an artifact of the specific threshold grid.","section":"§4, Figure 3"},{"comment":"The paper's own cluster-density discussion states that in 10 dimensions volume concentrates in outer shells and that this creates challenges for clustering and nearest-neighbor calculations, which is in direct tension with the use of a fixed Euclidean distance threshold to a mean centroid as a confidence score. Please provide distributional evidence—for example, per-class histograms or quantile plots of distances for correct versus incorrect predictions—showing that the distance separates the groups in the softmax space, and discuss whether a class-conditional or Mahalanobis distance would be more appropriate than absolute Euclidean distance.","section":"§3, Cluster Density"}],"minor_comments":[{"comment":"The term 'MNISTify' is used without definition; please define the transformation applied to English Handwritten Characters and CIFAR-10 images.","section":"§4"},{"comment":"The notation for the deferred answer is inconsistent ('not known' in the abstract, 'unknown' elsewhere); please choose one term and use it consistently.","section":"Abstract and §1"},{"comment":"In Equation (1), the scaling factor k appears both as the shell index and in r_k = k·r, which is confusing; please use distinct symbols for the shell index and the radius scaling factor.","section":"§3, Eq. (1)"},{"comment":"Algorithm 2 does not specify how the distance to the predicted class centroid is computed before the minimum is taken; please state the exact distance definition and whether it is the distance to the predicted-class centroid or to the true-class centroid.","section":"§3, Algorithm 2"},{"comment":"The axis labels and legend in Figure 3 are very small and the y-axis range (98–100%) makes the accuracy differences visually misleading; please enlarge the figure and add grid lines to improve readability.","section":"Figure 3"},{"comment":"Table 1 has formatting artifacts in the 'At/Above' columns (e.g., '0 .0' and '100 .0'); please fix the spacing and align the numbers.","section":"Table 1"},{"comment":"Reference [10] appears to be a duplicate of reference [9]; please check the reference list and remove redundant entries.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The main experimental figure uses training data, which is a serious issue that should be resolved before further review. I recommend requiring a revised evaluation on held-out test sets, with error bars and at least one baseline, and a direct evaluation of Algorithm 2's threshold. The paper would also benefit from a data/code availability statement, as the algorithms are simple enough to reproduce but the reported experiments are not currently verifiable from the manuscript alone."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nQuick take: the core abstention claim isn't supported as written. The main accuracy-retention evidence appears to come from the training set used to build the centroids and thresholds, not a held-out test set. Figure 3's caption says \"training dataset results are shown,\" and Table 1's totals (60k MNIST, 50k CIFAR-10) match the training splits. The prose calls this \"test set accuracy,\" but the numbers say otherwise. That makes the central result an in-sample fit rather than a predictive finding.\n\nWhat's genuinely useful: the observation that mean softmax vectors of correct predictions form separable class centroids, and that Euclidean distance to those centroids tracks accuracy, is plausible. The OOD experiments with English characters and MNISTified CIFAR-10 show a sensible pattern: OOD points fall farther from centroids. The method is cheap and easy to deploy. Those are real positives.\n\nThe soft spots are significant. No baselines—not even max-softmax probability or entropy, which are the obvious comparators. No error bars or multiple seeds. Algorithm 2 defines a per-class threshold as the minimum distance of incorrect training predictions, but the experiments never use it; they sweep thresholds manually. That threshold would trivially give 100% training accuracy, so it's circular as a claim. The novelty claim that clustering has never been applied to softmax space is unverifiable and likely overstated; the related-work section misses a lot.\n\nIf the authors rerun the evaluation on held-out test data, compare against standard confidence baselines, report variance, and actually test their proposed threshold, the core idea could stand. As it is, the evidence doesn't support the deployment claim.\n\nThis deserves a serious referee—the question is valid and the method is simple enough to evaluate properly—but I'd send it back for major revision, not accept it in current form.\n\nBest,\n[You]","headline":"The abstention claim is evaluated on the training data used to derive centroids and thresholds, so the central result is an in-sample fit; the idea is plausible but needs proper held-out evaluation and baselines.","tokens_in":14077,"tokens_out":5113,"would_cite":false,"duration_ms":49201,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that the mean softmax vector of a network's correct predictions can serve as a class centroid, and that Euclidean distance from a new prediction to that centroid is a reliable, training-free confidence score for deciding…","keywords":["uncertainty quantification","softmax probability space","distribution shift","out-of-distribution detection","confidence estimation","abstention","K-means clustering","not-known answers"],"falsifier":"Train a classifier where one class contains two visually distinct subpopulations, compute the per-class mean centroid and the minimum-incorrect-distance threshold, and measure whether correct predictions from either subpopulation fall beyond the threshold more often than incorrect predictions do; if a substantial share of correct near-mode examples are rejected while incorrect examples are accepted, the central proxy fails.","tokens_in":13141,"feed_emoji":"🎯","tokens_out":7428,"duration_ms":61905,"temperature":0.7,"pith_summary":"This paper tries to establish that a network's own softmax outputs contain enough information to say when the network should not answer. The proposal is simple: for each class, average the softmax vectors of all training examples the network got right, and treat that average as the class centroid. Then measure the Euclidean distance from any new prediction's softmax vector to the centroid of the predicted class. If that distance exceeds a per-class threshold, chosen as the smallest distance seen for a wrong prediction on the training set, the system answers 'not known' instead of committing to a label. If this works, any trained softmax classifier gains an abstention option with no retraining and no extra architecture.","feed_headline":"Softmax distance tells when to say 'I don't know'","feed_subtitle":"No retraining or extra architecture: distances to class centroids flag low-confidence predictions for human review.","key_machinery":"The central object is the class centroid, computed by Algorithm 1 as the arithmetic mean of all correct-prediction softmax vectors for a given class. The companion object is the threshold, computed by Algorithm 2 as the minimum distance from any incorrect training prediction to the centroid of the class it was assigned to. The method works by treating the K-dimensional softmax output as a point in the probability simplex, where all centroids are valid probability distributions and pairwise distances are bounded by $\\sqrt{2}$; Euclidean distance to the predicted-class centroid then functions as the confidence score, and the threshold creates a 'not known' region. The paper also uses shell-density computations in these high-dimensional spaces to characterize how in-distribution points concentrate near centroids while out-of-distribution points populate outer shells.","core_discovery":"The paper's central claim is that the mean of the softmax probability vectors for all correct predictions of a given class forms a usable centroid for that class in softmax space, and that Euclidean distance from a new prediction's softmax vector to that centroid tracks the network's confidence. Defining a class threshold as the smallest distance from an incorrect training prediction to the class centroid gives a conservative rule: predictions beyond the threshold are rejected and the network returns 'not known'. Across MNIST with a CNN and CIFAR-10 with a Vision Transformer, the authors find that correct predictions cluster tightly near their centroids, centroids are nearly maximally separated in the probability simplex (pairwise distances near $\\sqrt{2}$), and out-of-distribution inputs—English handwritten characters and CIFAR-10 images fed to the MNIST model—fall farther from centroids, producing high exclusion rates at small thresholds. The authors conclude that this distance metric is a consistent, lightweight proxy for confidence and a practical basis for deferring low-confidence decisions to human operators.","pith_inferences":["A natural stress test the paper does not run is multimodal classes: if one class is produced by two visually distinct subpopulations, the arithmetic-mean centroid could sit between the modes and the distance rule could reject confident correct predictions from either mode; a testable extension is to fit one centroid per mode and compare abstention quality.","Because the threshold is calibrated on the training set's wrong predictions, it inherits the network's blind spots: if training errors are few or unrepresentative, the threshold can be too tight or too loose, and per-class thresholds tuned on a held-out validation set would likely be a safer deployment recipe.","The near-$\\sqrt{2}$ centroid separation suggests the method's usefulness depends on the network being well trained and reasonably calibrated; for poorly calibrated networks, post-hoc recalibration before computing centroids could change results.","The method's 'not known' output is not a calibrated probability, so in safety-critical pipelines it would likely serve as a cheap first-stage filter rather than a complete uncertainty quantification."],"forward_implications":["Any softmax classifier can gain an abstention option by computing per-class centroids and thresholds from training-set predictions, with no retraining or architectural changes.","Decreasing the distance threshold trades retention against accuracy: on the reported settings, MNIST accuracy rises from about 98% at threshold 0.8 toward 99% at 0.05 while an additional 8% of correct predictions are rejected, and CIFAR-10/ViT shows even tighter clusters.","Inputs from a genuinely different distribution are more likely to land beyond threshold: at threshold 0.05, MNISTified CIFAR-10 rejects 97% of examples while in-distribution CIFAR-10 rejects 1.2%.","The threshold can be set from safety requirements: picking the minimum incorrect-prediction distance gives a conservative rule, while larger or smaller thresholds let an operator choose how much retention to sacrifice."],"supporting_citations":[{"why":"Supplies the K-means algorithm whose centroid-and-distance geometry the proposed confidence method reuses.","marker":"[27]"},{"why":"Provides the Vision Transformer architecture used for the CIFAR-10 half of the empirical evaluation.","marker":"[7]"},{"why":"Supplies the English handwritten character dataset used as out-of-distribution data for the MNIST-trained CNN.","marker":"[3]"},{"why":"Defines the misclassified and out-of-distribution detection task that the distance-threshold abstention rule addresses.","marker":"[21]"},{"why":"Provides the ImageNet-21k pretraining data used to initialize the ViT before fine-tuning on CIFAR-10.","marker":"[6]"}],"fun_headline_variants":["Softmax centroids reveal when AI is guessing","Neural networks learn to say 'I don't know' using softmax distances","Distance in softmax space flags low-confidence predictions","Class centroids in softmax space say when network is unsure"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that each class's softmax outputs cluster tightly and unimodally around their mean, so the arithmetic mean of correct predictions is a representative centroid and Euclidean distance to it is a trustworthy confidence score; the paper does not provide distributional evidence for this across classes.","fun_headline_variants_meta":{"raw":{"variants":["Softmax centroids reveal when AI is guessing","Neural networks learn to say 'I don't know' using softmax distances","Distance in softmax space flags low-confidence predictions","Class centroids in softmax space say when network is unsure"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00068,"raw_usage":{"total_tokens":3115,"prompt_tokens":998,"completion_tokens":2117,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":614,"completion_tokens_details":{"reasoning_tokens":2048}},"tokens_in":614,"tokens_out":2117,"duration_ms":20319,"temperature":1.0,"reasoning_tokens":2048,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T18:55:57.814771+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a classifier where one class contains two visually distinct subpopulations, compute the per-class mean centroid and the minimum-incorrect-distance threshold, and measure whether correct predictions from either subpopulation fall beyond the threshold more often than incorrect predictions do; if a substantial share of correct near-mode examples are rejected while incorrect examples are accepted, the central proxy fails.","supporting_citations":[{"cited_title":"In: Proceedings of the International Conference on Computer Vision Theory and Applications, Lisbon, Portugal (February 2009)","cited_arxiv_id":null,"evidence_quote":"Supplies the English handwritten character dataset used as out-of-distribution data for the MNIST-trained CNN."}],"review_version":1}