{"id":"c89da55f-4068-435f-bf23-67dd3bb75d1d","arxiv_id":"2508.00754","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A single-model OOD detection method that measures feature-space density with a Gaussian kernel (IPF) reports AUROC 93.18 on CIFAR-10 vs SVHN, a marginal gain over DDU's 92.90, with the kernel width selected to maximize AUROC.","lead":"This paper proposes using a kernel density estimate, called the information potential field, on neural network features to detect out-of-distribution samples with a single deterministic model. Tests on CIFAR-10 versus SVHN show a small AUROC gain over existing single-model methods, but the reported gain is partly driven by tuning the kernel width on the test set.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported AUROC is inflated because kernel width h is selected to maximize AUROC on the OOD test set; without this tuning, the claimed superiority over DDU is unsupported.","rationale":"The reader's weakest_assumption focuses on whether a 640-dimensional isotropic Gaussian KDE faithfully approximates the feature density. That is a legitimate concern, and the paper's own Section IV states Parzen estimation 'should not be used above 20 dimensions.' However, the more immediate and decisive problem is the evaluation protocol: the kernel width h is explicitly selected by maximizing AUROC on the test OOD distribution. This is a form of test-set leakage that inflates the reported AUROC and invalidates the direct comparison with DDU, which uses a fixed Gaussian mixture model without such tuning. The two concerns are related: if the KDE in 640 dimensions is not a reliable density estimator, then the AUROC ranking is driven by the specific choice of h, making the tuning issue even more damaging. I agree with the reader's REJECT verdict because the central claim of outperformance rests on a single, methodologically flawed comparison. The reader did mention h tuning in the rationale, but the stated weakest_assumption was the KDE fidelity, so my agreement is partial. The proposed concrete test is straightforward and would settle whether the claimed AUROC advantage persists when h is chosen honestly.","tokens_in":9323,"tokens_out":2816,"duration_ms":36918,"concrete_test":"Fix h before seeing any SVHN data, for example by choosing h via Silverman's rule on the CIFAR-10 training features or via a validation split drawn only from CIFAR-10, then compute AUROC on the full SVHN set. Alternatively, hold out a random half of SVHN exclusively for tuning h and report AUROC on the other half. If the resulting AUROC drops to within the reported standard deviation of DDU (92.90 ± 0.016) or below, the claimed superiority is an artifact of test-set tuning. As a secondary check, report the AUROC over the full h grid [0.01, 1]; if no value chosen without OOD labels reaches 93.18, the central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is empirical: IPF outperforms baselines, with Table I showing AUROC 93.18 for IPF versus 92.90 for DDU. The only free parameter, h, is chosen in Section III-C by 'performing cross-validation over the range [0.01, 1], selecting the value that maximized the AUROC score.' Since AUROC is computed on the CIFAR-10/SVHN test pair, this amounts to fitting h directly to the test OOD distribution. The reported 93.18 is therefore an optimistic, in-sample number, not an honest estimate of OOD performance. The acknowledged limitation that Parzen estimation should not be used above 20 dimensions (Section IV) makes the result even less secure: the isotropic Gaussian KDE is applied in a 640-dimensional feature space, and the same h is used across all 50,000 training points. Because the margin over DDU is only 0.28 AUROC and no code is released, the observed gap could easily be an artifact of test-set-tuned h rather than a genuine methodological advantage. For the central claim to hold, the reported AUROC must survive selection of h without access to OOD labels.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Information Potential Field (IPF), a kernel density estimate over the penultimate-layer features of a single deterministic neural network, as a method for uncertainty quantification and out-of-distribution (OOD) detection. The method is evaluated on two synthetic 2D datasets (Two Moons, Three Spirals) with visual uncertainty maps, and on CIFAR-10 vs. SVHN OOD detection using a WideResNet-28-10, reporting an AUROC of 93.18 for IPF versus 92.90 for DDU. The authors claim that IPF outperforms baseline methods while being simpler and requiring only a single forward pass.","tokens_in":9544,"tokens_out":3184,"duration_ms":39694,"significance":"If the reported result were robust, the paper would offer a simple, appealing alternative to class-conditional Gaussian density estimation (DDU) and distance-based methods, with no additional training cost at inference. The synthetic experiments are visually clear and illustrate the method's behavior in low dimensions. The paper also honestly identifies the main limitation of Parzen estimation in high dimensions and points to more advanced estimators as future work. However, the empirical case for the central claim is weakened by how the kernel-width hyperparameter is selected and by the application of the method in a 640-dimensional feature space, which the paper itself says is beyond the reliable regime for Parzen estimation. These issues currently limit the significance of the contribution.","major_comments":[{"comment":"The kernel width h is selected by \"performing cross-validation over the range [0.01, 1], selecting the value that maximized the AUROC score.\" Because AUROC is computed on the CIFAR-10 test set versus the SVHN OOD set, this procedure fits h directly to the test OOD pair. The reported AUROC of 93.18 is therefore an in-sample, optimistic estimate rather than an independent performance measure. Since the margin over DDU is only 0.28 AUROC, the central claim that IPF outperforms baselines is not supported unless h is selected on a separate validation set (or via nested cross-validation) and the AUROC is then reported on a held-out test pair.","section":"Section III-C, Table I"},{"comment":"The paper states that \"Parzen estimation does not scale well, and in practice it should not be used above 20 dimensions,\" yet the method applies an isotropic Gaussian KDE in the 640-dimensional feature space of WideResNet-28-10. This contradiction undermines the reliability of the density estimates underlying the reported AUROC. The claim that \"we were surprised with the quality of the Parzen estimator\" is not evidence. The authors should provide a sanity check of the density estimates in this setting, for example by comparing ID and OOD density score distributions, testing sensitivity to h across a wider range, or comparing against a higher-dimensional density estimator, before the AUROC numbers can be interpreted as evidence of a sound method.","section":"Section IV and Section III-C"},{"comment":"The OOD evaluation consists of a single dataset pair (CIFAR-10 as ID, SVHN as OOD) with no significance test and no variation of the in-distribution dataset or model architecture. The abstract and discussion claim that \"the results demonstrate that our method outperforms baseline models\" and that IPF \"can improve state of the art results for OOD detection,\" but a single pair with a 0.28 AUROC difference and overlapping standard deviations (93.18 ± 0.006 versus 92.90 ± 0.016) is insufficient to support a general superiority claim. The authors should either broaden the evaluation (e.g., multiple OOD datasets, multiple architectures) or temper the claims accordingly.","section":"Section III-C, Table I"}],"minor_comments":[{"comment":"Equation (3) omits the normalization constant of the Gaussian kernel, so ψ(z) in Eq. (2) is not a probability density but an unnormalized kernel sum. Since AUROC is rank-based this does not affect the reported metric, but the text should avoid calling ψ(z) an estimate of p(z) without noting the missing constant.","section":"Section II-C, Eq. (3)"},{"comment":"There is a typo in \"to improve the quality of the the features\" in the paragraph after Eq. (1); \"the the\" should be \"the.\"","section":"Section II-B"},{"comment":"The choice of the best kernel size (0.3) for the synthetic experiments is reported without a selection criterion; stating whether this was chosen visually or by a quantitative rule would improve reproducibility.","section":"Section III-B"},{"comment":"The phrase \"willl be pursued\" contains a typo; it should read \"will be pursued.\"","section":"Section IV"}],"recommendation":"major_revision","confidential_remarks":"The central claim is plausible and the method is simple, but the evaluation as presented is not sufficient to support it. The test-set-based selection of h is the most serious issue; if the authors can re-run the evaluation with a held-out validation set for h selection and provide evidence that the 640-dimensional KDE is not pathological, the paper could become a useful contribution. I would encourage the editor to request a revised version rather than reject outright, as the issues are fixable within the scope of the manuscript."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a clean but minor paper. The method is Gaussian KDE on penultimate-layer features, renamed \"information potential field.\" It is a known idea; DDU already estimates feature density (with a GMM), and Density-softmax applies kernel density ideas. What the paper does well is the 2D synthetic demonstration: the uncertainty maps are clear and show the method behaves sensibly, and spectral normalization visibly helps. The exposition is straightforward and the authors are candid about Parzen estimation's limitations.\n\nThe soft spots are serious, though. The main empirical claim—that IPF outperforms baselines—rests on a single CIFAR-10 vs SVHN comparison. The kernel width h is chosen to maximize AUROC on that same test pair, so the reported 93.18 is an in-sample number, not an independent estimate. The gain over DDU (92.90) is 0.28 AUROC, no significance test, likely within noise. And the paper uses 640-dimensional features while citing that Parzen estimation should not be used above 20 dimensions. That contradiction does not directly invalidate the method—the KDE might still give useful scores—but it means the reported superiority is not trustworthy.\n\nThe paper is not a total loss. For a reader new to density-based deterministic UQ, it is a readable illustration of how KDE on features can work, and the synthetic figures are useful for teaching. But as a research contribution, the evidence is too thin. The authors should select h on a validation split without OOD labels, test on multiple OOD pairs (or use a proper OOD benchmark), and report confidence intervals. If they do that, the paper could become a reasonable short paper or baseline study.\n\nI would not send this to a top venue as is. It deserves a workshop-level review at most, and honestly a desk reject at a journal would be defensible. I would tell the authors to fix the evaluation and resubmit.","headline":"A clean but minor paper: KDE on features, with a promising 2D demo but a headline AUROC claim that is not credible because h is tuned on the test set.","tokens_in":10089,"tokens_out":3478,"would_cite":false,"duration_ms":40620,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper's central claim is that an information potential field over a single network's feature vectors — a nonparametric density estimate — detects out-of-distribution images more accurately than existing baselines, while keeping…","keywords":["uncertainty quantification","out-of-distribution detection","kernel density estimation","information potential field","feature-space density","spectral normalization","distributional shift"],"falsifier":"Re-run the CIFAR-10/SVHN experiment with the kernel width chosen by Silverman's rule or by cross-validation on a held-out portion of the training set rather than by maximizing AUROC on the test set, and also try a second OOD set such as LSUN or Texture. If IPF no longer beats DDU and DUQ under honest bandwidth selection or generalizes to the second OOD set, the reported AUROC is an artifact of tuning rather than evidence that the density field is faithful.","tokens_in":9090,"feed_emoji":"🎯","tokens_out":8689,"duration_ms":93235,"temperature":0.7,"pith_summary":"Bayesian and ensemble methods give uncertainty quantification but cost many forward passes and large storage. This paper proposes a deterministic alternative: train one network with spectral normalization, then treat the density of its top-layer feature vectors as the uncertainty signal, computed with a kernel-density field called the information potential field (IPF). A test point landing in a low-density region of that field is labeled out-of-distribution. On CIFAR-10 as in-distribution and SVHN as out-of-distribution, the paper reports AUROC 93.18 with WideResNet-28-10, slightly above DDU's 92.90 and DUQ's 92.43. The contribution is showing that this simple Parzen-style density estimate can match or beat more elaborate single-model uncertainty methods without per-class distributional assumptions.","feed_headline":"Kernel density field beats single-model OOD detectors on CIFAR-10","feed_subtitle":"Information potential field from feature activations reaches AUROC 93.18 on CIFAR-10 vs SVHN, beating DDU and DUQ.","key_machinery":"The information potential field (IPF) is the central object: a sum of isotropic Gaussian kernels centered at the training feature vectors, $\\psi(z)=\\frac{1}{N}\\sum_i \\exp(-\\|z-z_i\\|^2/2h^2)$, used as a nonparametric stand-in for the feature density $p(z)$. It is computed once after training, and at test time the scalar field value at $z^*$ is the OOD score. Its behavior is governed by the kernel width $h$, which the paper selects by cross-validation, and its usefulness depends on the distance-preserving feature map produced by spectral normalization.","core_discovery":"The central discovery is that a nonparametric density estimate of the feature space of a single deterministic classifier is enough to separate in-distribution from out-of-distribution inputs. The method defines $\\psi(z)=\\frac{1}{N}\\sum_{i=1}^N G(z-z_i)$, where $G$ is an isotropic Gaussian kernel and the $z_i$ are the training-set feature embeddings from the layer before the classification head; low $\\psi(z^*)$ for a test point $z^*$ means high uncertainty and an OOD decision. Spectral normalization is used during training to enforce a bi-Lipschitz constraint on the feature map, so distinct inputs do not collapse to the same feature and the density field remains meaningful. The paper claims this yields AUROC 93.18 on CIFAR-10 vs SVHN, above softmax, ensembles, DUQ, and DDU, and that the method is simpler because it models the whole feature space rather than one Gaussian per class.","pith_inferences":["The paper does not report how the 640-dimensional kernel width was validated beyond maximizing AUROC on the test distribution; an independent evaluation with bandwidth chosen by Silverman's rule or by an internal train/validation split would show whether the reported margin over DDU is robust or an artifact of tuning.","Because the paper itself notes that Parzen estimation is not recommended above 20 dimensions, the strong result in 640 dimensions is surprising; replacing the isotropic Gaussian with a dimension-adaptive or infinitely divisible kernel density estimator, which the paper names as future work, is a direct test of whether the IPF principle or the simple kernel is what carries the performance.","The IPF assigns a scalar uncertainty to every input without retraining, so the same score could in principle be used for active-learning acquisition and for OOD detection on other benchmarks; neither application is tested here."],"forward_implications":["Out-of-distribution detection becomes a single forward pass plus density evaluation, so the computational and storage costs of ensembles and Bayesian approximations are avoided.","The density estimate makes no per-class Gaussian assumption, so it applies to feature spaces with overlapping or non-Gaussian class structure where DDU's mixture model is misspecified.","The kernel width $h$ acts as a single sensitivity knob, letting practitioners trade a tighter uncertainty region for broader tolerance.","For low-dimensional inputs, the same field can be evaluated directly in the data space, enabling fast OOD checks without any feature extractor."],"supporting_citations":[{"why":"Introduces the information potential field and the Parzen density estimator on which the method is built; also the source of the 20-dimension scaling caveat cited in the Discussion.","marker":"[37]"},{"why":"Supplies the spectral normalization / bi-Lipschitz constraint that keeps the feature map distance-aware and prevents feature collapse.","marker":"[30]"},{"why":"Defines the DDU baseline, whose per-class Gaussian assumption the IPF method is designed to avoid.","marker":"[31]"},{"why":"Defines the DUQ baseline, a single-model distance-based uncertainty method compared with IPF.","marker":"[29]"},{"why":"Defines the deep-ensemble baseline, the storage-intensive approach the paper seeks to replace.","marker":"[23]"},{"why":"Gives the Silverman rule-of-thumb for choosing the kernel width, the alternative to cross-validation mentioned in the method.","marker":"[40]"},{"why":"Provides the WideResNet-28-10 architecture used in the CIFAR-10/SVHN experiments.","marker":"[43]"},{"why":"Supplies the CIFAR-10 dataset used as in-distribution data in the OOD benchmark.","marker":"[38]"},{"why":"Supplies the SVHN dataset used as out-of-distribution data in the benchmark.","marker":"[39]"}],"fun_headline_variants":["Single deterministic model's feature space density detects OOD","One net's density estimate beats ensembles for uncertainty","Kernel density on feature embeddings tops OOD baselines","Simple density method achieves 93.18 AUROC on CIFAR-10 OOD","Feature-space density from one model beats multi-model OOD"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole comparison rests on the assumption that one Gaussian kernel width, applied in a 640-dimensional feature space, gives an honest estimate of where the training data live; the paper itself cautions that this style of density estimation is not expected to work above 20 dimensions.","fun_headline_variants_meta":{"raw":{"variants":["Single deterministic model's feature space density detects OOD","One net's density estimate beats ensembles for uncertainty","Kernel density on feature embeddings tops OOD baselines","Simple density method achieves 93.18 AUROC on CIFAR-10 OOD","Feature-space density from one model beats multi-model OOD"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001446,"raw_usage":{"total_tokens":5799,"prompt_tokens":891,"completion_tokens":4908,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":507,"completion_tokens_details":{"reasoning_tokens":4823}},"tokens_in":507,"tokens_out":4908,"duration_ms":41655,"temperature":1.0,"reasoning_tokens":4823,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T05:56:39.871880+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the CIFAR-10/SVHN experiment with the kernel width chosen by Silverman's rule or by cross-validation on a held-out portion of the training set rather than by maximizing AUROC on the test set, and also try a second OOD set such as LSUN or Texture. If IPF no longer beats DDU and DUQ under honest bandwidth selection or generalizes to the second OOD set, the reported AUROC is an artifact of tuning rather than evidence that the density field is faithful.","supporting_citations":[{"cited_title":"Reading digits in natural images with unsupervised feature learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the SVHN dataset used as out-of-distribution data in the benchmark."},{"cited_title":"Learning multiple layers of features from tiny images,","cited_arxiv_id":null,"evidence_quote":"Supplies the CIFAR-10 dataset used as in-distribution data in the OOD benchmark."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the information potential field and the Parzen density estimator on which the method is built; also the source of the 20-dimension scaling caveat cited in the Discussion."},{"cited_title":"Simple and principled uncertainty estimation with deterministic deep learning via distance awareness,","cited_arxiv_id":null,"evidence_quote":"Supplies the spectral normalization / bi-Lipschitz constraint that keeps the feature map distance-aware and prevents feature collapse."},{"cited_title":"Deep deterministic uncertainty: A new simple baseline,","cited_arxiv_id":null,"evidence_quote":"Defines the DDU baseline, whose per-class Gaussian assumption the IPF method is designed to avoid."},{"cited_title":"Uncertainty estimation using a single deep deterministic neural network,","cited_arxiv_id":null,"evidence_quote":"Defines the DUQ baseline, a single-model distance-based uncertainty method compared with IPF."},{"cited_title":"Simple and scalable predictive uncertainty estimation using deep ensembles,","cited_arxiv_id":null,"evidence_quote":"Defines the deep-ensemble baseline, the storage-intensive approach the paper seeks to replace."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the Silverman rule-of-thumb for choosing the kernel width, the alternative to cross-validation mentioned in the method."}],"review_version":1}