{"id":"e3b4fcb8-5454-42cf-979b-80c5aafb0371","arxiv_id":"2501.16718","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"HamOS generates diverse virtual outliers via Hamiltonian Monte Carlo in hyperspherical feature space, improving OOD detection FPR95 on CIFAR-10/100 and ImageNet-1K without natural outlier data.","lead":"This paper introduces HamOS, a method that synthesizes fake out-of-distribution examples by sampling with Hamiltonian Monte Carlo in the feature space of a neural network, using only in-distribution data. It reports large improvements in OOD detection accuracy on CIFAR and ImageNet benchmarks, potentially reducing the need for collecting real outlier data.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Hard/near-OOD coverage is the load-bearing weakness: HamOS samples only between class clusters, so OOD near a single cluster is underrepresented; Table 7 shows HamOS is not SOTA on the CIFAR-100 hard benchmark.","rationale":"The reader identified the between-cluster assumption as the weakest assumption, and my read agrees. I sharpen it with the paper's own Table 7 evidence: on the CIFAR-100 hard OOD benchmark, HamOS does not outperform SSD+ or CIDER, which is exactly the near-OOD regime that the sampling geometry is least equipped to cover. The concern is not about internal consistency—the HMC derivation and gradient are coherent—but about external validity of 'representative' when the synthesis support is structurally biased toward inter-cluster regions. The proposed check would settle whether hard OOD samples actually fall outside the synthesized outlier support. Since the reader already returned CONDITIONAL and my concern reinforces the need for qualification rather than overturning the core method, the verdict remains unchanged. The paper's released code and extensive ablations are real strengths; the weakness is the scope of the representativeness claim, not the mechanism itself.","tokens_in":50506,"tokens_out":8821,"duration_ms":92580,"concrete_test":"On CIFAR-100 as ID, compute the Eq. (1) OOD-ness distance for each sample in the hard OOD test sets (CIFAR-10 and Tiny-ImageNet) and compare this distribution with the OOD-ness distances of the synthesized outliers used in HamOS training. If the hard OOD samples predominantly have distances below the minimum synthesized-outlier distance, then the between-cluster sampler does not cover the near-OOD region, directly falsifying the 'representative outliers' claim for that regime. For a causal check, retrain HamOS with additional synthesis chains initialized near individual cluster boundaries and verify whether hard-OOD FPR95 improves beyond the 62.32 reported in Table 7.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that HamOS synthesizes 'diverse and representative outliers' rests on the assumption that the most useful OOD supervision lies between ID clusters (Eqs. 1-2). This assumption is load-bearing because the OOD-ness potential is defined as the average Euclidean distance to the k-th nearest neighbor in each of two classes, and HMC is initialized at cluster midpoints and driven by gradients that push away from both clusters (Eq. 4). Consequently, the support of the synthesized outlier distribution is concentrated in the interstitial regions between class clusters. Real OOD samples that are semantically similar to a single ID class—the 'hard OOD' regime—can lie close to one cluster boundary and have low OOD-ness under Eq. 1; they are therefore not represented by the synthesized distribution, and the hard margin in Eq. 5 only blocks points with high ID probability, not points just outside a single cluster. The paper's own hard-benchmark results support this concern: in Table 7 with CIFAR-100 as ID, HamOS achieves averaged FPR95 of 62.32, worse than SSD+ (58.80) and CIDER (60.62), while it is best on the standard five-dataset benchmark. This is not an internal inconsistency, but it shows the 'representative' and 'state-of-the-art' claims are benchmark-dependent and likely fail for near-distribution OOD. The mechanism works for between-cluster OOD, but generalizability to near-OOD is unestablished.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes HamOS, a method for out-of-distribution (OOD) detection that synthesizes virtual outliers in the hyperspherical feature space of a pretrained classifier using only in-distribution data. OOD-ness of a candidate point is defined (Eqs. 1–2) as the average Euclidean distance to the k-th nearest neighbor in each of two ID classes; spherical Hamiltonian Monte Carlo chains are initialized at the midpoints of nearby class prototypes and evolved by Leapfrog updates driven by the gradient of the potential (Eq. 4), with a hard margin (Eq. 5) that rejects proposals whose estimated von Mises-Fisher KDE ID probability is too high. The collected trajectory points serve as virtual outliers in an OOD-discernment loss (Eq. 7), jointly optimized with the CE loss and a CIDER-style ID contrastive loss (Eq. 8); inference uses the KNN distance to the ID embedding set (Section 3.4). The main experiments (Tables 1–2) report averaged FPR95 of 10.48% on CIFAR-10 and 46.68% on CIFAR-100 over five OOD test sets, and 44.59% on ImageNet-1K, surpassing the compared baselines on the standard benchmarks; the appendix contains ablations over sampling algorithms, scoring functions, contrastive losses, feature dimensions, and all hyperparameters, plus results on hard OOD benchmarks and from-scratch training.","tokens_in":50840,"tokens_out":13776,"duration_ms":122217,"significance":"The paper's strengths are empirical and reproducible: public code, multi-run means with standard deviations throughout, precise algorithm specifications (Algorithms 1–2), and unusually complete ablations covering sampling algorithms, scoring functions, contrastive losses, feature dimensions, and every free hyperparameter (Tables 9–24). If correct, the central result is genuinely useful: feature-space outlier synthesis is not restricted to Gaussian perturbations around boundary anchors, and gradient-guided exploration of the hypersphere provides cheap, effective OOD supervision without any natural outlier data. Table 9 offers a clean decomposition showing that the gain over random-walk exploration (CIFAR-100 FPR95 50.05 vs 46.68) is attributable to the gradient guidance rather than to the Markov-chain machinery itself.","major_comments":[{"comment":"The construction confines the support of the synthesized outliers to the regions between pairs of ID clusters: chains are initialized at cluster midpoints, the potential in Eq. (2) averages the distances to two clusters, and the gradient in Eq. (4) pushes points away from both clusters simultaneously, while the hard margin in Eq. (5) only blocks points that enter the bulk of the ID distribution. OOD inputs that resemble a single ID class (the hard/near-OOD regime) can therefore have low OOD-ness under Eq. (1) and are underrepresented in the synthesized distribution. This mechanism is consistent with the paper's own Table 7: with CIFAR-100 as ID, HamOS obtains averaged FPR95 of 62.32, behind SSD+ (58.80) and CIDER (60.62), although it is best on the standard five-dataset benchmark (Table 1) and does win Table 6 with CIFAR-10 as ID. The abstract and Section 3.1 claims that the framework generates 'diverse and representative outliers' exposing the model to 'miscellaneous potential OOD scenarios' should be qualified to the between-cluster regime, and the unconditional SOTA phrasing in Section 4.2 should carry the Table 7 caveat; the hedged wording in Appendix F.1 is welcome but the main text and abstract overstate the generality.","section":"Section 3.2 (Eqs. 1–2, Eq. 5); Appendix F.1 (Tables 6–7)"},{"comment":"The text states that 'the virtual outliers collected along the Markov chains obey the marginal distribution ∝ P_OOD', but this does not hold for the implemented procedure. First, the same paragraph explicitly disclaims convergence and the method runs only R=5 rounds from cluster midpoints, collecting trajectory samples rather than stationary samples from the target. Second, the hard-margin filter in Eq. (5) and Algorithm 1 line 19 is applied on top of the Metropolis acceptance probability, which breaks detailed balance; the accepted samples are not drawn from exp(−U_OOD). Third, the gradient in Eq. (4) treats the k-th nearest neighbors as constants although they depend on the current position z, and the potential in Eq. (1) is non-smooth where the k-th neighbor index changes, so the Leapfrog updates do not integrate the exact Hamiltonian defined by Eq. (2). None of this invalidates the empirical gains, which the ablation against Random Walk in Table 9 shows come from gradient-guided exploration, but the theoretical framing should be corrected: the paper should describe the chain outputs as guided exploration trajectories and drop or qualify the target-distribution claim.","section":"Section 3.2 (Eqs. 2–5); Algorithm 1 (lines 17–19)"}],"minor_comments":[{"comment":"Typos: 'establishs' (contribution list), 'demonstates' (Section 1), 'Temporature' (Appendix E.1), 'desnity' (Appendix G), 'verying' (text near Table 19); the manuscript would benefit from a proofread.","section":"Throughout"},{"comment":"Terminology: Eq. (1) defines P_OOD as a Euclidean distance, not a probability; the text variously calls it 'OOD-ness density', 'OOD-ness probability level', and 'likelihood' (Sections 2.2 and 3.2). Consistent terminology such as 'OOD-ness score' would avoid the implication that the quantity is normalized.","section":"Section 3.2 (Eqs. 1–2)"},{"comment":"The abstract and Sections 2.2 and 3.2 claim an acceptance rate 'almost close to 1', but no empirical acceptance rate is reported anywhere, and the hard-margin filter necessarily rejects some proposals; Table 9 reports only synthesis time. Reporting the measured acceptance rate per configuration would substantiate the claim.","section":"Abstract; Sections 2.2 and 3.2; Table 9"},{"comment":"The inference-time KNN scorer uses k=50 (Appendix E.1) whereas the synthesis potential uses k=200 (Table 4); a sentence on the choice and sensitivity of the inference-time k would improve reproducibility.","section":"Section 3.4; Appendix E.1; Table 4"},{"comment":"The synthesis potential (Eqs. 1–2) and the inference score (Section 3.4) both use KNN distance, which is a mild self-reference; the consistent gains under Mahalanobis distance (Table 10) and under the MSP/EBO/ASH/Scale/Relation scorers (Table 11) largely mitigate the concern, and this mitigation should be stated explicitly in the main text.","section":"Section 3.4; Tables 10–11"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a strong empirical systems paper whose main risk is over-citation as unconditional SOTA in follow-up work without the hard-benchmark caveat; a journal version should make the near-OOD limitation prominent. The contribution over VOS/NPOS is real but incremental (replacing Gaussian perturbation with gradient-guided Markov-chain exploration and a pairwise-class potential). The 'first to investigate Markov-chain outlier synthesis' claim is somewhat strong, though the specific application framing appears new. Scope-wise the paper fits an applied ML and representation-learning venue; its theoretical apparatus is a heuristic framing rather than a convergence analysis, so it should not be evaluated as a theory contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, what's actually new: HamOS is the first application of Hamiltonian Monte Carlo — or any Markov chain sampler — to virtual outlier synthesis in feature space. The mechanism is simple and clean. Define OOD-ness as the average kNN distance from a point to two ID class clusters, start chains at the midpoints between clusters on the hypersphere, and collect trajectory points. Against Gaussian-perturbation baselines like VOS and NPOS the diversity gain shows in the numbers: FPR95 of 10.48 on CIFAR-10 and 46.68 on CIFAR-100 averaged over five OOD sets, well ahead of NPOS, with ID accuracy essentially preserved. The experiments are the best part — ablations cover k, step size, leapfrog steps, hard margin, rounds, cluster count, feature dimension, contrastive losses, scoring functions, and alternative samplers. Code is public; CIFAR tables have multiple runs with standard deviations. I buy the central claim.\n\nNow the soft spots, in order of weight. Read Table 7 first: with CIFAR-100 as ID on the hard benchmark, HamOS is at 62.32 averaged FPR95, behind SSD+ (58.80) and CIDER (60.62). That is not a benchmark accident; it follows from the geometry. Chains start at midpoints and the potential repels from both clusters, so the synthesized distribution is concentrated between classes. Hard OOD close to a single ID class is exactly what this sampler does not produce, and the hard-margin filter only blocks high-ID-probability points; it does not add near-cluster coverage. 'Diverse and representative' is therefore true in a restricted sense — diverse across the between-cluster region, not representative of near-distribution OOD. The appendix honestly concedes no method wins that benchmark outright, but the abstract and intro do not carry the caveat.\n\nSecond, the MCMC framing is looser than it looks. The hard margin breaks detailed balance, the paper itself says convergence to the target is not required, and the abstract's 'acceptance rate almost close to 1' is never measured. What you actually have is gradient-guided trajectory exploration with an ID-likelihood filter — a perfectly good engineering recipe — but the theory section dresses it as principled sampling from the OOD-ness target. That framing should be rewritten. Minor: Table 2 has no error bars on ImageNet-1K, and 'paradigm shift' is overwrought.\n\nWho this is for: anyone in OOD detection or virtual outlier synthesis; HamOS becomes the baseline to beat on the standard CIFAR benchmarks. Does it deserve a serious referee? Yes — this is a solid, well-executed method paper, and an editor should send it to review rather than desk-reject. My recommendation for the review itself: accept after revisions that scope the 'diverse and representative' claim to between-cluster OOD, report the hard benchmark prominently, and make the acceptance-filter/MCMC language honest. Worth citing, worth having in the literature.","headline":"HamOS genuinely does something new — Hamiltonian Monte Carlo for virtual outlier synthesis — and the standard-benchmark gains hold up, but its own hard-OOD table shows the 'diverse and representative' claim is only true between clusters.","tokens_in":51366,"tokens_out":6950,"would_cite":true,"duration_ms":56142,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"HamOS generates diverse virtual outliers from in-distribution data alone by running Hamiltonian Monte Carlo chains in the hyperspherical feature space, and reports state-of-the-art OOD detection on CIFAR-10, CIFAR-100, and ImageNet-1K.","keywords":["out-of-distribution detection","virtual outlier synthesis","Hamiltonian Monte Carlo","hyperspherical embedding","Markov chain Monte Carlo","k-nearest-neighbor OOD-ness","contrastive learning","OOD discernment loss"],"falsifier":"Take a fixed pretrained encoder and compare HamOS against a variant that synthesizes outliers on the sphere's periphery far from all clusters, evaluating both on a near-OOD test set built by corrupting in-distribution images; if the peripheral variant wins on near-OOD data, the between-cluster prior is not capturing the OOD geometries that matter.","tokens_in":50286,"feed_emoji":"🎲","tokens_out":5886,"duration_ms":51656,"temperature":0.7,"pith_summary":"The paper tries to establish that out-of-distribution detection can be substantially improved without any natural outlier data, by synthesizing virtual outliers with Hamiltonian Monte Carlo. The method, HamOS, treats outlier generation as sampling from Markov chains that start at the midpoints between pairs of nearby in-distribution class clusters on a unit hypersphere, and are guided by a k-nearest-neighbor based OOD-ness energy. The sampled outliers span a wide range of OOD scores and are used to train an OOD discernment loss alongside classification and contrastive losses. On the standard benchmarks this yields FPR95 of 10.48% on CIFAR-10 and 46.68% on CIFAR-100 averaged over five OOD test sets, and 44.59% on ImageNet-1K, while keeping ID accuracy competitive.","feed_headline":"Markov-chain outliers top OOD detectors without extra data","feed_subtitle":"Synthetic outliers from in-distribution features alone cut CIFAR-10 FPR95 to 10.48% and CIFAR-100 to 46.68%.","key_machinery":"The load-bearing object is the OOD-ness potential energy $U^{OOD}(z; Z_u, Z_v) = -\\log P^{OOD}(z; Z_u, Z_v)$, where $P^{OOD}$ is the averaged Euclidean distance from $z$ to the $k$-th nearest neighbor in each of two ID class clusters; this function is used as the potential in spherical Hamiltonian Monte Carlo on the unit sphere. The machinery also includes the class-pair midpoints used as chain initializations, a hard margin derived from von Mises-Fisher kernel density estimation that rejects proposals landing near ID clusters, and the training objective $L_{HamOS} = L_{CE} + L_{ID-con} + \\lambda_d L_{OOD-disc}$ that pushes synthesized outliers away from ID prototypes.","core_discovery":"The central claim is that the missing ingredient for strong OOD detection is not real outlier data but a sampling scheme that explores the feature space with sufficient diversity; the paper argues that Markov-chain traversal guided by the averaged k-th nearest-neighbor distance to a pair of ID clusters produces exactly such outliers. Starting from cluster midpoints and moving along HMC trajectories from low to high OOD-ness regions, the chains collect points that occupy a continuum of OOD scores without entering the ID clusters, which the authors identify as the reason HamOS outperforms Gaussian-sampling baselines like VOS and NPOS. The authors further claim that this synthesis is cheap enough for large-scale use and that the framework is a general shell, working with several HMC variants, scoring functions, and ID contrastive losses.","pith_inferences":["The between-cluster prior is a substantive modeling choice: if deployment OOD data concentrates along other regions of the sphere (e.g., far from all clusters), the synthesized outliers may miss the critical boundary, and a hybrid that also samples peripheral regions could be needed.","The Markov-chain view suggests a direct bridge to score-based generative models: replacing the k-NN energy with a learned score function could adapt synthesis to the data manifold, at the cost of extra parameters.","The method's dependence on fine-grained cluster structure may make it sensitive to the number of classes and cluster separability, so a test on datasets with many overlapping classes (e.g., fine-grained recognition) would clarify how general the gain is."],"forward_implications":["OOD-aware training no longer depends on collecting natural outlier pools, which matters for domains where high-quality OOD data are scarce or expensive.","The reported gains on fine-tuned pretrained models suggest that any deployed classifier can be made more reliable in 20 epochs using only its own training data.","Because HamOS is compatible with multiple HMC variants, post-hoc scoring functions, and ID contrastive losses, improvements in any of those components can be absorbed directly.","The same synthesis scheme could regularize other representation-learning tasks that need explicit negative supervision beyond class boundaries."],"supporting_citations":[{"why":"Supplies the Hamiltonian Monte Carlo algorithm whose high acceptance rate and long trajectories make diverse outlier sampling practical.","marker":"[Duane et al., 1987]"},{"why":"Provides the spherical HMC update that keeps generated outliers on the unit hypersphere.","marker":"[Lan et al., 2013]"},{"why":"The VOS baseline that HamOS must beat; samples virtual outliers with Gaussian perturbations around ID clusters.","marker":"[Du et al., 2022b]"},{"why":"The NPOS baseline; non-parametric Gaussian outlier synthesis that HamOS compares against and outperforms.","marker":"[Tao et al., 2023]"},{"why":"The CIDER loss used as the default ID contrastive objective and the hyperspherical embedding setup HamOS builds on.","marker":"[Ming et al., 2023b]"},{"why":"The KNN distance used both as the inference-time scoring function and as the OOD-ness measure inside the sampler.","marker":"[Sun et al., 2022]"},{"why":"The Metropolis acceptance step used inside the HMC sampler to reject proposals that violate the hard margin.","marker":"[Metropolis et al., 1953]"}],"fun_headline_variants":["HMC-synthesized outliers beat OOD baselines using only ID data","Synthetic outliers via HMC make OOD detection SOTA without extra data","Markov-chain sampling crafts diverse outliers for top OOD scores","No real outliers needed: HMC synthesis boosts OOD detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole method rests on the premise that the most effective OOD supervision signals live in the region between and around the in-distribution class clusters in the hyperspherical embedding, and that the averaged k-th nearest-neighbor distance to two clusters is a faithful measure of how out-of-distribution a point is.","fun_headline_variants_meta":{"raw":{"variants":["HMC-synthesized outliers beat OOD baselines using only ID data","Synthetic outliers via HMC make OOD detection SOTA without extra data","Markov-chain sampling crafts diverse outliers for top OOD scores","No real outliers needed: HMC synthesis boosts OOD detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000573,"raw_usage":{"total_tokens":2681,"prompt_tokens":894,"completion_tokens":1787,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":510,"completion_tokens_details":{"reasoning_tokens":1710}},"tokens_in":510,"tokens_out":1787,"duration_ms":11413,"temperature":1.0,"reasoning_tokens":1710,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T11:11:49.931711+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a fixed pretrained encoder and compare HamOS against a variant that synthesizes outliers on the sphere's periphery far from all clusters, evaluating both on a near-OOD test set built by corrupting in-distribution images; if the peripheral variant wins on near-OOD data, the between-cluster prior is not capturing the OOD geometries that matter.","supporting_citations":[{"cited_title":"Spherical hamiltonian monte carlo for constrained target distributions","cited_arxiv_id":null,"evidence_quote":"Provides the spherical HMC update that keeps generated outliers on the unit hypersphere."},{"cited_title":"Non-parametric outlier synthesis","cited_arxiv_id":null,"evidence_quote":"The NPOS baseline; non-parametric Gaussian outlier synthesis that HamOS compares against and outperforms."},{"cited_title":"Equation of state calculations by fast computing machines","cited_arxiv_id":null,"evidence_quote":"The Metropolis acceptance step used inside the HMC sampler to reject proposals that violate the hard margin."}],"review_version":1}