{"id":"4d1bdcc0-a016-4d43-b8ec-2f2df276047d","arxiv_id":"2501.12204","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A GLRT-based score-combining rule slightly improves average AUROC and detection rate over CSI/SupCSI and classical p-value combination methods in dataset-vs-dataset and leave-one-class-out OOD experiments.","lead":"This paper proposes a statistics-based way to combine the multiple scores that contrastive out-of-distribution detectors produce, using a generalized likelihood ratio test. It reports small average gains over existing combining rules on image benchmarks, along with a conformal procedure that bounds false alarms.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Empirical claim of outperformance over Fisher/Stouffer is not statistically established: average AUROC gaps are ≤0.01, no error bars or repeated seeds are reported, and ε=0.25 is tuned on an evaluation OOD pair.","rationale":"The reader's verdict of CONDITIONAL is appropriate, but the single most load-bearing weakness is not the independence assumption itself. The paper explicitly acknowledges that independence is false (Section 3.1) and even provides a partial empirical check in Appendix A.2: replacing the identity covariance with the inlier sample covariance does not improve performance on the tested benchmarks, and in fact hurts. That result partially defuses the independence concern for the specific empirical claim, while correctly removing any theoretical optimality guarantee. The remaining and more serious weakness is that the claim 'GLRT outperforms Fisher and Stouffer' rests on extremely small average differences from a single run per condition. For example, in Table 6 the GLRT average AUROC is 0.9677 versus 0.9670 for both Fisher and Stouffer; in Table 8 the GLRT average DR is 86.2 versus 86.3 for Stouffer. These gaps are far below what one would expect from seed-to-seed variation in contrastive representation learning. Furthermore, the only hyperparameter of the method, ε, is chosen in Section 4.2 using the CIFAR-10-vs-CIFAR-100 task, which is also one of the evaluation tasks; this creates an optimistic bias that could explain part of the observed improvement. The proposed concrete test—multiple seeds, paired significance tests, and an ε=0 ablation that does not use OOD data—would settle whether the central empirical claim is real. This is exactly the kind of evidence needed before accepting the paper's headline. I therefore keep the reader's CONDITIONAL verdict unchanged: the contribution is technically reasonable, but the supporting experiments need to be strengthened before the claim can be accepted as stated.","tokens_in":20730,"tokens_out":11199,"duration_ms":114449,"concrete_test":"Run the complete pipeline (CSI/SupCSI training plus each score combiner) with K=10 independent seeds for all 8 dataset-vs-dataset and 10 leave-one-class-out configurations. For every configuration, compute the paired AUROC and DR(5%) differences: GLRT minus Fisher and GLRT minus Stouffer. Test the average paired difference over configurations against zero using a Wilcoxon signed-rank test, and report the median and interquartile range across seeds. In addition, repeat the full comparison with ε=0 instead of ε=0.25, so that no OOD data are used for hyperparameter selection. If the paired differences are not significant at p<0.05, or if the ε=0 ablation removes the advantage over Fisher/Stouffer, the central claim of outperformance is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is empirical: GLRT combining outperforms CSI/SupCSI heuristics and classical combiners. The data supporting the most novel part of this claim—outperformance over Fisher and Stouffer—consists of average AUROC differences as small as 0.0002–0.009 (Tables 6–9) and DR differences of 0.1–1.0 percentage points (Tables 8–9), from a single run per condition. Given the stochasticity of contrastive training (random augmentations, initialization, batch order), such differences are within plausible run-to-run noise. No error bars, number of seeds, or paired significance tests are reported. Additionally, Section 4.2 selects the single tunable parameter ε=0.25 using the CIFAR-10-inliers vs CIFAR-100-novelties evaluation task, which leaks OOD information into the method; the theoretically justified ε=0 (Wei et al. 2019) is abandoned because it performs slightly worse on that one task. Thus the headline result—that the GLRT is better than existing combiners—may not survive controlled evaluation. The independence assumption of eq. (15) is a secondary concern: Appendix A.2 partially tests it by substituting the sample covariance, and identity covariance still performs best in these benchmarks, so the theoretical misspecification does not by itself invalidate the empirical finding; it merely removes the theoretical optimality guarantee. The empirical support is the load-bearing weakness.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes to replace the heuristic score-combining rules used by the CSI and SupCSI contrastive OOD detectors (Eqs. (6) and (8)) with a generalized likelihood ratio test (GLRT) derived from a negative-means model on z-values obtained from an empirical CDF transform of the base scores (Eqs. (15) and (20)). The authors also give a conformal-prediction procedure that, using a validation set, yields a false-alarm-rate guarantee (Algorithms 2 and 3). Experiments on dataset-vs-dataset tasks with CIFAR-10, SVHN, LSUN, ImageNet, and CIFAR-100, and on leave-one-class-out tasks with CIFAR-10, claim that the GLRT combination outperforms the original CSI/SupCSI heuristics and the classical Fisher, Bonferroni, Simes/BH, Stouffer, and ALR combiners.","tokens_in":20999,"tokens_out":3045,"duration_ms":33721,"significance":"The statistical derivation of the GLRT statistic is sound and the proposed framework is a principled alternative to ad-hoc ensembling of contrastive scores. The false-alarm-rate guarantee in Section 3.3 is a useful contribution, as is the appendix's explicit examination of the independence assumption through a general-covariance variant. However, the central empirical claim of outperformance is currently not statistically established: the reported gains over Fisher and Stouffer are very small in AUROC and detection rate, the experiments appear to be single-run, and the single tuned parameter epsilon is selected using one of the evaluation tasks. If the empirical claim could be supported by repeated-seed results with proper selection of epsilon, the paper would be a genuinely useful contribution to OOD detection methodology.","major_comments":[{"comment":"The free parameter epsilon in Eq. (15) is set to 0.25 by tuning on the CIFAR-10-inliers vs CIFAR-100-novelties task, which is one of the evaluation tasks reported in Tables 2, 4, 6, and 8. This leaks OOD information into the method and makes the reported improvements over eps=0 and over the classical combiners hard to interpret. The authors should either choose epsilon on a held-out validation split that does not use any evaluation OOD dataset, or report the main comparisons for a range of epsilon values, including epsilon=0, to show that the conclusions are not artifacts of this choice.","section":"Section 4.2, Figure 4"},{"comment":"No error bars, confidence intervals, number of seeds, or paired significance tests are reported. The average AUROC differences between GLRT and Fisher/Stouffer are as small as 0.0002-0.009 (e.g., Table 6 avg 0.9677 vs 0.9670 for Fisher and Stouffer; Table 7 avg 0.9928 vs 0.9926/0.9916), and the DR differences are 0.1-1.0 percentage points. Given the stochasticity of contrastive training, these differences are within plausible run-to-run noise. The central claim that the GLRT outperforms Fisher and Stouffer therefore needs repeated-seed experiments with variance reporting or paired tests before it can be accepted.","section":"Tables 6-9 and 13-14"},{"comment":"The authors explicitly state that the independence assumption in Eq. (15) is not expected to hold in practice. The GLRT derivation and its Neyman-Pearson/minimax optimality rely on this assumption, while the actual z-values are derived from correlated scores. Appendix A.2 partially addresses this by substituting a sample covariance, but it only compares identity versus sample covariance on the dataset-vs-dataset AUROC tables and finds that identity performs better. This does not resolve the misspecification concern for correlated non-Gaussian z-values. The authors should either temper the optimality language to describe the GLRT as a heuristic score combination that is principled only under the NM model, or provide additional diagnostics on the actual distribution of the z-vectors.","section":"Section 3.1, Eq. (15)"},{"comment":"The claim that the GLRT outperforms all classical combiners is not uniformly supported by the tables. In Table 8, Stouffer achieves an average DR of 86.3 compared to 86.2 for GLRT, and in Tables 8-9 GLRT and Fisher tie on average. The text in Section 4.3 acknowledges this, but the abstract and conclusion state the outperformance claim without this qualification. The claims should be restated as 'on average over AUROC' and should acknowledge that the advantage over Fisher/Stouffer is not consistent across all metrics and tasks.","section":"Section 4.3, Tables 8-9"},{"comment":"The false-alarm-rate guarantee is stated as holding 'with probability at least 1-delta' over the random validation set, and the procedure in Algorithm 2 searches for a threshold a that controls the beta upper tail. This is a valid conformal-style argument, but the guarantee is average over the validation set under the assumption that Xval is drawn i.i.d. under H0. In the experiments, no results using this finite-sample guarantee are reported; all DR-versus-FAR results instead use a threshold computed directly on the test set. The paper should either include experiments demonstrating the conformal procedure, or explicitly state that the guarantee is a methodological contribution not evaluated empirically.","section":"Section 3.3, Eq. (24)"}],"minor_comments":[{"comment":"The name 'Benjamini-Hochwald' appears in the abstract and contribution list; this should be 'Benjamini-Hochberg'.","section":"Abstract and Section 1"},{"comment":"In Algorithm 3, the line 'return “OOD” if bq(x) ≤ a' is preceded by 'white guaranteeing a false-alarm rate'; this should read 'while guaranteeing'.","section":"Algorithm 3"},{"comment":"The conclusion refers to 'the non-negative means problem (15)', but Eq. (15) is the negative-means problem with means ≤ -ϵ; the wording should be corrected for consistency with Section 3.1.","section":"Section 5"},{"comment":"Figure 2 plots the proposed test statistic for m=1, but the paper does not mention in the caption or text that this is the univariate case; a brief note would avoid confusion.","section":"Figure 2"},{"comment":"The row for glrt-SupCSI+ marks both 'Heuristic' and 'GLRT' columns as applicable; it would be clearer to state explicitly that the base scores themselves are computed with the SupCSI/OC-SVM/Mahalanobis methods while the combining is GLRT-based.","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":"The statistical framework and derivation are sound, and I see no circularity in the GLRT itself. The load-bearing weakness is purely empirical: single-run results, very small gaps over Fisher/Stouffer, and epsilon tuned on an evaluation task. This is fixable within the scope of the manuscript by adding repeated-seed experiments, variance reporting, and a proper selection procedure for epsilon. I would not reject on the independence-assumption concern because the authors acknowledge it and the identity-covariance choice still performs best in their experiments; however, the optimality claims should be softened. I did not weigh the absence of released code in the verdict, but it would improve reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take on 2501.12204. The paper does something real: it replaces the heuristic score-averaging rules in CSI/SupCSI with a GLRT derived from a negative-means model over empirical z-values, and it adds a conformal calibration step that carries a finite-sample false-alarm guarantee. The derivation of t_GLRT (eq. 20) is correct, the writing is clear, and the honesty is refreshing: Section 3.1 states plainly that the independence assumption in (15) is not expected to hold, Appendix A.2 actually tests a covariance-generalized variant, and the Limitations paragraph admits that adding scores can hurt, which their own SupCSI+ results show.\n\nWhat is genuinely new: applying the convex-cone GLRT of Wei et al. (2019) to combine self-supervised OOD scores, with the epsilon > 0 extension and the empirical-CDF z-transform. Modest but real. The conformal false-alarm calibration in Section 3.3 is the cleanest part of the paper and is directly reusable.\n\nSoft spots, in proportion. The headline claim, that GLRT beats Fisher and Stouffer, is not established by the evidence. Tables 6-9 and 13-14 show average AUROC gaps of 0.0002-0.009 and DR gaps of a point or less, apparently from a single run per condition: no error bars, no seeds, no paired tests. In Table 8 (DR at 5% FAR with CIFAR-10 inliers), Stouffer actually wins. At that resolution the honest summary is 'competitive with, occasionally marginally better than, Fisher/Stouffer.' Second, epsilon = 0.25 is chosen using one of the evaluation tasks (Figure 4, CIFAR-100 vs CIFAR-10). The paper discloses this and fixes epsilon for all tasks, but the theoretically motivated epsilon = 0 was nonetheless abandoned on the basis of that single plot; a nested selection would be cleaner. Third, the independence assumption is false, but the stress-test note overweights it: Appendix A.2 shows identity covariance beats an estimated covariance, so the misspecification does not impeach the empirical comparison, it only voids the theoretical optimality claim. And no code is released, which sharpens the missing error bars.\n\nWho gets value: anyone working on OOD score ensembles or p-value/z-value combining. The GLRT formulation is worth engaging and the calibration trick is directly usable. I would not desk-reject on the empirics, but a referee should demand multiple seeds, error bars, and a proper evaluation of epsilon. Recommendation: yes, send to peer review. It is an honestly written methods paper with a correct derivation and a useful calibration tool; the empirical overreach is fixable with compute and a more careful writeup.","headline":"Clean GLRT-based score combining for contrastive OOD detection with an honest writeup, but the claim of beating Fisher/Stouffer rests on sub-0.01 AUROC gaps with no error bars.","tokens_in":21553,"tokens_out":4760,"would_cite":true,"duration_ms":43439,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F03","62H15"],"pacs":[],"model":"deepseek-v4-flash","headline":"A generalized likelihood-ratio test derived from a negative-means model of z-scored inlier scores outperforms heuristic score-combining rules for contrastive out-of-distribution detection.","keywords":["out-of-distribution detection","contrastive learning","hypothesis testing","generalized likelihood ratio test","score combining","negative-means model","conformal p-values","anomaly detection"],"falsifier":"Measure the off-diagonal correlations of the empirical z-value vectors on a held-out inlier set; if a GLRT built with the estimated inlier covariance (Appendix A.2) does not beat or match the identity-covariance GLRT on the same novelty sets, the independence assumption is doing essential work rather than being a harmless device.","tokens_in":20490,"feed_emoji":"🎯","tokens_out":15007,"duration_ms":123423,"temperature":0.7,"pith_summary":"Out-of-distribution (OOD) detection asks whether a test sample comes from a known inlier distribution or from something new. The leading contrastive detectors, CSI and SupCSI, compute many intermediate inlier scores and merge them with hand-crafted rules. This paper proposes replacing those rules with a formal hypothesis test: each score is converted to a z-value (the inverse-normal transform of its empirical percentile), an inlier is modeled as $\\mathcal{N}(0, I)$ on those z-values, and an OOD sample as having every z-coordinate mean at most $-\\epsilon$; the generalized likelihood ratio test (GLRT) of this negative-means model becomes the combining rule. On dataset-versus-dataset experiments with CIFAR-10, SVHN, LSUN, ImageNet, and CIFAR-100, and leave-one-class-out experiments on CIFAR-10, the GLRT combination improves the area under the ROC curve and the detection rate at a fixed false-alarm rate over the original CSI/SupCSI rules and over Fisher, Bonferroni, Simes, Benjamini-Hochberg, Stouffer, and ALR combinations. If the claim holds, it replaces heuristic weight tuning with a principled, extensible way to fuse any number of inlier scores.","feed_headline":"A likelihood-ratio test beats heuristic score fusion in OOD detection","feed_subtitle":"Principled z-value fusion beats Fisher, Bonferroni, Simes, Stouffer, and the CSI baseline on image OOD benchmarks.","key_machinery":"The load-bearing object is the negative-means (NM) hypothesis-testing model and its generalized likelihood ratio test statistic. The model (equation 15) says that inlier z-values are independent standard normals while OOD z-values are normals with every coordinate mean at most $-\\epsilon$; the GLRT statistic $t_{\\mathrm{GLRT}}(z) = (\\tfrac{1}{2} z^{-} - z)^T z^{-}$ with $z^{-}_l = \\min\\{z_l, -\\epsilon\\}$ is the log-likelihood ratio after replacing the unknown alternative mean by its constrained maximizer. The machinery also includes the empirical CDF transform $\\hat{z}_l(x) = \\Phi^{-1}(\\hat{F}_l(s_l(x)))$ that converts arbitrary score distributions to z-values, and a conformal p-value threshold (Algorithm 3) that certifies a user-specified false-alarm rate up to a chosen failure probability using a validation set.","core_discovery":"The paper's central claim is that score combining for contrastive OOD detection should be posed as a negative-means hypothesis test on z-values rather than as a heuristic sum, max, or weighted average. Under the null hypothesis $H_0 : Z \\sim \\mathcal{N}(0, I)$, each empirical z-value is standard normal; under the alternative $H_1 : Z \\sim \\mathcal{N}(\\mu, I)$ with $\\mu_l \\le -\\epsilon$ for every $l$, an OOD sample is, on average, worse on every score. Solving this composite testing problem with a GLRT yields the statistic $t_{\\mathrm{GLRT}}(z) = (\\tfrac{1}{2} z^- - z)^T z^-$, where $z^-_l = \\min\\{z_l, -\\epsilon\\}$. The paper claims that thresholding this statistic outperforms the heuristic combining rules of CSI and SupCSI on the tested image benchmarks, and also outperforms Fisher, Bonferroni, Simes/BH, Stouffer, and ALR p-value combiners.","pith_inferences":["Editorial inference: the average AUROC gap between the GLRT and the best classical combiners (Fisher, Stouffer) is well under a percentage point, while Bonferroni, Simes/BH, and ALR are much further behind, which suggests the GLRT's practical value is stability across datasets rather than a large leap over a well-chosen classical merger.","Editorial inference: because the independence assumption in (15) is false, a natural stress test is to compare the GLRT against a covariance-aware GLRT on benchmarks with strongly correlated scores; the paper's appendix finds the estimated-covariance variant underperforms, hinting that the identity covariance acts as a helpful inductive bias.","Editorial inference: the conformal false-alarm machinery of Section 3.3 does not depend on the GLRT statistic itself, so any score combiner could inherit the same finite-sample threshold guarantee by reusing Algorithm 3.","Editorial inference: since the pipeline only sees marginal z-values, it could fuse heterogeneous evidence (classifier logits, nearest-neighbor distances, density estimates) in other one-class and anomaly-detection tasks, not just contrastive image representations."],"forward_implications":["New inlier scores can be added to a contrastive detector by appending their empirical z-values, since the GLRT is agnostic to how the scores were produced; glrt-SupCSI+ combines 24 scores and improves dataset-vs-dataset AUROC over glrt-SupCSI.","The GLRT removes the need to hand-tune CSI's weighting coefficients $\\lambda^{\\mathrm{con}}_j$ and $\\lambda^{\\mathrm{shift}}_j$, because every score enters through its own z-value.","A validation set of size $v$ gives a threshold such that, with probability at least $1-\\delta$, the false-alarm rate is at most $\\alpha$, via the beta-distributed conformal p-value described in Section 3.3.","A positive offset $\\epsilon$ (the paper uses 0.25) improves AUROC over the $\\epsilon = 0$ cone problem.","The benefit of adding extra base scores is not guaranteed: glrt-SupCSI+ underperforms glrt-SupCSI on leave-one-class-out CIFAR-10, so base-score selection remains part of the design."],"supporting_citations":[{"why":"Supplies the CSI and SupCSI detectors, their 12 base scores, and the heuristic combining rules (equations 6 and 8) that the GLRT replaces.","marker":"(Tack et al., 2020)"},{"why":"Establishes minimax optimality of the GLRT for the $\\epsilon=0$ cone problem that the paper generalizes to $\\epsilon>0$.","marker":"(Wei et al., 2019)"},{"why":"Provides the LRT/GLRT framework, Neyman-Pearson optimality, and composite-testing background used to derive equation (20).","marker":"(Lehmann et al., 2005)"},{"why":"Supplies the conformal p-value distribution and the beta false-alarm bound used in Algorithm 3 to set thresholds.","marker":"(Vovk, 2012)"},{"why":"Supplies Fisher's p-value combination method, a baseline the GLRT must beat.","marker":"(Fisher, 1992)"},{"why":"Supplies Simes' method, a baseline for the global-null tests.","marker":"(Simes, 1986)"},{"why":"Supplies the BH procedure, the Simes-equivalent baseline used in the comparisons.","marker":"(Benjamini and Hochberg, 1995)"},{"why":"Supplies the ALR test statistic used as a score-combining baseline.","marker":"(Walther, 2013)"},{"why":"Supplies the CIFAR-10 and CIFAR-100 datasets used as inlier and novelty sets in the main experiments.","marker":"(Krizhevsky, 2009)"},{"why":"Supplies the SVHN dataset used as both an inlier and a novelty set in the experiments.","marker":"(Netzer et al., 2011)"}],"fun_headline_variants":["GLRT beats heuristic fusion in contrastive OOD detection","Principled z-test outperforms Fisher and CSI for OOD","New likelihood-ratio method improves OOD score combining","Negative-mean GLRT trumps classic p-value combiners"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The derivation assumes the z-values of an inlier are independent standard normals; the paper says this independence does not hold, so if the scores are strongly correlated, the GLRT is no longer a likelihood-ratio test for the true distribution.","fun_headline_variants_meta":{"raw":{"variants":["GLRT beats heuristic fusion in contrastive OOD detection","Principled z-test outperforms Fisher and CSI for OOD","New likelihood-ratio method improves OOD score combining","Negative-mean GLRT trumps classic p-value combiners"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000191,"raw_usage":{"total_tokens":1368,"prompt_tokens":992,"completion_tokens":376,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":608,"completion_tokens_details":{"reasoning_tokens":307}},"tokens_in":608,"tokens_out":376,"duration_ms":4107,"temperature":1.0,"reasoning_tokens":307,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:23:35.388986+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the off-diagonal correlations of the empirical z-value vectors on a held-out inlier set; if a GLRT built with the estimated inlier covariance (Appendix A.2) does not beat or match the identity-covariance GLRT on the same novelty sets, the independence assumption is doing essential work rather than being a harmless device.","supporting_citations":[],"review_version":1}