{"id":"db56e298-8063-4978-a260-c045609302af","arxiv_id":"2412.15668","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"AHGC clusters labeled and unlabeled images into subgraphs with a hierarchical graph cut, assigns pseudo-labels to unlabeled in-distribution images, and reports large FPR95 gains on CIFAR-10 and CIFAR-100 SC-OOD benchmarks.","lead":"This paper introduces AHGC, a graph-based method that separates in-distribution from out-of-distribution images when unlabeled data contains images from overlapping label categories at different granularities, such as 'cat' and 'tabby cat'. It reports large reductions in false positive rate on two standard benchmarks, but does not release code, and some claimed improvements are percentage-point differences rather than relative gains.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Hyperparameters are tuned on the same test benchmarks used for the headline FPR95 numbers, so the claimed SOTA margin is not yet supported as a validated result.","rationale":"I read the paper in good faith: the graph-cut framework is coherent and the empirical gap is large. But the load-bearing condition for 'consistently best' is that the reported numbers are honest estimates of performance on unseen data. The ablation section selects hyperparameters on the exact test benchmarks (Fig. 5, Fig. 6, Sec. V.E.2, V.E.5), and Sec. V.C reports the chosen values without any validation protocol. This is not an internal inconsistency, but it is a correctness risk: the reported FPR95 improvements could be inflated by selection over the test set. The reader's weakest-assumption (subgraph purity) is a mechanism concern, but contamination would tend to make OOD detection worse, not better, so it is less directly tied to the headline improvement; the paper should still measure contamination. My concrete test (held-out validation with fixed hyperparameters and multiple seeds) would settle whether the central empirical claim is real. Since the reader already conditioned acceptance on releasing artifacts and clarifying validation, my concern does not change the verdict.","tokens_in":25106,"tokens_out":8826,"duration_ms":76047,"concrete_test":"Split each benchmark into train/validation/test (e.g., hold out a random subset of the ID test images and of the known-ID/OOD unlabeled images for validation). Tune K, alpha, beta, gamma, delta, p_tau, rho only on the validation split, then freeze them and run 5 seeds on the untouched test split, reporting mean +/- std for FPR95, AUROC, AUPR, CCR. If the held-out margin over UDG (Tables I/II) is no longer large, the reported improvement is a test-selection artifact; if the margin persists, the central claim survives this objection.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that AHGC is state of the art on SC-OOD, with 40.47%/81.24% FPR95 gains (Abstract, Sec. V.D). The evidence for this claim is Tables I/II, but every important hyperparameter (subgraph count K, alpha, beta, gamma, delta, p_tau, rho) is selected by maximizing AUROC/AUPR on those same CIFAR-10/CIFAR-100 benchmark test sets: Sec. V.C fixes values chosen in Sec. V.E.5 via Fig. 6, and Sec. V.E.2 picks K via Fig. 5. No validation split or nested-protocol description is given. With six continuous hyperparameters and six OOD sets per benchmark, test-set selection can materially inflate FPR95/AUROC; a single run with no variance estimate cannot separate a real algorithmic margin from selection noise. The subgraph-contamination issue in the reader's verdict is real but secondary: if OOD samples were pseudo-labeled as ID, the model would be trained to give them high similarity scores, which should worsen (increase) FPR95, not improve it, so contamination does not explain the reported gains. The missing validation protocol is the load-bearing gap.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AHGC, an adaptive hierarchical graph-cut network for semantically coherent out-of-distribution detection. It builds a hierarchical KNN graph over labeled and unlabeled image features, cuts the graph into subgraphs using GAT-based linkage and density estimates, and pseudo-labels unlabeled samples in subgraphs whose labeled proportion exceeds a threshold rho. Training combines a classification loss, an equalization loss, an InfoNCE augmentation loss, and a co-training loss, while inference uses a temperature-scaled energy score. Experiments on the CIFAR-10 and CIFAR-100 SC-OOD benchmarks report large FPR95 improvements, including absolute drops of 40.47 and 81.24 percentage points on the Tiny-ImageNet unlabeled set.","tokens_in":25382,"tokens_out":6859,"duration_ms":56648,"significance":"If the reported results are trustworthy, AHGC is a novel and potentially important contribution to multi-granularity OOD detection: the graph-cut formulation with intra-subgraph label assignment is original, the ablation study is coherent and shows each loss component helping, and the reported margins over UDG and other SC-OOD methods are very large. The paper also gives a qualitative visualization supporting the intended semantic grouping. However, the absence of a validation protocol, the lack of error bars, and an internally inconsistent 'consistently best' caption substantially reduce the confidence that the headline SOTA claim is supported.","major_comments":[{"comment":"Hyperparameters alpha, beta, gamma, delta, p_tau, rho, and the subgraph number K are selected by maximizing AUROC/AUPR on the same CIFAR-10/CIFAR-100 benchmark test sets that appear in Tables I and II. No held-out validation split or nested selection protocol is described in Section V.C or Section V.E. With at least seven tunable quantities and six OOD test sets per benchmark, the reported SOTA margins may reflect selection on the test benchmark rather than a validated algorithmic gain. Please add a proper validation split (or nested cross-validation), fix all hyperparameters on that split, and then report test results, along with multiple-seed means and standard deviations.","section":"Section V.E.5 and Fig. 6; Section V.E.2 and Fig. 5"},{"comment":"The caption's claim that 'Our AHGC achieves consistently the best performance on all OOD detection metrics' is directly contradicted by the CIFAR-10 OOD row: AHGC obtains AUROC 74.47 versus 78.95 for EBO and 78.92 for OE; AUPR(In) 78.21 versus 81.95 for OE; AUPR(Out) 67.13 versus 74.44 for EBO; and CCR@FPR10^-1 44.42 versus 48.67 for EBO. Section V.D's analysis partially acknowledges this, but the caption and the 'consistently best' statement must be corrected or carefully qualified.","section":"Table II caption and CIFAR-10 row"},{"comment":"The paper never measures the contamination rate of pseudo-labeled subgraphs. The label-assignment rule in Section IV.E trusts any subgraph with rho > 0.5 and assigns the majority label to every unlabeled sample, so any OOD sample that lands in an ID-majority subgraph is trained as ID. The qualitative claim in Section V.F that 'no unlabeled OOD sample is mistakenly into ID cluster' is supported only by a few illustrative examples. Please report the fraction of OOD samples that receive pseudo-labels, the precision of the pseudo-labels on unlabeled ID samples, and ideally the label composition of the accepted subgraphs.","section":"Section IV.E and Section V.F"},{"comment":"All metrics are reported as single numbers with no error bars and no statement about the number of runs. Because the pipeline involves GAT, data augmentation, SGD, and iterative pseudo-labeling, a single run is insufficient to support claims of consistent superiority, especially in rows where margins are small (for example, Table II, Places365 AUROC: 81.73 for AHGC versus 80.96 for Scone, and Table I, CIFAR-100 AUROC: 92.72 versus 91.24). Please report mean and standard deviation over at least three to five seeds.","section":"Tables I and II"}],"minor_comments":[{"comment":"The '81.24%' and '40.47%' improvements are absolute FPR95 reductions (for example, 81.73 − 0.49 = 81.24 for CIFAR-100/Tiny-ImageNet), not relative improvements; the text should say 'percentage points' to avoid ambiguity.","section":"Abstract and Section V.D"},{"comment":"The InfoNCE denominator appears to include the positive pair twice: the first sum over b_j in B1 contains b^1_i, and the second sum over b_g in B0 contains b^0_i. Please clarify the intended contrastive form.","section":"Eq. (7)"},{"comment":"The termination condition for the hierarchy, 'E'_l = empty set', is unclear, since at each level every node with non-empty candidate set adds one edge; please specify how convergence is reached and how the chosen total subgraph number K relates to the edge-selection process in Eq. (3).","section":"Section IV.C"},{"comment":"There are typographical and wording errors, including 'CIAFR-10', 'semantics relationship', and the duplicated GAT reference (references [55] and [61] are the same paper); these should be corrected.","section":"Throughout"},{"comment":"The feature-encoder ablation is reported only for the CIFAR-10 benchmark, while the text says the method is evaluated on 'all the tasks'; please either add CIFAR-100 results or rephrase the claim.","section":"Section V.E.3"},{"comment":"No code-release or reproducibility statement is provided; please state whether the implementation will be made publicly available.","section":"Section V"}],"recommendation":"major_revision","confidential_remarks":"The main risk is that the headline results are not yet valid evidence because all hyperparameters appear to be tuned on the test benchmarks used for the reported numbers. This is fixable with a proper validation split and multi-seed evaluation, so I do not recommend rejection, but acceptance would be premature. The Table II caption contradiction should also be resolved before the paper can be considered reliable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a genuinely new method for the SC-OOD setting, and it's worth engaging with seriously. But treat the headline FPR95 gains as unverified; the evidence stops short of supporting them.\n\nWhat's new: the hierarchical KNN graph with linkage/density edge selection and intra-subgraph majority label assignment is a real combination that I don't think appears in UDG, Scone, ConjNorm, or the other baselines. The idea of exploiting the multi-granularity relationship—coarse labels in CIFAR-10 vs fine labels in Tiny-ImageNet—is exactly the right problem to work on. The ablation study is coherent and shows each module contributes. The qualitative result in Fig. 7 does what it claims, at least on the examples shown.\n\nSoft spots, in proportion: (1) The load-bearing claim is not yet supported. The hyperparameters K, alpha, beta, gamma, delta, p_tau, rho are all chosen by ablations on the same CIFAR-10/CIFAR-100 benchmark test sets (Fig. 5, Fig. 6), with no validation split or nested protocol described. With that many continuous knobs and six OOD sets per benchmark, single-run numbers without variance estimates can't separate real algorithmic margin from selection noise. The stress-test note is right that subgraph contamination is secondary: pseudo-labeling OOD as ID would push FPR95 up, not down, so that mechanism doesn't explain the gains. The missing validation protocol is the load-bearing gap. (2) The 'consistently best on all OOD detection metrics' caption is false as stated: in Table II's CIFAR-10 row, AHGC's AUROC (74.47) is below EBO (78.95) and OE (78.92), and AUPR(Out) is below several baselines. The analysis paragraph acknowledges this in passing, but the caption and abstract overreach. (3) No code, no data release, no error bars. For an empirical claim of this size, that's a red flag only because it's fixable—release the artifacts and the story becomes testable.\n\nBottom line: the method deserves a serious referee. It's a plausible contribution to SC-OOD, the exact setting where most detectors fail. But the current version should not be accepted as-is; it needs a proper validation protocol, variance estimates, and corrected claims. I'd bring it to reading group, but I wouldn't cite the numbers in my own work until the artifacts are out.","headline":"New graph-cut method for semantically coherent OOD detection with strong reported numbers, but the claimed SOTA margin isn't yet supported: hyperparameters are tuned on the same test benchmarks, no error bars, and the 'consistently best' caption is contradicted by its own Table II.","tokens_in":25944,"tokens_out":2584,"would_cite":false,"duration_ms":19701,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AHGC claims a hierarchical graph cut over image similarities can detect out-of-distribution samples better than prior methods, cutting FPR95 by 40.47% on CIFAR-10 and 81.24% on CIFAR-100.","keywords":["out-of-distribution detection","semantically coherent OOD","hierarchical graph cut","multi-granularity label","pseudo-labeling","graph attention network","energy-based detection","FPR95"],"falsifier":"Take the SC-OOD CIFAR-10 and CIFAR-100 benchmarks, run the AHGC training pipeline, and compare every pseudo-label assigned to an unlabeled Tiny-ImageNet image with that image's ground-truth semantic class. If more than about 5% of the unlabeled images that receive a pseudo-label are actually OOD under ground-truth semantics, the claimed mechanism is not doing the work.","tokens_in":24896,"feed_emoji":"🖼️","tokens_out":6232,"duration_ms":51548,"temperature":0.7,"pith_summary":"This paper proposes an adaptive hierarchical graph cut network (AHGC) for out-of-distribution detection under the semantically coherent SC-OOD setting, where the unlabeled pool contains both OOD images and in-distribution images labeled at a different granularity. The paper's central claim is that building a hierarchical KNN graph over image features and cutting it into subgraphs by linkage and density lets the model cluster fine-grained unlabeled images with their coarse-grained labeled counterparts, so those unlabeled ID images can be pseudo-labeled instead of being treated as outliers. On the CIFAR-10 and CIFAR-100 benchmarks, AHGC is reported to lower FPR95 by 40.47% and 81.24% respectively compared with the best prior method in the Tiny-ImageNet representative case, while also improving ID classification under the same false-positive constraints. The argument matters because real-world deployment data is usually a mixture at multiple label granularities, and treating all unlabeled data as OOD is a mismatch that current methods inherit.","feed_headline":"Graph cut beats prior OOD detectors by 81.24% on CIFAR-100","feed_subtitle":"It clusters images by label granularity, so fine-grained in-distribution samples no longer count as OOD.","key_machinery":"The load-bearing construction is the hierarchical KNN graph cut. Each image is a node; an attention-aware graph cut module predicts linkage probabilities $p_{ij}$ and a density proxy $d_i$ for every node, then keeps for each node the edge to the highest-similarity neighbor whose density is no lower, so low-density boundary nodes attach to denser cluster cores and the graph splits into subgraphs. The assignment rule then propagates the majority label of each subgraph to unlabeled nodes whenever the labeled share passes the threshold $\\rho$, and subgraph aggregation propagates peak-node and average features upward to the next level. The cosine-similarity maximization between two augmentations pushes the feature space to be more discriminative before the energy-based OOD score is applied.","core_discovery":"AHGC claims to solve multi-granularity OOD detection by treating unlabeled data as a mixture of ID and OOD rather than a pure OOD set. It constructs a hierarchical KNN graph on ResNet-18 features, uses a graph attention network to predict edge linkage and node density, cuts low-weight edges to form subgraphs, and assigns the majority ground-truth label within a subgraph to unlabeled images whenever the labeled fraction exceeds a threshold. The same encoder is then trained with the added pseudo-labeled samples, an equalization loss on remaining unlabeled samples, and an InfoNCE-style consistency loss between two augmentations. Detection uses temperature-scaled energy logits. The reported result is that AHGC achieves the best mean FPR95, AUROC, AUPR, and CCR across the six unlabeled datasets on both benchmarks, with representative FPR95 gains of 40.47% over UDG on CIFAR-10 and 81.24% over UDG on CIFAR-100 when Tiny-ImageNet is the unlabeled set.","pith_inferences":["The paper does not measure how often an OOD sample slips into a subgraph whose labeled share clears $\\rho=0.5$; measuring that contamination rate on the Tiny-ImageNet ground-truth IDs would directly test whether the reported separation comes from semantic alignment or from pseudo-label noise.","The same mechanism could be applied to open-set recognition or partial-label learning, where the unlabeled set is also assumed to contain known-class and novel-class samples at mixed granularity.","One testable extension is to vary $\\rho$ and the subgraph count while reporting pseudo-label precision; if precision degrades sharply for small $\\rho$, the threshold is the true control knob of the method.","The ablation study shows that the augmentation and equalization losses contribute much of the gain, so a reader should not assume the graph-cut module alone reproduces the headline numbers."],"forward_implications":["If AHGC's performance holds, an OOD detector can exploit a mixed unlabeled pool without knowing which samples are OOD, removing the unrealistic purity assumption of outlier exposure methods.","The pseudo-labeled ID samples recovered from the unlabeled pool also improve ID classification under low-FPR constraints, so detection and classification improve together.","The method's gains on Tiny-ImageNet, LSUN, and Places365 suggest that label-granularity-aware alignment, not just covariate shift, is a large part of what made those datasets hard.","Because the graph cut is trained with linkage and density supervision from labeled images, the approach can in principle be retrained whenever the labeled set changes, without new OOD data.","The authors state the intended next step is extending AHGC to video OOD detection."],"supporting_citations":[{"why":"Defines the semantically coherent OOD benchmarks and the UDG clustering baseline whose FPR95 numbers AHGC improves by 40.47% and 81.24%.","marker":"[1]"},{"why":"Supplies the ResNet-18 feature encoder used to extract image features for the graph construction.","marker":"[45]"},{"why":"Graph attention network used as the backbone that predicts linkage and density for the graph cut.","marker":"[61]"},{"why":"ODIN baseline; prior classification-based OOD method whose scores are compared on both benchmarks.","marker":"[20]"},{"why":"EBO baseline; energy-based detector that AHGC extends with temperature-scaled logits.","marker":"[21]"},{"why":"ConjNorm baseline; density-estimation method compared against AHGC on both benchmarks.","marker":"[44]"},{"why":"Outlier exposure baseline that trains on unlabeled auxiliary data as OOD, the assumption AHGC rejects.","marker":"[69]"},{"why":"Scone baseline using wild unlabeled data for OOD detection; compared on both benchmarks.","marker":"[70]"}],"fun_headline_variants":["Graph cut OOD detection: 81.24% FPR95 gain on CIFAR-100","Adaptive graph cut for multi-granularity OOD detection","Cutting KNN graphs boosts OOD detection by 81.24%","OOD detector adapts to label granularity with graph cut"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that any subgraph whose labeled share exceeds the threshold $\\rho$ is pure enough that assigning the majority label to every unlabeled image in it does not contaminate the training set; if OOD samples leak into those subgraphs, the reported separation is inflated.","fun_headline_variants_meta":{"raw":{"variants":["Graph cut OOD detection: 81.24% FPR95 gain on CIFAR-100","Adaptive graph cut for multi-granularity OOD detection","Cutting KNN graphs boosts OOD detection by 81.24%","OOD detector adapts to label granularity with graph cut"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000306,"raw_usage":{"total_tokens":1831,"prompt_tokens":1099,"completion_tokens":732,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":715,"completion_tokens_details":{"reasoning_tokens":648}},"tokens_in":715,"tokens_out":732,"duration_ms":6022,"temperature":1.0,"reasoning_tokens":648,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T11:12:32.010676+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the SC-OOD CIFAR-10 and CIFAR-100 benchmarks, run the AHGC training pipeline, and compare every pseudo-label assigned to an unlabeled Tiny-ImageNet image with that image's ground-truth semantic class. If more than about 5% of the unlabeled images that receive a pseudo-label are actually OOD under ground-truth semantics, the claimed mechanism is not doing the work.","supporting_citations":[{"cited_title":"Semantically coherent out-of-distribution detection,","cited_arxiv_id":null,"evidence_quote":"Defines the semantically coherent OOD benchmarks and the UDG clustering baseline whose FPR95 numbers AHGC improves by 40.47% and 81.24%."},{"cited_title":"Enhancing the reliability of out- of-distribution image detection in neural networks,","cited_arxiv_id":null,"evidence_quote":"ODIN baseline; prior classification-based OOD method whose scores are compared on both benchmarks."},{"cited_title":"Energy-based out-of-distribution detection,","cited_arxiv_id":null,"evidence_quote":"EBO baseline; energy-based detector that AHGC extends with temperature-scaled logits."},{"cited_title":"Conjnorm: Tractable density estimation for out-of-distribution detection,","cited_arxiv_id":null,"evidence_quote":"ConjNorm baseline; density-estimation method compared against AHGC on both benchmarks."},{"cited_title":"Deep anomaly detection with outlier exposure,","cited_arxiv_id":null,"evidence_quote":"Outlier exposure baseline that trains on unlabeled auxiliary data as OOD, the assumption AHGC rejects."},{"cited_title":"Feed two birds with one scone: Exploiting wild data for both out-of-distribution generalization and detection,","cited_arxiv_id":null,"evidence_quote":"Scone baseline using wild unlabeled data for OOD detection; compared on both benchmarks."}],"review_version":1}