{"id":"e14232b6-95af-46bc-b408-2dc17585e9bc","arxiv_id":"2506.19281","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"CMO adds a class-mean proximity constraint to distributionally robust optimization and NNR reweights graph samples by local label consistency, together improving minority-class accuracy in graph OOD experiments.","lead":"This paper proposes two training tweaks for graph neural networks that suffer when some classes are rare and some labels are noisy. One adds a constraint that focuses the optimizer on classes with similar feature means, and the other downweights samples whose neighbors disagree.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"NNR weights depend on an unspecified neighbor definition, so the reported gains may not be reproducible from the paper alone.","rationale":"The reader's strongest claim is the minority-class improvement from CMO+NNR, and the weakest assumption is the underspecified NNR neighbor definition. My independent check of the full text confirms that Eq. 18 is the sole mechanism generating the NNR weights, with no definition of the graph-instance neighborhood, no k or radius in the main text, and no ablation over definitions. Therefore the main experimental claim is not reproducible from the manuscript. This is the single most load-bearing concern because it sits directly between the method description and the headline numbers. It justifies a CONDITIONAL rather than REJECT verdict: the paper contains a concrete, testable request (exact code for neighbor construction and an ablation over definitions), and if the authors provide it, the reported gains may be verifiable; without it, the method is ambiguous as written.","tokens_in":16499,"tokens_out":1285,"duration_ms":12053,"concrete_test":"Ask the authors to release the exact neighbor-construction code and rerun Table 1 (synthetic, noise 0.2, ERM+NNR+CMO-KL; baseline ERM+CMO-KL) while ablating the neighbor definition: (a) cosine distance on the frozen GNN embedding at epoch t, (b) Euclidean distance on raw node statistics, (c) label-only kNN, and (d) random neighbor assignment. If the reported 80.2% Max% minority accuracy persists under random or label-only neighbors, then Eq. 18 is not the causal source of the gain and the contribution is vacuous; if the gain disappears for any reasonable definition, the paper needs to specify the definition and re-run all reported tables.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central experimental claim is that NNR (Eq. 18) reweights samples by N_hom(G_i)/N_sum(G_i), the fraction of same-class neighbors within a distance, improving minority-class accuracy (Tables 1 and 3). But for graph classification, each G_i is an entire graph; there is no neighborhood structure among graph instances. The paper never defines the distance or what 'same' means: input space, embedding space, or label-only. Assumption 3 asserts 'near sets' of graphs are disjoint and equal-sized, which is unlikely to hold in feature or embedding space and is not verified. The sensitivity analysis (Fig. 6) varies Gamma but never reports how changing the neighbor definition changes performance. If 'same-class neighbors' are defined via the true label, then NNR is a label-based density estimate requiring unlabeled test information or circular training labels; if defined via noisy labels, it collapses on exactly the samples NNR is meant to protect. Because Eq. 18 is the only mechanism producing the reported NNR gains, the method is underspecified as written and the headline improvement cannot be independently reproduced or falsified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes two modules for graph out-of-distribution (OOD) classification under class imbalance and label noise: Constrained Mean Optimization (CMO), which adds a class-mean constraint to a DRO objective, and Neighbor-Aware Noise Reweighting (NNR), which downweights samples with few same-class neighbors. The authors provide convergence theorems for the optimization, an OOD generalization bound based on PAC-Bayes arguments, and experiments on synthetic data plus DrugOOD, reporting gains in minority-class accuracy for the combined method.","tokens_in":16755,"tokens_out":6233,"duration_ms":59414,"significance":"If the empirical claims held, the work would address a practically relevant gap: minority-class robustness under both imbalance and structural noise in graph classification. The paper has some strengths: it benchmarks against a broad set of DRO baselines, it includes a code link, and the synthetic dataset explicitly varies noise and correlation strength. However, the theoretical component is not verifiable (proofs deferred to a missing supplement) and the key OOD bound relies on a circular assumption. More importantly, the NNR module is underspecified as written, and the headline empirical comparisons are based on best-over-seed numbers rather than averages. These issues undermine the central claims.","major_comments":[{"comment":"The NNR weight w_i = N_hom(G_i)/N_sum(G_i) is undefined as a method for graph classification. For graph-level classification, each G_i is an entire graph and there is no given neighborhood relation among the training graphs; the paper does not specify whether 'same-class neighbors within a certain distance' means neighbors in input space, in the GNN embedding space, or in label space, nor what distance function is used. Assumption 3 in Section 3.4 merely postulates that such near sets are disjoint and equal-sized without connecting to any concrete construction. As a result, Eq. (18) is not implementable from the description, and the reported NNR gains are not reproducible or falsifiable as written; if 'same-class' is defined using true labels, NNR would require oracle or unlabeled test information, and if it is defined using noisy training labels, it downweights precisely the clean minority samples the method is intended to protect.","section":"Section 3.3, Eq. (18)"},{"comment":"Assumption 5 postulates a sub-exponential tail bound on A = L_test - L_train conditioned on the norm of the classifier; this is exactly the generalization gap that Theorem 3 purports to bound. Since the theorem's conclusion is a bound on L(test) - L(train), and the only mechanism preventing A from being large is Assumption 5 itself, the argument is circular and does not establish that NNR or CMO controls OOD error. In addition, Eq. (25) is not a concrete bound: the right-hand side is an O(...) expression depending on terms involving C, w, mu, and g without specifying constants or conditions under which the sum is finite; it cannot be evaluated or verified.","section":"Section 3.4.2, Assumption 5 and Theorem 3"},{"comment":"Both convergence theorems have proofs only 'in the Supplementary material', but no supplementary material is provided with the manuscript, so the claims are unverifiable as submitted. Furthermore, Theorem 2 as stated, lim_{T→∞} ∑_{t=1}^T E[||∇θ L(θ_t,q_t)||] = 0, is not a valid convergence statement: because the summands are nonnegative, a limit of zero would force each gradient to vanish almost surely in a nontrivial problem; the intended statement is presumably about a Cesàro average or infimum of gradient norms, and the current form is false. Section 3.2 also states that Eq. (16) gives 'a rate of O(1/T)' while the displayed bound decays as 1/√T, and the constants G_θ and G_q in Eq. (16) are never defined.","section":"Section 3.2, Theorems 1 and 2"},{"comment":"The headline comparison '80.2% and 79.8% versus 71.6% and 74.7%' is based on the 'Max%' columns, i.e., the best result over the five random seeds, which is not a valid basis for claiming an improvement. The 'Average%' columns in the same table show that at noise level 0.2, ERM+NNR+CMO-KL achieves 59.3±0.8 and ERM+NNR+CMO-Chi achieves 58.5±0.7 minority accuracy, both below ERM+CMO-Chi at 61.6±0.5; at 0.15 the combined models are only marginally ahead (67.9 and 67.2 versus 67.8 and 67.5), with overlapping error bars. The all-class average accuracy of the combined models is also lower than plain ERM. Thus the central claim that CMO+NNR 'improve minority class robustness' is not supported by the averaged results, and no significance tests are provided. The same best-over-seed reporting appears in Table 4 for the real-world data.","section":"Section 4.5, Table 1 and Table 4"}],"minor_comments":[{"comment":"The f-divergence formula as written uses f_k(dP/dQ) inside the integral with respect to dP; for D_k(Q||P) the standard definition is ∫ f_k(dQ/dP) dP, so the argument of f_k appears to be inverted.","section":"Section 2.2, Eq. (6)"},{"comment":"Equation (20) uses C as the Lipschitz constant in the displayed inequality while the text says 'where L is the Lipschitz constant'; the notation is inconsistent.","section":"Section 3.4.1, Eq. (20)"},{"comment":"The notation for the number of same-class neighbors is inconsistent: the text defines N_hom(G_i) but earlier uses N_hm(G_i), and the relationship between the two is never clarified.","section":"Section 3.3"},{"comment":"There are several typographical issues: 'Eqal-Sized' in Assumption 3 should be 'Equal-Sized', and 'casual representation' in Assumption 1 should be 'causal representation'.","section":"Assumptions 1 and 3"},{"comment":"The sensitivity analysis reports results but does not specify the hyperparameters for the baseline methods or the ranges used for Γ, λ, and η_q; moreover, Figure 6 is largely unreadable in the manuscript due to encoding issues in the axes labels.","section":"Section 4.4 and Figure 6"}],"recommendation":"reject","confidential_remarks":"The manuscript is not in a publishable state: the main method is underspecified, the theory is circular or unverifiable, and the key empirical comparisons rely on best-seed selection. The combined method also does not consistently beat the best baseline in average minority accuracy according to Table 1, so the central claim would require new experiments and a substantially revised presentation to be salvageable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the combined CMO+NNR method consistently moves minority-class accuracy on DrugOOD, with average gains from 13% to 21% on assay and from 6% to 15% on scaffold. That is not just best-seed cherry-picking; the mean numbers improve, though overall accuracy dips slightly. Second, the paper's most load-bearing component, NNR, is never actually defined. Eq. 18 reweights each graph by the fraction of \"same-class neighbors within a certain distance,\" but the paper never says what distance, what representation, or what counts as a neighbor when each example is an entire graph. That makes the headline empirical claim unreproducible from the text.\n\nWhat is genuinely new is the combination: DRO with a class-mean proximity constraint plus local-consistency reweighting. I don't recall seeing that exact pair. The CMO idea is a natural extension of DRO, and the authors give a clear intuition for why the constraint helps in imbalanced classes. The experimental section is thorough, with synthetic data covering multiple noise rates and three DrugOOD splits, and the results are honestly annotated with variance.\n\nThe soft spots are real. Assumption 5 postulates a tail bound on A = test loss minus train loss, exactly the object Theorem 3 claims to bound. That is circular. All three theorems are deferred to a supplement that isn't in the preprint, so the theory line is unsupported as submitted. The \"Max%\" columns are best-of-five-seed selection; the averages help, but the text repeatedly leads with the max. The sensitivity analysis varies Gamma but never varies the neighbor definition, which is the actual unspecified choice.\n\nThis paper is for someone who wants a practical recipe for rare-class robustness in graph classification and is willing to inspect the code to fill in the gaps. As a scholarly artifact it needs major revision: specify the neighbor computation precisely, either prove the theorems without the circular assumption or cut the theory, and consistently report average rather than max. The empirical pattern is worth taking seriously, so I would not desk-reject it. Send it to referees, but they should be told the method description is incomplete and the theoretical claim needs hard scrutiny.","headline":"A useful empirical recipe for minority-class graph OOD, but NNR is underspecified and the theory's key assumption is circular; worth peer review if the method is pinned down.","tokens_in":17249,"tokens_out":4835,"would_cite":false,"duration_ms":43894,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Graph OOD minority-class accuracy rises with two ERM add-ons","keywords":["graph out-of-distribution learning","class imbalance","minority class accuracy","distributionally robust optimization","noise reweighting","graph neural networks","DrugOOD","PAC-Bayesian bound"],"falsifier":"Recompute the synthetic experiments with NNR's neighborhood defined in raw input space instead of embedding space; if the best maximum minority-class accuracy drops from 80.2% to near baseline, then the gain hinges on an unspecified distance choice rather than on the proposed mechanism.","tokens_in":16252,"feed_emoji":"📈","tokens_out":6355,"duration_ms":60437,"temperature":0.7,"pith_summary":"This paper argues that graph out-of-distribution classification fails hardest on rare classes when labels are imbalanced and structure is noisy, and that two simple add-ons to standard empirical risk minimization fix much of that failure. The first add-on, Constrained Mean Optimization (CMO), adds a class-similarity constraint to the worst-case (distributionally robust) objective; the second, Neighbor-Aware Noise Reweighting (NNR), down-weights samples whose local neighborhood is not label-consistent. On synthetic six-class data the combined method raises the best maximum minority-class accuracy from 71.6% and 74.7% to 80.2% and 79.8% at the two highest noise levels, and on the DrugOOD benchmark it lifts upper-bound minority accuracy to 47.2%, 34.9%, and 48.0% on the Assay, Scaffold, and Size splits. A reader should care because minority-class performance under shift and noise is a practical bottleneck for deploying graph models.","feed_headline":"Two tweaks lift graph OOD minority accuracy to 80 percent","feed_subtitle":"Adding class-similarity and noise-weight modules lifts rare-class accuracy on synthetic and DrugOOD.","key_machinery":"Two mechanisms carry the argument. CMO is a constrained optimization layer on top of distributionally robust optimization: it keeps the usual worst-case reweighting objective but adds the constraint $\\Delta(Q) = \\sum_{i<j} \\|\\mu(Q_i) - \\mu(Q_j)\\|^2 < \\rho_2$, the sum of squared distances between class-conditional feature means, so the worst-case distribution is chosen among assignments that respect class similarity. NNR is a per-sample loss weight $w_i = N_{\\mathrm{hom}}(G_i)/N_{\\mathrm{sum}}(G_i)$, the fraction of same-class neighbors within a graph distance $\\Gamma$, applied as $\\ell_i = w_i \\ell_i^{\\mathrm{raw}}$; it is supposed to make noisy samples, which have fewer same-class neighbors, contribute less. These are complemented by a Lagrange update for $q$ and $\\theta$, convergence theorems for convex and non-convex losses, and a PAC-Bayesian bound that expresses OOD error as three terms involving class centers, feature means, and neighbor ratios.","core_discovery":"The paper's central claim is that the minority-class drop in graph OOD classification is driven mainly by two correctable mechanisms—the model ignoring rare classes and the model overweighting noisy examples—and both can be corrected while keeping the standard ERM backbone. CMO regularizes the DRO update toward class distributions whose means are close, encoding the prior that hard classification cases are between similar classes; NNR multiplies each training loss by $w_i = N_{\\mathrm{hom}}(G_i)/N_{\\mathrm{sum}}(G_i)$, the fraction of nearby same-class neighbors, so isolated and likely noisy graphs contribute less. The authors report that on synthetic data the combined method reaches 80.2% and 79.8% maximum minority-class accuracy versus 71.6% and 74.7% for the best baselines, and on real-world DrugOOD data the upper-bound minority accuracy rises to 47.2% (Assay), 34.9% (Scaffold), and 48.0% (Size). They also provide convergence theorems for the CMO update and a PAC-Bayesian OOD generalization bound, arguing that the error bound decomposes into terms NNR's weights can control.","pith_inferences":["Beyond the paper's claims, a testable reading is that NNR's unspecified 'neighbor' relation can be pinned down as a nearest-neighbor query in the GNN embedding space; comparing Euclidean, cosine, and label-based neighborhoods would show which one reproduces the reported weights.","CMO's class-similarity prior is not graph-specific, so the same constraint could be tested on image and text classifiers, where class means are straightforward to estimate.","If NNR is pushed to higher noise levels, the assumption that noisy samples have fewer same-class neighbors may invert as labels themselves become random; tracking how NNR's weight distribution changes as the noise ratio crosses 0.2 would reveal where the method breaks."],"forward_implications":["If the claims hold, any ERM-trained graph classifier can be retrofitted with CMO and NNR without changing the backbone architecture.","At high noise ratios, maximum minority-class accuracy rises from 71.6% and 74.7% to 80.2% and 79.8%, making the model substantially more usable when rare classes matter.","The two modules are complementary: CMO stabilizes gains across settings, while NNR adds a noise-suppression effect that occasionally costs 0.2%–0.3% average accuracy.","On the DrugOOD benchmark splits, upper-bound minority accuracy improves to 47.2%, 34.9%, and 48.0% respectively, indicating the method transfers beyond synthetic data.","The theoretical sections imply the method converges at $O(1/T)$ for convex losses and to stationary points for non-convex losses, so the reported gains are not bought by unstable optimization."],"supporting_citations":[{"why":"Supplies the real-world DrugOOD benchmark datasets used to evaluate the method's generalization claims.","marker":"[9]"},{"why":"Provides the DORO DRO baselines and the CVaR/Chi-square variants that the paper's experiments must beat.","marker":"[28]"},{"why":"Supplies the group DRO formulation that CMO extends with its class-similarity constraint.","marker":"[18]"},{"why":"Provides the PAC-Bayesian framework used to derive the OOD generalization error bound for NNR.","marker":"[14]"},{"why":"Defines the Cressie-Read divergence family that underlies the divergence constraint in CMO.","marker":"[4]"},{"why":"Gives the optimal $O(1/\\epsilon^2)$ convergence rate for stochastic min-max problems that Theorem 1 claims to match.","marker":"[16]"}],"fun_headline_variants":["Rare-class graph OOD accuracy hits 80% via two fixes","Two modules push graph OOD minority accuracy to 80%","Graph OOD: fixing class imbalance and noise boosts rare-class scores","Minority graph OOD accuracy jumps to 80% with mean constraints and denoising","CMO+NNR: robust graph OOD with 80% minority accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that counting 'same-class neighbors' of a graph makes sense, yet it never defines what a neighbor of a graph is; if that count is meaningless or mistakes rare clean samples for noise, the reported gains disappear.","fun_headline_variants_meta":{"raw":{"variants":["Rare-class graph OOD accuracy hits 80% via two fixes","Two modules push graph OOD minority accuracy to 80%","Graph OOD: fixing class imbalance and noise boosts rare-class scores","Minority graph OOD accuracy jumps to 80% with mean constraints and denoising","CMO+NNR: robust graph OOD with 80% minority accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000861,"raw_usage":{"total_tokens":3739,"prompt_tokens":954,"completion_tokens":2785,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":570,"completion_tokens_details":{"reasoning_tokens":2686}},"tokens_in":570,"tokens_out":2785,"duration_ms":19047,"temperature":1.0,"reasoning_tokens":2686,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:34:05.425551+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the synthetic experiments with NNR's neighborhood defined in raw input space instead of embedding space; if the best maximum minority-class accuracy drops from 80.2% to near baseline, then the gain hinges on an unspecified distance choice rather than on the proposed mechanism.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the DORO DRO baselines and the CVaR/Chi-square variants that the paper's experiments must beat."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the PAC-Bayesian framework used to derive the OOD generalization error bound for NNR."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Cressie-Read divergence family that underlies the divergence constraint in CMO."}],"review_version":1}