{"id":"d9b2b7e3-49ef-46d6-8b0d-a021df19884c","arxiv_id":"2601.20753","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"GraphAllocBench provides a graph-based resource-allocation benchmark and two supplemental metrics that expose preference-consistency failures in multi-objective RL policies.","lead":"GraphAllocBench introduces a customizable \"city planning\" simulator for testing preference-conditioned multi-objective reinforcement learning, along with metrics for reliability and preference alignment. The paper shows current algorithms fail in ways hypervolume alone misses and that graph networks scale better than MLPs on large allocation tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Ordering Score assigns perfect score to constant reward sequences, so preference-ignoring policies can receive high OS; this undermines the claim that OS reveals preference consistency.","rationale":"The reader identified exactly the load-bearing weakness: the Ordering Score's handling of constant reward sequences contradicts its stated purpose of measuring preference consistency. I read the paper in good faith: GraphAllocBench is a useful environment contribution, the problem suite is diverse, and the HGNN results are suggestive. However, the OS flaw is not cosmetic—it affects the interpretation of a headline metric and the central claim that the new metrics expose failure modes. The proposed test would settle whether the flaw actually inflates reported numbers. Because the reader already recommends CONDITIONAL and this concern is fixable, the verdict should remain unchanged; no stronger action is warranted without evidence that the benchmark itself is unsound.","tokens_in":15969,"tokens_out":2104,"duration_ms":23997,"concrete_test":"Run a trivial constant-action policy (e.g., always allocate the same fixed vector, independent of the sampled preference) on Problem 1c and compute its Ordering Score with Algorithm 1. If the score is near 1 while the preference varies and the objective values stay constant, the metric is confirmed broken. Additionally, recompute the Problem 1c Ordering Scores in Table 4 after replacing the line 5–6 special case with s_i,j = 0 (or by dropping constant sequences); if the OS drops materially (e.g., > 0.1), the reported preference-awareness evidence is inflated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The Ordering Score (Algorithm 1, Appendix B) is one of two proposed supplementary metrics and is central to the claim that GraphAllocBench reveals preference-following failures. Lines 5–6 set s_i,j = 1 whenever all values in the reward sequence J_i are equal. This is not a benign convention: Spearman rank correlation is undefined for a constant sequence, and assigning it a perfect positive score means a policy that outputs the same objective value regardless of the preference vector is credited with perfect preference alignment along that dimension. In sparse-reward problems such as Problem 1c (J_i = max(0, P_j − 5) floor functions), many sampled preferences produce zero reward, so constant sequences are common. The high OS values reported for Problem 1c (0.945 for PCPL, 0.989 for PD-MORL in Table 4) may therefore be artifacts of the special case rather than evidence of preference-awareness. Since Section 4.3 explicitly states that OS 'evaluates how well the agent follows the preference it is given,' and the abstract credits the supplementary metrics with exposing failure modes that hypervolume misses, this definitional flaw directly undercuts the paper's central claim. The issue is fixable—constant sequences should be assigned correlation 0, or excluded—but as written the metric is not a valid measure of preference consistency.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces GraphAllocBench, a Gymnasium-based benchmark built on CityPlannerEnv, a graph-structured resource allocation environment with customizable objective functions, preference vectors, and dependency graphs. It proposes two supplementary evaluation metrics beyond hypervolume: the Proportion of Non-Dominated Solutions (PNDS) and the Ordering Score (OS), intended to measure prediction reliability and preference consistency in preference-conditioned multi-objective reinforcement learning (PCPL). The experimental section compares a PPO-based PCPL agent with MLP and HGNN feature extractors against PD-MORL on problems 0–5, and MLP versus HGNN on large-scale problems 6a–c. The central claims are that GraphAllocBench exposes failure modes (sharp rewards, sparse rewards, non-convex Pareto fronts, unbalanced objectives, local optima) that hypervolume alone does not capture, and that HGNN architectures improve hypervolume on large graphs. The authors provide a GitHub link and detailed environment definitions, which is a strength for reproducibility.","tokens_in":16303,"tokens_out":5933,"duration_ms":63975,"significance":"If the benchmark and its metrics are sound, GraphAllocBench addresses a real gap in MORL evaluation: most existing benchmarks are small grid or continuous test problems without the flexible, graph-structured allocation structure needed to stress-test preference-conditioned policies. The code release and customizable environment are valuable assets to the community. The attempt to supplement hypervolume with preference-consistency diagnostics is timely and relevant for PCPL. However, the OS metric, which is central to the preference-consistency claim, currently treats constant reward sequences as perfectly preference-aligned, which can reward preference-ignoring policies. In addition, the empirical tables largely omit variance information, weakening the support for the headline comparisons. With a corrected OS definition and proper uncertainty reporting, the benchmark could become a useful testbed; as written, the central claim about preference-awareness is not yet established.","major_comments":[{"comment":"Lines 5–6 of Algorithm 1 set s_i,j = 1 whenever all values in the reward sequence J_i are equal. Spearman rank correlation is undefined for a constant sequence; assigning it a perfect positive score means that a policy producing the same objective value regardless of the preference vector is credited with perfect preference alignment along that dimension. This directly contradicts Section 4.3's statement that OS evaluates 'how well the agent follows the preference it is given.' In sparse-reward settings such as Problem 1c (and in any problem with floor/zero regions), constant zero-reward sequences are common, so the high OS values in Table 4 (e.g., 0.989 for PD-MORL and 0.945 for PCPL on Problem 1c) may be artifacts of this special case rather than evidence of preference-awareness. Since the abstract and Section 4.3 credit the supplementary metrics with exposing preference-following fail","section":"Appendix B, Algorithm 1; Section 4.3"},{"comment":"The empirical claims are presented as 'significantly and consistently better' and 'best values,' but Table 4 reports no standard deviations or significance tests for any metric across the 5 seeds. Table 2 reports mean±std only for hypervolume, while PNDS and OS are given as bare means, with duplicated/garbled entries in the table body. Without variance information across seeds, the differences highlighted in the text (e.g., HGNN vs MLP hypervolume, PD-MORL vs PCPL ordering score) cannot be assessed. Please report full mean±std for every metric and seed, and either include appropriate significance tests or explicitly label the comparisons as exploratory.","section":"Tables 2 and 4"},{"comment":"The text describes Problem 1c as involving the floor function J_i(P) = max(0, P_j − 5), but Table 3 lists logistic-type objective functions for Problem 1c. This discrepancy matters because the OS constant-sequence artifact is most acute in sparse-reward problems where many preferences yield zero reward. Please clarify which problems actually contain constant-zero reward regions, connect the metric discussion to the concrete objective definitions, and update the text or table accordingly.","section":"Section 5.1, 'Sparse Rewards'"}],"minor_comments":[{"comment":"The table body has formatting errors: repeated values such as '0.880.88', '0.150.15', and '12.9±1.512.9±1.5' appear, and missing spaces separate columns. Please regenerate cleanly.","section":"Table 2"},{"comment":"The abstract and introduction refer to experiments with 'several state-of-the-art PCPL algorithms,' but the paper compares against only PD-MORL plus the authors' own baselines. Please rephrase or add additional PCPL baselines.","section":"Abstract and Section 1"},{"comment":"Section 4.3 defines OS as rank correlation between preference weight w_i and objective value J_i, but Algorithm 1 computes Spearman(J_i, sorted(J_i)). If the sweep index is intended to be a monotonic proxy for w_i, this should be stated explicitly; otherwise the computation should directly use w_i.","section":"Algorithm 1 vs. Section 4.3"},{"comment":"The Smooth Tchebycheff scalarization hyperparameter is mentioned as tunable, but no values or selection procedure are provided. Please include the hyperparameter settings used in the experiments.","section":"Appendix C.3"},{"comment":"The main text gives a GitHub URL while Appendix D points to an anonymous OpenReview-style link. These should be reconciled, and the final version should use a single persistent code link.","section":"Appendix D / Code Links"},{"comment":"The objective functions in Problem 0 use an undefined epsilon constant to avoid log(0). Please define epsilon and its value.","section":"Table 3"}],"recommendation":"major_revision","confidential_remarks":"The OS flaw is the key blocking issue; it directly undermines the paper's central claim that the supplementary metrics reveal preference consistency. The benchmark and environment are potentially valuable, and the fix is local (Algorithm 1 and re-run of OS values), so major revision rather than rejection seems appropriate. The missing variance in Tables 2 and 4 also needs to be addressed to support the empirical comparisons."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I read this one so you don't have to, and the short version is: the environment and problem suite are worth taking seriously, but one of the two new metrics is broken as written.\n\nThe actual contribution is the CityPlannerEnv sandbox and the GraphAllocBench problem suite. That fills a real gap—the MORL/PCPL world is stuck on 2D grid tasks like DST or FTN, and having a scalable, graph-structured allocation testbed with 100-demand instances is a meaningful step. The HGNN comparison is a nice empirical result, especially showing that attention pooling helps hypervolume with fewer parameters than the MLP. The code is linked, so the reproducibility package is there.\n\nNow the soft spots, in proportion. The ordering score algorithm in Appendix B has a genuine flaw: when the reward sequence for an objective is constant across the preference sweep, it assigns a perfect score of 1. Spearman correlation is undefined for a constant series, so they special-cased it—but the special case is wrong. A preference-immune policy that outputs the same allocation no matter what would get a high ordering score. In a sparse-reward problem like 1c, constant zero sequences are common, so the high OS numbers in Table 4 for that problem are likely artifacts. The paper's claim that the supplementary metrics reveal preference-consistency failures is undercut by this. The fix is trivial—set that case to 0 or discard it—but as is, the metric is not a valid measure of what the paper says it measures.\n\nAlso worth noting: Tables 2 and 4 report means without standard deviations for PNDS and OS, which makes it hard to judge significance. The PD-MORL baseline is admittedly not fully tuned, which is fine but limits the strength of the comparison. And PNDS is basically a known non-domination ratio; calling it new is a stretch, though it's still a reasonable thing to report.\n\nOverall, the benchmark is the contribution, not the metrics. The OS bug is fixable and doesn't sink the environment or the problem suite. I'd take this to a reading group—the metric flaw is a good discussion point—and I'd cite the benchmark in my own work, provided the authors fix the OS before publication. This deserves peer review; a good referee would ask for the fix, not a desk rejection.","headline":"A genuinely useful new benchmark environment, but the ordering score has a real bug that needs fixing before the metric can be trusted.","tokens_in":16769,"tokens_out":1722,"would_cite":true,"duration_ms":21313,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper introduces GraphAllocBench, a graph-based benchmark for preference-conditioned multi-objective policy learning, and argues that evaluating PCPL policies requires metrics beyond hypervolume — specifically an ordering score for pref","keywords":["Preference-conditioned policy learning","Multi-objective reinforcement learning","Benchmark","Resource allocation","Graph neural networks","Hypervolume","Ordering score","Pareto front"],"falsifier":"Compute the Ordering Score on a deliberately preference-blind policy — one that always outputs the same allocation regardless of preference — on Problem 1c (sparse rewards). If its OS approaches 1 while its hypervolume is near zero, the metric fails to punish preference-ignoring behavior, contradicting the paper's stated purpose.","tokens_in":15924,"feed_emoji":"📊","tokens_out":3898,"duration_ms":37790,"temperature":0.7,"pith_summary":"The paper argues that existing benchmarks for preference-conditioned multi-objective reinforcement learning are too small and rigid to reveal how well a single policy follows arbitrary user preferences. To fix this, it builds CityPlannerEnv, a configurable graph-based resource allocation sandbox, and packages it as GraphAllocBench with problems that vary objective shapes, Pareto fronts, dependency structures, and scale. It introduces two evaluation metrics — the proportion of non-dominated solutions (PNDS) and the ordering score (OS) — and shows experimentally that these expose failure modes that hypervolume alone misses, such as preference-ignoring behavior in sparse-reward and non-convex settings. It also shows that graph neural network feature extractors beat MLPs on the large 100-node problems, suggesting GNNs are a promising direction for high-dimensional allocation tasks.","feed_headline":"New benchmark exposes failure modes hypervolume alone cannot catch","feed_subtitle":"A graph-based task suite for preference-conditioned RL adds two metrics that expose poor preference-following hidden by hypervolume.","key_machinery":"CityPlannerEnv, a Gymnasium environment where an agent allocates units of resources to demands through a bipartite dependency graph, with objective functions defined over productions; the benchmark's evaluation protocol: Das-Dennis preference sampling, hypervolume ratio against analytically enumerated Pareto fronts where possible, PNDS, and Ordering Score (Spearman rank correlation between preference weights and objective values). The HGNN feature extractor with preference conditioning at node embeddings, residual connections, and attention pooling is the mechanism that scales to 100-demand graphs.","core_discovery":"On GraphAllocBench, hypervolume ratio alone gives an incomplete picture of a PCPL agent's quality. The paper's central discovery is that adding two cheap metrics — the proportion of non-dominated predicted solutions (PNDS) and the ordering score (OS), a Spearman rank correlation between preference weights and realized objective values — reveals preference-consistency failures that hypervolume hides, and that on large graph-structured allocation problems a heterogeneous graph neural network (HGNN) with preference-conditioned attention pooling achieves substantially higher hypervolume than an MLP with comparable or fewer parameters.","pith_inferences":["The ordering score's degenerate handling of constant reward sequences (score 1 when all values equal) may inflate the apparent preference-following of policies that simply ignore preferences in sparse-reward problems; a corrected metric or a robustness check with constant policies is a natural follow-up.","The PNDS metric could be combined with a diversity measure to distinguish 'many non-dominated but clustered' from 'many non-dominated and spread-out' predictions.","The benchmark's bipartite allocation structure could be extended to stochastic events or risk-aware objectives, which the authors list as future work, making it a testbed for uncertainty-aware PCPL.","One could test whether the ordering score's Spearman approach is robust to monotonic transformations of the scalarization, since PCPL policies may produce objective vectors that are monotone in preference but not linearly."],"forward_implications":["If the ordering score is taken seriously, PCPL benchmarks should report preference-following behavior, not just front quality.","The failure modes identified (sharp reward changes, sparse rewards, non-convex fronts, unbalanced objectives, local optima) give concrete targets for new PCPL algorithms.","GraphAllocBench's customizability means researchers can construct stress tests tailored to their algorithms.","HGNN-based feature extractors are a viable replacement for MLP extractors on large allocation graphs, achieving higher hypervolume with fewer parameters.","The scaling study with 20 objectives suggests that simple increasing objective functions may not degrade performance with more objectives, contrasting with typical evolutionary benchmarks."],"fun_headline_variants":["Two new metrics reveal preference failures hypervolume misses","GraphAllocBench: test preference-conditioned RL on graph tasks","Hypervolume hides poor preference-following; new metrics reveal it","New benchmark and metrics for preference-aware multi-objective RL"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The Ordering Score assumes that a constant reward sequence indicates perfect preference-following (it awards a score of 1), so a policy that produces zero or constant output for a sparse objective receives full credit for preference consistency in that dimension.","fun_headline_variants_meta":{"raw":{"variants":["Two new metrics reveal preference failures hypervolume misses","GraphAllocBench: test preference-conditioned RL on graph tasks","Hypervolume hides poor preference-following; new metrics reveal it","New benchmark and metrics for preference-aware multi-objective RL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000963,"raw_usage":{"total_tokens":3952,"prompt_tokens":772,"completion_tokens":3180,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":516,"completion_tokens_details":{"reasoning_tokens":3111}},"tokens_in":516,"tokens_out":3180,"duration_ms":25302,"temperature":1.0,"reasoning_tokens":3111,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T07:12:32.271957+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the Ordering Score on a deliberately preference-blind policy — one that always outputs the same allocation regardless of preference — on Problem 1c (sparse rewards). If its OS approaches 1 while its hypervolume is near zero, the metric fails to punish preference-ignoring behavior, contradicting the paper's stated purpose.","supporting_citations":[],"review_version":1}