{"id":"9ee44a93-6cc5-4a46-bce3-a1fef3c46f64","arxiv_id":"2412.10253","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":9,"one_line_summary":"GTDD combines GNN, Tree-LSTM, and Dueling-DQN to select SQL join orders, reporting lower costs than RTOS and JOGGER on JOB and TPC-H, but the comparison is weakened by modifying baselines to use its new reward function.","lead":"This paper introduces GTDD, a deep reinforcement learning framework for choosing SQL join orders, combining graph neural networks, tree-structured LSTMs, and a dueling DQN. It claims improved query plan costs and latencies over prior learned optimizers on standard benchmarks, but the experimental comparison has important caveats.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"GTDD's advantage over state-of-the-art is not established because §8.2 replaces RTOS/JOGGER's original reward with GTDD's log-ratio reward and no code is released; a fair comparison against published baselines is required.","rationale":"I read the paper as claiming that GTDD's integrated architecture beats published DRL optimizers. The load-bearing condition for that claim is that the experimental comparison reflects the published baselines. Section 8.2 explicitly violates this: 'all the related work employs the reward function we propose.' This turns the experiments into comparisons against re-implementations with GTDD's reward, not the state-of-the-art techniques as published. The missing code (reference [7] says 'available upon request') compounds the problem by making the modified baselines unauditable. The reader's identified weakest assumption, the action mask in Section 7.3, is a genuine limitation for disconnected query graphs, but for the connected JOB and TPC-H workloads the mask still permits all edge-respecting join trees, so it is less threatening. Since the central comparative claim cannot be assessed without a fair baseline comparison, I keep the reader's REJECT verdict rather than moving it.","tokens_in":17943,"tokens_out":5017,"duration_ms":603678,"concrete_test":"Run the original RTOS and JOGGER implementations (or exact published reward settings, network architectures, and hyperparameters; use official repositories if available) on the same 11-fold JOB train/test split described in §8.2.1, without substituting GTDD's log-ratio reward. Report MRC on the test templates using the final trained policy, not the best-episode value. If either baseline achieves MRC <= GTDD's 1.063 or GMRL <= 0.604, the central claim fails; if GTDD still wins by comparable margins, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 8.2 contains the sentence: 'For the sake of fairness, all the related work employs the reward function we propose.' This is the load-bearing problem. The paper's headline claim is that GTDD outperforms state-of-the-art techniques, but RTOS and JOGGER as evaluated are not the published RTOS and JOGGER: they are re-implementations using GTDD's log-ratio reward (Eq. 5), and likely other GTDD choices. The measured margins (MRC 1.224 vs 1.063; GMRL 0.833 vs 0.604) therefore conflate the reward function with the proposed GNN/Tree-LSTM/Dueling-DQN architecture. If the original reward designs and training protocols were used, those margins could shrink or reverse. Reference [7] says only that code is 'available upon request,' so there is no artifact that lets the reader verify whether the baselines were faithfully reproduced. The Section 7.3 action mask, in contrast, is not the principal threat: for connected query graphs (the JOB and TPC-H workloads here), every join tree can be built by repeatedly joining components connected by a query-graph edge, so forbidding Cartesian products does not preclude the optimal plan. The missing fair baseline comparison is what blocks the central claim.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GTDD, a deep reinforcement learning framework for join order selection that combines graph neural networks, Tree-LSTM, and Dueling-DQN, together with a log-ratio reward and curriculum learning. The authors evaluate GTDD on JOB and TPC-H, reporting that it outperforms the DRL-based baselines RTOS and JOGGER and performs comparably to PostgreSQL's dynamic programming optimizer. The central claim is that GTDD achieves a 'clear advantage' over state-of-the-art techniques in both estimated cost (MRC) and true latency (GMRL).","tokens_in":18290,"tokens_out":8412,"duration_ms":72911,"significance":"If the central claim were supported, GTDD would be a useful contribution to learned query optimization, since it integrates several representation-learning ideas (GNN, Tree-LSTM) with a relatively advanced RL algorithm (Dueling-DQN) and a normalization-based reward. The paper deserves credit for including an ablation of the RL component (GTD vs. GTDD) and for reporting both cost and latency metrics. However, the experimental validation has serious methodological flaws that undermine the claim: the baselines are re-implemented with the proposed reward function instead of their original reward designs, the reported MRC values are the best observed during training rather than the final policy's performance, and the action mask restricts the search space without justification. These issues are load-bearing for the paper's main conclusion.","major_comments":[{"comment":"The sentence 'For the sake of fairness, all the related work employs the reward function we propose' indicates that RTOS and JOGGER are not evaluated with their published reward functions but with the authors' log-ratio reward (Eq. 5). This does not establish that GTDD outperforms the state-of-the-art methods as published; it only shows that GTDD with its reward outperforms re-implemented variants that share that reward. The reported margins (e.g., MRC 1.063 vs. 1.224 in Table 1) may be due to the reward function rather than the architectural contributions. The authors must compare against the original RTOS and JOGGER implementations with their original reward functions, or clearly frame the current comparison as an ablation of the reward component.","section":"§8.2"},{"comment":"The MRC values reported in Table 1 are the 'Optimal MRC' values, i.e., the lowest MRC observed during training (as explicitly listed in Table 2), not the performance of the final policy at the end of training. This best-checkpoint reporting selects the most favorable evaluation point and overstates the methods' practical performance. Since these values are used as the headline numerical claims, the paper does not follow a standard evaluation protocol for RL agents. The authors should report the MRC of the final policy after a fixed training budget, or use a consistent protocol such as the average over the last K checkpoints.","section":"Tables 1 and 2"},{"comment":"The action mask prohibits joins that produce Cartesian products, restricting the agent to plans that only join components connected by an edge in the query graph. The paper does not justify this restriction. For the connected query graphs in JOB and TPC-H, an optimal plan may often be representable without cross products, but this is not guaranteed in general, and the paper does not provide an argument or empirical evidence that the mask never excludes the optimal plan for the tested workloads. The authors should either justify the restriction (e.g., by proving that for connected graphs with standard cost models, the optimal plan can always be expressed without Cartesian products) or evaluate on workloads where cross products are necessary.","section":"§7.3"},{"comment":"All reported results are from a single run without error bars, standard deviations, or multiple seeds. DRL training is notoriously noisy, and the observed differences (e.g., GTDD MRC 1.063 vs. RTOS 1.224) may not be statistically significant. The paper should report the mean and standard deviation over multiple independent runs, or at least provide learning curves for all runs that substantiate the claimed stability and convergence advantages.","section":"§8"}],"minor_comments":[{"comment":"There is a typo: 'quey' should be 'query'.","section":"§2.1"},{"comment":"The word 'presentation' appears to be a typo for 'representation' in several places, including the list of components and the text describing state representation.","section":"§4"},{"comment":"The paper first states that 90% of queries are used for training and 10% for testing, but Section 8.2.1 describes an 11-fold cross-validation with template-based splits. The relationship between these two evaluation protocols should be clarified.","section":"§8.1.2"},{"comment":"Reference [7] states only that the code is 'available upon request.' A public repository would support reproducibility and allow reviewers to verify the experimental setup; 'available upon request' is not sufficient for a scientific publication.","section":"References [7]"},{"comment":"The paper does not specify many hyperparameters used for GTDD or the baselines, such as hidden size, learning rate, batch size, replay buffer size, target network update frequency, exploration schedule, and curriculum parameters (k, I). These details are essential for reproduction.","section":"§8.1.1"},{"comment":"Several figures (especially Figure 3) are low-resolution and difficult to read; higher-quality versions should be provided.","section":"Figures"}],"recommendation":"major_revision","confidential_remarks":"The framework is interesting and the ablation of the RL component is a strength, but the experimental comparison as written does not support the central claim. The baseline issue and the best-so-far reporting are substantial, though fixable with additional experiments and re-analysis. If the authors can provide a fair comparison against the original published baselines and adopt a standard evaluation protocol, the paper could become acceptable. Otherwise, the manuscript may not meet the standard for the journal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper proposes GTDD, a DRL join-order optimizer that combines GNN query embeddings, Tree-LSTM join-tree encodings, Dueling-DQN, and a log-ratio reward relative to the DP plan. The reward idea is genuinely useful: it normalizes cost feedback across queries with wildly different magnitudes, which is a real pain point in this area. The ablation against GTD (same framework with plain DQN) is the right way to isolate the Dueling-DQN contribution, and the numbers on JOB and TPC-H look competitive when compared against PostgreSQL's DP.\n\nThe headline claim, however, is not established by this write-up. Section 8.2 says flatly that \"for the sake of fairness, all the related work employs the reward function we propose.\" That means RTOS and JOGGER as tested are not the published RTOS and JOGGER; they are re-implementations with GTDD's reward, and possibly other GTDD choices. The reported margins (MRC 1.22 vs 1.06; GMRL 0.83 vs 0.60) therefore conflate the reward function with the architecture. If the original rewards and training protocols were used, those gaps could shrink or reverse. Without that comparison, the results only show that GTDD with its own reward beats variants that use the same reward, which is a much weaker claim.\n\nTwo secondary issues. First, Table 2 reports the \"optimal MRC\" as the lowest value observed during training, not the final policy's MRC. That is cherry-picking the best moment on the training curve. The training curves do suggest GTDD is more stable, but the \"optimal\" column overstates its typical performance. Second, there is no code. Reference [7] says the code is available on request, which is a serious reproducibility gap for an empirical ML systems paper.\n\nThe action mask concern in the reader's report is a minor point. For the connected query graphs in JOB and TPC-H, every join tree can be built without Cartesian products, so the mask does not exclude optimal plans. It is a limitation worth stating, not a fatal flaw.\n\nWho gets value from this paper? People working on learned query optimization will find the reward normalization and the ablation design worth reading. The framework itself is a reasonable combination of known components. But as it stands, the central comparison is not trustworthy. A serious referee should see it because the problems are fixable: a fair comparison against the original baselines plus released code would let the results speak. I would not cite it in its current form, but I would encourage the authors to resubmit after fixing the evaluation.","headline":"Plausible DRL join-order framework with a useful reward idea, but the headline empirical claim is undercut by modified baselines and no code.","tokens_in":18776,"tokens_out":2867,"would_cite":false,"duration_ms":25654,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a deep reinforcement learning agent can learn SQL join orders that match or beat a database's dynamic-programming optimizer, by encoding the query graph and partial join tree and training with Dueling-DQN and a…","keywords":["join order selection","deep reinforcement learning","Dueling-DQN","Tree-LSTM","graph neural networks","query optimization","curriculum learning","Join Order Benchmark"],"falsifier":"Run GTDD on a query set that includes cyclic join graphs where exhaustive dynamic programming, with cross joins allowed, shows that the cheapest plan uses an intermediate Cartesian product—for example, a three-table cycle with a selective cross join between non-adjacent tables. Because GTDD's action mask omits such joins, its best expressible plan will be worse than the unrestricted DP plan, so its MRC on those queries will exceed 1 and the claim of DP-level plan quality would fail exactly where the mask is active.","tokens_in":17801,"feed_emoji":"🧠","tokens_out":19241,"duration_ms":162777,"temperature":0.7,"pith_summary":"Join order selection is the part of query optimization that decides the sequence in which a query's tables are joined, and the search space grows factorially, so optimizers trade plan quality against enumeration cost. The paper's claim is that a deep reinforcement learning agent, GTDD, can learn this choice by representing the query's join graph with a graph neural network (GNN), representing the partially built plan as a forest of join trees with Tree-LSTM, and training a Dueling-DQN with curriculum learning and a log-ratio reward. In the reported experiments on the 21-table JOB workload, GTDD reaches a mean relative cost of 1.06 against PostgreSQL's dynamic-programming plans and, after latency tuning, a geometric mean relative latency of 0.60, ahead of the prior DRL-based systems RTOS and JOGGER. If correct, this matters because a learned optimizer could adapt to real execution feedback and approach exhaustive-optimizer plan quality without enumerating the search space.","feed_headline":"Learned join-order policy beats prior deep-RL methods, matches PostgreSQL","feed_subtitle":"On 21-table JOB queries, the GTDD learner reaches 1.06 relative cost and 0.60 latency vs the database's best plans","key_machinery":"The load-bearing machinery is the state encoder paired with a normalized reward. For each query, a GNN over the join-graph adjacency matrix produces the query representation $R(q)$; the current partial plan is a forest of join trees, encoded by n-ary Tree-LSTM for each tree (preserving the order in which joins were chosen) and child-sum Tree-LSTM for the whole forest (independent of how many trees remain), giving $R(F)$. The state is the concatenation $R(s) = R(q) \\oplus R(F)$, fed into Dueling-DQN, where the value stream estimates the worth of being in the state and the advantage stream estimates the relative worth of each action before the two are combined. The terminal reward is $r = \\log(DP_{\\text{feedback}} / GTDD_{\\text{feedback}})$, compressing feedback from queries with very different cost magnitudes into a comparable scale. Curriculum learning sorts training queries by join count and adds harder partitions over time, and an action mask sets to zero any join that does not correspond to an existing join predicate. The framework's argument is that these pieces jointly give the agent a state representation that reflects both the query and the plan built so far, a reward that is comparable across heterogeneous queries, and an exploration space that excludes obviously invalid actions.","core_discovery":"The paper's central claim is that combining hierarchical plan representation with an improved deep-Q architecture yields better learned join orders than either ingredient alone. GTDD encodes a query at column, table, and query-graph levels—columns carry predicate and selectivity features, tables combine column embeddings with Node2Vec schema embeddings, and the query graph passes through a TransformerConv GNN—and encodes the current partial plan as a join forest using n-ary Tree-LSTM for each join tree and child-sum Tree-LSTM for the whole forest. That state feeds Dueling-DQN, which estimates state value and action advantage in separate streams. The reward is the log ratio of the DP baseline's cost (or latency) to GTDD's own, so queries of very different scales are scored on a common relative basis, and curriculum learning introduces queries with more joins only after simpler ones. On this setup the paper reports a mean relative cost (MRC) of 1.06313 for GTDD on JOB versus 1.13165 for the same framework with vanilla DQN (GTD), 1.22442 for RTOS, and 1.54444 for JOGGER; after latency tuning, the geometric mean relative latency (GMRL) is 0.60381 for GTDD versus 0.80425, 0.83313, and 1.16758 respectively, and 0.96399 on TPC-H. The authors take these numbers as showing that Dueling-DQN stabilises training and that the combination, not any single component, outperforms prior DRL join-order selection.","pith_inferences":["The paper leaves implicit that its action mask bans Cartesian-product joins; a testable extension is to relax the mask on cyclic or star-schema queries whose optimal plans may need a cross join, and check whether the log-ratio reward still converges to the unrestricted DP plan.","The reward requires a DP baseline for every training query, which is the expensive computation a learned optimizer is meant to avoid in deployment; substituting a cheaper heuristic or learned baseline as the denominator would measure how much of the reported margin survives without oracle feedback.","The saturation of all methods on TPC-H suggests the advantage is concentrated in large search spaces, so one prediction is that on schemas with more tables or cyclic join graphs the gap over attention-based JOGGER widens unless the action mask becomes the binding restriction.","The reported similarity between GTD and RTOS in final GMRL (0.804 vs 0.833), despite very different state encoders, suggests the dueling architecture rather than the GNN may be the main driver of latency improvement; separately ablating GNN and Tree-LSTM under the same RL algorithm would isolate the contributions."],"forward_implications":["A single learned policy can produce plans with DP-level estimated cost on a workload with up to 21 joins, meaning exhaustive enumeration is not necessary once the representation and reward are in place.","The same framework with vanilla DQN (GTD) is measurably worse than GTDD, so the choice of Dueling-DQN's separated value and advantage streams contributes to plan quality, not just training speed.","Cost-trained models transfer to latency tuning: GTDD's plans are about 40% lower in geometric-mean relative latency than DP plans on JOB, so a learned optimizer can be steered by real execution feedback rather than only by cost estimates.","The log-ratio reward makes one reward scale work for queries with very different costs, which is a prerequisite for training a curriculum from 3-join to 21-join queries in a single agent.","On the 8-table TPC-H schema, every learned method is close to DP, indicating the approach pays off where the join search space is large; that is the regime where traditional enumeration is most expensive."],"supporting_citations":[{"why":"RTOS supplies the Tree-LSTM join-forest representation and DQN baseline that GTDD is directly compared against in cost and latency experiments.","marker":"[24]"},{"why":"JOGGER is the self-attention-tree DRL baseline with curriculum learning that GTDD compares against on cost and latency.","marker":"[5]"},{"why":"Dueling-DQN is the architecture that separates value and advantage streams, the central reinforcement-learning change GTDD claims improves convergence and plan quality.","marker":"[22]"},{"why":"Tree-LSTM provides the n-ary and child-sum encoders that carry the join-tree and join-forest state representations.","marker":"[21]"},{"why":"TransformerConv is the GNN layer that produces the query-graph representation concatenated into each state.","marker":"[18]"},{"why":"Invalid-action masking is the mechanism that restricts the action space to existing join predicates and removes Cartesian products.","marker":"[8]"},{"why":"JOB is the 21-table workload with up to 21 joins where GTDD reports its largest margin over the baselines.","marker":"[11]"},{"why":"TPC-H is the 8-table workload where all methods approach dynamic-programming cost, bounding the claim to larger search spaces.","marker":"[16]"},{"why":"Dynamic programming is the plan-quality baseline whose feedback appears in the reward ratio and in the MRC and GMRL comparisons.","marker":"[1]"}],"fun_headline_variants":["GTDD deep-RL join-order framework beats prior methods on JOB","Hybrid GNN and Tree LSTM yields near-optimal join orders","Dueling-DQN join-order selection outperforms JOGGER and RTOS","Graph-aware deep RL for join order selection hits 1.06x DP cost","GTDD: Learned join-order policy with 1.06 relative cost on JOB"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The learned policy can only join tables that are connected by an explicit join condition in the query, so a plan that needs an intermediate cross join of unrelated tables can never be produced; if any query's optimal plan requires such a cross join, GTDD's search is restricted before learning begins.","fun_headline_variants_meta":{"raw":{"variants":["GTDD deep-RL join-order framework beats prior methods on JOB","Hybrid GNN and Tree LSTM yields near-optimal join orders","Dueling-DQN join-order selection outperforms JOGGER and RTOS","Graph-aware deep RL for join order selection hits 1.06x DP cost","GTDD: Learned join-order policy with 1.06 relative cost on JOB"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000227,"raw_usage":{"total_tokens":1523,"prompt_tokens":1046,"completion_tokens":477,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":662,"completion_tokens_details":{"reasoning_tokens":373}},"tokens_in":662,"tokens_out":477,"duration_ms":5282,"temperature":1.0,"reasoning_tokens":373,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T16:01:02.975355+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run GTDD on a query set that includes cyclic join graphs where exhaustive dynamic programming, with cross joins allowed, shows that the cheapest plan uses an intermediate Cartesian product—for example, a three-table cycle with a selective cross join between non-adjacent tables. Because GTDD's action mask omits such joins, its best expressible plan will be worse than the unrestricted DP plan, so its MRC on those queries will exceed 1 and the claim of DP-level plan quality would fail exactly where the mask is active.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Dynamic programming is the plan-quality baseline whose feedback appears in the reward ratio and in the MRC and GMRL comparisons."}],"review_version":1}