{"id":"d143693f-9bd6-4b1a-a8c7-d49d65238125","arxiv_id":"2505.06321","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"L2T adapts LLM reasoning step by step by graph-labeling thoughts and using a reinforcement-learned GNN to tune prompts and sampling settings.","lead":"This paper introduces L2T, a reasoning framework that builds a graph from an LLM's generated thoughts, asks the LLM to label each thought as continue, stop, finish, or backtrack, and uses a small graph neural network trained with reinforcement learning to adjust prompts and sampling settings. The authors report consistent accuracy gains over chain, tree, and graph reasoning baselines on Sudoku, the Game of 24, and logic puzzles.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unreported GNN training/evaluation split makes the 'without additional training' claim unverifiable; the missing L2T row in Table 4 leaves the only open-ended task unsupported.","rationale":"Good-faith reading: the method is plausible, and the three automatically scored tasks are strong. L2T w/o GNN already outperforms strong baselines, independently supporting the value of generic graph-structured self-annotation. The concern is not that results are fabricated; it is that the paper's broadest claim is not yet supported by the reported setup. The training/evaluation split determines whether the GNN module is a transferable meta-controller or a per-task optimizer, and the paper gives no training-data description, no eval split, and no code verification. The Creative Writing table's missing reference row makes the open-ended result uncheckable. These are load-bearing: if the controller was trained on the evaluation instances, the comparison is unfair and the central claim fails; if the controller was trained on disjoint data, the claim could survive. The concrete test will settle this. The reader's CONDITIONAL verdict therefore stands, with an added condition: report the training/evaluation split and supply the missing Table 4 row with an external judge for Creative Writing.","tokens_in":20314,"tokens_out":11290,"duration_ms":120250,"concrete_test":"Inspect the released code and training logs (https://github.com/zch65458525/L2T) to map the exact instances used in the 20 PPO epochs for each task; if the Game 24 games indexed 901-1000, the 13 Sudoku sets, the TruthQuest character sets, or the Creative Writing prompts reported in Tables 1-4 were used for controller training, the 'without additional training' claim is invalidated.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The abstract claims L2T improves reasoning 'without requiring additional training or task-specific prompt design.' The LLM is not fine-tuned, but the GNN controller is trained with PPO (Section 3.3) over 20 epochs (Appendix A.2). Nowhere does the paper state which problems generated the RL trajectories. If those trajectories came from the same Game of 24, Sudoku, TruthQuest, or Creative Writing instances later reported in Tables 1-4, then L2T is task-specifically optimized, and the comparison against zero-shot/few-shot baselines is not a test of a task-agnostic controller; the headline claim would be false literally and much weaker substantively. This is compounded by Section 3.3's reward r_k, which is the same GPT-4o's self-score, so the controller may be trained to satisfy the LLM judge rather than task correctness. Table 4 also omits the L2T row while reporting all percentages 'to L2T', so the only open-ended task cannot be independently checked from the paper; L2T w/o GNN is reported as sometimes scoring higher than L2T, which is unexplained if the GNN module is the contribution.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes L2T, a framework that represents an LLM's step-by-step reasoning as a graph, uses the LLM itself to classify each thought into one of four states (stop, continue, complete, backtrack), and introduces a GNN-based module that selects reasoning modes (branch count, temperature, top-p) during inference. The GNN module is trained with PPO. The authors claim that L2T improves reasoning performance across Sudoku, Game of 24, TruthQuest, and Creative Writing without task-specific prompts and without additional LLM training, and they compare against IO, CoT, ToT, GoT, and AoT baselines. The paper also reports ablations (L2T w/o GNN, L2T w/o RL, L2T w/ MLP) and token-consumption analyses.","tokens_in":20460,"tokens_out":5962,"duration_ms":61618,"significance":"If the claims hold, the method is a meaningful contribution: it offers a unified reasoning scaffold that adapts its search strategy online, and the objective results on Sudoku and Game of 24 show large improvements over structured prompting baselines. The inclusion of the L2T w/o GNN ablation is a strong feature, because it isolates the contribution of the graph-based thought-management mechanism from the learned GNN module. The paper also releases code and provides token-cost comparisons. However, the headline claim of working 'without additional training' is not supported as stated, because the GNN controller is trained with PPO, and the paper does not disclose the training/evaluation split that would establish task-agnostic behavior. The Creative Writing evaluation is also incomplete because Table 4 omits the L2T row.","major_comments":[{"comment":"The abstract claims that L2T improves reasoning 'without requiring additional training or task-specific prompt design,' but Section 3.3 and Appendix A.2 describe a PPO-trained GNN-based controller trained over 20 epochs. The paper never states which problems or data generated the reinforcement-learning trajectories. If those trajectories come from the same Sudoku, Game of 24, TruthQuest, or Creative Writing instances that are later reported in Tables 1-4, then the controller is task-specifically optimized and the comparison against zero-shot and few-shot baselines does not test a task-agnostic method. The authors must disclose the exact training/evaluation split and clarify in the abstract and intro that 'no additional training' refers only to the LLM, not to the GNN controller.","section":"Abstract; Section 3.3; Appendix A.2"},{"comment":"Table 4 omits the L2T row even though every percentage is defined relative to L2T ('Higher'/'Same'/'Lower'). Consequently, the reader cannot directly verify the text's claim that 'L2T achieves higher or equivalent scores in over 80% of cases' for the comparison against L2T w/o GNN; for the Sentence Formation (Less Hints) column, L2T is lower than L2T w/o GNN in 34.11% of cases and higher in only 15.05%. This pattern is not discussed and appears to contradict the paper's implication that the GNN module improves open-ended generation quality. Please add the L2T row (or an equivalent reference) and explain the relationship between L2T and L2T w/o GNN on Creative Writing.","section":"Table 4; Section 4.1 Creative Writing results"},{"comment":"The four-way node classification that drives the whole reasoning process is performed by the same GPT-4o model that generates the thoughts, and the reward r_k used to train the GNN is an integer from 0 to 10 assigned by that same LLM based on the LLM-generated evaluation criteria Xeva. The paper never measures the classification accuracy of Y_v against any ground truth, nor does it measure how well r_k correlates with final task success. If the node classifier is unreliable, or if r_k rewards the LLM's stylistic preferences rather than correctness, the PPO-trained controller could be optimizing a proxy that is disconnected from the objective task metrics. The authors should provide a human- or gold-labeled evaluation of the node classifier and a correlation analysis between r_k and final correctness on at least one task.","section":"Section 3.2 Eq. (2); Section 3.3 reward r_k"},{"comment":"The text repeatedly uses the phrase 'significant improvements,' but no statistical significance tests are reported anywhere in the paper. Tables 1-3 report means and standard deviations, and some differences are large (for example, 89.23 versus 69.41 on 5x5 Sudoku), but the claim of significance is not supported by any paired test or confidence interval. Given that the Sudoku results use only 13 puzzle sets per condition, the authors should either perform an appropriate paired test (e.g., Wilcoxon signed-rank) or soften the wording to 'consistent improvements' with the reported variability.","section":"Section 4.1 Results; Tables 1-3"}],"minor_comments":[{"comment":"Tree of Thoughts is cited to [Chu et al., 2024], but the correct reference for ToT is [Yao et al., 2023a], which is cited later in the Related Works section.","section":"Introduction, first paragraph"},{"comment":"The bar chart in Figure 1 contains untranslated placeholder labels such as '类别 1' and '系列4'; these should be replaced with the intended English axis labels and legend entries.","section":"Figure 1"},{"comment":"The set-builder notation in Eq. (1) is missing a set-membership symbol; it should read $\\tilde{E}_v^{(k-1)} = \\{(u,w) \\in E^{(k-1)} \\mid u \\in \\tilde{V}_v^{(k-1)}, w \\in \\tilde{V}_v^{(k-1)}\\}$.","section":"Section 3.2, Eq. (1)"},{"comment":"The Creative Writing task is described as 'a series of diverse writing challenges,' but Appendix B.4 describes only two specific tasks, and the mapping between those two tasks and the three columns of Table 4 (Sentence Formation with fewer hints, Sentence Formation with more hints, Text Expansion) is not explained.","section":"Section 4.1, Tasks; Appendix B.4"},{"comment":"The caption phrase 'All data represent the performance of L2T comparisons to other methods' is awkward; it would be clearer to state that each row gives the percentage of cases in which L2T scores higher, the same, or lower than the named method.","section":"Table 4 caption"},{"comment":"The phrasing 'Each solution must consist of at least two or more steps' mixes two comparatives; it should be 'at least two steps' or 'two or more steps.'","section":"Appendix A.5, Format Generation Prompt"}],"recommendation":"major_revision","confidential_remarks":"The decisive issue for the editor is the undisclosed training/evaluation split for the PPO-trained GNN. If the RL trajectories were generated from the same test instances used in Tables 1-4, the paper's central claim of task-agnostic reasoning would collapse, and this would be a reject-level flaw. If the authors can show a clean split (e.g., a separate training subset, or training on a different family of tasks), the method is likely salvageable as a major revision. The missing L2T row in Table 4 is also a serious reporting omission that must be corrected. I recommend sending the paper back with a request to disclose the training data and to add the missing row and significance tests."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid, imaginative LLM-reasoning paper whose central mechanism holds up on the objective tasks, but it reports itself a little too enthusiastically. The headline claim should be read as 'no LLM fine-tuning' — a GNN controller is trained with PPO over 20 epochs, and the paper never says which problem instances supplied those trajectories. If they came from the same Sudoku, Game-of-24, TruthQuest, or Creative Writing sets reported in Tables 1–4, then the comparison against zero-shot/few-shot baselines is not a test of a task-agnostic controller. That is my main concern, and it is a real one.\n\nWhat is genuinely new: combining an LLM-based node classifier on the reasoning-process graph with an RL-trained GNN that adjusts prompt parameters (branch count, temperature, top-p) in real time, all without task-specific prompts. The automatic generation of format and evaluation criteria from the task description is a neat trick. The results on the objective tasks are large and internally consistent: L2T w/o GNN alone beats ToT, GoT, and AoT by a wide margin on Sudoku and TruthQuest, and full L2T adds a further accuracy bump plus fewer generated nodes. The ablations isolating the GNN and the RL update are helpful.\n\nSoft spots, in order of severity:\n1. The train/test split for the GNN controller is missing from the paper. This is load-bearing for the 'task-agnostic' claim.\n2. Table 4 omits the L2T row, so the Creative Writing results cannot be independently checked. The text claims L2T is higher-or-equal in over 80% of cases, but the table as printed lets the reader verify neither the claim nor the magnitude. The L2T w/o GNN row also shows L2T being lower in roughly a third of cases, which is unexplained.\n3. The non-final-step reward r_k is GPT-4o's self-score, and Creative Writing is judged by the same LLM family. That is acceptable for Sudoku, Game of 24, and TruthQuest because final correctness is objective, but it is circular for open-ended writing.\n4. No significance tests are reported, and with 13 Sudoku puzzles and small TruthQuest subsets, some of the gaps between methods could be noise.\n5. The abstract's 'without additional training' is literally false — the GNN is trained. The paper should say 'without LLM fine-tuning.'\n\nNone of this kills the paper. The mechanism is believable, the objective results are strong, and the citation pattern looks fine. The fix is transparency: report the split, show the L2T row in Table 4, hedge the reward claim, and soften the abstract. This deserves a serious referee and, after a careful revision, publication. I would bring it to a reading group to discuss leakage risks in learned controllers.","headline":"L2T's graph-controlled reasoning is a real step forward, but the paper must disclose its RL training split before the 'no additional training' claim can stand.","tokens_in":21094,"tokens_out":2648,"would_cite":true,"duration_ms":27001,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"L2T claims that representing a model's reasoning as a graph and training a small GNN controller lets one LLM outperform CoT, ToT, GoT, and AoT across puzzles and writing tasks without task-specific prompts.","keywords":["large language models","reasoning","graph representation learning","graph neural networks","chain-of-thought","reinforcement learning","prompt engineering","adaptive reasoning"],"falsifier":"Take a sample of reasoning steps from the Sudoku and Game of 24 runs, have a human oracle label each pending node as stop, continue, final result, or backtrack, and compare with the LLM's assigned labels; then check whether the LLM's reward scores predict which reasoning threads reach the correct final answer. If label agreement is near chance or reward scores are uncorrelated with ultimate success, the bootstrap has no training signal and the reported gains would not replicate.","tokens_in":20022,"feed_emoji":"🧠","tokens_out":7569,"duration_ms":73129,"temperature":0.7,"pith_summary":"This paper introduces Learn to Think (L2T), a framework whose central claim is that a large language model can reason across many different tasks using one generic prompt, provided its reasoning process is represented as a graph and steered by a learned controller. Each thought becomes a node; the LLM labels open nodes as stop, continue, output, or backtrack, and a small graph neural network decides how the next thoughts are generated. L2T is evaluated on Sudoku, Game of 24, TruthQuest, and creative-writing benchmarks and reported to outperform CoT, ToT, GoT, and AoT, especially when task-specific prompts are removed. If correct, the framework points to a general reasoning controller that transfers across tasks without fine-tuning the underlying model.","feed_headline":"One graph teaches an LLM to reason without task prompts","feed_subtitle":"L2T beats CoT, ToT, GoT and AoT on puzzles while using the same generic prompt for every task.","key_machinery":"The central object is the reasoning process graph, a directed graph whose nodes are the LLM's thoughts and whose edges point from a thought to its successor, with each pending node assigned one of four labels by the LLM itself: stop, continue, final output, or backtrack. The graph unifies chain-, tree-, and graph-style reasoning because those are special cases. The load-bearing mechanism is the loop between the LLM classifier and a GNN controller: the LLM labels pending nodes from an induced subgraph rendered as text (Eq. 2), the controller's one-layer GCN plus MLP maps the graph to an action vector controlling branch count, temperature, and top-p, thought generation follows that action (Eq. 4), and PPO updates the controller using the LLM's evaluation score as reward. The graph state is what lets the controller see the whole reasoning process before choosing the next move.","core_discovery":"The paper claims that LLM reasoning can be bootstrapped without task-specific prompt design by making the reasoning process itself the learning target. Every thought the model produces is a node in a directed reasoning graph; the LLM labels each unprocessed node with one of four classes—stop, continue, output as final result, or backtrack—and only 'continue' nodes generate children. A one-layer graph convolutional network reads the current graph and outputs an adjustment vector that sets the number of branches, temperature, and top-p for the next thought, and this controller is trained with PPO using an integer LLM self-score as reward. In the reported experiments L2T outperforms CoT, ToT, GoT, and AoT on all four evaluated tasks, including the variants where task-specific prompts are stripped from the baselines, and the no-GNN variant also beats the baselines, supporting the claim that the graph-level labels carry most of the benefit.","pith_inferences":["Because the controller acts on a task-agnostic graph state, the same trained policy could be applied to other generative agent loops—tool use, multi-turn dialogue, or planning—where each action can be represented as a node.","The reliance on LLM-assigned labels and reward scores suggests a direct test: if a different, weaker base LLM is swapped in, the controller's performance will reveal how much of the gain comes from the graph policy versus the base model's own grading ability.","The action vector could be extended beyond sampling parameters to choices like which subgraph to revisit or which tool to call, which would make 'reasoning mode' a more general action space.","A zero-shot test on a novel task type not seen in training would tell whether the controller has learned to allocate exploration and exploitation in a transferable way rather than memorizing task-specific patterns."],"forward_implications":["A single generic prompt is enough for heterogeneous tasks such as Sudoku, arithmetic puzzles, logic puzzles, and creative writing, so users can stop hand-designing task-specific reasoning prompts.","Removing task-specific prompts sharply degrades CoT, ToT, GoT, and AoT, while L2T, which never uses them, keeps its reported full-strength performance.","The learned controller uses fewer reasoning steps: L2T generates fewer graph nodes than L2T without the GNN module, meaning the policy makes the model think more efficiently, not just more accurately.","Reinforcement learning on the LLM's own evaluation scores improves the controller beyond direct score-based training, as shown by the L2T versus L2T w/o RL ablation.","The controller adapts its strategy to the task, choosing correlated temperature/top-p values for the arithmetic task and anti-correlated values for creative writing."],"supporting_citations":[{"why":"Defines the Chain-of-Thought baseline that L2T compares against and supplies the step-by-step prompting idea the paper generalizes.","marker":"[Wei et al., 2022]"},{"why":"Tree-of-Thoughts baseline; also provides the Game of 24 dataset used for evaluation.","marker":"[Yao et al., 2023a]"},{"why":"Graph-of-Thoughts baseline and the graph-structured reasoning representation that L2T extends.","marker":"[Besta et al., 2024]"},{"why":"Algorithm-of-Thoughts baseline that L2T is evaluated against.","marker":"[Sel et al., 2024]"},{"why":"Supplies the graph convolutional network used as the backbone of the reasoning mode selection module.","marker":"[Kipf and Welling, 2017]"},{"why":"PPO is the reinforcement learning algorithm used to update the controller.","marker":"[Schulman et al., 2017]"},{"why":"Supplies the Sudoku benchmark used in the evaluation.","marker":"[Long, 2023]"},{"why":"Supplies the TruthQuest benchmark used in the evaluation.","marker":"[Mondorf and Plank, 2024]"}],"fun_headline_variants":["Graph learning bootstraps LLM reasoning, no task prompts","GNN guides LLM reasoning, beats CoT without prompt tuning","Reasoning graph with GNN controller outperforms prompt-based methods","LLM learns to think by labeling reasoning steps in a graph"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method stands on the assumption that the same LLM can reliably label its own intermediate thoughts with the four node classes and assign trustworthy 0–10 reward scores from a generic evaluation prompt; the paper does not measure either reliability directly.","fun_headline_variants_meta":{"raw":{"variants":["Graph learning bootstraps LLM reasoning, no task prompts","GNN guides LLM reasoning, beats CoT without prompt tuning","Reasoning graph with GNN controller outperforms prompt-based methods","LLM learns to think by labeling reasoning steps in a graph"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000197,"raw_usage":{"total_tokens":1353,"prompt_tokens":924,"completion_tokens":429,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":540,"completion_tokens_details":{"reasoning_tokens":357}},"tokens_in":540,"tokens_out":429,"duration_ms":5468,"temperature":1.0,"reasoning_tokens":357,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:57:27.372939+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a sample of reasoning steps from the Sudoku and Game of 24 runs, have a human oracle label each pending node as stop, continue, final result, or backtrack, and compare with the LLM's assigned labels; then check whether the LLM's reward scores predict which reasoning threads reach the correct final answer. If label agreement is near chance or reward scores are uncorrelated with ultimate success, the bootstrap has no training signal and the reported gains would not replicate.","supporting_citations":[{"cited_title":"Graph of thoughts: Solving elaborate problems with large language models","cited_arxiv_id":null,"evidence_quote":"Graph-of-Thoughts baseline and the graph-structured reasoning representation that L2T extends."},{"cited_title":"Algorithm of thoughts: Enhancing exploration of ideas in large language models","cited_arxiv_id":null,"evidence_quote":"Algorithm-of-Thoughts baseline that L2T is evaluated against."},{"cited_title":"Liar, liar, logical mire: A benchmark for suppositional reasoning in large language models","cited_arxiv_id":null,"evidence_quote":"Supplies the TruthQuest benchmark used in the evaluation."}],"review_version":1}