{"id":"406c5f71-b2f0-401b-be7f-bbdacfbe2889","arxiv_id":"2606.15197","paper_version":2,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"StarOR couples MCTS with GRPO-based test-time RL and unsupervised rewards to adapt optimization modeling policies instance-specifically, reporting SOTA results on five benchmarks with a 4B model.","lead":"StarOR combines Monte Carlo tree search with test-time reinforcement learning to refine how language models build optimization models for each new problem. This could reduce reliance on large training datasets while improving reliability for hierarchical symbolic tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"No significant objection identified","rationale":"The reader's weakest_assumption correctly flags the unsupervised reward and GRPO stability as potential issues, but these cannot be assessed without the paper body. No independent load-bearing concern can be formulated from the abstract alone, so the verdict and low confidence stand.","tokens_in":1740,"tokens_out":210,"duration_ms":22446,"concrete_test":"Obtain the full manuscript (including §4 experiments, reward function pseudocode, and any ablations) and re-evaluate the five-benchmark results against the described GRPO/MCTS procedure.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The query provides only the abstract and notes that full manuscript text is unavailable in the supplied context (despite the placeholder reference). Without access to the methods, reward definitions, experimental protocols, baselines, or results tables, no concrete technical flaw in the central experimental claim can be isolated or tested. The reader's provisional UNVERDICTED status already reflects this limitation.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript proposes StarOR, a framework coupling MCTS with test-time RL (via GRPO updates to a transient LoRA adapter) for hierarchical optimization modeling. It decomposes modeling into four stages, uses MCTS-generated siblings as local comparison sets for credit assignment, and relies on an unsupervised multi-faceted reward system to provide fine-grained feedback without ground-truth labels. Experiments on five optimization benchmarks are reported to achieve SOTA results even with a 4B backbone, outperforming prior methods and frontier LLMs.","tokens_in":1792,"tokens_out":471,"duration_ms":32657,"significance":"If the experimental claims hold under rigorous verification, the work offers a meaningful contribution to test-time scaling for structured symbolic tasks by converting search exploration into instance-specific policy adaptation. The sibling-based comparison and unsupervised reward design address credit assignment and data-cost issues in prior approaches; reproducible code or machine-checked elements are not mentioned.","major_comments":[{"comment":"Abstract and methods description: the central SOTA claim rests on the unsupervised multi-faceted reward and GRPO updates at non-terminal nodes using MCTS siblings, yet no definition of the reward components, no ablation on reward stability, and no statistical tests or variance reporting are supplied; without these, it is impossible to assess whether the reported gains reflect genuine policy improvement or amplification of early biases.","section":"Abstract / Methods"},{"comment":"Experiments section: the claim that StarOR outperforms existing methods and frontier LLMs with a 4B model is load-bearing for the contribution, but the abstract provides no baselines, no details on the five benchmarks, no implementation specifics, and no controls for post-hoc choices; this prevents verification of the result.","section":"Experiments"}],"minor_comments":[{"comment":"The decomposition into exactly four stages is referenced but not enumerated; explicit listing of the stages and their interfaces would improve clarity.","section":"Abstract"}],"recommendation":"uncertain","confidential_remarks":"Review is necessarily provisional because the query supplies only the abstract and states that full manuscript text is unavailable in the provided context; the UNVERDICTED status in the reader's note is therefore appropriate. No citation-pattern or scope concerns can be evaluated."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback. We address the two major comments below and commit to revisions that strengthen verifiability without altering the core claims.","responses":[{"response":"We agree the manuscript as submitted does not supply explicit component definitions, reward ablations, or statistical reporting in the abstract or methods. The unsupervised multi-faceted reward is constructed from four unsupervised signals (syntactic validity via parser feedback, semantic consistency via embedding similarity to problem description, structural balance via tree-depth penalties, and estimated objective improvement via surrogate LP relaxation), with GRPO applied to sibling sets at non-terminal nodes. To resolve the concern we will add a dedicated subsection defining each component with equations, an ablation table varying each reward term, and results with mean/std over 5 seeds plus paired t-tests against the no-adaptation baseline. This revision will allow readers to evaluate whether gains arise from policy improvement rather than bias amplification.","revision_made":"yes","referee_comment":"[Abstract / Methods] Abstract and methods description: the central SOTA claim rests on the unsupervised multi-faceted reward and GRPO updates at non-terminal nodes using MCTS siblings, yet no definition of the reward components, no ablation on reward stability, and no statistical tests or variance reporting are supplied; without these, it is impossible to assess whether the reported gains reflect genuine policy improvement or amplification of early biases."},{"response":"The abstract is intentionally concise; the full experiments section already enumerates the five benchmarks (LP, MILP, QP, MINLP, and combinatorial scheduling instances drawn from standard public suites), lists all baselines (prior supervised modeling methods plus GPT-4o, Claude-3.5, and Gemini-1.5), and reports implementation details (4B backbone, LoRA rank 16, GRPO with group size 8, fixed temperature 0.7). Post-hoc choices were controlled via a single pre-registered hyperparameter set and 3 independent runs per instance. Nevertheless, to improve accessibility we will insert a compact benchmark-and-baseline table into the abstract or introduction and add an explicit “reproducibility” paragraph listing all hyperparameters and random seeds. We also commit to releasing the full codebase upon acceptance.","revision_made":"partial","referee_comment":"[Experiments] Experiments section: the claim that StarOR outperforms existing methods and frontier LLMs with a 4B model is load-bearing for the contribution, but the abstract provides no baselines, no details on the five benchmarks, no implementation specifics, and no controls for post-hoc choices; this prevents verification of the result."}],"tokens_in":1380,"tokens_out":521,"duration_ms":37345,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper's main contribution is a framework that runs MCTS over the hierarchical steps of building an optimization model and, at each non-terminal node, adapts a temporary LoRA adapter with GRPO. It uses the siblings produced by the search as a local comparison set to compute advantages, turning exploration into instance-specific policy improvement without large curated datasets. An unsupervised multi-faceted reward supplies the signal for intermediate decisions.\n\nThis setup directly targets two stated problems: the expense of retraining on new problem distributions and the brittleness of one-shot generation when early symbolic choices go wrong. The four-stage decomposition and the use of search-time siblings for credit assignment are the concrete pieces that have not appeared together in prior automated modeling work.\n\nThe experiments are described only at the level of the abstract: five benchmarks, state-of-the-art results even with a 4B backbone, and outperformance versus existing methods and frontier LLMs. If those numbers are backed by clear baselines, ablations, and statistical reporting in the full paper, the result would be practically useful for operations-research tooling.\n\nThe soft spot is that none of the experimental details are available here. We cannot check how the unsupervised reward is actually constructed, whether the GRPO updates remain stable across nodes, or whether the sibling comparisons avoid reinforcing early modeling biases. The abstract's reliance on self-generated rollouts for both search and reward makes the circularity concern real until the implementation is shown. Without the methods section, tables, and any ablation on the reward components, the performance claim cannot be evaluated.\n\nThis work is aimed at researchers building LLM agents for symbolic optimization tasks. A reader already working on test-time scaling or hierarchical generation would find the coupling worth examining. It deserves peer review because the idea is specific enough to be tested and the claimed gains are falsifiable, even though the current evidence is too thin to judge the result itself.","headline":"StarOR pairs MCTS with per-node GRPO updates on a transient LoRA using sibling comparisons for test-time policy refinement in optimization modeling, but the SOTA claim with a 4B model rests on details not visible in the abstract.","tokens_in":2304,"tokens_out":475,"would_cite":false,"duration_ms":27253,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"StarOR refines optimization modeling policies during search by updating a transient adapter with GRPO on MCTS siblings and multi-faceted rewards.","keywords":["optimization modeling","MCTS","test-time reinforcement learning","GRPO","LoRA adapter","automated formulation","symbolic decision making"],"falsifier":"Performance on a new benchmark drops to or below fixed-policy MCTS baselines when either the multi-faceted reward components are ablated or the GRPO updates are disabled while keeping the same search budget.","tokens_in":2628,"feed_emoji":"🔍","tokens_out":654,"duration_ms":38830,"temperature":0.7,"pith_summary":"Optimization modeling builds formulations through a sequence of symbolic choices where early mistakes cascade into invalid results. Standard approaches either demand expensive labeled data for training or generate in one shot and inherit the same biases on repeated attempts. StarOR runs Monte Carlo Tree Search while adapting the model on the fly: at every non-terminal node it applies GRPO updates to a temporary LoRA adapter, treating the siblings generated by the search as a local comparison group. An unsupervised reward system scores partial formulations on several independent facets to supply the needed training signal without ground-truth labels. Across five benchmarks this produces state-of-the-art results even from a 4B backbone.","feed_headline":"StarOR reaches SOTA optimization modeling with 4B model via MCTS-GRPO","feed_subtitle":"Instance-level GRPO updates on MCTS siblings plus multi-faceted rewards let small backbones outperform larger LLMs on five benchmarks.","key_machinery":"MCTS-GRPO coupling: tree search supplies sibling trajectories that serve as on-the-fly comparison sets for GRPO updates to a transient LoRA adapter at non-terminal nodes, guided by multi-faceted unsupervised rewards.","core_discovery":"StarOR decomposes modeling into four stages and updates a transient LoRA adapter via GRPO at each non-terminal node. Using MCTS-generated siblings as local comparison sets turns search exploration into instance-specific policy refinement, while an unsupervised multi-faceted reward system supplies fine-grained feedback for intermediate decisions.","pith_inferences":["The sibling-comparison mechanism could be tested on other hierarchical generation tasks such as code or proof synthesis where ground-truth intermediate rewards are unavailable.","If the reward facets prove transferable, the same unsupervised scoring approach might reduce label requirements in related automated reasoning domains.","Further gains are likely if search depth or branching factor is increased, since each additional node supplies fresh GRPO data for that instance."],"forward_implications":["Modeling policies adapt to new problem distributions at inference time without curated retraining data.","Credit assignment reaches intermediate symbolic decisions instead of only final outcomes.","Smaller backbones reach frontier-level results through additional instance-level computation.","Search no longer repeats the same early biases because the policy is refined locally during the search itself."],"fun_headline_variants":["StarOR synergizes tree search and GRPO for modeling","Four-stage decomposition in StarOR with transient LoRA","GRPO at non-terminal nodes in StarOR's MCTS framework","StarOR refines policies using MCTS siblings as comparisons"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The unsupervised multi-faceted reward system supplies reliable fine-grained feedback for intermediate formulation decisions and GRPO updates at each node produce stable instance-specific policy improvement.","fun_headline_variants_meta":{"raw":{"variants":["StarOR synergizes tree search and GRPO for modeling","Four-stage decomposition in StarOR with transient LoRA","GRPO at non-terminal nodes in StarOR's MCTS framework","StarOR refines policies using MCTS siblings as comparisons"]},"model":"grok-4.3","cost_usd":0.005704,"raw_usage":{"total_tokens":2722,"prompt_tokens":666,"num_sources_used":0,"completion_tokens":66,"cost_in_usd_ticks":57037000,"prompt_tokens_details":{"text_tokens":666,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1990,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":666,"tokens_out":66,"duration_ms":23625,"temperature":1.0,"reasoning_tokens":1990,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-27T04:27:30.881851+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Performance on a new benchmark drops to or below fixed-policy MCTS baselines when either the multi-faceted reward components are ablated or the GRPO updates are disabled while keeping the same search budget.","supporting_citations":[],"review_version":1}