{"id":"18be33b4-1fc0-4f79-bf26-de3cbb4cf068","arxiv_id":"2512.09706","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"CrossAgent learns step-level action-interface selection via a three-stage SFT + single-turn GRPO + multi-turn GRPO pipeline, reporting 54.6% mean success on 800+ Minecraft tasks after RL on only 30 tasks.","lead":"CrossAgent, a 7B vision-language agent trained on Minecraft, decides at each step which of five action interfaces to use — raw mouse and keyboard, motion shortcuts, object-grounded targeting — via supervised fine-tuning, single-turn RL, and multi-turn GRPO. The paper reports 54.6% mean success across 800+ tasks after RL on just 30 tasks, versus 31.5% for the prior OpenHA model, arguing that learned interface choices beat fixed ones.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 2 OOD ASR for Mine/Craft is numerically identical to Table 1 all-task ASR despite the stated exclusion of 10 training tasks per category — arithmetically impossible; the generalization claim rests on these numbers.","rationale":"After reading the full manuscript, I focused on the central empirical claim: CrossAgent, trained on 30 tasks via MTRL, generalizes to over 800 tasks and outperforms fixed-action baselines. The strongest quantitative support for generalization is Table 2's OOD rows. However, the reported OOD ASR values for Mine Blocks and Craft Items are numerically identical to the all-task values in Table 1, despite the stated exclusion of the 10 training tasks per category. This is arithmetically impossible if the task pools differ: removing tasks with substantially different means (ID means 70.7 for Mine, 83.7 for Craft) must change the aggregate mean. The identical standard deviations to two decimal places reinforce that this is not a rounding artifact. The All Tasks row differs, which could indicate a different weighting or a partial copy, but the category rows are directly inconsistent. This issue is more load-bearing than the credit-assignment concern noted by the reader: even if the model only learned task-level interface biases, the performance gains could still be real; but if the OOD numbers are not actually OOD, the paper's headline generalization claim is unsupported. The credit-assignment concern is valid but is about mechanism interpretability, not about whether the reported numbers support the claim. I therefore recommend keeping the CONDITIONAL verdict, contingent on reconciling Table 1 and Table 2.","tokens_in":19181,"tokens_out":5337,"duration_ms":51964,"concrete_test":"Recompute Table 2 OOD ASR using the released evaluation code and the task split that excludes the 30 MTRL training tasks; verify the Mine Blocks and Craft Items entries. If recomputed values differ from 40.0±48.3 and 78.8±41.0, the tables are inconsistent. Also inspect the task-id lists for Table 1 and Table 2 OOD to confirm the exclusion.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table 2 (OOD) reports CrossAgent Mine Blocks ASR 40.0±48.3 and Craft Items 78.8±41.0. Table 1 (all tasks, including the 10 MTRL training tasks per category) reports identical values: 40.0±48.3 and 78.8±41.0. Since ASR is the mean success rate over tasks in the category, removing a subset of tasks (the training tasks, whose ID means are 70.7 and 83.7 respectively) cannot leave the mean and standard deviation unchanged to two decimal places. The All Tasks row differs slightly (49.1 vs 54.6), making a systematic re-use of all tasks unlikely but the category-level identity unexplained. If the OOD evaluation actually included the training tasks, the central generalization claim ('trained on 30 tasks, generalizes to 800+') is not supported by Table 2. If it is a copy-paste error, the paper must be corrected; either way the quantitative basis for the headline result is compromised. This is the most load-bearing issue because the paper's central contribution is the empirical demonstration of superior generalization, not just the training recipe.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CrossAgent, a 7B vision-language policy for Minecraft that can emit actions from three heterogeneous action spaces (motion, grounding, raw). Training proceeds in three stages: mixed-space SFT, single-turn GRPO over one-step action prediction with a semantic-equivalence parser, and multi-turn GRPO with an episodic success reward and a token penalty. The authors claim that this yields step-level, context-dependent action-space selection and state-of-the-art average success rate on over 800 OpenHA tasks after training on only 30 tasks, with improved out-of-distribution generalization relative to fixed-action baselines. The paper includes ablations removing the STRL stage, single-space MTRL baselines, and qualitative case studies of three tasks.","tokens_in":19330,"tokens_out":8260,"duration_ms":73021,"significance":"The direction is timely: training a single policy to choose among multiple action granularities is a meaningful step toward generalist agents, and the three-stage pipeline with cold-start SFT, STRL, and MTRL is a concrete, reproducible recipe. The paper also provides code/model links and detailed hyperparameters, which is a strength. If the reported numbers are correct, the ASR gains are substantial. However, the central quantitative support is compromised by the apparent identity between Table 1 and Table 2 category-level ASRs, and the step-level selection claim is supported only by qualitative evidence. The wording about 'emergent efficiency' is also inconsistent with the explicit token penalty in Eq. (6). These issues are fixable but need to be addressed before the claims can be accepted.","major_comments":[{"comment":"The OOD ASR values for Mine Blocks and Craft Items in Table 2 are numerically identical to the corresponding all-task ASRs in Table 1, for both CrossAgent and the w/o-STRL variant: CrossAgent Mine 40.0±48.3 and Craft 78.8±41.0; w/o-STRL Mine 39.0±46.5 and Craft 58.0±48.4. §4.1 states that MTRL trains on 10 tasks per category, and Table 2 reports ID ASRs for CrossAgent of 70.7 (Mine) and 83.7 (Craft). Excluding 10 training tasks with these means cannot leave the category mean and standard deviation unchanged to two decimal places. The All Tasks row differs (49.1 vs 54.6), so the identity is category-specific, but it is still arithmetically impossible under the stated split. If the OOD evaluation actually included the training tasks, the central 'trained on 30 tasks, generalizes to 800+' claim is unsupported; if it is a copy-paste error, the table must be recomputed. This is load-bearing f","section":"§4.4, Table 2; §4.1, Table 1"},{"comment":"The MTRL objective assigns the same episodic reward r(τ) to every intermediate response in a trajectory. This does not by itself provide per-step credit for action-space choice; the model can only learn step-level switching if the policy-gradient signal reliably distinguishes which action-space decisions contributed to success. The direct evidence for step-level selection is qualitative: density curves aggregated over 20 episodes of 3 tasks (§4.5, Figs. 5–6). There is no per-step router accuracy, no oracle-switch baseline, no seed-level variance, and no check against the possibility that the model learned only task-level interface biases. Please add a quantitative evaluation of action-space selection, or soften the central claim to task-adaptive rather than step-adaptive selection.","section":"§3.3, Appendix C.2 Eq. (8)"},{"comment":"The objective explicitly includes −λ·l_θ(τ), a hand-set token penalty, and the problem formulation in §3 includes λ_x·cost(a_t). The abstract and introduction nevertheless credit the model with 'emergent' efficiency and 'without human-specified rules.' Efficiency is directly rewarded, so it is not emergent; what is learned is the trade-off induced by the penalty. Please revise the wording to avoid overclaiming, or provide evidence that the observed efficiency improvements exceed what the penalty alone would induce.","section":"Eq. (6), §1"},{"comment":"The STRL reward and the relabeling rule in Eq. (4) both depend on a deterministic parser g that maps action strings from all spaces to a canonical raw representation. The paper does not describe how g handles high-level, ambiguous, or partially specified actions, nor does it report any validation of g's semantic equivalence. If g is not an exact oracle, the STRL reward is noisy and the relabeled dataset D_strl can contain incorrect labels. Please provide details of g, including its coverage across the three action spaces, and an error analysis.","section":"§3.2, Eqs. (2)–(4)"}],"minor_comments":[{"comment":"Model initialization is described as Qwen2-VL-7B-Instruct in §4.1 but as the open-source OpenHA model in Appendix D.1. Clarify which base model is actually used and how the two statements are consistent.","section":"§4.1 vs Appendix D.1"},{"comment":"The caption states that three metrics are reported (representative task success, FT, ASR), but the column headers only show FT and ASR. Add explicit headers/labels for the representative-task column and explain the icon.","section":"Table 1"},{"comment":"Figure numbering appears inconsistent: the text says 'Figure 5 further illustrates some key frames', but Figure 5 is the density-curve figure and Figure 6 is the example rollouts. Swap or renumber.","section":"§4.5"},{"comment":"The abstract's 'significantly outperforms fixed-action baselines' is only fully true for ASR. For FT, OpenHA is higher on All Tasks (62.8 vs 58.7) and on Mine Blocks (67.3 vs 45.2). Qualify the claim to avoid overstatement.","section":"Abstract"},{"comment":"Observation resolution is given as 360×640×3 in §4.1 and 640×360 in Appendix A.1. Use one consistent order.","section":"§4.1, Appendix A.1"},{"comment":"The baseline list mentions RawHA, but Table 1 has no RawHA row; RawHA-RL appears only in Table 2. Clarify the relationship or add the row.","section":"§4.1"}],"recommendation":"major_revision","confidential_remarks":"The Table 1/Table 2 identity is the most serious issue. Before any acceptance, the authors must supply a corrected OOD table and confirm that the OOD split excludes the 30 training tasks. If the numbers are corrected and the step-level selection evidence is strengthened, the paper could be acceptable; the current version's central generalization claim rests on an apparent numerical inconsistency. The overclaim about 'emergent efficiency' is wording-level and fixable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The novel piece here is the three-stage pipeline: mixed-space cold-start SFT, single-turn GRPO with an action-space-agnostic reward on parsed raw actions, then multi-turn GRPO with a token penalty. That's a real contribution — no one else has trained dynamic per-step selection among five heterogeneous action spaces with RL, and the recipe is clearly described. The STRL ablation is sensible and points in the right direction, and the environment is third-party. I also appreciate that they ship code and models.\n\nBut there's a load-bearing problem. Table 1 reports Mine Blocks ASR 40.0±48.3 and Craft Items 78.8±41.0 for CrossAgent; Table 2 reports exactly the same values for OOD tasks, even though Table 1 is supposed to include the 30 training tasks and the ID means for those categories are 70.7 and 83.7. Arithmetic makes that impossible unless the pools are not what they're described to be. The same identity holds for the w/o-STRL variant. Either the OOD evaluation includes training tasks, or Table 1 copied the OOD column. Either way, the central claim — that 30-task RL transfers to 800+ tasks — rests on numbers that don't yet stand up. This is not a minor typo; it's the headline result.\n\nOther issues are smaller but real. The \"emergent efficiency\" language is overreach: Eq. 6 has a hand-set λ·l(τ) token penalty, so efficiency is shaped, not emergent, and λ is never reported. \"State-of-the-art across all categories\" is also too strong: on Finished Tasks, OpenHA beats CrossAgent in 3 of 4 columns. And the central behavioral capability — step-level switching — is evidenced only by density plots from 20 episodes of 3 tasks, with no per-step router accuracy and no oracle-switch baseline. Given that multi-turn GRPO assigns the same episodic reward to all steps, the model may simply have learned task-level interface biases rather than context-appropriate per-step switching. The qualitative rollouts are suggestive but don't rule that out.\n\nThe idea is worth engaging with, and the pipeline deserves serious scrutiny — but as written, the quantitative foundation is compromised. A serious referee should be asked to check the tables, re-run the OOD evaluation with released code, and require seed-level variance and disclosed λ before this can be accepted. Send it to review, but flag the data integrity issue prominently; the authors need to correct or explain Tables 1 and 2 before the headline claim is credible.","headline":"Interesting training recipe for step-level action-space switching, but Tables 1 and 2 are arithmetically inconsistent on the two headline categories, so the central generalization claim is not currently supported by the reported numbers.","tokens_in":20002,"tokens_out":6350,"would_cite":false,"duration_ms":60146,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single vision-language agent trained with a three-stage reinforcement-learning pipeline can choose the right action interface at every step, and that ability transfers from 30 training tasks to more than 800 evaluation tasks in Minecraft.","keywords":["agentic AI","reinforcement learning","action-space selection","Minecraft","vision-language model","GRPO","generalization","step-level decision"],"falsifier":"Annotate test trajectories with the action space an oracle would have chosen at each step (knowing future success and execution cost). If the model's per-step choices match the oracle only at chance level, its claimed step-level switching is not supported by the data.","tokens_in":18872,"feed_emoji":"⛏️","tokens_out":4920,"duration_ms":46894,"temperature":0.7,"pith_summary":"This paper argues that the optimal action interface for an agent is not fixed but shifts from step to step, and that a single vision-language model can learn to choose the right interface on its own. CrossAgent is trained in three stages: cold-start supervised fine-tuning on mixed action types, single-turn reinforcement learning that rewards semantic correctness regardless of surface form, and multi-turn reinforcement learning that optimizes episodic success minus token cost. Trained on only 30 Minecraft tasks, it evaluates on over 800 and reports higher average success rates than every fixed-action-space baseline, with particularly large gains in crafting. The contribution is a recipe: action-space selection treated as a learnable, context-dependent decision rather than a design-time choice.","feed_headline":"One RL agent beats fixed-interface models on 800+ Minecraft tasks","feed_subtitle":"Step-by-step interface choice learned from 30 tasks transfers to the full 800+ task benchmark.","key_machinery":"The central mechanism is a composite action space with an implicit router: rather than a separate switching module, the same policy emits actions from any subspace, and the router is the policy's own choice. The parser g maps every action string to a canonical raw representation, so the single-turn reward (Eq. 2) is surface-form-agnostic; this lets GRPO reinforce whichever interface yields the correct raw action. The multi-turn stage (Eq. 6) then adds a token-length penalty, pushing the policy to prefer high-level interfaces when they suffice and reserve low-level actions for precision. The staged curriculum — cold start, single-turn, multi-turn — is what the claims rest on.","core_discovery":"The central claim is that dynamic, step-level action-space selection in a single policy can be learned from experience rather than specified by hand, and that this learned switching transfers to novel tasks. The authors demonstrate a three-stage pipeline — mixed-space SFT, single-turn GRPO with an action-space-agnostic reward, and multi-turn GRPO with an episodic success reward plus a token-length penalty — and show that the resulting model, CrossAgent, outperforms fixed-action-space baselines across more than 800 Minecraft tasks despite RL training on only 30. Qualitative rollouts indicate the model uses coarse motion primitives for exploration, grounding actions for visual tracking, and ra","pith_inferences":["The paper's qualitative evidence does not yet show per-step router accuracy; a direct test would be to compare each step's chosen action space against an oracle that knows which interface is optimal for that step. If the model only matches task-level biases, the step-level claim weakens.","The parser g is a semantic-equivalence oracle; deploying this in less formal environments (e.g., real GUIs, robotics) requires a similarly reliable mapping, whose availability is a practical bottleneck.","The token-length penalty conflates efficiency with token count; in domains where low-level actions are cheap and high-level actions are expensive, the learned preference may flip.","A natural extension would be to vary the cost coefficient per action space and measure how the policy's switching behavior adapts."],"forward_implications":["If correct, agent designers no longer need to fix one interface; a single post-trained model can cover APIs, GUI events, and primitives, simplifying system design.","Training on a small task set can transfer to a large benchmark, suggesting RL-tuned switching may generalize across task distributions.","The token-cost penalty yields more efficient trajectories without explicit hand-crafted rules.","Action-space selection becomes part of the policy, so it can adapt within a trajectory, e.g., switching from search API to GUI manipulation when a webpage blocks automation.","The approach positions RL as the mechanism for learning interface choice, not just action content."],"fun_headline_variants":["Step-level action-space choice learned from RL beats fixed interfaces","One agent masters API, GUI, and robot actions in Minecraft","Dynamic action interfaces: RL agent outperforms fixed baselines","CrossHA: RL teaches when to use coarse vs precise actions","800+ Minecraft tasks, one policy, learned action switching"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"Every action-space choice in a trajectory gets the same reward as the final task outcome, so the training signal cannot tell the model which interface decisions actually helped.","fun_headline_variants_meta":{"raw":{"variants":["Step-level action-space choice learned from RL beats fixed interfaces","One agent masters API, GUI, and robot actions in Minecraft","Dynamic action interfaces: RL agent outperforms fixed baselines","CrossHA: RL teaches when to use coarse vs precise actions","800+ Minecraft tasks, one policy, learned action switching"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000364,"raw_usage":{"total_tokens":1784,"prompt_tokens":720,"completion_tokens":1064,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":464,"completion_tokens_details":{"reasoning_tokens":981}},"tokens_in":464,"tokens_out":1064,"duration_ms":9897,"temperature":1.0,"reasoning_tokens":981,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T17:24:43.572469+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Annotate test trajectories with the action space an oracle would have chosen at each step (knowing future success and execution cost). If the model's per-step choices match the oracle only at chance level, its claimed step-level switching is not supported by the data.","supporting_citations":[],"review_version":1}