{"id":"9f8e1dc3-6e7d-4e76-9953-d59425f99af9","arxiv_id":"2607.05377","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":6,"one_line_summary":"A dual-system framework with a structured subtask interface, event-balanced training, and inference harness enables VLM-guided long-horizon robotic manipulation, achieving 95.5% on LIBERO-Long and 65% on real-world chemistry tasks.","lead":"Cortex pairs a vision-language model planner with a vision-language-action executor, using 32 standardized skill primitives and a text memory log to coordinate long multi-step robot tasks. A smart generalist might read it because it shows a practical path to robots that complete complex real-world procedures like chemistry experiments without task-specific retraining.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"Table 3's 0% vs 65% comparison may confound the Cortex planning interface with same-platform executor fine-tuning, leaving the 'infeasible through VLA fine-tuning alone' claim insufficiently isolated.","rationale":"The reader correctly identified that the 'zero-shot' label overstates generalization given same-platform executor fine-tuning. My concern is adjacent but more specific: it targets whether the Table 3 comparison actually supports the 'infeasible through VLA fine-tuning alone' claim, which is the load-bearing evidence for the paper's headline contribution. The ambiguity about whether baselines received equivalent fine-tuning is a real gap in the experimental design. However, the reader's CONDITIONAL verdict with MODERATE confidence already accounts for the gap between strong claims and the evidence base. My concern reinforces the existing conditions (particularly the reader's point 1 about zero-shot framing and point 3 about lack of error bars) without requiring a verdict change. The simulation results (Tables 1, 2, 5, 9) are independently controlled and do not suffer from this confound, so the architectural contributions remain supported. If the authors clarify that π_mem was trained on the same data with only conditioning differing, and add error bars, the verdict could move toward ACCEPT. If the baselines were not same-platform fine-tuned, the real-world claim weakens substantially and REJECT of the zero-shot headline claim would be warranted. The paper's engineering contributions (event-balanced sampling, harness engineering, 32-skill interface) are sound regardless of this concern.","tokens_in":25367,"tokens_out":3787,"duration_ms":194365,"concrete_test":"Fine-tune π_mem on the same 10 hours of same-platform data used for π_sub_mem, but conditioned on the full task instruction rather than subtask-segmented commands. Re-evaluate on the same 20-trial chemistry and washing suites. If π_mem still scores 0%, the 'infeasible through VLA fine-tuning alone' claim is supported. If it scores meaningfully above 0%, the claim requires qualification and the Cortex contribution is smaller than presented.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim — that Cortex enables zero-shot real-world long-horizon task completion 'infeasible through VLA fine-tuning alone' — rests on Table 3, where end-to-end baselines (π0.5, π_mem) score 0% while Cortex scores 65%. However, the experimental setup creates a potential confound. Cortex's executor π_sub_mem was fine-tuned on ~10 hours of same-platform, same-object data (Appendix A.5.2: 'trained on long-horizon demonstrations collected in the target robot setup'). The paper states π_mem has 'the same short-memory design' as π_sub_mem but does not explicitly confirm it was trained on the same 10 hours of data with only the conditioning text differing (full task instruction vs. subtask commands). If π0.5 and π_mem were not fine-tuned on this same-platform data, the 0% vs 65% gap reflects both (a) the Cortex planning interface and (b) same-platform motor adaptation, making the 'infeasible through VLA fine-tuning alone' attribution unsupported. Even in the best case where π_mem was trained on the same data, the comparison still conflates two changes: subtask-level conditioning and the full Cortex planning loop. Without an ablation that fine-tunes a VLA on the same 10 hours with full task instructions (no subtask segmentation), we cannot attribute the performance gap solely to the planning framework rather than to the conditioning format. This is distinct from the reader's concern about the 'zero-shot' label: even if one accepts that 'zero-shot' refers only to the VLM planner, the experimental evidence for 'infeasible through VLA fine-tuning alone' remains insufficiently controlled.","agreement_with_reader":"partial"},"referee_report":{"model":"glm-5.2","summary":"The paper introduces Cortex, a dual-system framework for long-horizon robotic manipulation that bidirectionally aligns a VLM planner (System-2) with a VLA executor (System-1) via a structured subtask interface. The framework standardizes manipulation into 32 canonical skill primitives, constructs training data through automatic annotation of 4k+ hours of video and 30 hours of simulation, and introduces an event-balanced sampling strategy for subtask transitions. Evaluations include open-loop VLM planning (Table 1), closed-loop simulation benchmarks (LIBERO-Long 95.5%, RoboTwin 86.8%), and real-world chemistry/kitchen tasks (Table 3: 65%/55% success). The central claim is that Cortex's generalist VLM enables zero-shot completion of unseen real-world long-horizon tasks infeasible through VLA fine-tuning alone.","tokens_in":26292,"tokens_out":1483,"duration_ms":261360,"significance":"The paper makes several solid contributions: (1) a scalable automatic annotation pipeline using dynamic programming for subtask boundary inference (Appendix A.1), (2) the event-balanced sampling strategy with a controlled ablation (Table 5) showing improved sample efficiency, (3) strong simulation results on LIBERO-Long and RoboTwin with component-wise ablations, and (4) real-world deployment on complex 14-step chemistry tasks. The harness engineering and timeout-driven recovery mechanism are practical contributions. However, the central real-world claim rests on a comparison whose experimental controls are insufficiently specified, which weakens the contribution's load-bearing evidence.","major_comments":[{"comment":"§4.3, Table 3: The central real-world claim — that Cortex enables zero-shot task completion 'infeasible through VLA fine-tuning alone' — rests on the 0% vs 65% comparison between end-to-end baselines (π0.5, π_mem) and Cortex. However, the experimental setup confounds two variables. Appendix A.5.2 states that π_sub_mem was fine-tuned on 'about 10 hours of automatically segmented subtask-to-action data collected in the target robot setup.' The paper states π_mem has 'the same short-memory design' as π_sub_mem but does not explicitly confirm whether π_mem and π0.5 were trained on the same 10 hours of same-platform data. If they were not, the 0% vs 65% gap reflects both (a) the Cortex planning interface and (b) same-platform motor adaptation. Even if π_mem was trained on the same data, the comparison still conflates subtask-level conditioning with the full Cortex planning loop. Without an ab","section":null},{"comment":"§4.3 and Abstract: The 'zero-shot' label is applied to the full system but Appendix A.5.2 reveals that System-1 (π_sub_mem) was fine-tuned on same-platform, same-object-type data (beakers, funnels, flasks). The system is zero-shot at the planning level but not at the motor-execution level. This distinction is not made explicit in the main text. The abstract's claim of 'zero-shot completion of unseen real-world long-horizon tasks' overstates the generalization: the executor has potentially memorized scene-specific motor patterns. The authors should clarify that 'zero-shot' refers specifically to the VLM planner's deployment without task-specific planning fine-tuning, and that the VLA executor was fine-tuned on same-platform data.","section":null},{"comment":"§4.1, Table 1: The LLM-as-a-Judge evaluation uses Qwen-3.5-9B to score a system (Cortex) built on Qwen3-VL-8B. This creates a same-family bias risk: the judge may systematically favor outputs from the Qwen family over those from GPT-5 or Gemini. The paper does not discuss this potential bias or report any robustness check (e.g., cross-model judge agreement, human evaluation on a subset). This undermines the open-loop evaluation's validity as evidence for the interface design. The authors should either (a) use a judge from a different model family, (b) report human-annotated scores on a subset for calibration, or (c) explicitly acknowledge this limitation and justify why same-family judging is acceptable here.","section":null}],"minor_comments":[{"comment":"§3.2: The 32 canonical skill primitives are listed in Appendix A.3 but the main text does not explain how this vocabulary was derived — whether through clustering of existing datasets, manual task analysis, or coverage analysis. A brief justification for the choice of 32 and the specific skills would strengthen the design.","section":null},{"comment":"Table 1: The scoring scale (out of 5 for subtask/memory, out of 10 for total) and the discrete judge ratings {0, 0.4, 0.9, 1.0} make interpretation difficult. The mapping from discrete ratings to the reported scale should be stated explicitly in the main text or table caption, not only in the appendix.","section":null},{"comment":"§3.3: The temporal margins ε₁ and ε₂ are described qualitatively (ε₂ > ε₁, ~1 second total) but concrete values are only given in Appendix A.2 (ε = 0.5s for RoboCerebra, 1.5s for Galaxea). The main text should include concrete values.","section":null},{"comment":"Figure 6: The RoboTwin results are presented as a bar chart with numerical values embedded. A table format (as in Table 8 in the appendix) would be easier to read and reference. Currently, cross-referencing between Figure 6 and Table 8 is cumbersome.","section":null},{"comment":"§4.2: The paper states Cortex uses 'π0.5 and maintains an exceptionally high success rate (88.00%)' — the sentence structure is slightly broken here and should be revised for clarity.","section":null},{"comment":"Appendix A.5.2, Table 10: The weight decay is listed as 1×10⁻², which differs from the simulation configuration in Table 7 (1×10⁻¹⁰). This large discrepancy should be explained or corrected.","section":null},{"comment":"The paper references GPT-5.4 [42] and Gemini-3.1-Pro [14] with future dates (2026). These appear to be preview/pre-release models. The exact versions and access dates should be noted for reproducibility.","section":null}],"recommendation":"major_revision","confidential_remarks":"The reader's concern about the Table 3 confound is well-founded and is the most important issue. The paper's claim of 'infeasible through VLA fine-tuning alone' is not adequately supported by the current experimental design. The fix is straightforward: either (a) confirm that π_mem was trained on the same 10 hours of data and add an ablation with full-task-instruction conditioning, or (b) soften the claim to 'infeasible without hierarchical subtask decomposition' rather than 'infeasible through VLA fine-tuning alone.' The same-family judge issue is a secondary concern but should be addressed for the open-loop evaluation to carry weight."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for a careful and constructive review. The three major comments identify legitimate gaps in our experimental controls and terminology. We address each below and commit to concrete revisions in all three cases: (1) clarifying and strengthening the baseline comparison for the real-world claim, (2) correcting the 'zero-shot' terminology to distinguish planner-level from executor-level generalization, and (3) adding robustness checks for the LLM-as-a-Judge evaluation including cross-model judge agreement and human-annotated calibration on a subset.","responses":[{"response":"The referee is correct that the main text does not explicitly state the training data configuration for the end-to-end baselines. To clarify: pi0.5 is the pretrained foundation model without any same-platform fine-tuning, while pi_mem was fine-tuned on the same 10 hours of same-platform subtask-to-action data as pi_sub_mem, with the only architectural difference being that pi_mem receives the full long-horizon task instruction rather than per-subtask commands. We agree this should have been stated explicitly in the main text and will add it. Regarding the deeper concern about confounding subtask-level conditioning with the full Cortex planning loop: we acknowledge that the 0% vs 65% comparison does not isolate the VLM planner's contribution from the subtask-conditioning benefit alone. The Human+pi_sub_mem row (75% success) partially addresses this by showing that even with a human providing optimal subtask commands, the executor alone achieves 75%, while Cortex's autonomous planning achieves 65%, demonstrating the planning loop closes most of the gap. However, we agree a cleaner ablation, pi_sub_mem with a non-Cortex planner (e.g., GPT-5 as planner), would further isolate the contribution. We will add this experiment and also revise the claim language to specify that the comparison is between end-to-end monolithic VLA execution and the full Cortex dual-system, not between subtask-conditioning alone and the full system.","revision_made":"partial","referee_comment":"[MAJOR-1] The central real-world claim (0% vs 65%) confounds Cortex's planning interface with same-platform motor adaptation, because it is unclear whether pi_mem and pi0.5 were trained on the same 10 hours of same-platform data as pi_sub_mem. Even if they were, the comparison conflates subtask-level conditioning with the full Cortex planning loop."},{"response":"This is a fair and accurate criticism. The abstract and main text use 'zero-shot' in a way that conflates two distinct generalization claims: the VLM planner operates zero-shot (no task-specific planning fine-tuning for the real-world chemistry tasks), while the VLA executor was fine-tuned on same-platform data with overlapping object types. We will revise the abstract and Section 4.3 to explicitly state that 'zero-shot' refers specifically to the VLM planner's deployment without task-specific planning fine-tuning, and that the VLA executor was fine-tuned on same-platform motor data. The revised phrasing will clarify that the contribution is the zero-shot transfer of the planning interface to unseen task compositions, not zero-shot motor execution. We appreciate the referee catching this overstatement.","revision_made":"yes","referee_comment":"[MAJOR-2] The 'zero-shot' label overstates generalization because System-1 (pi_sub_mem) was fine-tuned on same-platform, same-object-type data. The system is zero-shot at the planning level but not at the motor-execution level."},{"response":"The referee raises a valid methodological concern. Using a Qwen-family judge to evaluate a Qwen-family system does introduce potential same-family bias. We will address this in revision through two additions: (1) We will report human-annotated scores on a randomly sampled subset (approximately 100 samples across all three evaluation buckets) to calibrate the automated judge. (2) We will add a cross-model judge agreement check using a non-Qwen model (e.g., GPT-5) as a secondary judge on the same subset, reporting inter-judge agreement. If the human and cross-model scores are consistent with the Qwen-3.5-9B judge, this would indicate the same-family bias is not materially affecting conclusions. If discrepancies emerge, we will report them transparently. We will also add an explicit limitation paragraph acknowledging the same-family judge risk. We note that the judge evaluates structured JSON outputs (subtask text and memory text) against ground-truth references using a discrete rubric, which is less susceptible to stylistic preference bias than free-form generation judging, but we agree this does not fully eliminate the concern.","revision_made":"yes","referee_comment":"[MAJOR-3] The LLM-as-a-Judge evaluation uses Qwen-3.5-9B to score a system (Cortex) built on Qwen3-VL-8B, creating same-family bias risk. No robustness check is reported."}],"tokens_in":25412,"tokens_out":1040,"duration_ms":153492,"standing_objections":[]},"desk_editor":{"model":"glm-5.2","letter":"Cortex is a well-engineered dual-system framework for long-horizon manipulation. The main contribution is the specific combination of a 32-skill constrained planning interface, automatic subtask boundary annotation via dynamic programming on 4k hours of video, event-balanced sampling for transition phases, and a deployment harness with timeout-driven reset. The simulation results are solid: 95.5% on LIBERO-Long and 86.8% on RoboTwin, with clean ablations (Table 5) showing that event-balanced sampling helps despite using fewer total samples than the baseline. The RMBench results (Table 9) are a nice addition, demonstrating memory-dependent task performance. The system description is detailed enough for partial re-implementation, and the appendix is thorough on training protocols and prompt templates. The open-loop VLM evaluation (Table 1) showing Cortex outperforming GPT-5 and Gemini on planning is a meaningful result, though see below on the evaluation methodology. The stress-test concern about Table 3 is the real issue. The paper claims Cortex enables zero-shot real-world task completion that is 'infeasible through VLA fine-tuning alone,' citing 65% vs. 0% success. But the executor (π_sub_mem) was fine-tuned on ~10 hours of same-platform, same-object data (Appendix A.5.2), while the baselines (π0.5, π_mem) appear to lack this same-platform motor adaptation. The paper does not explicitly confirm that π_mem was trained on the same 10 hours with only the conditioning format differing. If the baselines were not fine-tuned on this data, the 0% vs. 65% gap conflates the Cortex planning interface with same-platform motor adaptation. Even in the best case where π_mem used the same data, the comparison still bundles two changes: subtask-level conditioning and the full planning loop. Without an ablation that fine-tunes a VLA on the same 10 hours with full task instructions (no subtask segmentation), the attribution claim is unsupported. This is the central weakness, and it is load-bearing for the paper's headline claim. The 'zero-shot' label is also imprecise: the VLM planner may be zero-shot on these tasks, but the executor is fine-tuned on same-platform data. The paper should make this distinction explicit. The LLM-as-a-Judge using Qwen-3.5-9B to evaluate a Qwen3-VL-8B-based system is a minor concern — same-family evaluator bias is possible but the rubric is detailed and the exact-match rule limits the worst cases. The real-world results lack error bars on 20-trial samples, which is standard but not disqualifying. This paper is for robotics researchers working on long-horizon manipulation and hierarchical control. The simulation work and data pipeline are genuinely useful contributions. The real-world demonstration is impressive but the comparison is not clean enough to support the strongest claims. It deserves a serious referee who can push the authors to either run the missing ablation or soften the attribution claim.","headline":"Solid dual-system robotics framework with a real confound in the headline real-world result","tokens_in":26342,"tokens_out":711,"would_cite":false,"duration_ms":181599,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"Dual-system robot hits 65% on 14-step chemistry tasks","keywords":[],"falsifier":"Run Cortex's VLM planner paired with a VLA executor fine-tuned on a genuinely different embodiment or object domain (e.g., a different arm morphology handling objects never seen in the executor's training data). If the 65% success rate on chemistry tasks drops to near zero, the zero-shot claim is specific to the planning layer and the executor is not generalizing.","tokens_in":25511,"feed_emoji":"🤖","tokens_out":1224,"duration_ms":50506,"temperature":0.7,"pith_summary":"Cortex argues that the reason today's Vision-Language-Action (VLA) models fail at long-horizon manipulation — tasks like running a multi-stage chemistry experiment or washing a beaker — is not a lack of motor skill but a lack of procedural memory. A reactive policy that maps only the current camera frame to actions cannot tell which stage of a multi-step procedure it is in, so it loops or jumps ahead. Cortex solves this by splitting the job into two asynchronously coupled systems: a high-level Vision-Language Model (VLM) that tracks progress in text-based semantic memory and dispatches one subtask at a time, and a low-level VLA that executes that subtask reactively. The central mechanism is a bidirectionally aligned subtask interface: the planner is constrained to output only subtasks drawn from 32 canonical skill primitives with strict language templates, and the executor is conditioned on those standardized commands rather than on the full task instruction. This interface is trained on over 4,000 hours of auto-annotated video and 30 hours of simulation data, with an event-balanced sampling strategy that oversamples the ambiguous moments around subtask boundaries so the planner learns when to hold the current subtask versus when to advance. A deployment harness normalizes the planner's language to executor-compatible commands and applies timeout-driven resets to escape perceptual deadlocks. The paper claims that this architecture, when paired with a fine-tuned VLA executor, can zero-shot complete 14-step real-world chemistry procedures at 65% success — something end-to-end VLA fine-tuning alone cannot achieve (0% success) — because the executor never has to infer global progress from raw observations.","feed_headline":"Dual-system robot hits 65% on 14-step chemistry tasks","feed_subtitle":"A VLM planner with text memory and a constrained 32-skill vocabulary lets a reactive VLA execute multi-stage procedures that end-to-end basi","key_machinery":"The bidirectionally aligned subtask interface: 32 canonical skill primitives with strict language templates, an actively updated text-based semantic memory log, event-balanced sampling with asymmetric temporal margins around subtask boundaries, and a deployment harness that normalizes planner output to executor-compatible commands and applies timeout-driven kinematic resets.","core_discovery":"The paper's central claim is that the bottleneck in long-horizon robotic manipulation is temporal-semantic ambiguity at subtask transitions, not motor execution capability. By constraining a VLM planner to a 32-skill vocabulary with actively updated text memory, conditioning the VLA executor on single subtask commands, and training with event-balanced sampling that densifies supervision around transition boundaries, the system resolves the ambiguity that causes monolithic VLAs to loop or skip stages. The proof point is that this planner can be paired with a VLA fine-tuned on generic subtask-to-action data and then complete entirely unseen 14-step real-world chemistry tasks at 65% success, a ","pith_inferences":["The zero-shot claim likely applies to the planning level but not the motor-execution level: the VLA executor was fine-tuned on ~10 hours of subtask-to-action data from the same platform with the same object types, so its motor patterns may be specialized to that embodiment and scene distribution. The paper does not make this distinction explicit.","If text-based memory is the bottleneck for spatial precision (as the authors acknowledge in their limitations), then tasks requiring pixel-level object-instance disambiguation — such as distinguishing two visually identical beakers — would expose a ceiling that the current architecture cannot break without adding visual memory retrieval.","The 32-skill vocabulary may be sufficient for tabletop chemistry and kitchen tasks but could be too coarse for domains requiring continuous force-controlled skills (e.g., deburring, suturing, assembly with tight tolerances), where the discrete subtask abstraction breaks down.","The event-balanced sampling result — that fewer transition-focused samples outperform more steady-state samples — suggests a general principle for data-efficient robot learning that could extend beyond the dual-system paradigm to any sequential decision-making model trained on demonstration data."],"forward_implications":["If the bidirectional alignment thesis holds, the path to longer-horizon robot autonomy runs through better planner-executor interfaces and transition-focused training data, not through scaling monolithic VLA models alone.","The 32-skill vocabulary could become a standard API layer: any VLM that outputs these primitives could swap between different VLA executors or robot embodiments without retraining the planner.","Event-balanced sampling suggests that data curation for robotic foundation models should prioritize transition frames over steady-state execution frames — a shift from volume-driven to event-driven data scaling.","The timeout-driven reset mechanism implies that perceptual deadlocks, not motor failures, may be the dominant failure mode in real-world long-horizon deployment, and that controlled physical perturbation is a viable recovery strategy."],"fun_headline_variants":["VLM planner with 32-skill vocabulary enables 65% success on unseen 14-step tasks","Bidirectional VLM–VLA alignment resolves subtask transition ambiguity in long-horizon mani","Constrained skill primitives let generalist planner complete multi-stage chemistry experim","Event-balanced sampling around transition boundaries beats monolithic VLAs by 3–4%","Text-memory planner paired with fine-tuned VLA achieves zero-shot long-horizon task comple"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The zero-shot transfer claim depends on the VLA executor being a general reactive policy, but the executor was fine-tuned on about 10 hours of subtask-to-action data collected on the same robot platform with the same object types, so its motor execution may be implicitly specialized to that scene distribution rather than truly zero-shot.","fun_headline_variants_meta":{"raw":{"variants":["VLM planner with 32-skill vocabulary enables 65% success on unseen 14-step tasks","Bidirectional VLM–VLA alignment resolves subtask transition ambiguity in long-horizon manipulation","Constrained skill primitives let generalist planner complete multi-stage chemistry experiments","Event-balanced sampling around transition boundaries beats monolithic VLAs by 3–4%","Text-memory planner paired with fine-tuned VLA achieves zero-shot long-horizon task completion"]},"model":"glm-5.2","effort":"high","cost_usd":0.0,"raw_usage":{"total_tokens":755,"prompt_tokens":657,"completion_tokens":98,"prompt_tokens_details":null},"tokens_in":657,"tokens_out":98,"duration_ms":11764,"temperature":1.0,"reasoning_tokens":null,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-07T13:44:03.747885+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"Run Cortex's VLM planner paired with a VLA executor fine-tuned on a genuinely different embodiment or object domain (e.g., a different arm morphology handling objects never seen in the executor's training data). If the 65% success rate on chemistry tasks drops to near zero, the zero-shot claim is specific to the planning layer and the executor is not generalizing.","supporting_citations":[],"review_version":1}