{"id":"42cfa5e6-da0b-40a3-a6ae-254abcc3226a","arxiv_id":"2611.06918","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"An encoder-free vision-language-action policy whose action head reads cached keys and values from many backbone depths outperforms encoder-based VLAs on LIBERO, LIBERO-plus, and RoboTwin 2.0, with early layers carrying much of the control signal.","lead":"This paper builds a robot-control model that feeds raw pixels straight into the language backbone, with no separate vision encoder, letting the action head read visual features from many network depths. The model beats encoder-based rivals on manipulation benchmarks, especially when the backbone is frozen, suggesting control-relevant visual information lives in early layers, not just the final encoder output.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"The load-bearing gap is that encoder removal is never manipulated while holding pretraining, capacity, and tokenization fixed; Table 2 and the scale claim rest on cross-family comparisons, and Table 3 tests only Qwen's unaligned vision-tower features, so [b9] remains a within-family depth result…","rationale":"In good faith, the paper is unusually transparent: it reports the layer-map ablations, the matched-noise control in Figure 4A, and explicitly acknowledges at [b32] that pretraining and capacity differences preclude attributing gains solely to encoder removal. Those internal controls credibly establish that readout depth matters within the authors' NEO-2B architecture. The problem is that the central claim [b9] and the abstract's scale claim [b6] are stated as general facts about encoder-free versus encoder-based VLA design, while the only head-to-head evidence is cross-family. The reader's weakest_assumption already identifies this confound; this stress pass agrees and sharpens it: Table 3's control does not create a matched interface contrast, because Qwen's vision-tower intermediate features are not language-conditioned and cannot reproduce what NEO early shared layers provide. A matched encoder-free versus encoder-based pair within one VLM family would settle whether the interface itself is responsible. Such a test is expensive but feasible, and without it the appropriate verdict remains CONDITIONAL: the depth-readout effect within NEO is supported, but its generalization to all VLAs is not yet established. No adjustment to the reader's verdict is needed.","tokens_in":20254,"tokens_out":6357,"duration_ms":47398,"concrete_test":"Construct a matched pair from one VLM family: take Qwen3-VL-2B and make an encoder-free variant by replacing its vision tower with a NEO-style patch embedder initialized into the LLM embedding space, while keeping the encoder-based original as the control. Pretrain both on the same OXE corpus for the same number of steps with the same 28-layer action expert and the same skip-28 layer map, then evaluate frozen-backbone LIBERO success over multiple seeds. If the encoder-free variant does not exceed the encoder-based variant by a margin comparable to the 11.4-point gap in Table 2, the 'encoder-free interface' effect asserted by [b6] and [b9] is not established. The reverse check, inserting a frozen SigLIP encoder in front of NEO and rerunning the comparison, would serve the same purpose.","verdict_should_be":"UNCHANGED","load_bearing_attack":"For [b9] and the abstract's claim that 'additional scale alone cannot recover it' [b6] to hold, the visual interface must be an independent causal variable: an encoder-free policy should beat an encoder-based policy when matched in everything except the interface. The paper's direct evidence never realizes this condition. In Table 2, NEO-2B differs from Qwen3-VL-2B and Gemma-3-4B in pretraining corpus and objective, tokenization, positional embedding, attention pattern, and capacity, not only in the absence of an encoder; the authors concede at [b32] that gains cannot be attributed solely to encoder removal. The Qwen encoder-access control in Table 3 routes expert layers to intermediate vision-tower features, but those features come from a separately pretrained tower without language or state conditioning, so the control narrows but does not isolate the claimed variable. The remaining controlled evidence, Table 4 layer maps and Figure 4 interventions, is strong for the claim that early readout helps within the authors' own native NEO-2B family, but a single family with one pretraining recipe cannot support a general 'fixed encoder output is suboptimal' principle across VLA designs. The central assertion is not internally contradicted; it is under-supported exactly where the paper generalizes from within-family ablations to a cross-family law.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"'From Pixels to Actions' proposes NEO-VLA, a vision-language-action policy that is structurally encoder-free: raw pixels are mapped by a lightweight two-stage convolutional patch embedder (stride 32) directly into a 40-layer, 2B-parameter backbone shared with language and state tokens, and a narrow flow-matching action expert reads cached keys and values from a configurable subset of backbone layers (the layer map). This makes visual readout depth an explicit design choice. The paper's evidence has four parts: (1) cross-family LIBERO comparisons of NEO-VLA against Qwen3-VL-2B and Gemma-3-4B under a matched action expert and robot corpus, in frozen, full-fine-tuning, and encoder-unfrozen regimes (Table 2); (2) a Qwen encoder-access control (Table 3), layer-map ablations within NEO with matched expert capacity (Table 4), and attachment-point interventions with a matched-noise control (Figure 4); (3) benchmark results on LIBERO, LIBERO-plus, LIBERO-Pro, RoboTwin 2.0, and a three-task real-world study; and (4) latency measurements (Table 11). The central thesis [b9] is that embodied control benefits from access to visual representations across the computational hierarchy rather than a fixed encoder output, and that action-critical signals often peak before the final visual representation.","tokens_in":20534,"tokens_out":14573,"duration_ms":101507,"significance":"If the depth-readout thesis holds, the paper makes a genuinely useful contribution. It identifies visual readout depth as an explicit, tunable policy variable, provides an open native-VLA system with strong open-model results on LIBERO-plus (89.7) and RoboTwin 2.0 Clean2Rand (69.4), and validates on a real robot. The within-family evidence is methodologically careful: the key layer-map contrast (Table 4, skip-28 vs end-28 frozen, 90.9% vs 86.7%) matches expert capacity and backbone depth; the intervention study (Figure 4A) includes a matched-noise control; and Table 3 is an honest negative result showing that unaligned intermediate encoder features do not transfer. The authors disclose the main confound ([b32]) and explicitly label cross-paper LIBERO comparisons as benchmark context rather than controlled comparisons ([b201]-[b202]).","major_comments":[{"comment":"The abstract's claims that native architectures 'outperform encoder-based counterparts under controlled comparisons' and that 'additional scale alone cannot recover it' are load-bearing but outrun the evidence. The only direct native-vs-encoder comparisons are cross-family: NEO-2B differs from Qwen3-VL-2B and Gemma-3-4B in pretraining corpus and objective, tokenization, positional embeddings, attention pattern, and capacity, and the authors themselves state at [b32] that 'differences in pretraining and model capacity preclude attributing the gains solely to encoder removal.' The scale claim rests entirely on Gemma-3-4B (4B, 84.5% frozen) trailing NEO-2B (2B, 90.9% frozen); that is exactly the comparison the acknowledgment says cannot bear causal weight, so 'additional scale alone cannot recover it' is not established as stated. Please either add a matched-interface control (an encoder-based variant of NEO trained with the identical three-stage recipe, or an encoder-free variant of an established VLM family) or rewrite [b6] to report the comparison-level fact with the confound stated in the same sentence.","section":"§5.1, Table 2; Abstract ([b6], [b32])","authors_acknowledge":"Although differences in pretraining and model capacity preclude attributing the gains solely to encoder removal, these results establish the native backbone as a strong and transferable foundation for action learning.","authors_acknowledge_p":0.93},{"comment":"The encoder-access control changes two variables at once: the depth from which the expert reads and the alignment status of the features it reads. The Qwen vision-tower features come from a separately pretrained encoder without language or state conditioning, so the drop from 79.5% to 64.9% under a frozen backbone shows that unaligned intermediate features are not directly usable by the expert; it does not test whether early aligned features would help, and it cannot serve as an interface-level test of encoder-free versus encoder-based design. The Finding paragraph ('not simply in accessing earlier visual features, but in how encoder-free pretraining shapes them into control-ready representations') already interprets Table 3 correctly, but the surrounding text invites the reader to treat it as a readout-depth control. Please state explicitly that Table 3 is a feature-alignment control and that the readout-depth evidence comes from Table 4 and Figure 4 within the NEO family.","section":"§5.1, Table 3","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The general principle in [b9]/[b36] — that effective control depends on which stages of visual computation remain accessible to action, and that the fixed encoder output is suboptimal — is supported only by ablations of a single backbone family (NEO-2B) with a single pretraining recipe. Table 4 and Figure 4 are internally clean, but a within-family depth result cannot by itself license a cross-architecture law about VLA interfaces. I recommend scoping the conclusion to the NEO family (for example, 'within the native NEO family, readout depth is a genuine policy variable') and presenting the cross-family results in Table 2 as evidence that the native backbone transfers well to action learning, rather than as confirmation of the depth mechanism.","section":"§5.2, §6 ([b9], [b36])","authors_acknowledge":null,"authors_acknowledge_p":null}],"minor_comments":[{"comment":"The same system is reported at 95.3% average LIBERO in Table 2 (full FT, step 30k, 500 episodes per suite) and 97.4% in Table 7 (step 27.5k, 2,000 episodes per suite); the discrepancy and the differing protocols are unexplained and should be reconciled in the text.","section":"Tables 2 and 7","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The reported request-to-chunk latency of 261.9 ms is smaller than the compiled policy forward pass of 282.5 ms listed immediately above it, which is only possible if the two measurements make different assumptions; please reconcile or annotate the table.","section":"Table 11","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The intervention analysis is performed on 'diagnostic skip-28 models,' but the text does not state whether these are frozen or fine-tuned; since the readout-depth conclusion differs between regimes (Table 4), this information matters for interpretation.","section":"§5.2, Figure 4","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The choice of 13 expert layers for the encoder-access control, rather than the 28 layers used in the layer-map ablations, and the identity of the vision-tower layers that are read, are not specified in the main text; the same applies to the appendix-referenced pretraining configurations, which are not present in the submitted manuscript.","section":"Table 3 and §5.1","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The phrase 'early visual representations are pretty effective for control' in the second contribution bullet is informal for a journal article; suggest 'consistently effective' or similar.","section":"Introduction, contributions","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"major_revision"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"You should know two things. First, the paper's real contribution is the systematic depth-readout evidence, not the encoder-free framework itself: layer-map ablations with matched expert capacity (Table 4), an encoder-access control on Qwen (Table 3), and KV-swap/masking interventions with a matched-noise control (Figure 4) converge on the claim that early visual layers carry action-relevant signal. Second, the headline claim that native architectures outperform encoder-based ones is weaker than the abstract suggests, and the authors mostly know it.\n\nThe within-family depth result holds up. Skip-28 beating end-28 on a frozen backbone (90.9 vs 86.7) is a clean comparison: same backbone, same 28-layer expert, same corpus, only the layer map changes. The first-14/last-14 comparison is muddier because it also shortens the backbone, and the paper says so. The intervention plots are a nice addition; the matched-noise control addresses the obvious confound that swaps are just bigger perturbations than nothing.\n\nWhere the paper strains is the native-versus-encoder conclusion. Table 2 compares NEO-2B against Qwen3-VL-2B and Gemma-3-4B across pretraining corpus, objective, tokenization, and capacity, so the gap cannot be attributed to encoder removal alone. The authors concede this at [b32], which is honest, but the abstract's 'additional scale alone cannot recover it' [b6] is still too strong for a single cross-family comparison. Table 3 narrows the gap by giving Qwen direct access to intermediate vision-tower features and showing it does not help, but those features come from an unaligned, separately pretrained tower, so it is not a matched test of the interface. The stress-test note has this right: encoder removal is never manipulated while holding everything else fixed. The depth principle is supported within one family; the general law about fixed encoder endpoints is not.\n\nSmaller issues: no error bars or significance tests on any headline number, the stage-3 'in-house action data' is undisclosed so the final weights cannot be reproduced, and the action-only baseline in Table 6 anchors a co-training recommendation but has no training recipe. None of these are load-bearing flaws, but a referee should ask for them.\n\nThis paper deserves a serious referee. The depth-readout study is carefully done on external benchmarks, the limitations are unusually explicit, and the robustness results on LIBERO-plus and RoboTwin 2.0 are worth taking seriously. I would send it out, and I would push either for a matched-scale native/encoder pair or for language that stops short of claiming scale alone cannot close the gap.","headline":"A genuinely useful depth-readout study on an encoder-free VLA; the within-family results are solid, and the cross-family 'native beats encoder' claim is real but under-matched.","tokens_in":21308,"tokens_out":1568,"duration_ms":15006},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"Robot policies work better when actions read visual depth, not one fixed encoder output.","keywords":["vision-language-action models","encoder-free visual representations","robot manipulation","flow-matching action expert","action readout depth","layer-wise probing","distribution shift robustness","LIBERO"],"falsifier":"Take the NEO-2B architecture and insert a separately pretrained visual encoder at the input while keeping the pretraining corpus, objective, and compute identical, then compare frozen-backbone LIBERO success: if the encoder-based variant reaches 90.9%, the native interface is not the cause. Separately, train the skip-28 and end-28 layer maps past 30,000 steps; if the late-only map converges to the same or higher success, the early-layer advantage is a training-dynamics effect rather than a stable property of where control information lives.","tokens_in":21577,"feed_emoji":"🤖","tokens_out":16508,"duration_ms":108773,"temperature":0.7,"pith_summary":"The paper sets out to establish that a robot manipulation policy should not be limited to the single visual feature vector produced by a pretrained vision encoder. It introduces NEO-VLA, a structurally encoder-free architecture in which raw pixels are patchified by a lightweight convolutional embedder and processed jointly with language throughout the backbone, with a flow-matching action expert that predicts continuous trajectories while reading cached attention keys and values from selected backbone layers. With the backbone frozen and only the expert trained, the native design reaches 90.9% average LIBERO success, versus 79.5% for Qwen3-VL-2B and 84.5% for Gemma-3-4B under the same expert and robot-training corpus; full fine-tuning narrows but does not erase the gap. Layer-map ablations and key–value swap interventions show action-critical visual information emerges early, often peaking before the final representation. If this is right, the depth at which a policy reads visual computation is a genuine design variable, and the default fixed-encoder interface leaves control-relevant signal unused.","feed_headline":"Robot policies gain by reading visual layers, not one fixed endpoint","feed_subtitle":"A pixel-native policy keeps early visual detail available to action, beating encoder-based backbones under frozen training.","key_machinery":"The load-bearing mechanism is the layer map $M=(m_1,\\dots,m_d)$, which assigns each layer of the action expert to the cached attention keys and values of a particular backbone layer; comparing first-28, skip-28, end-28, and full-40 maps isolates readout depth under matched expert capacity. The native visual interface is a two-stage convolutional patch embedder with overall stride 32 that turns pixels directly into backbone tokens, so image patches and instruction text co-evolve through a shared 40-layer transformer with no separately pretrained vision encoder, image codebook, or autoencoder. The action expert is a narrow flow-matching network that predicts a continuous velocity field over 30-step action chunks, attending jointly to its own action tokens and to the cached keys and values from its mapped backbone layers. The same attachment points support the interventions, in which one layer's cached keys and values are swapped with another observation or one input span is masked, revealing distinct depth profiles for appearance, language, and geometry signals.","core_discovery":"The central claim is that embodied control benefits from access to visual representations across the computational hierarchy, rather than a fixed encoder output. Three lines of evidence carry the argument: with the backbone frozen, the encoder-free NEO-VLA reaches 90.9% average LIBERO success against 79.5% for Qwen3-VL-2B and 84.5% for the larger Gemma-3-4B under the same action expert and robot corpus; a layer map that keeps early layers accessible (skip-28) beats a late-only map (end-28) by 90.9% versus 86.7% when frozen and 95.3% versus 92.5% after full fine-tuning; and swapping early-layer cached keys and values between observations perturbs predicted translation more than late-layer swaps, with a matched-noise control indicating the effect depends on content, not just disturbance size. The paper also claims that directly exposing an encoder-based model to intermediate vision-tower features does not recover the advantage, lowering frozen performance from 79.5% to 64.9%, so how the visual representations are formed and aligned matters as much as early access. The authors state that pretraining and capacity differences across backbones preclude attributing the gains solely to encoder removal.","pith_inferences":["A cleaner isolating experiment than any in the paper would be a pretraining-matched native-versus-encoder pair, holding corpus, objective, and scale fixed; that pair alone would decide how much of the frozen gap is an interface effect.","Since the layer map is hand-set, a natural extension is to learn or search the readout depths per task or embodiment; the intervention profiles suggest optimal depth is task-dependent.","The early-layer swap result predicts that real-robot policies with early readout should stay accurate under camera-intrinsic and viewpoint changes, because their control signal is tied to raw geometry rather than distilled semantics.","The co-training result implies a reporting convention worth adopting: manipulation benchmarks should be paired with a language-understanding score, since action-only post-training can look fine on task success while silently destroying generalization."],"forward_implications":["Readout depth becomes a tunable architectural choice: action experts can be attached to early, late, or distributed backbone layers, and frozen-backbone deployment is a realistic regime because the native representations already support control.","Policies that keep early visual layers accessible should be especially stable under camera and appearance changes, while spatial-layout and task-goal changes remain the harder failure mode, matching the paper's LIBERO-plus and LIBERO-Pro profile.","Preserving multimodal understanding during action post-training is part of policy robustness: in the paper, action-only fine-tuning drops RoboTwin clean-to-randomized success from 69.4% to 10.7% and destroys the model's ability to produce text.","Exposing an encoder-based model to intermediate vision-tower features is not a substitute for native representation formation, so future comparisons should control how visual features are produced, not only which layers the policy reads.","Aggregate LIBERO results are saturating, so benchmarking should move to perturbation-specific suites such as LIBERO-plus and RoboTwin 2.0 to reveal differences in robustness."],"supporting_citations":[{"why":"Supplies the structurally native visual interface, patch embedder, rotary positions, and multimodal attention that NEO-VLA inherits.","marker":"Diao et al., 2026a"},{"why":"Supplies the flow-matching action-expert formulation that NEO-VLA adapts for continuous action chunks.","marker":"Black et al., 2024"},{"why":"Provides Qwen3-VL-2B, the encoder-based backbone used as the primary controlled comparison and encoder-access control.","marker":"Bai et al., 2025"},{"why":"Provides Gemma-3-4B, the second encoder-based backbone compared under the same action expert and robot corpus.","marker":"Team, 2025b"},{"why":"Provides the Open X-Embodiment corpus used to pretrain all compared backbones for the matched 75k-step experiments.","marker":"OXE Team et al., 2023"},{"why":"Provides the LIBERO benchmark whose four suites generate the frozen and fine-tuned success-rate comparisons.","marker":"Liu et al., 2023a"},{"why":"Provides LIBERO-plus, the seven-axis distribution-shift evaluation behind the robustness claims.","marker":"Fei et al., 2025"},{"why":"Provides RoboTwin 2.0, the randomized-environment benchmark for clean-to-randomized generalization.","marker":"Chen et al., 2025"}],"fun_headline_variants":["Why robot action policies should see all visual layers, not just the final one","Encoder-free robot policy taps early visual detail for stronger control","Visual layers beat fixed encoder outputs in robot policy tests","Action signals peak before the final visual layer, so policies should look earlier","NEO-VLA: robot control improves by reading vision at every depth"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The argument depends on the compared models being similar enough that the frozen-backbone gap comes from removing the encoder rather than from their different pretraining, objectives, or sizes; the authors themselves say those differences prevent attributing the gains solely to encoder removal, and the depth result is inferred from ablations on one backbone family.","fun_headline_variants_meta":{"raw":{"variants":["Why robot action policies should see all visual layers, not just the final one","Encoder-free robot policy taps early visual detail for stronger control","Visual layers beat fixed encoder outputs in robot policy tests","Action signals peak before the final visual layer, so policies should look earlier","NEO-VLA: robot control improves by reading vision at every depth"]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":0.000153,"raw_usage":{"cost":0.0001525524,"is_byok":false,"cost_details":{"upstream_inference_cost":0.0001525524,"upstream_inference_prompt_cost":3.26144e-05,"upstream_inference_completions_cost":0.000119938},"total_tokens":1751,"prompt_tokens":1092,"completion_tokens":659,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":917,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":581}},"tokens_in":1092,"tokens_out":659,"cost_source":"openrouter_usage","duration_ms":6844,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":581,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-11-05T06:23:38.250994+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":"Take the NEO-2B architecture and insert a separately pretrained visual encoder at the input while keeping the pretraining corpus, objective, and compute identical, then compare frozen-backbone LIBERO success: if the encoder-based variant reaches 90.9%, the native interface is not the cause. Separately, train the skip-28 and end-28 layer maps past 30,000 steps; if the late-only map converges to the same or higher success, the early-layer advantage is a training-dynamics effect rather than a stable property of where control information lives.","supporting_citations":[],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}