{"id":"93d6c1f8-ef91-4773-99e2-378d610d73ae","arxiv_id":"2607.03387","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.5,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Residual tactile representations plus surprise-aware gating let VLA policies master contact-rich robot tasks that vision-only models fail.","lead":"ResTacVLA turns tactile signals into residual surprise against visual predictions, then gates them into a VLA policy only when vision is unreliable. The method lifts success rates on precision insertion, screwing and wiping tasks where pure vision collapses under occlusion and contact dynamics.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Residual orthogonality and information-gain claims rest solely on in-distribution end-task success; no direct probe of residual vs. raw tactile or OOD transfer exists.","rationale":"The reader correctly isolates the untested wrist-camera residual assumption as the weakest link supporting the mechanism that produces the headline numbers. My concern is essentially the same: the orthogonality and information-gain properties are inferred from in-distribution task success rather than independently measured or ablated. The experimental design is otherwise careful (phase-wise metrics, component ablations of VQ and gating, robustness suite), the margins are large enough that small-N binomial noise is unlikely to reverse the ranking, and no mathematical inconsistency appears. Therefore the CONDITIONAL verdict already accounts for the soft spot; no further downgrade is warranted. Strengthening the residual-specific ablation or adding a modest OOD contact set would convert the verdict to ACCEPT.","tokens_in":13184,"tokens_out":568,"duration_ms":31686,"concrete_test":"Retrain the policy after replacing r_t with the raw UniT latent z_t inside the same VQ bottleneck and SAG (still driven by the original σ_t), then re-evaluate the five-task suite of Table I under identical trial counts. If average success drops by less than 10 absolute points relative to full ResTacVLA, the residual formulation itself is not the primary driver of the reported gains.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central performance claim (Table I: 62.8 % avg, +34.6 % over π0.5) is attributed to the Residual Tactile Representation r_t = z_t − μ_t produced by a wrist-camera-only CMP (§III-B, Eq. 1). This residual is asserted to be dense, high-value, and orthogonal to vision, thereby resolving modality collapse. Yet the only evidence is end-to-end success rates, t-SNE clustering (Fig. 4), and gating traces (Fig. 5) on the same five tasks used to pre-train the CMP. There is no measurement of mutual information I(r_t; multi-view visual features) versus I(z_t; ·), no ablation that feeds raw z_t (instead of residual) through the identical VQ + SAG pipeline, and no evaluation of the frozen CMP residual on held-out contact dynamics or novel objects. If the residual largely re-encodes task-specific patterns already partially predictable from the full visual stack, the predictive-coding justification and the claimed bandwidth-mismatch solution do not hold beyond the reported regime.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes ResTacVLA, a VLA policy for contact-rich manipulation that reformulates tactile input as a Residual Tactile Representation r_t = z_t − ẑ_t, where ẑ_t is a wrist-camera visual prior produced by a Cross-Modal Predictor (CMP). Residuals are quantized via a VQ bottleneck into Latent Contact Primitives and injected into a flow-matching action expert through a Surprise-Aware Gate driven by the predictor’s aleatoric uncertainty σ_t. Built on π0.5 and evaluated on five real-robot tasks (lightbulb screwing, plug insertion, peg-in-hole, peg transfer, plate wiping), ResTacVLA reports 62.8% average success (+34.6% over vision-only π0.5), phase-wise gains on interaction stages, ablations of VQ and gating (Table II), and robustness to grasp noise, mid-execution displacements, and surface-height changes (Table III).","tokens_in":13560,"tokens_out":1408,"duration_ms":18532,"significance":"Contact-rich manipulation remains a clear bottleneck for vision-centric VLAs, and modality collapse under naive tactile fusion is a recognized practical failure mode. The residual-plus-surprise-gate formulation is a clean, biologically motivated systems idea that is architecture-agnostic (also improves Diffusion Policy) and is supported by real-robot results, phase-wise metrics, interpretability plots (t-SNE of primitives, gate traces), and three robustness suites. If the residual truly supplies orthogonal information gain rather than task-specific re-encoding, the approach offers a reusable recipe for tactile integration in generalist policies. Strengths include multi-task CMP pretraining, frozen-then-finetune protocol, and explicit comparison against both ResNet and UniT tactile baselines.","major_comments":[{"comment":"§III-B and Abstract claim that r_t = z_t − ẑ_t is dense, high-value, and orthogonal to vision, thereby ‘inherently resolving the bandwidth mismatch.’ The only supporting evidence is end-to-end success (Table I), t-SNE (Fig. 4), and gate traces (Fig. 5) on the same five-task distribution used to train the CMP. There is no mutual-information comparison I(r_t; multi-view vision) vs I(z_t; ·), and no ablation that routes raw (non-residual) tactile latents through the identical VQ + SAG pipeline. Without that control, the performance gap versus π0.5 w/ T-UniT (42.3% → 62.8%) cannot be attributed specifically to residualization rather than to VQ discretization or gating alone. A residual-vs-raw ablation (or an explicit MI / reconstruction probe) is load-bearing for the central mechanistic claim.","section":"§III-B, Eq. (1), Table I"},{"comment":"All quantitative claims rest on 15–25 physical trials per condition with point estimates only (Tables I–III). No standard errors, confidence intervals, or statistical tests are reported. Given the absolute gains claimed (+34.6% average, up to +46.7% on individual phases) and the known trial-to-trial variance of contact-rich real-robot evaluation, the absence of uncertainty quantification weakens the strength of the superiority statements. At minimum, bootstrap or binomial CIs (or repeated seeds) should be added for the main table and the robustness suite.","section":"Table I, Table III, §IV-A"},{"comment":"The CMP is pre-trained and frozen on multi-task interaction data drawn from the same five tasks later used for policy evaluation (§III-A, §IV-A). Consequently, residual quality and the semantics of the VQ codebook (Fig. 4) are never probed on held-out objects, novel contact dynamics, or sensor placements outside the training distribution. The robustness suite (Table III) perturbs execution conditions but not the residual extractor itself. A modest OOD residual probe (e.g., novel object geometry or different GelSight mounting) would substantially strengthen the claim that Latent Contact Primitives capture general physical events rather than task-specific patterns.","section":"§III-B, Fig. 4, Table III"}],"minor_comments":[{"comment":"Eq. (1) writes L_pred = λ_σ log σ²_t + ∥z_t − μ_t∥² / σ²_t; the conventional Gaussian NLL also includes a ½ factor and often a constant. Clarify whether the omitted constants are absorbed into λ_σ or whether the loss is intentionally unnormalized.","section":"§III-B, Eq. (1)"},{"comment":"Table II reports a single average over only Plug Insertion and Plate Wiping; per-task numbers (or at least the two individual rates) would make the −26.7% / −13.3% deltas easier to interpret.","section":"Table II"},{"comment":"Fig. 1 caption and the main text both use ‘Surprise Signal’ / ‘surprise-aware gate’; ensure consistent capitalization and that g_t is defined before first use in the figure.","section":"Fig. 1, §III-C"},{"comment":"The abstract states ‘up to 86.7% task success’; Table I shows 86.7% only for Peg-A (alignment). Clarify that the peak is phase-specific, not full-task success, to avoid overstatement.","section":"Abstract, Table I"},{"comment":"Related Work §II-B correctly notes that contrastive alignment can suppress residual information; a short citation to predictive-coding robotics or forward-model residual work (beyond the neuroscience refs) would better situate the contribution for the robotics audience.","section":"§II-B"}],"recommendation":"major_revision","confidential_remarks":"Solid real-robot systems paper with a clean idea; the residual story is currently under-supported relative to the abstract’s claims, but the empirical gains and ablations of VQ/gating are real. I expect the residual-vs-raw ablation and basic error bars to be straightforward to add. Scope fits a robotics journal well; novelty is incremental but useful. No integrity concerns."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful takeaway is simple: residualizing tactile against a wrist-camera prior, quantizing the residual into contact primitives, and gating by the prior’s uncertainty lifts a strong π0.5 baseline by roughly 35 points absolute on five real contact-rich tasks, and the same residual idea also helps Diffusion Policy. That is a concrete systems result, not just a re-labeling of “add tactile.”\n\nWhat is new is the pipeline, not the neuroscience slogan. CMP (ResNet-18 + MLP predicting μ, σ from wrist view only), residual r = z − μ, VQ into Latent Contact Primitives, then SAG g = sigmoid(MLP(σ)) that interpolates between the primitive and a learned “no-contact” token before the flow-matching action expert. They freeze the CMP after multi-task pre-training and fine-tune the policy. Baselines are fair: vision-only π0.5 and DP, naïve ResNet tactile, UniT tactile, and residual-on-DP. Phase-wise metrics (alignment vs interaction), ablations of VQ and gating (Table II: −27 and −13 points), t-SNE of the codebook, gate traces, and three robustness suites (grasp noise, mid-execution target shift, height change) all point the same direction. Math is standard NLL + VQ + flow matching; citations cover the right VLA and tactile papers.\n\nSoft spots are real but secondary. Trial counts are 15–25 with no error bars or tests, so the 62.8 % average is a point estimate. The stress-test concern is fair: there is no direct MI comparison of residual vs raw tactile, no ablation that runs raw z through the identical VQ+SAG path, and no OOD objects or held-out dynamics for the frozen CMP. The “dense orthogonal information gain” claim therefore rests on end-task success and the free-space vs contact clustering. That does not sink the empirical result inside the reported regime, but it does mean the predictive-coding story is stronger than the evidence that residual is strictly better than a well-gated raw tactile stream. Hyperparameters (λσ, λp, codebook size) are free and un-swept in the text.\n\nThis is for people building tactile-augmented VLAs or contact policies who want a working residual+gate recipe they can try. It deserves a serious referee; the experiments are honest enough and the gains large enough that the missing residual probes and statistics can be requested rather than used for desk rejection. I would engage with it.","headline":"Solid residual-tactile VLA recipe with real gains on five contact tasks; predictive-coding framing is mostly packaging and residual orthogonality is only shown end-to-end.","tokens_in":14153,"tokens_out":617,"would_cite":true,"duration_ms":5655,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Robots overcome vision-dominated failure in contact tasks by sensing only the unexpected touch residuals that vision cannot predict.","keywords":["residual tactile representation","vision-language-action","contact-rich manipulation","predictive coding","modality collapse","surprise-aware gating","latent contact primitives"],"falsifier":"Train the identical residual-gated policy and a strong naïve-touch baseline on the same five tasks, then evaluate both on a held-out contact-rich task under occlusion; if residual gating no longer yields a clear success-rate advantage, or if the gate stays near zero throughout contact phases, the central claim fails.","tokens_in":14064,"feed_emoji":"✋","tokens_out":848,"duration_ms":16538,"temperature":0.7,"pith_summary":"Simply bolting tactile sensors onto vision-language-action robot policies usually fails: continuous high-bandwidth vision drowns out the sparse, event-driven touch signals. This paper claims the fix is to stop feeding raw touch and instead feed only the residual—the difference between what the wrist camera predicts the contact should feel like and what the tactile sensor actually reports. Those residuals are turned into a small set of discrete Latent Contact Primitives and admitted into the policy only when the visual prediction itself is uncertain. On five real contact-rich tasks (screwing, insertion, wiping) the resulting policy raises average success from roughly 28 % to 63 % and stays robust when grasps are noisy or targets move mid-execution. A reader who cares about generalist robots should care because contact skills that humans take for granted remain the clearest bottleneck for today’s vision-centric models.","feed_headline":"Residual touch lifts robot contact success by 35 points","feed_subtitle":"Sensing only what vision cannot predict keeps policies robust under occlusion and disturbance.","key_machinery":"Residual Tactile Representation: the discrepancy r_t = z_t − ẑ_t produced by a Cross-Modal Predictor, discretized by a vector-quantized bottleneck into Latent Contact Primitives, then modulated by a Surprise-Aware Gate driven by the predictor’s own uncertainty σ_t. It converts sparse touch into dense information gain that is injected only when vision is unreliable.","core_discovery":"ResTacVLA shows that reformulating tactile input as the residual between a visual prior and physical sensation, quantizing that residual into Latent Contact Primitives, and gating it by visual-prediction uncertainty systematically prevents modality collapse and produces large, consistent gains on contact-rich manipulation.","pith_inferences":["The same residual-coding idea could keep other sparse modalities (force, audio, proprioceptive spikes) from being drowned by vision or language.","If the visual prior is trained on broader data, the residual stream may surface rarer contact events without extra human labels.","Failure under lighting shifts or tactile-sensor recalibration would be the natural next stress test of the orthogonality claim."],"forward_implications":["Contact-rich VLA policies can obtain large success gains without redesigning the vision-language backbone.","Tactile input becomes useful exactly in the occluded or force-critical phases where vision fails, rather than acting as constant noise.","The residual-plus-gate pattern is architecture-agnostic and already improves a Diffusion Policy baseline by the same mechanism.","Robustness to grasp noise, mid-execution target shifts and surface-height changes follows directly from prioritizing residual surprise.","The learned Latent Contact Primitives form a compact, cross-task vocabulary of physical events that vision alone cannot see."],"fun_headline_variants":["Residual touch captures what vision misses for contact robots","Tactile residuals halt vision collapse in contact-rich tasks","Unexpected touch residuals boost contact success under disturbance","Gated residual touch primitives fix robot modality clash","Vision-unpredictable tactile codes raise contact-rich success"],"cache_read_input_tokens":128,"weakest_assumption_plain":"The method assumes a wrist-camera predictor can extract residuals that carry enough orthogonal physical information for a simple uncertainty gate to decide, reliably, when touch should override vision.","fun_headline_variants_meta":{"raw":{"variants":["Residual touch captures what vision misses for contact robots","Tactile residuals halt vision collapse in contact-rich tasks","Unexpected touch residuals boost contact success under disturbance","Gated residual touch primitives fix robot modality clash","Vision-unpredictable tactile codes raise contact-rich success"]},"model":"grok-4.5","effort":"low","cost_usd":0.006626,"raw_usage":{"total_tokens":1663,"prompt_tokens":742,"num_sources_used":0,"completion_tokens":78,"cost_in_usd_ticks":66260000,"prompt_tokens_details":{"text_tokens":742,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":843,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":742,"tokens_out":78,"duration_ms":7023,"temperature":1.0,"reasoning_tokens":843,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-12T02:50:51.200859+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Train the identical residual-gated policy and a strong naïve-touch baseline on the same five tasks, then evaluate both on a held-out contact-rich task under occlusion; if residual gating no longer yields a clear success-rate advantage, or if the gate stays near zero throughout contact phases, the central claim fails.","supporting_citations":[],"review_version":1}