Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

A decoupled multimodal diffusion policy with a plugin tactile adapter achieves 72.25% average success on bimanual dexterous tasks, and the adapter adds another 10.25 points while tuning under 10% of parameters.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 04:13 UTC pith:V756AH4F

load-bearing objection The DECO-50 dataset and ~2,000 real-robot rollouts make this worth serious attention, but the tactile adapter's headline gain conflates tactile injection with LoRA fine-tuning, and the statistics are thinner than the numbers suggest. the 4 major comments →

arxiv 2602.05513 v3 pith:V756AH4F submitted 2026-02-05 cs.RO cs.AI

DECO: Decoupled Multimodal Diffusion Transformer for Bimanual Dexterous Manipulation with a Plugin Tactile Adapter

Xukun Li , Yu Sun , Lei Zhang , Bosheng Huang , Yibo Peng , Yuan Meng , Haojun Jiang , Shaoxuan Xie
show 5 more authors
Guocai Yao Alois Knoll Zhenshan Bing Xinlong Wang Zhenguo Sun
This is my paper
classification cs.RO cs.AI MSC 68T40
keywords bimanual dexterous manipulationmultimodal diffusion transformertactile sensingLoRA adapterparameter-efficient fine-tuningcontact-rich manipulationflow matchingrobot imitation learning dataset
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that fusing vision, proprioception, and touch into a single coupled conditioning stream wastes information, because the modalities play different roles in generating actions. It proposes DECO, a diffusion transformer that gives each modality its own injection pathway — joint self-attention for images, adaptive layer normalization for joint states, and cross-attention for tactile signals — and shows this decoupled design outperforms coupled fusion on real bimanual dexterous tasks. The second claim is that tactile sensing can be added to an already-trained vision policy through a small LoRA-based adapter, without retraining the backbone, and that this yields its largest gains on contact-rich tasks like lid closing and assembly. Alongside the method, the paper releases a 50-hour, 5M-frame bimanual dataset with tactile readings so other groups can reproduce or challenge the results.

Core claim

DECO's central claim is that how a modality is injected matters as much as whether it is injected. The model builds a Multimodal Diffusion Transformer block in which image tokens and action tokens share joint self-attention, proprioceptive state and task conditions enter through adaptive layer normalization, and tactile embeddings are injected via cross-attention with a LoRA adapter. Trained with a flow-matching velocity objective on the released DECO-50 dataset and evaluated in over 2,000 real rollouts, DECO reports the best per-task performance across all four bimanual tasks, a 72.25% average success rate (21% higher than the strongest baseline), and, with the tactile adapter, an additiona

What carries the argument

The load-bearing mechanism is the decoupled conditioning scheme inside the MMDiT block: visual information is fused with action tokens through joint self-attention; proprioception, task one-hot conditions, and diffusion timesteps are folded in via AdaLN (adaptive layer normalization); and tactile signals are injected through a dedicated cross-attention module. A LoRA adapter in the attention projections lets the second-stage tactile adapter update only a small fraction of the frozen vision-action backbone, which is what makes the 'plugin' claim concrete. The two-stage schedule — train vision-only first, freeze, then add tactile — turns tactile integration into a parameter-efficient fine-tuni

Load-bearing premise

The comparisons are controlled enough that the measured gains can be attributed to the decoupled architecture and tactile adapter, even though the baselines use a different image encoder (ResNet-18 vs ResNet-34), different numbers of parameters, different inference schedulers (10 DDIM steps vs 5 flow-matching steps), and different training epochs (200 vs 150).

What would settle it

Run DECO with the exact baseline configuration — ResNet-18 image encoder, 10 DDIM inference steps, 200 training epochs — and re-do the 2,000-rollout evaluation. If the average success rate drops to the baseline's 51–57% range, decoupling itself is not what produces the 21% gain. For the adapter claim, an independent test is to freeze the vision backbone and tune only the LoRA parameters on a new contact-rich task never seen in DECO-50; if the 20% gain does not appear, the plugin's generality is unsupported.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • On weakly tactile tasks like pick-and-place and material sorting, vision and proprioception suffice; the paper's results imply tactile collection effort can be skipped there.
  • On contact-rich tasks, the adapter's stage-wise breakdowns show touch specifically rescues lid-closing and assembly stages where vision cannot tell whether contact or completion has occurred.
  • Because the backbone stays frozen, the adapter can in principle be attached to any pretrained vision-based policy, not only DECO.
  • The released dataset gives other groups a way to compare tactile fusion strategies on the same 50-hour, 5M-frame corpus.
  • Coupled injection of tactile embeddings (DECO.cs) performs no better than the vision-only model, which the paper treats as evidence that decoupling, not just adding data, drives the gain.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the decoupling result transfers, the design principle — route each sensor modality through the conditioning path that matches its sparsity and role — could be applied to larger vision-language-action models, where freezing a pretrained policy and grafting tactile via LoRA cross-attention would be far cheaper than retraining.
  • The 20% gain on contact-rich tasks suggests that many failures in current bimanual systems come from invisible contact states; equipping policies with even low-dimensional force or touch cues may close that gap without architectural changes.
  • A testable extension: train DECO.p with tactile but no task-level one-hot condition to see whether the adapter partially substitutes for task identification, which would indicate the tactile stream carries object- or phase-specific information.
  • The OOD material-sorting numbers hint that tactile helps most on small, slippery objects; a focused benchmark varying object size and surface friction would sharpen the claim about when touch is worth the sensor cost.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes DECO, a diffusion-transformer policy with decoupled multimodal conditioning: visual tokens attend jointly with action tokens, proprioception/task/diffusion-step information enters through AdaLN, and tactile embeddings enter through cross-attention. A second contribution is a LoRA-based plugin tactile adapter applied to a frozen pretrained DECO, updating 7.97M of 91.02M parameters. The authors release DECO-50, a 50-hour bimanual dexterous manipulation dataset with tactile sensing, and evaluate on four real-robot tasks with over 2,000 rollouts. Headline results are a 72.25% average success rate for DECO vs 57.25% for the best baseline ACT, and an 82.50% average for DECO with the tactile adapter (DECO.p), i.e. +10.25 points over DECO and +20 points on contact-rich tasks. An ablation on assembly tasks compares coupled tactile addition (DECO.cs), decoupled from-scratch injection (DECO.ds), and the plugin adapter (DECO.p).

Significance. If the empirical claims hold, the decoupled conditioning design and parameter-efficient tactile adapter are a useful contribution to extending pretrained visuomotor policies with tactile sensing, and DECO-50 is a substantial community resource. The paper's strengths include real physical rollouts with large trial counts, released code/dataset, and ablations showing that decoupled cross-attention-based tactile injection outperforms simple coupled addition. There are no fitted-constant predictions or circular metrics; the results are measured success counts. However, the current evidence does not yet fully control for the adapter's LoRA fine-tuning effect or for capacity/training differences in baselines, so the causal attribution of the headline gains needs strengthening.

major comments (4)
  1. [§3.3, Table 1, Table 5, Table 6] The claim that the tactile adapter adds 10.25 points (DECO.p 82.50% vs DECO 72.25% in Table 1) conflates tactile signal injection with LoRA fine-tuning. DECO.p trains 7.97M LoRA parameters on the pretrained attention projections (Fig. 4, Table 6), whereas the DECO baseline is frozen and receives no task-specific adaptation. No tactile-absent LoRA control is provided. The from-scratch ablation in Table 5 compares coupled vs decoupled tactile injection but does not isolate the adapter's fine-tuning effect, since DECO.ds is trained from scratch and DECO.p combines LoRA with tactile. Please add a control with the same LoRA adapter trained with tactile inputs masked/removed, or a version with tactile cross-attention but no LoRA, to attribute the observed gain to tactile content.
  2. [Table 10, §4.2] Baselines are not capacity- or inference-matched. DECO uses ResNet-34 while ACT/DP use ResNet-18; model sizes are 83.05M vs 51.60/76.46M; DECO uses 5 flow-matching steps vs 10 DDIM steps for DP; training epochs differ (150 vs 200); ACT uses temporal ensembling and an execution chunk size of 1. Thus the '21% improvement over the baseline' and the per-task comparisons in Table 1 may be driven by architecture capacity, compute, or inference budget rather than the decoupled design. Please provide capacity-matched baselines using the same image encoder, comparable parameter counts, and identical denoising-step budgets, or hold these factors constant in an ablation.
  3. [Appendix B.1, Tables 1–5] Success counts are single-deployment point estimates with no repeated seeds or confidence intervals. Appendix B.1 states that 'we select the best-performing checkpoint based on validation performance and deploy it for real-world evaluation.' Since the central claims are differences of 10–20 percentage points and each condition uses only 20 trials per object, selection bias from validation-checkpoint picking and binomial noise need to be quantified. Please report mean ± std or confidence intervals over at least three training seeds and deployment repetitions, or otherwise justify why the checkpoint-selection procedure does not inflate the reported numbers.
  4. [Table 3 vs Table 13] The waste-disposal counts are internally inconsistent. Table 3 reports DP.t as Stage1 72/80, Stage2 65/80, Stage3 48/80 and DECO.p Stage2 76/80, while Appendix Table 13 sums to DP.t 77/43/38 and DECO.p Stage2 72/80. These discrepancies affect the baseline comparison and the characterization of tactile gains in Table 1. The tables need to be reconciled and corrected, and the corrected totals should be used in the headline percentages.
minor comments (4)
  1. [Table 4] The DECO.p row reports '55/100' but the four per-object denominators are each 20, and Table 14 confirms the sum is 55/80. This arithmetic typo should be corrected.
  2. [Abstract, Table 1] The phrase 'additional 10.25% average success rate' and '20% gain' should be clarified as percentage-point improvements, not relative improvements.
  3. [Eq. (4)] The notation Sinusoid(σ(W·T)·T) is ambiguous. Please specify the dimensions of T and W, how the sinusoid embedding is computed, and how the gating is applied before the MLP.
  4. [Table 10 notes] The note says DP's U-Net channels were modified to 'similar parameter scale to our model,' but the resulting parameter count still differs. State the original and modified channel configurations explicitly.

Circularity Check

0 steps flagged

No significant circularity found: the central results are real-robot success rates compared against external baselines and internal ablations, not reductions to fitted inputs or self-cited theorems.

full rationale

DECO's load-bearing claims are measured success rates from physical rollouts (Tables 1–5, 11–14), evaluated against re-implemented ACT and DP baselines and against internal ablations (DECO.cs, DECO.ds, DECO.p). The diffusion/flow-matching objective (Eqs. 1–2) is a standard training loss, not a derivation that assumes its own conclusion, and no fitted parameter is later renamed as a prediction. The tactile-adapter comparison (DECO vs DECO.p) changes both the tactile input and LoRA fine-tuning, which is an experimental confound and a real correctness risk, but it is not circularity: the reported gain is not equal to the adapter's construction, and Table 5's from-scratch decoupled-vs-coupled ablation provides independent evidence about the injection mechanism. The paper cites overlapping-author works (e.g., RoboCOIN, RoboMIND) only for dataset-scarcity context; these citations are not used to derive the success-rate claims. Selecting the best validation checkpoint is a standard evaluation choice, not a fitted-input-called-prediction step. No circular step can be exhibited by equation or by construction, so the honest finding is a score of 0.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

No new physical entities. The method introduces learned parameters only. The central claim rests on domain assumptions about touch informativeness and evaluation representativeness, plus a set of hand-picked training hyperparameters.

free parameters (4)
  • Inference steps k = 5 for DECO, 10 for DP
    Hand-chosen; differs across compared methods, so it is part of the comparison confound in Table 1.
  • LoRA rank/alpha of tactile adapter = not reported
    Controls capacity of the tactile adapter; rank and alpha are omitted from the text.
  • Sigmoid flow-matching timestep schedule = sigma(xi)=1/(1+exp(-xi))
    Introduced in Section 3.1 without ablation; changes the loss weighting over flow timesteps.
  • Tactile gating matrix W = learned
    Learnable linear projection in Eq. (4); fitted to data during adapter training.
axioms (4)
  • domain assumption Tactile pads (17 per hand) plus the two-branch encoder preserve contact information needed for contact-rich tasks.
    Central to the tactile adapter's benefit; no sensor-noise or information-content analysis is provided.
  • domain assumption Human teleoperation demonstrations are a representative training distribution for autonomous execution.
    Policy trained on demos and evaluated in the same setup; no coverage analysis is given.
  • standard math Flow-matching loss (Eq. 1) is a valid objective for action-chunk generation.
    Follows established generative modeling, though the paper does not cite the flow-matching originals.
  • domain assumption Ten to twenty real-robot rollouts per condition with one selected checkpoint quantify policy quality.
    Small trial counts and validation checkpoint selection make reported success rates noisy.

pith-pipeline@v1.3.0-alltime-deepseek · 19639 in / 17659 out tokens · 176069 ms · 2026-08-03T04:13:00.359539+00:00 · methodology

0 comments
read the original abstract

Bimanual dexterous manipulation relies on integrating multimodal inputs to perform complex real-world tasks. To address the challenges of effectively combining these modalities, we propose DECO, a decoupled multimodal diffusion transformer that disentangles vision, proprioception, and tactile signals through specialized conditioning pathways, enabling structured and controllable integration of multimodal inputs, with a lightweight adapter for parameter-efficient injection of additional signals. Alongside DECO, we release DECO-50 dataset for bimanual dexterous manipulation with tactile sensing, consisting of 50 hours of data and over 5M frames, collected via teleoperation on real dual-arm robots. We train DECO on DECO-50 and conduct extensive real-world evaluation with over 2,000 robot rollouts. Experimental results show that DECO achieves the best performance across all tasks, with a 72.25% average success rate and a 21% improvement over the baseline. Moreover, the tactile adapter brings an additional 10.25% average success rate across all tasks and a 20% gain on complex contact-rich tasks while tuning less than 10% of the model parameters.

Figures

Figures reproduced from arXiv: 2602.05513 by Alois Knoll, Bosheng Huang, Guocai Yao, Haojun Jiang, Lei Zhang, Shaoxuan Xie, Xinlong Wang, Xukun Li, Yibo Peng, Yuan Meng, Yu Sun, Zhenguo Sun, Zhenshan Bing.

Figure 1
Figure 1. Figure 1: Overview of the Proposed DECO Framework. DECO is a DiT-based policy that decouples multimodal conditioning. Image and action tokens interact via joint self attention, while proprioceptive states and optional conditions are injected through adaptive layer normalization. Tactile signals are injected via cross attention, while a lightweight LoRA-based adapter is used to efficiently fine-tune the pretrained po… view at source ↗
Figure 2
Figure 2. Figure 2: Two-Stage Training Paradigm for DECO. In the first stage, a vision–action policy is trained with images, proprioceptive states and task-level conditions. In the second stage, the pretrained policy is frozen, and tactile signals are incorporated via a lightweight adapter and cross attention, enabling parameter-efficient adaptation to tactile-aware manipulation without retraining the entire model. visual and… view at source ↗
Figure 3
Figure 3. Figure 3: Multimodal Diffusion Transformer Block with Decoupled Conditioning. Images via self-attention, pro￾prioceptive states via AdaLN, and tactile signals via cross￾attention, enabling independent and efficient integration of each modality. The velocity predictor is built upon our Multi-Modal Dif￾fusion Transformer (MMDiT) Block, as illustrated in Fig￾ure 3. The core design principle of MMDiT is to decouple moda… view at source ↗
Figure 4
Figure 4. Figure 4: Plugin Tactile Adapter. Raw tactile information is encoded by the tactile encoder and integrated into the pretrained policy via LoRA for efficient adaptation. while Low Rank Adaptation(LoRA) (Hu et al., 2021) se￾lectively fine-tunes the attention layers of the pretrained vision–action backbone. During the second training stage, the pretrained policy is frozen, and only the adapter param￾eters are optimized… view at source ↗
Figure 6
Figure 6. Figure 6: Objects used in (a) Pick-and-Place, (b) Waste Dis￾posal, (c) ablation experiments, (d) Assembly, (e) Material Sorting, and (f) the piece–correspondence mapping. into the input and the model is trained from scratch, and DECO.p, where the tactile adapter is integrated into our pretrained vision-based policy and fine-tuned. Real-World Setting. We follow the same setup as in data collection to ensure consisten… view at source ↗
Figure 7
Figure 7. Figure 7: DECO with and without tactile on Waste Disposal and Assembly tasks two strongly tactile-dependent phases: opening and closing the bin lid. We report success counts up to each of three stages: Stage 1 (open lid), Stage 2 (pick and throw trash), and Stage 3 (close lid). Successfully opening or closing the lid requires applying appropriate torque, which depends on both the applied force and the moment arm det… view at source ↗
Figure 8
Figure 8. Figure 8: Data collection setup with the active camera. A.3. Data Collection We set up a bimanual teleoperation system with a custom active camera mounted on Unitree H1-2. The robot is equipped with a pair of Inspire RH56DFTP hands; each hand has 6 DOF and 17 tactile pads (1062 contact points in total, each with values 0–4096). The dual arms have 14 DOF and the active camera has yaw and pitch motors. During teleoper… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Data Pyramid for Embodied Manipulation

    cs.RO 2026-07 conditional novelty 3.0

    Embodied training data form a five-layer pyramid—real-robot, UMI, ego/exo, simulation, general V–L—ordered by the trade-off between scale and robot alignment, and model capabilities track how those layers are mixed.

Reference graph

Works this paper leans on

26 extracted references · 15 linked inside Pith · cited by 1 Pith paper

  1. [5]

    Collaboration, O

    ISSN 0278-3649, 1741-3176. Collaboration, O. X.-E., O’Neill, A., Rehman, A., Gupta, A., Maddukuri, A., Gupta, A., Padalkar, A., Lee, A., Pooley, A., Gupta, A., Mandlekar, A., Jain, A., Tung, A., Bewley, A., Herzog, A., Irpan, A., Khazatsky, A., Rai, A., Gupta, A., Wang, A., Kolobov, A., Singh, A., Garg, A., Kembhavi, A., Xie, A., Brohan, A., Raffin, A., S...

    Pith/arXiv arXiv

  2. [6]

    ViTacFormer: Learning Cross-Modal Representation for Visuo-Tactile Dexterous Manipulation.Preprint at arXiv:2506.15953,

    Heng, L., Geng, H., Zhang, K., Abbeel, P., and Malik, J. ViTacFormer: Learning Cross-Modal Representation for Visuo-Tactile Dexterous Manipulation.Preprint at arXiv:2506.15953,

    Pith/arXiv arXiv

  3. [7]

    K., Boots, B., Kaess, M., Lambeta, M., Wu, T., Liu, Z., Hogan, F

    Higuera, C., Sharma, A., Fan, T., Bodduluri, C. K., Boots, B., Kaess, M., Lambeta, M., Wu, T., Liu, Z., Hogan, F. R., and Mukadam, M. Tactile Beyond Pixels: Multisensory Touch Representations for Robot Manipulation.Preprint at arXiv:2506.14754,

    Pith/arXiv arXiv

  4. [8]

    RoboMIND 2.0: A Multimodal, Bimanual Mo- bile Manipulation Dataset for Generalizable Embodied Intelligence.Preprint at arXiv:2512.24653,

    Hou, C., Wu, K., Liu, J., Che, Z., Wu, D., Liao, F., Li, G., He, J., Feng, Q., Jin, Z., Gu, C., Liu, Z., Han, N., Mi, X., Lv, Y ., Fu, Y ., Dai, G., Gu, L., Li, T., Zhang, Y ., Zhang, Y ., Wang, X., Fan, S., Li, M., Zhao, Z., Liu, N., Xu, Z., Ren, P., Ji, J., Liu, H., Cheng, K., Zhang, S., and Tang, J. RoboMIND 2.0: A Multimodal, Bimanual Mo- bile Manipul...

    arXiv

  5. [9]

    J., Shen, Y ., Wallis, P., Allen-Zhu, Z., Li, Y ., Wang, S., Wang, L., and Chen, W

    Hu, E. J., Shen, Y ., Wallis, P., Allen-Zhu, Z., Li, Y ., Wang, S., Wang, L., and Chen, W. LoRA: Low-Rank Adaptation of Large Language Models.Preprint at arXiv:2106.09685,

    Pith/arXiv arXiv

  6. [11]

    Lambeta, M., Wu, T., Sengul, A., Most, V . R., Black, N., Sawyer, K., Mercado, R., Qi, H., Sohn, A., Taylor, B., Tydingco, N., Kammerer, G., Stroud, D., Khatha, J., Jenk- ins, K., Most, K., Stein, N., Chavira, R., Craven-Bartle, T., Sanchez, E., Ding, Y ., Malik, J., and Calandra, R. Digitizing Touch with an Artificial Multimodal Fingertip. Preprint at ar...

    Pith/arXiv arXiv

  7. [12]

    The De- velopments and Challenges towards Dexterous and Em- bodied Robotic Manipulation: A Survey.Preprint at arXiv:2507.11840,

    Li, G., Wang, R., Xu, P., Ye, Q., and Chen, J. The De- velopments and Challenges towards Dexterous and Em- bodied Robotic Manipulation: A Survey.Preprint at arXiv:2507.11840,

    arXiv

  8. [13]

    RDT-1B: A Diffusion Foundation Model for Bimanual Manipulation

    Liu, S., Wu, L., Li, B., Tan, H., Chen, H., Wang, Z., Xu, K., Su, H., and Zhu, J. RDT-1B: A Diffusion Foundation Model for Bimanual Manipulation. InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025,

    2025

  9. [14]

    Being-H0: Vision- Language-Action Pretraining from Large-Scale Human Videos.Preprint at arXiv:2507.15597,

    Luo, H., Feng, Y ., Zhang, W., Zheng, S., Wang, Y ., Yuan, H., Liu, J., Xu, C., Jin, Q., and Lu, Z. Being-H0: Vision- Language-Action Pretraining from Large-Scale Human Videos.Preprint at arXiv:2507.15597,

    Pith/arXiv arXiv

  10. [15]

    Being-H0.5: Scaling Human-Centric Robot Learn- ing for Cross-Embodiment Generalization.Preprint at arXiv:2601.12993,

    Luo, H., Wang, Y ., Zhang, W., Zheng, S., Xi, Z., Xu, C., Xu, H., Yuan, H., Zhang, C., Wang, Y ., Feng, Y ., and Lu, Z. Being-H0.5: Scaling Human-Centric Robot Learn- ing for Cross-Embodiment Generalization.Preprint at arXiv:2601.12993,

    arXiv

  11. [16]

    RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version)

    Mu, Y ., Chen, T., Peng, S., Chen, Z., Gao, Z., Zou, Y ., Lin, L., Xie, Z., and Luo, P. RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version). Preprint at arXiv:2409.02920,

    Pith/arXiv arXiv

  12. [17]

    Sliwowski, D., Jadav, S., Stanovcic, S., Orbik, J., Heiders- berger, J., and Lee, D

    IEEE. Sliwowski, D., Jadav, S., Stanovcic, S., Orbik, J., Heiders- berger, J., and Lee, D. REASSEMBLE: A Multimodal Dataset for Contact-rich Robotic Assembly and Disas- sembly.Preprint at arXiv:2502.05086,

    Pith/arXiv arXiv

  13. [18]

    J., Du, M., Zheng, C., Zhao, T., Hansen-Estruch, P., Vuong, Q., He, A., Myers, V ., Fang, K., Finn, C., and Levine, S

    Walke, H., Black, K., Lee, A., Kim, M. J., Du, M., Zheng, C., Zhao, T., Hansen-Estruch, P., Vuong, Q., He, A., Myers, V ., Fang, K., Finn, C., and Levine, S. BridgeData V2: A Dataset for Robot Learning at Scale.Preprint at arXiv:2308.12952,

    Pith/arXiv arXiv

  14. [19]

    Robo- MIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulation.Preprint at arXiv:2412.13877, 2025a

    Wu, K., Hou, C., Liu, J., Che, Z., Ju, X., Yang, Z., Li, M., Zhao, Y ., Xu, Z., Yang, G., Fan, S., Wang, X., Liao, F., Zhao, Z., Li, G., Jin, Z., Wang, L., Mao, J., Liu, N., Ren, P., Zhang, Q., Lyu, Y ., Liu, M., He, J., Luo, Y ., Gao, Z., Li, C., Gu, C., Fu, Y ., Wu, D., Wang, X., Chen, S., Wang, Z., An, P., Qian, S., Zhang, S., and Tang, J. Robo- MIND: ...

    Pith/arXiv arXiv 2025

  15. [20]

    ForceVLA: Enhancing VLA Models with a Force- aware MoE for Contact-rich Manipulation.Preprint at arXiv:2505.22159,

    Yu, J., Liu, H., Yu, Q., Ren, J., Hao, C., Ding, H., Huang, G., Huang, G., Song, Y ., Cai, P., Lu, C., and Zhang, W. ForceVLA: Enhancing VLA Models with a Force- aware MoE for Contact-rich Manipulation.Preprint at arXiv:2505.22159,

    arXiv

  16. [21]

    VTLA: Vision-Tactile-Language-Action Model with Preference Learning for Insertion Manipulation.Preprint at arXiv:2505.09577, 2025a

    Zhang, C., Hao, P., Cao, X., Hao, X., Cui, S., and Wang, S. VTLA: Vision-Tactile-Language-Action Model with Preference Learning for Insertion Manipulation.Preprint at arXiv:2505.09577, 2025a. Zhang, T., Li, D., Li, Y ., Zeng, Z., Zhao, L., Sun, L., Chen, Y ., Wei, X., Zhan, Y ., Li, L., and He, X. Empower- ing Embodied Manipulation: A Bimanual-Mobile Robo...

    Pith/arXiv arXiv

  17. [22]

    TA-VLA: Elucidating the Design Space of Torque-aware Vision-Language-Action Models

    Zhang, Z., Xu, H., Yang, Z., Yue, C., Lin, Z., Gao, H.-a., Wang, Z., and Zhao, H. TA-VLA: Elucidating the Design Space of Torque-aware Vision-Language-Action Models. Preprint at arXiv:2509.07962, 2025b. Zhao, T. Z., Kumar, V ., Levine, S., and Finn, C. Learn- ing Fine-Grained Bimanual Manipulation with Low-Cost Hardware. In Bekris, K. E., Hauser, K., Herb...

    Pith/arXiv arXiv

  18. [23]

    D., Kolobov, A., Huang, F., and Yang, J

    Zheng, R., Liang, Y ., Huang, S., Gao, J., III, H. D., Kolobov, A., Huang, F., and Yang, J. TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policies. InThe Thirteenth Inter- national Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025,

    2025

  19. [24]

    Dataset Details A.1

    12 Decoupled Multimodal Diffusion Transformer for Bimanual Dexterous Manipulation with a Plugin Tactile Adapter Supplementary Material A. Dataset Details A.1. Dataset Size Table 7.Detailed success statistics and total collected hours for the four tasks. Task Object Succ./Total Traj. Succ./Total Hours Task 1 Onion 132/160 1.001/1.267 Apple 113/131 1.003/1....

    1944

  20. [25]

    All components are printed in black or white Bambu Lab PLA Basic, except for the soft hose in the custom IF pair, which is printed in TPU

    Additionally, we design a custom pair of interference-fit (IF) parts for the ablation study. All components are printed in black or white Bambu Lab PLA Basic, except for the soft hose in the custom IF pair, which is printed in TPU. 13 Decoupled Multimodal Diffusion Transformer for Bimanual Dexterous Manipulation with a Plugin Tactile Adapter (a) Data Coll...

    2024

  21. [2021]

    VT-Refine: Learning Bimanual Assembly with Visuo-Tactile Feedback via Simulation Fine-Tuning.Preprint at arXiv:2510.14930, 2025a

    Huang, B., Xu, J., Akinola, I., Yang, W., Sundaralingam, B., O’Flaherty, R., Fox, D., Wang, X., Mousavian, A., Chao, Y .-W., and Li, Y . VT-Refine: Learning Bimanual Assembly with Visuo-Tactile Feedback via Simulation Fine-Tuning.Preprint at arXiv:2510.14930, 2025a. Huang, J., Wang, S., Lin, F., Hu, Y ., Wen, C., and Gao, Y . Tactile-VLA: Unlocking Vision...

    arXiv

  22. [2023]

    RoboTwin 2.0: A Scalable Data Genera- tor and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation.Preprint at arXiv:2506.18088,

    Chen, T., Chen, Z., Chen, B., Cai, Z., Liu, Y ., Li, Z., Liang, Q., Lin, X., Ge, Y ., Gu, Z., Deng, W., Guo, Y ., Nian, T., Xie, X., Chen, Q., Su, K., Xu, T., Liu, G., Hu, M., Gao, H.-a., Wang, K., Liang, Z., Qin, Y ., Yang, X., Luo, P., and Mu, Y . RoboTwin 2.0: A Scalable Data Genera- tor and Benchmark with Strong Domain Randomization for Robust Bimanua...

    Pith/arXiv arXiv

  23. [2024]

    OmniVTLA: Vision-Tactile-Language- Action Model with Semantic-Aligned Tactile Sensing

    Cheng, Z., Zhang, Y ., Zhang, W., Li, H., Wang, K., Song, L., and Zhang, H. OmniVTLA: Vision-Tactile-Language- Action Model with Semantic-Aligned Tactile Sensing. Preprint at arXiv:2508.08706,

    arXiv

  24. [2025]

    Touch100k: A Large-Scale Touch-Language-Vision Dataset for Touch-Centric Multimodal Representation

    Cheng, N., Guan, C., Gao, J., Wang, W., Li, Y ., Meng, F., Zhou, J., Fang, B., Xu, J., and Han, W. Touch100k: A Large-Scale Touch-Language-Vision Dataset for Touch-Centric Multimodal Representation. Preprint at arXiv:2406.03813,

    Pith/arXiv arXiv

  25. [2026]

    ISSN 1545-5955, 1558-3783. Black, K., Brown, N., Driess, D., Esmail, A., Equi, M., Finn, C., Fusai, N., Groom, L., Hausman, K., Ichter, B., Jakubczak, S., Jones, T., Ke, L., Levine, S., Li-Bell, A., Mothukuri, M., Nair, S., Pertsch, K., Shi, L. X., Tanner, J., Vuong, Q., Walling, A., Wang, H., and Zhilinsky, U. $ π 0$: A Vision-Language-Action Flow Model ...

    Pith/arXiv arXiv

  26. [2048]

    We re-implement ACT and DP by closely following the official LeRobot repository. To ensure a comparable model capacity across different baselines, we modify the default channel configuration of the conditional U-Net used in DP from [512, 1024, 2048] to [256, 512, 1024], resulting in a similar parameter scale to our model. For DP.t, tactile observations ar...

    2048