Pith. sign in

REVIEW 4 major objections 5 minor 31 references

Auditing Instruction-Trajectory Mismatches in Multimodal Robot Demonstrations

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that instruction–trajectory mismatches in multimodal robot demonstration datasets can be detected and corrected post-hoc by a training-free fusion of per-modality neighborhood and prototype evidence, improving downstream…

desk verdict A useful, clearly-written data-curation method for ITM auditing, with real-robot support, but the synthetic LIBERO results are partly inflated by the noise generator's use of the same embeddings. read the letter →

arxiv 2608.07895 v1 pith:ZUE45AZ6 submitted 2026-08-08 cs.RO cs.LG

Simon Holk , Ryosuke Takanami , Tatsuya Matsushima , Yusuke Iwasawa , Yutaka Matsuo , Yueh-Hua Wu , Kei Ota This is my paper · ORCID
classification cs.ROcs.LG
keywords instruction-trajectorymismatchrobotdemonstrationauditingmultimodalfusionproduct-of-expertslabelerrordetectiondatacurationlearningfromvision-language-actionmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Robot demonstrations used to train vision-language-action policies can be behaviorally correct yet paired with the wrong language instruction, a failure mode the paper calls an instruction–trajectory mismatch (ITM). The paper proposes Multimodal Probabilistic Fusion (MMPF), a training-free auditor that, for each sensing modality, builds a task-label distribution from local neighborhood agreement and global prototype similarity, then fuses the modalities with reliability-weighted product of experts. On LIBERO benchmarks with injected mismatches and on two real-robot datasets, MMPF achieves the strongest ITM detection and label-correction accuracy among the compared baselines. The paper also shows that auditing improves downstream policy learning most when language is needed to disambiguate the task, and that relabeling can beat filtering in small real-robot datasets.

What carries the argument

The central object is the reliability-weighted product of experts over unimodal task-label distributions. For modality $m$ and episode $i$, MMPF forms a distribution $p_m(y|i)$ by taking the geometric mixture of an RBF-weighted kNN vote (Eq. 2) and an exponential distance to per-task prototypes (Eq. 3). Predictive entropy converts each unimodal posterior into a reliability weight $r_{i,m}=1-H(p_m)/\log C$, and the final posterior is $\log p(y|i)\propto \sum_m r_{i,m}\log p_m(y|i)$. This mechanism does the work: a modality that is occluded, noisy, or task-ambiguous produces a diffuse posterior, gets downweighted, and is prevented from dragging the joint prediction; agreement across modalities sharpens the final label. The same reliability scores can be thresholded to flag sensor-corrupted episodes independently of label errors.

What would settle it

On a dataset where proprioception and motor trajectories are nearly identical across two tasks that differ only by object identity, inject 30% label flips and run MMPF: if correction accuracy remains near 100% while proprioception-only evidence is at chance, the method does not need per-modality task geometry; if it collapses, the embedding-geometry assumption is the operative load-bearing factor.

Watch

Extended reading notes

Core claim

The paper claims that instruction–trajectory mismatches can be found and corrected without training a classifier or invoking a VLM, by exploiting the geometry of pretrained trajectory embeddings. Each modality (head camera, wrist camera, proprioception) votes on the true task using an RBF-weighted k-nearest-neighbor histogram and a distance-to-prototype distribution; the two are combined by a geometric mixture, and the resulting distributions are fused by a product of experts weighted by per-episode predictive entropy. Across the eight LIBERO settings with instance-dependent and clustered noise, MMPF reaches 88.9–100.0 F1, near-perfect AUROC/AUPRC, and 100.0% correction accuracy among detected true ITMs. On real-robot bottle-mug and table datasets it attains 91.8 and 94.3 F1 with 97.9% and 94.9% correction accuracy. Downstream, MMPF-curated training raises LIBERO-Goal success from 58.7 to 77.5 and LIBERO-Spatial from 70.3 to 76.7, and on a counterfactual object-grounding benchmark it increases faithful grasp rate from 28.3% to 41.4% while cutting biased grasps from 40.3% to 14.3%.

Load-bearing premise

MMPF assumes that, within each modality, demonstrations of the same task are geometrically close and demonstrations of different tasks are far apart, so that nearest neighbors and prototypes reveal the true task rather than viewpoint, object color, or robot-state noise; the paper states the same dependence in its conclusion.

Editorial extensions

If this is right

  • ITM auditing becomes a distinct stage in robot data curation, complementary to filtering failed rollouts or redundant trajectories.
  • Relabeling flagged demonstrations can outperform dropping them when trajectories remain valid, especially in small real-robot datasets.
  • The per-modality reliability score doubles as a sensor-corruption detector, so one auditing pass surfaces both label errors and broken observations.
  • Counterfactual grounding metrics reveal downstream benefits of auditing that clean-environment success alone hides.
  • Because the audit stage is training-free, the method transfers to any dataset that has pretrained trajectory embeddings, including future foundation-model encoders.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial: The framework's contribution is best read as an auditing head on top of whatever trajectory embeddings exist, so its accuracy in practice will ride on the next generation of proprioceptive and video foundation models rather than on the fusion itself.
  • Editorial: The reliability score could be used online during data collection to flag uncertain or sensor-corrupted episodes in real time, turning a post-hoc auditor into a collection-time filter.
  • Editorial: The finite-task-label assumption could be relaxed by replacing hard classes with soft neighborhoods in instruction-embedding space (as the paper itself notes), which would let the same fusion audit open-vocabulary paraphrases; a concrete test is to apply MMPF to a dataset with naturally occurring instruction paraphrases and compare corrections against human audit.
  • Editorial: A natural stress test is to inject mismatches with adversarial similarity, choosing wrong labels whose trajectories are closest in embedding space; the paper's synthetic instance-dependent noise approximates this, but an explicitly adversarial version would bound the method's robustness.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies post-hoc auditing of Instruction–Trajectory Mismatches (ITMs) in multimodal robot demonstration datasets. It proposes MMPF, a classifier-free auditor that derives a per-modality task-label distribution from local kNN agreement and global prototype similarity, scores each modality by predictive entropy, and fuses the modalities through a reliability-weighted product of experts. The method is evaluated on LIBERO with two kinds of injected synthetic noise, on two real-robot datasets with 30% uniform noise, and in downstream policy-learning experiments with smolvla and pi0.5, including a counterfactual grounding evaluation. The central claims are that MMPF achieves the strongest overall ITM detection and label-correction accuracy and that auditing improves downstream policy learning where language is needed to disambiguate the task.

Significance. If the claims hold, the paper makes a useful applied contribution: it formalizes ITM auditing for robot demonstrations, provides a post-embedding training-free detector that requires no clean-label supervision, and demonstrates a practical filtering-versus-relabeling trade-off on real hardware. The method is a sensible integration of established ingredients (weighted kNN voting, prototype distances, product-of-experts fusion), and the paper is honest about relying on embedding quality. The most valuable parts are the real-robot auditing results and the counterfactual grounding evaluation, which directly probe instruction grounding rather than only clean-environment success. However, the headline LIBERO claims are weakened by the fact that the synthetic instance-dependent noise is generated from the same trajectory-embedding geometry that MMPF's kNN evidence uses, and several central quantitative comparisons lack uncertainty estimates. These issues are fixable and do not invalidate the approach, but they currently prevent accepting the strongest comparative claims without qualification.

major comments (4)
  1. [IV.A.c and Table I] The instance-dependent LIBERO noise construction is aligned with MMPF's own decision rule. Section IV.A.c states that episodes are corrupted when their nearest neighbors in trajectory-embedding space include demonstrations from different tasks, and that the replacement label is taken from the nearest different-task neighbor. This is exactly the signal that MMPF's local evidence in Eq. (2) keys on, using the same Cosmos-Embed1 embeddings. The near-ceiling AUROC/AUPRC values and the 100% correction accuracy in Table I therefore partly measure how faithfully the noise generator instantiates MMPF's kNN assumption, not how well MMPF detects naturally occurring ITMs. The real-robot uniform-noise results are independent and supportive, but they do not settle the magnitude of the LIBERO advantage. Please add an experimental condition in which synthetic noise is generated without using the auditing embeddings—for example, uniform label noise, noise based on task semantics, or noise injected before embedding extraction—and report the same metrics for that condition.
  2. [Tables I, II, V, VI] The main auditing comparisons are reported as point estimates without uncertainty. Table I has no error bars or significance tests for any of the eight LIBERO settings, and Tables II, V, and VI likewise report single numbers, despite Table III appropriately reporting 95% Student-t intervals. Because the headline claim is 'strongest overall ITM detection and label correction accuracy,' the reader needs to know whether the observed margins over Confident Learning, Retrieval, and LEMoN are stable under dataset construction or seed variation. Please provide confidence intervals or bootstrap/variance estimates for at least the principal detection and correction metrics in Tables I, II, and V, and perform a significance test for the comparisons that support the abstract's strongest-claim wording.
  3. [Table VII and Section IV.G] The real-robot downstream policy claims are not statistically supported at the reported confidence level. Under filtering-based cleaning on Table, MMPF's 78.8% [68.6,86.3] overlaps the unfiltered baseline's 73.8% [63.2,82.1]; under correction-based cleaning on Bottle-mug, MMPF's 77.5% [62.5,87.7] overlaps the unfiltered baseline's 62.5% [47.0,75.8]. The text says MMPF 'obtains the highest observed point estimate' and 'attains an overall better success rate,' which is fair as a point-estimate statement, but the conclusion should be tempered or backed by additional rollouts, seeds, or a paired comparison. As written, the downstream advantage over no filtering is suggestive rather than established.
  4. [IV.A.e and Table I] The correction-accuracy metric is conditional on true-positive detection, which makes the reported 100% values harder to interpret. CorrAcc is defined as |{i in I_TP : y_i_hat = y_i}| / |I_TP|, so it measures the reliability of proposed corrections only among episodes already correctly flagged as ITMs. In settings where recall is substantially below precision, a high CorrAcc can coexist with a large number of uncorrected mismatches. The claim that MMPF has 'highly reliable' correction labels is therefore not the same as a claim that MMPF corrects most ITMs in the dataset. Please also report, for each condition, the absolute fraction of all injected ITMs that are both detected and correctly relabeled, or provide a full confusion matrix for the LIBERO results.
minor comments (5)
  1. [Abstract and Section III] The abstract and conclusion call MMPF 'training-free,' but the robot-state encoder in Section IV.A.b is trained (albeit transductively and without task labels). Please consistently say 'training-free after embedding extraction' to avoid misleading readers.
  2. [IV.A.c] The clustered noise setting also selects clusters in the same Cosmos-Embed1 embedding space used for auditing; while less directly aligned with the kNN vote than the instance-dependent setting, it still deserves a sentence acknowledging that this shared geometry may favor methods built on the same embedding.
  3. [IV.A.d] The description of the Retrieval baseline is ambiguous: it is said to use 'CLIP-style scoring to the same pretrained trajectory embeddings used by MMPF' and also 'ridge regression to align the video and text embedding spaces.' Please clarify whether the baseline is genuine CLIP scoring or a learned linear alignment, since this affects whether it is a strong comparator.
  4. [IV.A.a] CF-Object is described as an independent re-implementation of the benchmark in [29] because the original is not public. Please state explicitly how the re-implementation was validated for equivalence to the original counterfactual protocol, since the grounding claims in Table IV depend on this variant.
  5. [Equations (1)-(6)] The notation would benefit from a short worked example or a table summarizing the values of K, lambda, sigma_m, tau_m, epsilon, and the reliability threshold, since the reader must otherwise infer these from the text in Section IV.A.b and Table VI.

Circularity Check

1 steps flagged · score 6.0 of 10

Partial circularity: LIBERO instance-dependent ITMs are generated from the same Cosmos-Embed1 trajectory-embedding kNN geometry that MMPF's Eq. (2) votes over, so Table I's detection/correction numbers partly measure the constructor's use of MMPF's own decision rule.

  1. self definitional [Section IV.A.c (Noise Injection) vs. Section III.A, Eq. (2)]
    "For each episode, we compute its nearest neighbors in trajectory-embedding space and identify episodes whose closest neighbors include demonstrations from different tasks. We then sample corrupted episodes from these boundary-near regions and replace their label with a confusable task label, chosen from the nearest different-task neighbor. ... Both synthetic corruption settings are constructed using the Cosmos-Embed1 video trajectory embeddings shared by all methods."

    Eq. (2) defines p_kNN^m(y|i) as a weighted histogram of labels among the K nearest neighbors of z_{i,m} in the same modality embedding space. The instance-dependent noise generator selects episodes by their nearest neighbors in trajectory-embedding space and relabels them with the label of the nearest different-task neighbor, using the same Cosmos-Embed1 trajectory embeddings that MMPF uses for its visual modalities. Hence, for every injected ITM, the replacement label is, by construction, the label of a nearby episode in the very space where Eq. (2) performs its vote. The kNN histogram is therefore concentrated on the replacement label; MMPF flags the episode and its argmax correction is exactly the injected label.

full rationale

The method's main inference machinery is not fitted to the target metric: K=30, lambda=0.5, sigma_m, and tau_m are fixed or set from unsupervised embedding statistics, and the robot-state encoder is trained transductively without task-label supervision. No load-bearing self-citation appears: the cited counterfactual benchmark [29] is external, and the paper explicitly frames its contribution as integrating established components rather than claiming a new first-principles result. However, the synthetic LIBERO instance-dependent ITM construction is definitionally aligned with Eq. (2): mismatched episodes are selected and relabeled using nearest-neighbor structure in the same Cosmos-Embed1 trajectory embedding space that MMPF's local evidence votes over, so Table I's detection and correction numbers partly reduce by construction. The clustered-noise setting is also constructed from the same embeddings, though less directly tied to a single equation. The real-robot uniform-noise experiments, the CF-Object grounding evaluation, and the downstream policy comparisons provide independent support and prevent a higher score, but the paper's first benchmark claim ('Across LIBERO benchmarks with injected instruction mismatches ... strongest overall') is inflated by the benchmark-construction circularity. Overall: one central benchmark prediction reduces by construction, while the method's core formulation and real-robot results retain independent content; score 6 reflects this partial circularity.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

MMPF introduces no new physical entities. Its free parameters are standard hyperparameters and scale choices (K, lambda, sigma_m, tau_m, epsilon, reliability threshold, encoder loss weights); none are fitted to the target auditing metric, so circularity burden is low. The main axioms are representation quality, label-majority assumption, finite label set, independent-expert fusion, and embedding suitability.

free parameters (7)
  • K (number of nearest neighbors) = 30
    Fixed hyperparameter in local evidence estimation, Eq. (1)-(2); not tuned per dataset but chosen by hand and affects the local label vote.
  • lambda (local/global mixing weight) = 0.5
    Geometric mixture weight in Eq. (4), set to equal weight for local and global evidence; chosen by hand.
  • sigma_m (RBF bandwidth) = mean pairwise distance per modality
    Bandwidth in Eq. (1), set to data-dependent mean pairwise distance; not fitted to labels but is a free scale choice.
  • tau_m (prototype temperature) = mean prototype distance per modality
    Temperature in Eq. (3), set to mean prototype distance per modality; a scale choice that affects posterior sharpness.
  • epsilon (log smoothing constant) = small constant, value not reported
    Prevents zero-count kNN classes from producing undefined log probabilities in Eq. (4).
  • minimum reliability threshold = 0.1
    Used in the main results to flag noisy episodes; the paper reports sensitivity to this operating point in Table VI.
  • robot-state encoder loss weights = 1, 1, 0.5, 0.5
    Weights for masked state-delta reconstruction, masked action reconstruction, inverse dynamics, and forward-versus-reversed classification; chosen by hand.
assumptions (5)
  • domain assumption Correctly labeled demonstrations of the same task cluster in the embedding space of each modality.
    The local kNN vote (Eq. 2) and prototype similarity (Eq. 3) produce meaningful task evidence only if embedding geometry reflects task identity; the paper acknowledges in Section V that MMPF depends on embedding quality.
  • domain assumption The audited dataset is mostly correctly labeled; label noise is a minority.
    Eq. (2) takes a weighted majority vote of neighbor labels; if most labels were wrong, the vote would be wrong. The paper states in Section I that the setting is "a robot dataset in which most labels are correct."
  • domain assumption Task labels come from a finite set of canonical instructions C, with multiple episodes per task.
    The posterior is over discrete classes 1..C in Section III; the conclusion explicitly limits the method to finite canonical task instructions.
  • domain assumption Per-modality posteriors can be fused as independent experts via a weighted product of experts.
    Eq. (6) multiplies modality distributions, implicitly assuming conditional independence; reliability weighting mitigates but does not remove this assumption.
  • domain assumption Pretrained Cosmos-Embed1 embeddings are suitable trajectory representations for visual modalities, and no task labels leak into the embeddings.
    Section IV-A-b applies Cosmos-Embed1 uniformly to visual modalities; if the embeddings encode scene layout instead of behavior, prototype similarity can be misleading.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Auditing Instruction-Trajectory Mismatches in Multimodal Robot Demonstrations." pith.science (2026). https://pith.science/paper/ZUE45AZ6

@misc{pith2026260807895,
  author       = {Pith},
  title        = {Pith review of: Auditing Instruction-Trajectory Mismatches in Multimodal Robot Demonstrations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZUE45AZ6}},
  note         = {Machine review of arXiv:2608.07895}
}
read the original abstract

Robot demonstration datasets used to train vision-language-action policies can contain a subtle but harmful failure mode: trajectories that are behaviorally correct but paired with the wrong language instruction. We study post-hoc auditing of these Instruction-Trajectory Mismatches (ITMs). Unlike failed rollouts, ITMs often look plausible, and can corrupt the language-behavior mapping learned by the policy. We propose Multimodal Probabilistic Fusion (MMPF), a training-free auditing framework that treats each modality as an expert, estimates a task-label distribution from local neighborhood agreement and global prototype similarity, and then fuses modalities with predictive-entropy weighting in a product of experts. Across LIBERO benchmarks with injected instruction mismatches and noisy real-robot data, MMPF achieves the strongest overall ITM detection and label correction accuracy. We also show that auditing improves most downstream policy learning in settings where language is needed to disambiguate the task. We demonstrate in real robot experiments that our method can achieve improved policy performance and show the trade-off of filtering demonstrations compared to relabeling.

Figures

Figures reproduced from arXiv: 2608.07895 by the authors.

Figure 1
Figure 1. Example ITM auditing case. The provided instruction is “Place bottle in [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. MMPF auditing framework. Each modality estimates a task-label distribution by combining local [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The first dataset, denoted Table, decomposes similar tabletop behavior into primitive actions: focus, navigation, picking, and placing. It includes two objects, a paper and a mug where the tasks are to focus on, navigate to, pick up, and place the target object. In total it has 1260 demonstrations split across all primitive actions. The second dataset, denoted Bottle-mug, contains two fami￾lies of two-stage manipula… view at source ↗
Figures from the paper (1 more)
Figure 3
Figure 3. Figure 3: Real-robot environments used for downstream evaluation. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 8 canonical work pages

  1. [1]

    Rt-1: Robotics transformer for real-world control at scale,

    A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, J. Hsuet al., “Rt-1: Robotics transformer for real-world control at scale,”arXiv preprint arXiv:2212.06817, 2022

    arXiv 2022

  2. [2]

    Rt-2: Vision-language-action models transfer web knowledge to robotic control,

    B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahidet al., “Rt-2: Vision-language-action models transfer web knowledge to robotic control,” inConference on Robot Learning. PMLR, 2023, pp. 2165–2183

    2023

  3. [3]

    Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collab- oration 0,

    A. O’Neill, A. Rehman, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, A. Jainet al., “Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collab- oration 0,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 6892–6903

    2024

  4. [4]

    Octo: An open-source generalist robot policy,

    Octo Model Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xuet al., “Octo: An open-source generalist robot policy,”arXiv preprint arXiv:2405.12213, 2024

    arXiv 2024

  5. [5]

    Openvla: An open-source vision-language-action model,

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketiet al., “Openvla: An open-source vision-language-action model,”arXiv preprint arXiv:2406.09246, 2024

    arXiv 2024

  6. [6]

    𝜋 0: A vision-language-action flow model for general robot control,

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, S. Jakubczak, T. Jones, L. Ke, S. Levine, A. Li-Bell, M. Mothukuri, S. Nair, K. Pertsch, L. X. Shi, J. Tanner, Q. Vuong, A. Walling, H. Wang, and U. Zhilinsky, “𝜋 0: A vision-language-action flow model for general robot control,” 2024. [Online]. Availabl...

    arXiv 2024

  7. [7]

    𝜋 0.5: a vision- language-action model with open-world generalization,

    Physical Intelligence, K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusaiet al., “𝜋 0.5: a vision- language-action model with open-world generalization,”arXiv preprint arXiv:2504.16054, 2025

    arXiv 2025

  8. [8]

    Cupid: Curating data your robot loves with influence functions,

    C. Agia, R. Sinha, J. Yang, R. Antonova, M. Pavone, H. Nishimura, M. Itkina, and J. Bohg, “Cupid: Curating data your robot loves with influence functions,”arXiv preprint arXiv:2506.19121, 2025

    arXiv 2025

Show all 31 references
  1. [9]

    Robot data curation with mutual information estimators,

    J. Hejna, S. Mirchandani, A. Balakrishna, A. Xie, A. Wahid, J. Tompson, P. Sanketi, D. Shah, C. Devin, and D. Sadigh, “Robot data curation with mutual information estimators,”arXiv preprint arXiv:2502.08623, 2025

    2025 arXiv

  2. [10]

    Datamil: Selecting data for robot imitation learning with datamodels,

    S. Dass, A. Khaddaj, L. Engstrom, A. Madry, A. Ilyas, and R. Mart ´ın- Mart´ın, “Datamil: Selecting data for robot imitation learning with datamodels,”arXiv preprint arXiv:2505.09603, 2025

    2025 arXiv

  3. [11]

    Curating Demonstrations using Online Experience,

    A. S. Chen, A. M. Lessing, Y. Liu, and C. Finn, “Curating Demonstrations using Online Experience,” inProceedings of Robotics: Science and Systems, Los Angeles, CA, USA, June 2025

    2025

  4. [12]

    Scizor: A self-supervised approach to data curation for large-scale imitation learning,

    Y. Zhang, Y. Xie, H. Liu, R. Shah, M. Wan, L. Fan, and Y. Zhu, “Scizor: A self-supervised approach to data curation for large-scale imitation learning,”arXiv preprint arXiv:2505.22626, 2025

    2025 arXiv

  5. [13]

    Robotic skill acquisition via in- struction augmentation with vision-language models,

    T. Xiao, H. Chan, P. Sermanet, A. Wahid, A. Brohan, K. Haus- man, S. Levine, and J. Tompson, “Robotic skill acquisition via in- struction augmentation with vision-language models,”arXiv preprint arXiv:2211.11736, 2022

    2022 arXiv

  6. [14]

    Clip-rt: Learning language-conditioned robotic policies from natural language supervision,

    G.-C. Kang, J. Kim, K. Shim, J. K. Lee, and B.-T. Zhang, “Clip-rt: Learning language-conditioned robotic policies from natural language supervision,” arXiv preprint arXiv:2411.00508, 2024

    2024 arXiv

  7. [15]

    Sprint: Scalable policy pre- training via language instruction relabeling,

    J. Zhang, K. Pertsch, J. Zhang, and J. J. Lim, “Sprint: Scalable policy pre- training via language instruction relabeling,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 9168– 9175

    2024

  8. [16]

    Cast: Counterfactual labels improve instruction following in vision-language- action models,

    C. Glossop, W. Chen, A. Bhorkar, D. Shah, and S. Levine, “Cast: Counterfactual labels improve instruction following in vision-language- action models,”arXiv preprint arXiv:2508.13446, 2025

    2025 arXiv

  9. [17]

    Consistency matters: Defining demonstration data quality metrics in robot learning from demonstration,

    M. Sakr, J. Zhang, H. M. V. d. Loos, D. Kulic, and E. Croft, “Consistency matters: Defining demonstration data quality metrics in robot learning from demonstration,”ACM Transactions on Human-Robot Interaction, 2024

    2024

  10. [18]

    Evaluating uncertainty and quality of visual language action-enabled robots,

    P. Valle, C. Lu, S. Ali, and A. Arrieta, “Evaluating uncertainty and quality of visual language action-enabled robots,”arXiv preprint arXiv:2507.17049, 2025

    2025 arXiv

  11. [19]

    Re-mix: Optimizing data mixtures for large scale imitation learning,

    J. Hejna, C. Bhateja, Y. Jiang, K. Pertsch, and D. Sadigh, “Re-mix: Optimizing data mixtures for large scale imitation learning,”arXiv preprint arXiv:2408.14037, 2024

    2024 arXiv

  12. [20]

    Scaling robot policy learning via zero-shot labeling with foundation models,

    N. Blank, M. Reuss, M. R¨ uhle, ¨O. E. Ya˘gmurlu, F. Wenzel, O. Mees, and R. Lioutikov, “Scaling robot policy learning via zero-shot labeling with foundation models,”arXiv preprint arXiv:2410.17772, 2024

    2024 arXiv

  13. [21]

    Confident learning: Estimating uncertainty in dataset labels,

    C. Northcutt, L. Jiang, and I. Chuang, “Confident learning: Estimating uncertainty in dataset labels,”Journal of Artificial Intelligence Research, vol. 70, pp. 1373–1411, 2021

    2021

  14. [22]

    Deep k-nn for noisy labels,

    D. Bahri, H. Jiang, and M. Gupta, “Deep k-nn for noisy labels,” in International Conference on Machine Learning. PMLR, 2020, pp. 540– 550

    2020

  15. [23]

    Lemon: Label error detection using multimodal neighbors,

    H. Zhang, A. Balagopalan, N. Oufattole, H. Jeong, Y. Wu, J. Zhu, and M. Ghassemi, “Lemon: Label error detection using multimodal neighbors,”arXiv preprint arXiv:2407.18941, 2024

    2024 arXiv

  16. [24]

    Seeing what’s wrong: A trajectory-guided approach to caption error detection,

    G. I. Afriat, R. Lucas, X. Meng, Y. Hou, Y. Zhu, and R. Mazumder, “Seeing what’s wrong: A trajectory-guided approach to caption error detection,” Openreview, 2025

    2025

  17. [25]

    Training products of experts by minimizing contrastive divergence,

    G. E. Hinton, “Training products of experts by minimizing contrastive divergence,”Neural computation, vol. 14, no. 8, pp. 1771–1800, 2002

    2002

  18. [26]

    Generalized product-of-experts for learning multimodal representations in noisy environments,

    A. Joshi, N. Gupta, J. Shah, B. Bhattarai, A. Modi, and D. Stoyanov, “Generalized product-of-experts for learning multimodal representations in noisy environments,” inProceedings of the ACM International Con- ference on Multimodal Interaction (ICMI), 2022, pp. 83–93

    2022

  19. [27]

    Generalized product of experts for automatic and principled fusion of gaussian process predictions,

    Y. Cao and D. J. Fleet, “Generalized product of experts for automatic and principled fusion of gaussian process predictions,”arXiv preprint arXiv:1410.7827, 2014

    2014 arXiv

  20. [28]

    Libero: Benchmarking knowledge transfer for lifelong robot learning,

    B. Liu, Y. Zhu, C. Gao, Y. Feng, Q. Liu, Y. Zhu, and P. Stone, “Libero: Benchmarking knowledge transfer for lifelong robot learning,”Advances in Neural Information Processing Systems, vol. 36, pp. 44 776–44 791, 2023

    2023

  21. [29]

    When vision overrides language: Evaluating and mitigating counterfactual failures in vlas,

    Y. Fang, Y. Feng, D. Jing, J. Liu, Y. Yang, Z. Wei, D. Szafir, and M. Ding, “When vision overrides language: Evaluating and mitigating counterfactual failures in vlas,”arXiv preprint arXiv:2602.17659, 2026

    2026 arXiv

  22. [30]

    Cosmos world foundation model platform for physical ai,

    NVIDIA, “Cosmos world foundation model platform for physical ai,” arXiv preprint arXiv:2501.03575, 2025, cosmos-Embed1 trajectory em- bedding model

    2025 arXiv

  23. [31]

    Smolvla: A vision-language-action model for affordable and efficient robotics,

    M. Shukor, D. Aubakirova, F. Capuano, P. Kooijmans, S. Palma, A. Zoui- tine, M. Aractingi, C. Pascal, M. Russi, A. Marafiotiet al., “Smolvla: A vision-language-action model for affordable and efficient robotics,”arXiv preprint arXiv:2506.01844, 2025

    2025 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.