Pith. sign in

REVIEW 2 major objections 5 minor 26 references

A closed-weight robot model, accessible only through a managed fine-tuning API, can be driven to near-perfect task success by a closed-loop flywheel that converts deployment-time reward feedback into advantage-conditioned supervised data, w

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 12:18 UTC pith:WAMNVY2R

load-bearing objection Clever API-only closed-loop fine-tuning method, but the evaluation reuses the training configurations, so the near-perfect success rates are likely inflated and the 'specialist' claims outrun the evidence. the 2 major comments →

arxiv 2607.29172 v1 pith:WAMNVY2R submitted 2026-07-31 cs.RO cs.AI

CLIFT: Turning Gemini Robotics On-Device into Humanoid Specialists via Non-Invasive Closed-Loop Iterative Fine-Tuning

Yuxin Chen , Hari Srikanth , Nathan Jew , Menglin Wu , Pengcheng Wang , Junli Ren , Masayoshi Tomizuka , Peng Xu
show 2 more authors
Jinyu Xie Thomas Tian
This is my paper
classification cs.RO cs.AI
keywords robot foundation modelsmanaged SFT APIclosed-loop fine-tuninghumanoid manipulationadvantage conditioningreward modeldeployment-time learningclosed-weight adaptation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper asks whether a robot foundation model that can only be adapted by submitting training data to a managed supervised fine-tuning API—no weights, gradients, or losses visible—can still improve closed-loop on a real humanoid. It introduces CLIFT, a flywheel that deploys the current policy, scores each rollout with a preference-calibrated dense reward model, labels action chunks with a binary advantage token via retrieval against visually similar states, and fine-tunes on those chunks through the API. After two cycles, the method takes the model from 93% to 100% on box packing, 70% to 98% on cup insertion, and 53% to 96% on bimanual plate handover. The same pipeline improves an open-weight VLA, but to a much lower ceiling, suggesting the pretrained prior—not the access regime—sets the adaptation limit. A sympathetic reader would care because it shows closed-loop reinforcement-style improvement is possible inside an API-only regime, turning black-box adaptation interfaces into practical policy-improvement tools.

Core claim

The central claim is that reinforcement feedback can be encoded directly into supervised training data, making closed-loop policy improvement compatible with a managed SFT API. CLIFT works by scoring deployment rollouts with a dense reward model that is calibrated by 100 human pairwise preferences selecting among VLM-generated per-step reward candidates, then distilling those into a fixed reward model. Each action chunk is assigned a binary advantage token by comparing its discounted return against chunks retrieved from visually similar starting states (top-30% threshold). The next SFT dataset includes these relabeled chunks, and conditioning on the positive token at deployment steers the po

What carries the argument

The flywheel's central mechanism is the advantage-conditioned SFT tuple: an observation, language instruction, action chunk, and a binary token that encodes whether the chunk's return ranks in the top 30% of chunks from visually similar states. The token is produced by a retrieval-based advantage estimator that compares each chunk's discounted return over an 1.8-second look-ahead window against peers found with a frozen vision encoder, so the threshold auto-adapts to state difficulty. The reward model R_theta that feeds these returns is itself a VLM fine-tuned on per-step labels distilled from VLM candidates that best matched 100 human pairwise preferences. This pipeline converts deployment-

Load-bearing premise

The entire learning signal flows through the dense reward model R_theta, which is trained on about 200 rollouts whose per-frame labels come from VLM-generated candidates filtered by only 100 human pairwise preferences, and is never validated against held-out human judgments or task outcomes; if those rewards reward surface progress or clean-looking motion rather than true task progression, CLIFT amplifies the wrong behavior and the reported gains would not transfer.

What would settle it

Train R_theta as described, but before running CLIFT, validate its per-step rewards against a held-out set of human preferences and against ground-truth task outcomes. Then run two flywheel cycles using an oracle reward that is perfectly calibrated to task outcomes: if the oracle-driven loop substantially outperforms the R_theta-driven loop on the same tasks, the reward model is the bottleneck; if the two match, R_theta is adequate. Alternatively, run CLIFT with R_theta deliberately corrupted (e.g., rewarding smooth motion regardless of task progress) and show success rates drop—this would set

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If true, managed SFT APIs can serve as general-purpose closed-loop improvement interfaces for robot foundation models, not just one-shot customization.
  • The result implies that the ceiling of API-adaptation is set by the pretrained prior, so securing API access to the strongest closed-weight models is worthwhile even without internal access.
  • Chunk-level advantage labeling from failed rollouts converts failure into training signal, which is what closes the gap on the hardest task (bimanual handover 53% to 96% vs. episode-selection ~84%).
  • The same non-invasive pipeline transfers across models and access regimes, improving an open-weight VLA as well, so it is not specific to one API.
  • Conditioning on a positive advantage token at inference acts as a weak form of guidance, and the emergent retry and pre-grasp reorientation behaviors suggest closed-loop practice can compose new behavior beyond the demonstration distribution.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to validate R_theta against held-out human judgments and task outcomes before running the flywheel; if a corrupted reward model (e.g., rewarding smooth motion regardless of task progress) eliminates the gains, that would confirm the reward model is the load-bearing component.
  • The retrieval-based advantage estimator makes a strong assumption that cosine similarity of frozen vision embeddings captures state equivalence; a sensitivity study swapping the encoder or using task-specific state features would clarify how robust the credit assignment is.
  • Because the paper's conclusion that the pretrained prior sets the ceiling rests on a single invasive baseline, a broader comparison across several open-weight models of comparable scale would strengthen or qualify that claim.
  • A natural extension the paper flags is coupling CLIFT with a control-aware world model to reduce the number of real-robot rollouts per cycle; such a world model would also make the approach more practical for safety-sensitive deployment.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. CLIFT is a closed-loop fine-tuning procedure for closed-weight robot VLAs exposed only through a managed SFT API. Starting from a demonstration-trained policy π0, it deploys the policy on hardware, scores each rollout with a preference-calibrated dense reward model R_θ, labels action chunks with binary advantage tokens obtained by comparing discounted chunk returns against chunks retrieved from visually similar states, and fine-tunes the base model through the API on demonstrations plus relabeled rollouts. The loop repeats. The paper evaluates on a Unitree G1 humanoid on three tabletop but whole-body tasks: box packing, cup insertion, and bimanual plate handover. Across two flywheel cycles, the dense advantage-conditioned variant raises GROD from 93/70/53% to 100/98/96% success on the three tasks, while the same pipeline applied to open-weight π0.5 reaches only 76/56/30%. An invasive FiLM-conditioned variant of π0.5 does not close the gap. The paper concludes that API-only access can support closed-loop policy improvement and that the pretrained prior, rather than access level, sets the adaptation ceiling.

Significance. If the headline results are reliable, this is a significant empirical contribution: it identifies a practically important access regime (managed SFT API) and demonstrates a non-invasive mechanism for closed-loop improvement that requires no weights, gradients, or likelihoods, with carefully matched controls between models (same demonstrations, reward model, rollout budget, deployment configuration). The retrieval-based advantage labeling is a plausible way to normalize returns by state difficulty without learning a value function. The paper also discloses its limitations. However, the significance is conditional on two load-bearing points: the evaluation must be free of train/eval contamination, and the reward model must be shown to encode genuine task progress. As written, neither point is adequately established.

major comments (2)
  1. [Sec. 4.1 and App. E.2] The evaluation protocol suffers from train/eval contamination. Sec. 4.1 states that 'deployment and evaluation coincide: each cycle deploys the current policy for 100 rollouts per task under a fixed evaluation suite, and these same rollouts both define its success rate and, once relabeled, form the training data for the next cycle.' App. E.2 confirms that the same fixed configurations are 'reused identically for every model, every variant, and every flywheel cycle.' Consequently, the success rates reported for π1 and π2 are computed on initial scene layouts whose rollouts from earlier cycles are in the cumulative training set D_k. The observed improvement can therefore be explained by memorizing these specific layouts rather than by acquiring a transferable closed-loop skill; the claims of 'task mastery' and 'humanoid specialist' go beyond what this protocol demonstrates. Please add held
  2. [Sec. 3.1, App. B, Eq. (5)] The reward model R_θ is the sole source of the learning signal (advantage tokens in Eq. (5) and the top-30% episode selection), yet the paper provides no validation of R_θ against held-out human preferences or task outcomes. It is trained on roughly 200 rollouts with 100 pairwise preference labels, with VLM candidate sequences selected to match those preferences; no accuracy, correlation, or ablation is reported. If R_θ rewards surface progress or smooth motion rather than true task progression, CLIFT would amplify the wrong behavior and the reported success gains would not transfer. Please report (i) R_θ's agreement with held-out human pairwise preferences and with binary task success, and (ii) an ablation replacing the dense reward with task success/failure or a no-reward/random-label baseline, to demonstrate that the dense reward signal is what drives the improvement.
minor comments (5)
  1. [Fig. 5 / Sec. 4.2] Success rates are point estimates over 100 trials with no confidence intervals or significance tests. Report binomial confidence intervals, and clarify how the two independent training runs were used (the text says the run with lowest held-out demonstration loss was selected, which introduces selection bias).
  2. [Sec. 3.2 / App. C] The method relies on several global hyperparameters with no sensitivity analysis: retrieval threshold δ, look-ahead horizon H, discount factor γ, the top-30% percentile, and the classifier-free guidance scale β. Please report sensitivity to at least δ, H, and the percentile for one task, since the method's robustness depends on these choices.
  3. [App. B] The candidate-selection procedure for choosing one reward sequence per rollout to maximize agreement across the 100 human preference pairs is described only verbally. Please specify the optimization method (e.g., greedy, ILP) and its computational feasibility, as the number of combinations grows exponentially in K and the number of rollouts.
  4. [Fig. 6 / Sec. 4.2] The 'emergent behaviors' are presented qualitatively. Quantify them, for example by measuring the frequency of reorientation/retry events in evaluation rollouts, to support the claim that these behaviors are acquired and not cherry-picked.
  5. [Sec. 4.2] The detailed benchmarking is referenced to an interactive server on the project website; provide a static archival artifact (e.g., a downloadable dataset or a DOI-linked document) for reproducibility and long-term access.

Circularity Check

1 steps flagged

Reported near-perfect success rates are measured on the same fixed configurations that supply the relabeled training rollouts; the 'task mastery' claim reduces to training-set performance.

specific steps
  1. fitted input called prediction [Sec. 4.1 ('CLIFT protocol and evaluation') and App. E.2 ('Evaluation protocol'); headline results in Sec. 4.2 / Fig. 5]
    "A key property of the on-device setting is that deployment and evaluation coincide: each cycle deploys the current policy for 100 rollouts per task under a fixed evaluation suite, and these same rollouts both define its success rate and, once relabeled, form the training data for the next cycle. ... Crucially, the same configurations are reused identically for every model, every variant, and every flywheel cycle reported in this paper ... Because deployment and evaluation coincide in our on-device setting, these same rollouts also serve as the data relabeled for the next flywheel cycle."

    The reported 'near-perfect success after two flywheel cycles' (Fig. 5) is evaluated on the exact fixed configuration set whose rollouts are appended to the SFT dataset D_k used to train the evaluated policy. Each policy is therefore tested on initial scene layouts already present in its training data from earlier cycles. Improvements across cycles can be explained by memorizing these fixed layouts rather than by acquiring a transferable task skill, yet the paper glosses the numbers as 'task mastery' and 'humanoid specialist.' This is a construction-level equivalence: evaluation inputs coincide with training inputs by design, independent of the reward model's calibration.

full rationale

The main derivation chain of CLIFT—converting deployment reward into advantage-conditioned SFT tuples and feeding them back through a managed API—is not itself circular: the advantage labels are computed from the fitted reward model and the final success criteria are binary task-completion checks external to that model. The reward model is unvalidated against held-out human judgments, and the zero-shot/citation-based components are a correctness risk, but that is not a circularity. The significant circularity is in the evaluation claim: the paper explicitly makes deployment and evaluation coincide and reuses the identical configurations across every cycle, so the reported success rates are measurements on the training distribution, not predictions about task mastery on novel layouts. This does not invalidate the methodological contribution, but it does mean the headline 'near-perfect success' result is partly forced by the construction of the data loop. Score 6 reflects one central prediction reducing to its training inputs by construction.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 0 invented entities

The load-bearing inputs the reader pays for upstream: a black-box SFT API whose contract is unverifiable, a once-trained reward model validated nowhere, a heuristic (not derived) mapping from the stated RL objective to the data flywheel, and a visual-similarity proxy for state similarity. Seven hand-chosen thresholds/budgets (δ, 30% percentile, H=1.8s, γ, β=0.2, K=12, 100 preferences) gate the quality of the advantage signal. No new physical or theoretical entities are postulated; the advantage token is inherited from π*0.6 [5] and R_θ is a fitted artifact with no falsifiable handle outside this paper's evaluation.

free parameters (7)
  • δ (retrieval similarity threshold) = single global value tuned so each comparison set holds |N| neighbors on average
    App. C: hand-tuned threshold deciding which rollouts contribute peer chunks to the advantage comparison set; directly controls the quality of the advantage baseline.
  • advantage percentile (top 30%) = 0.30
    Eq. (5): chunk labeled positive iff its discounted return exceeds the 70th percentile of its comparison set; chosen without sensitivity analysis.
  • look-ahead horizon H = 1.8 s of future observations
    App. C, Eq. (4): window over which chunk returns are accumulated; chosen, no ablation.
  • discount factor γ = not stated
    Eq. (4): used in computing chunk returns; the value is never reported.
  • classifier-free guidance scale β = 0.2
    App. D: deployment-time strength of the positive advantage token; fixed across all variants.
  • VLM reward candidates K = 12
    App. B: number of candidate per-step reward sequences sampled per rollout; chosen without analysis.
  • human preference pairs = 100
    Sec. 4.1, App. B: the entire reward-calibration budget; small for dense per-frame labels and no saturation analysis is provided.
axioms (5)
  • domain assumption The managed SFT operator F_SFT (GROD) exists and behaves as a fine-tuner from the base checkpoint on each submission, with no hidden access to internals.
    Sec. 2 formalizes F_SFT as the only update operation; its contract is cited to [10], a private-preview API that reviewers and most readers cannot inspect.
  • domain assumption R_θ, trained once, yields reward sequences that remain valid across all flywheel cycles and all tasks.
    Sec. 3.3: 'The reward model R_θ is trained once and held fixed throughout.' No drift, distribution-shift, or validity check is reported.
  • ad hoc to paper The deployment-time data flywheel approximates the KL-regularized objective in Eq. (1).
    Sec. 2: 'We instead approximate π* with a deployment-time data flywheel.' The correspondence between binarized top-30% chunk selection plus SFT and the Eq. (1) objective is asserted, not derived.
  • domain assumption DINOv3 cosine similarity between starting frames is a valid proxy for task-relevant state similarity.
    App. C: comparison sets (Eq. 3) are built from visual embeddings; if similar-looking frames differ in task-relevant state (object pose, contact, controller phase), the advantage baseline is miscalibrated.
  • standard math Standard background: expectation, KL divergence, discounted returns, percentile ranking, flow-matching and MSE losses.
    Used throughout Secs. 2–3 and App. E; no special assumptions beyond standard definitions.

pith-pipeline@v1.3.0-daily-deepseek · 14978 in / 25684 out tokens · 218617 ms · 2026-08-03T12:18:39.700868+00:00 · methodology

0 comments
read the original abstract

While robot foundation models are growing increasingly capable, the strongest models are typically trained on proprietary data and remain closed-source, limiting downstream users' ability to adapt them to new tasks, embodiments, and deployment settings. Following the LLM community, an emerging access paradigm for closed-weight robot foundation models is the managed supervised fine-tuning (SFT) API, where users submit training data and receive a tuned policy without access to model weights, gradients, or training internals. While such APIs let downstream users leverage powerful proprietary foundation models, they restrict policy improvement to pure imitation, ruling out reinforcement learning and other closed-loop methods that rely on internal training signals. This limitation is particularly acute for agile, contact-rich humanoid manipulation, where the gap between policy outputs and deployed behavior is large due to novel states, action tracking dynamics, latency, and controller-specific failure modes. We study how effective this managed-API regime is for humanoid adaptation, and how closed-loop improvement can be realized within it to push policies toward task mastery. We conduct one of the first empirical studies of managed-API adaptation on a real humanoid, instantiated on Gemini Robotics On-Device (GROD). We find that direct SFT through the API substantially outperforms a leading open-weight VLA trained on the same demonstrations, yet still falls short of deployment-level mastery on agile, contact-rich tasks. To close this gap, we introduce CLIFT: Closed-Loop Iterative Fine-Tuning, which turns deployment-time reward feedback into API-compatible supervised data and enables closed-loop policy improvement without accessing weights, gradients, likelihoods, or losses-pushing GROD to near-perfect success after two flywheel cycles, all without "opening the model box."

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

26 extracted references · 12 linked inside Pith

  1. [1]

    Open X-embodiment: Robotic learning datasets and RT-X models

    Open X-Embodiment Collaboration. Open X-embodiment: Robotic learning datasets and RT-X models. In IEEE International Conference on Robotics and Automation (ICRA), pages 6892–6903, 2024

    2024

  2. [2]

    Gemini robotics: Bringing ai into the physical world.arXiv preprint arXiv:2503.20020, 2025

    Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslie, Jean-Baptiste Alayrac, Montserrat Gonzalez Arenas, Travis Armstrong, Ashwin Balakrishna, Robert Baruch, Maria Bauza, Michiel Blokzijl, et al. Gemini robotics: Bringing ai into the physical world.arXiv preprint arXiv:2503.20020, 2025

    Pith/arXiv arXiv 2025

  3. [3]

    Gr00t n1: An open foundation model for generalist humanoid robots

    Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, Joel Jang, Zhenyu Jiang, Jan Kautz, Kaushil Kundalia, Lawrence Lao, Zhiqi Li, Zongyu Lin, Kevin Lin, Guilin Liu, Edith Llontop, Loic Magne, Ajay Mandlekar, Avnish Narayan, Soroush Nasiriany, Scott Reed, You Liang Tan, Gua...

    Pith/arXiv arXiv 2025

  4. [4]

    arXiv preprint arXiv:2603.12263, 2026

    Songlin Wei, Hongyi Jing, Boqian Li, Zhenyu Zhao, Jiageng Mao, Zhenhao Ni, Sicheng He, Jie Liu, Xiawei Liu, Kaidi Kang, et al.𝜓0: An open foundation model towards universal humanoid loco-manipulation. arXiv preprint arXiv:2603.12263, 2026

    arXiv 2026

  5. [5]

    arXiv preprint arXiv:2511.14759, 2025

    Ali Amin, Raichelle Aniceto, Ashwin Balakrishna, Kevin Black, Ken Conley, Gavin Connors, James Darpinian, Karan Dhabalia, Joseph DiCarlo, Danny Driess, et al.𝜋∗ 0.6: a VLA that learns from experience. arXiv preprint arXiv:2511.14759, 2025

    Pith/arXiv arXiv 2025

  6. [6]

    Helix: A vision-language-action model for generalist humanoid control, 2025

    Figure AI. Helix: A vision-language-action model for generalist humanoid control, 2025

    2025

  7. [7]

    Fine-tuning – openai api documentation, 2025

    OpenAI. Fine-tuning – openai api documentation, 2025. Accessed: 2026-04-13

    2025

  8. [8]

    About supervised fine-tuning for Gemini models – Vertex AI documentation, 2025

    Google Cloud. About supervised fine-tuning for Gemini models – Vertex AI documentation, 2025. Accessed: 2026-04-13

    2025

  9. [9]

    Position: Good embodied reward models need bad behavior data

    Ran Tian, Yilin Wu, and Andrea Bajcsy. Position: Good embodied reward models need bad behavior data. InInternational Conference on Machine Learning (ICML), 2026

    2026

  10. [10]

    Gemini robotics on-device, 2025

    Google DeepMind. Gemini robotics on-device, 2025. Private preview, available via Trusted Tester Program. Accessed: 2026-04-13

    2025

  11. [11]

    The physical intelligence layer, February 2026

    Physical Intelligence. The physical intelligence layer, February 2026. Accessed: 2026-04-13

    2026

  12. [12]

    Humanoid everyday: A comprehensive robotic dataset for open-world humanoid manipulation.arXiv preprint arXiv:2510.08807, 2025

    Zhenyu Zhao, Hongyi Jing, Xiawei Liu, Jiageng Mao, Abha Jha, Hanwen Yang, Rong Xue, Sergey Zakharov, Vitor Guizilini, and Yue Wang. Humanoid everyday: A comprehensive robotic dataset for open-world humanoid manipulation.arXiv preprint arXiv:2510.08807, 2025

    Pith/arXiv arXiv 2025

  13. [13]

    Tune to learn: How controller gains shape robot policy learning.arXiv preprint arXiv:2604.02523, 2026

    Antonia Bronars, Younghyo Park, and Pulkit Agrawal. Tune to learn: How controller gains shape robot policy learning.arXiv preprint arXiv:2604.02523, 2026

    Pith/arXiv arXiv 2026

  14. [14]

    Self-improving vision-language-action models with data generation via residual RL.arXiv preprint arXiv:2511.00091, 2025

    Wenli Xiao, Haotian Lin, Andy Peng, Haoru Xue, Tairan He, Yuqi Xie, Fengyuan Hu, Jimmy Wu, Zhengyi Luo, Linxi Fan, Guanya Shi, and Yuke Zhu. Self-improving vision-language-action models with data generation via residual RL.arXiv preprint arXiv:2511.00091, 2025

    arXiv 2025

  15. [15]

    Robo-Dopamine: General process reward modeling for high-precision robotic manipula- tion.arXiv preprint arXiv:2512.23703, 2025

    Huajie Tan et al. Robo-Dopamine: General process reward modeling for high-precision robotic manipula- tion.arXiv preprint arXiv:2512.23703, 2025

    arXiv 2025

  16. [16]

    Steering your diffusion policy with latent space reinforcement learning.arXiv preprint arXiv:2506.15799, 2025

    Andrew Wagenmaker, Mitsuhiko Nakamoto, Yunchu Zhang, Seohong Park, Waleed Yagoub, Anusha Naga- bandi, Abhishek Gupta, and Sergey Levine. Steering your diffusion policy with latent space reinforcement learning.arXiv preprint arXiv:2506.15799, 2025

    Pith/arXiv arXiv 2025

  17. [17]

    Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Manuel Y. Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Devin LeBlanc, Sergey Levine, et al.𝜋0.5: a 11 vision-language-action model with open-world generalization. InP...

    2025

  18. [18]

    Proximal policy opti- mization algorithms.arXiv preprint arXiv:1707.06347, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy opti- mization algorithms.arXiv preprint arXiv:1707.06347, 2017

    Pith/arXiv arXiv 2017

  19. [19]

    Advantage-weighted regression: Simple and scalable off-policy reinforcement learning.arXiv preprint arXiv:1910.00177, 2019

    Xue Bin Peng, Aviral Kumar, Grace Zhang, and Sergey Levine. Advantage-weighted regression: Simple and scalable off-policy reinforcement learning.arXiv preprint arXiv:1910.00177, 2019

    Pith/arXiv arXiv 1910

  20. [20]

    Vision language models are in-context value learners

    Yecheng Jason Ma, Joey Hejna, Chuyuan Fu, Dhruv Shah, Jacky Liang, Zhuo Xu, Sean Kirmani, Peng Xu, Danny Driess, Ted Xiao, Osbert Bastani, Dinesh Jayaraman, Wenhao Yu, Tingnan Zhang, Dorsa Sadigh, and Fei Xia. Vision language models are in-context value learners. InInternational Conference on Learning Representations (ICLR), 2025

    2025

  21. [21]

    Maximizing alignment with minimal feedback: Efficiently learning rewards for visuomotor robot policy alignment

    Ran Tian, Yilin Wu, Chenfeng Xu, Masayoshi Tomizuka, Jitendra Malik, and Andrea Bajcsy. Maximizing alignment with minimal feedback: Efficiently learning rewards for visuomotor robot policy alignment. arXiv preprint arXiv:2412.04835, 2024

    Pith/arXiv arXiv 2024

  22. [22]

    XR-Teleoperate: An open-source teleoperation framework and data collection toolkit for embodied intelligence

    Unitree Robotics. XR-Teleoperate: An open-source teleoperation framework and data collection toolkit for embodied intelligence. https://github.com/unitreerobotics/xr_teleoperate, 2024. Accessed: 2026-02

    2024

  23. [23]

    PaliGemma: A versatile 3B VLM for transfer.arXiv preprint arXiv:2407.07726, 2024

    Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, et al. PaliGemma: A versatile 3B VLM for transfer.arXiv preprint arXiv:2407.07726, 2024

    Pith/arXiv arXiv 2024

  24. [24]

    FiLM:Visualreasoning with a general conditioning layer

    EthanPerez, FlorianStrub, HarmdeVries, VincentDumoulin, andAaronCourville. FiLM:Visualreasoning with a general conditioning layer. InAAAI Conference on Artificial Intelligence, 2018

    2018

  25. [25]

    Dinov3.arXivpreprintarXiv:2508.10104, 2025

    Oriane Siméoni, Huy V Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khali- dov,MarcSzafraniec,SeungeunYi,MichaëlRamamonjisoa,etal. Dinov3.arXivpreprintarXiv:2508.10104, 2025

    Pith/arXiv arXiv 2025

  26. [26]

    frame_index

    Kevin Frans, Seohong Park, Pieter Abbeel, and Sergey Levine. Diffusion guidance is a controllable policy improvement operator.arXiv preprint arXiv:2505.23458, 2025. 12 Appendix A Related Work Closed-weight API-exposed foundation models. Despite the rapid progress of open-source models, many of the most capable large language models and vision-language mod...

    Pith/arXiv arXiv 2025