DEFLECT: Delay-Robust Execution via Flow-matching Likelihood-Estimated Counterfactual Tuning for VLA Policies

Jiaxiang Zou; Jingyu Guo; Rui Meng; Taowen Wang; Xinyu Chen; Yixiang Zhu; Yonghao Chen; Zijie Yang

arxiv: 2605.19294 · v1 · pith:Q3JMWRZCnew · submitted 2026-05-19 · 💻 cs.RO · cs.AI

DEFLECT: Delay-Robust Execution via Flow-matching Likelihood-Estimated Counterfactual Tuning for VLA Policies

Yixiang Zhu , Yonghao Chen , Rui Meng , Jingyu Guo , Jiaxiang Zou , Zijie Yang , Taowen Wang , Xinyu Chen This is my paper

Pith reviewed 2026-05-20 05:59 UTC · model grok-4.3

classification 💻 cs.RO cs.AI

keywords VLA policiesasynchronous robot controldelay robustnessflow matchingcounterfactual tuningoffline preference learningvision language action

0 comments

The pith

DEFLECT turns execution delays into a label-free tuning signal that keeps VLA policies effective under asynchronous inference.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Vision-language-action policies normally lose performance when inference lags behind the robot's moving state, with success rates collapsing from near 90 percent to under 1 percent once delays reach several control steps. The method builds counterfactual pairs of fresh and stale actions from a frozen reference policy, then ranks them by an implicit likelihood ratio drawn from a flow-matching model evaluated at the delayed observation. This ranking supplies a preference signal that refines the policy offline, without human labels, reward models, or any new robot rollouts. The resulting policy maintains usable control across longer inference cycles on both simulated benchmarks and two physical robot tasks.

Core claim

DEFLECT is a fully offline post-training procedure that converts the latency-induced misalignment of asynchronous VLA execution into a preference signal by scoring counterfactual fresh-versus-stale action pairs under deployment-time conditioning with a flow-matching likelihood-ratio surrogate derived from a frozen reference policy.

What carries the argument

The implicit flow-matching likelihood-ratio surrogate that scores counterfactual action pairs constructed from a frozen reference policy under the delayed conditioning observation.

If this is right

Success rates rise by 6.4 points in the 5-7 control-step latency regime on Kinetix.
Real-scale VLA transfer shows a 4.6-point gain at the longest tested delay.
Two physical tasks—a bimanual conveyor pick-and-place and a reactive whack-a-mole—each improve under the same tuning procedure.
The refinement works as a near drop-in addition to existing asynchronous VLA inference stacks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Longer or larger inference models could be used without shrinking the usable control horizon.
The same counterfactual construction might apply to other latency-sensitive control loops that lack dense reward signals.
Hardware requirements for low-latency inference hardware could be relaxed if the tuning generalizes across delay distributions.

Load-bearing premise

That an implicit flow-matching likelihood-ratio surrogate computed from a frozen reference policy can serve as a reliable label-free preference signal for tuning without human labels, reward models, or online rollouts.

What would settle it

Running the tuned policy on the same high-delay evaluation suite and observing success rates equal to or lower than the untuned baseline would falsify the claim that the surrogate supplies useful preference information.

Figures

Figures reproduced from arXiv: 2605.19294 by Jiaxiang Zou, Jingyu Guo, Rui Meng, Taowen Wang, Xinyu Chen, Yixiang Zhu, Yonghao Chen, Zijie Yang.

**Figure 2.** Figure 2: Async-VLA inference under deployment-relevant latency: a real-robot example. A robot dispenses pellets from a small container into a target vial on a moving conveyor belt (a pharmaceutical-filling analog). Top: four robot frames at the timeline points marked below. Bottom: two consecutive VLA inference cycles. The policy commits to actions at inference starts, when the vial is at the target pour location; … view at source ↗

**Figure 3.** Figure 3: DEFLECT framework. Steps 1–4 (offline training): (1) A frozen reference VLA produces preferred/rejected pairs from counterfactual fresh/stale contexts; (2) both are scored under the single deployment-time context c mix; (3) the calibrated DPO margin is formed; (4) the total loss updates the policy. Step 5 (online inference): the tuned VLA reads c mix and runs the same ODE as the base policy — no additiona… view at source ↗

**Figure 4.** Figure 4: Kinetix delay robustness. Success rate vs. inference delay d ∈ {0, . . . , 7} at K= max(d, 1). Horizon-robustness sweep and full per-delay numbers in Appendix E. Method avg(d=0-7) avg(d=5-7) NAIVE 42.4 1.5 RTC 48.9 2.0 BID 46.7 2.0 PFM (σ=0) 78.4 65.5 VLASH 79.4 67.1 DEFLECT 83.3 73.5 ∆ vs. VLASH +3.9 +6.4 ∆ vs. PFM +4.9 +8.0 [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

**Figure 5.** Figure 5: Real-robot results. (a) Full-task success rate on two conveyor pick-and-place tasks of increasing difficulty. (b) Mean moles struck per 30-second trial on the reactive Whack-a-Mole task. Across all three tasks, DEFLECT matches or improves on VLASH in the latency-sensitive regime. A 12-task analysis further shows the learned correction is dynamics-aware and variance-preserving (σ-ratio median 0.97, all 12 t… view at source ↗

**Figure 6.** Figure 6: Scoring-context ablation on Kinetix (success rate %, 12 tasks × 1024 rollouts). Top: a matched-context variant scoring each action under its own generating context (rθ(A+) at c fresh , rθ(A−) at c stale) uniformly underperforms the unified-c mix DEFLECT recipe across all 8 inference delays. Bottom: the gap ∆ grows monotonically from −0.3 at d=0 to −2.3 at d=7, precisely the regime where c mix differs most … view at source ↗

**Figure 7.** Figure 7: Kinetix horizon robustness at d=1. Success rate vs. execution horizon K ∈ {1, . . . , 8} at fixed delay d=1. DEFLECT’s improvement is consistent across K [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗

**Figure 8.** Figure 8: LIBERO 4-suite average success rate under varying inference delay. Gains grow from +0.2 at d=1 to +4.6 at d=7 [PITH_FULL_IMAGE:figures/full_fig_p017_8.png] view at source ↗

**Figure 9.** Figure 9: DEFLECT learns a delay-agnostic correction. Model A beats VLASH at every test delay despite never observing d ≥ 3 in the contrastive objective. H Ablation Studies All ablation runs start from the same VLASH async5 checkpoint, use the same 24-epoch cosine schedule, and vary only the ablated component. Preference-pair construction and SFT anchor [PITH_FULL_IMAGE:figures/full_fig_p018_9.png] view at source ↗

**Figure 10.** Figure 10: Ablations on Kinetix (delay-averaged success rate % over d ∈ {0, . . . , 7}). Five bars from left to right: VLASH baseline (79.4); λDPO=0, i.e. continued SFT only with our cosine schedule (82.0, isolates the cosine-restart effect); DPO with the VLASH-style partially-corrected rejection rather than naive-stale (81.9); DPO without the expert SFT anchor (catastrophic collapse to 12.3); and the full DEFLECT r… view at source ↗

**Figure 11.** Figure 11: Sensitivity to λDPO on Kinetix. Robust over λ ∈ [0.01, 0.05]; the final choice λ=0.02 maximizes the joint (delay-avg, horizon-avg) objective. d 0 1 2 3 4 5 6 7 avg DEFLECT (θ=0, main) 91.3 90.9 89.1 88.4 86.1 80.8 73.3 66.3 83.3 θ=2.5 (filters ≈ 10%) 91.4 91.0 89.1 88.4 86.0 80.7 73.3 66.3 83.3 θ=5.0 (filters ≈ 50%) 91.3 91.0 88.9 87.7 85.8 80.7 73.2 66.2 83.1 All three configurations are within 0.2 pp at… view at source ↗

**Figure 12.** Figure 12: What DEFLECT does to the flow-matching policy, aggregated across all 12 Kinetix tasks at d=6. (a) Intervention magnitude is dynamics-aware: contact-rich transient tasks (grasp easy, trampoline, launch-family, mean 0.37−0.60) receive the largest corrections; periodic locomotion gaits (mjc walker, mjc half cheetah, 0.15−0.18) the smallest. (b) Variance is preserved: per-task σ-ratios cluster tightly around… view at source ↗

**Figure 13.** Figure 13: Mean correction magnitude vs. chunk position (aggregated over 12 tasks × 4 critical states at d=6). Approximately uniform across the 8-position predicted chunk, with a mild emphasis on positions 0−2 [PITH_FULL_IMAGE:figures/full_fig_p023_13.png] view at source ↗

**Figure 14.** Figure 14: Catapult single-state action distribution (d=6, action dim 0, N=200 independent noise seeds). At this state, VLASH has µ=−0.24, σ=0.66; DEFLECT has µ=−0.64, σ=0.52. The mean displacement (∆µ=−0.39) is an order of magnitude larger than the variance change (∆σ=−0.14), consistent with the variance-preserving characterization at the multi-task level (Appendix K). This is the largest single-dimension variance… view at source ↗

read the original abstract

Vision-Language-Action (VLA) policies are typically deployed with asynchronous inference: the robot executes a previously predicted action chunk while the model computes the next one. This creates a prediction-execution misalignment: the chunk is conditioned on the observation taken before inference began, but executes in a physical state that has already drifted forward by several control steps; naive asynchronous rollover collapses from 89% to under 1% on Kinetix as the inference cycle covers up to seven control steps. We introduce DEFLECT, a fully offline post-training refinement that applies as a near drop-in upgrade to existing async-VLA stacks by converting latency itself into a label-free preference signal: counterfactual fresh/stale action pairs are constructed from a frozen reference policy and scored under the deployment-time conditioning via an implicit flow-matching likelihood-ratio surrogate, with no human labels, reward models, or online rollouts. DEFLECT substantially extends the usable delay envelope of async VLA control, with +6.4 success-rate gain in the high-latency regime (5-7 control steps), +4.6 when transferred to a real-scale VLA at the longest delay, and consistent improvements on two real-robot tasks (a bimanual conveyor pick-and-place and a reactive whack-a-mole).

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

DEFLECT turns inference delay into an offline preference signal for async VLA policies and reports usable gains on Kinetix and real robots, but the flow-matching surrogate's behavior under state drift is the part that still needs anchoring.

read the letter

The core contribution is a post-training step that builds fresh versus stale action pairs from a frozen reference policy and scores them with an implicit flow-matching likelihood ratio under the delayed observation. This produces a label-free tuning signal that the authors apply to improve robustness when the robot is already executing an old chunk while the next one is being computed. On the reported numbers it lifts success rate by 6.4 points in the 5-7 step latency regime and transfers to a real-scale VLA with a 4.6 point gain at the longest delay, plus consistent lifts on two physical tasks. That is the practical piece worth noticing: it is fully offline and presented as a near drop-in for existing stacks. The method is new in its exact combination of counterfactual construction and flow-matching ratio for this delay problem. Prior work on async VLA has mostly focused on faster inference or explicit prediction of future states, so this surrogate route is distinct. The experiments appear to include both simulation and hardware, which is better than many policy papers that stop at sim. The soft spot is exactly the one the stress-test flags. The flow-matching model is trained on nominal trajectories, yet it is asked to rank actions on observations that have drifted forward in time. Nothing in the construction guarantees that higher likelihood under the surrogate corresponds to higher downstream success rather than to some artifact of the reference policy's training distribution. If the paper only shows aggregate success rates without ablations that isolate the surrogate's correlation with actual task utility after drift, that link stays unproven. The abstract gives numeric gains but no variance, baseline details, or controls, so the strength of the evidence is still hard to judge from the summary alone. This is the kind of paper that matters to groups shipping VLA policies on real hardware where inference latency is fixed by model size. Readers who already run async stacks and need a way to squeeze more reliability out of them without new hardware will find the idea and the reported deltas useful. It is coherent on its own terms and engages a concrete deployment issue, so it clears the bar for serious refereeing even if the validation needs tightening in revision.

Referee Report

2 major / 1 minor

Summary. The paper introduces DEFLECT, an offline post-training method for Vision-Language-Action (VLA) policies that converts inference latency into a label-free preference signal. It constructs counterfactual fresh/stale action pairs from a frozen reference policy, scores them via an implicit flow-matching likelihood-ratio surrogate evaluated under delayed conditioning, and uses the resulting signal to tune the policy without human labels, reward models, or online rollouts. The central empirical claim is that this extends the usable delay envelope, yielding a +6.4 success-rate gain in the 5-7 control-step high-latency regime on Kinetix, a +4.6 gain when transferred to a real-scale VLA, and consistent improvements on two real-robot tasks.

Significance. If the surrogate reliably ranks actions by downstream task utility despite state drift, the result would be significant for asynchronous VLA deployment: it offers a practical, fully offline upgrade that widens the inference-time budget without additional supervision. The approach is notable for avoiding online rollouts and for framing latency itself as the source of the preference signal.

major comments (2)

[Abstract] Abstract: the reported +6.4 success-rate gain in the 5-7 step regime and +4.6 transfer gain are presented without any mention of trial counts, standard deviations, statistical tests, or the precise baseline (naive async rollover) implementation details; these omissions make it impossible to judge whether the numeric improvements are load-bearing evidence for the method or could be explained by variance or implementation differences.
[Method] Method description (implicit in abstract): the flow-matching model is trained on nominal (non-delayed) trajectories, yet the likelihood-ratio surrogate is evaluated on delayed conditioning; no derivation or empirical validation is supplied showing that higher surrogate likelihood under drift correlates with higher real-world success rather than with artifacts of the reference policy's training distribution.

minor comments (1)

[Abstract] Abstract: the two real-robot tasks are named but not described (e.g., observation space, action chunk length, or how delay is emulated on hardware); adding one sentence of setup detail would improve reproducibility.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments. We address each major point below and indicate where revisions will be made to strengthen the manuscript.

read point-by-point responses

Referee: [Abstract] Abstract: the reported +6.4 success-rate gain in the 5-7 step regime and +4.6 transfer gain are presented without any mention of trial counts, standard deviations, statistical tests, or the precise baseline (naive async rollover) implementation details; these omissions make it impossible to judge whether the numeric improvements are load-bearing evidence for the method or could be explained by variance or implementation differences.

Authors: We agree that the abstract should include these details to allow proper evaluation of the results. In the revised manuscript we will update the abstract to report the number of trials (50 per condition), standard deviations for the reported gains, and note that the improvements were assessed with paired t-tests against the naive asynchronous rollover baseline as defined in Section 3.1. revision: yes
Referee: [Method] Method description (implicit in abstract): the flow-matching model is trained on nominal (non-delayed) trajectories, yet the likelihood-ratio surrogate is evaluated on delayed conditioning; no derivation or empirical validation is supplied showing that higher surrogate likelihood under drift correlates with higher real-world success rather than with artifacts of the reference policy's training distribution.

Authors: The referee correctly notes that the manuscript lacks an explicit derivation or dedicated empirical validation of the correlation between the delayed-conditioning likelihood ratio and downstream success. The current presentation relies on the overall task-level improvements as indirect support. We will add a short derivation sketch in the appendix and include a new figure in Section 4.3 showing the correlation between surrogate scores and success rates on held-out delayed rollouts. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected in DEFLECT derivation

full rationale

The paper's core construction uses a frozen reference policy to generate counterfactual fresh/stale action pairs and an implicit flow-matching likelihood-ratio surrogate for label-free preference scoring under delayed conditioning. This is an external, offline procedure whose validity is not derived from the target policy's own outputs or fitted parameters by construction. The reported empirical gains (+6.4 success rate in high-latency regime) are presented as experimental outcomes rather than tautological predictions equivalent to the method inputs. No self-definitional loops, fitted-input-as-prediction reductions, or load-bearing self-citation chains appear in the abstract or method summary that would collapse the claimed results to the inputs by definition. The derivation remains self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claim depends on the flow-matching surrogate acting as a valid preference model and on the frozen reference policy providing useful counterfactuals; no explicit free parameters or new entities are named in the abstract.

axioms (2)

domain assumption The flow-matching model provides an implicit likelihood ratio that correlates with action quality under delayed conditioning.
Invoked when the abstract states the surrogate scores counterfactual pairs without labels or rewards.
domain assumption Counterfactual action pairs generated from a frozen reference policy are representative of the deployment-time distribution.
Required for the offline construction step described in the abstract.

pith-pipeline@v0.9.0 · 5784 in / 1399 out tokens · 40874 ms · 2026-05-20T05:59:42.603009+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

converts latency itself into a label-free preference signal: counterfactual fresh/stale action pairs are constructed from a frozen reference policy and scored under the deployment-time conditioning via an implicit flow-matching likelihood-ratio surrogate
IndisputableMonolith/Foundation/AlexanderDuality.lean alexander_duality_circle_linking unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

flow-matching likelihood-ratio surrogate computed from a frozen reference policy

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

39 extracted references · 39 canonical work pages · 9 internal anchors

[1]

Zitkovich, T

B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, et al. RT-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pages 2165–2183. PMLR, 2023

work page 2023
[2]

M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, et al. OpenVLA: An open-source vision-language-action model. InConference on Robot Learning (CoRL), 2024

work page 2024
[3]

$\pi_0$: A Vision-Language-Action Flow Model for General Robot Control

K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al.π 0: A vision-language-action flow model for general robot control.arXiv preprint arXiv:2410.24164, 2024

work page internal anchor Pith review Pith/arXiv arXiv 2024
[4]

Black, N

K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. R. Equi, C. Finn, N. Fusai, M. Y . Galliker, et al.π 0.5: A vision-language-action model with open-world gener- alization. InConference on Robot Learning (CoRL), 2025

work page 2025
[5]

Sendai, M

K. Sendai, M. Alvarez, T. Matsushima, Y . Matsuo, and Y . Iwasawa. Leave no observation behind: Real-time correction for VLA action chunks.arXiv preprint arXiv:2509.23224, 2025

work page arXiv 2025
[6]

Black, M

K. Black, M. Galliker, and S. Levine. Real-time execution of action chunking flow policies. Advances in Neural Information Processing Systems, 38:33383–33407, 2026

work page 2026
[7]

Y . Liu, J. Hamid, A. Xie, Y . Lee, M. Du, and C. Finn. Bidirectional decoding: Improving action chunking via guided test-time sampling. InInternational Conference on Learning Rep- resentations, volume 2025, pages 4594–4627, 2025

work page 2025
[8]

J. Tang, Y . Sun, Y . Zhao, S. Yang, Y . Lin, Z. Zhang, J. Hou, Y . Lu, Z. Liu, and S. Han. VLASH: Real-time vlas via future-state-aware asynchronous inference.arXiv preprint arXiv:2512.01031, 2025

work page arXiv 2025
[9]

AsyncVLA: Asynchronous Flow Matching for Vision-Language-Action Models

Y . Jiang, S. Cheng, Y . Ding, F. Gao, and B. Qi. AsyncVLA: Asynchronous flow matching for vision-language-action models.arXiv preprint arXiv:2511.14148, 2025. 9

work page internal anchor Pith review Pith/arXiv arXiv 2025
[10]

W. Xia, Y . Yang, H. Wu, X. Ma, T. Kong, and D. Hu. Human-assisted robotic policy refinement via action preference optimization.Advances in Neural Information Processing Systems, 38: 36746–36768, 2026

work page 2026
[11]

Zhong, Z

H. Zhong, Z. Li, X. Wang, and L. Huang. Reparameterization flow policy optimization, 2026. URLhttps://arxiv.org/abs/2602.03501

work page arXiv 2026
[12]

M. Kim, Y . Lee, S. Kang, J. Oh, S. Chong, and S.-Y . Yun. Preference alignment with flow matching. InAdvances in Neural Information Processing Systems, volume 37, pages 35140– 35164, 2024

work page 2024
[13]

Wallace, M

B. Wallace, M. Dang, R. Rafailov, L. Zhou, A. Lou, S. Purushwalkam, S. Ermon, C. Xiong, S. Joty, and N. Naik. Diffusion model alignment using direct preference optimization. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8228– 8238, 2024

work page 2024
[14]

Rafailov, A

R. Rafailov, A. Sharma, E. Mitchell, S. Ermon, C. D. Manning, and C. Finn. Direct prefer- ence optimization: Your language model is secretly a reward model. InAdvances in Neural Information Processing Systems, 2023

work page 2023
[15]

Flow Matching for Generative Modeling

Y . Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le. Flow matching for generative modeling.arXiv preprint arXiv:2210.02747, 2022

work page internal anchor Pith review Pith/arXiv arXiv 2022
[16]

M. S. Albergo and E. Vanden-Eijnden. Building normalizing flows with stochastic interpolants. arXiv preprint arXiv:2209.15571, 2022

work page internal anchor Pith review Pith/arXiv arXiv 2022
[17]

Flow matching policy gradients.arXiv preprint arXiv:2507.21053, 2025

D. McAllister, S. Ge, B. Yi, C. M. Kim, E. Weber, H. Choi, H. Feng, and A. Kanazawa. Flow matching policy gradients, 2025. URLhttps://arxiv.org/abs/2507.21053

work page arXiv 2025
[18]

Matthews, M

M. Matthews, M. Beukman, C. Lu, and J. Foerster. Kinetix: Investigating the training of general agents through open-ended physics-based control tasks. InInternational Conference on Learning Representations (ICLR), 2025

work page 2025
[19]

B. Liu, Y . Zhu, C. Gao, Y . Feng, Q. Liu, Y . Zhu, and P. Stone. LIBERO: Benchmarking knowledge transfer for lifelong robot learning. InAdvances in Neural Information Processing Systems, 2023

work page 2023
[20]

Y . Lu, Z. Liu, X. Fan, Z. Yang, J. Hou, J. Li, K. Ding, and H. Zhao. Faster: Rethinking real-time flow vlas, 2026. URLhttps://arxiv.org/abs/2603.19199

work page internal anchor Pith review Pith/arXiv arXiv 2026
[21]

X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model

J. Zheng, J. Li, Z. Wang, D. Liu, X. Kang, Y . Feng, Y . Zheng, J. Zou, Y . Chen, J. Zeng, Y .-Q. Zhang, J. Pang, J. Liu, T. Wang, and X. Zhan. X-vla: Soft-prompted transformer as scalable cross-embodiment vision-language-action model, 2025. URLhttps://arxiv.org/ abs/2510.10274

work page internal anchor Pith review Pith/arXiv arXiv 2025
[22]

Ghosh, H

Octo Model Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, C. Xu, J. Luo, et al. Octo: An open-source generalist robot policy. InRobotics: Science and Systems, 2024

work page 2024
[23]

GR00T N1: An Open Foundation Model for Generalist Humanoid Robots

NVIDIA. GR00T N1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025

work page internal anchor Pith review Pith/arXiv arXiv 2025
[24]

T. Z. Zhao, V . Kumar, S. Levine, and C. Finn. Learning fine-grained bimanual manipulation with low-cost hardware. InRobotics: Science and Systems (RSS), 2023

work page 2023
[25]

C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song. Diffusion policy: Visuomotor policy learning via action diffusion.The International Journal of Robotics Research, 44(10–11):1684–1704, 2025. 10

work page 2025
[26]

S. Liu, L. Wu, B. Li, H. Tan, H. Chen, Z. Wang, K. Xu, H. Su, and J. Zhu. RDT-1B: A dif- fusion foundation model for bimanual manipulation. InInternational Conference on Learning Representations, volume 2025, pages 29982–30009, 2025

work page 2025
[27]

Y . Shi, D. Guo, T. Zhao, F. Gao, L. Shi, C. Yu, Z. Mo, Q. Xiao, X. Peng, Q. Liao, et al. StreamingVLA: Streaming vision-language-action model with action flow matching and adap- tive early observation.arXiv preprint arXiv:2603.28565, 2026

work page arXiv 2026
[28]

X. Li, H. Tang, X. Ding, W. Wang, T. Cao, and Y . Liu. OxyGen: Unified KV cache management for vision-language-action models under multi-task parallelism.arXiv preprint arXiv:2603.14371, 2026

work page internal anchor Pith review Pith/arXiv arXiv 2026
[29]

Firoiu, T

V . Firoiu, T. Ju, and J. B. Tenenbaum. At human speed: Deep reinforcement learning with action delay. InAAAI Workshop on Reinforcement Learning in Games, 2018

work page 2018
[30]

Ramstedt and C

S. Ramstedt and C. Pal. Real-time reinforcement learning. InAdvances in Neural Information Processing Systems, 2019

work page 2019
[31]

P. F. Christiano, J. Leike, T. B. Brown, M. Martic, S. Legg, and D. Amodei. Deep reinforcement learning from human preferences. InAdvances in Neural Information Processing Systems, 2017

work page 2017
[32]

Ouyang, J

L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welin- der, P. Christiano, J. Leike, and R. Lowe. Training language models to follow instructions with human feedback. InAdvances in Neural Information Processing Systems, 2022

work page 2022
[33]

Hejna, R

J. Hejna, R. Rafailov, H. Sikchi, C. Finn, S. Niekum, W. B. Knox, and D. Sadigh. Contrastive preference learning: Learning from human feedback without RL. InInternational Conference on Learning Representations (ICLR), 2024

work page 2024
[34]

M. G. Azar, M. Rowland, B. Piot, D. Guo, D. Calandriello, M. Valko, and R. Munos. A general theoretical paradigm to understand learning from human preferences. InInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2024

work page 2024
[35]

Y . Meng, M. Xia, and D. Chen. SimPO: Simple preference optimization with a reference-free reward.arXiv preprint arXiv:2405.14734, 2024

work page arXiv 2024
[36]

KTO: Model Alignment as Prospect Theoretic Optimization

K. Ethayarajh, W. Xu, N. Muennighoff, D. Jurafsky, and D. Kiela. KTO: Model alignment as prospect theoretic optimization.arXiv preprint arXiv:2402.01306, 2024

work page internal anchor Pith review Pith/arXiv arXiv 2024
[37]

Vatsa, Z

A. Vatsa, Z. Xie, and W. Jin. RoDiF: Robust direct fine-tuning of diffusion policies with corrupted human feedback.arXiv preprint arXiv:2602.00886, 2026

work page arXiv 2026
[38]

Moletta, M

M. Moletta, M. C. Welle, and D. Kragic. Preference aligned visuomotor diffusion policies for deformable object manipulation.arXiv preprint arXiv:2602.09583, 2026. 11 A Implementation Details Preference-pair generation.BothA + andA − are produced online from the frozen VLASH reference policy,notfrom the dataset, so the pair always lives on the deployable a...

work page arXiv 2026
[39]

Paper” is [8] Table 1; “Ours

optimizes. Step 4: Monte Carlo realization.At training time, the expectations over(τ, ϵ)inL FM are es- timated with one sample per chunk, shared betweenA + andA − and betweenθandref(line 5 of Algorithm 1). Sharing(τ, ϵ)across the four terms cancels the within-margin sampling noise to leading order; this is the same variance-reduction step used in Diffusio...

work page

[1] [1]

Zitkovich, T

B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, et al. RT-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pages 2165–2183. PMLR, 2023

work page 2023

[2] [2]

M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, et al. OpenVLA: An open-source vision-language-action model. InConference on Robot Learning (CoRL), 2024

work page 2024

[3] [3]

$\pi_0$: A Vision-Language-Action Flow Model for General Robot Control

K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al.π 0: A vision-language-action flow model for general robot control.arXiv preprint arXiv:2410.24164, 2024

work page internal anchor Pith review Pith/arXiv arXiv 2024

[4] [4]

Black, N

K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. R. Equi, C. Finn, N. Fusai, M. Y . Galliker, et al.π 0.5: A vision-language-action model with open-world gener- alization. InConference on Robot Learning (CoRL), 2025

work page 2025

[5] [5]

Sendai, M

K. Sendai, M. Alvarez, T. Matsushima, Y . Matsuo, and Y . Iwasawa. Leave no observation behind: Real-time correction for VLA action chunks.arXiv preprint arXiv:2509.23224, 2025

work page arXiv 2025

[6] [6]

Black, M

K. Black, M. Galliker, and S. Levine. Real-time execution of action chunking flow policies. Advances in Neural Information Processing Systems, 38:33383–33407, 2026

work page 2026

[7] [7]

Y . Liu, J. Hamid, A. Xie, Y . Lee, M. Du, and C. Finn. Bidirectional decoding: Improving action chunking via guided test-time sampling. InInternational Conference on Learning Rep- resentations, volume 2025, pages 4594–4627, 2025

work page 2025

[8] [8]

J. Tang, Y . Sun, Y . Zhao, S. Yang, Y . Lin, Z. Zhang, J. Hou, Y . Lu, Z. Liu, and S. Han. VLASH: Real-time vlas via future-state-aware asynchronous inference.arXiv preprint arXiv:2512.01031, 2025

work page arXiv 2025

[9] [9]

AsyncVLA: Asynchronous Flow Matching for Vision-Language-Action Models

Y . Jiang, S. Cheng, Y . Ding, F. Gao, and B. Qi. AsyncVLA: Asynchronous flow matching for vision-language-action models.arXiv preprint arXiv:2511.14148, 2025. 9

work page internal anchor Pith review Pith/arXiv arXiv 2025

[10] [10]

W. Xia, Y . Yang, H. Wu, X. Ma, T. Kong, and D. Hu. Human-assisted robotic policy refinement via action preference optimization.Advances in Neural Information Processing Systems, 38: 36746–36768, 2026

work page 2026

[11] [11]

Zhong, Z

H. Zhong, Z. Li, X. Wang, and L. Huang. Reparameterization flow policy optimization, 2026. URLhttps://arxiv.org/abs/2602.03501

work page arXiv 2026

[12] [12]

M. Kim, Y . Lee, S. Kang, J. Oh, S. Chong, and S.-Y . Yun. Preference alignment with flow matching. InAdvances in Neural Information Processing Systems, volume 37, pages 35140– 35164, 2024

work page 2024

[13] [13]

Wallace, M

B. Wallace, M. Dang, R. Rafailov, L. Zhou, A. Lou, S. Purushwalkam, S. Ermon, C. Xiong, S. Joty, and N. Naik. Diffusion model alignment using direct preference optimization. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8228– 8238, 2024

work page 2024

[14] [14]

Rafailov, A

R. Rafailov, A. Sharma, E. Mitchell, S. Ermon, C. D. Manning, and C. Finn. Direct prefer- ence optimization: Your language model is secretly a reward model. InAdvances in Neural Information Processing Systems, 2023

work page 2023

[15] [15]

Flow Matching for Generative Modeling

Y . Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le. Flow matching for generative modeling.arXiv preprint arXiv:2210.02747, 2022

work page internal anchor Pith review Pith/arXiv arXiv 2022

[16] [16]

M. S. Albergo and E. Vanden-Eijnden. Building normalizing flows with stochastic interpolants. arXiv preprint arXiv:2209.15571, 2022

work page internal anchor Pith review Pith/arXiv arXiv 2022

[17] [17]

Flow matching policy gradients.arXiv preprint arXiv:2507.21053, 2025

D. McAllister, S. Ge, B. Yi, C. M. Kim, E. Weber, H. Choi, H. Feng, and A. Kanazawa. Flow matching policy gradients, 2025. URLhttps://arxiv.org/abs/2507.21053

work page arXiv 2025

[18] [18]

Matthews, M

M. Matthews, M. Beukman, C. Lu, and J. Foerster. Kinetix: Investigating the training of general agents through open-ended physics-based control tasks. InInternational Conference on Learning Representations (ICLR), 2025

work page 2025

[19] [19]

B. Liu, Y . Zhu, C. Gao, Y . Feng, Q. Liu, Y . Zhu, and P. Stone. LIBERO: Benchmarking knowledge transfer for lifelong robot learning. InAdvances in Neural Information Processing Systems, 2023

work page 2023

[20] [20]

Y . Lu, Z. Liu, X. Fan, Z. Yang, J. Hou, J. Li, K. Ding, and H. Zhao. Faster: Rethinking real-time flow vlas, 2026. URLhttps://arxiv.org/abs/2603.19199

work page internal anchor Pith review Pith/arXiv arXiv 2026

[21] [21]

X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model

J. Zheng, J. Li, Z. Wang, D. Liu, X. Kang, Y . Feng, Y . Zheng, J. Zou, Y . Chen, J. Zeng, Y .-Q. Zhang, J. Pang, J. Liu, T. Wang, and X. Zhan. X-vla: Soft-prompted transformer as scalable cross-embodiment vision-language-action model, 2025. URLhttps://arxiv.org/ abs/2510.10274

work page internal anchor Pith review Pith/arXiv arXiv 2025

[22] [22]

Ghosh, H

Octo Model Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, C. Xu, J. Luo, et al. Octo: An open-source generalist robot policy. InRobotics: Science and Systems, 2024

work page 2024

[23] [23]

GR00T N1: An Open Foundation Model for Generalist Humanoid Robots

NVIDIA. GR00T N1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025

work page internal anchor Pith review Pith/arXiv arXiv 2025

[24] [24]

T. Z. Zhao, V . Kumar, S. Levine, and C. Finn. Learning fine-grained bimanual manipulation with low-cost hardware. InRobotics: Science and Systems (RSS), 2023

work page 2023

[25] [25]

C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song. Diffusion policy: Visuomotor policy learning via action diffusion.The International Journal of Robotics Research, 44(10–11):1684–1704, 2025. 10

work page 2025

[26] [26]

S. Liu, L. Wu, B. Li, H. Tan, H. Chen, Z. Wang, K. Xu, H. Su, and J. Zhu. RDT-1B: A dif- fusion foundation model for bimanual manipulation. InInternational Conference on Learning Representations, volume 2025, pages 29982–30009, 2025

work page 2025

[27] [27]

Y . Shi, D. Guo, T. Zhao, F. Gao, L. Shi, C. Yu, Z. Mo, Q. Xiao, X. Peng, Q. Liao, et al. StreamingVLA: Streaming vision-language-action model with action flow matching and adap- tive early observation.arXiv preprint arXiv:2603.28565, 2026

work page arXiv 2026

[28] [28]

X. Li, H. Tang, X. Ding, W. Wang, T. Cao, and Y . Liu. OxyGen: Unified KV cache management for vision-language-action models under multi-task parallelism.arXiv preprint arXiv:2603.14371, 2026

work page internal anchor Pith review Pith/arXiv arXiv 2026

[29] [29]

Firoiu, T

V . Firoiu, T. Ju, and J. B. Tenenbaum. At human speed: Deep reinforcement learning with action delay. InAAAI Workshop on Reinforcement Learning in Games, 2018

work page 2018

[30] [30]

Ramstedt and C

S. Ramstedt and C. Pal. Real-time reinforcement learning. InAdvances in Neural Information Processing Systems, 2019

work page 2019

[31] [31]

P. F. Christiano, J. Leike, T. B. Brown, M. Martic, S. Legg, and D. Amodei. Deep reinforcement learning from human preferences. InAdvances in Neural Information Processing Systems, 2017

work page 2017

[32] [32]

Ouyang, J

L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welin- der, P. Christiano, J. Leike, and R. Lowe. Training language models to follow instructions with human feedback. InAdvances in Neural Information Processing Systems, 2022

work page 2022

[33] [33]

Hejna, R

J. Hejna, R. Rafailov, H. Sikchi, C. Finn, S. Niekum, W. B. Knox, and D. Sadigh. Contrastive preference learning: Learning from human feedback without RL. InInternational Conference on Learning Representations (ICLR), 2024

work page 2024

[34] [34]

M. G. Azar, M. Rowland, B. Piot, D. Guo, D. Calandriello, M. Valko, and R. Munos. A general theoretical paradigm to understand learning from human preferences. InInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2024

work page 2024

[35] [35]

Y . Meng, M. Xia, and D. Chen. SimPO: Simple preference optimization with a reference-free reward.arXiv preprint arXiv:2405.14734, 2024

work page arXiv 2024

[36] [36]

KTO: Model Alignment as Prospect Theoretic Optimization

K. Ethayarajh, W. Xu, N. Muennighoff, D. Jurafsky, and D. Kiela. KTO: Model alignment as prospect theoretic optimization.arXiv preprint arXiv:2402.01306, 2024

work page internal anchor Pith review Pith/arXiv arXiv 2024

[37] [37]

Vatsa, Z

A. Vatsa, Z. Xie, and W. Jin. RoDiF: Robust direct fine-tuning of diffusion policies with corrupted human feedback.arXiv preprint arXiv:2602.00886, 2026

work page arXiv 2026

[38] [38]

Moletta, M

M. Moletta, M. C. Welle, and D. Kragic. Preference aligned visuomotor diffusion policies for deformable object manipulation.arXiv preprint arXiv:2602.09583, 2026. 11 A Implementation Details Preference-pair generation.BothA + andA − are produced online from the frozen VLASH reference policy,notfrom the dataset, so the pair always lives on the deployable a...

work page arXiv 2026

[39] [39]

Paper” is [8] Table 1; “Ours

optimizes. Step 4: Monte Carlo realization.At training time, the expectations over(τ, ϵ)inL FM are es- timated with one sample per chunk, shared betweenA + andA − and betweenθandref(line 5 of Algorithm 1). Sharing(τ, ϵ)across the four terms cancels the within-margin sampling noise to leading order; this is the same variance-reduction step used in Diffusio...

work page