Pith.

Robotics · arXiv:2608.29768 v1 · submitted 30 Aug 2026 · 60 references

SmoothRL: Online Reinforcement Learning During Asynchronous Execution

Guang Gao, Yuxuan Nong, Baifu Huang, Jianan Wang

The paper says

“Deploying robot policies in the physical world requires satisfying two fundamental desiderata: reliability and smooth real-time execution. However, deploying state-of-the-art generalist models presents challenges on both fronts.”

more of the abstract

Achieving the precision and robustness required for real-world deployment necessitates sample-efficient online reinforcement learning (RL) to adapt pretrained models. Meanwhile, the increasing scale of robot foundation models has led to higher inference latency. To satisfy real-time constraints under high latency, modern systems adopt asynchronous inference with action chunking, overlapping policy computation with chunk execution to hide latency and enable smooth control. Despite their complementary roles, integrating asynchronous execution with gradient-based online RL remains underexplored. We present SmoothRL, an online RL framework that fine-tunes a pretrained policy within an asynchronous inference loop. SmoothRL follows a value-gradient paradigm, directly updating policy parameters using gradients of the action-value function with respect to policy actions. To enable correct optimization under asynchronous execution, SmoothRL explicitly models the asynchronous inference process during training. Specifically, each generated action chunk is partitioned by frame index into three regions: a committed region, consisting of actions committed by the previous inference cycle; an execution region, containing newly generated actions executed by the robot; and a discarded region, containing actions superseded by the next inference cycle. Gradients are propagated only through the execution region, ensuring policy optimization aligns with the trajectory distribution induced by asynchronous execution. We evaluate SmoothRL on real-world robotic tasks requiring high precision, as well as highly dynamic tasks that necessitate asynchronous execution.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 9 Sep 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. Well-motivated method for a genuine gap, but the load-bearing latency budget assumption needs more teeth and the evaluation lacks statistical replication. This line is wrong

4 major objections · 4 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 60 references say.

The paper says, at §4.4, Table 1

The review did not quote this passage.

Pith says: Objection 1 of 4 · major

The evaluation protocol evaluates each initial-state configuration exactly once, yielding only 10, 18, and 12 episodes per checkpoint, and all nonzero checkpoints come from a single RL run per task. The reported success rates are therefore point estimates with very wide binomial confidence intervals; for example, box opening at 150 episodes is 2/10, which is statistically indistinguishable from the base policy's 3/10.

more of objection 1

The claim that SmoothRL 'substantially outperforms' the frozen base policy needs repeated runs, confidence intervals, or at least a statement of the statistical significance of the observed differences.

The paper says, at §4.5, §3.3

The review did not quote this passage.

Pith says: Objection 2 of 4 · major

The core technical contribution — gradient truncation to the execution region — is not ablated. The experiments compare the full SmoothRL system against the frozen base policy, but never against a variant that propagates the value gradient through the full chunk, nor against a synchronous online RL baseline.

more of objection 2

Since the actor loss also contains a BC anchor and a smoothness regularizer, the observed gains could in principle come from those terms or from residual RL generally rather than from the truncation. An ablation isolating the truncation is needed to support the paper's central claim.

The paper says, at Eq. (5), §1 and §3.3

The review did not quote this passage.

Pith says: Objection 3 of 4 · major

The paper asserts that policy updates 'depend exclusively on actions whose consequences are observed,' but Eq. (5) contradicts this: the BC term ||a − a_target||^2 over [0,2n) and the smoothness penalty Δ^k a over [0,2n) both propagate gradients through the committed region [0,n), whose actions are overwritten by the preceding chunk and never executed.

more of objection 3

The claim is accurate only for the value-gradient pathway, not for the full actor loss. Please either restrict the regularizers to the execution region or revise the claim to describe the value-gradient pathway specifically.

The paper says, at §3.1, §5 Limitation 1

The review did not quote this passage.

Pith says: Objection 4 of 4 · major

The fixed latency budget n is load-bearing: the deterministic region boundaries [0,n), [n,2n), and the state augmentation all rely on chunk handover occurring exactly at the scheduled frame. The paper acknowledges that exceeding the budget destabilizes the loop, and it cites a TT-RTC scheduler for enforcing timing, but it provides no measurements of inference latency variability and no sensitivity analysis with respect to budget violations.

more of objection 4

Since the theoretical justification collapses if the timing assumption is violated, the paper should report measured inference latencies relative to n, or evaluate robustness to occasional budget overruns.

4 minor objections

The paper says, at §4.4, Figure 7

Pith says: minor objection 1 of 4

The learning curves would be much easier to interpret if the figure showed binomial confidence intervals or at least the raw numerators and denominators at each checkpoint.

The paper says, at §4.1, §4.3

Pith says: minor objection 2 of 4

The paper says the base policy π0.5 is 'fine-tuned for each task' in §4.1 but later describes 'supervised pretraining' using task demonstrations before online RL; please clarify whether these are two stages and how the task-specific pretraining relates to the released π0.5 checkpoint.

show 2 more

The paper says, at §4.3

Pith says: minor objection 3 of 4

The hyperparameter description gives G, D, batch size, and the correction bound, but omits the learning rate, discount factor, ensemble size N, and the smoothness weights w_smooth and w_k; these should be listed for reproducibility.

The paper says, at Algorithm 1

Pith says: minor objection 4 of 4

Line 14 uses the notation τ_{t-2n} ← τ_{t-2n} ∪ (r_{t-2n}, s_t, z_t, ä_{t:t+2n}, a_{[0,n)}), but the appended base-policy reference appears to be the reference at the bootstrap state, not at time t-2n; please align the notation with the transition fields defined in line 12.

What would settle it

Run the asynchronous loop with artificial latency perturbations: for a random 10% of inferences, add a delay of 1.5n frames (i.e., inference takes 1.5 times the budget). Compare the success rate on dynamic tossing after 250 episodes against the nominal (no delay) baseline. If the success rate drops by more than 10 percentage points (e.g., from 94% to below 84%), the fixed-budget assumption is load-bearing in practice and the method's robustness to timing violations is insufficient.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 9 Sep 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic agrees 24 s · 18,683 tokens read, 2,404 written
  3. Editor wrote the verdict above 19 s · 20,137 tokens read, 736 written
  4. Referee major revision: 4 major objections, 4 minor 58 s · 19,200 tokens read, 5,797 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

The paper identifies a real problem: online RL fine-tuning under asynchronous inference with action chunking. Prior work either assumes synchronous execution or works in latent space. SmoothRL's chunk-role partitioning and gradient truncation to the execution region is a clean, explicit fix for the mismatch between generated and executed actions. The formulation is rigorous, the instantiation is practical (raw action space, human intervention handles naturally), and the real-robot results look impressive—e.g., pen capping from 8% to 83% success.

What I like: they don't oversell. The limitations section is upfront about the fixed latency budget and the bounded expressiveness of residual corrections. The paper is clearly written and the method is reproducible in principle.

Soft spots: The load-bearing assumption—fixed latency budget n—is acknowledged but not stress-tested. If inference ever exceeds n, the chunk handover timing shifts and the entire gradient truncation argument loses its foundation. The paper provides no analysis of latency fluctuations in their hardware, nor any sensitivity study. This gap matters because real robot systems have jitter.

Second, the evaluation is thin: one episode per configuration per checkpoint, no error bars, a single run per task. With only 10–18 episodes per checkpoint, the success rates could swing by ±15% or more. They report monotonic improvement but without replication we can't tell if it's robust.

Third, no code or data release. The experiments use proprietary hardware (Astribot S1). Independent verification is impossible.

That said, this is an honest, nicely structured paper that tackles a genuine gap. The central claim—gradient truncation aligned with asynchronous execution—is plausible and well-articulated. The limitations are stated, not hidden. I think it deserves a serious referee, but the referee should push for multiple independent runs with standard errors and a robustness analysis of the latency budget. The paper would be stronger with a sensitivity experiment: artificially jitter inference times and measure success rate degradation.

Bottom line: yes, send to peer review. It's a solid engineering contribution with a clear theoretical framing, even if the experimental support is a bit fragile.

The strongest objection

The core technical contribution of SmoothRL is gradient truncation to the execution region [n,2n) of each action chunk. This partition requires the committed and execution region lengths to be fixed to n frames, which in turn requires that every policy inference completes within a known worst-case latency budget n. The paper acknowledges this as a limitation (Section 5) but provides no analysis of robustness to latency violations. In practical robotic deployments, inference time fluctuates due to system load, input variability, or model complexity; a fixed upper bound cannot be guaranteed without hard real-time scheduling. If inference occasionally exceeds n, the chunk handover occurs later than scheduled, the actual execution window shifts relative to frame indices, and the gradient computed against the nominal [n,2n) segment no longer aligns with the actions that were actually executed. The TD target (Eq. 3) and the state augmentation (using the committed region from the preceding chunk) both assume strict adherence to the timing model. Violations introduce uncorrected bias that can degrade or invalidate the policy update. The paper neither measures inference latency fluctuations in its experiments nor evaluates how sensitive the success rate is to violations of the budget. This gap is particularly concerning because the method's theoretical justification (unbiased chunk-skip bootstrap, correct gradient path) collapses when the timing assumption is violated.

The reader's weakest assumption was the fixed latency budget n, and I agree that this is the most load-bearing concern. The entire region partitioning and gradient truncation rest on deterministic boundaries [0,n), [n,2n). The paper acknowledges the limitation but does not analyze its impact, and the evaluation does not test robustness to latency jitter. My recommended check directly tests whether this assumption holds in practice. I leave the verdict unchanged because the method remains plausible and the limitation is explicitly stated; however, if the test reveals fragility, the verifier should downgrade to REJECT. The agreement is 'agree' because the reader correctly identified the same assumption as weakest.

The referee's report

If the claims hold, the paper addresses a genuine and underexplored gap: aligning online RL objectives with the action-execution semantics of asynchronous deployment. The chunk-role partitioning and the distinction between agreement and differentiability are useful conceptual contributions, and the raw-action-space treatment of human intervention is practically attractive. The derivation in Section 3.3 is clear and internally consistent, and Algorithm 1 provides a concrete, reproducible training loop. However, the empirical support is currently thin: the evaluation uses one episode per configuration, a single run per task, no ablation of the truncation mechanism, and no measurement of the latency-budget assumption that the entire formulation rests on. These gaps prevent the paper from fully establishing the strength of its central claims.

Pith's reading of the paper (a machine's summary, not the authors')

The paper addresses a fundamental mismatch in robot learning: online reinforcement learning methods typically assume synchronous execution, where the robot pauses at chunk boundaries, but real-world deployment uses asynchronous inference to hide latency. The authors argue that this mismatch causes policy updates to be optimized under dynamics that differ from deployment, preventing reliable fine-tuning. They introduce SmoothRL, a framework that explicitly models asynchronous execution during training by partitioning each action chunk into committed, execution, and discarded regions based on a fixed latency budget. By truncating the value gradient to only the execution region, the policy is updated exclusively on actions that actually reach the environment, restoring alignment between optimization and deployment. On three real robotic tasks, the approach raises success rates from 8-39% to 83-94% with 250 episodes of fine-tuning.

What carries the argument. Chunk role partitioning and value-gradient truncation under a fixed latency budget n. Each action chunk of horizon H is partitioned by frame index into committed [0,n), execution [n,2n), and discarded [2n,H) regions; the policy's value gradient ∇_a Q is computed only with respect to the execution region, so that the gradient path passes through actions that reach the environment. The critic is conditioned on the full action chunk with the committed region treated as a state augmentation (stop-gradient), which models the concurrent decision process and preserves the chunk-skip Bellman backup over the 2n-frame effective horizon.

Core claim. The paper's central claim is that value-gradient online RL can be correctly integrated with asynchronous inference by formalizing the partial execution of action chunks as a known constraint. Under a fixed latency budget n, each generated chunk splits into three frame-indexed regions: committed (frames [0,n), actions overwritten by the previous chunk), execution (frames [n,2n), actions actually executed), and discarded (frames [2n,H), actions superseded before execution). The key identity is that backpropagating ∇_a Q only through the execution region ensures policy updates depend exclusively on actions whose consequences are observed, while the critic retains the full chunk for unbiased temporal-difference backup. This gradient truncation, combined with running the asynchronous loop during training rollouts, aligns the optimization objective with the trajectory distribution induced by asynchronous deployment, enabling online fine-tuning that would otherwise be contaminated by unexecuted actions.

The premise it rests on. The entire chunk partitioning and gradient truncation rely on the inference loop being held to a fixed latency budget n; if inference ever takes longer than n, the scheduled chunk handover cannot be guaranteed and the deterministic region boundaries break down—the paper acknowledges this as a limitation.

If the paper is right (Pith's inferences, not the authors')

Robot policies can be fine-tuned with online RL in the same asynchronous loop used for deployment, eliminating the distribution mismatch that plagued earlier methods that trained under synchronous assumptions.

The gradient truncation principle applies to any value-gradient actor-critic algorithm operating on action chunks, not just the specific TD3+BC instantiation used in the paper, enabling a general class of asynchronous online RL methods.

Human interventions in raw action space can be incorporated directly as supervised regression targets and value-labeled transitions, because the RL variable remains in the same space as the human demonstration.

The fixed latency budget n makes region boundaries deterministic, simplifying the training objective but imposing a hard real-time constraint: if inference exceeds n, the chunk handover schedule breaks and the framework's guarantees no longer hold.

The gradient truncation idea could extend to settings beyond robotics, such as online fine-tuning of language models or game-playing agents that generate multi-step plans where only a prefix is actually executed before re-planning.

The reliance on a fixed latency budget may be relaxed by learning to predict region boundaries from observed inference times, or by using a scheduling policy that dynamically adjusts the budget to maintain timing guarantees without wasting slack.

Combining the method with action blending (mixing overlapping chunks) would require computing gradients through every chunk that contributes to the executed trajectory, which adds complexity but might improve smoothness; the paper flags this as future work but does not pursue it.

The 52% reduction in acceleration and 47% reduction in jerk reported for the throwing task suggest that online RL under asynchronous execution not only improves success but also naturally produces smoother motion, plausibly because the optimization objective penalizes discontinuities through the smoothness regularizer in equation (5).

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Deploying robot policies in the physical world requires satisfying two fundamental desiderata: reliability and smooth real-time execution. However, deploying state-of-the-art generalist models presents challenges on both fronts. Achieving the precision and robustness required for real-world deployment necessitates sample-efficient online reinforcement learning (RL) to adapt pretrained models. Meanwhile, the increasing scale of robot foundation models has led to higher inference latency. To satisfy real-time constraints under high latency, modern systems adopt asynchronous inference with action chunking, overlapping policy computation with chunk execution to hide latency and enable smooth control. Despite their complementary roles, integrating asynchronous execution with gradient-based online RL remains underexplored. We present SmoothRL, an online RL framework that fine-tunes a pretrained policy within an asynchronous inference loop. SmoothRL follows a value-gradient paradigm, directly updating policy parameters using gradients of the action-value function with respect to policy actions. To enable correct optimization under asynchronous execution, SmoothRL explicitly models the asynchronous inference process during training. Specifically, each generated action chunk is partitioned by frame index into three regions: a committed region, consisting of actions committed by the previous inference cycle; an execution region, containing newly generated actions executed by the robot; and a discarded region, containing actions superseded by the next inference cycle. Gradients are propagated only through the execution region, ensuring policy optimization aligns with the trajectory distribution induced by asynchronous execution. We evaluate SmoothRL on real-world robotic tasks requiring high precision, as well as highly dynamic tasks that necessitate asynchronous execution.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-09-23

The authors' main claim, as the classifier picked it out 0.45

“We present SmoothRL, an online RL framework that fine-tunes a pretrained policy within an asynchronous inference loop.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.10 · states limitation 0.13 · human participants 0.04 · controlled comparison 0.06 · randomized assignment 0.02 · states quantitative result 0.04 · compares with prior results 0.08 · mentions code or data availability 0.02

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Deploying robot policies in the physical world requires satisfying two fundamental desiderata: reliability and smooth real-time execution. background 1.00 0.05 0.03 asserted 1.00
However, deploying state-of-the-art generalist models presents challenges on both fronts. background 0.91 0.07 0.02 asserted 0.99
Achieving the precision and robustness required for real-world deployment necessitates sample-efficient online reinforcement learning (RL) to adapt pretrained models. background 0.99 0.06 0.04 asserted 1.00
Meanwhile, the increasing scale of robot foundation models has led to higher inference latency. background 1.00 0.08 0.02 asserted 1.00
To satisfy real-time constraints under high latency, modern systems adopt asynchronous inference with action chunking, overlapping policy computation with chunk execution to hide latency and enable smooth control. background 1.00 0.05 0.04 asserted 1.00
Despite their complementary roles, integrating asynchronous execution with gradient-based online RL remains underexplored. gap 1.00 0.06 0.04 asserted 1.00
We present SmoothRL, an online RL framework that fine-tunes a pretrained policy within an asynchronous inference loop. contribution 1.00 0.94 0.22 asserted 1.00
SmoothRL follows a value-gradient paradigm, directly updating policy parameters using gradients of the action-value function with respect to policy actions. method 0.96 0.81 0.09 asserted 1.00
To enable correct optimization under asynchronous execution, SmoothRL explicitly models the asynchronous inference process during training. method 0.93 0.86 0.08 asserted 1.00
Specifically, each generated action chunk is partitioned by frame index into three regions: a committed region, consisting of actions committed by the previous inference cycle; an execution region, containing newly generated actions executed by the robot; and a discarded region, containing actions superseded by the next inference cycle. method 0.99 0.77 0.05 asserted 1.00
Gradients are propagated only through the execution region, ensuring policy optimization aligns with the trajectory distribution induced by asynchronous execution. method 0.99 0.83 0.13 asserted 1.00
We evaluate SmoothRL on real-world robotic tasks requiring high precision, as well as highly dynamic tasks that necessitate asynchronous execution. contribution 0.56 0.95 0.11 asserted 0.99

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.95

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 8,796 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

References (60)
  1. [1] Kevin Black et al. Real-Time Execution of Action Chunking Flow Policies. arXiv:2506.07339, 2025 on PithPith's reviewarXiv
  2. [2] Kevin Black et al. Training-Time Action Conditioning for Efficient Real-Time Chunking. arXiv:2512.05964, 2025 on PitharXiv
  3. [3] Yufeng Liu et al. Learning Native Continuation for Action Chunking Flow Policies. arXiv:2602.12978, 2026 on PithPith's reviewarXiv
  4. [4] Dongyang Liu et al. Action-Prior Denoising for Smooth Real-Time Chunking. arXiv:2605.25537, 2026 on PithPith's reviewarXiv
  5. [5] Cheng Chi et al. Diffusion Policy: Visuomotor Policy Learning via Action Diffusion. arXiv:2303.04137, 2023 on PithPith's reviewarXiv
  6. [6] Tony Z. Zhao et al. Learning Fine-Grained Bimanual Manipulation with Low-Cost Hard- ware. arXiv:2304.13705, 2023 on PithPith's reviewarXiv
  7. [7] Perry Dong et al. EXPO-FT: Sample-Efficient Reinforcement Learning Finetuning for Vision- Language-Action Models. arXiv:2605.25477, 2026 on PithPith's reviewarXiv
  8. [8] Chenyu Yang et al. SERNF: Sample-Efficient Real-World Dexterous Policy Fine-Tuning via Action-Chunked Critics and Normalizing Flows. arXiv:2602.09580, 2026 on PithPith's reviewarXiv
  9. [9] Charles Xu et al. RL Token: Bootstrapping Online RL with Vision-Language-Action Models. arXiv:2604.23073, 2026 on PithPith's reviewarXiv
  10. [10] Dongchi Huang et al. CO-RFT: Efficient Fine-Tuning of Vision-Language-Action Models through Chunked Offline Reinforcement Learning. arXiv:2508.02219, 2025 on PithPith's reviewarXiv
  11. [11] Wenli Xiao et al. Self-Improving Vision-Language-Action Models with Data Generation via Residual RL. arXiv:2511.00091, 2025 on PitharXiv
  12. [12] Lars Ankile et al. Residual Off-Policy RL for Finetuning Behavior Cloning Policies. arXiv:2509.19301, 2025 on PitharXiv
  13. [13] Guozheng Ma et al. What Makes Value Learning Efficient in Residual Reinforcement Learning? arXiv:2602.10539, 2026 on PitharXiv
  14. [14] Qiyang Li et al. Decoupled Q-Chunking. arXiv:2512.10926, 2025 on PitharXiv
  15. [15] Guanxing Lu et al. VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning. arXiv:2505.18719, 2025 on PithPith's reviewarXiv
  16. [16] Haozhan Li et al. SimpleVLA-RL: Scaling VLA Training via Reinforcement Learning. arXiv:2509.09674, 2025 on PithPith's reviewarXiv
  17. [17] Yanjiang Guo et al. Improving Vision-Language-Action Model with Online Reinforcement Learning. arXiv:2501.16664, 2025 on PithPith's reviewarXiv
  18. [18] Si-Cheng Wang et al. VLA Model Post-Training via Action-Chunked PPO and Self Behavior Cloning. arXiv:2509.25718, 2025. 23 on PitharXiv
  19. [19] Kang Chen et al. 𝜋RL: Online RL Fine-tuning for Flow-based Vision-Language-Action Models. arXiv:2510.25889, 2025 on PitharXiv
  20. [20] Siting Wang et al.𝜋-StepNFT: Wider Space Needs Finer Steps in Online RL for Flow-based VLAs. arXiv:2603.02083, 2026 on PithPith's reviewarXiv
show 40 more
  1. [21] Allen Z. Ren et al. Diffusion Policy Policy Optimization. arXiv:2409.00588, 2024 on PithPith's reviewarXiv
  2. [22] Tonghe Zhang et al. ReinFlow: Fine-tuning Flow Matching Policy with Online Reinforce- ment Learning. arXiv:2505.22094, 2025 on PitharXiv
  3. [23] Seohong Park et al. Flow Q-Learning. arXiv:2502.02538, 2025 on PithPith's reviewarXiv
  4. [24] Kun Lei et al. RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learning. arXiv:2510.14830, 2025 on PitharXiv
  5. [25] Yuhui Chen et al. ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consis- tency Policy. arXiv:2502.05450, 2025 on PithPith's reviewarXiv
  6. [26] Yi Wang et al. Learning While Deploying: Fleet-Scale Reinforcement Learning for Generalist Robot Policies. arXiv:2605.00416, 2026 on PithPith's reviewarXiv
  7. [27] Perry Dong et al. EXPO: Stable Reinforcement Learning with Expressive Policies. arXiv:2507.07986, 2025 on PithPith's reviewarXiv
  8. [28] Jianlan Luo et al. Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning. arXiv:2410.21845, 2024 on PithPith's reviewarXiv
  9. [29] Checheng Yu et al.𝜒0: Resource-Aware Robust Manipulation via Taming Distributional Inconsistencies. arXiv:2602.09021, 2026 on PitharXiv
  10. [30] Physical Intelligence.𝜋 ∗ 0.6: a VLA That Learns From Experience. arXiv:2511.14759, 2025 on PithPith's reviewarXiv
  11. [31] ByteDance Seed. GR-RL: Going Dexterous and Precise for Long-Horizon Robotic Manipu- lation. arXiv:2512.01801, 2025 on PitharXiv
  12. [32] Andrew Wagenmaker et al. Steering Your Diffusion Policy with Latent Space Reinforcement Learning. arXiv:2506.15799, 2025 on PithPith's reviewarXiv
  13. [33] Qiyang Li et al. Reinforcement Learning with Action Chunking. arXiv:2507.07969, 2025 on PithPith's reviewarXiv
  14. [34] Ge Li et al. TOP-ERL: Transformer-based Off-Policy Episodic Reinforcement Learning. arXiv:2410.09536, 2024 on PithPith's reviewarXiv
  15. [35] Ted Xiao et al. Thinking While Moving: Deep Reinforcement Learning with Concurrent Control. arXiv:2004.06089, 2020 on PithPith's reviewarXiv
  16. [36] Ayoub Agouzoul. Understanding Asynchronous Inference Methods for Vision-Language- Action Models. arXiv:2605.08168, 2026 on PithPith's reviewarXiv
  17. [37] Jiaming Tang et al. VLASH: Real-Time VLAs via Future-State-Aware Asynchronous Infer- ence. arXiv:2512.01031, 2025 on PithPith's reviewarXiv
  18. [38] Astribot Team. Asynchronous Fast-Slow Vision-Language-Action Policies for Whole-Body Robotic Manipulation. arXiv:2512.20188, 2025. 24 on PitharXiv
  19. [39] Kohei Sendai et al. Leave No Observation Behind: Real-time Correction for VLA Action Chunks. arXiv:2509.23224, 2025 on PitharXiv
  20. [40] Physical Intelligence.𝜋0: A Vision-Language-Action Flow Model for General Robot Control. arXiv:2410.24164, 2024 on PithPith's reviewarXiv
  21. [41] Physical Intelligence.𝜋0.5: a Vision-Language-Action Model with Open-World Generaliza- tion. arXiv:2504.16054, 2025 on PithPith's reviewarXiv
  22. [42] Songming Liu et al. RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation. arXiv:2410.07864, 2024 on PithPith's reviewarXiv
  23. [43] Yue Su et al. World Guidance: World Modeling in Condition Space for Action Generation. arXiv:2602.22010, 2026 on PitharXiv
  24. [44] Lin Li et al. Causal World Modeling for Robot Control. arXiv:2601.21998, 2026 on PithPith's reviewarXiv
  25. [45] BeingBeyond Team. Being-H0.7: A Latent World-Action Model from Egocentric Videos. arXiv:2605.00078, 2026 on PithPith's reviewarXiv
  26. [46] Astribot Team. Towards Predictive, Aligned, and Scalable Robot Learning. arXiv:2607.11270, 2026 on PithPith's reviewarXiv
  27. [47] NVIDIA. GR00T N1: An Open Foundation Model for Generalist Humanoid Robots. arXiv:2503.14734, 2025 on PithPith's reviewarXiv
  28. [48] Tianyuan Yuan et al. Fast-WAM: Do World Action Models Need Test-time Future Imagina- tion? arXiv:2603.16666, 2026 on PithPith's reviewarXiv
  29. [49] Moo Jin Kim et al. Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning. arXiv:2601.16163, 2026 on PithPith's reviewarXiv
  30. [50] Xue Bin Peng et al. Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learning. arXiv:1910.00177, 2019 on PithPith's reviewarXiv
  31. [51] Kevin Frans et al. Diffusion Guidance Is a Controllable Policy Improvement Operator. arXiv:2505.23458, 2025 on PithPith's reviewarXiv
  32. [52] Chenxiao Gao et al. FlowRL: A Taxonomy and Modular Framework for Reinforcement Learning with Diffusion Policies. arXiv:2603.27450, 2026 on PitharXiv
  33. [53] Mitsuhiko Nakamoto et al. Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance. arXiv:2410.13816, 2024 on PithPith's reviewarXiv
  34. [54] Jacky Kwok et al. RoboMonkey: Scaling Test-Time Sampling and Verification for Vision- Language-Action Models. arXiv:2506.17811, 2025 on PithPith's reviewarXiv
  35. [55] Qiyang Li, Sergey Levine. Q-learning with Adjoint Matching. arXiv:2601.14234, 2026 on PithPith's reviewarXiv
  36. [56] Xinyue Chen et al. Randomized Ensembled Double Q-Learning: Learning Fast Without a Model. arXiv:2101.05982, 2021 on PithPith's reviewarXiv
  37. [57] Jimmy Lei Ba et al. Layer Normalization. arXiv:1607.06450, 2016 on PithPith's reviewarXiv
  38. [58] Scott Fujimoto et al. Addressing Function Approximation Error in Actor-Critic Methods. arXiv:1802.09477, 2018. 25 on PithPith's reviewarXiv
  39. [59] Scott Fujimoto, Shixiang Shane Gu. A Minimalist Approach to Offline Reinforcement Learning. arXiv:2106.06860, 2021 on PithPith's reviewarXiv
  40. [60] Tuomas Haarnoja et al. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. arXiv:1801.01290, 2018. 26 on PithPith's reviewarXiv
Cited by 1 Pith paper
How the literature uses it

What this paper does with the arXiv papers it cites

Each citing sentence is taken from the citing paper's LaTeX source and typed by Pith's classifier. A label appears at probability 0.8 or more (0.95 for support), a contest at 0.7 or more; the quoted words are the citing authors'. How each label was checked · JSON

What it cites · 51 sentences without a confident label

Uses the method or tool of 6 papers
Compares with 5 papers
  • GR-RL [ref] performs online learning under asynchronous execution, but places the RL optimization variable in the latent noise space of a frozen diffusion policy [ref] .

    2512.01801 · Asynchronous Restricted Online RL · 0.96

  • GR-RL [ref] performs online learning under asynchronous execution, but places the RL optimization variable in the latent noise space of a frozen diffusion policy [ref] .

    Steering Your Diffusion Policy with Latent Space Reinforcement Learning · Asynchronous Restricted Online RL · 0.96

  • Chunk-level off-policy methods outside the frozen-policy setting [ref] nevertheless adopt the same chunk-level optimization unit as our approach.

    TOP-ERL: Transformer-based Off-Policy Episodic Reinforcement Learning · Synchronous Online RL · 0.83

Cites as background 24 papers
  • A family of chunk-stitching methods [ref] therefore studies how to maintain continuity between newly generated actions and those already committed for execution.

    2512.05964 · Introduction · 0.83

  • A family of chunk-stitching methods [ref] therefore studies how to maintain continuity between newly generated actions and those already committed for execution.

    Action-Prior Denoising for Smooth Real-Time Chunking · Introduction · 0.83

  • A family of chunk-stitching methods [ref] therefore studies how to maintain continuity between newly generated actions and those already committed for execution.

    Learning Native Continuation for Action Chunking Flow Policies · Introduction · 0.83

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment