Pith. sign in

REVIEW 4 major objections 6 minor 24 references

Expert drone policies can be distilled into compact vision-language-action models that follow semantic navigation commands and generalize to unseen object-color pairings.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 04:32 UTC pith:2F3ZWOVR

load-bearing objection Useful expert-to-VLA pipeline for drone navigation with solid sim numbers and released data; zero-shot compositionality claim is softer than the abstract suggests. the 4 major comments →

arxiv 2607.03146 v1 pith:2F3ZWOVR submitted 2026-07-03 cs.RO

Exp2VLA: Enabling Vision-Language-Action for Drone Navigation from Expert Demonstrations

Van Huyen Dang , Kabilesh Rajendran , Erdi Sayar , Erdal Kayacan This is my paper
classification cs.RO
keywords vision-language-actiondrone navigationexpert distillationlanguage-conditioned controlaerial roboticsimitation learningsim-to-real
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that large, heavy vision-language-action models are not required for practical language-conditioned drone flight. Instead, high-quality expert rollouts—whether from reinforcement learning, teleoperation, or classical controllers—can be recorded in simulation and converted into training data that fine-tunes smaller VLA architectures. The resulting policies take first-person RGB images plus a natural-language command and output continuous velocity commands, enabling end-to-end navigation without hand-crafted intermediate modules. Closed-loop experiments show that a mid-scale model reaches roughly two-thirds success on multi-object scenes whose color-shape combinations appeared in training and still exceeds 50 percent success on color-permuted targets never seen before. The practical payoff is that existing specialized controllers can be transferred into a single reusable language-guided policy, lowering the barrier to deploying new aerial behaviors on resource-constrained platforms.

Core claim

Expert demonstration data collected from goal-directed policies can be distilled into compact vision-language-action models that execute language-conditioned drone navigation in multi-object scenes and retain nontrivial success on previously unseen object-color compositions.

What carries the argument

Exp2VLA: an expert-distillation pipeline that records RGB frames, drone state, velocity actions, and language instructions from oracle policies, converts them to a standard dataset format, and fine-tunes only the action expert of a mid-scale VLA while freezing the vision backbone.

Load-bearing premise

That clean, goal-offset expert trajectories collected in a structured simulator, combined with freezing the vision encoder, are enough for the model to learn true compositional color-shape grounding rather than scene-specific visuomotor shortcuts.

What would settle it

Train and evaluate the same pipeline on a held-out set of object-color compositions never present in any expert episode; if zero-shot success collapses to near chance while in-distribution success remains high, the claimed compositional generalization fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Existing RL, teleoperation, or classical drone controllers can be converted into language-conditioned VLA policies without redesigning perception and planning stacks.
  • Mid-scale VLAs (around 3–4 B parameters) become viable for aerial navigation under consumer-GPU memory budgets when only the action expert is updated.
  • Sim-to-sim and off-board SITL deployments become a practical intermediate step toward real-time language-guided flight on resource-limited airframes.
  • Multi-task training with varied color-shape instructions yields policies that transfer across distinct target objects rather than overfitting to a single object class.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If expert quality is the dominant factor, the same pipeline could absorb human teleoperation data from outdoor flights and close the remaining sim-to-real gap more quickly than pure synthetic rollouts.
  • The performance gap between the mid-scale and ultra-compact backbones implies that visual-encoder capacity, not action-chunk length alone, is the binding constraint for multi-object aerial scenes.
  • Standardized multi-object aerial VLA benchmarks with explicit color-shape permutations would make the zero-shot claims directly comparable across future methods.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces Exp2VLA, an expert-distillation pipeline that converts demonstrations from a PPO oracle (or other controllers) in Isaac Lab into LeRobot-format datasets and fine-tunes compact VLA models (π0.5 ~4B and SmolVLA ~0.45B) for language-conditioned quadrotor navigation. Expert rollouts record RGB, body-frame state, velocity-style actions (vx, vz, yaw rate), and natural-language instructions; fine-tuning freezes the vision encoder and updates the action expert under imitation learning with action chunking (K=50). Closed-loop evaluation in Isaac Lab reports success rates over 500 episodes per task (ϵ=0.60 m) on a single-object baseline, a multi-object heterogeneous scene (~65% average for π0.5), and a color-permuted zero-shot scene (~51.7% average), plus latency/VRAM profiling and a qualitative sim-to-sim / SITL demonstration with off-board inference via ROS2. Datasets are released publicly.

Significance. If the empirical claims hold under stronger controls, Exp2VLA is a useful systems contribution for aerial robotics: it shows how existing expert policies can be transferred into a unified language-conditioned VLA without hand-engineered perception stacks, and it documents that a mid-scale architecture (π0.5) can outperform an ultra-compact backbone on multi-object aerial navigation while remaining closer to edge budgets than 7B-class models. Concrete strengths include public LeRobot-format datasets, systematic multi-object and color-permuted evaluation with large episode counts, training curves, and measured loop latency/VRAM. The work is primarily an engineering pipeline and empirical study rather than a new learning algorithm; its value for the community is reproducibility and a practical path from expert controllers to language-guided flight.

major comments (4)
  1. §IV-B and Table V: The central claim that the fine-tuned model “generalize[s] to unseen object–color compositions” (abstract, contributions, §IV-B) is only weakly supported. Zero-shot success averages 51.7% but varies sharply across targets (green cube 51%, red cylinder 75%, blue cone 29%). Combined with the training regime in §III-A/B—PPO goals fixed 0.5 m in front of the visual target, fixed 247-step (~5 s) episodes, reduced action manifold, and a frozen SigLIP/PaliGemma vision encoder with only the Flow Matching action expert updated—the policy can succeed via residual visual/spatial shortcuts (blob size/position, approach stereotypes) rather than compositional binding of color and shape tokens. Without controls that break residual visual cues while holding language fixed (e.g., novel shapes, distractor-matched colors, language-only ablations, or unfreezing/comparing the vision encode
  2. §IV-B, Tables III–V: There are no non-VLA or alternative-policy baselines for the same language-conditioned navigation tasks (e.g., classical open-vocabulary detector + goal-conditioned controller, goal-conditioned behavioral cloning without a VLM backbone, or a larger OpenVLA-style model under the same data budget). The only comparison is π0.5 vs SmolVLA under Exp2VLA. Without such baselines it is hard to isolate what the VLA architecture and language conditioning contribute beyond the expert’s stereotyped approach trajectories distilled into the dataset.
  3. Tables III–V: Success rates are reported as point estimates over 500 episodes per task with no standard errors, confidence intervals, or statistical tests. Given the large zero-shot variance across objects and the binary success definition (Eq. 1, ϵ=0.60 m), uncertainty quantification is load-bearing for interpreting both the ~65% in-distribution multi-object average and the 51.7% zero-shot average.
  4. §IV-C and Fig. 6: The SITL / real-platform claim is only qualitative (trajectory snapshots with off-board workstation inference). No quantitative success rates, failure modes, latency under ROS2, or domain-gap metrics are reported for the physical drone. The abstract and conclusion present “simulation-in-the-loop” as supporting real-time deployability; that claim needs at least a small quantitative SITL evaluation or should be clearly scoped as a feasibility demo only.
minor comments (6)
  1. §II: Several UAV-VLA / VLN citations are arXiv preprints with placeholder-style author lists (“et al.”); please verify completeness and add any concurrent peer-reviewed aerial VLA work for fair positioning.
  2. Fig. 3: Training loss and gradient norm are shown only for π0.5; adding SmolVLA curves (or stating they were omitted) would make the capacity comparison clearer.
  3. Algorithm 1: The success check is nested inside the action-chunk execution loop; a short note on whether partial chunks are truncated on success would aid reimplementation.
  4. Table II / §III-A: Action dimensions are described as spanning [−1,1] after normalization, but the mapping from normalized commands to physical velocities is not stated; please specify the denormalization used at deployment.
  5. Typographical consistency: “UA Vs” / “UAVs”, “π 0.5” spacing, and “A V1” codec appear inconsistently; unify notation throughout.
  6. §IV-A: Episode duration is stated as 5 s while data-collection episodes are 247 steps (4.94 s); clarify control frequency and whether evaluation horizon matches training.

Circularity Check

0 steps flagged

No circularity: empirical imitation-learning pipeline with held-out evaluation; success rates are measured outcomes, not quantities forced by construction from fitted inputs.

full rationale

Exp2VLA is a systems paper that collects expert rollouts (PPO oracle in Isaac Lab), converts them to LeRobot format, fine-tunes existing VLA backbones (π0.5 / SmolVLA) via imitation learning with a frozen vision encoder, and reports closed-loop success rates under randomized initial conditions and a color-permuted zero-shot scene. There is no first-principles derivation, no uniqueness theorem, no parameter fitted to a subset of the target metric and then re-presented as a prediction, and no load-bearing self-citation that reduces the central claim to an unverified prior result by the same authors. The expert goal offset (0.5 m), fixed episode length, reduced action manifold, and frozen SigLIP are design choices that affect what the policy can learn; they do not make the reported success percentages (Tables III–V) true by definition. Evaluation uses independent episode randomizations and an unseen color permutation, so the numbers are empirical measurements rather than tautologies. Minor self-citations to the authors’ earlier drone-perception papers appear only as background and are not required for the pipeline or the success-rate claims. Consequently the derivation chain contains no circular steps.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 1 invented entities

The central claim rests on standard RL/imitation assumptions, simulator fidelity, and several hand-chosen training and evaluation knobs. No new physical entities are postulated; the main invented construct is the Exp2VLA pipeline itself as an engineering assembly of existing pieces (PPO expert, LeRobot format, frozen-backbone VLA fine-tuning).

free parameters (6)
  • success radius ϵ = 0.60 m
    Evaluation success is defined by min distance ≤ ϵ with ϵ=0.60 m; this threshold directly sets reported success rates.
  • action chunk size K = 50
    Closed-loop control executes K predicted actions before re-inference; K=50 is chosen and affects latency and tracking behavior.
  • fine-tuning learning rate and schedule = 5e-4, 60k steps
    lr=5e-4 with cosine decay over 60k steps is a hand-set optimization choice that shapes the final policy.
  • batch sizes (π0.5 / SmolVLA) = 16 / 64
    Batch 16 vs 64 chosen for memory/throughput on a single RTX 4080 Super; affects optimization trajectory.
  • expert goal offset = 0.5 m
    Expert goals are placed 0.5 m in front of visual targets to aid VLA fine-tuning; this shapes demonstration distribution.
  • episode length for data collection = 247 steps
    Fixed 247 steps (~4.94 s) per episode determines temporal coverage of demos.
axioms (5)
  • domain assumption Imitation learning on expert velocity trajectories is sufficient to transfer goal-directed navigation into a language-conditioned VLA policy.
    Core of §III-B; no theoretical guarantee, only empirical fine-tuning success.
  • domain assumption Isaac Lab RGB observations and rigid-body quadrotor dynamics are adequate proxies for the intended closed-loop navigation task.
    All quantitative claims in §IV are sim-based; SITL still uses sim perception/control loop with off-board inference.
  • ad hoc to paper Freezing the vision encoder while training only the action expert preserves enough visual-semantic features for multi-object discrimination.
    Explicit training regime in §III-B for memory reasons; performance gap vs SmolVLA is later attributed to encoder capacity.
  • domain assumption PPO with explicit spatial goals yields expert behavior suitable as an oracle for semantic language commands.
    §III-A: language labels are attached to goal-directed expert rollouts rather than learned jointly from language.
  • standard math Standard supervised/imitation optimization and Flow Matching action modeling behave as in prior VLA literature.
    Uses established PPO, imitation learning, and π0.5/SmolVLA architectures without new proofs.
invented entities (1)
  • Exp2VLA pipeline no independent evidence
    purpose: Name the end-to-end assembly of expert collection, LeRobot conversion, compact VLA fine-tuning, and closed-loop aerial evaluation.
    Engineering construct, not a new physical or mathematical object; independent evidence is the released datasets and reported experiments, not an external falsifiable entity.

pith-pipeline@v1.1.0-grok45 · 15996 in / 3555 out tokens · 37258 ms · 2026-07-12T04:32:58.154515+00:00 · methodology

0 comments
read the original abstract

Vision-language-action (VLA) models open a new path toward intuitive robot control by directly linking perception, language, and action in a single end-to-end framework. Yet for UAVs, practical adoption remains difficult because existing solutions are either computationally heavy or insufficiently capable in complex environments. In this work, we propose a practical expert-distillation pipeline (Exp2VLA) for language-conditioned drone navigation. The core idea is to distill expert behavior, obtained from reinforcement learning, teleoperation, or other controllers, into training data that can be used to fine-tune compact VLA models. This allows existing control strategies to be transferred into a unified language-guided navigation model, reducing manual system integration and lowering the barrier for deploying new robot behaviors. Experiments in both sim-to-sim and simulation-in-the-loop settings across multi-object scenes show that the fine-tuned models can handle varied semantic commands and generalize to unseen target compositions. The proposed framework demonstrates how expert-policy distillation can help mechatronic systems move from specialized control modules toward more flexible and reusable robot intelligence.

Figures

Figures reproduced from arXiv: 2607.03146 by Erdal Kayacan, Erdi Sayar, Kabilesh Rajendran, Van Huyen Dang.

Figure 1
Figure 1. Figure 1: Overview of the proposed Exp2VLA pipeline. Expert policies in Isaac [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The Exp2VLA pipeline consists of three stages: (1) collecting expert demonstrations in Isaac Lab and converting them into the LeRobot format, [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Training metrics for fine-tuning π0.5 on datasets obtained via Exp2VLA over 60k steps (batch size 16, lr = 5e−4 , BF16 precision). The SigLIP vision encoder is frozen and only the Flow Matching action expert is updated. total step count. The training loss as well as the gradient norm for the π0.5 fine-tuning is shown in [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Training environments: (a) single-object dataset (Dataset 1) and (b) [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Measured inference-loop latency for Exp2VLA( [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: The top row (a) presents the SITL pipeline, in which the trained [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 10 linked inside Pith

  1. [1]

    Deep learning for vision-based navigation in autonomous drone racing,

    H. X. Pham, H. I. Ugurlu, J. Le Fevre, D. Bardakci, and E. Kayacan, “Deep learning for vision-based navigation in autonomous drone racing,” inDeep learning for robot perception and cognition. Elsevier, 2022, pp. 371–406

    2022

  2. [2]

    Pen- cilnet: Zero-shot sim-to-real transfer learning for robust gate perception in autonomous drone racing,

    H. X. Pham, A. Sarabakha, M. Odnoshyvkin, and E. Kayacan, “Pen- cilnet: Zero-shot sim-to-real transfer learning for robust gate perception in autonomous drone racing,”IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 11 847–11 854, 2022

    2022

  3. [3]

    Vds-nav: V olumetric depth-based safe navigation for aerial robots–bridging the sim-to-real gap,

    V . H. Dang, A. Redder, H. X. Pham, A. Sarabakha, and E. Kay- acan, “Vds-nav: V olumetric depth-based safe navigation for aerial robots–bridging the sim-to-real gap,”IEEE Robotics and Automation Letters, vol. 10, no. 10, pp. 11 038–11 045, 2025

    2025

  4. [4]

    Navila: Legged robot vision-language- action model for navigation,

    A.-C. Cheng, Y . Ji, Z. Yang, Z. Gongye, X. Zou, J. Kautz, E. Bıyık, H. Yin, S. Liu, and X. Wang, “Navila: Legged robot vision-language- action model for navigation,”arXiv preprint arXiv:2412.04453, 2024

    Pith/arXiv arXiv 2024

  5. [5]

    Mobilevla- r1: Reinforcing vision-language-action for mobile robots,

    T. Huang, D. Li, R. Yang, Z. Zhang, Z. Yang, and H. Tang, “Mobilevla- r1: Reinforcing vision-language-action for mobile robots,”arXiv preprint arXiv:2511.17889, 2025

    arXiv 2025

  6. [6]

    Momanipvla: Trans- ferring vision-language-action models for general mobile manipulation,

    Z. Wu, Y . Zhou, X. Xu, Z. Wang, and H. Yan, “Momanipvla: Trans- ferring vision-language-action models for general mobile manipulation,” inProceedings of the Computer Vision and Pattern Recognition Confer- ence, 2025, pp. 1714–1723

    2025

  7. [7]

    Openvla: An open-source vision-language-action model,

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. P. Foster, P. R. Sanketi, Q. Vuong, T. Kollar, B. Burchfiel, R. Tedrake, D. Sadigh, S. Levine, P. Liang, and C. Finn, “Openvla: An open-source vision-language-action model,” inProceedings of The 8th Conference on Robot Learning, ser. Proceedings of Machine Learning Res...

  8. [8]

    2679–2713

    PMLR, 06–09 Nov 2025, pp. 2679–2713. [Online]. Available: https://proceedings.mlr.press/v270/kim25c.html

    2025

  9. [9]

    Cognitivedrone: Towards vision-language- action aerial robotics,

    Z. Huang, Y . Chenet al., “Cognitivedrone: Towards vision-language- action aerial robotics,”arXiv preprint arXiv:2409.06620, 2024. [Online]. Available: https://arxiv.org/abs/2409.06620

    Pith/arXiv arXiv 2024

  10. [11]

    Available: https://arxiv.org/abs/2502.01181

    [Online]. Available: https://arxiv.org/abs/2502.01181

    Pith/arXiv arXiv

  11. [12]

    Racevla: Vla-based racing drone navigation with human-like behaviour,

    Z. Wanget al., “Racevla: Vla-based racing drone navigation with human-like behaviour,”arXiv preprint arXiv:2503.01245, 2025. [Online]. Available: https://arxiv.org/abs/2503.01245

    Pith/arXiv arXiv 2025

  12. [13]

    Uav-vln: Vision-language navigation for uavs via flight chain-of- thought,

    J. Dong, Y . Bi, D. Ye, Z. Zhao, Y . Chen, C. Xu, K. Wang, Y . Yang, K. Li, L. Wang, D. Lin, Y . Qiao, H. Fei, and Z. Liu, “Uav-vln: Vision-language navigation for uavs via flight chain-of- thought,”arXiv preprint arXiv:2504.21432, 2025. [Online]. Available: https://arxiv.org/abs/2504.21432

    arXiv 2025

  13. [15]

    Available: https://arxiv.org/abs/2506.10756

    [Online]. Available: https://arxiv.org/abs/2506.10756

    Pith/arXiv arXiv

  14. [16]

    Grounded vision-language navigation for aerial robotics in dynamic environments,

    F. Wang, H. Liu, M. Chenet al., “Grounded vision-language navigation for aerial robotics in dynamic environments,”arXiv preprint arXiv:2512.08639, 2025. [Online]. Available: https://arxiv.org/abs/2512 .08639

    Pith/arXiv arXiv 2025

  15. [17]

    Smolvla: A lightweight vision- language-action model for robotic manipulation,

    S. Belkhale, K. Pertschet al., “Smolvla: A lightweight vision- language-action model for robotic manipulation,”arXiv preprint arXiv:2409.18976, 2024. [Online]. Available: https://arxiv.org/abs/2409 .18976

    arXiv 2024

  16. [18]

    π 0.5: A vision-language-action model with open-world generalization,

    K. Black, C. Finnet al., “π 0.5: A vision-language-action model with open-world generalization,”Physical Intelligence Technical Report,

  17. [19]

    Available: https://www.pi.website/download/pi05.pdf

    [Online]. Available: https://www.pi.website/download/pi05.pdf

  18. [20]

    Paligemma: A versatile 3b vision-language model for transfer,

    L. Beyeret al., “Paligemma: A versatile 3b vision-language model for transfer,”arXiv preprint arXiv:2407.07726, 2024

    Pith/arXiv arXiv 2024

  19. [21]

    Libero-pro: Towards robust and fair evaluation of vision-language-action models beyond memorization,

    X. Zhou, A. Zeng, K. Pertschet al., “Libero-pro: Towards robust and fair evaluation of vision-language-action models beyond memorization,”arXiv preprint arXiv:2406.08912, 2024. [Online]. Available: https://arxiv.org/abs/2406.08912

    Pith/arXiv arXiv 2024

  20. [22]

    Prox- imal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox- imal policy optimization algorithms,”arXiv preprint arXiv:1707.06347, 2017

    Pith/arXiv arXiv 2017

  21. [23]

    rl-games: A high-performance framework for reinforcement learning,

    D. Makoviichuk and V . Makoviychuk, “rl-games: A high-performance framework for reinforcement learning,” https://github.com/Denys88/rl games, May 2021

    2021

  22. [24]

    Isaac lab: A gpu- accelerated simulation framework for multi-modal robot learning,

    M. Mittal, P. Roth, J. Tigue, A. Richard, O. Zhang, P. Du, A. Serrano- Mu˜noz, X. Yao, R. Zurbr ¨ugg, N. Rudinet al., “Isaac lab: A gpu- accelerated simulation framework for multi-modal robot learning,”arXiv preprint arXiv:2511.04831, 2025

    Pith/arXiv arXiv 2025

  23. [25]

    Robot learning: A tutorial,

    F. Capuano, C. Pascal, A. Zouitine, T. Wolf, and M. Aractingi, “Robot learning: A tutorial,”arXiv preprint arXiv:2510.12403, 2025

    arXiv 2025

  24. [26]

    Sigmoid loss for language image pre-training (siglip),

    X. Zhai, B. Mustafa, A. Kolesnikov, and L. Beyer, “Sigmoid loss for language image pre-training (siglip),” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023

    2023