Pith. sign in

REVIEW 4 major objections 5 minor 25 references

A frozen 7B vision-language model can drive at 20 Hz when a 337M action module reads its cached memory every tick.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 22:43 UTC pith:KHSAEPKI

load-bearing objection A genuinely useful fast-slow VLA architecture for closed-loop driving, honestly reported, but the core claim that randomized-staleness training closes the asynchronous gap is under-supported: teacher-forced state tokens and an unspecified δmax leave the deployment distribution untested. the 4 major comments →

arxiv 2607.15621 v1 pith:KHSAEPKI submitted 2026-07-17 cs.RO cs.AI

Think at 5 Hz, Act at 20 Hz: Asynchronous Fast-Slow Vision-Language-Action Inference for Closed-Loop Driving

Yun Li , Jiachen Gong , Simon Thompson , Ehsan Javanmardi , Qunli Zhang , Zifan Zeng , Shiming Liu , Peng Wang
show 2 more authors
Zixuan Guo Manabu Tsukada
This is my paper
classification cs.RO cs.AI
keywords fast-slow architecturevision-language-action modelclosed-loop drivingkey-value cacheasynchronous inferencerandomized stalenessCARLAreal-time control
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper is trying to establish that the latency problem of language-guided driving is architectural, not fundamental. It shows that a frozen 7B vision-language backbone can act as a slowly updated context provider—maintaining a per-layer key-value cache of instruction and visual history—while a 337M action expert reads that cache plus the current camera frame at every 50 ms simulation tick to regress fresh waypoints. The central claim is that the cache lag this creates can be absorbed by training the expert under randomized staleness, and the reported result is a route-completion jump from 37.0 to 94.0 over a frame-skipping baseline at a per-tick model cost of 32 ms on a consumer GPU. A sympathetic reader would care because it offers a concrete way to keep a large language model's instruction-following and scene reasoning in the control loop without sacrificing control rate.

Core claim

On the paper's own terms, the discovery is that a frozen 7B vision-language backbone, which never attends back to the expert, can be turned into a cached representation of the scene that a 337M action expert consumes at full control rate. Trained with randomized staleness to match the asynchronous cache lag it will meet at deployment, the expert regresses five waypoints per 50 ms tick and lifts closed-loop route completion on LangAuto-Short from 37.0 to 94.0 over the frame-skipping baseline, transfers zero-shot to unseen towns at 84–94% completion, and reduces open-loop waypoint error by nearly a factor of four, all at a per-tick model cost of 32 ms that does not grow with history length.

What carries the argument

The central object is the per-layer key-value cache of the frozen backbone, maintained by an incremental four-token append every K=4 ticks and rebuilt only on instruction changes, notice arrivals, or episode boundaries. The expert's ten tokens project into the backbone's attention geometry and cross-attend into the cached keys and values at each of 32 layers, while the backbone never attends to the expert, so the cache is identical with or without the expert running. Randomized-staleness training draws a random lag δ for each sample and truncates the backbone prefix visible to the expert, teaching the expert to fuse an outdated scene summary with fresh per-tick evidence.

Load-bearing premise

The load-bearing premise is that randomized-staleness training with teacher-forced previous waypoints closes the train/deploy gap for an expert that at test time reads a cache lagging by up to K ticks and consumes its own previously predicted waypoints; if that gap is not closed, the 20 Hz closed-loop results would not transfer to deployment.

What would settle it

In closed loop, run the same trained expert at 20 Hz but force the backbone cache to be appended only every 8 ticks instead of 4 (or freeze it for the whole episode), keeping everything else identical. If route completion holds near 94.0, randomized-staleness training is not carrying the result; if completion drops sharply, the claim that test-time staleness must stay inside the trained range is confirmed.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Fresh control at every simulation tick is achievable for a language-conditioned driving agent on a single consumer GPU; the rate mismatch is an architectural choice, not an invariant of large vision-language models.
  • The route-completion gain and the reductions in red-light violations, route deviations, and timeouts come from per-tick freshness rather than from the expert's waypoint quality alone, as the frame-skip ablation shows.
  • Zero-shot transfer to unseen towns, holding 84–94% route completion where the baseline reaches 31–41%, indicates the expert learns to read the backbone's cache and the current frame rather than memorizing a town's geometry.
  • Incremental cache appending is numerically faithful to full recomputation (waypoint differences below 4 mm), so cache reuse preserves the backbone's representation.
  • Because the per-tick cost is independent of history length, the approach keeps the 50 ms tick budget even as the agent's visual memory grows.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • We infer that the asymmetric attention design (backbone never attends to expert) makes the cache a reusable interface: the same cached scene representation could feed multiple fast modules—such as a safety monitor or a different action head—without extra backbone cost, a property the paper notes but does not exploit.
  • We infer that randomized-staleness training is a general recipe for any policy that consumes a slowly updated latent context alongside fast sensor input, not just driving; robot manipulation with a slow VLM planner and a fast low-level policy is a natural next application.
  • We infer that the paper's own long-route result (85% completion but collapsed infraction score) suggests the fast path's next bottleneck is hazard negotiation under dense traffic, pointing to training on longer, signalized clips rather than a change in architecture.
  • We infer that the measured 32 ms per-tick cost on a consumer GPU leaves headroom for quantization or distillation of the expert, which could be tested by checking whether the 20 Hz cadence and completion gains survive on lower-cost hardware.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes an asynchronous fast-slow vision-language-action architecture for closed-loop driving. A frozen 7B LMDrive backbone is treated as a slow context provider: it incrementally appends visual tokens to its per-layer KV cache every K=4 ticks (5 Hz), and the cached representation is reused by a 337M action expert that runs at every 50 ms simulation tick (20 Hz). The expert cross-attends into the cached backbone keys/values at all 32 layers, plus the current frame and a state token encoding previous waypoints, previous control, speed, and target point. To match deployment, training uses randomized staleness: for each supervised frame, the backbone prefix is masked to end δ frames earlier, with δ drawn uniformly from {0,...,δmax}. The paper reports open-loop waypoint L1 improvements (0.123 → 0.031), a golden test for cache equivalence (waypoint drift < 4 mm), CARLA LangAuto-Short closed-loop results (route completion 37.0 → 94.0 over the frame-skipping LMDrive baseline, and 82.1 → 94.0 when moving the same expert from 10 Hz to 20 Hz), zero-shot transfer to unseen towns (84–94% completion), and a flat per-tick model cost of about 32 ms independent of history length. The central claim is that per-tick freshness, affordable only through cached context, is what drives the route-completion gain, while the expert itself drives the driving-score gain.

Significance. If the claims hold, this is a useful architectural contribution: it shows that a large frozen VLA backbone can be repurposed as an incrementally updated cache while a small expert produces fresh control at a higher rate, directly addressing the inference-latency bottleneck of language-conditioned driving agents. The paper has several concrete strengths: the cache-equivalence golden test (< 4 mm) is a machine-checkable safeguard; the evaluation uses a public LMDrive checkpoint and a standard CARLA benchmark; the architecture leaves the backbone frozen and avoids backprop through 7B parameters; and the randomized-staleness training idea is a clean, falsifiable mechanism for aligning training with asynchronous deployment. The result is potentially significant for closed-loop VLA systems because it offers a way to separate expensive semantic/temporal reasoning from fast reactive control without retraining the backbone. However, the empirical evidence is currently too thin to fully support the headline numbers: the key freshness isolation row is a single run, transfer results are single-run, the staleness bound δmax is unspecified, and the exposure-bias gap between teacher-forced traini

major comments (4)
  1. [§3.2 and §3.3] The value of δmax is never specified. At deployment the cache lags by up to K−1 = 3 ticks, and §3.3 asserts that this 'stays inside the training distribution,' but without δmax this containment cannot be verified: if δmax < 3, the deployed staleness is out of distribution; if δmax is much larger, the training distribution is much broader than deployment. The authors should report δmax and ideally show sensitivity to the chosen δmax, including a closed-loop sweep over K or over test-time staleness. This is load-bearing because randomized staleness is the mechanism claimed to align training with asynchronous execution.
  2. [§4.3, Table 2] The key freshness isolation — 10 Hz frame-skipped vs 20 Hz per-tick, raising route completion from 82.1 to 94.0 and cutting red-light violations — rests on a single run for the 10 Hz row, and the transfer table (Table 4) also has single runs per agent and town. The two-run baseline and ours give only half-range error bars, so the 82.1→94.0 difference has no variance estimate. Given that the headline abstract claim is built on this comparison, at least 5–10 independent runs per configuration (or an explicit statement of computational limits and a bootstrap analysis) are needed before the per-tick-freshness effect can be considered robust.
  3. [§3.2 and §4.2] The train/deploy gap is asserted, not demonstrated. During training the state token is teacher-forced with ground-truth previous waypoints under Gaussian noise and random dropout; at deployment the expert consumes its own previous predictions produced under a potentially stale cache. The open-loop L1 improvements and the randomized-δ ablation are evaluated under the synchronous δ=0 protocol with teacher forcing, so they do not directly test either self-feeding or cache lag. The paper's Limitations section flags small run counts and long-route hazards but not this exposure-bias gap. A targeted closed-loop ablation (e.g., feeding ground-truth state tokens vs self-generated state tokens at test time, or varying test-time K over a range and measuring route completion) would substantially strengthen the claim that staleness-augmented training transfers to asynchronous deployment.
  4. [§4.4] There is a tension between the title/abstract claim of 'Act at 20 Hz' and the measured latencies. The median end-to-end agent step is 58 ms and the wall-clock rate is about 17 Hz, while the per-tick model compute is 32 ms. The text says that in CARLA synchronous mode the vehicle still receives fresh control at every tick, but if the agent step takes 58 ms it cannot sustain true 50 ms ticks. The authors should clarify whether the simulator tick is slowed by the agent computation, and report the actual control rate at deployment. The 20 Hz framing is central to the paper, so this must be made precise.
minor comments (5)
  1. [§4.3] The phrase 'Our infraction score is lower' is ambiguous or inverted: in CARLA, a higher infraction score is better. The authors appear to mean that the infraction score is worse (0.37 vs 0.80). Please state the direction explicitly.
  2. [Figure 5] The legend for the randomized-δ curve appears as 'randomized (ours)' with a missing δ symbol; please fix the typography so the notation matches §3.2.
  3. [§4.2] The authors acknowledge that the open-loop gap partly reflects the privileged state-token signal. This is good, but the paper should also report open-loop performance with self-generated (non-teacher-forced) state tokens, since the closed-loop comparison is the only place this is checked.
  4. [Table 1 and Table 2] The captions mention 'T able' with a space; please correct the table formatting. Also, the units for per-kilometer infractions in §4.3 should be stated consistently (per kilometer is mentioned in prose, but not in the table).
  5. [General] No code or trained expert weights are provided. Releasing the expert and evaluation scripts would materially improve reproducibility, especially given the small number of runs and the dependence on the public LMDrive checkpoint.

Circularity Check

0 steps flagged

No significant circularity: main results are measured in simulation against a public checkpoint, and no prediction reduces to a fitted constant or self-citation.

full rationale

The paper's central claims are empirical rather than derived from fitted inputs. Route completion, driving score, infraction score, and latency are measured in the CARLA simulator using the public LMDrive checkpoint, with unseen-town transfer providing external grounding. Randomized-staleness training is a training-time augmentation: it changes how the expert is trained, but it does not fit the evaluation metric, and the δ=0 ablation compares two training variants under the same evaluation protocol. The frame-skip row uses the same trained expert at two control rates, so the 20 Hz vs 10 Hz comparison is a measured behavioral difference rather than a quantity forced by construction. The cache-equivalence check (monolithic vs incremental prefill differing by <4 mm) is an independent numeric validation that the cache faithfully reproduces the backbone states. The only mild self-reference is the citation of DriveVLA-W0 [8] for the design choice of learned positional embeddings and no rotary encoding; even if that citation overlaps with the present authors, it is not load-bearing for the paper's main closed-loop results, which are measured against an external baseline. The unspecified δmax and the assertion that test-time staleness 'stays inside the training distribution' are robustness/correctness concerns, not circularity, because they do not make any measured outcome equal to an input by construction.

Axiom & Free-Parameter Ledger

2 free parameters · 5 axioms · 0 invented entities

The paper introduces no new physical entities, but it does rely on a set of architectural and empirical assumptions. Two free parameters (K and the undisclosed δmax) directly affect the timing and staleness guarantees. The most fragile assumption is that randomized-staleness training plus teacher forcing is sufficient for closed-loop operation, which is not directly tested.

free parameters (2)
  • K (cache append interval / max deployment staleness) = 4 ticks
    Chosen by hand in §3.3. It trades cache maintenance cost against maximum staleness and defines the 5 Hz / 20 Hz split. Changing K would change the latency budget and the staleness distribution that training must cover.
  • δmax (upper bound of randomized staleness) = not reported
    Introduced in §3.2 as the support of U{0,...,δmax} for staleness augmentation. The paper never gives its value; if δmax is smaller than the deployment lag bound K−1, training does not cover test-time staleness. This is a load-bearing, undisclosed hyperparameter.
axioms (5)
  • standard math Append-only KV caching is numerically exact for a causal transformer with rotary position embeddings.
    Invoked in §3.3 ('Rotary phases make append-only caching exact') and checked with a golden test. The entire asynchronous schedule depends on this equivalence; if it failed, the cached context would not match the frozen backbone's own representation.
  • domain assumption The frozen LMDrive perception encoder, Q-Former, and LLaMA-7B backbone provide a sufficient representation of instruction and visual history for control.
    The paper inherits this from prior work (§3) and does not validate the backbone's representation quality beyond waypoint regression. The action expert is entirely dependent on the cached backbone features.
  • domain assumption The CARLA LangAuto benchmark, 32 short routes in town05, with the reuse of LMDrive's PID controller, is a valid closed-loop proxy for driving performance.
    The closed-loop results are measured in this specific simulator configuration (§4.1). The paper acknowledges that simulation is not road readiness, but the central quantitative claims are only about this benchmark.
  • domain assumption A 337M expert consuming ten tokens per frame (state token, 4 visual tokens, 5 waypoint queries) can regress safe waypoints at 20 Hz.
    The architecture (§3.1) fixes this input/output interface. The per-tick latency and waypoint accuracy claims both rest on this modeling choice; no architecture search or comparison is provided.
  • ad hoc to paper Randomized staleness during training with teacher-forced state tokens transfers to closed-loop deployment where the expert sees only its own predictions.
    This is the central training/deployment alignment assumption in §3.2. It is supported only indirectly by the open-loop ablation and the single frame-skip run, not by a dedicated closed-loop staleness robustness experiment.

pith-pipeline@v1.3.0-alltime-deepseek · 9971 in / 8968 out tokens · 96736 ms · 2026-08-01T22:43:20.690327+00:00 · methodology

0 comments
read the original abstract

Large language models bring instruction following and scene reasoning to end-to-end driving, but their inference latency collides with the control rate a vehicle requires. Existing closed-loop agents hide this gap by invoking the model on alternate simulation ticks and replaying the previous command in between, so half of all control outputs ignore the newest observations. We present a fast-slow architecture that removes this compromise. A frozen 7B vision-language backbone acts as the slow system, digesting navigation instructions and visual history at low frequency while exposing its per-layer key-value cache as a standing representation of the scene. A lightweight action expert acts as the fast system, attending to this cache and to the current camera frame at every simulation tick to regress waypoints in a single forward pass. Since the cache lags behind the world at deployment, we train the expert under randomized staleness, aligning training with asynchronous execution. On LangAuto-Short routes in CARLA, our system produces fresh control at every 50 ms simulation tick and lifts route completion from 37.0 to 94.0 over the frame-skipping baseline. A frame-skip ablation with the same expert separates the two factors at work: the expert raises the driving score on its own, while per-tick freshness raises completion from 82.1 to 94.0 and cuts red-light violations by a third. Trained on a single town, the expert transfers zero-shot to two unseen towns, holding 84-94% route completion where the baseline reaches 31-41%. It reduces open-loop waypoint error by nearly a factor of four compared to the backbone's own action head, at a per-tick model cost of 32 ms that is independent of history length on a single consumer GPU.

Figures

Figures reproduced from arXiv: 2607.15621 by Ehsan Javanmardi, Jiachen Gong, Manabu Tsukada, Peng Wang, Qunli Zhang, Shiming Liu, Simon Thompson, Yun Li, Zifan Zeng, Zixuan Guo.

Figure 1
Figure 1. Figure 1: One second of control under the two schedules, at the measured per-stage laten￾cies of [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Asynchronous fast-slow inference. The frozen backbone (left) digests instruction and history every K ticks, appending four tokens per frame to its per-layer KV cache through an incremental forward pass; the cache is rebuilt only on invalidation events. The action expert (right) runs at every simulation tick: its ten tokens cross-attend into the cached keys and values at each of the 32 layers and regress fi… view at source ↗
Figure 3
Figure 3. Figure 3: The four evaluation towns. The expert is trained on short routes from town05 only; towns 01, 02, and 03 are never seen during expert training, and town03 is evalu￾ated on the long-route tier. 3.2 Staleness-Augmented Training At deployment the cache lags the current frame by up to K ticks. We replicate this gap during training by drawing δ ∼ U {0, . . . , δmax} for each sample and masking the backbone prefi… view at source ↗
Figure 4
Figure 4. Figure 4: Closed-loop driving in town05. Each panel shows the forward view with side views, the active navigation instruction, the runtime notice, and the predicted way￾points; the bottom-right panel brakes for a red light flagged by a notice. 11.2; the expert completes routes through traffic it has not learned to negotiate safely, a trade-off we revisit in the transfer experiments and the limitations. Infractions t… view at source ↗
Figure 5
Figure 5. Figure 5: Left: per-step model latency versus history length. The legacy path re-encodes the full history each step and exceeds the 50 ms tick budget at every history length, while our per-tick cost stays flat at 32 ms. Right: validation waypoint L1 across training epochs; the randomized-δ expert leads its δ=0 twin at every epoch and both leave the frozen backbone head far behind. only at instruction changes, notice… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

25 extracted references · 2 canonical work pages · 2 internal anchors

  1. [1]

    In: Robotics: Science and Systems 2025 (21 Jun 2025).https://doi.org/ 10.48550/arXiv.2410.24164,http://dx.doi.org/10.48550/arXiv.2410.24164 2, 4, 5

    Black, K., Brown, N., Driess, D., Esmail, A., Equi, M., Finn, C., Fusai, N., Groom, L., Hausman, K., Ichter, B., Jakubczak, S., Jones, T., Ke, L., Levine, S., Li-Bell, A., Mothukuri, M., Nair, S., Pertsch, K., Shi, L.X., Tanner, J., Vuong, Q., Walling, A., Wang, H., Zhilinsky, U.:π0: A vision-language-action flow model for general robot control. In: Robot...

    Pith/arXiv arXiv doi:10.48550/arxiv.2410.24164 2025

  2. [2]

    In: Robotics: Science and Systems XIX

    Chi, C., Feng, S., Du, Y., Xu, Z., Cousineau, E., Burchfiel, B., Song, S.: Diffusion policy: Visuomotor policy learning via action diffusion. In: Robotics: Science and Systems XIX. Robotics: Science and Systems Foundation (10 Jul 2023).https: //doi.org/10.15607/rss.2023.xix.026,http://dx.doi.org/10.15607/rss. 2023.xix.0264

    doi:10.15607/rss.2023.xix.026 2023

  3. [3]

    IEEE Trans

    Chitta, K., Prakash, A., Jaeger, B., Yu, Z., Renz, K., Geiger, A.: TransFuser: Imitation with transformer-based sensor fusion for autonomous driving. IEEE Trans. Pattern Anal. Mach. Intell.45(11), 12878–12895 (Nov 2023).https: //doi.org/10.1109/TPAMI.2022.3200245,http://dx.doi.org/10.1109/TPAMI. 2022.32002454

    arXiv 2023

  4. [4]

    In: Levine, S., Vanhoucke, V., Goldberg, K

    Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., Koltun, V.: CARLA: An Open Urban Driving Simulator. In: Levine, S., Vanhoucke, V., Goldberg, K. (eds.) Pro- ceedings of the 1st Annual Conference on Robot Learning. Proceedings of Machine Learning Research, vol. 78, pp. 1–16. PMLR (2017),https://proceedings.mlr. press/v78/dosovitskiy17a.html3 12 Y. Li et al

    2017

  5. [5]

    Transactions on Machine Learning Re- search (07/2025) (23 Jul 2025).https://doi.org/10.48550/arXiv.2410.23262, https://openreview.net/forum?id=kH3t5lmOU84

    Hwang, J.J., Xu, R., Lin, H., Hung, W.C., Ji, J., Choi, K., Huang, D., He, T., Cov- ington, P., Sapp, B., Zhou, Y., Guo, J., Anguelov, D., Tan, M.: EMMA: End-to-end Multimodal Model for autonomous driving. Transactions on Machine Learning Re- search (07/2025) (23 Jul 2025).https://doi.org/10.48550/arXiv.2410.23262, https://openreview.net/forum?id=kH3t5lmOU84

    Pith/arXiv arXiv doi:10.48550/arxiv.2410.23262 2025

  6. [6]

    IEEE Trans

    Katsikopoulos, K.V., Engelbrecht, S.E.: Markov decision processes with delays and asynchronous cost collection. IEEE Trans. Automat. Contr.48(4), 568–574 (Apr 2003).https://doi.org/10.1109/tac.2003.809799,http://dx.doi.org/10. 1109/TAC.2003.8097995

    arXiv 2003

  7. [7]

    In: International Conference on Machine Learning

    Li, J., Li, D., Savarese, S., Hoi, S.: BLIP-2: Bootstrapping language-image pre- training with frozen image encoders and large language models. In: International Conference on Machine Learning. arXiv (2023).https://doi.org/10.48550/ ARXIV.2301.125975

    Pith/arXiv arXiv doi:10.48550/arxiv.2301.12597 2023

  8. [8]

    In: The Fourteenth International Conference on Learning Representations (18 Dec 2025).https: / / doi

    Li, Y., Shang, S., Liu, W., Zhan, B., Wang, H., Wang, Y., Chen, Y., Wang, X., An, Y., Tang, C., Hou, L., Fan, L., Zhang, Z.: DriveVLA-W0: World models amplify data scaling law in autonomous driving. In: The Fourteenth International Conference on Learning Representations (18 Dec 2025).https: / / doi . org / 10 . 48550 / arXiv . 2510 . 12796,https : / / ope...

    2025

  9. [9]

    In: NeurIPS 2023 Foundation Models for Decision Making Workshop (2 Oct 2023),http://arxiv.org/abs/2310.014154

    Mao, J., Qian, Y., Ye, J., Zhao, H., Wang, Y.: GPT-Driver: Learning to Drive with GPT. In: NeurIPS 2023 Foundation Models for Decision Making Workshop (2 Oct 2023),http://arxiv.org/abs/2310.014154

    Pith/arXiv arXiv 2023

  10. [10]

    In: Conference on Language Modeling (7 Oct 2024) 4

    Mao, J., Ye, J., Qian, Y., Pavone, M., Wang, Y.: A language agent for autonomous driving. In: Conference on Language Modeling (7 Oct 2024) 4

    2024

  11. [11]

    arXiv [cs.LG] (22 Apr 2025).https://doi.org/10.48550/ arXiv.2504.16054,http://arxiv.org/abs/2504.160544

    Physical Intelligence, Black, K., Brown, N., Darpinian, J., Dhabalia, K., Driess, D., Esmail, A., Equi, M., Finn, C., Fusai, N., Galliker, M.Y., Ghosh, D., Groom, L., Hausman, K., Ichter, B., Jakubczak, S., Jones, T., Ke, L., LeBlanc, D., Levine, S., Li-Bell, A., Mothukuri, M., Nair, S., Pertsch, K., Ren, A.Z., Shi, L.X., Smith, L., Springenberg, J.T., St...

    Pith/arXiv arXiv doi:10.48550/arxiv.2504.16054 2025

  12. [12]

    In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Prakash, A., Chitta, K., Geiger, A.: Multi-modal fusion transformer for end-to-end autonomous driving. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 7073–7083. IEEE (Jun 2021).https://doi. org/10.1109/cvpr46437.2021.00700,http://dx.doi.org/10.1109/CVPR46437. 2021.007004

    arXiv 2021

  13. [13]

    Real-Time Reinforcement Learning

    Ramstedt, S., Pal, C.: Real-Time Reinforcement Learning. arXiv [cs.LG] (11 Nov 2019).https://doi.org/10.48550/arXiv.1911.04448,http://arxiv.org/abs/ 1911.044485

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.1911.04448 2019

  14. [14]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Renz, K., Chen, L., Arani, E., Sinavski, O.: SimLingo: Vision-Only Closed- Loop Autonomous Driving with Language-Action Alignment. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11993–12003. openaccess.thecvf.com (2025),http://openaccess.thecvf. com / content / CVPR2025 / html / Renz _ SimLingo _ Vision - Only _...

    2025

  15. [15]

    In: 2024 Conference on Computer Vision and Pattern Recog- nition (14 Jun 2024) 4

    Renz, K., Chen,L., Marcu,A.M.,Hünermann, J.,Hanotte, B.,Karnsund, A., Shot- ton,J.,Arani,E.,Sinavski,O.:CarLLaVA:Visionlanguagemodelsforcamera-only Asynchronous Fast-Slow VLA Inference for Closed-Loop Driving 13 closed-loop driving. In: 2024 Conference on Computer Vision and Pattern Recog- nition (14 Jun 2024) 4

    2024

  16. [16]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2024), http://arxiv.org/abs/2312.074882, 4, 5

    Shao, H., Hu, Y., Wang, L., Waslander, S.L., Liu, Y., Li, H.: LMDrive: Closed- Loop End-to-End Driving with Large Language Models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2024), http://arxiv.org/abs/2312.074882, 4, 5

    Pith/arXiv arXiv 2024

  17. [17]

    In: Liu, K., Kulic, D., Ich- nowski, J

    Shao, H., Wang, L., Chen, R., Li, H., Liu, Y.: Safety-Enhanced Autonomous Driv- ing Using Interpretable Sensor Fusion Transformer. In: Liu, K., Kulic, D., Ich- nowski, J. (eds.) Proceedings of The 6th Conference on Robot Learning. Pro- ceedings of Machine Learning Research, vol. 205, pp. 726–737. PMLR (2023), https://proceedings.mlr.press/v205/shao23a.html4

    2023

  18. [18]

    ReasonNet: End-to-End Driving with Temporal and Global Reasoning

    Shao, H., Wang, L., Chen, R., Waslander, S.L., Li, H., Liu, Y.: ReasonNet: End- to-end driving with temporal and global reasoning. arXiv [cs.CV] (17 May 2023). https://doi.org/10.48550/arXiv.2305.10507,http://arxiv.org/abs/2305. 105074

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2305.10507 2023

  19. [19]

    In: European Conference on Computer Vision 2024 (29 Sep 2024) 2, 4

    Sima, C., Renz, K., Chitta, K., Chen, L., Zhang, H., Xie, C., Luo, P., Geiger, A., Li, H.: DriveLM: Driving with graph visual question answering. In: European Conference on Computer Vision 2024 (29 Sep 2024) 2, 4

    2024

  20. [20]

    In: 8th Annual Conference on Robot Learning (5 Sep 2024), https://openreview.net/pdf?id=928V4Umlys2, 4

    Tian, X., Gu, J., Li, B., Liu, Y., Wang, Y., Zhao, Z., Zhan, K., Jia, P., Lang, X., Zhao, H.: DriveVLM: The Convergence of Autonomous Driving and Large Vision- Language Models. In: 8th Annual Conference on Robot Learning (5 Sep 2024), https://openreview.net/pdf?id=928V4Umlys2, 4

    2024

  21. [21]

    arXiv [cs.CL] (27 Feb 2023),http://arxiv.org/abs/2302.139715

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., Lample, G.: LLaMA: Open and efficient foundation language models. arXiv [cs.CL] (27 Feb 2023),http://arxiv.org/abs/2302.139715

    Pith/arXiv arXiv 2023

  22. [22]

    In: Advances in Neural Information Processing Systems 35

    Wu, P., Jia, X., Chen, L., Yan, J., Li, H., Qiao, Y.: Trajectory-guided control prediction for end-to-end autonomous driving: A simple yet strong baseline. In: Advances in Neural Information Processing Systems 35. pp. 6119–6132. Neural In- formation Processing Systems Foundation, Inc. (NeurIPS), San Diego, California, USA (2022).https://doi.org/10.52202/0...

    arXiv 2022

  23. [23]

    In: International Conference on Learning Representations (25 Sep 2019),https: //openreview.net/forum?id=SJexHkSFPS5

    Xiao, T., Jang, E., Kalashnikov, D., Levine, S., Ibarz, J., Hausman, K., Herzog, A.: Thinking While Moving: Deep Reinforcement Learning with Concurrent Control. In: International Conference on Learning Representations (25 Sep 2019),https: //openreview.net/forum?id=SJexHkSFPS5

    2019

  24. [24]

    IEEE Robot

    Xu, Z., Zhang, Y., Xie, E., Zhao, Z., Guo, Y., Wong, K.Y.K., Li, Z., Zhao, H.: DriveGPT4: Interpretable end-to-end autonomous driving via large language model. IEEE Robot. Autom. Lett.9(10), 8186–8193 (Oct 2024).https://doi. org/10.1109/lra.2024.34400972, 4

    arXiv 2024

  25. [25]

    arXiv [cs.RO] (23 Apr 2023).https://doi

    Zhao, T.Z., Kumar, V., Levine, S., Finn, C.: Learning fine-grained bimanual ma- nipulation with low-cost hardware. arXiv [cs.RO] (23 Apr 2023).https://doi. org/10.48550/arXiv.2304.13705,http://arxiv.org/abs/2304.137054

    Pith/arXiv arXiv doi:10.48550/arxiv.2304.13705 2023