Pith. sign in

REVIEW 2 major objections 4 minor 43 references

LADY proposes the first fully linear attention model for end-to-end autonomous driving, aiming to fuse arbitrarily long sensor history at constant inference cost.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 06:29 UTC pith:PKFUFXBX

load-bearing objection A credible engineering contribution with a real constant-time inference story, but the memory claim needs fixing and LICA deserves an ablation. the 2 major comments →

arxiv 2512.15038 v3 pith:PKFUFXBX submitted 2025-12-17 cs.AI

LADY: Linear Attention for Autonomous Driving Efficiency without Transformers

Jihao Huang , Xi Xia , Zhiyuan Li , Tianle Liu , Jingke Wang , Junbo Chen , Tengju Ye This is my paper
classification cs.AI
keywords end-to-end autonomous drivinglinear attentionlinear cross-attentiontemporal context fusiondiffusion policytrajectory planningedge deploymentmulti-modal trajectories
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

End-to-end driving systems today lean on attention blocks whose cost grows quadratically with the number of tokens, which makes long multi-frame history and on-vehicle deployment hard. This paper claims that a fully linear attention stack—where a fixed-size recurrent state carries all past camera and LiDAR information—can replace every transformer component in a driving planner, including decoder cross-attention, without sacrificing planning quality. To keep the whole pipeline linear, the authors introduce LICA, a lightweight cross-attention built from two causal passes and a concatenation trick, and pair it with a diffusion-based trajectory decoder. On standard driving benchmarks the model is competitive with transformer-based systems, and on the closed-loop benchmark an unbounded-history variant matches top baselines; at the same time, inference latency and memory stay flat as the number of past frames grows. If the claim holds, long-horizon temporal context becomes nearly free at run time, which is exactly the setting where today's methods are most taxed.

Core claim

LADY's central claim is that a fully linear attention architecture is sufficient for end-to-end planning: a recurrent block with a delta-rule hidden-state update compresses multi-frame camera and LiDAR tokens causally, and at inference each new frame is folded into a fixed-size state, so time and memory per step are constant regardless of how many frames have come before. The paper's own mechanism, LICA, performs cross-attention without a quadratic softmax: learnable query tokens are first encoded by one recurrent pass, concatenated after the fused bird's-eye-view and ego-feature sequence, and run through a second recurrent pass; the last query positions become the cross-attended queries tha

What carries the argument

Two pieces of machinery carry the argument. The backbone is a recurrent linear-attention block with a delta-rule state update: it processes token sequences causally, compressing all past tokens into a fixed-size hidden state, so the per-token cost is constant in sequence length. The paper's own contribution is LICA (lightweight linear cross-attention): to let decoder queries attend to fused scene features, the queries are first encoded by one recurrent block, then concatenated after the BEV-ego token sequence and fed through a second recurrent block; the last query-position outputs are the cross-attended queries. This converts the typical quadratic query-to-feature softmax into a single caus

Load-bearing premise

The design's success hinges on assuming that a single causal pass over the concatenated feature-and-query sequence passes all the information a bidirectional cross-attention would, even though features never look back at the queries.

What would settle it

Benchmark an identical model whose decoder cross-attention is replaced by standard quadratic attention with the same parameter count and training budget: if the quadratic version raises the planner's score on the open-loop benchmark by more than a noise-level margin, the causal design is losing information; if it does not, the linear shortcut is sufficient.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Long sensor history becomes nearly free at inference: a vehicle can keep a running state from every frame it has seen, so behaviors that unfold over many seconds can inform the plan at constant marginal cost.
  • The same model fits on embedded vehicle computers, where transformer planners with long input windows would exceed real-time budgets.
  • The quality of the generated candidate trajectories is already near-human; a better learned scorer alone could lift the model past state-of-the-art results without changing the planner.
  • The framework's linear-attention backbone can be swapped for other recurrent linear-attention layers, so the efficiency property is not tied to one specific implementation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The causal, one-directional nature of LICA means features never directly see the queries; if bidirectional feature-to-query information proves necessary in tricky scenes, the constant-cost property may force a compromise. An ablation replacing LICA with quadratic cross-attention would quantify this.
  • The O(1) guarantee is an inference-time property; training still processes a fixed-length window in parallel, so the asymptotic advantage applies to deployment, not to the cost of fitting the model.
  • The 'infinite frames' variant effectively turns the driving model into a learned state-space memory; this suggests a connection to memory-augmented world models, where the hidden state could also be used for prediction and surprise detection beyond planning.
  • Since the best-of-N already exceeds human PDMS, the scoring head, not the trajectory generator, is the current ceiling; training the scorer on richer rule-based labels or closed-loop outcomes could unlock SOTA with this exact architecture.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. LADY proposes a fully linear-attention end-to-end autonomous driving model that replaces transformer attention with RWKV-7 linear attention for multi-frame camera/LiDAR fusion, and introduces LICA, a causal RWKV-7-based linear cross-attention mechanism inside a diffusion-based trajectory decoder. The paper claims constant per-frame inference cost and memory independent of history length, reports a NAVSIM PDMS of 90.9, closed-loop Bench2Drive results showing the benefit of temporal context, and validates deployment on an NVIDIA Jetson edge device.

Significance. If the efficiency and accuracy claims hold, the contribution is useful: it extends modern linear-attention sequence models to cross-modal end-to-end planning and demonstrates a constant per-frame inference overhead as historical context grows. The paper provides direct benchmark measurements, a temporal-context ablation, and edge-device validation, which are concrete strengths. However, the headline complexity claim is overstated (the recurrent state is d×d, not O(d)), and the LICA design lacks an ablation against standard cross-attention, so the central claim that linear cross-attention is an effective replacement is not yet fully supported.

major comments (2)
  1. [Table I, Eq. (1), Abstract] The memory complexity claim is misstated. Equation (1) defines S_t as a d×d matrix, so the recurrent temporal state requires O(d^2) memory per frame, not O(d) as listed in Table I. The abstract's 'O(1) regardless of the history length' is only true with respect to the history length T, not as an absolute constant. The time column O(Td) is a total cost over T frames, not a per-frame cost; the claimed constant per-frame cost should be stated as O(d^2) per step independent of T. Please correct the table and abstract accordingly.
  2. [Sec. III-B, Fig. 4] LICA is a single causal pass: feature tokens are processed first, their outputs are discarded, and query tokens never affect the feature representation. The last M query outputs see feature information only through the fixed d×d recurrent state. The statement that the state 'naturally attend[s] to both earlier feature tokens and later query tokens' is misleading for the extracted query outputs. Since LICA is a core contribution supporting 'effective cross-modal interaction', the paper needs an ablation comparing LICA against standard transformer cross-attention (or a bidirectional linear attention variant) to quantify any accuracy loss. Without such an ablation, the fully-linear design's accuracy claim is not fully supported.
minor comments (4)
  1. [Sec. III-A, Eq. (1), parallel form] The text says 'the rows of S∈R^{B×d} are the hidden states S_0,...,S_{B-1}', but S_t is d×d. The dimensions in this sentence are inconsistent and should be clarified.
  2. [Table III] The frame-number row '15' is typeset as '1598.0', merging the frame count and the numeric score; please fix the formatting.
  3. [Table I caption] 'start-of-the-art' should be 'state-of-the-art'; also, the footnote notation 'MT b' should be expanded and the table footnote numbering cleaned up.
  4. [Sec. II-B] The sentence on Mamba-2's decay, 'resulting them becoming a smaller proportion of the numerically increasing state', is grammatically unclear and should be rephrased.

Circularity Check

0 steps flagged

No significant circularity; the only author-overlapping citation (FusionAD) is background, and all load-bearing results are measured or follow from externally supported RWKV-7 equations.

full rationale

LADY's central results are direct empirical measurements on NAVSIM and Bench2Drive (Tables I and II), not derivations from fitted inputs. The claimed constant-time, constant-memory inference follows from the cited RWKV-7 recurrence (Eqs. 1–4), which is imported from the external FLA library and the RWKV-7 paper [29,40], so the linear-attention machinery is independently supported. LICA (Sec. III-B, Fig. 4) is an architectural construction—concatenating BEV-ego and query tokens in a causal RWKV-7 pass and extracting the last M outputs—not a renamed known result or a parameter fitted to a subset and then called a prediction. The absence of an ablation against standard cross-attention and the acknowledged scorer limitation (Sec. VII-G) are correctness/evidence concerns, not circularity. The only self-citation is FusionAD [17], listed among regression-based approaches; no load-bearing argument depends on it. Under the required quote-and-reduction standard, I cannot exhibit a circular step; the score reflects only the minor, non-load-bearing self-citation.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 0 invented entities

The paper contributes no new physical entities or fitted constants; its central claims are architectural and empirical. The free parameters are hand-chosen hyperparameters that influence the reported benchmark numbers, and the axioms are standard machine-learning assumptions plus the specific LICA design assumption.

free parameters (7)
  • History length for training = 10 frames
    Chosen by hand; Table III shows performance varies with frame count, so the 10-frame setting directly affects reported results.
  • Number of anchor trajectories = 100
    Inherited from DiffusionDrive-style diffusion decoder; affects multi-modal trajectory diversity and computation.
  • Diffusion schedule truncation = 50 of 1000 steps
    Truncated diffusion schedule chosen to reduce training cost; affects trajectory quality.
  • Number of denoising steps at inference = 2
    Inference-time setting that trades quality for latency; directly affects planning results.
  • Number of diffusion decoder layers = 2
    Cascaded decoder depth chosen by hand; affects model capacity and runtime.
  • Learning rate and schedule = 1e-4, ReduceLROnPlateau
    Standard optimization hyperparameters selected for training; influence final PDMS.
  • Training epochs and batch size = 100 epochs, batch size 4
    Hand-picked training budget; results may vary with these choices.
axioms (5)
  • domain assumption The RWKV-7 delta-rule state update in Eq. (1) correctly implements linear attention and compresses all needed historical information into the d×d state S_t.
    The paper imports RWKV-7 from [29] and relies on it for the constant-cost and constant-memory claims. If the recurrence loses safety-critical temporal information, the central efficiency-accuracy tradeoff changes.
  • ad hoc to paper LICA's causal concatenation of queries after feature tokens is a sufficient cross-attention substitute.
    Sec. III-B assumes queries attending to earlier feature tokens via the recurrent state is equivalent to bidirectional cross-attention; feature tokens never attend to queries, and this asymmetry is not analyzed.
  • domain assumption NAVSIM PDMS and Bench2Drive metrics are reliable proxies for real-world driving safety and performance.
    Open-loop NAVSIM evaluation is non-reactive, and closed-loop CARLA Bench2Drive is simulated; both may not fully capture real-world edge cases.
  • domain assumption A fixed-size recurrent state can store long-range temporal context without needing to revisit past frames.
    The O(1) temporal fusion claim assumes lossless-enough compression of an unbounded history into a d×d matrix, which is the core of the efficiency claim.
  • domain assumption The FLA library [40] implements RWKV-7 faithfully and efficiently.
    The paper cites FLA for parallel training and inference; if the implementation deviates from Eq. (1), the measured latency and memory behavior may differ.

pith-pipeline@v1.3.0-alltime-deepseek · 12682 in / 12293 out tokens · 118281 ms · 2026-08-04T06:29:18.597986+00:00 · methodology

0 comments
read the original abstract

End-to-end autonomous driving has emerged as a promising paradigm. However, state-of-the-art methods rely heavily on Transformer architectures. The inherent quadratic complexity of Transformers restricts their ability to model long-range spatial and temporal dependencies, particularly on resource-constrained edge platforms. Given the inherent demand for efficient temporal modeling in autonomous driving, this computational bottleneck severely constrains real-time deployment. While linear attention mechanisms offer a computationally efficient alternative, existing architectures are predominantly limited to self-attention, lacking the cross-modal capabilities essential for autonomous driving. In this work, we propose LADY, the first fully linear attention-based generative model for end-to-end autonomous driving. LADY incorporates a novel, lightweight linear cross-attention (LICA) mechanism to enable effective cross-modal interaction while preserving linearity. A key advantage of our framework is its ability to fuse long-range temporal contexts during inference with constant computational and memory costs ($O(1)$), regardless of the historical sequence length. Experiments on the NAVSIM and Bench2Drive benchmarks demonstrate that LADY achieves performance comparable to state-of-the-art methods, delivering competitive planning accuracy with significantly reduced latency. Furthermore, efficiency benchmarking on edge devices validates the model's feasibility for resource-limited scenarios.

Figures

Figures reproduced from arXiv: 2512.15038 by Jihao Huang, Jingke Wang, Junbo Chen, Tengju Ye, Tianle Liu, Xi Xia, Zhiyuan Li.

Figure 1
Figure 1. Figure 1: Comparison of different end-to-end paradigms: (a - b) [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overall architecture of our approach LADY. The model’s inputs comprise the ego vehicle’s current status, consecutive multi-frame [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overall architecture of the RWKV-7 block. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Overall architecture of linear cross-attention block. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Overall architecture of the diffusion-based decoder layer. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Inference time and memory usage comparison between [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Visualization of representative NAVSIM scenarios of our proposed model LADY. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Comparison of representative Bench2Drive scenarios between our proposed LADY model and iPad [33]. [PITH_FULL_IMAGE:figures/full_fig_p008_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

43 extracted references · 17 linked inside Pith

  1. [1]

    Autonomous driving system: A comprehensive survey,

    J. Zhao, W. Zhao, B. Deng, Z. Wang, F. Zhang, W. Zheng, W. Cao, J. Nan, Y . Lian, and A. F. Burke, “Autonomous driving system: A comprehensive survey,”Expert Systems with Applications, vol. 242, p. 122836, 2024

    2024

  2. [2]

    End-to-end autonomous driving: Challenges and frontiers,

    L. Chen, P. Wu, K. Chitta, B. Jaeger, A. Geiger, and H. Li, “End-to-end autonomous driving: Challenges and frontiers,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 12, pp. 10 164– 10 183, 2024

    2024

  3. [3]

    Genad: Generative end-to-end autonomous driving,

    W. Zheng, R. Song, X. Guo, C. Zhang, and L. Chen, “Genad: Generative end-to-end autonomous driving,” inEuropean Conference on Computer Vision. Springer Nature Switzerland, 2025, pp. 87–104

    2025

  4. [4]

    End- to-end autonomous driving through v2x cooperation,

    H. Yu, W. Yang, J. Zhong, Z. Yang, S. Fan, P. Luo, and Z. Nie, “End- to-end autonomous driving through v2x cooperation,”Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, pp. 9598–9606, 2025

    2025

  5. [5]

    A survey of optimization-based task and motion planning: From classical to learning approaches,

    Z. Zhao, S. Cheng, Y . Ding, Z. Zhou, S. Zhang, D. Xu, and Y . Zhao, “A survey of optimization-based task and motion planning: From classical to learning approaches,”IEEE/ASME Transactions on Mechatronics, pp. 1–27, 2024

    2024

  6. [6]

    Real-time performance-focused localization techniques for autonomous vehicle: A review,

    Y . Lu, H. Ma, E. Smart, and H. Yu, “Real-time performance-focused localization techniques for autonomous vehicle: A review,”IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 7, pp. 6082–6100, 2022

    2022

  7. [7]

    Unitr: A unified and efficient multi-modal transformer for bird’s-eye- view representation,

    H. Wang, H. Tang, S. Shi, A. Li, Z. Li, B. Schiele, and L. Wang, “Unitr: A unified and efficient multi-modal transformer for bird’s-eye- view representation,” in2023 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE Computer Society, 2023, pp. 6769– 6779

    2023

  8. [8]

    Drivevlm: The convergence of au- tonomous driving and large vision-language models,

    X. Tian, J. Gu, B. Li, Y . Liu, Y . Wang, Z. Zhao, K. Zhan, P. Jia, X. Lang, and H. Zhao, “Drivevlm: The convergence of au- tonomous driving and large vision-language models,”arXiv preprint arXiv:2402.12289, 2024

    Pith/arXiv arXiv 2024

  9. [9]

    Video mamba suite: State space model as a versatile alternative for video understanding,

    G. Chen, Y . Huang, J. Xu, B. Pei, Z. Chen, Z. Li, J. Wang, K. Li, T. Lu, and L. Wang, “Video mamba suite: State space model as a versatile alternative for video understanding,”arXiv preprint arXiv:2403.09626, 2024

    Pith/arXiv arXiv 2024

  10. [10]

    A safe motion planning and reliable control framework for autonomous vehicles,

    H. Pan, M. Luo, J. Wang, T. Huang, and W. Sun, “A safe motion planning and reliable control framework for autonomous vehicles,” IEEE transactions on intelligent vehicles, vol. 9, no. 4, pp. 4780– 4793, 2024

    2024

  11. [11]

    Llm3: Large language model-based task and motion planning with motion failure reasoning,

    S. Wang, M. Han, Z. Jiao, Z. Zhang, Y . N. Wu, S.-C. Zhu, and H. Liu, “Llm3: Large language model-based task and motion planning with motion failure reasoning,” in2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024, pp. 12 086–12 092

    2024

  12. [12]

    Motion planning for autonomous driving: The state of the art and future perspectives,

    S. Teng, X. Hu, P. Deng, B. Li, Y . Li, Y . Ai, D. Yang, L. Li, Z. Xuanyuan, F. Zhu, and L. Chen, “Motion planning for autonomous driving: The state of the art and future perspectives,”IEEE Transac- tions on Intelligent Vehicles, vol. 8, no. 6, pp. 3692–3711, 2023

    2023

  13. [13]

    Recent advancements in end-to-end au- tonomous driving using deep learning: A survey,

    P. S. Chib and P. Singh, “Recent advancements in end-to-end au- tonomous driving using deep learning: A survey,”IEEE Transactions on Intelligent Vehicles, vol. 9, no. 1, pp. 103–118, 2023

    2023

  14. [14]

    Is ego status all you need for open-loop end-to-end autonomous driving?

    Z. Li, Z. Yu, S. Lan, J. Li, J. Kautz, T. Lu, and J. M. Alvarez, “Is ego status all you need for open-loop end-to-end autonomous driving?” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 14 864–14 873

    2024

  15. [15]

    Transfuser: Imitation with transformer-based sensor fusion for au- tonomous driving,

    K. Chitta, A. Prakash, B. Jaeger, Z. Yu, K. Renz, and A. Geiger, “Transfuser: Imitation with transformer-based sensor fusion for au- tonomous driving,”IEEE transactions on pattern analysis and machine intelligence, vol. 45, no. 11, pp. 12 878–12 895, 2022

    2022

  16. [16]

    Planning-oriented autonomous driving,

    Y . Hu, J. Yang, L. Chen, K. Li, C. Sima, X. Zhu, S. Chai, S. Du, T. Lin, W. Wanget al., “Planning-oriented autonomous driving,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 17 853–17 862

    2023

  17. [17]

    Fusionad: Multi-modality fusion for pre- diction and planning tasks of autonomous driving,

    T. Ye, W. Jing, C. Hu, S. Huang, L. Gao, F. Li, J. Wang, K. Guo, W. Xiao, W. Maoet al., “Fusionad: Multi-modality fusion for pre- diction and planning tasks of autonomous driving,”arXiv preprint arXiv:2308.01006, 2023

    Pith/arXiv arXiv 2023

  18. [18]

    Vad: Vectorized scene representation for efficient autonomous driving,

    B. Jiang, S. Chen, Q. Xu, B. Liao, J. Chen, H. Zhou, Q. Zhang, W. Liu, C. Huang, and X. Wang, “Vad: Vectorized scene representation for efficient autonomous driving,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 8340–8350

    2023

  19. [19]

    Vadv2: End-to-end vectorized autonomous driving via probabilistic planning,

    S. Chen, B. Jiang, H. Gao, B. Liao, Q. Xu, Q. Zhang, C. Huang, W. Liu, and X. Wang, “Vadv2: End-to-end vectorized autonomous driving via probabilistic planning,”arXiv preprint arXiv:2402.13243, 2024

    Pith/arXiv arXiv 2024

  20. [20]

    Hydra-mdp: End-to-end multimodal planning with multi-target hydra-distillation,

    Z. Li, K. Li, S. Wang, S. Lan, Z. Yu, Y . Ji, Z. Li, Z. Zhu, J. Kautz, Z. Wuet al., “Hydra-mdp: End-to-end multimodal planning with multi-target hydra-distillation,”arXiv preprint arXiv:2406.06978, 2024

    Pith/arXiv arXiv 2024

  21. [21]

    Hydra-mdp++: Advancing end-to-end driving via expert- guided hydra-distillation,

    K. Li, Z. Li, S. Lan, Y . Xie, Z. Zhang, J. Liu, Z. Wu, Z. Yu, and J. M. Alvarez, “Hydra-mdp++: Advancing end-to-end driving via expert- guided hydra-distillation,”arXiv preprint arXiv:2503.12820, 2025

    Pith/arXiv arXiv 2025

  22. [22]

    Diffusion policy: Visuomotor policy learning via ac- tion diffusion,

    C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song, “Diffusion policy: Visuomotor policy learning via ac- tion diffusion,”The International Journal of Robotics Research, p. 02783649241273668, 2023

    2023

  23. [23]

    Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving,

    B. Liao, S. Chen, H. Yin, B. Jiang, C. Wang, S. Yan, X. Zhang, X. Li, Y . Zhang, Q. Zhanget al., “Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving,”arXiv preprint arXiv:2411.15139, 2024

    Pith/arXiv arXiv 2024

  24. [24]

    Drama: An efficient end-to-end motion planner for autonomous driving with mamba,

    C. Yuan, Z. Zhang, J. Sun, S. Sun, Z. Huang, C. D. W. Lee, D. Li, Y . Han, A. Wong, K. P. Teeet al., “Drama: An efficient end-to-end motion planner for autonomous driving with mamba,”arXiv preprint arXiv:2408.03601, 2024

    Pith/arXiv arXiv 2024

  25. [25]

    Mamba: Linear-time sequence modeling with selective state spaces,

    A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,”arXiv preprint arXiv:2312.00752, 2023

    Pith/arXiv arXiv 2023

  26. [26]

    Transformers are ssms: Generalized models and efficient algorithms through structured state space duality,

    T. Dao and A. Gu, “Transformers are ssms: Generalized models and efficient algorithms through structured state space duality,”arXiv preprint arXiv:2405.21060, 2024

    Pith/arXiv arXiv 2024

  27. [27]

    Rwkv: Reinvent- ing rnns for the transformer era,

    B. Peng, E. Alcaide, Q. Anthony, A. Albalak, S. Arcadinho, S. Bider- man, H. Cao, X. Cheng, M. Chung, M. Grellaet al., “Rwkv: Reinvent- ing rnns for the transformer era,”arXiv preprint arXiv:2305.13048, 2023

    Pith/arXiv arXiv 2023

  28. [28]

    Eagle and finch: Rwkv with matrix-valued states and dynamic recurrence,

    B. Peng, D. Goldstein, Q. Anthony, A. Albalak, E. Alcaide, S. Bider- man, E. Cheah, T. Ferdinan, H. Hou, P. Kazienkoet al., “Eagle and finch: Rwkv with matrix-valued states and dynamic recurrence,”arXiv preprint arXiv:2404.05892, vol. 3, 2024

    Pith/arXiv arXiv 2024

  29. [29]

    Rwkv-7” goose

    B. Peng, R. Zhang, D. Goldstein, E. Alcaide, X. Du, H. Hou, J. Lin, J. Liu, J. Lu, W. Merrillet al., “Rwkv-7” goose” with expressive dynamic state evolution,”arXiv preprint arXiv:2503.14456, 2025

    Pith/arXiv arXiv 2025

  30. [30]

    Kimi linear: An expressive, efficient attention architecture,

    K. Team, “Kimi linear: An expressive, efficient attention architecture,” arXiv preprint arXiv:2510.26692, 2025

    Pith/arXiv arXiv 2025

  31. [31]

    Transdiffuser: End-to-end trajectory generation with decorrelated multi-modal representation for autonomous driving,

    X. Jiang, Y . Ma, P. Li, L. Xu, X. Wen, K. Zhan, Z. Xia, P. Jia, X. Lang, and S. Sun, “Transdiffuser: End-to-end trajectory generation with decorrelated multi-modal representation for autonomous driving,” arXiv preprint arXiv:2505.09315, 2025

    arXiv 2025

  32. [32]

    Navsim: Data-driven non-reactive autonomous vehicle simulation and benchmarking,

    D. Dauner, M. Hallgarten, T. Li, X. Weng, Z. Huang, Z. Yang, H. Li, I. Gilitschenski, B. Ivanovic, M. Pavone, A. Geiger, and K. Chitta, “Navsim: Data-driven non-reactive autonomous vehicle simulation and benchmarking,” inAdvances in Neural Information Processing Systems, vol. 37, 2024, pp. 28 706–28 719

    2024

  33. [33]

    ipad: Iterative proposal-centric end-to-end autonomous driving,

    K. Guo, H. Liu, X. Wu, J. Pan, and C. Lv, “ipad: Iterative proposal-centric end-to-end autonomous driving,”arXiv preprint arXiv:2505.15111, 2025

    Pith/arXiv arXiv 2025

  34. [34]

    Retentive network: A successor to transformer for large language models,

    Y . Sun, L. Dong, S. Huang, S. Ma, Y . Xia, J. Xue, J. Wang, and F. Wei, “Retentive network: A successor to transformer for large language models,”arXiv preprint arXiv:2307.08621, 2023

    Pith/arXiv arXiv 2023

  35. [35]

    Gated linear atten- tion transformers with hardware-efficient training,

    S. Yang, B. Wang, Y . Shen, R. Panda, and Y . Kim, “Gated linear atten- tion transformers with hardware-efficient training,” inProceedings of the 41st International Conference on Machine Learning, 2024, p. 23

    2024

  36. [36]

    Linear transformers are secretly fast weight programmers,

    I. Schlag, K. Irie, and J. Schmidhuber, “Linear transformers are secretly fast weight programmers,” inInternational conference on machine learning, 2021, pp. 9355–9366

    2021

  37. [37]

    Adaptive switching circuits,

    B. Widrow and M. E. Hoff, “Adaptive switching circuits,” inNeuro- computing: foundations of research. MIT Press, 1988, pp. 123–134

    1988

  38. [38]

    Parallelizing linear transformers with the delta rule over sequence length,

    S. Yang, B. Wang, Y . Zhang, Y . Shen, and Y . Kim, “Parallelizing linear transformers with the delta rule over sequence length,”arXiv preprint arXiv:2406.06484, 2024

    Pith/arXiv arXiv 2024

  39. [39]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017

    2017

  40. [40]

    Fla: A triton-based library for hardware- efficient implementations of linear attention mechanism,

    S. Yang and Y . Zhang, “Fla: A triton-based library for hardware- efficient implementations of linear attention mechanism,” 2024. [Online]. Available: https://github.com/fla-org/flash-linear-attention

    2024

  41. [41]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016

    2016

  42. [42]

    Carla: An open urban driving simulator,

    A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V . Koltun, “Carla: An open urban driving simulator,” inConference on robot learning, 2017, pp. 1–16

    2017

  43. [43]

    Bench2drive: Towards multi-ability benchmarking of closed-loop end-to-end autonomous driving,

    X. Jia, Z. Yang, Q. Li, Z. Zhang, and J. Yan, “Bench2drive: Towards multi-ability benchmarking of closed-loop end-to-end autonomous driving,”Advances in Neural Information Processing Systems, vol. 37, pp. 819–844, 2024

    2024