Towards Latency-Aware 3D Streaming Perception for Autonomous Driving

Jiangmiao Pang; Jiaqi Peng; Tai Wang; Yuan Shen

REVIEW 3 major objections 5 minor 40 references

Towards Latency-Aware 3D Streaming Perception for Autonomous Driving

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A 3D detector designed around runtime latency can keep online accuracy close to offline accuracy on an edge GPU without acceleration.

desk verdict The core idea is useful, but the benchmark's 12 Hz ground-truth labels are asserted without provenance, and that makes the headline gains provisional. read the letter →

arxiv 2504.19115 v1 pith:BNZEOECZ submitted 2025-04-27 cs.CV cs.RO

Jiaqi Peng , Tai Wang , Jiangmiao Pang , Yuan Shen This is my paper · ORCID

classification cs.CVcs.RO

keywords 3Dobjectdetectionstreamingperceptionlatency-awareautonomousdrivingquery-baseddetectortrajectorypredictionedgedeploymentirregulartimeseries

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that 3D perception for autonomous driving should be evaluated and designed for the latency a detector actually experiences on the vehicle, not only for offline accuracy on a fixed benchmark. To support that, it introduces a streaming benchmark that samples camera sweeps at irregular intervals set by a target device's runtime and grades predictions at multiple query times instead of only at frame boundaries. On top of that benchmark, it proposes LASP, a query-based 3D detector with two latency-aware parts: historical object queries are propagated to the current time through a continuous linear ODE, and a lightweight trajectory predictor compensates for the delay by shifting detections along predicted motion. The paper reports that on a Jetson AGX Orin, this design keeps online accuracy close to 80 percent of its own offline result without acceleration toolkits, that it outperforms the TensorRT-accelerated detector on the same device, and that on RTX 3090 and GTX 1060 it posts the highest mAP among the listed detectors. A sympathetic reader would take the central claim to be that treating latency as a first-class design variable recovers much of the performance that naive deployment leaves on the table.

What carries the argument

The load-bearing mechanism is continuous query propagation in the hidden space of object queries. Each object query carries a reference center and a context embedding; a nonlinear layer maps the embedding to a hidden state $z_t$, and the paper assumes that over a small time step the hidden state obeys the linear ODE $dz = A^m_{\tau(t)\to t} z\,dt$, where the transition matrix is a motion-conditioned mixture of $K$ basis matrices. Since the basis matrices share orthonormal eigenvectors, the matrix exponential becomes an elementwise exponential, so propagating a query from any past timestamp to the current time costs little. A second component, the intention-guided head, turns each detection query into $K$ trajectory endpoints using learnable intention queries seeded by k-means clusters of trajectory endpoints, and the posterior latency then selects where on those predicted trajectories the next output should be placed.

What would settle it

Regenerate the 12 Hz extended-nuScenes labels from the original 2 Hz keyframes using a different interpolation or label-propagation rule, then rerun the streaming comparisons; if LASP's mAP advantage shrinks or flips, part of the reported gain reflects the label-generation rule rather than perception quality.

Watch

Extended reading notes

Core claim

The central discovery the authors put forward is that the two failures induced by runtime latency—irregularly spaced historical frames and time-lagged outputs—can be modeled explicitly rather than patched by faster inference. They model each object query's hidden state as evolving under a linear ordinary differential equation whose transition matrix depends on ego pose and object velocity, and solve the propagation to the current time with a matrix exponential; because the basis matrices share eigenvectors, the exponential is cheap enough for edge deployment. For the delay itself, they split latency compensation into a latency-agnostic trajectory prediction head, trained with intention queries derived from k-means clusters of ground-truth trajectory endpoints, and a posterior step that shifts the just-computed detections along the predicted trajectories until the next result is ready. This lets one model answer queries at any time within a running frame, which they say is the first incorporation of trajectory prediction into the evaluation of 3D perception. On their extended 12 Hz version of the nuScenes dataset, they report that LASP on a Jetson AGX Orin reaches an online mAP of 0.353 without acceleration, above the TensorRT-accelerated model's 0.334 on the same hardware, and that on RTX 3090 and GTX 1060 LASP posts the highest mAP among the listed detectors.

Load-bearing premise

The reported results assume that the extended nuScenes video has reliable 3D labels at 12 Hz, and this assumption enters in Section V-A, yet the paper never states how those labels were made from the original 2 Hz keyframe annotations.

Editorial extensions

If this is right

Streaming evaluation with irregular query times can replace fixed-frame-rate evaluation for camera-based 3D detectors, because the benchmark derives its query times from real per-frame latency rather than a uniform schedule.
Latency compensation by trajectory prediction makes a detector a continuous-time estimator, so the same model can answer perception queries at any time within a frame rather than only at frame boundaries.
On edge hardware, recovering most of offline accuracy does not require specialized acceleration if the model is trained to predict and compensate for its own delay.
The method's advantage should grow as latency grows, because the predictive-detection component explicitly moves detections along predicted trajectories during the blocked interval.
A single trained model can serve across multiple streaming frame rates and hardware platforms without per-platform retraining, as the experiments evaluate one model under several simulated latencies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The 12 Hz ground-truth labels in the extended nuScenes set are the load-bearing evaluation currency, and the paper never states how they were produced; if they are interpolated or propagated from the original 2 Hz keyframes, part of the reported mAP advantage could reflect agreement with that interpolation rule rather than perception quality.
An unstated consequence of the ODE formulation is that the same mechanism could absorb other sources of timestamp irregularity, such as asynchronous camera sweeps or dropped frames, not just latency-induced irregularity.
Posterior-latency compensation assumes the system knows the latency after it happens; a deployed stack would need a runtime estimator, so a closed-loop variant that predicts latency in advance is a natural extension.
Combining the continuous history integration with existing acceleration software is an obvious follow-up; the paper treats acceleration as a competing baseline rather than a component that could be stacked with the proposed modules.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a plausible, well-engineered paper on making 3D streaming perception latency-aware, with a real practical motivation. But the benchmark it introduces rests on a dataset claim that is currently unverifiable: nuScenes does not ship 12 Hz 3D box annotations, yet the paper evaluates at 12 Hz without saying where those labels came from. That is a load-bearing gap, not a nitpick.

What's genuinely new: the paper transfers streaming perception from 2D to multi-frame 3D, and adds two concrete mechanisms—continuous ODE-based history integration over irregularly spaced frames, and trajectory-compensated detection that adjusts output for the measured latency. The ODE machinery is borrowed from continuous RNNs and state-space models, but the combination with query-based 3D detection is sensible and the ablations show each module helps. Running on Jetson AGX Orin and comparing against TensorRT-accelerated baselines is the right kind of evaluation for this problem. The writing is clear and the self-citations to StreamPETR and DORT are legitimate.

The soft spots, in order of weight. First, the ground-truth provenance. Section V-A states the extended nuScenes dataset has 12 Hz annotations and 6x the original samples, but offers no reference, protocol, or release link. If those labels are interpolated or propagated from the 2 Hz keyframes, then the mAP numbers—including the headline 0.353 on the Orin—may partly reward models that match the interpolation rule rather than models that perceive better. The stress-test note is right: this is load-bearing because every query-time evaluation in Eq. (4) depends on those labels. Second, there are no error bars or multiple runs, which is common in this field but still matters when gains are small (e.g., 0.353 vs 0.334 TensorRT). Third, the 'derivation' in Section IV-B is really an assumption—a linear ODE with learned parameters—not a derivation from physics. That is fine as a design choice, but the paper should not oversell it.

The paper also claims 'for the first time' incorporating trajectory prediction into 3D perception evaluation. Given ASAP [17] already addressed 3D streaming perception, that claim is too strong; the more defensible novelty is handling irregular intervals and multi-frame history, not the first trajectory-augmented evaluation. Minor but worth fixing.

Who this is for: researchers working on real-time 3D perception on edge hardware, and anyone building streaming benchmarks. It deserves a serious referee—the problem is real and the method is plausible—but the referee must demand a full account of the 12 Hz annotation protocol before any numbers are trusted. I'd engage with it critically.

Referee Report

3 major / 5 minor

Summary. The paper proposes a latency-aware streaming benchmark for camera-based 3D perception and an accompanying framework, LASP, that combines continuous-time history integration via a linear ODE model with a predictive detection head that compensates for runtime latency. The method is built on top of a query-based 3D detector (StreamPETR-style) and is evaluated on an 'extended nuScenes dataset' with annotations at 12 Hz. Experiments on RTX 3090 and Jetson AGX Orin report that LASP outperforms StreamPETR and other baselines, with a Jetson AGX Orin online mAP of 0.353 without hardware acceleration. The paper includes ablations of the two proposed modules, showing consistent improvements over no-alignment, MLN, zero-hold, velocity-based, and forecasting baselines.

Significance. If the results are valid, the paper addresses a practical gap in deploying 3D perception models on edge devices by explicitly considering runtime latency in both evaluation and model design. The continuous history-integration formulation and the intention-guided predictive detection are reasonable extensions of existing query-based architectures, and the ablations are internally consistent, giving moderate support to the method's effectiveness. The paper also makes a useful move by evaluating on edge hardware and comparing against TensorRT acceleration. However, the credibility of all quantitative claims rests on an unexplained 12 Hz ground-truth annotation protocol for nuScenes, and the absence of error bars leaves the magnitude of reported gains uncertain. The paper honestly states the shared-eigenvector assumption in Eq. (11), which is a positive aspect, though that assumption remains underexamined.

major comments (3)

[Section V-A, Datasets] The paper asserts that the 'extended nuScenes dataset' has annotations at a rate of 12 Hz, roughly six times the original keyframe rate, but provides no reference, release link, or annotation protocol. Original nuScenes provides human-annotated 3D boxes only at 2 Hz keyframes. If the 12 Hz labels were obtained by interpolation, propagation, or an automated pseudo-labeler, then every metric computed via Eq. (4) at non-keyframe query times compares predictions against synthetic targets, and the reported gains in Table I (including the Jetson AGX Orin mAP of 0.353) may partly reflect how well a method reproduces the interpolation rule rather than true perception quality. The authors must specify the annotation procedure, make the extended dataset available, or restrict the evaluation to times with real ground truth.
[Tables I, II, III] All reported mAP, NDS, and TP metrics are single numbers with no error bars or statistical significance measures. Since the streaming benchmark samples random latency, the differences between methods (e.g., LASP vs. TensorRT: 0.353 vs. 0.334 in Table III) may be within run-to-run variance. The authors should report mean and standard deviation over at least three training runs or evaluation seeds and specify the sampling distribution used for latency.
[Section IV-B, Eq. (11)] The derivation assumes that all basis matrices A^(k) share the same orthogonal eigenvectors, which is an unstated and unmotivated restriction on the transition matrix. This assumption substantially simplifies the matrix exponential, but the paper does not justify why such a restricted family is sufficient to model object motion in query-embedding space, nor does it ablate this design choice. The authors should either provide a justification, discuss the expressiveness limitations, or add an ablation that varies this assumption (e.g., using different eigenvectors per basis matrix).

minor comments (5)

[Abstract and Section V-B] The phrase 'closely aligns with 80% of its offline evaluation' is ambiguous; please specify the offline mAP for LASP on Jetson AGX Orin and the exact ratio (e.g., online mAP / offline mAP = 0.80).
[Section III-A, Eq. (2)] The query times set {t_j}_{j=0}^{N} is defined as {t_j | t+tau <= t_j < t+2tau}, but the number N and the precise sampling of points within the interval are not specified. Please clarify how N is chosen and whether the query times are uniformly sampled.
[Section V-B, Table I] The paper states that StreamPETR was retrained 'with video input at the streaming frame rate of each platform,' but does not report the training hyperparameters for StreamPETR or other baselines. Please provide these details to ensure a fair comparison.
[Section V-B] There is a typo: 'SreamPETR' should be 'StreamPETR' in the sentence preceding Table I.
[Throughout] The paper claims 'for the first time, we incorporate trajectory prediction results into the evaluation of 3D perception tasks.' Given that the ASAP benchmark (Ref. [17]) and DORT (Ref. [24]) both involve trajectory or predictive elements in 3D perception, please clarify the precise novelty of this claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: benchmark and model are externally evaluated; the 12 Hz label provenance is a data-validity concern, not a definitional reduction.

full rationale

The paper's central claims rest on an external benchmark (Eqs. 1-4) that compares online predictions against ground-truth labels y_j, and on two model components that are explicit supervised modeling choices rather than renamings of fitted inputs. The continuous history-integration ODE (Eqs. 8-11) is introduced as an assumption ('we assume that in small time step dt, the hidden state z_t also follows motion-aware linear transition'), not derived from the benchmark's outputs and not later relabeled as a prediction. The intention-guided predictive detection head (Eqs. 12-13) uses k-means cluster centers from training ground-truth trajectories as learnable query embeddings, then supervises the actual trajectory outputs; the test-time detection and trajectory predictions are therefore not equal to the cluster centers by construction. The self-citations, notably DORT [24] for temporal modeling and StreamPETR [7] as a baseline, motivate design choices and provide comparison methods but do not carry the claimed result; the method is evaluated against external baselines and external validation ground truth. The only serious concern is the unproven provenance of the 'extended nuScenes dataset [30], with annotations at a rate of 12Hz' in Section V-A, since the cited nuScenes paper provides keyframe annotations at 2 Hz. This is a benchmark-validity and reproducibility risk, not a circularity, because the evaluation labels are still external to the model's own fitted outputs and the paper does not define its ground truth in terms of its predictions.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

No new physical entities are introduced. The intention queries are learned representations derived from training data, not independent entities. The ODE formulation is a mathematical model, not an invented object with independent evidence.

free parameters (2)

number of intention points K = 6 per category
K-means on the endpoints of ground-truth trajectories in the training set; used to define intention queries in the predictive detection head (Eq. 12-13).
number of basis matrices in history integration = 10
Chosen for the locally linear transition model in Eq. 10; a hyperparameter of the ODE-based query propagation.

assumptions (3)

domain assumption The query hidden state z_t follows a linear ODE dz = A z dt with a motion-aware transition matrix over small time steps (Eq. 8).
Stated in Section IV-B. This is a modeling assumption, not derived from physical laws; the transition parameters are learned from data.
ad hoc to paper The basis matrices A^(k) share the same orthogonal eigenvectors, so the matrix exponential reduces to an elementwise exponential (Eq. 11).
A design choice to make the ODE solution computationally cheap; it restricts the family of learnable transitions and is not justified independently of the paper's goals.
domain assumption The extended nuScenes dataset contains valid 3D annotations at 12 Hz.
Invoked in Section V-A. Original nuScenes annotates keyframes at 2 Hz; the paper does not disclose the interpolation or annotation procedure.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Towards Latency-Aware 3D Streaming Perception for Autonomous Driving." pith.science (2026). https://pith.science/paper/BNZEOECZ

@misc{pith2026250419115,
  author       = {Pith},
  title        = {Pith review of: Towards Latency-Aware 3D Streaming Perception for Autonomous Driving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BNZEOECZ}},
  note         = {Machine review of arXiv:2504.19115}
}

read the original abstract

Although existing 3D perception algorithms have demonstrated significant improvements in performance, their deployment on edge devices continues to encounter critical challenges due to substantial runtime latency. We propose a new benchmark tailored for online evaluation by considering runtime latency. Based on the benchmark, we build a Latency-Aware 3D Streaming Perception (LASP) framework that addresses the latency issue through two primary components: 1) latency-aware history integration, which extends query propagation into a continuous process, ensuring the integration of historical feature regardless of varying latency; 2) latency-aware predictive detection, a module that compensates the detection results with the predicted trajectory and the posterior accessed latency. By incorporating the latency-aware mechanism, our method shows generalization across various latency levels, achieving an online performance that closely aligns with 80\% of its offline evaluation on the Jetson AGX Orin without any acceleration techniques.

Figures

Figures reproduced from arXiv: 2504.19115 by the authors.

**Figure 2.** The left figure illustrates the runtime distribution on the edge device. We sampled runtime of each frame, applying discrete transformations to [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Our method builds on end-to-end sparse query-based 3D object detectors: 1) We maintain a memory bank containing historical query context [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Online performance under different streaming frame rate [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

**Figure 5.** Figure 5: Visualization of baseline and our results on nuScenes dataset. We show 3D bboxes predictions in camera images and the bird’s-eye-view. CAM_FRONT_LEFT CAM_FRONT CAM_FRONT_RIGHT CAM_BACK_LEFT CAM_BACK CAM_BACK_RIGHT [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

**Figure 6.** Figure 6: Visualization of predictive detection results. We show detection boxes for each object and their future trajectories. TABLE II THE ABLATION STUDY ON TEMPORAL ALIGNMENT Methods mAP↑ mATE↓ mASE↓ mAOE↓ mAAE↓ No align 0.327 0.713 0.272 0.442 0.212 MLN 0.343 0.704 0.276 0.4…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 19 canonical work pages

[17]

Are we ready for vision-centric driving streaming perception? the asap benchmark,

X. Wang, Z. Zhu, Y . Zhang, G. Huang, Y . Ye, W. Xu, Z. Chen, and X. Wang, “Are we ready for vision-centric driving streaming perception? the asap benchmark,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 9600–9610

work page 2023
[1]

Vision-centric bev perception: A survey,

Y . Ma, T. Wang, X. Bai, H. Yang, Y . Hou, Y . Wang, Y . Qiao, R. Yang, D. Manocha, and X. Zhu, “Vision-centric bev perception: A survey,” arXiv preprint arXiv:2208.02797 , 2022

arXiv 2022
[2]

3d object detection for autonomous driving: A comprehensive survey,

J. Mao, S. Shi, X. Wang, and H. Li, “3d object detection for autonomous driving: A comprehensive survey,” International Journal of Computer Vision , vol. 131, no. 8, pp. 1909–1963, 2023

work page 1909
[3]

Fcos3d: Fully convolutional one-stage monocular 3d object detection,

T. Wang, X. Zhu, J. Pang, and D. Lin, “Fcos3d: Fully convolutional one-stage monocular 3d object detection,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2021, pp. 913–922

2021
[4]

Bevdet: High- performance multi-camera 3d object detection in bird-eye-view,

J. Huang, G. Huang, Z. Zhu, Y . Ye, and D. Du, “Bevdet: High- performance multi-camera 3d object detection in bird-eye-view,” arXiv preprint arXiv:2112.11790, 2021

arXiv 2021
[5]

Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers,

Z. Li, W. Wang, H. Li, E. Xie, C. Sima, T. Lu, Y . Qiao, and J. Dai, “Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers,” in European conference on computer vision . Springer, 2022, pp. 1–18

work page 2022
[6]

Time will tell: New outlooks and a baseline for tem- poral multi-view 3d object detection,

J. Park, C. Xu, S. Yang, K. Keutzer, K. M. Kitani, M. Tomizuka, and W. Zhan, “Time will tell: New outlooks and a baseline for tem- poral multi-view 3d object detection,” in The Eleventh International Conference on Learning Representations , 2022

work page 2022
[7]

Exploring object- centric temporal modeling for efficient multi-view 3d object detec- tion,

S. Wang, Y . Liu, T. Wang, Y . Li, and X. Zhang, “Exploring object- centric temporal modeling for efficient multi-view 3d object detec- tion,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 3621–3631

work page 2023

Show all 40 references

[8]

Center-based 3d object detec- tion and tracking,

T. Yin, X. Zhou, and P. Krahenbuhl, “Center-based 3d object detec- tion and tracking,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2021, pp. 11 784–11 793

2021
[9]

Pointpillars: Fast encoders for object detection from point clouds,

A. H. Lang, S. V ora, H. Caesar, L. Zhou, J. Yang, and O. Beijbom, “Pointpillars: Fast encoders for object detection from point clouds,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 12 697–12 705

2019
[10]

An energy and gpu-computation efficient backbone network for real-time object detection,

Y . Lee, J.-w. Hwang, S. Lee, Y . Bae, and J. Park, “An energy and gpu-computation efficient backbone network for real-time object detection,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops , 2019, pp. 0–0

2019
[11]

Multi-dimensional pruning: Joint channel, layer and block pruning with latency constraint,

X. Sun, B. Lakshmanan, M. Shen, S. Lan, J. Chen, and J. Alvarez, “Multi-dimensional pruning: Joint channel, layer and block pruning with latency constraint,” arXiv preprint arXiv:2406.12079 , 2024

2024 arXiv
[12]

Towards streaming perception,

M. Li, Y .-X. Wang, and D. Ramanan, “Towards streaming perception,” in Computer Vision–ECCV 2020: 16th European Conference, Glas- gow, UK, August 23–28, 2020, Proceedings, Part II 16 . Springer, 2020, pp. 473–488

2020
[13]

Real-time object detection for streaming perception,

J. Yang, S. Liu, Z. Li, X. Li, and J. Sun, “Real-time object detection for streaming perception,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 5385–5395

2022
[14]

Longshortnet: Exploring temporal and semantic features fusion in streaming perception,

C. Li, Z.-Q. Cheng, J.-Y . He, P. Li, B. Luo, H. Chen, Y . Geng, J.- P. Lan, and X. Xie, “Longshortnet: Exploring temporal and semantic features fusion in streaming perception,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASS...

2023
[15]

Damo-streamnet: Optimizing streaming perception in autonomous driving,

J.-Y . He, Z.-Q. Cheng, C. Li, W. Xiang, B. Chen, B. Luo, Y . Geng, and X. Xie, “Damo-streamnet: Optimizing streaming perception in autonomous driving,” arXiv preprint arXiv:2303.17144 , 2023

2023 arXiv
[16]

Adap- tive streaming perception using deep reinforcement learning,

A. Ghosh, A. Nambi, A. Singh, H. Yvs, and T. Ganu, “Adap- tive streaming perception using deep reinforcement learning,” arXiv preprint arXiv:2106.05665, vol. 1, no. 2, p. 7, 2021

2021 arXiv
[18]

Bevdepth: Acquisition of reliable depth for multi-view 3d object detection,

Y . Li, Z. Ge, G. Yu, J. Yang, Z. Wang, Y . Shi, J. Sun, and Z. Li, “Bevdepth: Acquisition of reliable depth for multi-view 3d object detection,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 2, 2023, pp. 1477–1485

2023
[19]

Sparsebev: High- performance sparse 3d object detection from multi-camera videos,

H. Liu, Y . Teng, T. Lu, H. Wang, and L. Wang, “Sparsebev: High- performance sparse 3d object detection from multi-camera videos,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 18 580–18 590

2023
[20]

Ray denoising: Depth-aware hard negative sampling for multiview 3d object detection,

F. Liu, T. Huang, Q. Zhang, H. Yao, C. Zhang, F. Wan, Q. Ye, and Y . Zhou, “Ray denoising: Depth-aware hard negative sampling for multiview 3d object detection,” arXiv preprint arXiv:2402.03634 , vol. 10, 2024

2024 arXiv
[21]

Bevdet4d: Exploit temporal cues in multi- camera 3d object detection,

J. Huang and G. Huang, “Bevdet4d: Exploit temporal cues in multi- camera 3d object detection,” arXiv preprint arXiv:2203.17054 , 2022

2022 arXiv
[22]

Sparse4d: Multi-view 3d object detection with sparse spatial-temporal fusion,

X. Lin, T. Lin, Z. Pei, L. Huang, and Z. Su, “Sparse4d: Multi-view 3d object detection with sparse spatial-temporal fusion,” arXiv preprint arXiv:2211.10581, 2022

2022 arXiv
[23]

Petrv2: A unified framework for 3d perception from multi-camera images,

Y . Liu, J. Yan, F. Jia, S. Li, A. Gao, T. Wang, and X. Zhang, “Petrv2: A unified framework for 3d perception from multi-camera images,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 3262–3272

2023
[24]

Dort: Modeling dynamic objects in recurrent for multi-camera 3d object detection and tracking,

L. Qing, T. Wang, D. Lin, and J. Pang, “Dort: Modeling dynamic objects in recurrent for multi-camera 3d object detection and tracking,” in Conference on Robot Learning . PMLR, 2023, pp. 3749–3765

2023
[25]

Predictive visual track- ing: A new benchmark and baseline approach,

B. Li, Y . Li, J. Ye, C. Fu, and H. Zhao, “Predictive visual track- ing: A new benchmark and baseline approach,” arXiv preprint arXiv:2103.04508, 2021

2021 arXiv
[26]

Pvt++: a simple end-to-end latency-aware visual tracking framework,

B. Li, Z. Huang, J. Ye, Y . Li, S. Scherer, H. Zhao, and C. Fu, “Pvt++: a simple end-to-end latency-aware visual tracking framework,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 10 006–10 016

2023
[27]

Bot-sort: Robust asso- ciations multi-pedestrian tracking,

N. Aharon, R. Orfaig, and B.-Z. Bobrovsky, “Bot-sort: Robust asso- ciations multi-pedestrian tracking,” arXiv preprint arXiv:2206.14651 , 2022

2022 arXiv
[28]

Vip3d: End-to-end visual trajectory prediction via 3d agent queries,

J. Gu, C. Hu, T. Zhang, X. Chen, Y . Wang, Y . Wang, and H. Zhao, “Vip3d: End-to-end visual trajectory prediction via 3d agent queries,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 5496–5506

2023
[29]

Query-centric trajectory prediction,

Z. Zhou, J. Wang, Y .-H. Li, and Y .-K. Huang, “Query-centric trajectory prediction,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 17 863–17 873

2023
[30]

nuscenes: A multimodal dataset for autonomous driving,

H. Caesar, V . Bankiti, A. H. Lang, S. V ora, V . E. Liong, Q. Xu, A. Krishnan, Y . Pan, G. Baldan, and O. Beijbom, “nuscenes: A multimodal dataset for autonomous driving,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2020, pp. 11 621–11 631

2020
[31]

Deep residual learning for image recognition,

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778

2016
[32]

Finding structure in time,

J. L. Elman, “Finding structure in time,” Cognitive science , vol. 14, no. 2, pp. 179–211, 1990

1990
[33]

Mime: Multilevel medical embedding of electronic health records for predictive healthcare,

E. Choi, C. Xiao, W. Stewart, and J. Sun, “Mime: Multilevel medical embedding of electronic health records for predictive healthcare,” Advances in neural information processing systems , vol. 31, 2018

2018
[34]

Modeling irregular time series with continuous recurrent units,

M. Schirmer, M. Eltayeb, S. Lessmann, and M. Rudolph, “Modeling irregular time series with continuous recurrent units,” in International conference on machine learning . PMLR, 2022, pp. 19 388–19 405

2022
[35]

Kalmannet: Neural network aided kalman filtering for partially known dynamics,

G. Revach, N. Shlezinger, X. Ni, A. L. Escoriza, R. J. Van Sloun, and Y . C. Eldar, “Kalmannet: Neural network aided kalman filtering for partially known dynamics,” IEEE Transactions on Signal Processing , vol. 70, pp. 1532–1547, 2022

2022
[36]

Deep variational bayes filters: Unsupervised learning of state space models from raw data,

M. Karl, M. Soelch, J. Bayer, and P. Van der Smagt, “Deep variational bayes filters: Unsupervised learning of state space models from raw data,” arXiv preprint arXiv:1605.06432 , 2016

2016 arXiv
[37]

A disentangled recognition and nonlinear dynamics model for unsupervised learning,

M. Fraccaro, S. Kamronn, U. Paquet, and O. Winther, “A disentangled recognition and nonlinear dynamics model for unsupervised learning,” Advances in neural information processing systems , vol. 30, 2017

2017
[38]

J. B. Rawlings, D. Q. Mayne, M. Diehl et al., Model predictive control: theory, computation, and design . Nob Hill Publishing Madison, WI, 2017, vol. 2

2017
[39]

Mtr++: Multi-agent motion prediction with symmetric scene modeling and guided intention query- ing,

S. Shi, L. Jiang, D. Dai, and B. Schiele, “Mtr++: Multi-agent motion prediction with symmetric scene modeling and guided intention query- ing,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

2024
[40]

Motion transformer with global intention localization and local movement refinement,

Shi, Shaoshuai and Jiang, Li and Dai, Dengxin and Schiele, Bernt, “Motion transformer with global intention localization and local movement refinement,” Advances in Neural Information Processing Systems, vol. 35, pp. 6531–6543, 2022

2022

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [17]

Are we ready for vision-centric driving streaming perception? the asap benchmark,

X. Wang, Z. Zhu, Y . Zhang, G. Huang, Y . Ye, W. Xu, Z. Chen, and X. Wang, “Are we ready for vision-centric driving streaming perception? the asap benchmark,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 9600–9610

work page 2023

[2] [1]

Vision-centric bev perception: A survey,

Y . Ma, T. Wang, X. Bai, H. Yang, Y . Hou, Y . Wang, Y . Qiao, R. Yang, D. Manocha, and X. Zhu, “Vision-centric bev perception: A survey,” arXiv preprint arXiv:2208.02797 , 2022

arXiv 2022

[3] [2]

3d object detection for autonomous driving: A comprehensive survey,

J. Mao, S. Shi, X. Wang, and H. Li, “3d object detection for autonomous driving: A comprehensive survey,” International Journal of Computer Vision , vol. 131, no. 8, pp. 1909–1963, 2023

work page 1909

[4] [3]

Fcos3d: Fully convolutional one-stage monocular 3d object detection,

T. Wang, X. Zhu, J. Pang, and D. Lin, “Fcos3d: Fully convolutional one-stage monocular 3d object detection,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2021, pp. 913–922

2021

[5] [4]

Bevdet: High- performance multi-camera 3d object detection in bird-eye-view,

J. Huang, G. Huang, Z. Zhu, Y . Ye, and D. Du, “Bevdet: High- performance multi-camera 3d object detection in bird-eye-view,” arXiv preprint arXiv:2112.11790, 2021

arXiv 2021

[6] [5]

Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers,

Z. Li, W. Wang, H. Li, E. Xie, C. Sima, T. Lu, Y . Qiao, and J. Dai, “Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers,” in European conference on computer vision . Springer, 2022, pp. 1–18

work page 2022

[7] [6]

Time will tell: New outlooks and a baseline for tem- poral multi-view 3d object detection,

J. Park, C. Xu, S. Yang, K. Keutzer, K. M. Kitani, M. Tomizuka, and W. Zhan, “Time will tell: New outlooks and a baseline for tem- poral multi-view 3d object detection,” in The Eleventh International Conference on Learning Representations , 2022

work page 2022

[8] [7]

Exploring object- centric temporal modeling for efficient multi-view 3d object detec- tion,

S. Wang, Y . Liu, T. Wang, Y . Li, and X. Zhang, “Exploring object- centric temporal modeling for efficient multi-view 3d object detec- tion,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 3621–3631

work page 2023