REVIEW 3 major objections 5 minor 1 cited by
RoPETR: Improving Temporal Camera-Only 3D Detection by Integrating Enhanced Rotary Position Embedding
T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read RoPETR claims that decomposing rotary position embeddings into spatial and temporal components improves temporal modeling enough to set a new camera-only 3D detection benchmark, reaching 70.9% NDS on the nuScenes test set.
desk verdict Benchmark claim with a load-bearing confound: no ablation isolates the proposed rotary embedding from RayDN. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is Multimodal Rotary Position Embedding (M-RoPE), a rotary position embedding that turns normalized bird's-eye-view coordinates $(x,y)$ and a normalized frame time $t$ into rotation angles via log-frequency vectors, then applies channel-pair rotations to queries and keys in both self-attention and cross-attention. This gives the transformer an explicit relative-position and relative-time signal, which is what lets StreamPETR's propagated object queries encode motion dynamics rather than only appearance and depth.
What would settle it
Train StreamPETR with RayDN but without the rotary embedding, using the same 24-epoch recipe; if its NDS and mAVE match RoPETR's 69.0 NDS and 0.163 mAVE, then the rotary embedding's claimed contribution is not supported.
Extended reading notes
Core claim
Multimodal Rotary Position Embedding (M-RoPE) normalizes each object's BEV center $(x,y)$ to $[0,1]^2$, multiplies it by a logarithmic frequency vector $\omega$ to obtain rotation angles $\theta_x$ and $\theta_y$, and separately encodes a normalized frame identifier $t \in [0,1]$ with another frequency vector $\omega_t$ to obtain $\theta_t$. These angles rotate channel pairs in query/key operations in self-attention and cross-attention, so attention becomes explicitly sensitive to relative BEV offsets and relative time between frames. In StreamPETR's streaming decoder, which already propagates object queries over time, the paper reports that this lowers mean velocity error from 0.220 to 0.163 on the test set and raises NDS from 68.6% to 69.0%, with the larger RoPETR-e variant reaching 70.9% NDS and 64.8% mAP.
Load-bearing premise
The paper's attribution of the NDS gain to the proposed embedding assumes that the concurrently added RayDN module does not account for the improvement.
Editorial extensions
If this is right
- If the central claim is right, the main measurable effect is a large drop in mean velocity error: RoPETR reports a test mAVE of 0.163 versus 0.220 for the RayDN baseline.
- NDS can rise without mAP rising: RoPETR reports 69.0 NDS with 61.9 mAP against 68.6 NDS with 63.1 mAP for RayDN, so the gain is concentrated in motion-related metrics.
- The embedding introduces no new depth network or dense supervision, so the reported gain does not come from added inference cost.
- The method stacks with higher resolution and test-time augmentation, giving the enhanced RoPETR-e result of 70.9 NDS and 64.8 mAP on the test set.
Reading between the lines
- The paper's attribution of the gain specifically to M-RoPE is not directly tested, because the RayDN module is added in the same step; a dedicated ablation would make the causal claim exact.
- If relative-time encoding is the operative ingredient, varying the temporal frequency scale $\omega_t$ should shift the velocity-error distribution in a predictable way, which is a direct behavioral test.
- The same rotary decomposition is detector-agnostic and could be applied to other streaming query-based detectors, where the same velocity bottleneck likely appears.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes RoPETR, a modification of the StreamPETR camera-only 3D detector. The claimed contribution is a Multimodal Rotary Position Embedding (M-RoPE) applied to query-key operations, intended to improve temporal modeling and, in particular, velocity estimation on the nuScenes benchmark. The authors also state that they incorporate the RayDN module into the same framework. Experiments report an NDS of 69.0% (mAP 61.9%) on the nuScenes test set with a ViT-L backbone, and 70.9% NDS (mAP 64.8%) with higher resolution and test-time augmentation, which the paper describes as state-of-the-art for camera-only 3D object detection. A validation-set comparison is also provided for a V2-99 backbone, where RoPETR reaches 61.4% NDS versus 57.1% for StreamPETR. The manuscript is short, method details are given only in prose, and no ablation study is included.
Significance. If the reported results are correctly attributed, the paper would represent a meaningful practical advance: improving velocity estimation is widely recognized as a bottleneck for NDS on nuScenes, and the reported mA VE reductions (0.236 to 0.163 in the test table) are substantial. The problem framing is sensible, and the use of rotary position embeddings for spatiotemporal queries is a plausible direction. However, the manuscript currently provides no formal specification of the proposed embedding, no ablations, and no baseline that separates the effect of M-RoPE from the concurrently added RayDN module. As a result, the central claim that M-RoPE drives the improvement is unsupported by the presented evidence. The paper's strengths are its clear motivation and the use of a standard benchmark, but these do not compensate for the missing isolation of the contribution.
major comments (3)
- [Section IV-B, Table II] The reported RoPETR results are confounded with the RayDN module. Section IV-B states, "Additionally, we incorporate the RayDN [16] module into our framework," so every RoPETR row presumably includes both M-RoPE and RayDN. Table II shows RayDN (ViT-L) alone achieving 68.6 NDS and 0.220 mA VE, while RoPETR achieves 69.0 NDS and 0.163 mA VE. No StreamPETR+RayDN baseline and no M-RoPE-only (without RayDN) variant are reported. Since RayDN alone already improves NDS and mA VE over StreamPETR, the differences in Table II cannot be attributed to M-RoPE. The sentence in Section IV-C claiming that the results "clearly demonstrat[e] the efficacy of our rotational positional encoding approach" is therefore not supported by the experiments.
- [Section III-B] The proposed method is not formally specified. No equations are provided for the frequency vectors omega and omega_t, the rotation angles theta_x, theta_y, and theta_t, or the exact mechanism by which these rotations are applied to Q/K in self-attention and cross-attention. The phrase "multiplied by a logarithmic frequency vector" leaves unclear whether the frequencies are fixed constants, learned parameters, or a combination. The reference to [26] (Qwen2-VL) does not identify a specific section or formula, and the adaptation to StreamPETR is described only qualitatively. This makes the contribution impossible to reproduce or to verify as a distinct technical novelty.
- [Table I, Section IV-C] The validation evidence suffers from the same attribution problem. Table I compares RoPETR (V2-99) with StreamPETR and Stream3DPPE, but if the RoPETR row includes RayDN, as Section IV-B indicates, then the +4.3 NDS improvement over StreamPETR may be partly or wholly due to RayDN rather than M-RoPE. No validation row for RayDN and no ablation with and without M-RoPE are provided. Additionally, Table I uses a different backbone than the test-set SOTA claim, so the test results are not accompanied by a validation-table counterpart that would help assess generalization of the proposed component.
minor comments (5)
- [Abstract and Table II] The abstract reports a state-of-the-art NDS of 70.86%, while Table II gives 70.9%; these values should be made consistent.
- [Section IV-C] The citation "ViT-L [27] [20]" for the ViT-L backbone is incorrect: [20] is SOLOFusion, not the ViT paper. The intended reference appears to be [4] (Dosovitskiy et al.).
- [Section III-B] The name "Multimodal Rotary Position Embedding" is used, but the text does not explain what "multimodal" refers to in this application, given that only camera inputs are used. The relationship to the M-RoPE of Qwen2-VL should be clarified.
- [Table II] RoPETR (ViT-L) has a lower mAP than RayDN (61.9 vs 63.1) while having a higher NDS. The paper does not discuss this trade-off, which is relevant because the stated goal is to improve velocity estimation without harming detection accuracy.
- [Section IV-C] The sentence "Prior methods such as DETR3D and MV2D reported moderate results of 47.9% and 51.4% NDS" lists numbers that correspond to specific input resolutions in Table II; the text should state the configurations to avoid misleading comparisons.
Circularity Check
No circular derivation: RoPETR is an empirical configuration of an external embedding (M-RoPE) with no fitted parameters; the RayDN confound is an attribution risk, not a circularity.
full rationale
The paper's proposed contribution is the adoption of Multimodal Rotary Position Embedding (M-RoPE) from Qwen2-VL [26] into StreamPETR. The method section contains no closed-form derivation and no fitted parameter that is later re-reported as a prediction; the rotation angles are described as normalized coordinates multiplied by a logarithmic frequency vector, following the cited external work. The experimental section compares RoPETR against StreamPETR and Stream3DPPE baselines. A legitimate weakness is that Section IV-B states 'we incorporate the RayDN [16] module into our framework,' and no StreamPETR+RayDN or M-RoPE-only ablation is reported, so the NDS and mA VE gains cannot be cleanly attributed to M-RoPE. However, this is a missing control or attribution problem, not a circular reduction: the result is not defined in terms of the claim, no fitted value is renamed as a prediction, and no self-citation chain or uniqueness theorem is invoked. Therefore no circularity step can be exhibited, and the appropriate score is 0.
Assumptions & free parameters
free parameters (1)
- Rotary frequency vectors omega and omega_t
assumptions (3)
- domain assumption M-RoPE transfers from Qwen2-VL to BEV coordinate and temporal encoding in StreamPETR
- domain assumption The RayDN module does not account for the observed improvement
- domain assumption The publicly reported nuScenes test numbers are accurate as stated
Cite this review
Pith. "Pith review of RoPETR: Improving Temporal Camera-Only 3D Detection by Integrating Enhanced Rotary Position Embedding." pith.science (2026). https://pith.science/paper/DGZSTF7F
@misc{pith2026250412643,
author = {Pith},
title = {Pith review of: RoPETR: Improving Temporal Camera-Only 3D Detection by Integrating Enhanced Rotary Position Embedding},
year = {2026},
howpublished = {\url{https://pith.science/paper/DGZSTF7F}},
note = {Machine review of arXiv:2504.12643}
}
read the original abstract
This technical report introduces a targeted improvement to the StreamPETR framework, specifically aimed at enhancing velocity estimation, a critical factor influencing the overall NuScenes Detection Score. While StreamPETR exhibits strong 3D bounding box detection performance as reflected by its high mean Average Precision our analysis identified velocity estimation as a substantial bottleneck when evaluated on the NuScenes dataset. To overcome this limitation, we propose a customized positional embedding strategy tailored to enhance temporal modeling capabilities. Experimental evaluations conducted on the NuScenes test set demonstrate that our improved approach achieves a state-of-the-art NDS of 70.86% using the ViT-L backbone, setting a new benchmark for camera-only 3D object detection.
Forward citations
Cited by 1 Pith paper
-
Geometry-Grounded Unified 3D Perception for Autonomous Driving
A single transformer latent pretrained for 3D reconstruction, adapted with camera calibration and split attention, drives depth, detection, and occupancy for self-driving cameras.
Reference graph
Works this paper leans on
-
[16]
Ray denoising: Depth-aware hard negative sampling for multi-view 3d object detection
Feng Liu, Tengteng Huang, Qianjing Zhang, Haotian Yao, Chi Zhang, Fang Wan, Qixiang Ye, and Yanzhao Zhou. Ray denoising: Depth-aware hard negative sampling for multi-view 3d object detection. InEuropean Conference on Computer Vision, pages 200–217. Springer, 2024
work page 2024
-
[26]
Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2- vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024
arXiv 2024
-
[1]
nuscenes: A multimodal dataset for autonomous driving
Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11621–11631, 2020
2020
-
[2]
End-to-end object detection with transformers
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. InEuropean conference on computer vision, pages 213–229. Springer, 2020
2020
-
[3]
Depth- net: A recurrent neural network architecture for monocular depth pre- diction
Arun CS Kumar, Suchendra M Bhandarkar, and Mukta Prasad. Depth- net: A recurrent neural network architecture for monocular depth pre- diction. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 283–291, 2018
work page 2018
-
[4]
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weis- senborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020
arXiv 2010
-
[5]
Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Girshick. Mask r-cnn. InProceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017
work page 2017
-
[6]
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016
2016
Show all 33 references
-
[7]
Bevdet4d: Exploit temporal cues in multi-camera 3d object detection.arXiv preprint arXiv:2203.17054, 2022
Junjie Huang and Guan Huang. Bevdet4d: Exploit temporal cues in multi-camera 3d object detection.arXiv preprint arXiv:2203.17054, 2022
2022 arXiv
-
[8]
Bevdet: High-performance multi-camera 3d object detection in bird-eye- view.arXiv preprint arXiv:2112.11790, 2021
Junjie Huang, Guan Huang, Zheng Zhu, Yun Ye, and Dalong Du. Bevdet: High-performance multi-camera 3d object detection in bird-eye- view.arXiv preprint arXiv:2112.11790, 2021
2021 arXiv
-
[9]
Centermask: Real-time anchor-free instance segmentation
Youngwan Lee and Jongyoul Park. Centermask: Real-time anchor-free instance segmentation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13906–13915, 2020
2020
-
[10]
Dn-detr: Accelerate detr training by introducing query denoising
Feng Li, Hao Zhang, Shilong Liu, Jian Guo, Lionel M Ni, and Lei Zhang. Dn-detr: Accelerate detr training by introducing query denoising. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13619–13627, 2022
2022
-
[11]
Bevstereo: Enhancing depth estimation in multi-view 3d object detection with temporal stereo
Yinhao Li, Han Bao, Zheng Ge, Jinrong Yang, Jianjian Sun, and Zeming Li. Bevstereo: Enhancing depth estimation in multi-view 3d object detection with temporal stereo. InProceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 1486–1494, 2023
2023
-
[12]
Bevdepth: Acquisition of reliable depth for multi-view 3d object detection
Yinhao Li, Zheng Ge, Guanyi Yu, Jinrong Yang, Zengran Wang, Yukang Shi, Jianjian Sun, and Zeming Li. Bevdepth: Acquisition of reliable depth for multi-view 3d object detection. InProceedings of the AAAI conference on artificial intelligence, volume 37, pages 1477–1485, 2023
2023
-
[13]
Bevformer: learning bird’s-eye-view representation from lidar-camera via spatiotemporal transformers.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024
Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chonghao Sima, Tong Lu, Qiao Yu, and Jifeng Dai. Bevformer: learning bird’s-eye-view representation from lidar-camera via spatiotemporal transformers.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024
2024
-
[14]
Fb-bev: Bev representation from forward-backward view transformations
Zhiqi Li, Zhiding Yu, Wenhai Wang, Anima Anandkumar, Tong Lu, and Jose M Alvarez. Fb-bev: Bev representation from forward-backward view transformations. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 6919–6928, 2023
2023
-
[15]
Sparse4d v3: Advancing end-to-end 3d detection and tracking.arXiv preprint arXiv:2311.11722, 2023
Xuewu Lin, Zixiang Pei, Tianwei Lin, Lichao Huang, and Zhizhong Su. Sparse4d v3: Advancing end-to-end 3d detection and tracking.arXiv preprint arXiv:2311.11722, 2023
2023 arXiv
-
[17]
Petr: Position embedding transformation for multi-view 3d object detection
Yingfei Liu, Tiancai Wang, Xiangyu Zhang, and Jian Sun. Petr: Position embedding transformation for multi-view 3d object detection. InEuropean conference on computer vision, pages 531–548. Springer, 2022
2022
-
[18]
Petrv2: A unified framework for 3d perception from multi-camera images
Yingfei Liu, Junjie Yan, Fan Jia, Shuailin Li, Aqi Gao, Tiancai Wang, and Xiangyu Zhang. Petrv2: A unified framework for 3d perception from multi-camera images. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 3262–3272, 2023
2023
-
[19]
Is pseudo-lidar needed for monocular 3d object detection? InProceed- ings of the IEEE/CVF International Conference on Computer Vision, pages 3142–3152, 2021
Dennis Park, Rares Ambrus, Vitor Guizilini, Jie Li, and Adrien Gaidon. Is pseudo-lidar needed for monocular 3d object detection? InProceed- ings of the IEEE/CVF International Conference on Computer Vision, pages 3142–3152, 2021
2021
-
[20]
Time will tell: New outlooks and a baseline for temporal multi-view 3d object detection.arXiv preprint arXiv:2210.02443, 2022
Jinhyung Park, Chenfeng Xu, Shijia Yang, Kurt Keutzer, Kris Kitani, Masayoshi Tomizuka, and Wei Zhan. Time will tell: New outlooks and a baseline for temporal multi-view 3d object detection.arXiv preprint arXiv:2210.02443, 2022
-
[21]
Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d
Jonah Philion and Sanja Fidler. Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d. InComputer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIV 16, pages 194–210. Springer, 2020
2020
-
[22]
Hop: History-and-order aware pre-training for vision-and- language navigation
Yanyuan Qiao, Yuankai Qi, Yicong Hong, Zheng Yu, Peng Wang, and Qi Wu. Hop: History-and-order aware pre-training for vision-and- language navigation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15418–15427, 2022
2022
-
[23]
3dppe: 3d point positional encoding for transformer-based multi-camera 3d object detection
Changyong Shu, Jiajun Deng, Fisher Yu, and Yifan Liu. 3dppe: 3d point positional encoding for transformer-based multi-camera 3d object detection. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 3580–3589, 2023
2023
-
[24]
Sparse r-cnn: End-to-end object detection with learnable proposals
Peize Sun, Rufeng Zhang, Yi Jiang, Tao Kong, Chenfeng Xu, Wei Zhan, Masayoshi Tomizuka, Lei Li, Zehuan Yuan, Changhu Wang, et al. Sparse r-cnn: End-to-end object detection with learnable proposals. InProceedings of the IEEE/CVF conference on computer vision and pattern recogni...
2021
-
[25]
Attention is all you need.Advances in neural information processing systems, 30, 2017
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017
2017
-
[27]
Exploring object-centric temporal modeling for efficient multi-view 3d object detection
Shihao Wang, Yingfei Liu, Tiancai Wang, Ying Li, and Xiangyu Zhang. Exploring object-centric temporal modeling for efficient multi-view 3d object detection. InProceedings of the IEEE/CVF international conference on computer vision, pages 3621–3631, 2023
2023
-
[28]
Detr3d: 3d object detection from multi- view images via 3d-to-2d queries
Yue Wang, Vitor Campagnolo Guizilini, Tianyuan Zhang, Yilun Wang, Hang Zhao, and Justin Solomon. Detr3d: 3d object detection from multi- view images via 3d-to-2d queries. InConference on Robot Learning, pages 180–191. PMLR, 2022
2022
-
[29]
Object as query: Lifting any 2d object detector to 3d detection
Zitian Wang, Zehao Huang, Jiahui Fu, Naiyan Wang, and Si Liu. Object as query: Lifting any 2d object detector to 3d detection. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 3791–3800, 2023
2023
-
[30]
Cape: Camera view position embedding for multi-view 3d object detection
Kaixin Xiong, Shi Gong, Xiaoqing Ye, Xiao Tan, Ji Wan, Errui Ding, Jingdong Wang, and Xiang Bai. Cape: Camera view position embedding for multi-view 3d object detection. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 21570– 21579, 2023
2023
-
[31]
Dino: Detr with improved denoising anchor boxes for end-to-end object detection.arXiv preprint arXiv:2203.03605, 2022
Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung-Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection.arXiv preprint arXiv:2203.03605, 2022
2022 arXiv
-
[32]
Class-balanced grouping and sampling for point cloud 3d object detection.arXiv preprint arXiv:1908.09492, 2019
Benjin Zhu, Zhengkai Jiang, Xiangxin Zhou, Zeming Li, and Gang Yu. Class-balanced grouping and sampling for point cloud 3d object detection.arXiv preprint arXiv:1908.09492, 2019
1908 arXiv
-
[33]
Deformable detr: Deformable transformers for end-to-end object detection.arXiv preprint arXiv:2010.04159, 2020
Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection.arXiv preprint arXiv:2010.04159, 2020
2010 arXiv
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.