REVIEW 4 major objections 6 minor 1 cited by
Towards Accurate State Estimation: Motion Dynamics Kalman Filter for 3D Multi-Object Tracking
T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read An adaptive Kalman filter that weights velocity, acceleration, and jerk from an object's recent motion improves 3D tracking accuracy and occlusion recovery at a cost of about 0.078 ms per frame.
desk verdict A plausible adaptive motion-model KF for 3D MOT with real but small gains; missing calibration parameters and an untested multimodel comparison keep it conditional. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the motion-dynamics weighting matrix $\hat{W}_t = \mathrm{diag}(1,\hat{w}_v,\hat{w}_a,\hat{w}_j)$, inserted between the transition matrix $F_t$ and the previous state in the Kalman prediction. Each weight is computed as $\hat{w} = \min(\sigma(\Delta\theta_{t\leftarrow s})/\ell, 1)$, where $\sigma(\Delta\theta_{t\leftarrow s})$ is the sample standard deviation of finite differences of the relevant motion parameter over a window of $k$ measurements, and $\ell$ is a tuned normalization constant. The weights are updated after each measurement using a noise-cleaned position estimate, and the window size $k$ controls how quickly the model switches between velocity-, acceleration-, and jerk-dominated behavior. This machinery turns the Kalman filter from a fixed-order polynomial predictor into a data-adaptive one while keeping a single motion model.
What would settle it
On a dataset with ground-truth trajectories and high detector noise, compute the adaptive weights and measure state-estimation error for each object; if there is a class of maneuvers, such as stop-and-go traffic or sudden lane changes, for which the adaptive-weighted filter has consistently larger error than a constant-velocity filter, the central claim fails.
Extended reading notes
Core claim
The central claim is that the state-transition step itself should be adapted per object per frame. The paper writes the prediction as $\hat{x}_{t|t-1}=F_t\hat{W}_{t-1}\hat{x}_{t-1|t-1}$, where $\hat{W}$ is a diagonal matrix of weights in $[0,1]$ applied to the velocity, acceleration, and jerk components of the motion model. The weights come from a Gaussian estimate of the object's motion dynamics: the standard deviation of finite differences of position and velocity over a smoothing window, normalized by hand-tuned factors $\ell_v,\ell_a,\ell_j$ and clipped to one. With this formulation, the filter automatically behaves like a constant-velocity model when the object moves steadily, like a constant-acceleration model during acceleration, and like a jerk model during rapid maneuvers, without running several models in parallel. The paper reports that this single adaptive model improves state-estimation localization and trajectory quality, especially for occluded and distant objects, and reduces identity switches.
Load-bearing premise
The whole gain rests on the assumption that the standard deviation of recent changes in position, velocity, and acceleration, divided by hand-tuned constants, is a faithful measure of the object's true motion dynamics; if that mapping is unreliable, the adaptive weights can distort predictions instead of correcting them.
Editorial extensions
If this is right
- A tracking pipeline can replace a constant-velocity or constant-acceleration Kalman filter with the weighted prediction $\hat{x}_{t|t-1}=F_t\hat{W}_{t-1}\hat{x}_{t-1|t-1}$ and inherit the accuracy gains without changing its detector or association logic.
- The reported gains persist across five different detectors on KITTI and two detectors on Waymo, so the improvement is tied to the state estimator rather than to one detector's failure modes.
- For occluded targets, the adaptive filter reduces trajectory drift and improves re-identification; the paper reports gains up to 1.22% HOTA and 1.55% MOTA on 20-frame simulated occlusions.
- The added computation is about 0.068–0.094 ms per frame, with an average of 0.078 ms, which is small enough for real-time autonomous-driving stacks.
Reading between the lines
- Editorial inference: because the weighting only uses a scalar time series of positions, the same mechanism could transfer to 2D MOT, pedestrian tracking, or radar tracks, provided the normalization factors are re-tuned.
- Editorial inference: the paper leaves the normalization factors ($\ell_v,\ell_a,\ell_j$) and the smoothing-window size $k$ as hand-set hyperparameters; a principled calibration schedule or a learned mapping from measurement noise to these constants is a natural next step the paper does not explore.
- Editorial inference: if the standard-deviation signal is computed on detections that are themselves noisy, the weights may under-weight fast maneuvers just when they matter; a testable variant would feed the detector's confidence or noise covariance into the normalization.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a motion-dynamics Kalman filter (MD-KF) for 3D multi-object tracking. The prediction step is modified to x̂_{t|t-1} = F_t Ŵ_{t-1} x̂_{t-1|t-1}, where Ŵ is a diagonal matrix of weights applied to the velocity, acceleration, and jerk components of a jerk motion model. The weights are computed as min(σ/ℓ, 1), with σ the sample standard deviation of finite differences of noise-mitigated detections over a smoothing window, and ℓ_v, ℓ_a, ℓ_j hand-defined normalization factors. The method is embedded in RobMOT and evaluated on KITTI and the Waymo Open Dataset. The reported results show small improvements over the RobMOT baseline on KITTI test (+0.07 HOTA, +0.07 MOTA), consistent small gains across five detectors and two motion models, larger gains in simulated occlusion scenarios, and a runtime overhead of about 0.078 ms per frame. The paper claims this outperforms both constant-motion and multimodel Kalman filters at lower computational cost.
Significance. The adaptive single-model idea is attractive and the occlusion-simulation protocol is a useful contribution for evaluating trackers under missing detections. The manuscript provides evaluations with multiple detectors and uses official KITTI and Waymo evaluation tools. However, the claimed advantage over multimodel filters is not experimentally supported, and the main benchmark gains are marginal relative to what could be explained by the unreported calibration parameters. The value of the contribution depends on full parameter disclosure, a noise-propagation analysis, and a direct comparison with a multimodel baseline.
major comments (4)
- [Section 4, Eq. (9) and Section 3.4] The normalization factors ℓ_v, ℓ_a, ℓ_j and the smoothing-window size k are never reported. These parameters determine when the acceleration and jerk terms are activated, so the reported gains cannot be separated from tuning on the same benchmarks. In addition, the paper does not analyze how detection noise propagates through Eq. (7): for a constant-velocity target with zero-mean measurement noise of standard deviation η, σ(Δz) ≈ √2·η and σ(Δ²z) ≈ √6·η on finite differences, so if ℓ_a and ℓ_j are not sufficiently larger than that floor, the filter will spuriously activate acceleration and jerk terms. Please report ℓ and k for all experiments, add a sensitivity analysis, and quantify the noise floor for Eq. (7).
- [Abstract, Section 5.2.1, Table 1 and Table 3] The abstract claims margins of 0.56% and 0.81% in HOTA and MOTA on KITTI and Waymo, but Table 1 shows RobMOT (Dynamic) versus RobMOT (Baseline) with +0.07 HOTA and +0.07 MOTA, and Table 3 reports only MOTA and Miss deltas without any HOTA values. The stated margins are therefore not supported by the experimental tables. The abstract and conclusion should be revised to match the reported numbers, and the missing HOTA results for the Waymo experiments should be added or the claim removed.
- [Abstract, Section 2, Section 5.2] The manuscript claims that MD-KF consistently outperforms constant and multimodel KF and that it reduces the computational cost of multimodel approaches, but no multimodel KF (for example, the interacting multiple model baseline of [4] or [16]) appears in any experiment. Table 4 compares runtime only between RobMOT with the baseline KF and with MD-KF. To support the central comparative claim, the authors should include an IMM or other multimodel baseline in Tables 1, 3, and 4, or restrict the claim to comparison with a single-model constant-motion KF.
- [Section 5.3, Table 5] The large occlusion gains (up to +1.22 HOTA and +1.55 MOTA) come from a simulated protocol in which detections are withheld for 10–20 frames; during the occlusion interval no measurement noise enters the update, so these results do not validate the weighting mechanism under the noisy-detection conditions of normal frames, where the Table 1 margins are only +0.07. The authors should report the variance of the main metrics across runs or a statistical test, and should explicitly state that the occlusion results are conditional on the simulation assumptions and do not transfer directly to the standard benchmark numbers.
minor comments (6)
- [Section 5.2.1] In the text, 'donated by RobMOT (Baseline)' should read 'denoted by RobMOT (Baseline)', and the asterisks in Table 1 are not explained.
- [Figure 7] The caption contains duplicated words ('AccelerationAcceleration JerkJerk') and the plot axes are unlabeled, making it hard to read the HOTA and MOTA values.
- [Equation (7)] The symbol k is used both as the index in the sum and as the total number of observations (k = t - s + 1) and later as the smoothing-window size; please use separate symbols for these quantities.
- [Section 3.3] The statement 'σ(Δz) ∝ acceleration fluctuations' is asserted without derivation; if retained, the exact relation under a constant-acceleration model should be provided.
- [Equations (9)–(10)] Equation (10) is redundant with the min-clipping in Eq. (9) and is not used in the derivation; please remove it or explain the equivalence clearly.
- [Table 3] The color-key legend for Table 3 is informal; please provide numerical values with confidence intervals or standard deviations instead of color thresholds.
Circularity Check
No significant circularity: the adaptive weights are computed from measurement finite differences and tested against independent tracking metrics, not from the claimed outputs.
full rationale
The derivation chain is not circular. The predicted state (Eq. 4) is x̂_{t|t-1} = F_t Ŵ_{t-1} x̂_{t-1|t-1}, where Ŵ is formed from weights computed in Eq. 9: ŵ_v = min(σ(z_{t→k})/ℓ_v, 1), ŵ_a = min(σ(Δz_{t→k})/ℓ_a, 1), ŵ_j = min(σ(Δ²z_{t→k})/ℓ_j, 1). These weights are functions of sample standard deviations of the measured position and its finite differences over a sliding window (Eqs. 7-9), i.e., of past measurements, not of the HOTA/MOTA/IDF1 targets used for evaluation. The fine-tuned factors ℓ_v, ℓ_a, ℓ_j and the window size k are not reported, which is a reproducibility and possible overfitting concern, but no equation reduces the predicted state to these fitted constants by construction, and no reported benchmark score is a renamed fit. The self-citations to RobMOT [1] supply the baseline tracker and the D_t noise-mitigation term; both the baseline and the proposed variant share those components, so the comparison isolates the motion-dynamics weighting. The choice of the post-measurement localization term is justified by a ground-truth comparison on KITTI (Section 3.2), which is an external check, not a circular import. No uniqueness theorem or ansatz is smuggled in via citation, and no prediction is statistically forced by a fitted parameter in the sense of an Eq. X = Eq. Y reduction. The remaining concerns—unreported hyperparameter values and the lack of noise-propagation analysis for Eq. 9—are correctness or reproducibility risks, not circularity.
Assumptions & free parameters
free parameters (2)
- Motion dynamics normalization factor ℓ_v, ℓ_a, ℓ_j =
Not reported
- Smoothing window size k =
Not reported
assumptions (4)
- domain assumption Motion dynamics over a sliding window are Gaussian-distributed
- domain assumption σ(Δθ) approximates the fluctuation of the corresponding motion parameter (velocity or acceleration)
- domain assumption Weighted jerk equation (Eq. 2) with weights in [0,1] interpolates correctly among constant velocity, constant acceleration, and constant jerk models
- domain assumption The post-measurement localization ẑ_t = z_t - H K_t diag(D_t) from [1] removes detector noise
Cite this review
Pith. "Pith review of Towards Accurate State Estimation: Motion Dynamics Kalman Filter for 3D Multi-Object Tracking." pith.science (2026). https://pith.science/paper/6ANRXPJN
@misc{pith2026250507254,
author = {Pith},
title = {Pith review of: Towards Accurate State Estimation: Motion Dynamics Kalman Filter for 3D Multi-Object Tracking},
year = {2026},
howpublished = {\url{https://pith.science/paper/6ANRXPJN}},
note = {Machine review of arXiv:2505.07254}
}
read the original abstract
Precise 3D state estimation in multi-object tracking (MOT) is critical for self-driving cars, particularly for objects occluded. Motion modeling in the Kalman filter with a constant motion assumption is widely used in MOT methods, but it neglects the continuous changes in objects' motion caused by traffic in urban environments. Although recent research introduces a multimodel Kalman filter that incorporates multiple motion models, these approaches incur significant computational overhead from the simultaneous processing of multiple models. To this end, this work introduces a motion-dynamics Kalman filter (MD-KF) that overcomes the constant-motion assumption while preserving the singularity of the motion model. MD-KF models the changes in objects' motion over successive measurements as Gaussian distributions, and adaptively adjusts a weighted motion model to account for these variations. MD-KF consistently outperforms constant and multimodel KF across multiple datasets with a significant reduction in computation latency compared to multimodel approaches. The proposed approach demonstrates its superiority in trajectory estimation during occlusion and state estimation stability for stationary objects.
Figures
Figures from the paper (6 more)
Forward citations
Cited by 1 Pith paper
-
Polycepta: Object-Centric Appearance Estimation for Multi-Object Tracking
Polycepta recursively estimates per-object appearance states so visual cues improve over time, reducing identity switches and lifting tracking-by-detection performance at real-time speed.
Reference graph
Works this paper leans on
-
[4]
Adaptive target tracking with interacting heterogeneous motion models
Ki-In Na, Sunglok Choi, and Jong-Hwan Kim. Adaptive target tracking with interacting heterogeneous motion models. IEEE Transactions on Intelligent Transportation Systems, 23(11):21301–21313, 2022
work page 2022
-
[16]
Sensor- agnostic graph-aware kalman filter for multi-modal multi-object tracking
Depanshu Sani, Anirudh Iyer, Prakhar Rai, Saket Anand, Anuj Srivastava, and Kaushik Kalyanaraman. Sensor- agnostic graph-aware kalman filter for multi-modal multi-object tracking. InInternational Conference on Pattern Recognition, pages 380–398. Springer, 2024
work page 2024
-
[1]
Mohamed Nagy, Naoufel Werghi, Bilal Hassan, Jorge Dias, and Majid Khonji. Robmot: Robust 3d multi-object tracking by observational noise and state estimation drift mitigation on lidar pointcloud, 2024
work page 2024
-
[2]
Are we ready for autonomous driving? the kitti vision benchmark suite
Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In Conference on Computer Vision and Pattern Recognition (CVPR) , 2012
2012
-
[3]
A maneuvering target tracking algorithm based on the interacting multiple models
Liu Yan-Chang and Zuo Xian-Gang. A maneuvering target tracking algorithm based on the interacting multiple models. TELKOMNIKA Indonesian Journal of Electrical Engineering , 11(7):3997–4003, 2013
work page 2013
-
[5]
Mohamed Nagy, Majid Khonji, Jorge Dias, and Sajid Javed. Dfr-fastmot: Detection failure resistant tracker for fast multi-object tracking based on sensor fusion. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 827–833, 2023
work page 2023
-
[6]
3d multi-object tracking in point clouds based on prediction confidence-guided data association
Wu et al. 3d multi-object tracking in point clouds based on prediction confidence-guided data association. IEEE Transactions on Intelligent Transportation Systems, 2022
work page 2022
-
[7]
Msa-mot: Multi-stage association for 3d multimodality multi-object tracking
Ziming Zhu, Jiahao Nie, Han Wu, Zhiwei He, and Mingyu Gao. Msa-mot: Multi-stage association for 3d multimodality multi-object tracking. Sensors, 22(22):8650, 2022
work page 2022
Show all 28 references
-
[8]
3d multi-object tracking based on informatic divergence-guided data association
Jiawei He, Chunyun Fu, Xiyang Wang, and Jianwen Wang. 3d multi-object tracking based on informatic divergence-guided data association. Signal Processing, 222:109544, 2024
2024
-
[9]
Monocular 3d multi-object tracking with an ekf approach for long- term stable tracks
Andreas Reich and Hans-Joachim Wuensche. Monocular 3d multi-object tracking with an ekf approach for long- term stable tracks. In 2021 IEEE 24th International Conference on Information Fusion (FUSION) , pages 1–7, 2021
2021
-
[10]
Towards robust reference system for autonomous driving: Rethinking 3d mot
Leichen Wang, Jiadi Zhang, Pei Cai, and Xinrun Lil. Towards robust reference system for autonomous driving: Rethinking 3d mot. In 2023 IEEE International Conference on Robotics and Automation (ICRA) , pages 8319– 8325, 2023
2023
-
[11]
3d multi-object tracking: A baseline and new evaluation metrics
Xinshuo Weng, Jianren Wang, David Held, and Kris Kitani. 3d multi-object tracking: A baseline and new evaluation metrics. 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 10359–10366, 2019
2020
-
[12]
Immortal tracker: Tracklet never dies
Qitai Wang, Yuntao Chen, Ziqi Pang, Naiyan Wang, and Zhaoxiang Zhang. Immortal tracker: Tracklet never dies. arXiv preprint arXiv:2111.13672, 2021
2021 arXiv
-
[13]
Simpletrack: Understanding and rethinking 3d multi-object tracking
Ziqi Pang, Zhichao Li, and Naiyan Wang. Simpletrack: Understanding and rethinking 3d multi-object tracking. In ECCV Workshops, 2021
2021
-
[14]
Eagermot: 3d multi-object tracking via sensor fusion
Aleksandr Kim, Aljo ˇsa Oˇsep, and Laura Leal-Taix ´e. Eagermot: 3d multi-object tracking via sensor fusion. In 2021 IEEE International Conference on Robotics and Automation (ICRA) , pages 11315–11321, 2021
2021
-
[15]
Mono- camera 3d multi-object tracking using deep learning detections and pmbm filtering
Samuel Scheidegger, Joachim Benjaminsson, Emil Rosenberg, Amrit Krishnan, and Karl Granstr ¨om. Mono- camera 3d multi-object tracking using deep learning detections and pmbm filtering. In 2018 IEEE Intelligent V ehicles Symposium (IV), pages 433–440. IEEE, 2018. 19 A PREPRINT ...
2018
-
[17]
Scalability in perception for autonomous driving: Waymo open dataset
Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aur ´elien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timo- feev, Scott Ettinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Yu Zh...
2020
-
[18]
Triplettrack: 3d object tracking using triplet embeddings and lstm
Nicola Marinello, Marc Proesmans, and Luc Van Gool. Triplettrack: 3d object tracking using triplet embeddings and lstm. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4500–4510, 2022
2022
-
[19]
Aleksandr Kim, Guillem Bras ´o, Aljo ˇsa Oˇsep, and Laura Leal-Taix ´e. Polarmot: How far can geometric rela- tions take us in 3d multi-object tracking? In Shai Avidan, Gabriel Brostow, Moustapha Ciss ´e, Giovanni Maria Farinella, and Tal Hassner, editors, Computer Vision – EC...
2022
-
[20]
Deepfusionmot: A 3d multi-object track- ing framework based on camera-lidar fusion with deep association
Xiyang Wang, Chunyun Fu, Zhankun Li, Ying Lai, and Jiawei He. Deepfusionmot: A 3d multi-object track- ing framework based on camera-lidar fusion with deep association. IEEE Robotics and Automation Letters , 7(3):8260–8267, 2022
2022
-
[21]
3d multi-object tracking in point clouds based on prediction confidence-guided data association
Hai Wu, Wenkai Han, Chenglu Wen, Xin Li, and Cheng Wang. 3d multi-object tracking in point clouds based on prediction confidence-guided data association. IEEE Transactions on Intelligent Transportation Systems , 23(6):5668–5677, 2022
2022
-
[22]
Xiangyang Wu, Yiming Sun, Liangjian He, Wei Li, Yifan Wu, Mingqiang Ding, Changhu Lu, and Ping Li. Virtual sparse convolution for multimodal 3d object detection.https://openaccess.thecvf.com/content/ CVPR2023/supplemental/Wu_Virtual_Sparse_Convolution_CVPR_2023_supplemental.pd...
2023
-
[23]
Pv-rcnn: Point-voxel feature set abstraction for 3d object detection
Shaoshuai Shi, Chaoxu Guo, Li Jiang, Zhe Wang, Jianping Shi, Xiaogang Wang, and Hongsheng Li. Pv-rcnn: Point-voxel feature set abstraction for 3d object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020
2020
-
[24]
Pointrcnn: 3d object proposal generation and detection from point cloud
Shaoshuai Shi, Xiaogang Wang, and Hongsheng Li. Pointrcnn: 3d object proposal generation and detection from point cloud. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2019
2019
-
[25]
Second: Sparsely embedded convolutional detection
Yan Yan, Yuxing Mao, and Bo Li. Second: Sparsely embedded convolutional detection. Sensors, 18(10):3337, 2018
2018
-
[26]
Virtual sparse convolution for multimodal 3d object detection
Wu et al. Virtual sparse convolution for multimodal 3d object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2023
2023
-
[27]
Casa: A cascade attention network for 3-d object detection from lidar point clouds
Wu et al. Casa: A cascade attention network for 3-d object detection from lidar point clouds. IEEE Transactions on Geoscience and Remote Sensing , 2022
2022
-
[28]
Once detected, never lost: Surpassing human performance in offline lidar based 3d object detection
Lue Fan, Yuxue Yang, Yiming Mao, Feng Wang, Yuntao Chen, Naiyan Wang, and Zhaoxiang Zhang. Once detected, never lost: Surpassing human performance in offline lidar based 3d object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pa...
2023
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.