REVIEW 4 major objections 6 minor 32 references
Bayesian Approximation-Based Trajectory Prediction and Tracking with 4D Radar
T0 review · 4 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash
Pith's one-line read Bayes-4DRTrack claims that replacing fixed Kalman covariance with MC Dropout variance and adding Doppler association lifts AMOTA on K-Radar from 0.485 to 0.542.
desk verdict Useful integration of known uncertainty tricks for 4D radar MOT, but the headline gain mostly comes from detection-side changes, not the advertised adaptive process noise. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is a Kalman filter whose noise is not fixed. In the prediction step, $N_P=10$ MC Dropout forward passes on a transformer motion model produce predicted states $\hat{p}^i_{t+1}$; their empirical mean $\mu_{\hat{p}_{t+1}}$ becomes the predicted state and their empirical variance $\sigma^2_{\hat{p}_{t+1}}$ (Eq. 2) becomes the process noise covariance. Detection uses the same idea: $N_D=10$ MC Dropout samples, IoU clustering, and Loss Attenuation (Eq. 1) that weights the regression loss by an input-dependent observation noise $\sigma(x_i)$. Matching runs in two stages, first Mahalanobis distance with the dynamically updated covariance, then a Doppler velocity affinity $A_R(i,j)=\exp(-\|\Delta v_{ij}\|^2 / 2\sigma^2)$, so objects with similar positions but different radial velocities are separated.
What would settle it
Measure the calibration of the MC Dropout variance on K-Radar: compute the empirical squared error of $\mu_{\hat{p}_{t+1}}$ against the true next state and compare it with $\sigma^2_{\hat{p}_{t+1}}$ across tracks, for instance by binning predictions by predicted variance and checking that observed error matches. If the variance is systematically miscalibrated, or if permuting variance values across tracks (removing per-track information) leaves AMOTA at 0.542, then the reported gain is not caused by per-track uncertainty estimation.
Extended reading notes
Core claim
The paper's core discovery, stated on its own terms, is that a tracking-by-detection 3D MOT system for 4D radar reaches 0.542 AMOTA on K-Radar by applying Bayesian approximation, via MC Dropout and Loss Attenuation, to both the RTNH detector and a transformer-based motion predictor; using the sample variance of $N_P=10$ MC Dropout trajectory predictions as the Kalman filter's process noise covariance; and adding a Doppler-velocity association stage after Mahalanobis matching. The ablation results attribute the largest gain to this adaptive prediction variance: with Loss Attenuation in detection, MC Dropout prediction plus estimated variance moves AMOTA from 0.515 to 0.542, whereas feeding detection variance into measurement noise slightly lowers it to 0.537. The full system also reports the best true-positive count (11,682) and the fewest identity switches (246) among the methods compared in Table III.
Load-bearing premise
The load-bearing premise is that the sample variance of the $N_P=10$ MC Dropout trajectory predictions is a calibrated measure of how wrong the predicted position is, so feeding it into the Kalman filter as process noise genuinely adapts the filter rather than merely rescaling its gains.
Editorial extensions
If this is right
- Replacing fixed process noise with MC-Dropout-derived variance raises AMOTA from 0.495 to 0.542 in the full configuration, indicating the adaptive variance is the largest single contributor.
- Doppler-based two-stage association consistently increases true positives and decreases false positives relative to Mahalanobis-only matching in the reported ablation.
- Loss Attenuation in detection alone lifts the constant-velocity baseline from 0.492 to 0.515 AMOTA.
- With a three-frame input horizon the transformer motion model outperforms the constant-velocity model; longer horizons of four or five frames degrade AMOTA.
- The full Bayes-4DRTrack outperforms AB3DMOT, Probabilistic 3D MOT, and SimpleTrack on K-Radar, beating the strongest of those baselines by 4.5 AMOTA points.
Reading between the lines
- If the MC Dropout variance is calibrated, the same adaptive-noise recipe could transfer to camera- or LiDAR-based trackers, whose Kalman filters also use fixed covariances; the paper does not test that transfer.
- The Doppler association stage is likely to matter most in dense traffic with many similar-position targets at different speeds; a stress test on highway scenes with high relative velocities would isolate its contribution.
- A direct calibration check of Eq. (2), comparing predicted variance against empirical squared error on held-out K-Radar tracks, would tell whether the AMOTA gain comes from well-calibrated uncertainty or merely from a variance schedule that happens to help the filter.
- The asymmetry reported in the ablation, where prediction variance helps and detection variance hurts, suggests that over-trusting noisy detections is worse than being conservative in motion; the paper asserts this asymmetry but does not explain it.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Bayes-4DRTrack, a 3D multi-object tracking system for 4D radar built on a tracking-by-detection pipeline. The detector is RTNH with MC Dropout and Loss Attenuation; the prediction stage is a transformer-based motion prediction network with MC Dropout, whose sample variance is used as an adaptive process-noise covariance for the Kalman filter; and data association combines Mahalanobis distance with a Doppler-based velocity affinity. The method is evaluated on the K-Radar dataset, reporting an AMOTA of 0.542 against 0.485 for a constant-velocity Mahalanobis baseline, described as a 5.7% gain. Ablations examine input horizon, two-stage association, and the contributions of Bayesian approximation on the detection and prediction sides.
Significance. If the reported results are statistically reliable, Bayes-4DRTrack would be a useful demonstration that 4D radar Doppler information and learned motion models can improve tracking in adverse weather. The paper has notable strengths: the use of a common detector (RTNH) across all compared tracking methods is a fair-control design; the formulas for Loss Attenuation, MC Dropout averaging, and the association cost are standard and correctly stated; and the ablation study is structured to separate detection-side and prediction-side contributions. However, the headline claim is ambiguous (percentage points vs. relative percent), and the specific contribution of the adaptive prediction variance—which the paper presents as a core novelty—is very small in the ablation (about 0.003 AMOTA) and is reported without error bars or significance tests. The paper also omits key experimental details needed to assess reproducibility, including the dataset split, training protocol for the prediction network, and several hyperparameter values. The contribution is therefore plausible but not yet substantiated at the level claimed.
major comments (4)
- [Section IV-B.3, Table II] The central claim that using MC Dropout variance from the prediction step as process noise 'meaningfully boosts tracking performance' is not supported by the ablation. In Table II, rows 7–10, the baseline with detection-side MC Dropout and Loss Attenuation achieves 0.537; adding MC Dropout prediction gives 0.539; adding the estimated variance V gives 0.542. The marginal contribution of V is therefore 0.003 AMOTA (and 0.001 in rows 2–3 and 0.002 in rows 5–6). These differences are within the range that could arise from the stochasticity of MC Dropout alone, and the paper reports no repeated runs, error bars, or significance tests. The authors should either provide statistical evidence that the V component is a real improvement or reframe the paper's contribution away from adaptive prediction noise.
- [Abstract and Section IV.C] The abstract's '5.7% gain' is an absolute percentage-point difference: 0.542 minus 0.485 equals 0.057, not a 5.7% relative improvement, which would be about 11.8%. The same issue appears in Section IV.C, where a '4.5%' gain over SimpleTrack is also absolute percentage points (0.542 vs. 0.497). The text should consistently say 'percentage points' and should clearly state the selected baseline; against the stronger SimpleTrack baseline in Table III the gain is 4.5 percentage points, not 5.7. This matters because the comparison baseline affects how the contribution is perceived.
- [Section III-C and Section IV-B] The manuscript does not provide enough experimental detail to reproduce or fully assess the prediction network. There is no description of the transformer architecture, the loss function, the dropout rate or placement, the optimizer, the number of training epochs, or the train/validation/test split of K-Radar. In particular, if input horizon n=3 and other hyperparameters were selected on the same data used for the final evaluation, the comparisons in Table I may overstate the advantage. The authors should report the evaluation protocol and training split, and ideally provide code or a detailed configuration.
- [Section III-C, Eq. (2) and Section IV-B.3] The MC Dropout predictive variance from Eq. (2) is used as the Kalman filter's process noise covariance with no calibration check. A raw sample variance over NP stochastic forward passes is only meaningful as a covariance if the network's uncertainty is well calibrated; the paper provides no reliability diagrams, coverage analysis, or comparison of predicted variance against actual trajectory errors. Since the entire adaptive-noise component rests on this variance, the authors should add a calibration study or at least verify that the estimated variance is correlated with prediction error, rather than simply reporting a small AMOTA change.
minor comments (6)
- [Table II] The meaning of 'V' in the table header is ambiguous: in rows 3, 6, 10, and 12 it appears to denote prediction variance used as process noise, while in rows 8 and 11 it denotes detection variance used as measurement noise. The table should use distinct labels, such as V_pred and V_det, to make the ablation interpretable at a glance.
- [Section III-D] The two-stage association description is internally inconsistent with the final cost formula. The text says Stage 1 uses Mahalanobis distance and Stage 2 refines unmatched tracks with Doppler, but then defines a final association cost A(i,j) = w1*DM(i,j) + w2*(1-AR(i,j)), which suggests a combined cost for all pairs. Please clarify whether the matching is sequential or joint, and specify the matching algorithm (e.g., Hungarian or greedy) and any gating thresholds.
- [Section III-D] The Doppler affinity term AR(i,j) uses a velocity difference Δv_ij but does not define whether this is a full 3D velocity vector, a radial Doppler velocity, or a projected component. Since 4D radar provides radial velocity, the definition should be explicit, and the values of w1, w2, and the Doppler sigma should be reported.
- [References] Reference [32] is cited for the AMOTA/AMOTP metrics, but the cited paper (Henriques et al., ECCV 2012) is about circulant tracking kernels. The appropriate citation is likely Weng et al. [10] or the nuScenes benchmark where AMOTA is defined.
- [Section III-C] Eq. (2) uses the sample variance with a 1/NP normalization, but it is not stated whether the predicted states p_i include the orientation θ and how the variance for θ is combined with the positional variances when forming the Kalman process-noise matrix. Please specify the state vector and the exact mapping from (μ, σ^2) to Q.
- [Section IV-B.1] Table I explores input horizons n=2,...,5, but the paper does not explain how tracks shorter than n frames are initialized or handled during the first frames (e.g., whether a constant-velocity fallback is used). This detail is needed to interpret the performance at different horizons.
Circularity Check
No significant circularity; the reported AMOTA gains are an empirical comparison, not a derivation that reduces to its inputs.
full rationale
The paper's central claim is an empirical AMOTA comparison on the K-Radar benchmark. Equation (2) defines the mean and variance of MC Dropout trajectory samples, and using that variance as process noise in the Kalman filter is a design choice, not a definitional identity with the tracked output. The ablation in Table II shows that the estimated-variance component adds only 0.003 AMOTA (0.539 to 0.542), which undermines the authors' interpretive claim that 'properly modeling uncertainty in the prediction step can meaningfully boost tracking performance,' but this is an overstatement, not circularity. Self-citations to [2] for the K-Radar dataset and RTNH detector are normal use of prior work; all comparison methods share the same detector, which makes the comparison fair rather than biased. No step in the derivation chain is equivalent by construction to its inputs, and no fitted parameter is renamed as a prediction. A calibration check of the MC Dropout variance would strengthen the paper, but its absence is a robustness concern, not a circularity concern.
Assumptions & free parameters
free parameters (5)
- Input horizon n =
3
- MC Dropout sample counts ND, NP =
10, 10
- Association weights w1 and w2 =
not reported
- Doppler affinity sigma =
not reported
- Kalman fixed noise covariance value =
not reported
assumptions (3)
- domain assumption MC Dropout samples approximate the posterior predictive distribution of the detector and the motion predictor, so the sample variance is a valid uncertainty measure.
- domain assumption The RTNH detector from prior work [2] remains accurate when extended with loss attenuation and MC Dropout, and the K-Radar dataset annotations are reliable ground truth.
- standard math The standard Kalman filter with constant-velocity model is an appropriate baseline, and replacing its fixed process noise with MC Dropout variance is a valid operation.
Cite this review
Pith. "Pith review of Bayesian Approximation-Based Trajectory Prediction and Tracking with 4D Radar." pith.science (2026). https://pith.science/paper/WC2SB5QV
@misc{pith2026250201357,
author = {Pith},
title = {Pith review of: Bayesian Approximation-Based Trajectory Prediction and Tracking with 4D Radar},
year = {2026},
howpublished = {\url{https://pith.science/paper/WC2SB5QV}},
note = {Machine review of arXiv:2502.01357}
}
read the original abstract
Accurate 3D multi-object tracking (MOT) is vital for autonomous vehicles, yet LiDAR and camera-based methods degrade in adverse weather. Meanwhile, Radar-based solutions remain robust but often suffer from limited vertical resolution and simplistic motion models. Existing Kalman filter-based approaches also rely on fixed noise covariance, hampering adaptability when objects make sudden maneuvers. We propose Bayes-4DRTrack, a 4D Radar-based MOT framework that adopts a transformer-based motion prediction network to capture nonlinear motion dynamics and employs Bayesian approximation in both detection and prediction steps. Moreover, our two-stage data association leverages Doppler measurements to better distinguish closely spaced targets. Evaluated on the K-Radar dataset (including adverse weather scenarios), Bayes-4DRTrack demonstrates a 5.7% gain in Average Multi-Object Tracking Accuracy (AMOTA) over methods with traditional motion models and fixed noise covariance. These results showcase enhanced robustness and accuracy in demanding, real-world conditions.
Figures
Reference graph
Works this paper leans on
-
[1]
Vehicle detection and tracking in adverse weather using a deep learning framework,
M. Hassaballah, M. A. Kenk, K. Muhammad, and S. Minaee, “Vehicle detection and tracking in adverse weather using a deep learning framework,” IEEE transactions on intelligent transportation systems , vol. 22, no. 7, pp. 4230–4242, 2020
work page 2020
-
[2]
K-radar: 4d radar object detection for autonomous driving in various weather conditions,
D.-H. Paek, S.-H. Kong, and K. T. Wijaya, “K-radar: 4d radar object detection for autonomous driving in various weather conditions,” Advances in Neural Information Processing Systems , vol. 35, pp. 3819–3829, 2022
work page 2022
-
[3]
Detection and tracking on automotive radar data with deep learning,
J. F. Tilly, S. Haag, O. Schumann, F. Weishaupt, B. Duraisamy, J. Dickmann, and M. Fritzsche, “Detection and tracking on automotive radar data with deep learning,” in 2020 IEEE 23rd International Conference on Information Fusion (FUSION) . IEEE, 2020, pp. 1–7
work page 2020
-
[4]
J.-H. Kim, M.-C. Lee, and T.-S. Lee, “Deep-learning based multi- object detection and tracking using range-angle map in automo- tive radar systems,” in 2022 IEEE 95th Vehicular Technology Conference:(VTC2022-Spring). IEEE, 2022, pp. 1–6
work page 2022
-
[5]
Tj4dradset: A 4d radar dataset for au- tonomous driving,
L. Zheng, Z. Ma, X. Zhu, B. Tan, S. Li, K. Long, W. Sun, S. Chen, L. Zhang, M. Wan et al. , “Tj4dradset: A 4d radar dataset for au- tonomous driving,” in 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2022, pp. 493–498
work page 2022
-
[6]
Multi- class road user detection with 3+ 1d radar in the view-of-delft dataset,
A. Palffy, E. Pool, S. Baratam, J. F. Kooij, and D. M. Gavrila, “Multi- class road user detection with 3+ 1d radar in the view-of-delft dataset,” IEEE Robotics and Automation Letters , vol. 7, no. 2, pp. 4961–4968, 2022
2022
-
[7]
D.-H. Paek, S.-H. Kong, and K. T. Wijaya, “Enhanced k-radar: Optimal density reduction to improve detection performance and accessibility of 4d radar tensor-based object detection,” in 2023 IEEE Intelligent Vehicles Symposium (IV) . IEEE, 2023, pp. 1–6
work page 2023
-
[8]
A new approach to linear filtering and prediction problems,
R. E. Kalman, “A new approach to linear filtering and prediction problems,” 1960
1960
Show all 32 references
-
[9]
Attention is all you need,
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017
2017
-
[10]
3d multi-object tracking: A baseline and new evaluation metrics,
X. Weng, J. Wang, D. Held, and K. Kitani, “3d multi-object tracking: A baseline and new evaluation metrics,” in 2020 IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems (IROS) . IEEE, 2020, pp. 10 359–10 366
2020
-
[11]
Probabilistic 3d multi- modal, multi-object tracking for autonomous driving,
H.-K. Chiu, J. Li, R. Ambrus ¸, and J. Bohg, “Probabilistic 3d multi- modal, multi-object tracking for autonomous driving,” in 2021 IEEE International Conference on Robotics and Automation (ICRA) , 2021, pp. 14 227–14 233
2021
-
[12]
Simpletrack: Understanding and rethinking 3d multi-object tracking,
Z. Pang, Z. Li, and N. Wang, “Simpletrack: Understanding and rethinking 3d multi-object tracking,” in European Conference on Computer Vision. Springer, 2022, pp. 680–696
2022
-
[13]
Score refinement for confidence-based 3d multi-object tracking,
N. Benbarka, J. Schr ¨oder, and A. Zell, “Score refinement for confidence-based 3d multi-object tracking,” in 2021 IEEE/RSJ Inter- national Conference on Intelligent Robots and Systems (IROS). IEEE, 2021, pp. 8083–8090
2021
-
[14]
Dropout as a bayesian approximation: Representing model uncertainty in deep learning,
Y . Gal and Z. Ghahramani, “Dropout as a bayesian approximation: Representing model uncertainty in deep learning,” in international conference on machine learning . PMLR, 2016, pp. 1050–1059
2016
-
[15]
What uncertainties do we need in bayesian deep learning for computer vision?
A. Kendall and Y . Gal, “What uncertainties do we need in bayesian deep learning for computer vision?” Advances in neural information processing systems, vol. 30, 2017
2017
-
[16]
Center-based 3d object detec- tion and tracking,
T. Yin, X. Zhou, and P. Krahenbuhl, “Center-based 3d object detec- tion and tracking,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2021, pp. 11 784–11 793
2021
-
[17]
3dmodt: Attention-guided affinities for joint detection & tracking in 3d point clouds,
J. Kini, A. Mian, and M. Shah, “3dmodt: Attention-guided affinities for joint detection & tracking in 3d point clouds,” in 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 841–848
2023
-
[18]
Monocular quasi-dense 3d object tracking,
H.-N. Hu, Y .-H. Yang, T. Fischer, T. Darrell, F. Yu, and M. Sun, “Monocular quasi-dense 3d object tracking,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 45, no. 2, pp. 1992– 2008, 2022
1992
-
[19]
Cc-3dt: Panoramic 3d object tracking via cross-camera fusion,
T. Fischer, Y .-H. Yang, S. Kumar, M. Sun, and F. Yu, “Cc-3dt: Panoramic 3d object tracking via cross-camera fusion,” arXiv preprint arXiv:2212.01247, 2022
2022 arXiv
-
[20]
Mutr3d: A multi-camera tracking framework via 3d-to-2d queries,
T. Zhang, X. Chen, Y . Wang, Y . Wang, and H. Zhao, “Mutr3d: A multi-camera tracking framework via 3d-to-2d queries,” in Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 4537–4546
2022
-
[21]
Standing between past and future: Spatio-temporal modeling for multi-camera 3d multi-object tracking,
Z. Pang, J. Li, P. Tokmakov, D. Chen, S. Zagoruyko, and Y .-X. Wang, “Standing between past and future: Spatio-temporal modeling for multi-camera 3d multi-object tracking,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2023, pp. 17 928–17 938
2023
-
[22]
Gnn3dmot: Graph neural network for 3d multi-object tracking with 2d-3d multi-feature learning,
X. Weng, Y . Wang, Y . Man, and K. M. Kitani, “Gnn3dmot: Graph neural network for 3d multi-object tracking with 2d-3d multi-feature learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020, pp. 6499–6508
2020
-
[23]
Joint multi-object detection and tracking with camera-lidar fusion for autonomous driving,
K. Huang and Q. Hao, “Joint multi-object detection and tracking with camera-lidar fusion for autonomous driving,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2021, pp. 6983–6989
2021
-
[24]
Cross-modal 3d object detection and tracking for auto-driving,
Y . Zeng, C. Ma, M. Zhu, Z. Fan, and X. Yang, “Cross-modal 3d object detection and tracking for auto-driving,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2021, pp. 3850–3857
2021
-
[25]
Simple and scalable predictive uncertainty estimation using deep ensembles,
B. Lakshminarayanan, A. Pritzel, and C. Blundell, “Simple and scalable predictive uncertainty estimation using deep ensembles,” Advances in neural information processing systems , vol. 30, 2017
2017
-
[26]
Deep multi-view depth estimation with predicted uncertainty,
T. Ke, T. Do, K. Vuong, K. Sartipi, and S. I. Roumeliotis, “Deep multi-view depth estimation with predicted uncertainty,” in 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021, pp. 9235–9241
2021
-
[27]
Stochas- tic segmentation networks: Modelling spatially correlated aleatoric un- certainty,
M. Monteiro, L. Le Folgoc, D. Coelho de Castro, N. Pawlowski, B. Marques, K. Kamnitsas, M. van der Wilk, and B. Glocker, “Stochas- tic segmentation networks: Modelling spatially correlated aleatoric un- certainty,” Advances in neural information processing systems, vol. 33, pp...
2020
-
[28]
Towards safe autonomous driving: Capture uncertainty in the deep neural network for lidar 3d vehicle detection,
D. Feng, L. Rosenbaum, and K. Dietmayer, “Towards safe autonomous driving: Capture uncertainty in the deep neural network for lidar 3d vehicle detection,” in 2018 21st international conference on intelligent transportation systems (ITSC) . IEEE, 2018, pp. 3266–3273
2018
-
[29]
Capturing object detection uncertainty in multi-layer grid maps,
S. Wirges, M. Reith-Braun, M. Lauer, and C. Stiller, “Capturing object detection uncertainty in multi-layer grid maps,” in 2019 IEEE Intelligent Vehicles Symposium (IV) . IEEE, 2019, pp. 1520–1526
2019
-
[30]
Model uncertainty guides visual object tracking,
L. Zhou, A. Ledent, Q. Hu, T. Liu, J. Zhang, and M. Kloft, “Model uncertainty guides visual object tracking,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 35, no. 4, 2021, pp. 3581– 3589
2021
-
[31]
A review and comparative study on probabilistic object detection in autonomous driving,
D. Feng, A. Harakeh, S. L. Waslander, and K. Dietmayer, “A review and comparative study on probabilistic object detection in autonomous driving,” IEEE Transactions on Intelligent Transportation Systems , vol. 23, no. 8, pp. 9961–9980, 2021
2021
-
[32]
Exploiting the circulant structure of tracking-by-detection with kernels,
J. F. Henriques, R. Caseiro, P. Martins, and J. Batista, “Exploiting the circulant structure of tracking-by-detection with kernels,” in Com- puter Vision–ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part IV 12 . Spri...
2012
Reviewed August 9, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.