REVIEW 3 major objections 5 minor 11 references
A Real-Time DETR Approach to Bangladesh Road Object Detection for Autonomous Vehicles
T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read The paper reports the first RT-DETR benchmark on the BadODD Bangladesh road-object dataset, with mAP50 0.41518 on the public test split and 0.28194 on the private split at 22.44 ms average inference.
desk verdict A straightforward RT-DETR fine-tuning report on Bangladesh's BadODD dataset, whose only new quantitative claims are two mAP50 test scores that are undercut by a training/test metrics mix-up in the paper's main table. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is RT-DETR, a Real-Time Detection Transformer that combines a transformer encoder-decoder with efficient hybrid encoding and flexible decoder layers so inference speed can be adjusted without retraining. This architecture lets the paper use self-attention to capture global context in road scenes while keeping 22.44 ms per-image latency, something the original DETR could not do. The other load-bearing element is the BadODD dataset itself, whose 13 classes and public/private split define the metric the paper reports.
What would settle it
Run the published model and hyperparameters against the official BadODD test split and recompute mAP50: if the 0.41518/0.28194 values cannot be reproduced, or if Table III's 0.861 figure is found to use the 5,896-image training set, the central performance claim collapses.
Extended reading notes
Core claim
The paper's central claim is that a fine-tuned RT-DETR achieves mAP50 0.41518 on the public 60% test set and 0.28194 on the private 40% test set of BadODD, with an average 22.44 ms inference time per image, making it the first reported RT-DETR benchmark on this dataset. The authors attribute the dataset's difficulty to class imbalance: frequent classes such as auto-rickshaw and three-wheeler dominate, while rare classes like train and wheelchair have only one or two instances. Per-class results across the full 5,896-image set show high precision and recall on common classes and much weaker performance on rare ones, and the model is noted to struggle with congested scenes where it may predict multiple objects in one space.
Load-bearing premise
The paper's result depends on the reported mAP50 values of 0.41518 and 0.28194 actually being computed on the held-out public and private test splits of BadODD, rather than on training images or a mislabeled split.
Editorial extensions
If this is right
- At 22.44 ms per image, the model runs at roughly 45 frames per second, fast enough for autonomous driving decision loops.
- The reported public/private split scores give future researchers a direct benchmark to beat on BadODD.
- Because rare classes such as train and wheelchair have almost no training instances, adding data or augmentation for those classes should improve overall mAP more than tuning the common classes.
- The explored preprocessing for flares, night images, stains, and blur suggests clear headroom: the authors expect integrating these steps to raise robustness, especially in low-visibility conditions.
Reading between the lines
- If the abstract's test numbers are correct, RT-DETR provides a real-time transformer baseline, but the gap between the 0.861 mAP50 in Table III and the 0.415/0.282 test scores suggests the table may be reporting a different split; confirming split labels is an editorial check before treating these as comparable benchmarks.
- The 22.44 ms figure implies that pruning or quantization of RT-DETR could push it onto embedded automotive hardware while retaining real-time performance.
- The class-imbalance pattern implies that a simple class-balanced reweighting or synthetic data for rare classes might lift mAP50-95 more than architectural changes.
- A direct head-to-head with YOLOv8 on the same BadODD splits would clarify whether the transformer's global attention actually helps on Bangladeshi road scenes or whether the accuracy gap is a cost of the dataset's low-visibility conditions.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript applies the RT-DETR object-detection architecture to the BadODD dataset of Bangladeshi road scenes. The authors report a mAP50 of 0.41518 on a 60% public test split and 0.28194 on a 40% private test split, with an average inference time of 22.44 ms per image. The paper also describes dataset-specific challenges (flares, night images, windshield stains, motion blur), preprocessing and augmentation experiments that were not ultimately integrated, the chosen hyperparameters, and per-class results in Table III that are computed on 5,896 images, the same count as the training set.
Significance. If the reported test-set numbers are accurate, the paper offers the first public RT-DETR benchmark on BadODD and shows that a transformer-based detector can process images in real time on this dataset. The focus on an under-represented geographic domain and the candid discussion of dataset-specific challenges are strengths. However, the manuscript does not provide a reproducible evaluation protocol, a baseline comparison, or a clear separation between training-set and test-set metrics, so the significance is currently limited. The reported mAP scores on the public and private test splits are potentially valuable reference points, but they are not independently auditable from the manuscript as written.
major comments (3)
- [Section III and Table III] Section III introduces Table III as 'the performance metrics for the model' and then quotes test mAP scores of 0.4151 and 0.2891, but Table III reports results on 5,896 images, which is exactly the training-set size stated in Section II-A. The per-class table shows an overall mAP50 of 0.861, far above the abstract's test numbers. This conflation of training-set metrics with held-out test performance makes the central quantitative claim unverifiable. The authors must state explicitly which split Table III refers to, provide per-class results on the actual public and private test splits, and describe the evaluation protocol (checkpoint selection, input resolution, confidence threshold, and any post-processing). The large gap between the Table III mAP and the test mAP also undermines the sentence in Section III that 'these results underscore the model's ability to generalize well'; the gap itself needs discussion.
- [Section III] The paper provides no comparison with a baseline detector. The introduction claims that RT-DETR outperforms YOLO and Fast R-CNN variants, but the experiments do not include any such comparison on BadODD. Without a baseline trained and evaluated under the same protocol, the reader cannot judge whether the reported mAP values and the 22.44 ms inference time are competitive. Adding at least one baseline, such as YOLOv8 or Faster R-CNN, would substantially strengthen the paper's central claim.
- [Abstract and Section III] The two headline numbers, mAP50 0.41518 on the public 60% test set and 0.28194 on the private 40% test set, are not tied to any concrete evaluation procedure. The manuscript does not specify how the 60/40 split is defined, which checkpoint was used, whether the 416x416 training resolution was also used at inference, or what detection threshold was applied. This makes the central quantitative claim non-reproducible. The authors should provide the split configuration, evaluation code or command, and per-class results for both test splits.
minor comments (5)
- [Section II-C] The text states that the RT-DETR architecture expects 416x416 images, but official RT-DETR models commonly use 640x640; please verify this is the resolution actually used and state it consistently in the evaluation protocol.
- [Table II] Table II includes an entry 'loss 0.0ms' as a runtime component, which is not meaningful for inference time; remove it or clarify that it is not part of the per-image inference pipeline.
- [Introduction and References] There are several typos, including 'spotlite' in the Introduction and multiple occurrences of 'it's' where 'its' is intended. Also, reference [10] appears in the reference list but is not cited in the text; please add the citation or remove the reference.
- [Section II-A] The footnotes marked '2', '3', and '4' in Section II-A are not presented as actual footnotes, and the text about the Flare7k, Night-to-Day, and reflection-removal experiments is scattered; consider moving these preprocessing experiments to a dedicated subsection so the reader can distinguish them from the final integrated pipeline.
- [Figure 8] The confusion matrix is referenced but never explained in the text; please describe how to interpret it and what conclusions the authors draw from it.
Circularity Check
No significant circularity: the reported mAP50 values are direct measurements on held-out test splits, not derived quantities or fitted inputs.
full rationale
The paper's central claim is an empirical benchmark result: applying the existing RT-DETR architecture to the BadODD dataset and measuring mAP50 on the public 60% and private 40% test splits ("Our results gave a mAP50 score of 0.41518 in the public 60% test set, and 0.28194 in the private 40% test set"). No quantity is defined in terms of another claim, no fitted parameter is relabeled as a prediction, and the model choice is justified by external prior work (Lv et al., [3]), not by a self-citation chain. The inconsistency between Table III's all-class mAP50 of 0.861 over 5,896 images (the stated training-set size) and the abstract's lower test scores is a correctness and auditability concern about which numbers correspond to held-out evaluation, but it is not circularity: the two sets of numbers are not equal by construction, and the test scores are not inputs to the training procedure. There are no self-citations and no uniqueness or ansatz arguments imported from the authors' own prior work. The derivation chain is therefore self-contained with respect to circularity, even though the evaluation protocol is not fully auditable from the manuscript alone.
Assumptions & free parameters
free parameters (5)
- learning_rate =
0.001
- momentum =
0.937
- batch_size =
16
- warmup_iterations =
3
- epochs =
50
assumptions (3)
- domain assumption The BadODD dataset annotations and the public/private test split are accurate and correctly used.
- domain assumption The RT-DETR implementation and pretrained weights behave as described in the cited RT-DETR paper [3].
- domain assumption mAP is computed according to the standard COCO/VOC protocol.
Cite this review
Pith. "Pith review of A Real-Time DETR Approach to Bangladesh Road Object Detection for Autonomous Vehicles." pith.science (2026). https://pith.science/paper/URMAC5WD
@misc{pith2026241115110,
author = {Pith},
title = {Pith review of: A Real-Time DETR Approach to Bangladesh Road Object Detection for Autonomous Vehicles},
year = {2026},
howpublished = {\url{https://pith.science/paper/URMAC5WD}},
note = {Machine review of arXiv:2411.15110}
}
read the original abstract
In the recent years, we have witnessed a paradigm shift in the field of Computer Vision, with the forthcoming of the transformer architecture. Detection Transformers has become a state of the art solution to object detection and is a potential candidate for Road Object Detection in Autonomous Vehicles. Despite the abundance of object detection schemes, real-time DETR models are shown to perform significantly better on inference times, with minimal loss of accuracy and performance. In our work, we used Real-Time DETR (RTDETR) object detection on the BadODD Road Object Detection dataset based in Bangladesh, and performed necessary experimentation and testing. Our results gave a mAP50 score of 0.41518 in the public 60% test set, and 0.28194 in the private 40% test set.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin. "Attention Is All You Need". arXiv preprint arXiv:1706.03762, 2023
arXiv 2023
-
[2]
End-to-End Object Detection with Transformers
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, Sergey Zagoruyko. "End-to-End Object Detection with Transformers". arXiv preprint arXiv:2005.12872, 2020
arXiv 2005
-
[3]
DETRs Beat YOLOs on Real-time Object Detection
Wenyu Lv, Yian Zhao, Shangliang Xu, Jinman Wei, Guanzhong Wang, Cheng Cui, Yuning Du, Qingqing Dang, Yi Liu. "DETRs Beat YOLOs on Real-time Object Detection". arXiv preprint arXiv:2304.08069, 2023
arXiv 2023
-
[4]
Glenn Jocher, Ayush Chaurasia, Jing Qiu. "Ultralytics YOLO". GitHub repository, version 8.0.0, 2023
work page 2023
-
[5]
BadODD: Bangladeshi Autonomous Driving Object Detection Dataset
Mirza Nihal Baig, Rony Hajong, Mahdi Murshed Patwary, Mohammad Shahidur Rahman, Husne Ara Chowdhury. "BadODD: Bangladeshi Autonomous Driving Object Detection Dataset". arXiv preprint arXiv:2401.10659, 2024
work page Pith review arXiv 2024
-
[6]
Single Image Reflection Removal Exploiting Misaligned Training Data and Network Enhancements
Kaixuan Wei, Jiaolong Yang, Ying Fu, David Wipf, Hua Huang. "Single Image Reflection Removal Exploiting Misaligned Training Data and Network Enhancements". IEEE Conference on Computer Vision and Pattern Recognition, 2019
work page 2019
-
[7]
Flare7K: A Phenomenological Nighttime Flare Removal Dataset
Yuekun Dai, Chongyi Li, Shangchen Zhou, Ruicheng Feng, Chen Change Loy. "Flare7K: A Phenomenological Nighttime Flare Removal Dataset". Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022
work page 2022
-
[8]
Flare7K++: Mixing Synthetic and Real Datasets for Nighttime Flare Removal and Beyond
Yuekun Dai, Chongyi Li, Shangchen Zhou, Ruicheng Feng, Yihang Luo, Chen Change Loy. "Flare7K++: Mixing Synthetic and Real Datasets for Nighttime Flare Removal and Beyond". 2023
work page 2023
Show all 11 references
-
[9]
Nighttime Smartphone Reflective Flare Removal using Optical Center Symmetry Prior
Yuekun Dai, Yihang Luo, Shangchen Zhou, Chongyi Li, Chen Change Loy. "Nighttime Smartphone Reflective Flare Removal using Optical Center Symmetry Prior". Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023
2023
-
[10]
Adverse weather image translation with asymmetric and uncertainty-aware GAN
Jeong-gi Kwak, Youngsaeng Jin, Yuanming Li, Dongsik Yoon, Donghyeon Kim, Hanseok Ko. "Adverse weather image translation with asymmetric and uncertainty-aware GAN". arXiv preprint arXiv:2112.04283, 2021
2021 arXiv
-
[11]
11em plus .33em minus .07em 4000 4000 100 4000 4000 500 `\.=1000 = #1 \@IEEEnotcompsoconly \@IEEEcompsoconly #1 * [1] 0pt [0pt][0pt] #1 * [1] 0pt [0pt][0pt] #1 * \| ** #1 \@IEEEauthorblockNstyle \@IEEEcompsocnotconfonly \@IEEEauthorblockAstyle \@IEEEcompsocnotconfonly \@IEEEco...
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.