REVIEW 5 major objections 6 minor 3 references
YOLOatr : Deep Learning Based Automatic Target Detection and Localization in Thermal Infrared Imagery
T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read YOLOatr, a modified YOLOv5s, reaches 99.6% mAP on correlated thermal images and 37.7% mAP on longer-range test images.
desk verdict A straightforward YOLOv5s tweak applied to DSIAC, but the evaluation is too inconsistent to support the SOTA claim. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism is a modified YOLOv5s architecture. An extra P2 detection head operates on a higher-resolution feature map so targets that occupy only a few pixels get a dedicated prediction scale; the default PANet neck is replaced with a BiFPN neck that learns weighted bidirectional cross-scale feature fusion. Around this, a custom augmentation profile keeps brightness and contrast variation, keeps mosaic augmentation low so small targets are not shrunk, disables shear, and sets high mixup and copy-paste values. Training is done from scratch on thermal images rather than from ImageNet-pretrained weights, which the paper found slightly better for this modality. The paper interprets this combination as the reason YOLOatr detects small targets in clutter and improves decorrelated-range generalization.
What would settle it
Recompute mAP@0.5 from the saved test predictions using the standard definitions (precision = TP/(TP+FP), recall = TP/(TP+FN)) and compare with Table 4; if the evaluation used the paper's printed formula, the numbers cannot stand. Independently, averaging the four target-wise mAP rows in Table 4 does not reproduce the reported 0.377 'All' value, so that arithmetic discrepancy is checkable directly from the table.
Extended reading notes
Core claim
The central discovery claimed is architectural: adding a small-object head and weighted cross-scale feature fusion to YOLOv5s produces a detector that keeps 99.6% mAP within the training range and generalizes substantially better when the test range is longer. The decisive evidence is the decorrelated protocol, where YOLOv5s collapses to 27.1% mAP while YOLOatr reaches 37.7% mAP, a gain the paper attributes to the extra P2 head and BiFPN neck rather than to more parameters. The gain is uneven across classes, with the T72 tank at 62.2% mAP and the Pickup truck at 16.3% mAP at 3 km, which the paper ties to target size and thermal signature. The paper also reports 110 fps inference with about 7.1 million parameters, making the claimed accuracy compatible with real-time operation.
Load-bearing premise
The central claim rests on the assumption that the evaluation code computed precision and mean average precision correctly, because the paper's printed precision formula is actually accuracy and its two tables disagree on the decorrelated result.
Editorial extensions
If this is right
- At the claimed 110 fps with roughly 7.1 million parameters and 16.4 GFLOPs, the detector's accuracy is achievable in real time on modest GPU hardware.
- Training from scratch on thermal data slightly outperformed ImageNet transfer learning, indicating that RGB pretraining carries little advantage for MWIR automatic target recognition.
- Because the gain over YOLOv5s is concentrated in the decorrelated protocol, the extra P2 head and BiFPN neck act mainly as range-generalization improvements rather than as within-range accuracy boosts.
- Target-wise results show large class disparities at 3 km, so the model's real-world utility depends on target type: heavy tactical vehicles with strong thermal signatures are far better recognized than civilian vehicles.
Reading between the lines
- Because the paper leaves false-alarm rate for future work, an operational evaluation should use false-alarm-rate versus probability-of-detection curves rather than mAP alone.
- An ablation that removes only the BiFPN neck while keeping the P2 head would test which component carries the decorrelated-range gain; the paper reports the combined model but not this decomposition.
- The same head-and-augmentation recipe could be applied to visible-band small-object benchmarks; the paper's evidence is limited to the MWIR dataset, so cross-modality generalization remains untested.
- Stricter evaluation at a higher IoU threshold or under lower signal-to-clutter ratios would test whether the near-perfect correlated score reflects precise localization or mainly generous overlap at the 0.5 threshold.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes YOLOatr, a modified YOLOv5s detector with an added P2 small-object head, a BiFPN neck, and a custom data-augmentation profile, and evaluates it on the public DSIAC MWIR ATR dataset. The authors report state-of-the-art results: 99.6% mAP@0.5 on correlated testing (T1) and 37.7% mAP on decorrelated testing at 3 km (T2), with gains of 0.02 and 11.4 percentage points over baseline YOLOv5s. The central claim is that these architectural and augmentation choices deliver a near-perfect, real-time-capable ATR system that generalizes to unseen ranges.
Significance. If the results were reliable, the paper would provide a practically useful engineering contribution: it uses a public military-relevant benchmark, reports a lean architecture with fast inference, and compares correlated and decorrelated testing. However, the evaluation has load-bearing methodological flaws. Equation (1) defines precision as accuracy, the target-wise T2 mAP scores do not average to the reported 'All' value, Table 5 contains a 73.3% decorrelated mAP that contradicts Table 4, and no code or predictions are released. These issues invalidate the quantitative claims as written. The paper's significance is therefore conditional on a complete re-evaluation and data release; in its current form it cannot substantiate the claimed SOTA performance.
major comments (5)
- [§4.1, Eq. (1)] Equation (1) defines Precision as (TP+TN)/(TP+TN+FP+FN), which is accuracy, not precision. The stated simplification to TP/All Detections is only valid when TN=0, which does not hold in object detection where background is not a scored class. If the evaluation code follows this equation, every precision and mAP value in Tables 4 and 5 is invalid, including the headline 99.6% and 37.7% figures. The central claim therefore rests on an unsupported quantitative basis.
- [§4.3, Table 4 (Right)] The target-wise T2 mAP values are 0.622 (T72), 0.214 (BTR70), 0.393 (SUV), and 0.163 (Pickup). Their mean is 0.348, not the 0.377 reported in the 'All' row. This directly contradicts the definition of mAP in Eq. (3) as the mean over classes. The discrepancy suggests the numbers were computed or assembled inconsistently, and it cannot be resolved from the paper alone.
- [§5, Table 5] Table 5 lists decorrelated testing results for other methods with mAP values of 73.3% (e.g., Cuellar et al., YOLOv3 (SF), testing at 5.0 km after training at 4.0–4.5 km). If those numbers are comparable to the paper's T2 protocol, they are far higher than the reported 37.7% mAP for YOLOatr in Table 4, contradicting the claim that YOLOatr achieves SOTA decorrelated performance. If they are not comparable, the table lacks the necessary protocol annotations to make any comparison meaningful. Either way, the comparative claims in Section 5 are internally inconsistent.
- [§4.3, Tables 1 and 2] All experiments appear to be single runs on a single train/test split, with no error bars, repeated runs, or statistical tests. The claimed improvement over YOLOv5s on correlated T1 testing is only 0.002 in mAP (0.996 vs 0.994), a difference that is well within the random variation expected for a single split. Without variance estimates, the 'near-perfect' and 'SOTA' claims are not supported. This issue is load-bearing because the small gain is presented as the main correlated-range result.
- [General reproducibility] No code, evaluation script, ground-truth predictions, or trained weights are released. Given that Eq. (1) appears to document an incorrect metric and that the reported numbers are internally inconsistent, the absence of release prevents any external verification of the results. Reproducibility is a necessary condition for the empirical claims to be accepted.
minor comments (6)
- [§3.2] The phrase 'two learning methodologies two learning methodologies' duplicates words; it should be corrected to 'two learning methodologies.'
- [§3.1, Table 1] Table 1 states '13 different MWIR Visible,' but the text and the target list describe 10 vehicles and 2 humans, which totals 12 classes. The discrepancy between the table and the text should be resolved.
- [References] The reference '[Millikan et al., 2018]' appears twice with different content: once as a URL to the DSIAC database and once as a CVPR workshop paper. The database URL should be cited separately or removed.
- [§4.3, Table 4] The row 'Performance Gain +0.02% +11.4%' should use percentage points rather than percent when describing differences between mAP values (e.g., '0.2 percentage point' or '11.4 percentage points').
- [§Figure 5 caption] The caption contains a duplicated phrase: 'Target detection at 5000m (Bottom Right, Detection at 5000m range (Bottom Right)'. This should be cleaned up.
- [§3.2] The BiFPN neck is mentioned but the implementation details (e.g., whether learnable weights or a specific number of repeats are used) are not provided, making the architectural modification difficult to reproduce.
Circularity Check
No material circularity: YOLOatr's results are empirical benchmark measurements, not derivations that reduce to their inputs.
full rationale
The paper's central claims are performance measurements on a held-out DSIAC test split after architectural and augmentation modifications to YOLOv5s. There is no derivation chain in which a predicted quantity is defined in terms of the same quantity, no fitted parameter renamed as a prediction, and no load-bearing self-citation: the cited prior work is external (DSIAC, YOLOv5, and other ATR studies), and no argument in the paper depends on an unverified result from the present authors' own previous publications. The custom augmentation profile and head/neck modifications were selected through an ablation-style experimental process and then evaluated on separate testing protocols; even if this introduces mild selection bias, it is not circularity by construction. The internal inconsistency in Equation (1) — precision written as (TP+TN)/(TP+TN+FP+FN) alongside the correct simplification TP/All Detections — is a correctness and reproducibility concern, not a circularity one, because it does not make the reported mAP equivalent to a training input or to the model definition. Likewise, the mismatch between target-wise T2 mAP values and the reported 'All' value is an arithmetic inconsistency, not a definitional reduction. The stated limitations in Section 5 (no genetic-algorithm hyperparameter search, no false-alarm-rate comparison, no head-structure exploration) are honest scoping statements and do not constitute circular steps. Overall, the paper is an empirical benchmark study, so no circularity score above 0 is warranted.
Assumptions & free parameters
free parameters (5)
- mosaic augmentation probability =
0.1
- mixup augmentation probability =
0.4
- copy_paste augmentation probability =
0.5
- input image size =
640x640
- training epochs =
100
assumptions (3)
- domain assumption DSIAC ground-truth annotations are accurate and the four selected vehicle classes represent the ATR problem
- domain assumption Ultralytics YOLOv5 implementation and the described P2/BiFPN modifications are implemented correctly
- domain assumption The metrics are computed with a correct mAP implementation
Cite this review
Pith. "Pith review of YOLOatr : Deep Learning Based Automatic Target Detection and Localization in Thermal Infrared Imagery." pith.science (2026). https://pith.science/paper/SVKQOJ4T
@misc{pith2026250711267,
author = {Pith},
title = {Pith review of: YOLOatr : Deep Learning Based Automatic Target Detection and Localization in Thermal Infrared Imagery},
year = {2026},
howpublished = {\url{https://pith.science/paper/SVKQOJ4T}},
note = {Machine review of arXiv:2507.11267}
}
read the original abstract
Automatic Target Detection (ATD) and Recognition (ATR) from Thermal Infrared (TI) imagery in the defense and surveillance domain is a challenging computer vision (CV) task in comparison to the commercial autonomous vehicle perception domain. Limited datasets, peculiar domain-specific and TI modality-specific challenges, i.e., limited hardware, scale invariance issues due to greater distances, deliberate occlusion by tactical vehicles, lower sensor resolution and resultant lack of structural information in targets, effects of weather, temperature, and time of day variations, and varying target to clutter ratios all result in increased intra-class variability and higher inter-class similarity, making accurate real-time ATR a challenging CV task. Resultantly, contemporary state-of-the-art (SOTA) deep learning architectures underperform in the ATR domain. We propose a modified anchor-based single-stage detector, called YOLOatr, based on a modified YOLOv5s, with optimal modifications to the detection heads, feature fusion in the neck, and a custom augmentation profile. We evaluate the performance of our proposed model on a comprehensive DSIAC MWIR dataset for real-time ATR over both correlated and decorrelated testing protocols. The results demonstrate that our proposed model achieves state-of-the-art ATR performance of up to 99.6%.
Reference graph
Works this paper leans on
-
[1]
[Zhao et al., 2022] Zhao, M., Li, W., Li, L., Hu, J., Ma, P., & Tao, R. (2022). Single-frame infrared small-target detection: A survey. IEEE Geoscience and Remote Sensing Magazine. [Berg et al., 2015] Berg, A., Ahlberg, J., & Felsberg, M. (2015, August). A thermal object tracking benchmark. In 2015 12th IEEE International Conference on Advanced Video and ...
work page 2022
-
[2023]
[Millikan et al., 2018] Millikan, B., Foroosh, H., & Sun, Q. (2018). Deep co nvolutional neural networks with integrated quadratic correlation filters for automatic target recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (pp. 1222-1229). [d'Acremont et al., 2019] d'Acremont, A. , Fablet, R., Baussard,...
work page 2018
-
[2040]
[Arif & Mahalanobis, 2020] Arif, M., & Mahalanobis, A. (2020). Multiple view generation and classi fication of mid-wave infrared images using deep learning. arXiv preprint arXiv:2008.07714. [Arif & Mahalanobis, 2021] Arif, M., & Mahalanobis, A. (2021). Infrared target recognition using realistic training images generated by modifying latent features of an...
work page Pith review arXiv 2020
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.