{"id":"baebb8b4-f261-4c54-9065-429ccf69e40b","arxiv_id":"2501.08639","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Fine-tuning YOLO detectors on fire and smoke data raised aerial wildfire detection mAP from 45.7% to 79.2% on the authors' AFSE dataset, but did not improve edge-computing metrics.","lead":"Researchers tested whether fine-tuning pretrained object detectors on fire and smoke data improves aerial wildfire detection from drones. The best model reached 79.2% mAP on their small private test set, but transfer learning did not improve speed, power, or energy use on edge hardware.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Scratch versus fine-tune comparison is confounded by different learning rates; the claimed TL benefit is not uniquely attributable to transfer learning.","rationale":"The paper's headline result is an empirical comparison between transfer learning and training from scratch. The reader's identified weakest assumption was AFSE dataset representativeness and annotation quality, which concerns external validity. My concern targets internal validity: the comparison is confounded by hyperparameter differences, so the causal attribution of the mAP improvement to transfer learning is not cleanly established. This is load-bearing because the central claim is precisely that TL improves detection precision; if the scratch baseline was run with a suboptimal learning rate, the conclusion could be overstated. The reader's verdict already notes per-model hyperparameter tuning in the SOTA comparison, but not the scratch versus TL confound. Therefore I partially agree with the reader: the paper is reasonably analyzed, but a different, more fundamental experimental control issue deserves emphasis. The appropriate verdict remains CONDITIONAL (accept only if the controlled comparison and dataset/code release are provided), which is unchanged from the reader's verdict. The concrete test directly settles the concern by equating hyperparameters across conditions. I emphasize that the paper has positive features: it reports negative results for cascaded TL and for TL's effect on edge metrics, and it provides detailed tables of results. These do not, however, resolve the confounding issue at the heart of the primary claim.","tokens_in":15877,"tokens_out":10463,"duration_ms":92823,"concrete_test":"Re-run the YOLOv5n 'Train from scratch' condition from Table VIII for 150 epochs on the AFSE dataset using the exact hyperparameters of the FASDD fine-tuning row (initial learning rate 0.001, batch size 16, image size 640, same scheduler and augmentations), with random initialization. If the resulting mAP@0.5 remains near 45.7, the TL advantage is confirmed; if it rises substantially (e.g., above 70), the reported benefit is partly an artifact of the higher learning rate used for the scratch baseline. Report results over at least three random seeds to also bound seed variance.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central comparative claim in Table VIII (YOLOv5n: Fine-Tune FASDD, 0 frozen, 150 epochs, mAP@0.5 = 79.2% vs Train from scratch, 150 epochs, mAP@0.5 = 45.7%) is not based on a controlled experiment. Table VII lists the initial learning rate for YOLOv5n Fine-Tuning as 0.001, but the scratch rows do not specify a learning rate, so they use the default (typically 0.01 in YOLOv5). Section IV.A states that 'the number of epochs and initial learning rate varied based on the model evaluated as well as the pre-trained data used for TL. This was done to limit overfitting.' Consequently, the observed improvement may be partly due to the lower learning rate, which is known to stabilize training on small datasets, rather than the pretrained initialization itself. The same confound affects the source-dataset comparison: COCO fine-tuning used 300 epochs while FASDD used 150, so the claimed superiority of homogeneous TL is also entangled with epoch count. Without an ablation that keeps the training schedule identical and varies only the initialization, the paper's statement that 'TL applied with Fine-Tuning significantly improves the detection precision compared with training from scratch on the same epoch setting' is not fully supported by the data.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies transfer learning (TL) for UAV-based wildfire flame and smoke detection, using the authors' private Aerial Fire and Smoke Essential (AFSE) dataset as the target and either FASDD or COCO as the source dataset. The authors compare training from scratch with fine-tuning for YOLOv5n and other lightweight YOLO variants, also evaluating a cascaded TL scheme, a comparison against several non-YOLO detectors, and edge-computing metrics (FPS, power, energy-delay product) on a Raspberry Pi 5. The headline empirical finding is that fine-tuning YOLOv5n from FASDD pre-trained weights reaches 79.2% mAP@0.5 on the AFSE test split, versus 45.7% for 150 epochs of scratch training and 69.2% for 600 epochs of scratch training. The paper also reports that TL does not materially change edge inference metrics and that YOLOv5n is nearly twice as fast as YOLO8n/YOLO11n on the CPU-only edge device.","tokens_in":16147,"tokens_out":5250,"duration_ms":46498,"significance":"If the central comparison were properly controlled, the result would be practically valuable: it suggests that homogeneous-domain pre-training can substantially compensate for small annotated datasets in aerial wildfire detection, and it identifies YOLOv5n as an attractive model for CPU-only UAV edge devices. The paper includes real power measurements, a range of YOLO versions and non-YOLO detectors, and a 5-fold cross-validation attempt, which are useful strengths. However, the main accuracy claim is currently supported only by experiments in which the training schedule (learning rate, epoch count) is varied together with the initialization, so the reported improvement cannot be uniquely attributed to transfer learning. The edge-computing portion is better controlled and the null result there is reported honestly.","major_comments":[{"comment":"","section":"IV.A, Table VII, Table VIII"},{"comment":"","section":"IV.B.4, Table XI"},{"comment":"","section":"IV.B.2, Fig. 4"}],"minor_comments":[{"comment":"The sentence \"TL alone did not benefit from the edge computing metrics evaluated\" is garbled; it should read \"TL alone did not benefit the edge computing metrics evaluated.\"","section":"Abstract"},{"comment":"The YOLOv5n test mAP@0.5 is 79.2% in Table VIII but 79.3% in Table XI for what appears to be the same configuration; please reconcile these values or state that they are from different runs/splits.","section":"Table VIII vs Table XI"},{"comment":"In the MERGED row, validation mAP values are shown as dashes, yet the text comments on validation results for the merged case; please fill in these values or clarify why they are omitted.","section":"Table X"},{"comment":"The block of numbers preceding Fig. 4 is difficult to parse; it should be formatted as a table with labeled rows for training condition and columns for mean, variance, and standard deviation of APfire, APsmoke, and mAP@0.5.","section":"Fig. 4"},{"comment":"The layout of Table V is confusing because multiple versions are packed into each row (e.g., \"5 | 6 8 | 9 10 | 11\"); use separate rows or clearer column grouping for each YOLO version.","section":"Table V"},{"comment":"The AFSE dataset is described as developed in the authors' prior work, but there is no statement about its public availability; if it is private, the reproducibility of the headline numbers is limited, and this should be acknowledged.","section":"General"},{"comment":"Reference [29] is an informal blog post for average precision; consider replacing it with a more standard reference, and verify the formatting of the reference for [8].","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a practically relevant problem and contains a useful set of measurements, but the main empirical claim currently rests on a confounded comparison. The edge-computing part is interesting and the negative result on TL's impact on FPS/power is honestly reported. The lack of a controlled ablation and the unclear SOTA-comparison protocol are the main technical issues; both are fixable within the manuscript's scope. I also note that the target dataset is private, which limits external validation, and that several tables/figures would benefit from clearer presentation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short take: this is a useful, honest empirical follow-up to the authors' own prior work, with one important confound in the headline claim.\n\nWhat's new: specific measurements — FASDD-pretrained YOLOv5n hitting 79.2 mAP on a small aerial wildfire set, edge FPS/power/EDP before and after transfer learning, a cascaded TL comparison, and the YOLOv5n-vs-YOLO11n CPU speed gap. None of these appear in the cited literature. The paper also reports negative results cleanly: cascaded TL gives no benefit, and TL does not improve edge metrics. That is the right thing to do and makes the paper more credible than the abstract suggests.\n\nWhat's soft: the central scratch-vs-fine-tune comparison is not controlled. Table VII lists lr0=0.001 for YOLOv5n fine-tuning; the scratch rows use the YOLOv5 default, typically 0.01. So the 45.7 vs 79.2 mAP comparison at 150 epochs confounds initialization with learning rate. The paper even says epochs and learning rate varied by model to limit overfitting, which makes the 'same epoch setting' claim misleading. The FASDD-vs-COCO comparison is also confounded by epoch count (150 vs 300). A controlled ablation holding the schedule fixed and varying only the initialization would fix this. Second, AFSE is a private 282-image dataset and no code or weights are released, so the headline numbers are not independently checkable. Third, the abstract's 'reduces training time' only holds if you ignore the 9.6 hours spent obtaining the FASDD weights; the body qualifies it, but the abstract does not. The SOTA comparison also varies epochs and batch size per model.\n\nNone of this kills the paper. The general TL benefit is well established, the cross-validation section shows reduced variance, and the edge measurements are concrete data practitioners want. But as it stands, the specific quantitative claims should be read as conditional on an ablation that has not been run.\n\nWho it's for: people choosing a lightweight YOLO for CPU-only wildfire detection on a drone, and researchers working on small aerial fire datasets. It deserves a serious referee, but the referee should ask for the controlled ablation and, ideally, code and data release.","headline":"Useful, honest empirical follow-up on transfer learning for CPU-only wildfire detection, but the headline scratch-vs-fine-tune comparison is confounded by different learning rates and the private dataset limits reproducibility.","tokens_in":16664,"tokens_out":2543,"would_cite":false,"duration_ms":24277,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fine-tuning a wildfire detector from a large fire/smoke dataset lifts its accuracy from 45.7% to 79.2% mAP on a small aerial test set.","keywords":["wildfire detection","transfer learning","fine-tuning","YOLOv5n","object detection","edge computing","AFSE dataset","FASDD"],"falsifier":"Train YOLOv5n with the same FASDD-pretrained fine-tuning recipe and evaluate it on an independently annotated aerial wildfire image set the authors did not use; if the fine-tuned model does not beat a from-scratch model trained for the same number of epochs on that set, the transfer-learning advantage claimed here would not generalize.","tokens_in":15691,"feed_emoji":"🔥","tokens_out":5217,"duration_ms":49557,"temperature":0.7,"pith_summary":"The paper argues that transfer learning, specifically fine-tuning a pretrained YOLOv5n detector, is the practical route to accurate aerial wildfire flame-and-smoke detection when only a small annotated dataset is available. On its 282-image Aerial Fire and Smoke Essential (AFSE) target set, fine-tuning from homogeneous Flame and Smoke Detection Dataset (FASDD) weights reaches 79.2% mAP@0.5, whereas 150 epochs of training from scratch reaches 45.7% and even 600 epochs reaches only 69.2%. A heterogeneous source (COCO) helps less than the homogeneous source, a second cascaded transfer stage adds no clear benefit, and transfer learning does not change inference speed, power draw, or energy-delay product on the edge device. The same experiments position YOLOv5n as the fastest lightweight detector on CPU-only hardware, processing images at about twice the rate of newer YOLOv8n and YOLO11n variants.","feed_headline":"Fine-tuning lifts wildfire detection from 45.7% to 79.2% mAP","feed_subtitle":"Pretrained YOLOv5n on a 282-image aerial set beats scratch training and stays fast on CPU-only hardware.","key_machinery":"The load-bearing mechanism is fine-tuning as a two-step transfer-learning procedure: first train a YOLOv5n detector on a large source dataset (COCO or FASDD), then continue training on AFSE with a lowered learning rate, either freezing a number of layers or leaving all layers unfrozen. The comparison that carries the argument is the unfrozen, homogeneous-source case, because that is where the gain is largest. Supporting machinery includes the AFSE target dataset, the 70/15/15 train/validation/test split, mAP@0.5 with IoU threshold 0.5, stratified 5-fold cross-validation for generalizability, and the normalized energy-delay product for edge efficiency.","core_discovery":"The central claim is that a one-stage fine-tuning transfer-learning recipe — start from weights trained on a large fire/smoke dataset, then fine-tune on the small aerial target with all layers unfrozen and a low learning rate — turns a lightweight detector into a usable wildfire detector. On the AFSE test split, YOLOv5n fine-tuned from FASDD achieves 79.2% mAP@0.5, with fire AP 70.0% and smoke AP 88.5%, compared with 45.7% mAP for 150-epoch scratch training and 69.2% for 600-epoch scratch training. Fine-tuning from COCO reaches 64.8%, showing that homogeneous source data matters. Cascaded transfer learning, whether through D-Fire or a merged FASDD+D-Fire pretraining stage, lands at 78.0–80.5% mAP, so the extra stage is not a reliable win over the single FASDD stage. The paper also reports that transfer learning leaves edge-computing metrics essentially unchanged, since the architecture and input size are the same.","pith_inferences":["Because AFSE has only 282 images and one split, the 79.2% figure should be read as a single-split result; an external evaluation on unseen aerial wildfire footage would be needed before treating it as a deployment guarantee.","The near-identical FPS and power before and after transfer learning suggest the accuracy gain is free at runtime, but also imply that any future edge gains must come from architecture or quantization, not from the choice of pretrained weights.","A natural next test is to apply the same FASDD-to-AFSE fine-tuning recipe to other aerially captured fire datasets and check whether the mAP jump persists; the paper's cross-validation variance suggests smaller variation, but not transfer across datasets.","The conclusion that cascaded transfer is unnecessary could be tested by varying the order of source datasets; a larger first-stage set or a closer intermediate domain might still help."],"forward_implications":["Teams with small bespoke wildfire datasets can get usable detectors by fine-tuning from an existing large flame/smoke dataset rather than training from scratch.","The 600-epoch scratch baseline (69.2%) being below the 150-epoch fine-tuned result (79.2%) means transfer learning can save training time while improving accuracy.","Homogeneous pretraining matters: FASDD pretraining beats COCO pretraining by 14.4 mAP points, so source-domain similarity is a first-order choice.","Cascaded transfer is not worth the extra training time; merging datasets into a single pretraining stage is at least as good and cheaper.","On CPU-only edge hardware, YOLOv5n remains competitive and roughly twice as fast as newer nano variants, making it a reasonable default for UAV deployment."],"supporting_citations":[{"why":"Supplies the homogeneous source dataset (FASDD) whose pretrained weights produce the 79.2% mAP result.","marker":"[9]"},{"why":"Supplies the 282-image AFSE target dataset and the prior result this work extends.","marker":"[8]"},{"why":"Provides the YOLOv5 detector whose nano variant is the main model throughout the experiments.","marker":"[7]"},{"why":"Supplies the heterogeneous COCO pretraining comparison showing source-domain similarity matters.","marker":"[28]"},{"why":"Supplies D-Fire, the secondary dataset used in cascaded and merged pretraining tests.","marker":"[10]"},{"why":"Provides the detection toolbox used to instantiate the comparison SOTA detectors.","marker":"[35]"}],"fun_headline_variants":["Fine-tuning boosts wildfire mAP from 45.7% to 79.2%","Transfer learning lifts wildfire detection to 79.2% mAP","YOLOv5n fine-tuned for wildfire: 79.2% mAP","One-stage fine-tuning beats cascaded TL for wildfire mAP","Edge device YOLOv5n hits 79.2% mAP with fine-tuning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The AFSE dataset — 282 un-augmented images from an aerial perspective, with its ground-truth boxes — is representative enough of real wildfire scenes that the measured mAP gains will carry over to other aerial imagery.","fun_headline_variants_meta":{"raw":{"variants":["Fine-tuning boosts wildfire mAP from 45.7% to 79.2%","Transfer learning lifts wildfire detection to 79.2% mAP","YOLOv5n fine-tuned for wildfire: 79.2% mAP","One-stage fine-tuning beats cascaded TL for wildfire mAP","Edge device YOLOv5n hits 79.2% mAP with fine-tuning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000718,"raw_usage":{"total_tokens":3295,"prompt_tokens":1087,"completion_tokens":2208,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":703,"completion_tokens_details":{"reasoning_tokens":2103}},"tokens_in":703,"tokens_out":2208,"duration_ms":17020,"temperature":1.0,"reasoning_tokens":2103,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:20:48.648014+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train YOLOv5n with the same FASDD-pretrained fine-tuning recipe and evaluate it on an independently annotated aerial wildfire image set the authors did not use; if the fine-tuned model does not beat a from-scratch model trained for the same number of epochs on that set, the transfer-learning advantage claimed here would not generalize.","supporting_citations":[{"cited_title":"An automatic fire detection system based on deep convolutional neural networks for low - power, resource-constrained devices,","cited_arxiv_id":null,"evidence_quote":"Supplies D-Fire, the secondary dataset used in cascaded and merged pretraining tests."},{"cited_title":"FASDD: an open-access 100,000-level flame and smoke detection dataset for deep learning in fire detection,","cited_arxiv_id":null,"evidence_quote":"Supplies the homogeneous source dataset (FASDD) whose pretrained weights produce the 79.2% mAP result."},{"cited_title":"Transfer learning enhanced deep learning model for wildfire flame and smoke detection,","cited_arxiv_id":null,"evidence_quote":"Supplies the 282-image AFSE target dataset and the prior result this work extends."},{"cited_title":"YOLOv5 Ultralytics,","cited_arxiv_id":null,"evidence_quote":"Provides the YOLOv5 detector whose nano variant is the main model throughout the experiments."},{"cited_title":"Microsoft COCO: common objects in context,","cited_arxiv_id":null,"evidence_quote":"Supplies the heterogeneous COCO pretraining comparison showing source-domain similarity matters."}],"review_version":1}