{"id":"81a17822-df70-4d26-bd7c-de1d79cca429","arxiv_id":"2506.21451","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"The authors present a 7,049-image thermal dataset of underground miners in five posture classes and report transfer-learning baselines, with best mAP50 around 80%.","lead":"This paper introduces a thermal imaging dataset of 7,049 images of underground miners in five postures and evaluates several YOLO and RT-DETR models on it. It matters as a potential benchmark resource for thermal miner detection in emergency response and search-and-rescue robotics.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Random frame-level split of video-derived thermal frames likely leaks near-duplicate scenes into both training and validation, so the headline mAP50 (80.2) may overstate transfer-learning performance.","rationale":"The reader's conditional verdict already requires a held-out evaluation and data access; I agree with those requirements but locate the most acute threat one step earlier. Because every image is a video frame, the train/validation split must be grouped by source sequence, otherwise the reported 80.2 mAP50 and the transfer-learning improvement in Table 3 are not trustworthy evidence that the dataset transfers to unseen mine scenes. The representativeness concern is real but secondary: even a perfectly representative dataset cannot be judged if the evaluation protocol permits frame-level leakage. The proposed re-split is a small, concrete check that either confirms the headline numbers or exposes their fragility, so no verdict change is needed beyond the conditional acceptance already recommended.","tokens_in":7361,"tokens_out":6173,"duration_ms":64214,"concrete_test":"Re-split the dataset with all frames from the same source video/scene kept in the same fold (release per-frame video or session IDs), retrain YOLO11-n, YOLO11-l, and RT-DETR-X with the Section 4 hyperparameters, and compare mAP50 and F1 with Tables 3 and 4. If the sequence-grouped mAP50 drops by more than about 5 points, the reported numbers are inflated by temporal leakage; if it stays within seed-to-seed variability, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.4 states that images were extracted as frames from Spot CAM+IR video streams, and Section 4 splits the 7,049 frames into 4,584 training and 2,465 validation images without stating that the split is grouped by source video, scene, or session. Consecutive frames from a staged mine sequence are near-duplicates: same miner, pose, background, and thermal conditions. If the split is random at frame level, many validation frames have almost identical training counterparts, so Tables 3 and 4 may measure near-duplicate frame reconstruction rather than detection of unseen miners or new emergency scenarios. The transfer-learning gain in Table 3 (e.g., YOLO11-x from 57.1 to 79.0 mAP50) is the central evidence for the dataset's utility, and the 'no overfitting' reading of Fig. 5 does not rule out leakage because a leaked validation set is also easy. This internal evaluation-protocol issue should be settled before the single-site representativeness question from Section 3.2 can be meaningfully assessed.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Thermal UHD, a thermal imaging dataset of 7,049 frames of underground miners in five postures (lying, bending, sitting, squatting, standing), collected at the Missouri S&T Experimental Mine with a Spot CAM+IR camera mounted on a Boston Dynamics Spot robot. The dataset spans three phases: work with heat/smoke, normal work, and rest. The authors evaluate YOLOv8, YOLOv10, YOLO11, and RT-DETR variants with and without transfer learning, reporting that transfer learning on this dataset improves detection, with YOLO11-l reaching 80.2% mAP50 on the validation split. The paper claims the dataset fills a gap in publicly accessible thermal datasets for underground miner detection and could support emergency response research.","tokens_in":7596,"tokens_out":3370,"duration_ms":36552,"significance":"The dataset addresses a real and under-served need: public thermal imagery of underground miners in emergency-relevant conditions. The three-phase collection (heat/smoke, normal work, rest) and five posture classes are sensible design choices, and the authors obtained IRB approval and used staged volunteers. The baseline evaluation across multiple YOLO variants and RT-DETR provides a useful reference point. The transfer-learning comparison against random initialization is a defensible protocol and the paper honestly discusses limitations such as class imbalance and posture misclassification. If the dataset is released and the evaluation protocol is tightened, this could be a valuable community resource for mining safety and thermal person detection research.","major_comments":[{"comment":"Section 3.4 states that images were extracted as frames from Spot CAM+IR video streams, and Section 4 reports a split of 4,584 training and 2,465 validation images without specifying whether the split groups frames by source video, scene, or session. Consecutive frames from the same staged sequence are near-duplicates (same miner, posture, background, and thermal conditions). If the split is frame-level random, many validation frames will have nearly identical counterparts in training, so the mAP50 and F1 scores in Tables 3 and 4 may largely reflect near-duplicate-frame memorization rather than detection of unseen miners or new scenarios. The authors should split by video/scene/session (with an appropriate temporal gap) and rerun the experiments, or provide evidence that the reported results are robust to a grouped split.","section":"Sections 3.4 and 4"},{"comment":"All results are single-run evaluations on the validation split; there is no separate test set, no repeated runs, and no error bars. The reported differences between models are often small (e.g., YOLO11-l at 80.2% vs YOLO11-n at 80.1% in Table 4) and cannot be distinguished from noise. The authors should report mean and standard deviation over multiple training runs with different seeds, and evaluate on a held-out test split or via cross-validation. This is necessary to support the central transfer-learning claim and any model ranking.","section":"Section 4, Tables 3 and 4"},{"comment":"Data were collected at a single experimental mine with staged volunteers and one thermal camera model. The abstract and conclusion describe the dataset as a foundation for reliable thermal-based miner detection in real emergencies, but the paper provides no external validation or evidence that models trained on this dataset transfer to other mines, different sensors, or unscripted operational conditions. The representativeness limitation should be explicitly acknowledged and the claims softened, or the authors should add at least one external validation scenario.","section":"Sections 3.2 and 6"},{"comment":"The manuscript does not state where or how the dataset can be accessed, nor does it mention a license, a DOI, or a release URL. For a dataset contribution, public availability is essential. The authors should provide a download link, license, and dataset documentation including annotation format, sequence or session metadata, and the exact split definition.","section":"General (dataset availability)"}],"minor_comments":[{"comment":"The text says 'ROS Neotic'; the correct name is 'ROS Noetic.'","section":"Section 3.1, Table 1"},{"comment":"The caption says 'one of the YOLO11 variants' without identifying which variant; please specify the exact model used for the confusion matrix.","section":"Section 5.1, Fig. 6"},{"comment":"The class-instance distribution is shown graphically but exact counts are not reported; please include the numbers in the caption or text for reproducibility.","section":"Section 5.1, Fig. 4"},{"comment":"The author affiliation line contains 'Rolla, USA Princeton University, Princeton NJ 08544, USA,' which appears to be a formatting error and should be corrected.","section":"Title block"},{"comment":"Model naming is inconsistent (e.g., 'YOLOv8-x' vs 'YOLOV8X' and 'RTDETR-X' vs 'RT-DETR-X'); please unify the notation throughout.","section":"Tables 3 and 5"},{"comment":"The paper does not report the number of video sequences or sessions from which the 7,049 frames were extracted; this metadata is important for assessing diversity and for replicating or auditing the split.","section":"Section 3.4"}],"recommendation":"major_revision","confidential_remarks":"The dataset is a potentially useful community resource, but the evaluation-protocol issues—especially the unclear frame-level split—are the main technical blockers. If the authors can re-run with a grouped split and provide dataset release details, the paper could become suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The Thermal UHD dataset is a real contribution. As far as I can tell from the paper's own survey, there is no public thermal benchmark for underground miner detection with posture annotations, and this fills that niche with 7,049 images from an experimental mine, covering lying, bending, sitting, squatting, and standing across normal and emergency-like conditions. The data collection is thoughtful: IRB approval, three planned phases with heat and smoke variation, and annotation via a standard tool. That is worth something, and the authors deserve credit for putting it together.\n\nThe evaluation, though, has a soft spot that is not minor. The images are frames extracted from video streams, and the 65/35 train/validation split is described only as a random split of the 7,049 images. Nothing in the paper says the split is grouped by source video, scene, or session. If it is a random frame-level split, consecutive frames from the same staged sequence are near-duplicates, so the validation set is largely reconstructing training frames. That would make the headline numbers (e.g., YOLO11-l at 80.2% mAP50 vs. 57% from scratch) look better than a model would do on genuinely unseen miners or new scenarios. The paper's own Fig. 5 claim of \"no overfitting\" does not address this, because a leaked validation set is easy too. The stress-test note is right to flag this.\n\nAlso missing: a held-out test set, repeated runs, error bars, and any public link or license for the dataset. The reader's verdict of conditional acceptance is correct, and I would make the split protocol and data availability preconditions. The single-site representativeness question is real but secondary; you can at least assess how much of the reported gain is real only after the split is fixed.\n\nThat said, the paper is not sloppy in its reasoning. It acknowledges class imbalance, explains why smaller YOLO variants do well on a small simple dataset, and does not overclaim the emergency coverage. The transfer-learning direction is plausible even if the specific numbers are currently unreliable.\n\nFor a reader: this is for people working on underground search-and-rescue robotics or thermal person detection in industrial settings. I would not cite the numbers as they stand, but I would cite the dataset if it is released with a proper split. It deserves a serious referee, not a desk reject, but the referee should push hard on the evaluation protocol and data availability.","headline":"A genuinely useful dataset contribution whose reported numbers are currently undermined by a likely frame-level train/validation leak; fix the split and release the data before trusting the mAP claims.","tokens_in":8052,"tokens_out":1107,"would_cite":false,"duration_ms":14711,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper introduces the Thermal UHD dataset, 7,049 thermal images of underground miners in five postures, and shows that transfer learning on it raises detector mAP50 from roughly 57% to 80%.","keywords":["thermal imaging","underground mining","miner detection","object detection","dataset","YOLO","transfer learning","search and rescue"],"falsifier":"Collect a matched set of thermal images from a different underground mine during a real or realistic emergency, with different mine geometry, a different camera, and real smoke or fire, then test a model fine-tuned only on Thermal UHD; if mAP50 drops well below the 80% reported on the validation split, the dataset's representativeness claim fails.","tokens_in":7198,"feed_emoji":"⛏️","tokens_out":6507,"duration_ms":66739,"temperature":0.7,"pith_summary":"This paper builds and releases the Thermal UHD dataset: 7,049 thermal images of underground miners in five postures (standing, sitting, squatting, bending, lying), captured at an experimental mine under normal work, rest, and emergency-like conditions with heat and smoke. The authors' central claim is that this dataset fills a real gap—no public thermal dataset is tailored to underground mine emergencies—and that fine-tuning object detectors on it makes them substantially better at finding miners. They support this with transfer-learning benchmarks: model mAP50 rises from roughly 57–61% trained from scratch to 75–85% after fine-tuning on Thermal UHD. If correct, the dataset gives search-and-rescue developers a common starting point for training and comparing thermal miner-detection systems.","feed_headline":"7,049 thermal images raise miner detection to 80% mAP50","feed_subtitle":"Fine-tuning YOLO and RT-DETR on staged underground mine scenes beats training from scratch by about 20 points.","key_machinery":"The load-bearing mechanism is the Thermal UHD dataset itself plus the transfer-learning protocol applied to it. The dataset is 7,049 video-extracted thermal frames, resized to 640×640 and annotated with bounding boxes for five miner postures; its three-phase collection scheme is what creates the diversity the paper argues is missing from existing thermal datasets. The protocol is standard fine-tuning of pretrained detectors (YOLOv8, YOLOv10, YOLO11, RT-DETR) on the new labels, and the comparison of fine-tuned versus from-scratch training is what makes the dataset's contribution measurable.","core_discovery":"The paper's central claim is that a purpose-built thermal dataset for underground mines is the missing piece that lets existing object detectors become usable for miner search and rescue. Thermal UHD contains 7,049 frames taken from a thermal camera carried by a legged robot through an experimental mine, split into 4,584 training and 2,465 validation images and labeled with five posture classes: standing, sitting, squatting, bending, and lying. The capture protocol deliberately varied conditions across three phases—working with an introduced heat source and smoke, working under normal temperature while holding tools, and resting—so that models see emergency-like and normal scenes. On this data, fine-tuning raises mAP50 from 57–61% (trained from scratch) to 75–85%, with YOLO11-l reaching 80.2% mAP50 and the lightweight YOLO11-n close behind at 80.1%.","pith_inferences":["The paper does not test whether models fine-tuned on Thermal UHD generalize to a different mine or a different thermal camera; the implied claim is that they will, but that is an assumption to verify.","Because the images come from a video stream on a mobile robot, the dataset could also support temporal or tracking models, which the current single-frame benchmarks do not evaluate.","The same posture classes could be used for routine safety monitoring, such as detecting a miner who has fallen or is resting unexpectedly, in addition to emergency response.","Merging Thermal UHD with visible-light underground imagery could produce multi-spectral detectors, but the paper evaluates thermal data only."],"forward_implications":["Researchers get a public, posture-labeled thermal benchmark for underground mines, so future miner-detection models can be trained and compared on common ground.","A rescue robot carrying a thermal camera can, after fine-tuning, spot a miner in any of the five postures, including lying down, which is the posture most relevant to a collapsed or incapacitated miner.","Lightweight YOLO variants perform almost as well as much larger ones, so the approach is usable on robot hardware with limited compute and power.","Posture-level detection is feasible but the bending class is frequently confused with standing, so the paper's own results imply more balanced data or a better loss is needed before posture labels can be trusted in emergencies."],"supporting_citations":[{"why":"Shows that the diversity of a training dataset drives how well transfer learning works for thermal person detection, the premise behind fine-tuning on Thermal UHD.","marker":"[3]"},{"why":"Documents the low-light detection challenge in underground coal mines that motivates thermal imaging for miner detection.","marker":"[4]"},{"why":"Cited as an underground rescue study relying on detection, illustrating why a dedicated miner dataset is needed.","marker":"[5]"},{"why":"Prior thermal pedestrian detection for underground mine vehicles, establishing the limited existing approach the dataset extends.","marker":"[6]"},{"why":"A rescue-support project using infrared thermography and an unmanned ground vehicle, connecting thermal data to emergency response.","marker":"[7]"},{"why":"Demonstrates human detection in thermal images with YOLOv8 for search and rescue, the direct application the dataset targets.","marker":"[11]"},{"why":"YOLOv8 source used as one of the benchmark architectures.","marker":"[16]"},{"why":"YOLO11 source; YOLO11-l is the best-performing model in the study.","marker":"[18]"},{"why":"RT-DETR source; the transformer-based detector benchmark.","marker":"[19]"}],"fun_headline_variants":["Thermal dataset enables 80% miner detection in underground mines","Fine-tuning on 7,049 thermal images boosts miner detection to 80% mAP50","New thermal dataset for miner rescue hits 80% mAP50 with YOLO11","7,049 thermal frames give miners a detection edge at 80% mAP50","Underground miner detection hits 80% mAP50 with thermal data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The single experimental mine, staged volunteers, and one thermal camera model are assumed to represent the range of real underground mines and emergencies closely enough that models trained on this data will still find real miners.","fun_headline_variants_meta":{"raw":{"variants":["Thermal dataset enables 80% miner detection in underground mines","Fine-tuning on 7,049 thermal images boosts miner detection to 80% mAP50","New thermal dataset for miner rescue hits 80% mAP50 with YOLO11","7,049 thermal frames give miners a detection edge at 80% mAP50","Underground miner detection hits 80% mAP50 with thermal data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000778,"raw_usage":{"total_tokens":3422,"prompt_tokens":913,"completion_tokens":2509,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":529,"completion_tokens_details":{"reasoning_tokens":2402}},"tokens_in":529,"tokens_out":2509,"duration_ms":18715,"temperature":1.0,"reasoning_tokens":2402,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:24:11.131153+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect a matched set of thermal images from a different underground mine during a real or realistic emergency, with different mine geometry, a different camera, and real smoke or fire, then test a model fine-tuned only on Thermal UHD; if mAP50 drops well below the 80% reported on the validation split, the dataset's representativeness claim fails.","supporting_citations":[{"cited_title":"Sensors (Switzerland)","cited_arxiv_id":null,"evidence_quote":"Shows that the diversity of a training dataset drives how well transfer learning works for thermal person detection, the premise behind fine-tuning on Thermal UHD."},{"cited_title":"In: 2020 IEEE Aerospace Conference","cited_arxiv_id":null,"evidence_quote":"Cited as an underground rescue study relying on detection, illustrating why a dedicated miner dataset is needed."},{"cited_title":"In: IEEE Africon’11","cited_arxiv_id":null,"evidence_quote":"Prior thermal pedestrian detection for underground mine vehicles, establishing the limited existing approach the dataset extends."},{"cited_title":"Remote Sens (Basel)","cited_arxiv_id":null,"evidence_quote":"A rescue-support project using infrared thermography and an unmanned ground vehicle, connecting thermal data to emergency response."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"YOLOv8 source used as one of the benchmark architectures."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"YOLO11 source; YOLO11-l is the best-performing model in the study."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"RT-DETR source; the transformer-based detector benchmark."}],"review_version":1}