{"id":"c32d6684-5d42-4678-9f41-f5d776fe711b","arxiv_id":"2411.10627","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A 35-person thermal/optical dataset shows CNN-based thermography detecting induced cooling and erythema changes with high accuracy, but random image-level splitting and a proxy design limit the conclusions.","lead":"This paper tests whether thermal imaging, combined with a simple CNN, can detect temperature changes in darker-skinned patients better than ordinary photos. It reports near-perfect thermal image classification on a new 35-person dataset, but the evaluation leaks participant identity between training and test sets.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Random image-level split introduces participant leakage; reported AUC/F1 in Table 1 cannot support the central claim without a participant-level evaluation.","rationale":"The reader’s weakest assumption identifies the random image-level split as the critical flaw, and my independent review of the manuscript reaches the same conclusion. The manuscript’s quantitative evidence for the central claim—thermography-based CNN robustness to protocols across skin tones—depends entirely on the AUC/F1 values in Table 1. Those values are computed on a test set that likely contains images from the same participants used in training, so the model can memorize participant-specific appearance and texture instead of learning the temperature/erythema signal. This is not an external disagreement with consensus; it is an internal statistical invalidity that no amount of careful protocol variation can fix. The paper does include some independent support: it introduces a new dataset and reports detailed hyperparameters, and the controlled simulation design is sensible as a pilot. However, no code or data are released, and the reported perfect scores on a task with only 35 participants are a red flag that the evaluation is not conservative. A participant-level split is the standard remedy and would directly test the central claim. Since the reader’s verdict of REJECT is already based on this precise concern, my stress-test does not change the disposition; it reinforces it.","tokens_in":8790,"tokens_out":2722,"duration_ms":30243,"concrete_test":"Re-run the full MobileNetV2 pipeline with a participant-stratified split: leave-one-participant-out cross-validation for both the cooling and erythema tasks, using the same preprocessing, augmentation, and hyperparameters as in Section 2.2. Pool the held-out predictions across all 35 folds and recompute AUC and F1. If the cooling AUC/F1 drops from 1.0 or the erythema AUC/F1 drops by more than 0.1 relative to Table 1, the random-split results are inflated by within-participant leakage. This single check would settle whether the central claim holds.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that thermography-based CNNs are robust to data collection protocols for all skin tones—rests on the evaluation in Table 1. The experimental design in Section 2.2 uses a random image-level 80/20 split, and Appendix B confirms “Data is split randomly into training and test sets.” Each cooling participant contributes 48 images (2 cameras x 2 lighting conditions x 2 distances x 3 postures x 2 control/cool) and each erythema participant contributes 9 sequential images. A random image-level split will place images from the same participant in both training and test partitions, violating the i.i.d. assumption required for unbiased performance estimates. A CNN with MobileNetV2’s capacity can exploit participant-specific texture, background, stone placement, or cupping marks to classify images, rather than learning the actual signal of induced temperature change or erythema. The perfect AUC/F1 of 1.0 on the cooling task and the near-perfect erythema scores are therefore not trustworthy evidence of generalization to unseen patients. The paper’s own limitations (Section 2.3) mention small sample size and simulated conditions, but do not acknowledge this statistical confounding. Without a subject-level split, the reported robustness across protocols and skin tones is unsupported, regardless of the proxy design.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a new dataset of thermal and optical images from 35 healthy adults, predominantly with darker skin tones, collected under a cooling protocol and a cupping (erythema) protocol with deliberately varied imaging conditions (camera, lighting, distance, posture). The authors fine-tune MobileNetV2 on three image modalities (optical, grayscale thermal, and color thermal) for two binary classification tasks: cooling versus control and erythema-positive versus control. They report that thermal images achieve perfect or near-perfect AUC/F1 on both tasks, while optical images perform worse, and they conclude that thermography-based CNNs are robust to data collection protocols and may be viable for pressure injury detection in dark skin. The central claim rests entirely on the test-set metrics in Table 1.","tokens_in":9013,"tokens_out":2897,"duration_ms":32000,"significance":"If the reported results were statistically valid, the paper would provide an important, clinically relevant comparison of thermal versus optical imaging for early pressure injury detection across darker skin tones, where visual inspection is known to be unreliable. The dataset itself, with its multi-protocol collection focused on darker skin tones, is a useful contribution. However, the manuscript's central quantitative claims are undermined by a fundamental evaluation-design flaw (participant leakage), so the significance of the current results is not established. The paper is appropriately framed as preliminary, but the evaluation methodology does not support even a preliminary conclusion about generalization to unseen patients.","major_comments":[{"comment":"The random image-level 80/20 split creates participant leakage. Each cooling participant contributes 48 images (2 cameras × 2 lighting conditions × 2 distances × 3 postures × 2 control/cool) and each erythema participant contributes 9 images. With a random split, images from the same participant appear in both training and test sets. MobileNetV2 can exploit participant-identifying cues (skin texture, background, stone placement, cupping marks, or the exact positioning of the stone) rather than learning the actual temperature or erythema signal. Consequently, the perfect AUC/F1 of 1.0 for the cooling task and the near-perfect scores for the erythema task in Table 1 cannot be interpreted as evidence of generalization to unseen patients. The authors must re-run the evaluation with a participant-level split (e.g., leave-participants-out or grouped k-fold) and report metrics with confidence intervals. This is the load-bearing issue for the paper's central claim.","section":"Section 2.2, Appendix B"},{"comment":"The abstract and conclusion claim that the thermography-based CNN is 'robust to data collection protocols for all skin tones,' but no per-protocol or per-skin-tone statistical analysis is provided for thermal images. Table 1 gives only aggregate test-set metrics, and the protocol-level analysis in Figure 2 is restricted to misclassified optical images. The robustness claim is therefore not directly supported by the evidence; even setting aside the leakage problem, the paper does not show that the thermal model's performance is invariant across lighting, distance, and posture. Please report thermal performance stratified by each protocol factor (or a proper interaction analysis) before claiming protocol robustness.","section":"Section 2.2 and Section 2.3 (Impact of Image Protocol)"},{"comment":"The per-skin-tone analysis for erythema (Table 2) consists of raw correct/incorrect counts from a single test split, with many skin-tone categories containing only 3–6 participants (Appendix A, Table 3). With this sample size, the counts cannot support the conclusion that thermography is more reliable than optical imaging 'for all skin tones' or that the InterMid category is particularly challenging. The manuscript should either provide confidence intervals or a more appropriate statistical comparison (e.g., bootstrap or mixed-effects model) and should temper the claim to reflect the exploratory nature of the results.","section":"Table 2 and Section 2.3"}],"minor_comments":[{"comment":"The section header 'T ask Classification' contains a spacing typo; it should read 'Task Classification.'","section":"Section 2.2"},{"comment":"The text uses 'lightning' where 'lighting' is intended (e.g., 'Variations in lightning, distance, cameras, and posture').","section":"Appendix A"},{"comment":"The eczema skin-tone category labels are capitalized inconsistently (e.g., 'Intermediate Mid' versus 'InterMid'), and Figure 4 is referenced but not described in the main text; please align terminology and refer to the figure explicitly.","section":"Table 3 and Figure 4"},{"comment":"The hyperparameter section states a fixed learning rate of 1 × 10^-3 but does not report the range considered in tuning; please clarify whether the learning rate was tuned or fixed.","section":"Appendix B"},{"comment":"The confusion matrix for the cooling task reports 174 true negatives and 169 true positives, but the test set size is stated as 343 in Table 4; this sums to 343, but the breakdown by class should be stated explicitly for reproducibility.","section":"Appendix C"}],"recommendation":"major_revision","confidential_remarks":"The participant-leakage issue is fundamental and would ordinarily push toward rejection, but I recommend major revision because the flaw is identifiable and fixable: re-running the experiments with a participant-level split would directly address the central concern. The authors should also be asked to provide confidence intervals and to substantially soften the 'robust to all protocols and skin tones' language unless per-protocol/per-skin-tone results are reported. I note that the paper does not mention a data or code availability statement; given that the dataset is a core contribution, the authors should be encouraged to release it (with appropriate IRB considerations) to allow verification."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. First, the dataset is a real contribution: 35 participants, mostly dark skin, thermal and optical images with controlled variations in lighting, distance, pose, and camera, plus two tasks (cooling and cupping-induced erythema). Second, the central claim—that a thermography-based CNN is robust to collection protocol across skin tones—is not supported by the evaluation as written, because the random image-level split lets images from the same participant appear in both train and test.\n\nWhat the paper does well: it addresses an equity-relevant gap, it gathers a new dataset rather than reusing an old one, and it compares thermal vs optical imaging head-to-head on the same data. The protocol variation is thoughtful and the proxy design (induced temperature change and erythema) is a reasonable first step. The authors are transparent about the small sample and simulated setting.\n\nThe soft spot is the split. Each cooling participant contributes 48 images; each erythema participant contributes 9. An 80/20 random image split will put the same person on both sides. With 35 participants and a powerful model like MobileNetV2, the model can exploit participant identity—background, stone placement, skin texture, cupping marks—rather than the signal of temperature change. The perfect AUC/F1 on cooling and near-perfect erythema numbers therefore tell you little about generalization to a new patient. This is not a small quibble; it is the statistical basis for every headline number. The paper's limitations section lists sample size and simulated conditions but never mentions this confounding. Also missing: confidence intervals, a participant-level split, and any plan to release the data or code. The erythema positive threshold (6 c.u.) is a free parameter, but that is minor compared with the leakage.\n\nMy take: the dataset is worth keeping, and the question deserves serious work, but the current results do not demonstrate what the abstract claims. A corrected analysis with a participant-level split, confidence intervals, and ideally real PI images (or at least a clearer argument for why induced erythema transfers) could change the verdict. I would send this to peer review, but I would want the authors to redo the evaluation before publication. If the journal is willing to accept a revised version, this becomes a useful preliminary study; as is, it overclaims.","headline":"Genuinely useful new dataset, but the image-level split invalidates the headline AUC/F1; needs participant-level evaluation before the robustness claim can stand.","tokens_in":9524,"tokens_out":2144,"would_cite":false,"duration_ms":22037,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A CNN trained on thermal images detects temperature changes and erythema in dark skin more reliably than optical images, with perfect cooling-task scores and an AUC above 0.93 for erythema.","keywords":["pressure injury detection","thermography","thermal imaging","dark skin tones","erythema detection","MobileNetV2","convolutional neural network","image classification"],"falsifier":"Re-run the same MobileNetV2 pipeline with a participant-level split so that all images from one person are kept in either training or testing, and compare the cooling and erythema AUC and F1 to Table 1; if the thermal advantage shrinks or the absolute scores fall substantially, the reported robustness is an artifact of image-level leakage.","tokens_in":8605,"feed_emoji":"🌡️","tokens_out":8524,"duration_ms":69778,"temperature":0.7,"pith_summary":"This paper tests whether thermography can replace visual inspection for early pressure-injury detection in dark skin, where redness is hard to see. The authors collected a new dataset of 35 participants, predominantly with dark skin tones, and induced temperature changes and erythema on the lower back under varied imaging protocols. They trained a small convolutional neural network on thermal and optical images for two binary tasks, cooling detection and erythema detection. The thermal-trained model achieved perfect scores on the cooling task and outperformed the optical model on erythema, suggesting that thermography is a viable, pigment-independent route to pressure-injury screening.","feed_headline":"Thermal imaging spots erythema in dark skin better than optical","feed_subtitle":"A CNN on thermal images hit perfect cooling scores and over 0.93 AUC for erythema in 35 dark-skinned participants.","key_machinery":"The central object is MobileNetV2, a compact convolutional neural network with 19 residual bottleneck layers, fine-tuned from ImageNet pretraining with a fixed learning rate and Adam optimizer. The argument runs through the dataset: thermal and optical images of the lower back from 35 healthy adults spanning four Eumelanin skin-tone categories, with temperature changes induced by a cooled stone and a cupping device, and acquisition varied by camera, lighting, distance, and pose. Thermal images carry the temperature difference directly, whereas optical images carry color that melanin masks; the network is trained separately on optical, grayscale thermal, and color thermal images so the two representations can be compared on identical binary classification tasks and test splits.","core_discovery":"The central claim is that a thermography-based CNN is robust to data collection protocols for all skin tones and detects induced temperature changes and erythema in darker skin more reliably than optical images. On the cooling task, MobileNetV2 reaches an AUC and F1 of 1.0 with both grayscale and color thermal images, versus 0.818 and 0.711 with optical images. On the erythema task, the thermal models achieve an F1 of at least 0.914 and an AUC of at least 0.935, compared with 0.868 and 0.909 for optical images. The study interprets this as evidence that temperature-based imaging bypasses the melanin-related visual masking that makes visual inspection unreliable, and that strict imaging protocols may not be necessary for thermal acquisition.","pith_inferences":["A participant-level (stratified) train/test split is the natural next test: because the current split is random at the image level, images from the same participant appear in both training and testing, so the model may have learned participant identity rather than generalizable temperature patterns.","The erythema result is based on a single imaging protocol, so the claim of protocol robustness applies mainly to the cooling task and should be re-tested for erythema under varied lighting and distance.","If a patient-level split erases most of the thermal-versus-optical gap, the practical conclusion would shift from thermography being categorically better to both modalities performing similarly once identity leakage is removed, which would change the clinical recommendation.","A natural extension is to test on real pressure injuries rather than induced temperature changes, since real injuries involve tissue damage, moisture, and chronic inflammation that may alter thermal signatures differently than cooling or cupping."],"forward_implications":["Clinical adoption of thermography for pressure-injury screening would not require strict control of lighting, camera distance, or patient pose, since thermal model performance stayed high across protocol variations.","Dark-skin patients, who are currently at higher risk of missed early pressure injuries, could be screened with thermal imaging instead of relying on visual redness.","A compact network like MobileNetV2 is light enough for bedside or portable devices, making real-time thermographic screening feasible in routine nursing workflows.","Thermal imaging may serve as a more reliable early-warning signal than optical imaging for erythema, the first visible sign of pressure injury, across intermediate and high eumelanin skin tones."],"supporting_citations":[{"why":"Defines MobileNetV2, the network architecture used for all classification tasks in the study.","marker":"Sandler et al., 2018"},{"why":"Provides ImageNet, the pretraining dataset from which the CNN is fine-tuned.","marker":"Russakovsky et al., 2015"},{"why":"Introduces the Eumelanin Human Skin Colour Scale used to categorize participants' skin tones.","marker":"Dadzie et al., 2022"},{"why":"Supplies the revised Eumelanin skin tone categories that the study applies to its 35 participants.","marker":"Sonenblum et al., 2023"},{"why":"Establishes the prior CNN approach for classifying infrared thermal images for pressure injury, which this study extends to dark skin tones.","marker":"Wang et al., 2021"},{"why":"Demonstrates MobileNetV2 for pressure injury detection on thermal images, the direct methodological precedent for the model choice.","marker":"Pandey et al., 2022"},{"why":"Documents racial disparities in pressure injury outcomes, motivating the study's focus on dark skin tones.","marker":"Black et al., 2023"},{"why":"Reports weak correlations between erythema index and temperature changes, cited to justify restricting the erythema protocol to a single acquisition setting.","marker":"Bates-Jensen et al., 2024"}],"fun_headline_variants":["Thermal imaging outshines optical for dark skin pressure injury detection","Thermography detects skin damage in dark tones where visual fails","Cooling test: thermal CNN perfect, optical lags on dark skin","Heat maps beat visible light for pressure injuries in dark skin"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim rests on treating every image as an independent sample, even though dozens of images come from each participant, so the same person's images appear in both the training and test sets and the model could be recognizing participants rather than temperature changes.","fun_headline_variants_meta":{"raw":{"variants":["Thermal imaging outshines optical for dark skin pressure injury detection","Thermography detects skin damage in dark tones where visual fails","Cooling test: thermal CNN perfect, optical lags on dark skin","Heat maps beat visible light for pressure injuries in dark skin"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001234,"raw_usage":{"total_tokens":5023,"prompt_tokens":858,"completion_tokens":4165,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":474,"completion_tokens_details":{"reasoning_tokens":4092}},"tokens_in":474,"tokens_out":4165,"duration_ms":27449,"temperature":1.0,"reasoning_tokens":4092,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T19:29:04.104727+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same MobileNetV2 pipeline with a participant-level split so that all images from one person are kept in either training or testing, and compare the cooling and erythema AUC and F1 to Table 1; if the thermal advantage shrinks or the absolute scores fall substantially, the reported robustness is an artifact of image-level leakage.","supporting_citations":[{"cited_title":"The eumelanin human skin colour scale: a proof-of-concept study","cited_arxiv_id":null,"evidence_quote":"Introduces the Eumelanin Human Skin Colour Scale used to categorize participants' skin tones."},{"cited_title":"Using technology to detect erythema across skin tones","cited_arxiv_id":null,"evidence_quote":"Supplies the revised Eumelanin skin tone categories that the study applies to its 35 participants."},{"cited_title":"Infrared thermal images classification for pressure injury prevention incorporating the convolutional neural networks","cited_arxiv_id":null,"evidence_quote":"Establishes the prior CNN approach for classifying infrared thermal images for pressure injury, which this study extends to dark skin tones."},{"cited_title":"A deep learning approach for automated detection and segmentation of pressure ulcers using infrared-based thermal imaging","cited_arxiv_id":null,"evidence_quote":"Demonstrates MobileNetV2 for pressure injury detection on thermal images, the direct methodological precedent for the model choice."},{"cited_title":"Current perspectives on pressure injuries in persons with dark skin tones from the national pressure injury advisory panel","cited_arxiv_id":null,"evidence_quote":"Documents racial disparities in pressure injury outcomes, motivating the study's focus on dark skin tones."},{"cited_title":"Thermal measurement of erythema across skin tones: Implications for clinical identification of early pressure injury","cited_arxiv_id":null,"evidence_quote":"Reports weak correlations between erythema index and temperature changes, cited to justify restricting the erythema protocol to a single acquisition setting."}],"review_version":1}