{"id":"10e36939-e223-46ef-b74f-eed2ca724983","arxiv_id":"2506.17954","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A mobile app measures tuberculin skin test indurations with ARCore and DeepLabv3, but its accuracy claims rest on circular tests with clay mock-ups.","lead":"This paper describes a smartphone app that uses ARCore depth sensing and a DeepLabv3 segmentation model to measure the size of Mantoux tuberculin skin test reactions. The authors say it improves accuracy and reliability, but the only tests were done on clay models, not real patients.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported millimeter accuracy is an artifact of evaluating on the same clay mocks used to calibrate the pixel-to-mm conversion; no evidence that this transfers to real human TST indurations.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the calibration/evaluation circularity on clay mocks. The paper's only quantitative accuracy evidence is the 9.91 mm measurement of a 10 mm clay mock, which is exactly the sort of in-sample fit that cannot support a claim of clinical accuracy. The manuscript itself acknowledges in Section 5 that 'obtaining approval and accessing real patient data will ensure clinical applicability,' confirming that such validation is absent. The design is plausible as an engineering prototype, but the central claim of accurate and reliable automated TST measurement is not established. The duplicate section numbers, missing reference, and use of an unrelated segmentation dataset are secondary; the fundamental issue is that the accuracy result is circular. Therefore the verdict should remain REJECT, in agreement with the reader.","tokens_in":131,"tokens_out":1552,"duration_ms":26154,"concrete_test":"Re-run the measurement protocol on real TST indurations (or, if ethically unavailable, on skin-mimicking phantoms with known induration sizes not used in any calibration step). Compare app measurements against blinded clinician measurements using the standard ballpoint/ruler method, across multiple operators, skin tones, and lighting conditions. Report mean absolute error and Bland-Altman limits of agreement. If the error exceeds ±2 mm or the calibration factors must be re-fit to the new data, the claimed accuracy does not transfer beyond the clay mocks.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim is that the app measures TST indurations with millimeter accuracy. In §4.5, the pixel-to-millimeter factors (0.1197, 0.1523, 0.1499) were empirically fitted in Experiment 3 using clay mock indurations of 5, 10, and 15 mm. Experiment 2 then reports that a 10 mm clay mock is measured as 9.91 mm. Because the evaluation set is the same set of clay mocks used for calibration, this agreement is a restatement of the fit, not independent validation. The load-bearing premise is that clay mocks reproduce the optical properties (edge contrast, sharpness, curvature) and depth response of real TST indurations, and that ARCore's depth estimation, which §4.2 optimizes using a contrasting background, performs equally well on human skin with varied pigmentation, hair, and edema. No real patient data, no error bars, no comparison to the ballpoint/ruler clinical standard, and no held-out test set are provided. The segmentation model trained on PAD-UFES-20 (dermatoscopic images) is also not shown to segment TST indurations. Therefore the abstract's claim of 'significant improvements in accuracy and reliability' is unsupported by the evidence presented.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a mobile Android application (TBCheck) for automated measurement of Mantoux tuberculin skin test (TST) indurations. The system uses ARCore for depth estimation, DeepLabv3 for image segmentation, and a pixel-to-millimeter conversion calibrated through experiments with clay mock indurations of 5, 10, and 15 mm. The authors report an optimal capture depth of 219–220 mm, at which a 10 mm mock is measured as 9.91 mm, and claim that the application improves accuracy and reliability over standard clinical practice. The evaluation consists of four experiments: segmentation model selection (on the PAD-UFES-20 skin lesion dataset), depth optimization, calibration of conversion factors, and a usability study with ten participants.","tokens_in":8148,"tokens_out":2241,"duration_ms":25691,"significance":"If the measurement pipeline were shown to be accurate on real TST indurations, the application could offer a meaningful contribution to TB screening in resource-limited settings by standardizing and automating reading of the Mantoux test. The paper also demonstrates a systematic attempt to combine ARCore depth information, machine-learning segmentation, and a sticker-based scaling approach. However, the manuscript provides no validation on real patients, no comparison with the standard ballpoint pen and ruler method, no segmentation accuracy metrics on induration images, and no statistical analysis. The principal quantitative evidence is circular because the pixel-to-millimeter calibration factors are derived from and then evaluated on the same clay mock objects. The central claim of 'significant improvements in accuracy and reliability' is therefore unsupported by the evidence as presented.","major_comments":[{"comment":"The load-bearing evidence for millimeter accuracy is circular. In §4.5, the pixel-to-millimeter conversion factors (0.1197, 0.1523, 0.1499) are presented as calibrated values, and §4.6 Experiment 3 states that these factors were obtained using mock indurations of 5, 10, and 15 mm. Experiment 2 then reports that the same type of clay mock (10 mm) is measured as 9.91 mm at the optimal depth. Because the evaluation and calibration sets are the same clay objects, the small reported error is a restatement of the fitted conversion factors rather than an independent test of accuracy. A valid evaluation would require held-out objects, a different ground-truth measurement modality, or real patient indurations with independent clinical measurement.","section":"§4.5 and §4.6, Experiments 2 and 3"},{"comment":"The abstract's claim that the application 'was evaluated against standard clinical practices, demonstrating significant improvements in accuracy and reliability' is not supported by the experiments in §4.6. No comparison to the ballpoint pen and ruler method or to any other clinical standard is reported; the only usability data come from ten participants completing a level-of-agreement questionnaire, with no quantitative accuracy comparison. The manuscript should either present such a comparison or explicitly restrict the claims to a technical demonstration on clay phantoms.","section":"Abstract and §4.6 (conclusion of Experiments)"},{"comment":"The segmentation model is trained exclusively on PAD-UFES-20, a dataset of dermatoscopic skin lesion images, and the manuscript provides no evidence that this model can segment TST indurations. No segmentation metrics (IoU, Dice, precision/recall) are reported for induration images, and Figure 3, labeled 'Successful and Failed Segmented of Skin Induration,' appears to include failures without quantitative analysis. The claim that DeepLabv3 provides 'robust image segmentation' for TST measurement is therefore not established, and the effect of segmentation errors on the reported diameter measurements is unknown.","section":"§4.4 and §4.6, Experiment 1"},{"comment":"The ARCore depth estimation and plane detection are optimized using a 'contrasting background' (Figure 2), but the application is intended to operate on human skin, which exhibits varied pigmentation, hair, edema, and irregular surfaces. The transfer of depth accuracy from the laboratory setup with clay mocks on a contrasting cloth to real clinical conditions is a central assumption that is never tested. The authors themselves note that depth stability and accuracy under varied lighting conditions remain future work, which underscores that this assumption is unvalidated in the present manuscript.","section":"§4.2 and §4.3"},{"comment":"No error bars, confidence intervals, or statistical tests are reported for any of the quantitative or usability measurements. For example, the finding that '60% felt confident' and 'only 50% found it easy to use' is based on ten participants, yet no uncertainty is attached to these percentages. More importantly, the repeated measurements from which the optimal depth of 219–220 mm and the conversion factors were derived are not reported with variance, making it impossible to assess the reliability of the calibration.","section":"§4.6, Experiments 2–4"}],"minor_comments":[{"comment":"The manuscript contains two consecutive sections numbered 4.5: 'Automated Measurement of Induration' and 'Patient Information Collection and Reminder Function.' This numbering error should be corrected.","section":"§4.5 (two sections numbered 4.5)"},{"comment":"The PAD-UFES-20 dataset is cited as [70] in §4.4, but the reference list contains only 24 entries; the reference should be added or the citation number corrected.","section":"Reference list"},{"comment":"There are several typographical and grammatical errors, for example 'pappers' in §2.1, 'di-agnostics' in the abstract, and the incomplete sentence in §2.1 starting with 'this app utilizes.' These should be fixed through a careful edit.","section":"Throughout"},{"comment":"The three examples shown in Figure 4 (15.00 mm, 9.23 mm, and 4.97 mm) are presented as measurement results, but the figure does not identify which clay mock sizes these correspond to, making the figure uninformative without additional caption detail.","section":"Figure 4"}],"recommendation":"reject","confidential_remarks":"The core quantitative claim rests on a circular validation, and the manuscript's scope does not include the real patient data or independent comparator that would be required to fix this issue. The citation of [70] without a corresponding reference also suggests editorial carelessness. As an image-analysis application paper in eess.IV, the lack of segmentation evaluation on the target domain is a further mismatch with the journal's standards. I would not encourage resubmission unless the authors perform a genuinely independent clinical validation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the app exists, the system description is coherent, but the accuracy claim rests on a circular calibration–evaluation loop with clay mocks. I would not trust the reported millimeter numbers until real patient data appear.\n\nWhat is worth something: the design choice to use ARCore depth plus a scaling sticker is a reasonable alternative to the photogrammetry used in the UCT work [9–11], and the authors integrate a full pipeline (auto-capture, orientation panel, DeepLabv3 segmentation, reminder function). The usability study with ten participants is a real attempt to gather feedback. They also state plainly in the future work that clinical approval and real patient data are still needed, which is honest.\n\nThe soft spots are serious. The pixel-to-mm factors in §4.5 (0.1197, 0.1523, 0.1499) were fitted to clay mocks of 5, 10, and 15 mm, and Experiment 2 then reports 9.91 mm for a 10 mm mock. That is not validation; it is a restatement of the fit. There are no error bars, no held-out objects, no comparison with the ballpoint-pen ruler method, and no segmentation accuracy numbers. The DeepLabv3 model was trained on PAD-UFES-20, which is dermatoscopic skin lesion images, not photographs of TST indurations; no evidence shows this transfers. The abstract's claim of 'significant improvements in accuracy and reliability' is unsupported.\n\nThere are also presentation problems: duplicate section numbers 4.5, a reference to '[70]' that does not exist, and an introduction that promises a Section 6 that never appears. None of these are fatal on their own, but they add to the impression of a not-yet-ready manuscript.\n\nWho is this for? It could be a useful starting point for a graduate student working on mHealth TST measurement, or an example in a methods course of why calibration and evaluation sets must differ. But as a scientific claim, it does not hold up.\n\nMy recommendation: do not send this to peer review as is. The correct next step is a small clinical pilot with real indurations, independent of the calibration objects. If the authors bring that data, the paper would deserve referee time. Right now the evidence is too circular.","headline":"The app is a plausible engineering prototype, but the reported millimeter accuracy is an artifact of calibrating and then evaluating on the same clay mocks, so the paper isn't yet a validated scientific contribution.","tokens_in":8672,"tokens_out":2905,"would_cite":false,"duration_ms":34245,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a smartphone app using ARCore depth sensing, a scaling-sticker reference, and DeepLabv3 segmentation can measure Mantoux skin-test indurations at millimeter accuracy, replacing the manual ballpoint-pen method.","keywords":["Mantoux test","tuberculin skin test","induration measurement","ARCore depth estimation","DeepLabv3 segmentation","mobile health","latent tuberculosis","scaling sticker calibration"],"falsifier":"Capture a set of real TST indurations across varied skin tones and lighting, measure each with two blinded clinicians using the ballpoint-pen method and with the app, then compare both to an independent reference such as high-frequency ultrasound or calipers; if the app's errors exceed the 1-2 mm clinical tolerance or track systematically with skin tone, the central accuracy claim does not transfer to patients.","tokens_in":7663,"feed_emoji":"📱","tokens_out":7928,"duration_ms":79719,"temperature":0.7,"pith_summary":"This paper claims that a smartphone can replace the manual ballpoint-pen/ruler reading of the Mantoux tuberculin skin test. The proposed Android app uses ARCore depth sensing, a scaling sticker placed next to the induration as a real-world reference, and a DeepLabv3 segmentation model to isolate the raised area and measure its diameter in millimeters. The authors argue this removes subjective interpretation and improves accessibility in low-resource settings, and they report that at the optimal capture distance of 219-220 mm a 10 mm clay mock induration was measured as 9.91 mm. The broader claim is that automated measurement of this standard TB-screening test can be done entirely on a mobile device.","feed_headline":"Phone app reads Mantoux skin-test reactions at millimeter accuracy","feed_subtitle":"A scaling sticker plus ARCore depth could replace the subjective ballpoint-pen method; a 10 mm mock read as 9.91 mm.","key_machinery":"The load-bearing mechanism is the scaling-sticker calibration: a sticker with known physical size appears in the same image plane as the induration, and ARCore's Depth API supplies the millimeter depth of the skin surface. The app then measures the largest diameter of the segmented induration in pixels and converts it with a depth-calibrated factor chosen by pixel diameter (0.1197 below 50 pixels, 0.1523 between 50 and 80, 0.1499 between 80 and 200). This piecewise factor is what turns a dimensionless pixel measurement into a physical millimeter reading, and the paper's reported accuracy is essentially an evaluation of that calibration on the same clay objects used to derive it.","core_discovery":"The central discovery is a measurement pipeline in which ARCore's depth estimation provides the distance from camera to skin, and a scaling sticker supplies a known physical reference, so pixel distances can be converted to millimeters without photogrammetric 3D reconstruction. The paper reports piecewise conversion factors: 0.1197 for pixel diameters below 50, 0.1523 for 50-80, and 0.1499 for 80-200, calibrated on 5, 10, and 15 mm clay mocks. On the decisive test, a 10 mm mock at 219-220 mm capture depth was measured as 9.91 mm. The authors state that the evaluated system shows significant improvements in accuracy and reliability over standard clinical practice and conclude that on-device segmentation plus depth-based scaling is a viable path to standardized TST evaluation.","pith_inferences":["Inference beyond the paper: the same depth-plus-sticker pipeline could measure induration height or volume, not just diameter, because ARCore already provides per-pixel depth over the raised area.","Inference beyond the paper: the piecewise conversion factors are discontinuous at 50 and 80 pixels, so a smooth calibration curve fitted over the full range would likely reduce boundary artifacts and could be tested against the 5, 10, and 15 mm mocks.","Inference beyond the paper: if real-patient validation confirms the clay-mock accuracy, sticker-reference measurement could transfer to other raised skin lesions by retraining the segmentation model."],"forward_implications":["A clinician or trained worker can obtain an automated reading by holding the phone at the app-guided depth and orientation, removing the ballpoint-pen/ruler step and its inter-observer variability.","The on-device DeepLabv3 model computes the reading locally, so the app can work in settings without a network connection or specialized equipment.","Combining the measured diameter with the app's risk-factor questionnaire yields a positive or negative latent-TB classification, as demonstrated by the 10-15 mm induration threshold.","The reported optimal capture depth of 219-220 mm gives a concrete operating point for validation studies on real patients."],"supporting_citations":[{"why":"Establishes the photogrammetric 3D-reconstruction approach to measuring indurations from smartphone images, the method this app replaces.","marker":"[10]"},{"why":"Provides the integrated image-capture-and-analysis mobile system for latent tuberculosis that this work extends with ARCore depth and on-device machine learning.","marker":"[9]"},{"why":"Introduces deep-learning and GAN-based synthetic induration image generation, motivating the segmentation approach used here.","marker":"[11]"},{"why":"Supplies the DeepLab v3 atrous-convolution architecture the paper adapts for real-time induration segmentation.","marker":"[20]"},{"why":"Names the public annotated skin-lesion dataset of 2,298 images used to train the segmentation model, though the reference list omits the entry.","marker":"[70]"}],"fun_headline_variants":["AR depth plus sticker gives Mantoux reads within 0.09 mm","Scaling sticker, not 3D, makes TST app accurate","Phone measures Mantoux indurations to 0.1 mm using ARCore","ARCore+sticker app hits 9.91 mm on 10 mm Mantoux mock"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The accuracy result rests on the assumption that clay mocks faithfully mimic the optical, depth, and edge properties of real human TST indurations, and that ARCore depth is equally accurate on skin as on the contrasting background used in the experiments.","fun_headline_variants_meta":{"raw":{"variants":["AR depth plus sticker gives Mantoux reads within 0.09 mm","Scaling sticker, not 3D, makes TST app accurate","Phone measures Mantoux indurations to 0.1 mm using ARCore","ARCore+sticker app hits 9.91 mm on 10 mm Mantoux mock"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000598,"raw_usage":{"total_tokens":2785,"prompt_tokens":924,"completion_tokens":1861,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":540,"completion_tokens_details":{"reasoning_tokens":1776}},"tokens_in":540,"tokens_out":1861,"duration_ms":12669,"temperature":1.0,"reasoning_tokens":1776,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:22:38.259057+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Capture a set of real TST indurations across varied skin tones and lighting, measure each with two blinded clinicians using the ballpoint-pen method and with the app, then compare both to an independent reference such as high-frequency ultrasound or calipers; if the app's errors exceed the 1-2 mm clinical tolerance or track systematically with skin tone, the central accuracy claim does not transfer to patients.","supporting_citations":[{"cited_title":"Measurement of Skin Induration Size Using Smartphone Images and Photogrammetric Reconstruction: Pilot Study,","cited_arxiv_id":null,"evidence_quote":"Establishes the photogrammetric 3D-reconstruction approach to measuring indurations from smartphone images, the method this app replaces."},{"cited_title":"Mobile phone-based evaluation of latent tuberculosis infection: Proof of concept for an integrated image capture and analysis system,","cited_arxiv_id":null,"evidence_quote":"Provides the integrated image-capture-and-analysis mobile system for latent tuberculosis that this work extends with ARCore depth and on-device machine learning."},{"cited_title":"Image analysis for a mobile phone-based assessment of latent tuberculosis infection,","cited_arxiv_id":null,"evidence_quote":"Introduces deep-learning and GAN-based synthetic induration image generation, motivating the segmentation approach used here."}],"review_version":1}