{"id":"3b3e1483-1571-44db-b442-68c0ff5e829d","arxiv_id":"2501.18270","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"The iToBoS dataset provides 16,954 region-level skin images with lesion bounding boxes and metadata for training and benchmarking lesion detection algorithms.","lead":"This paper introduces iToBoS, a dataset of 16,954 skin-region images from 100 people, with lesions marked by bounding boxes. It gives AI researchers a resource for training systems that detect skin lesions in full skin context rather than isolated close-ups.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Annotation reliability is the load-bearing risk, and the paper's own statistics contradict the stated 2.5 mm minimum-lesion threshold, leaving the smallest bounding boxes unvalidated.","rationale":"The reader's weakest assumption was that the bounding-box annotations are accurate and complete enough to serve as ground truth. I agree that this is the central load-bearing issue. My stress-test sharpens it by pointing to an internal inconsistency in the paper's own numbers: the stated 2.5 mm minimum threshold is contradicted by reported diameters as small as 0.14 mm and by the presence of thousands of sub-3 mm lesions. This is not merely a missing validation metric; it suggests either protocol drift in annotation or unreliable physical-size conversion, both of which directly affect the trustworthiness of the smallest labels. The dataset still appears to exist, be publicly accessible, and contain a large number of plausibly useful region-level images, so the reader's CONDITIONAL verdict is appropriate. I would not move to accept or reject on this basis alone, but the condition should explicitly include a quantitative annotation audit and reconciliation of the threshold discrepancy.","tokens_in":11016,"tokens_out":6364,"duration_ms":69817,"concrete_test":"Compute the full distribution of hull-based diameters in the released YOLO/COCO labels and count annotations whose estimated physical diameter is below 2.5 mm using the provided pixel spacing. If more than a small fraction (e.g., >1%) fall below the stated threshold, the annotation protocol is not as described. Then have two independent dermatologists re-annotate a stratified random sample of 200 tiles, oversampling lesions <3 mm, and report per-box recall and localization error against the released labels to quantify label noise on the smallest lesions.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The dataset's fitness as ground truth for lesion detection rests on the claim that all tiles were manually reviewed by dermatologists and that annotations follow a 2.5 mm minimum lesion size. The paper provides no inter-annotator agreement, no error-rate audit, and no external validation of the bounding boxes. More concretely, the reported statistics contradict the stated protocol: Figure 9b and the accompanying text report lesion diameters as low as 0.14 mm (train) and 0.28 mm (test), with 5,471 train and 4,855 test lesions in the <3 mm category, even though Methods states that a 2.5 mm lower threshold was established. Either the threshold was not enforced, so the inclusion criterion is not what is described, or the pixel-to-mm conversion/metadata is unreliable in the small-lesion regime. In either case, the reliability of the smallest boxes, which are the most difficult for detection models, is unestablished. Because the central contribution is a labeled detection dataset, systematic label noise or protocol drift would directly bias models trained and benchmarked on it.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper describes the iToBoS dataset: 16,954 2D skin-surface tiles extracted from 3D total-body photographs of 100 participants at two clinical sites, with bounding-box annotations of skin lesions, metadata (anatomical region, age, sex-at-birth, sun damage score), and train/test splits for the iToBoS-2024 Kaggle detection challenge. The authors detail the three-phase construction pipeline (acquisition, annotation, public subset selection), report descriptive statistics on lesion size, spatial distribution, sun damage, and image dimensions, and provide access instructions, data formats, and helper code. The central claim is that the dataset provides a novel, publicly available resource for training and benchmarking lesion detection AI models.","tokens_in":11147,"tokens_out":4516,"duration_ms":48814,"significance":"If the bounding-box labels are reliable, the dataset fills a real gap: most public skin-image datasets are lesion-centric or crop-based, whereas iToBoS provides region-level tiles with surrounding skin context, which is important for lesion detection. The two-site collection, stratified sampling, metadata, YOLO/COCO formats, and public hosting on Kaggle with helper code are concrete strengths. The paper also documents privacy protections and ethical approvals. However, the central claim of utility as a detection benchmark is not yet fully supported: the paper provides no quantitative annotation-quality metrics (inter-annotator agreement, error-rate audit, or external validation) and no baseline detection results, and the reported lesion-size statistics appear to contradict the stated 2.5 mm minimum inclusion threshold. These issues are load-bearing because systematic label noise or protocol drift would directly bias models trained and benchmarked on the dataset.","major_comments":[{"comment":"The stated inclusion criterion of a 2.5 mm minimum lesion diameter is contradicted by the reported diameter statistics. Figure 9b reports minimum diameters of 0.14 mm (train) and 0.28 mm (test), and Figure 9a places 5,471 train and 4,855 test lesions in the '<3 mm' category. If the 2.5 mm threshold was enforced, lesions below 2.5 mm should be absent; if it was not enforced, the Methods description is inaccurate. In either case, the reliability of the smallest bounding boxes, which are the most difficult for detection models, is unestablished. Please clarify whether the threshold was enforced, report the number of boxes below 2.5 mm in each split, and provide an audit of the pixel-to-millimeter conversion used for these measurements.","section":"Data Analysis, Figure 9 vs. Methods (Data annotation)"},{"comment":"The only validation offered is the assertion that 'All annotated tiles were manually reviewed by a team of dermatologists.' For a dataset whose primary use is ground truth for lesion detection, this is insufficient. No inter-annotator agreement, no error-rate audit, no sample-level re-annotation statistics, and no external comparison (e.g., against VECTRA's automated detection or dermoscopic confirmation) are reported. Please add quantitative QA metrics, ideally broken down by lesion size, so that users can calibrate the expected label noise, particularly for small or subtle lesions.","section":"Technical Validation"},{"comment":"The paper claims the dataset is intended for 'training and benchmarking,' but no baseline detection results are provided and the test-set labels are private 'at least until the completion of the iToBoS lesion detection challenge.' This makes the benchmarking claim currently untestable by readers. Please include at least one reproducible baseline detector trained on the public train split, with evaluation on the test set once labels are released or on a clearly described held-out subset, and state explicitly the timeline and conditions for label release.","section":"Data Accessibility and Usage Notes"}],"minor_comments":[{"comment":"The directory structure in Figure 7 shows a 'labels' folder and labels.json for the test set, while the text states that test labels are private; please reconcile this inconsistency or annotate the figure to indicate that the test labels are placeholders or withheld.","section":"Figure 7"},{"comment":"The caption in Figure 11 begins 'Figure 11. Figure 11.'; the duplication should be removed.","section":"Appendix A, Figure 11"},{"comment":"The product name 'VectraDBTool' appears with broken spacing ('V ectraDBTooldeveloped'); please correct the formatting.","section":"Methods: 2D tile generation from 3D-TBP avatar"},{"comment":"The competing interests statement is a placeholder sentence ('The corresponding author is responsible for providing...'); the manuscript should include an actual statement from the authors.","section":"Competing interests"}],"recommendation":"major_revision","confidential_remarks":"The threshold contradiction in Figure 9 is the most serious internal inconsistency and should be resolved before publication. If the authors can provide quantitative annotation-quality metrics and clarify the inclusion protocol, the paper would be a useful data descriptor; I would not recommend rejection at this stage. The paper is within the scope of a data-description venue but needs stronger technical validation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know before you read it. First, this is a real resource: 16,954 skin tiles from 3D total body photography, with bounding boxes, sun damage scores, anatomical labels, and public code. That is a genuine gap relative to HAM10000, BCN20000, and SLICE-3D, which are lesion-centric or crop-based. Second, the paper does not actually verify the annotation quality, and there is a specific internal inconsistency that matters if you plan to train on it.\n\nWhat the paper does well: the construction pipeline is described clearly — recruitment, VECTRA WB360 capture, tiling, cloud annotation, and a stratified sampling procedure for the public split. Privacy measures are thoughtful: face masking, tattoo inpainting, randomized filenames, removal of 3D coordinates. The metadata (age, sex, sun damage, anatomical region) is useful for context-aware models. The data is public on Kaggle with YOLO and COCO formats, plus helper scripts.\n\nThe soft spots are real but not fatal. Most importantly, the stress-test note is right: Methods says a 2.5 mm minimum lesion diameter threshold was established, but Figure 9 reports train and test min diameters of 0.14 mm and 0.28 mm, with over 10,000 lesions in the <3 mm category. Either the threshold was not enforced or the pixel-to-mm conversion/metadata is unreliable in the small-lesion regime. That is a load-bearing issue for a detection dataset because the smallest boxes are the hardest for models and the most likely to be noisy. The paper also provides no inter-annotator agreement, no error-rate audit, and no baseline detection results. The test labels are private, so the claimed benchmarking utility is currently unverified. And the homogeneity (Fitzpatrick I–II, European ancestry) limits generalizability, which the paper does acknowledge.\n\nProportionately: the dataset itself is valuable and likely usable, but the absence of label validation means early adopters should do their own audit on a sample before trusting the boxes as ground truth. The discrepancy about the 2.5 mm threshold needs a public correction or an explicit admission that the threshold was advisory.\n\nWho is this for? Anyone building or evaluating lesion detection models on 3D total body photography. They should read the paper, download the data, and treat the annotations as provisional until the authors document the protocol drift and release some form of validation. It deserves a serious referee; I would send it to review with major revisions, asking for IAA on a subset, a baseline model, and a clear explanation of the diameter discrepancy.","headline":"A genuinely useful region-level skin lesion detection dataset, but the annotation protocol has an internal contradiction and no quantitative validation of label quality.","tokens_in":11779,"tokens_out":1961,"would_cite":true,"duration_ms":20927,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper introduces the iToBoS dataset: 16,954 region-level skin images tiled from 3D total-body photographs of 100 participants, each annotated with lesion bounding boxes and metadata, publicly released to train and benchmark…","keywords":["skin lesion detection","3D total body photography","dataset release","bounding box annotation","VECTRA WB360","skin cancer screening","iToBoS challenge","deep learning"],"falsifier":"Re-annotate a random sample of about 200 tiles with two independent dermatologists who have not seen the original labels, following the same 2.5 mm minimum-size rule, and measure lesion-level agreement (matched by overlap) against the published boxes. If agreement is markedly lower for lesions under 3 mm or in high-sun-damage tiles, the ground truth would be too noisy to support reliable benchmarking; the same check on tiles that share the 45-pixel overlap could show whether duplicated lesions are labelled consistently.","tokens_in":10819,"feed_emoji":"🩺","tokens_out":11892,"duration_ms":99883,"temperature":0.7,"pith_summary":"This paper sets out to close a gap in skin-cancer AI data: most public datasets are dermoscopic images of single, isolated lesions centered in the frame, so models never learn what normal surrounding skin looks like. The authors present the iToBoS dataset, 16,954 high-resolution image tiles of skin regions from 100 participants, captured with the VECTRA WB360 3D total-body photography system at two clinical sites in Spain and Australia. Each tile covers roughly a 7 × 9 cm patch of skin, every suspicious lesion is marked with a bounding box, and metadata give anatomical location, age group, and a sun damage score for each image. The dataset is released publicly for the iToBoS-2024 lesion detection challenge, with the stated goal of letting researchers train and benchmark detectors on context-rich clinical images rather than isolated lesion crops, ultimately supporting early skin-cancer detection in non-clinical settings. The paper's contribution is the resource itself—its construction, privacy safeguards, and statistical characterisation—not a trained detection model.","feed_headline":"16,954 tiled skin images give detectors the missing context","feed_subtitle":"Tiles from 3D total-body scans carry lesion boxes plus skin context that isolated-lesion datasets lack.","key_machinery":"The mechanism that carries the whole contribution is the tiling-and-annotation pipeline. The VECTRA WB360 scanner's 92 cameras reconstruct a full-body 3D avatar; a dedicated tiling tool divides the avatar surface into overlapping 2D tiles of about 1012 × 827 pixels with a 45-pixel overlap, tagging each tile with an anatomical region and omitting the head to protect anonymity. On the V7 Darwin annotation platform, a trained team of medical students, nurses, and doctors draws bounding boxes around every lesion at least 2.5 mm across, assigns a sun damage score from 1 to 3, and tags tattoos and distinguishing marks for inpainting, with reviewers and then dermatologists re-checking each tile. Finally, a hierarchical stratified sampler that borrows the Wallace rule of nines allocates tiles to train and test splits, balancing anatomical region, sex-at-birth, and lesion presence while preserving rare under-30 participants. This pipeline is what converts raw total-body scans into a ground-truth detection dataset.","core_discovery":"The central claim, stated on the paper's own terms, is that a publicly releasable, clinically useful lesion-detection dataset can be produced directly from 3D total-body photographs by tiling the reconstructed avatar into 2D skin patches. The resulting dataset has 8,473 training and 8,481 test images across 100 patients, carrying 29,403 and 30,594 bounding-box annotations respectively, built through a three-phase pipeline of capture and tiling, annotation with expert review, and hierarchically stratified public-subset selection. The authors argue that the tiles preserve the surrounding-skin context that lesion-centric datasets discard, while privacy processing—masked faces, removed 3D coordinates, randomised file names, and inpainted tattoos—makes the release ethically and legally possible. Their statistical analysis shows lesion diameters peaking near 4 mm with a right-skewed tail, bounding boxes small and consistent across splits, and a roughly 4:1 ratio of lesion-bearing to lesion-free images, which they present as evidence that the data reflect real-world clinical variability. Taken together, the paper claims this resource is suitable for training and benchmarking lesion-detection algorithms in realistic conditions.","pith_inferences":["An implicit test the paper leaves open: because adjacent tiles overlap by 45 pixels, some lesions appear in multiple tiles, and comparing detector outputs on those duplicates would quantify consistency and the effect of tile-boundary context.","The paper deliberately stops at dataset construction and statistics, so a baseline detection experiment—training a standard detector on the train split and scoring the private test labels after the challenge—is the natural next step the authors do not perform.","The participant pool is limited to fair skin (Fitzpatrick I–II) and predominantly European ancestry, and most annotated lesions are benign; the dataset is therefore best read as a detection benchmark for finding lesions in fair skin, not as evidence about malignancy classification or generalisation across skin types.","Because 3D coordinates were removed and file names randomised, the public release cannot support per-patient longitudinal tracking or body-surface-level analysis; those capabilities would require access beyond the published tiles and aggregate metadata."],"forward_implications":["Researchers can train and benchmark lesion-detection algorithms on region-level images that include surrounding skin, which the paper argues is critical context that isolated-lesion datasets lack.","The 8,481 test images have labels held private until the iToBoS-2024 challenge ends, so the split can serve as a common, leakage-free benchmark for comparing detectors.","The metadata—anatomical location, age group, and sun damage score—let models condition on demographic and environmental context, which the authors say helps distinguish lesions from healthy skin.","The 2.5 mm annotation threshold and sun-damage tagging mean the dataset is aimed at clinically meaningful lesion detection, including the small and subtle cases that challenge both clinicians and algorithms.","The stratified sampling that preserves rare cases, such as participants under 30, is intended to keep clinically significant but uncommon presentations available in both training and test sets."],"supporting_citations":[{"why":"The SLICE-3D dataset built from the same VECTRA WB360 system for ISIC 2024; it establishes the prior art of lesion crops from 3D-TBP and the context the iToBoS tiles add.","marker":"[19]"},{"why":"The annotation protocol for total body photography that structures how tiles are reviewed and labelled for machine-learning use.","marker":"[20]"},{"why":"Clinical study of micro-melanomas that justifies the 2.5 mm minimum lesion diameter threshold used during annotation.","marker":"[21]"},{"why":"The anatomical burn-surface rule of nines adapted to set sampling proportions across body regions for the public subset.","marker":"[22]"},{"why":"The HAM10000 dataset, the canonical example of lesion-centric dermoscopic datasets that the iToBoS dataset is designed to complement.","marker":"[13]"}],"fun_headline_variants":["Tiled 3D body scans yield 16,954 skin images with lesion boxes","Skin-context dataset from 3D total-body photos for lesion detection","16,954 tiled skin patches enrich lesion detection with context","New dataset: tiled skin regions from 3D scans with annotations","3D total-body photos tiled into lesion-detection training set"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the bounding-box annotations are accurate and complete enough to serve as ground truth for training and benchmarking, yet the paper reports no inter-annotator agreement or error-rate figures and concedes in the usage notes that users may find occasional inconsistencies.","fun_headline_variants_meta":{"raw":{"variants":["Tiled 3D body scans yield 16,954 skin images with lesion boxes","Skin-context dataset from 3D total-body photos for lesion detection","16,954 tiled skin patches enrich lesion detection with context","New dataset: tiled skin regions from 3D scans with annotations","3D total-body photos tiled into lesion-detection training set"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000173,"raw_usage":{"total_tokens":1282,"prompt_tokens":949,"completion_tokens":333,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":565,"completion_tokens_details":{"reasoning_tokens":237}},"tokens_in":565,"tokens_out":333,"duration_ms":3881,"temperature":1.0,"reasoning_tokens":237,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T00:06:16.562507+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-annotate a random sample of about 200 tiles with two independent dermatologists who have not seen the original labels, following the same 2.5 mm minimum-size rule, and measure lesion-level agreement (matched by overlap) against the published boxes. If agreement is markedly lower for lesions under 3 mm or in high-sun-damage tiles, the ground truth would be too noisy to support reliable benchmarking; the same check on tiles that share the 45-pixel overlap could show whether duplicated lesions are labelled consistently.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The SLICE-3D dataset built from the same VECTRA WB360 system for ISIC 2024; it establishes the prior art of lesion crops from 3D-TBP and the context the iToBoS tiles add."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The annotation protocol for total body photography that structures how tiles are reviewed and labelled for machine-learning use."},{"cited_title":"The exposure treatment of burns","cited_arxiv_id":null,"evidence_quote":"The anatomical burn-surface rule of nines adapted to set sampling proportions across body regions for the public subset."}],"review_version":1}