{"id":"a3722a94-d7df-4097-a884-8fd639ee1b6a","arxiv_id":"1908.05621","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A deep learning ensemble segments geographic atrophy in color fundus images with human-level Dice scores and reveals that atrophy area grows quadratically up to about 12 mm² before growth slows.","lead":"Researchers trained an ensemble of deep learning models to automatically outline geographic atrophy, a form of age-related macular degeneration, in color photographs of the retina, reaching agreement close to that of expert graders. Applying the model to more than 5,000 images from the AREDS study, they report that atrophy area grows quadratically until about 12 mm², after which growth slows.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unvalidated transfer of the segmentation model to AREDS could make the observed growth plateau after 12 mm² an artifact of size-dependent segmentation bias; no manual AREDS validation is reported.","rationale":"The reader's verdict is CONDITIONAL, with the weakest assumption identified as the untested generalization of the segmentation model to AREDS. My analysis agrees: the natural-history result is the most load-bearing claim, and it depends on automatic segmentations that are validated only on RS/BMES. The specific mechanism I add is size-dependent segmentation bias: because the outcome is growth rate as a function of lesion area, any systematic over- or under-estimation of area that correlates with lesion size will directly bias the slope estimates and could produce a spurious plateau near 12 mm². The paper provides no quantitative check of model accuracy on AREDS, only sample images in the supplement. A stratified manual validation with bias correction would settle whether the observed deceleration is biological or computational. Since the reader already flagged exactly this assumption and recommended conditional acceptance pending additional validation, my assessment does not move the verdict.","tokens_in":12206,"tokens_out":3677,"duration_ms":41585,"concrete_test":"Manually delineate GA in a stratified random sample of AREDS images, oversampling lesions larger than 12 mm², using the same consensus grading protocol as for RS/BMES. Compare model segmentations to these manual delineations, and estimate the mean area bias as a function of true lesion size. Then recompute the Figure 4 growth-rate curve using bias-corrected automatic areas (or using the manual areas directly for the sampled eyes). If the plateau or decrease after 12 mm² persists after correction, the central claim is supported; if it disappears, the claim should be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that GA area grows quadratically up to about 12 mm² and then stabilizes or decreases—rests entirely on automatic segmentations applied to AREDS color fundus images. The model was validated only on RS/BMES, with a mean Dice of 0.72 ± 0.26 and an area ICC of 0.83; no manual delineations on AREDS are reported to confirm that the segmentation remains unbiased in this new domain. This matters because the growth-rate-versus-area curve in Figure 4 is computed from slopes of automatically segmented area over 2-year windows. If segmentation error is correlated with lesion size—for example, if large, low-contrast, or more confluent lesions are systematically under-segmented on AREDS images—then the estimated growth rates for large lesions would be biased downward, potentially creating or exaggerating the apparent deceleration after 12 mm². Conversely, small-lesion over-segmentation would inflate early growth rates and steepen the quadratic phase. The model's known failure cases and the paper's own admission of imperfect Dice make such size-dependent bias plausible. The 50% stereoscopic-exclusion criterion could also interact with this: eyes with inconsistent stereo estimates (possibly those with difficult, large, or fast-growing lesions) are removed, which could selectively alter the high-area tail of the curve. Without a direct accuracy check on AREDS, the natural-history conclusion is not securely supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript presents an ensemble deep-learning model for automatic segmentation of geographic atrophy (GA) in color fundus images (CFIs), trained and validated on 409 images from the Rotterdam Study and Blue Mountains Eye Study, achieving a Dice coefficient of 0.72 ± 0.26 against consensus manual delineations, comparable to inter-grader Dice values of 0.72–0.82. The model is then applied to 5,379 CFIs from 625 eyes in the Age-Related Eye Disease Study (AREDS) to measure GA growth. The authors report that eight automatically extracted structural biomarkers are significantly associated with the square-root annual growth rate, and that combining all per-eye growth estimates suggests GA area grows quadratically up to about 12 mm², after which the growth rate stabilizes or decreases. The paper concludes that the model enables fully automatic GA segmentation and can support large-scale natural-history analyses.","tokens_in":12493,"tokens_out":7326,"duration_ms":65234,"significance":"If the findings hold, the paper makes two useful contributions. First, it provides a fully automatic deep-learning pipeline for GA segmentation in CFIs, with validation against multiple expert graders on two population-based cohorts; the model's performance is close to the inter-grader variability, which is a meaningful benchmark. Second, it demonstrates the feasibility of applying such a model to a large longitudinal dataset (AREDS) and reproduces known associations (e.g., multifocal and extrafoveal lesions growing faster), while proposing novel morphometric biomarkers. The central natural-history claim—a quadratic growth phase followed by a plateau—would be of substantial clinical and trial-design interest if it survives the additional validation described below. The paper also includes a fairly detailed model description in the appendix, which aids reproducibility, though code and trained models are not provided.","major_comments":[{"comment":"The model's segmentation accuracy on AREDS is never assessed; all reported performance metrics (Dice 0.72, ICC 0.83) come from RS/BMES. Because the growth-rate-versus-area curve in Figure 4 is derived entirely from automatic segmentations on AREDS, a size-dependent segmentation bias could create or exaggerate the apparent deceleration after ~12 mm². For example, under-segmentation of large, low-contrast, or confluent lesions would lower estimated growth rates for large lesions, while over-segmentation of small lesions would inflate early growth. The authors should validate the model on a manually graded AREDS subset (the AREDS reading-center gradings referenced in ref. 11 are an appropriate source) and report Dice and ICC stratified by lesion size and image quality, or at least compare automatic areas with available manual AREDS area measurements.","section":"Methods, 'GA growth rate' (first paragraph)"},{"comment":"The exclusion of visits with more than 50% relative difference in automatically segmented GA area between stereoscopic images is applied without a reported rationale or a characterization of the 41 excluded eyes (625 enrolled, 584 analyzed). If stereoscopic discordance is more common for large or fast-growing lesions, the exclusion could selectively remove the large-area tail of the growth curve and contribute to the observed plateau. The authors should report the excluded eyes' baseline characteristics and test the robustness of Figure 4 to alternative thresholds (e.g., 30% and 70%).","section":"Methods, 'GA growth rate' (stereoscopic exclusion)"},{"comment":"The claim that GA area grows quadratically up to ~12 mm² and then stabilizes is based on pooling per-eye 2-year growth slopes across eyes and fitting a quadratic curve with a data-dependent cutoff at 12 mm². This procedure does not demonstrate that individual eyes follow this trajectory; eyes reaching large areas are a selected subset, and cross-sectional pooling can mask heterogeneous individual paths. The authors should analyze individual longitudinal data with a prespecified mixed-effects model (e.g., including linear and quadratic time terms with a changepoint) and report how many eyes cross the 12 mm² threshold and their growth patterns.","section":"Methods, 'GA growth rate' (last paragraph) and Figure 4"},{"comment":"The fixed pixel-to-millimeter conversion based on an assumed 4.5 mm fovea-to-disc distance affects all area and growth estimates, including the 12 mm² threshold. The authors acknowledge this limitation but do not quantify its impact. A sensitivity analysis with alternative conversion factors (e.g., ±10%) should be reported, and if possible the authors should calibrate magnification per image using the measured fovea-to-disc distance.","section":"Methods, 'Data' (pixel-to-mm conversion) and Discussion"}],"minor_comments":[{"comment":"The study is described as 'Prospective, multicenter, natural history study,' but the model development and validation are retrospective analyses of existing cohort data; the design should be described as a retrospective analysis of prospective cohorts.","section":"Abstract, 'Design' line"},{"comment":"The participant counts (409 CFIs for development, 5,379 CFIs for analysis) are reported inconsistently with the evaluation N=315 in Results; the distinction between images and unique visits should be stated consistently.","section":"Abstract and Methods"},{"comment":"The claim that this is 'the first deep learning model for segmentation of GA in CFI' should be reconciled with reference 21 (Keenan et al.), which describes a deep-learning approach for automated detection of GA from color fundus photographs; if that work also performs segmentation, the novelty claim should be narrowed.","section":"Introduction and reference 21"},{"comment":"The feature 'Fovea region' in Table 3 corresponds to 'foveal involvement' in the appendix; the naming should be unified.","section":"Table 3 and appendix"},{"comment":"The right panel's 'evolution of GA area over time' is obtained by numerical integration starting from an assumed area of 0.5 mm² at t=0; the dependence of the resulting curve on this initial condition and on the integration method should be stated.","section":"Figure 4"},{"comment":"The window selection criterion ('for which the number of available CFIs was highest for the respective eye') should specify how ties are resolved and how many timepoints were typically available in the chosen window.","section":"Methods, 'GA growth rate'"},{"comment":"The paper should state whether the trained model or code is publicly available; this would enhance reproducibility.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's central natural-history claim rests on the unvalidated application of the segmentation model to AREDS. Given that AREDS has existing reading-center gradings, the absence of a validation step is the main gap; the revision should either add that validation or substantially temper the claim. Also, the novelty claim ('first deep learning model for segmentation of GA in CFI') should be checked against reference 21 before final acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid segmentation paper with a natural-history add-on that is more suggestive than conclusive. The first deep learning model for GA in color fundus images, built from an ensemble of encoder-decoders and validated with five-fold cross-validation against four graders. Dice of 0.72 is right in the range of inter-grader agreement (0.72–0.82), and the ICC on area is 0.83. That part is real and useful.\n\nThe growth-rate analysis on AREDS (5,379 images, 625 eyes) is the flashier claim: GA area grows quadratically up to about 12 mm² and then stabilizes or decreases. But the model was trained on RS/BMES and applied to AREDS without any manual validation on AREDS images. The supplement shows example segmentations, not quantified agreement. So the central curve in Figure 4 could be biased if segmentation error is size-dependent. That concern is legitimate. The paper's own Dice of 0.72 ± 0.26 includes failure cases, and large low-contrast lesions could plausibly be under-segmented, flattening the high-area tail.\n\nThe mitigating fact is that the slowdown for large lesions was already reported by Keenan et al. using manual graders (ref 41), and the paper includes that comparison. So the artifact story would have to produce exactly the pattern that manual measurements already showed. That reduces the risk but doesn't remove it, because the 12 mm² cutoff and the exact curve shape could still be wrong.\n\nOther soft spots: the pixel-to-mm conversion is a fixed 4.5 mm for all eyes, which the authors acknowledge. The 50% stereoscopic exclusion is post hoc and could remove fast-growing or difficult eyes. The multivariate model explains only 18% of growth variance. None of these are fatal, but they all push in the direction of treating the natural-history result as hypothesis-confirming rather than new discovery.\n\nWho benefits: anyone working on automatic grading of retinal images, and trialists who need scalable endpoints for GA. The segmentation model is a step forward. The natural-history part is best read as a large-scale replication of known patterns.\n\nRecommendation: send to peer review. The method deserves referee time. Ask for either a manual validation subset on AREDS or a sensitivity analysis of the exclusion threshold and segmentation uncertainty. That would turn a conditional accept into a solid one.","headline":"Solid segmentation model, but the natural-history claim depends on unvalidated transfer to AREDS—treat the 12 mm² plateau as suggestive until manual validation appears.","tokens_in":13086,"tokens_out":2349,"would_cite":true,"duration_ms":22965,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A deep learning model can segment geographic atrophy in color fundus images accurately enough to trace its long-term natural history, revealing that GA area grows quadratically up to about 12 mm² and then stabilizes or decreases.","keywords":["geographic atrophy","deep learning segmentation","color fundus imaging","age-related macular degeneration","disease progression","natural history","structural biomarkers","ensemble model"],"falsifier":"Take a stratified sample of images from the target dataset spanning small and large lesions, have expert graders delineate GA, and compare automatic and manual areas per size bin; if the automatic-versus-manual bias changes systematically as lesions approach 12 mm², the plateau may be a measurement artifact rather than true natural history.","tokens_in":12005,"feed_emoji":"👁️","tokens_out":5233,"duration_ms":50231,"temperature":0.7,"pith_summary":"This paper develops a fully automatic deep learning segmentation model for geographic atrophy (GA), the progressive death of retinal pigment epithelium in advanced age-related macular degeneration, and validates it against four expert graders on 409 color fundus images. The model reaches a Dice coefficient of 0.72 and an intraclass correlation of 0.83 for GA area, close to inter-grader agreement. Applied to 5,379 images from a large longitudinal dataset, the segmentations yield square-root annual growth rates across 584 eyes. The central natural-history claim is that GA area grows quadratically with time up to a lesion area of about 12 mm², after which growth rate stabilizes or decreases. The paper also identifies eight automatically computed structural features—area, filled area, convex area, convex solidity, eccentricity, roundness, foveal involvement, and perimeter—that are significantly associated with future growth rate.","feed_headline":"Eye atrophy growth plateaus after 12 mm²","feed_subtitle":"Automatic segmentation of 5,379 color fundus images traces geographic atrophy and maps its natural history.","key_machinery":"The central object is an ensemble of 20 encoder-decoder deep convolutional networks with residual blocks and shortcut connections, trained on consensus manual delineations from four graders in two population-based cohorts. Each network maps a color fundus image, together with a contrast-enhanced version, to a per-pixel likelihood of GA; ensemble predictions are combined after per-model threshold correction. The other load-bearing device is the square-root transformation of GA area: growth rate is measured as the slope of a linear regression through the square root of area over time, which removes the baseline-size dependence and lets the authors pool growth rates across eyes into a single curve.","core_discovery":"The paper's central claim is that an ensemble of encoder-decoder deep networks can segment GA in color fundus images with accuracy approaching inter-grader agreement, and that the resulting segmentations, applied at scale to a longitudinal dataset, reveal a consistent natural-history pattern: square-root-transformed GA area grows roughly linearly (that is, area grows quadratically) while lesions are small, but the growth rate in mm² per year stops increasing and stabilizes or declines once the atrophic area reaches approximately 12 mm². The authors argue this pattern explains the dependence of growth rate on baseline area and appears both in cross-sectional pooling of growth rates and in individual eyes followed over many years.","pith_inferences":["The pooled growth curve mixes many eyes at different disease stages, so the plateau near 12 mm² could reflect a cohort effect rather than a universal within-eye phase; tracking individual lesions through that size would directly test the claim.","If the plateau is confirmed in within-eye data, the standard square-root transformation, which assumes constant radial expansion, would need revision for large lesions because radial speed would appear to slow.","Applying the same segmentation pipeline to fundus autofluorescence or OCT images could test whether the plateau is specific to color fundus imaging or a true biological feature of atrophy growth."],"forward_implications":["Fully automatic segmentation makes it practical to measure GA area and growth in thousands of eyes from standard color fundus images without manual grading.","If the plateau near 12 mm² is real, square-root annual growth rate cannot be treated as constant across the disease; clinical trials should stratify or adjust for baseline lesion size.","Eight automatically computed structural features predict future growth, offering a route to enrich clinical trials with fast-progressing eyes.","The quadratic-to-saturating growth curve provides a quantitative natural-history benchmark that future interventions can be compared against."],"supporting_citations":[{"why":"Provides the longitudinal AREDS dataset and prior manual area-progression measurements that the model's application builds on.","marker":"11"},{"why":"Introduces the square-root transformation of GA area that removes the dependence of growth rate on baseline lesion size, the metric used throughout the study.","marker":"26"},{"why":"Supplies the encoder-decoder architecture with shortcut connections that the segmentation ensemble adapts to color fundus images.","marker":"39"},{"why":"Reports similar GA growth measurements in a related study and is plotted alongside this paper's estimates for comparison.","marker":"41"},{"why":"Provides the fovea-to-disc distance used to convert pixels to millimeters, setting the scale for all area and growth values.","marker":"37"},{"why":"Establishes circularity index as a risk factor for GA progression, a prior association that the structural-feature analysis extends.","marker":"23"}],"fun_headline_variants":["GA growth slows after 12 mm², deep learning reveals","Deep learning maps geographic atrophy growth plateau","Eye atrophy growth plateaus at 12 mm², AI shows","Quadratic growth of geographic atrophy stops past 12 mm²","AI segmentation uncovers natural history of geographic atrophy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The model is trained on one set of color fundus images and applied to a second large dataset without any manual delineations on that second dataset, so segmentation errors that vary with lesion size or image quality could, in principle, create or mask the observed growth plateau.","fun_headline_variants_meta":{"raw":{"variants":["GA growth slows after 12 mm², deep learning reveals","Deep learning maps geographic atrophy growth plateau","Eye atrophy growth plateaus at 12 mm², AI shows","Quadratic growth of geographic atrophy stops past 12 mm²","AI segmentation uncovers natural history of geographic atrophy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000556,"raw_usage":{"total_tokens":2694,"prompt_tokens":1040,"completion_tokens":1654,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":656,"completion_tokens_details":{"reasoning_tokens":1575}},"tokens_in":656,"tokens_out":1654,"duration_ms":12061,"temperature":1.0,"reasoning_tokens":1575,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:08:00.699767+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a stratified sample of images from the target dataset spanning small and large lesions, have expert graders delineate GA, and compare automatic and manual areas per size bin; if the automatic-versus-manual bias changes systematically as lesions approach 12 mm², the plateau may be a measurement artifact rather than true natural history.","supporting_citations":[{"cited_title":"Change in area of geographic atrophy in the Age -Related Eye Disease Study: AREDS report number 26","cited_arxiv_id":null,"evidence_quote":"Provides the longitudinal AREDS dataset and prior manual area-progression measurements that the model's application builds on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the square-root transformation of GA area that removes the dependence of growth rate on baseline lesion size, the metric used throughout the study."},{"cited_title":"EyeNED workstation: development of a multi-modal vendor-independent application for annotation, spatial alignment and analysis of retinal images","cited_arxiv_id":null,"evidence_quote":"Supplies the encoder-decoder architecture with shortcut connections that the segmentation ensemble adapts to color fundus images."},{"cited_title":"Kaggle diabetic retinopathy detection competition report","cited_arxiv_id":null,"evidence_quote":"Reports similar GA growth measurements in a related study and is plotted alongside this paper's estimates for comparison."},{"cited_title":"The Rotterdam Study: 2018 update on objectives, design and main results","cited_arxiv_id":null,"evidence_quote":"Provides the fovea-to-disc distance used to convert pixels to millimeters, setting the scale for all area and growth values."},{"cited_title":"Circularity index as a risk factor for progression of geographic atrophy","cited_arxiv_id":null,"evidence_quote":"Establishes circularity index as a risk factor for GA progression, a prior association that the structural-feature analysis extends."}],"review_version":1}