{"id":"bc754a92-5199-44ca-9d81-f58a7c64e1e9","arxiv_id":"2607.03643","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Confidence-guided transformer fusion of multiple fundus images yields higher balanced accuracy and sensitivity at matched coverage than quality-cascade or single-image baselines on mBRSET and BRSET.","lead":"A lightweight AI pipeline fuses multiple smartphone fundus photos and rejects low-confidence cases, beating traditional image-quality filters for diabetic retinopathy screening. It could make mobile eye screening more reliable in places with few specialists.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.5","headline":"The headline gains rest on a cascade baseline that is not matched for multi-image aggregation, so the ~12%/6% BA lift may overstate the value of confidence itself.","rationale":"The reader correctly flags calibration and Brazilian fixed-count data as the weakest assumption and assigns CONDITIONAL with medium correctness risk. That is directionally right, but the more load-bearing internal issue is experimental confounding: the cascade is single-image while the proposed method is multi-image, so the ~12%/6% BA and large sensitivity gains cannot be attributed cleanly to confidence. Tables 3–4 already quantify a substantial fusion-only lift; the selective-prediction curves therefore mix two effects. A matched multi-image quality cascade (or quality-then-fuse arm) is the single check that would settle whether confidence is doing the claimed work. Until that is shown, the translational claim remains conditional on the same grounds the reader gave, plus this attribution gap. No derivation is broken and code/data are public, so I do not move to REJECT; I keep CONDITIONAL and note that the reader’s weakest_assumption under-weights the unmatched baseline.","tokens_in":17994,"tokens_out":655,"duration_ms":5954,"concrete_test":"Re-run the cascade on mBRSET/BRSET by first applying the quality model to every available image, then fusing only the images that pass a quality threshold (mean/max or the same transformer with masking) and evaluate BA/sensitivity at the exact 70% and 50% coverage points used in Figs. 3–4. If the multi-image quality cascade closes most of the gap to the confidence-fusion curves, the headline attribution to confidence is overstated.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim (Abstract/Results: ~12% and ~6% BA gains at 70% coverage; 94%/96% vs 61%/86% sensitivity at 50% coverage) attributes superiority to confidence-guided multi-image fusion over “cascade filtering.” But the cascade is defined as a single-image quality filter followed by a single-image DR model (Methods §Single-Image…; Fig. 1a), while the proposed method fuses 2–4 images with a transformer then applies a symmetric confidence margin on the fused probability (Fig. 2; Proposed Framework). Tables 3–4 already show that multi-image fusion alone lifts BA from 0.77→0.84 (mBRSET) and 0.90→0.92 (BRSET) without any confidence threshold. Figures 3–4 therefore compare an unfused cascade against a fused confidence system; the large reported deltas conflate the benefit of multi-view aggregation with the benefit of confidence-based deferral. The paper’s own calibration (Fig. 6) further shows systematic overconfidence, so the operating points that produce the headline numbers may not transfer. Without a multi-image quality cascade (or quality-filtered fusion) at matched coverage, the claim that “confidence-based filtering consistently outperforms image quality-based cascaded pipelines” is not cleanly supported.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes a confidence-guided multi-image fusion pipeline for patient-level diabetic retinopathy (DR) screening on fundus photographs. Using a lightweight RETFoundGreen backbone, it (i) fuses multiple images per patient with a small transformer (or mean/max pooling baselines), then (ii) applies a symmetric confidence margin around 0.5 on the fused probability to defer low-confidence cases. On mBRSET (mobile, n=1,234 patients after filtering) and BRSET (clinic, n=7,599), the authors report that at 70% coverage their method reaches ~91% and ~97% balanced accuracy—gains of ~12 and ~6 points over a single-image quality-cascade baseline—and higher sensitivity at matched coverage. They further argue that human image-quality labels are only weakly associated with diagnostic accuracy (Table 6, saliency), so model confidence is a better filter than explicit quality assessment, and that the pipeline is suitable for offline mobile screening.","tokens_in":18387,"tokens_out":1246,"duration_ms":18569,"significance":"If the comparative claims hold under cleaner ablations, the work is practically relevant for LMIC mobile DR screening: it shows that selective deferral plus multi-view aggregation can improve the accuracy–coverage tradeoff without Bayesian multi-pass uncertainty, using a ~25M-parameter offline model and public datasets with released code (SureSight). The independent finding that human quality labels poorly track diagnostic utility (Table 6; occlusion maps) is useful and challenges common cascade designs. Strengths include patient-level splits, mean/max/transformer fusion baselines, coverage-matched operating curves, calibration analysis, and saliency. The main significance risk is attribution: headline gains currently mix multi-image fusion with confidence filtering against a single-image quality cascade, so the incremental value of confidence versus multi-view aggregation is not fully isolated.","major_comments":[{"comment":"The headline BA/sensitivity gains (Abstract; Results text for Figs. 3–4: ~12%/6% BA at 70% coverage; 94%/96% vs 61%/86% sensitivity at 50% coverage) compare a single-image quality cascade (Methods, Fig. 1a) to confidence-based multi-image fusion (Fig. 2). Tables 3–4 already show multi-image fusion alone lifts BA (mBRSET 0.77→0.84; BRSET 0.90→0.92) without confidence thresholds. Figs. 3–4 therefore conflate multi-view aggregation with confidence-based deferral. Please add matched multi-image quality-cascade (or quality-filtered fusion) and report confidence-only vs fusion-only deltas at fixed coverage so the claim that “confidence-based filtering consistently outperforms image quality-based cascaded pipelines” is cleanly supported.","section":null},{"comment":"Fig. 6 shows systematic overconfidence (mean predicted probability exceeds observed DR rate in most bins). Operating points that produce the Abstract numbers are obtained by sweeping the symmetric margin T on the test set (Proposed Framework; Evaluation). For translational claims about “prompting retakes” and reduced incorrect diagnoses, either (a) fix T on validation only and report test metrics once, or (b) apply a simple calibration step (e.g., temperature scaling) and re-evaluate coverage–BA/sensitivity curves. Directional calibration supports ranking, but uncorrected overconfidence weakens the reliability interpretation of high-confidence negatives.","section":null},{"comment":"Methods restrict evaluation to patients with fixed image counts (exactly two per eye for mBRSET; one per eye for BRSET; Tables S1–S2), then assign patient labels by max DR across eyes. Real screening has variable numbers of captures and missing eyes. Please quantify selection bias (how many patients/images were dropped and whether DR prevalence differs) and report performance when the transformer is evaluated with naturally incomplete sets (beyond the masking used for two-image mBRSET), or state this as a hard limitation on the coverage claims.","section":null}],"minor_comments":[{"comment":"Fig. 5 caption: “mage quality” → “Image quality”.","section":null},{"comment":"Abstract Methods lists three comparators including “confidence-based prediction,” but Results lead with cascade vs multi-image fusion; make the single-image confidence arm as explicit in the Abstract numbers as in Figs. 3–4.","section":null},{"comment":"Equation (1): notation for class weights is hard to parse in the text rendering (w0/w1 definitions); rewrite with standard weighted CE notation.","section":null},{"comment":"Tables 3–4 highlight “highest mean” but several metrics are within reported SD of each other; consider marking statistical comparisons or confidence intervals rather than bold alone.","section":null},{"comment":"Related work on uncertainty-aware DR (refs 26–27) is cited; a short quantitative note on wall-clock cost vs MC-dropout/ensemble on the same backbone would strengthen the “single inference pass” deployment claim.","section":null},{"comment":"BRSET DR prevalence is only 6.4% (Table 2); emphasize that high specificity/BA can be driven by the negative class and report prevalence-sensitive metrics at the chosen operating points more prominently.","section":null}],"recommendation":"major_revision","confidential_remarks":"The skeptic’s conflation concern is real and load-bearing for the Abstract wording, but the paper still has a solid secondary result (quality labels ≠ diagnostic utility) and usable engineering contribution. I would not reject; major revision with a multi-image quality cascade and validation-fixed thresholds should be enough. Scope fits eess.IV / translational imaging well. No integrity red flags; code and data are public."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful takeaway is straightforward: on mBRSET and BRSET, patient-level transformer fusion of 2–4 fundus images plus a simple confidence margin around 0.5 beats a quality-cascade pipeline at matched coverage, and human quality labels barely track diagnostic accuracy. That second point is the cleaner contribution. Tables 6a/b and the saliency examples show low- and high-quality images give nearly identical balanced accuracy; the quality model itself is decent (Table 5) yet cascade filtering still plateaus. Confidence-based rejection plus multi-view aggregation is the practical fix they demonstrate.\n\nWhat they do well: patient-level splits, mean/max/transformer fusion ablations (Tables 3–4), coverage-matched curves (Figs 3–4), calibration (Fig 6), open code/data, and a lightweight RETFoundGreen backbone that is actually mobile-plausible. The fusion lift alone is real (0.77→0.84 BA on mBRSET, 0.90→0.92 on BRSET). Sensitivity at 50% coverage jumps a lot under their protocol. Citation pattern is fair; they engage DeepFundus, binocular CNNs, and uncertainty work without overclaiming novelty of the individual pieces.\n\nSoft spot, in proportion: the stress-test is right that the cascade is single-image while the proposed system is multi-image. Figures 3–4 therefore mix the benefit of fusion with the benefit of confidence. A multi-image quality cascade (or quality-filtered fusion) at the same coverage would isolate the claim that “confidence-based filtering consistently outperforms image quality-based cascaded pipelines.” Calibration also shows systematic overconfidence, so the exact operating points may shift. Both datasets are Brazilian with fixed image counts; generalizability is the usual open question, not a fatal hole.\n\nThis is for people building or evaluating smartphone DR screening pipelines who care about accuracy-coverage tradeoffs and whether to keep a separate quality model. It is not a foundational methods paper. I would send it to peer review; the evidence is reproducible enough and the clinical framing is honest. Engage if you work on mobile fundus AI; otherwise skim the quality-label analysis and move on.","headline":"Solid practical result on confidence + multi-image fusion for mobile DR screening, but the headline ~12%/6% gains partly conflate fusion with confidence because the cascade baseline stays single-image.","tokens_in":18926,"tokens_out":541,"would_cite":true,"duration_ms":5471,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Model confidence plus multi-image fusion beats human quality labels for reliable diabetic retinopathy screening from fundus photos.","keywords":["diabetic retinopathy","fundus imaging","model confidence","multi-image fusion","selective prediction","mobile screening","image quality","patient-level diagnosis"],"falsifier":"On a new multi-image screening cohort from a different population, measure whether a confidence-thresholded fusion model still beats a quality-cascade baseline at matched coverage on balanced accuracy and sensitivity; if the cascade equals or exceeds fusion, or if calibration collapses so that high-confidence bins no longer track true disease rates, the central claim fails.","tokens_in":18943,"feed_emoji":"👁️","tokens_out":866,"duration_ms":12558,"temperature":0.7,"pith_summary":"This paper argues that for diabetic retinopathy screening from smartphone or clinic fundus images, the model's own prediction confidence is a better filter for unreliable cases than human-annotated image-quality scores. By fusing several images per patient with a lightweight transformer and then accepting only high-confidence patient-level predictions, the method raises balanced accuracy and sensitivity while allowing low-confidence cases to be deferred for a retake. On two Brazilian datasets (mobile and tabletop), the approach substantially outperforms a classic cascade that first discards \"poor-quality\" images and then diagnoses the rest. The practical payoff is more trustworthy automated screening at capture time in settings where specialists are scarce, using a small model that needs only one forward pass per image.","feed_headline":"Confidence beats quality labels for diabetic eye screening","feed_subtitle":"Fusing multiple fundus images and keeping only high-confidence predictions lifts accuracy while flagging retakes","key_machinery":"Confidence-guided multi-image fusion: a shared lightweight vision backbone embeds each fundus image; a small transformer aggregates the embeddings into one patient-level probability (supporting variable image counts via masking); symmetric confidence margins around 0.5 then decide whether to accept the diagnosis or defer the patient for retake.","core_discovery":"Human quality labels are only weakly linked to actual diagnostic accuracy, whereas confidence-based rejection of patient-level predictions, especially after multi-image fusion, consistently yields higher balanced accuracy and sensitivity at the same coverage rates. At 70 percent patient coverage the fused confidence pipeline reaches roughly 91 percent balanced accuracy on the mobile set and 97 percent on the clinic set, gains of about 12 and 6 points over quality cascade filtering; at 50 percent coverage it reaches 94 percent and 96 percent sensitivity versus 61 percent and 86 percent for the cascade.","pith_inferences":["If confidence truly tracks diagnostic utility better than visual quality, many existing \"ungradable\" rejection rules in retinal AI pipelines may be discarding usable images and lowering overall screening yield.","Mild overconfidence may be tolerable for high-sensitivity screening thresholds, but the same system would need recalibration or different margins before high-confidence negatives could safely discharge patients without human review.","The masking-friendly transformer suggests a practical path for clinics that capture uneven numbers of images per eye without forcing a fixed two-image binocular architecture."],"forward_implications":["Screening programs can replace or de-emphasize separate quality classifiers and simply use the diagnostic model's confidence to decide which patients need an immediate retake.","Patients kept at high-confidence operating points receive fewer incorrect automated diagnoses, reducing both missed disease and unnecessary referrals.","Because the pipeline is lightweight and needs only one inference per image, it can run offline on mobile devices used for field fundus capture.","The same confidence-plus-fusion pattern can be applied wherever multiple fundus views of a patient are routinely acquired."],"fun_headline_variants":["Confidence beats quality labels for DR screening accuracy","Multi-image confidence fusion lifts diabetic eye diagnosis","Model confidence filters beat human quality labels in DR","Fundus fusion via confidence improves retinopathy screening","High-confidence multi-view fusion outperforms quality cascade"],"cache_read_input_tokens":128,"weakest_assumption_plain":"The load-bearing premise is that the model's fused probability, even when mildly overconfident, ranks diagnostic reliability well enough that rejecting low-confidence patients improves real screening outcomes.","fun_headline_variants_meta":{"raw":{"variants":["Confidence beats quality labels for DR screening accuracy","Multi-image confidence fusion lifts diabetic eye diagnosis","Model confidence filters beat human quality labels in DR","Fundus fusion via confidence improves retinopathy screening","High-confidence multi-view fusion outperforms quality cascade"]},"model":"grok-4.5","effort":"low","cost_usd":0.005126,"raw_usage":{"total_tokens":1521,"prompt_tokens":904,"num_sources_used":0,"completion_tokens":53,"cost_in_usd_ticks":51260000,"prompt_tokens_details":{"text_tokens":904,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":564,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":904,"tokens_out":53,"duration_ms":4254,"temperature":1.0,"reasoning_tokens":564,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-12T00:58:13.611818+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On a new multi-image screening cohort from a different population, measure whether a confidence-thresholded fusion model still beats a quality-cascade baseline at matched coverage on balanced accuracy and sensitivity; if the cascade equals or exceeds fusion, or if calibration collapses so that high-confidence bins no longer track true disease rates, the central claim fails.","supporting_citations":[],"review_version":1}