{"id":"ff1a04e9-2fca-4b49-b8ce-e1d801c00498","arxiv_id":"2412.13599","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A co-evolutionary framework that alternates between chest X-ray abnormality detection and radiology report generation, using each task to refine the other's pseudo-labels, achieves state-of-the-art results on MS-CXR and PD-CXR.","lead":"CoE-DG couples a chest X-ray abnormality detector and a radiology report generator so that each task generates pseudo-labels that improve the other, using images that have reports but no bounding boxes as extra training data. On MS-CXR and PD-CXR, it outperforms separate detection and report generation baselines by modest but consistent margins, with the largest gains when box annotations are scarce.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DIP encodes detector omissions as 'no abnormality' labels: when detection finds no box on a weakly labeled image, the generator's class token is supervised as background despite the paired report, and GIP then filters pseudo-labels with that same biased signal, reinforcing missed lesions.","rationale":"The reader's weakest assumption is the mutual pseudo-label loop. I agree with the direction but believe there is a sharper, untested mechanism that makes the concern concrete: Section III.F and Eq. (7) together turn detector omissions into false negative supervision for the generator's class token on weakly labeled images, and Eq. (8) then uses that same class token to filter detector pseudo-labels. This is not just shared noise; it is an asymmetric channel through which the detector's initial blind spots are written into the generator and then re-imposed on future detectors. I chose this over the missing error bars because the statistical concern, while real, is a general property of the evaluation, whereas the DIP background-label issue is specific to the proposed architecture and directly targets the central mutual-promotion claim. The proposed ablation isolates the mechanism by changing only the classification supervision for no-detection weak images. If the concern lands, the central claim remains plausible but requires an additional condition; if it does not, the current conditional verdict stands. Hence verdict remains UNCHANGED from the reader's CONDITIONAL.","tokens_in":26370,"tokens_out":9843,"duration_ms":97927,"concrete_test":"Run an ablation on MS-CXR: for weakly labeled images where the current detector emits no box above the 0.9 threshold, modify DIP so the auxiliary classification loss L_R^cls is not supervised with the whole-image background label (either drop the term for those samples or use report-derived image-level labels from MIMIC-CXR/CheXBert as y_I). Retrain the co-evolution loop under the same protocol. Additionally, measure the generator's class-token recall on fully labeled test images where the initial teacher detector misses the ground-truth abnormality. If final detection mAP@0.5 or report AUC improves, or recall on missed-lesion images is low, the background-label rule is actively reinforcing detector omissions and the mutual-promotion claim is weakened; if performance stays flat or drops, the rule is harmless.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III.F specifies that when the student detector finds no abnormality in a weakly labeled CXR, the whole image is treated as a background pseudo-box (y=0, B=(0,0,W,H)) and fed to the generator. Through Eq. (7), the auxiliary classification term L_R^cls = CE(y_I, yhat_R) then trains the generator's class token to predict 'no abnormality' for that image, regardless of whether its paired report actually contains findings. The report-generation term still sees the true text, but the class token—the exact signal used by GIP's filter in Eq. (8)—is pushed toward false negatives. In the next detector iteration, GIP only retains pseudo-labels whose categories appear in the generator's predicted abnormality set, so a generator trained to call missed-lesion images normal will suppress any newly proposed correct boxes. This is a concrete feedback path for shared blind spots, more specific than generic pseudo-label noise: DIP does not merely omit information about missed abnormalities; it injects an erroneous negative supervision signal. The mechanism is consistent with the semi-oracle result (Table III: 38.05 vs 37.89 at mAP@0.5) and failure cases (Fig. 9), but those observations do not isolate it. Because the paper's central claim is that the two tasks mutually improve each other, an asymmetric error-propagation path that can actively erase hard positives is the most load-bearing risk.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CoE-DG, a co-evolutionary framework that jointly trains an abnormality detector and a radiology report generator on chest X-rays, using a small fully labeled set (boxes plus reports) and a large weakly labeled set (reports only). The method alternates two information-propagation mechanisms: GIP, in which the generator's predictions and features refine the detector's pseudo labels and input features, and DIP, in which the detector's boxes and categories condition the generator and provide auxiliary classification supervision. The authors report state-of-the-art results on MS-CXR and PD-CXR for detection (mAP at several IoU thresholds) and for report generation (BLEU/METEOR/ROUGE/AUC) with three generation backbones, together with ablations, a semi-oracle comparison, normal-case validation, convergence analysis, and computational-cost measurements. The central claim is that the two tasks mutually improve each other, enabling both accurate abnormality localization and clinically useful report generation from mostly weakly labeled data.","tokens_in":26677,"tokens_out":5011,"duration_ms":49013,"significance":"If the claims hold, the paper makes a practically valuable contribution: it shows that the abundant report-only supervision in hospital archives can be exploited to improve a box-level detector, and that detector outputs can in turn improve report generation, without requiring large paired box-and-report datasets. The manuscript is strengthened by extensive experiments and ablations on two public datasets, a semi-oracle experiment that upper-bounds the GIP filtering approximation, an explicit normal-case evaluation, release of code, and a clear comparison with several strong baselines. The main risk is that the mutual-bootstrap design has an asymmetric error-propagation path that can actively reinforce missed abnormalities, and the detection-side SOTA claim currently rests on single-run mAP numbers without significance testing.","major_comments":[{"comment":"The no-detection handling introduces a systematically biased negative supervision signal into the GIP loop. When the student detector finds no box on a weakly labeled image, the whole image is treated as a background pseudo-box (y=0, B=(0,0,W,H)) and fed to the generator; through the L_R^cls term in Eq. (7), the class token is trained to predict 'no abnormality' regardless of whether the paired report actually describes findings. Because Eq. (8) retains only detector pseudo-labels whose categories appear in the generator's predicted abnormality set, this biased class token can suppress newly proposed correct boxes in the next co-evolution iteration. This is a concrete asymmetric error-propagation path for shared blind spots, not merely generic pseudo-label noise. The semi-oracle experiment (Table III, 38.05 vs 37.89 mAP@0.5) and the failure cases in Fig. 9 are consistent with this mechanism but do not isolate it. Please quantify how often the no-detection path co-occurs with report-described abnormalities, and provide an ablation that replaces y_I=0 with report-derived positive supervision (or otherwise breaks the negative feedback) to show the framework is robust to this path.","section":"III.F, Eqs. (7)-(8)"},{"comment":"Detection results are reported as single mAP values without variance or significance testing, although the claimed SOTA margins over Soft Teacher are 2.07, 1.30, and 1.15 percentage points at IoU thresholds 0.25, 0.5, and 0.75 on MS-CXR. Given the known sensitivity of pseudo-label semi-supervised detection to initialization and seed, please report mean plus/minus standard deviation over at least three runs and apply a paired statistical test (for example, the Wilcoxon signed-rank test over per-test-image scores, as already used for report generation in Section IV.A) to support the central claim that CoE-DG outperforms the compared detection methods.","section":"IV.D, Table III"}],"minor_comments":[{"comment":"The sentence introducing Eq. (8) says 'given a weakly labeled image x in Dl'; this should be Dw, since GIP is applied to weakly labeled images.","section":"III.E.2"},{"comment":"In DIP, the projection network f_R_prj is defined on image patches cropped according to detected boxes, but in GIP the same notation is used as f_R_prj(x) on the full image; please clarify the input resolution and architecture used for the whole-image embedding to ensure reproducibility.","section":"III.E.1 and III.E.2"},{"comment":"In the observations paragraph after Table III, 'MX-CXR' should be 'MS-CXR'.","section":"IV.D"},{"comment":"The normal-case evaluation is based on 30 randomly selected cases, and the reported TNR differences (10% and 3.33%) correspond to only a few cases; please provide a confidence interval or justify the sample size.","section":"IV.F, Tables VII-VIII"},{"comment":"The comparison with LLaVA-Med, CheXagent, and XrayGPT uses officially released pretrained models for inference without fine-tuning on MIMIC-CXR, so their absolute numbers are not directly comparable with the task-specific models trained on the target datasets; this limitation should be stated more prominently.","section":"IV.E"},{"comment":"The phrase 'language-efficiency metrics' is used in several places (e.g., Section IV.E and Section V); 'language-efficacy' or 'language quality' would be more accurate and consistent.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The main reason for major revision is the negative-feedback path in Section III.F: if the authors can experimentally close or bound it, the central mutual-improvement claim would be substantially stronger. The detection-side lack of significance testing is also important given the small margins over Soft Teacher. No concerns about novelty disclosure: the prior conference paper is explicitly cited and the differences are clearly stated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, what to know: this is a competent engineering contribution that extends the authors' prior CEIRD detector to joint report generation, with thorough ablations and a public code release. The gains are real but modest, and there's a specific feedback path I want you to keep in mind because I think it's the load-bearing risk.\n\nWhat's actually new: the GIP-DIP co-evolution loop, where a report generator filters detector pseudo-labels via class predictions, and the detector's boxes supervise the generator via abnormality tokens and location embeddings, plus SA-NMS to combine teacher/student detections. The empirical study on MS-CXR and PD-CXR is new, and the paper is transparent about building on CEIRD. The ablations (Tables IX, X) show each component helps, and the framework plugs into three different report generators with consistent gains on language and AUC metrics. That's real evidence.\n\nSoft spots: the stress-test note is right. Section III.F says that when the student detector finds no box on a weakly labeled CXR, the whole image is treated as a background pseudo-box, and the generator is trained to predict 'no abnormality' on its class token via Eq. (7), regardless of whether the paired report actually describes findings. Then GIP's filter in Eq. (8) keeps only detector pseudo-labels whose categories appear in the generator's predicted set. So a generator trained to call missed-lesion images normal will actively suppress newly proposed correct boxes. This is not generic pseudo-label noise; it's an error injection that can reinforce shared blind spots. The semi-oracle comparison (Table III) bounds the GIP approximation but doesn't isolate this path. I'd want to see a variant where no-box images are either skipped or supervised by the report's own labels to test whether this negative supervision is harmful.\n\nSecond, the detection numbers are reported without variance or significance tests, and key hyperparameters (pseudo-label threshold, number of iterations, labeled-to-weak ratio) are selected on validation. That's not fatal, but it makes the SOTA claims thinner than they look. Seed-averaged results and a sensitivity analysis would help.\n\nThe paper deserves a serious referee. It's a coherent, well-ablated contribution with a plausible central claim, and the code is out. The reviewer should focus on the DIP negative-supervision path and demand the robustness experiments. I'd accept it for peer review but push for revision before publication.","headline":"A well-ablated CXR detection and report generation co-evolution framework with real but modest gains, undercut by a concrete negative-supervision feedback path that can erase hard positives and by missing robustness statistics.","tokens_in":27266,"tokens_out":3268,"would_cite":true,"duration_ms":33201,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A detector and a report generator trained in a co-evolutionary loop improve each other on chest X-rays, achieving state-of-the-art detection and report quality from mostly report-only data.","keywords":["chest X-ray","abnormality detection","radiology report generation","semi-supervised learning","co-evolutionary training","teacher-student distillation","pseudo-label refinement","vision-language learning"],"falsifier":"Measure per-category recall on small or heavily overlapping lesions across co-evolution iterations, for example cardiomegaly overlapped by consolidation as in the paper's Fig. 9(a)–9(b): if enabling the generator-guided filter lowers recall on such categories compared with a detector trained without GIP, the filtering is sacrificing true positives the generator also misses, and the mutual-promotion claim for those lesion types would be refuted.","tokens_in":26172,"feed_emoji":"🩻","tokens_out":5428,"duration_ms":44695,"temperature":0.7,"pith_summary":"The paper sets out to show that two clinical AI tasks on chest X-rays—localizing abnormalities and writing the radiology report—can be trained together so each task makes the other better. It proposes a co-evolutionary loop in which a detector and a report generator alternately train, with the generator filtering and enriching the detector's pseudo-labels on weakly labeled images and the detector feeding its detected boxes and categories into the generator. Using only about 1,000 box-annotated images alongside more than 100,000 report-only images from MIMIC-CXR, the paper reports state-of-the-art detection accuracy and improved report quality on two benchmarks. The practical stake is that large archives of unannotated X-rays with text reports could be exploited to train fine-grained localizers, reducing the burden of manual bounding-box annotation.","feed_headline":"Detector and report model co-evolve to read chest X-rays","feed_subtitle":"A two-task loop turns report-only X-ray archives into detection and report training that beats state-of-the-art baselines.","key_machinery":"The load-bearing mechanism is the co-evolutionary training loop built on teacher-student distillation. Three components carry the argument: SA-NMS (self-adaptive non-maximum suppression) runs NMS on the union of teacher pseudo-boxes and student predictions so confident student boxes replace stale teacher ones; GIP (generator-guided information propagation) concatenates generator features to detector features and keeps a detector pseudo-label only if its category appears in the generator's class-token prediction; DIP (detector-guided information propagation) prepends detected abnormality tokens and location embeddings to the generator's input and adds an auxiliary multi-label classification supervised by detector pseudo-labels. The alternation of GIP and DIP across iterations is what converts weakly labeled report-only data into detection supervision and box annotations into report supervision.","core_discovery":"On the paper's own terms, CoE-DG demonstrates that abnormality detection and report generation are mutually informative rather than independent: a detector trained with report-guided pseudo-label filtering reaches mAP@0.5 of 37.89 on MS-CXR and 20.19 on PD-CXR, exceeding the best prior semi-supervised detector; the same detector's outputs, fed as abnormality tokens and location embeddings, lift BLEU-4 and clinical AUC of three different report generators over their baselines. The improvement is attributed to a closed loop where each model's weaknesses are corrected by the other's strengths—GIP removes detector pseudo-labels whose categories the generator does not confirm, and DIP supplies the generator with localized evidence the whole-image encoder tends to miss.","pith_inferences":["The filtering rule in GIP (keep only detector pseudo-labels whose category the generator also predicts) is conservative: it likely removes true positives that both models miss, so extending it to a soft weighting scheme with uncertainty estimates could recover additional recall without admitting more noise.","The co-evolution principle should transfer to other imaging domains with paired free-text reports, such as mammography or CT, where the same ratio of scarce box annotations to abundant reports holds.","One testable consequence of the paper's logic is that detection performance should keep improving as more report-only data is added even when box-annotated data stays fixed; this could be verified by scaling the weakly labeled pool.","The failure cases on overlapping and tiny lesions suggest the mutual filtering can lock in shared blind spots; a variant that periodically re-injects fully supervised training on hard cases might prevent that lock-in."],"forward_implications":["If the central claim holds, institutions with large archives of CXR images and free-text reports can train fine-grained abnormality localizers without manually drawing boxes on all images.","The same detector's outputs can simultaneously improve report generation, so a single inference pipeline can output both bounding boxes and a full report at negligible extra compute.","The framework's generality claim is supported by experiments with three different report generator backbones (R2Gen, R2GenCMN, ORGan), each improved by DIP.","Because the authors show the biggest gains at 25% and 50% of the fully labeled data, the method is most valuable in low-annotation regimes.","The reported gains over Soft Teacher (up to 2.07% mAP) indicate the mechanism extracts signal from reports that pure image-based semi-supervised learning cannot."],"supporting_citations":[{"why":"Soft Teacher is the strongest semi-supervised detection baseline the paper compares against and claims to beat by up to 2.07% mAP.","marker":"[10]"},{"why":"Teacher-student distillation is the detection training paradigm the framework builds on and extends with GIP and SA-NMS.","marker":"[11]"},{"why":"R2Gen is the base report generation model used as the generator backbone and as a primary baseline for report quality.","marker":"[7]"},{"why":"MS-CXR provides the bounding box annotations for part of MIMIC-CXR, defining the fully labeled subset and evaluation benchmark.","marker":"[18]"},{"why":"PD-CXR supplies the second evaluation benchmark focused on pneumonia and pneumothorax detection.","marker":"[19]"},{"why":"MIMIC-CXR is the large source of weakly labeled CXRs with free-text reports that the weakly labeled training pool is drawn from.","marker":"[20]"},{"why":"The authors' preliminary detection-only framework is the predecessor that CoE-DG extends with report generation and feature enhancement.","marker":"[21]"},{"why":"RetinaNet is the detection architecture used as the detector backbone throughout the framework.","marker":"[23]"},{"why":"R2GenCMN is one of the alternate generator backbones that DIP improves, supporting the claim of framework generality.","marker":"[55]"},{"why":"CheXbert is the automatic labeler used to extract abnormality labels from generated reports for computing the clinical-efficacy AUC metric.","marker":"[62]"}],"fun_headline_variants":["Chest X-ray detection and reporting improve via co-evolution","Co-evolving detector and report generator boosts chest X-ray analysis","Weakly labeled X-rays train a dual-task loop for detection and reports","Mutual promotion: detector and report model improve each other on CXR","Detector and report generator co-evolve using weakly labeled chest X-rays"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The loop assumes the two models' pseudo-labels on unannotated images are jointly reliable, so when both models share a blind spot—tiny lesions, overlapping findings—each round of co-evolution can reinforce the same omission instead of correcting it.","fun_headline_variants_meta":{"raw":{"variants":["Chest X-ray detection and reporting improve via co-evolution","Co-evolving detector and report generator boosts chest X-ray analysis","Weakly labeled X-rays train a dual-task loop for detection and reports","Mutual promotion: detector and report model improve each other on CXR","Detector and report generator co-evolve using weakly labeled chest X-rays"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00062,"raw_usage":{"total_tokens":2879,"prompt_tokens":953,"completion_tokens":1926,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":569,"completion_tokens_details":{"reasoning_tokens":1834}},"tokens_in":569,"tokens_out":1926,"duration_ms":11612,"temperature":1.0,"reasoning_tokens":1834,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T12:58:37.778346+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure per-category recall on small or heavily overlapping lesions across co-evolution iterations, for example cardiomegaly overlapped by consolidation as in the paper's Fig. 9(a)–9(b): if enabling the generator-guided filter lowers recall on such categories compared with a detector trained without GIP, the filtering is sacrificing true positives the generator also misses, and the mutual-promotion claim for those lesion types would be refuted.","supporting_citations":[{"cited_title":"End-to-end semi-supervised object detection with soft teacher,","cited_arxiv_id":null,"evidence_quote":"Soft Teacher is the strongest semi-supervised detection baseline the paper compares against and claims to beat by up to 2.07% mAP."},{"cited_title":"Generating radiology reports via memory-driven transformer,","cited_arxiv_id":null,"evidence_quote":"R2Gen is the base report generation model used as the generator backbone and as a primary baseline for report quality."},{"cited_title":"Making the most of text semantics to improve biomedical vision–language processing,","cited_arxiv_id":null,"evidence_quote":"MS-CXR provides the bounding box annotations for part of MIMIC-CXR, defining the fully labeled subset and evaluation benchmark."},{"cited_title":"Weakly supervised one-stage vision and language disease detection using large scale pneumonia and pneumothorax studies,","cited_arxiv_id":null,"evidence_quote":"PD-CXR supplies the second evaluation benchmark focused on pneumonia and pneumothorax detection."},{"cited_title":"MIMIC-CXR-JPG-chest radiographs with structured labels,","cited_arxiv_id":null,"evidence_quote":"MIMIC-CXR is the large source of weakly labeled CXRs with free-text reports that the weakly labeled training pool is drawn from."},{"cited_title":"You’ve got two teachers: Co-evolutionary image and report distillation for semi-supervised anatomical abnormality detection in chest X-ray,","cited_arxiv_id":null,"evidence_quote":"The authors' preliminary detection-only framework is the predecessor that CoE-DG extends with report generation and feature enhancement."}],"review_version":1}