{"id":"20a83cbd-d804-455d-a35a-a2f231893c78","arxiv_id":"2505.06185","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Adding segmentation and image-reconstruction tasks to a Swin-Unet classifier slightly improved F1 on same-hospital test slices and AUC on cross-hospital test slices.","lead":"This paper tests whether training a Swin Transformer model on several tasks at once, including segmentation and image reconstruction, improves detection of a brain-bleed marker in CT scans. The authors report small gains in F1 on same-hospital test data and AUC on held-out hospitals over single-task classifiers.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Claimed superiority is metric-dependent: MTL-Swin-Unet wins F1 on same-patient but loses F1 on covariate shift; wins AUC on covariate shift but loses AUC on same-patient.","rationale":"The reader's weakest_assumption focused on the undocumented selection of the 179-image covariate-shift test set, which is a real concern for that specific benchmark. However, the more load-bearing issue is that the central claim of superiority is achieved only by choosing F1 in the same-patient setting and AUC in the covariate-shift setting. The paper's own Table 1 shows that under the alternative metric in each setting, the proposed method is not the best: Joint-SwinTransformer has higher AUC on Hospitals 1-4, and Joint-SwinTransformer-tiny has higher F1 on Hospitals 5-11. This means the headline result is not robust to metric choice, and the single-split, no-error-bar evaluation cannot rule out chance. The proposed condition is to require a pre-specified primary metric (or explicit justification for different metrics) and to report confidence intervals or multiple seeds. This does not change the reader's conditional verdict but sharpens the condition: without a fixed metric, the claim of simultaneous F1 and AUC superiority is unsupported. Agreement is partial because the reader did identify metric switching in the rationale, but the weakest_assumption was on sampling rather than on this more fundamental inconsistency.","tokens_in":8778,"tokens_out":6889,"duration_ms":62833,"concrete_test":"Re-rank Table 1 twice: once with F1 as the sole metric for both test sets, once with AUC as the sole metric for both test sets. If MTL-Swin-Unet (cls+seg+rec) fails to rank first in both settings for either fixed metric (it will fail F1 on Hospitals 5-11 and AUC on Hospitals 1-4), the central claim is metric-dependent. As a complementary check, request the per-image scores or confusion matrices and compute a DeLong test or bootstrap 95% CI for the AUC difference of 0.011-0.027 on the 179 covariate-shift samples; if the interval crosses zero, the reported superiority is not statistically significant.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is supported by switching the evaluation metric between the two settings. In Table 1, on Test Data (Hospitals 1-4), MTL-Swin-Unet (cls+seg+rec) has the best F1 (0.903) but not the best AUC: Joint-SwinTransformer reaches 0.974 versus 0.967. On Test Data (Hospitals 5-11), MTL-Swin-Unet (cls+seg+rec) has the best AUC (0.799) but not the best F1: Joint-SwinTransformer-tiny reaches 0.626 versus 0.618. Thus the phrase \"outperformed other classifiers\" is only true under the metric chosen ad hoc for each setting. If a single metric (F1 or AUC) were applied across both test sets, the proposed method is not first in both. This makes the claimed advantage of multitask learning (seg+rec) over the listed baselines non-robust: the \"improvement\" is within the noise of a single split and no error bars are given. The burden is on the authors to justify the metric choice or show the advantage holds under a consistently pre-specified metric.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes two Transformer-based methods for classifying hypodensity markers in brain CT: MTL-Swin-Unet, which extends Swin-Unet by adding classification and image-reconstruction heads trained jointly with the segmentation head via the weighted loss in Eq. (5), and Joint-SwinTransformer, which concatenates representations from a frozen Swin-Unet segmentation encoder with a trainable SwinTransformer encoder for classification. The experiments use a private dataset of 11,780 CT images from 11 hospitals. On test data from hospitals 1-4, MTL-Swin-Unet (cls+seg+rec) achieves the highest reported F1 (0.903) and accuracy (0.961); on test data from hospitals 5-11, it achieves the highest reported AUC (0.799). The authors interpret these results as evidence that multitask learning with segmentation and reconstruction improves classification and mitigates spurious correlation under covariate shift.","tokens_in":9023,"tokens_out":5994,"duration_ms":57171,"significance":"The paper addresses a practically important question: whether auxiliary segmentation and reconstruction objectives improve a Swin-based classifier for a scarce medical label, and whether such multitask training helps under cross-hospital distribution shift. The strengths include a reasonably detailed architecture description and a head-to-head empirical comparison against standard baselines (ResNet152, Swin Transformer, SwinTransformer-tiny, joint-CNN) on the same private dataset. The reported differences, however, are small and are evaluated on a single split without error bars or significance tests. If the evaluation is made robust and the advantage persists, the result would be a useful empirical contribution to multitask learning for medical imaging. The manuscript does not provide code or a reproducibility statement, and the data are private, so independent verification is not currently possible.","major_comments":[{"comment":"The abstract and conclusion claim that the proposed method outperformed other classifiers in F1 under no covariate shift and in AUC under covariate shift, but Table 1 shows that the advantage is metric-dependent: on Hospitals 1-4, MTL-Swin-Unet (cls+seg+rec) has the best F1 (0.903) but not the best AUC (0.967 vs Joint-SwinTransformer's 0.974); on Hospitals 5-11, it has the best AUC (0.799) but not the best F1 (0.618 vs Joint-SwinTransformer-tiny's 0.626). The claim therefore holds only when a different metric is selected as the headline in each setting. The authors should pre-specify one primary metric or report both metrics with a consistent decision rule, and discuss the trade-off explicitly.","section":"Result, Table 1"},{"comment":"The experimental design does not rule out patient-level leakage in the same-hospital test setting. The text states that for hospital 1, 179 images were allocated as test data while the remaining 9/10 of hospital 1 was used for training, and the table footnote says non-overlapping slices from the same patient are allowed for both training and test data. If slices from the same patient appear in both training and test, the reported F1 advantage (0.903 vs 0.895) may be inflated by memorization of patient-specific features. The authors should either use a patient-stratified split for the no-covariate-shift setting or explicitly report the number of patients and the overlap statistics.","section":"Settings of Two Datasets"},{"comment":"The covariate-shift test set of 179 images from hospitals 5-11 is described only as randomly selected with a positive ratio matching the facility-1 test data. The manuscript does not state the total number of available images or patients, the selection algorithm, or whether selection was stratified by hospital and patient. Because the central cross-hospital AUC claim rests on this sample, the authors must document the selection procedure and report the class distribution, hospital distribution, and patient counts. Without this, the sample cannot be judged as a fair covariate-shift benchmark.","section":"Settings of Two Datasets"},{"comment":"The claim that the model avoids spurious correlations (e.g., focusing on the skull) is supported only by two illustrative Grad-CAM examples. The paper provides no quantitative comparison of Grad-CAM maps against the segmentation masks, nor a comparison of heatmaps between the proposed method and a baseline classifier that reportedly focuses on the skull. The authors should add a quantitative faithfulness or overlap metric (e.g., Dice or IoU between the Grad-CAM region and the lesion mask) over the test set, or temper the spurious-correlation claim accordingly.","section":"Verification of Prediction Basis, Figure 4"},{"comment":"The multitask loss weights (lambda_cls = 0.3, lambda_seg = 0.4, lambda_rec = 0.4) are hand-set, and no sensitivity analysis or ablation over these weights is reported. Because the central comparison is between task combinations, the reader cannot determine whether the observed benefit of adding segmentation and reconstruction is robust to the weight choice. The authors should include a small sensitivity study over the lambda values or provide a principled rationale for the chosen values.","section":"Loss Function, Eq. (5)"}],"minor_comments":[{"comment":"There are typos and terminology inconsistencies: 'sprious-correlation' should be 'spurious correlation', and 'covariance shift' should be 'covariate shift' in the abstract, the Settings of Two Datasets section, and the Conclusion.","section":"Abstract and throughout"},{"comment":"The text refers to 'Table ??' and to 'Section .' as unresolved cross-references; these placeholders should be fixed before a version is submitted for review.","section":"Impact of Model Size and Method"},{"comment":"The table footnote describes joint-CNN as based on EfficientNet, while the main text calls it 'CNN based joint-Learning' and contains the placeholder 'avoid reference for anonymity'; this should be resolved and the baseline described consistently.","section":"Table 1 footnote and Settings of the Learner"},{"comment":"The dataset description says '8/10 of the 9,912 images from facilities 2 to 4' were used for training; please clarify whether the remaining 2/10 are validation or excluded, and how the stated total of 11,780 images is composed.","section":"Dataset"},{"comment":"The epoch counts are given as 600 for classification and 200 for segmentation, but it is not stated how many epochs are used for the joint multi-task models; please specify the training schedule and any early-stopping criterion for every method.","section":"Settings of the Learner"},{"comment":"The caption for panel (b), 'hematoma excluding hypodensity', would be clearer as 'hematoma without hypodensity', and the figure should state the CT window/level used for display.","section":"Figure 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript uses a private dataset and does not mention ethical approval or data-use agreements for the 11 institutions; the journal should verify the authors' right to publish aggregate results. The placeholder 'avoid reference for anonymity' suggests the text was prepared for a double-blind submission and was not cleaned up, which should be corrected. In addition, the lack of code or a reproducibility statement is a concern for a cs.LG submission whose only evidence is empirical."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague —\n\nThe paper is a small combination of known components: Swin-Unet extended with classification and reconstruction heads, plus a joint-learning variant that uses a frozen segmentation encoder. That is a legitimate extension, and the authors describe the architecture and losses cleanly. The actual empirical claim is narrower than the abstract suggests. In the same-patient test, MTL-Swin-Unet (cls+seg+rec) has the best F1 (0.903) but not the best AUC (Joint-SwinTransformer reaches 0.974). In the covariate-shift test, it has the best AUC (0.799) but not the best F1 (Joint-SwinTransformer-tiny reaches 0.626). The results text acknowledges this, but the abstract and conclusion say 'outperformed' without the metric qualification. That is an overstatement.\n\nWhat the paper does well: it is transparent about loss weights and training details, and it reports the negative result that reconstruction alone hurts classification, which is a useful observation. The architecture is a reasonable baseline for future work on multitask medical imaging.\n\nThe soft spots are real. The evaluation uses a single split with no error bars or significance tests, so the small numerical advantages (0.008 in F1, 0.011 in AUC) are likely within noise. The covariate-shift test of 179 images from hospitals 5–11 is underdocumented: no information on how many patients, how the selection was made, or whether the positive-ratio matching biases the result. No code or data is released, so nothing is independently checkable. One baseline (joint-CNN) appears as 'avoid reference for anonymity,' which has to be fixed for any publication. The spurious-correlation claim rests on a qualitative Grad-CAM figure, with no baseline comparison or quantitative measure. These are not fatal flaws in the idea, but they make the central performance claims unverified.\n\nWho this is for: readers working on multitask vision transformers for medical imaging, and anyone interested in how evaluation metric choice can flip rankings. It is not a significant advance, but it is not nonsense.\n\nMy recommendation: a serious editor should send this to peer review, not desk reject it, because the empirical question is legitimate and the method description is clear enough to engage with. The review should demand multiple splits or bootstrapping, a pre-specified metric, a real citation for the missing baseline, and either released artifacts or a clear data-availability statement. With those changes it could become a solid workshop-level contribution; without them, the claims should not be accepted as stated.","headline":"Honest small extension, but the headline 'outperformed' depends on switching F1 and AUC between settings, and the single-split evaluation is too thin to trust.","tokens_in":9539,"tokens_out":3281,"would_cite":false,"duration_ms":31970,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A Swin Transformer trained at once for classification, segmentation, and reconstruction recognizes brain hematoma markers better than a single-task Swin classifier.","keywords":["multitask learning","Swin Transformer","Swin-Unet","brain hematoma","hypodensity","semantic segmentation","image reconstruction","covariate shift"],"falsifier":"Re-run the comparison with patient-level splits instead of slice-level splits: train on all patients from hospitals 1-4 and test on a fresh random draw of slices from hospitals 5-11, with the positive ratio matched by a documented procedure and multiple draws to estimate variance; if MTL-Swin-Unet no longer beats Swin Transformer in AUC, the reported advantage is an artifact of the chosen subset.","tokens_in":8547,"feed_emoji":"🧠","tokens_out":6564,"duration_ms":55578,"temperature":0.7,"pith_summary":"The paper tries to establish that MTL-Swin-Unet, a Swin-Unet trained jointly to classify, segment, and reconstruct brain CT images, learns a representation that recognizes the hypodensity marker better than a plain Swin Transformer classifier. On test slices drawn from the same patients as the training data, the three-task model achieves the highest accuracy and F1 among the compared classifiers. On slices from hospitals the model never saw, it achieves the highest AUC. If this holds, auxiliary pixel-level objectives, segmentation especially, can counteract the spurious correlations that make such models look at the skull instead of the lesion.","feed_headline":"Multitask Swin-Unet lifts hematoma classification on new hospitals","feed_subtitle":"Three-task training beats single-task Swin Transformer on F1 and on cross-hospital AUC for brain CT hypodensity.","key_machinery":"The central object is MTL-Swin-Unet: a U-shaped Swin Transformer network with a shared encoder whose representations feed three heads, a classification head on global average pooled features, a segmentation decoder, and an image reconstruction decoder. The three task losses are combined as a weighted sum, $L = \\lambda_{\\mathrm{cls}} L_{\\mathrm{cls}} + \\lambda_{\\mathrm{seg}} L_{\\mathrm{seg}} + \\lambda_{\\mathrm{rec}} L_{\\mathrm{rec}}$, with the segmentation loss itself a weighted mix of cross-entropy and Dice loss. The shared encoder is initialized from ImageNet pretrained weights, and the decoder structure follows Swin-Unet with skip connections, letting dense pixel-level objectives regularize the classification representation.","core_discovery":"The central claim is that multitask learning with segmentation and image reconstruction improves Swin-based classification of hypodensity in brain CT images. The paper reports that MTL-Swin-Unet (cls + seg + rec) reaches an accuracy of 0.961, precision 0.852, recall 0.809, and F1 of 0.903 on the same-hospital test set, ahead of every classifier it is compared with, and an AUC of 0.799 on the held-out-hospital test set, also the highest. The comparison also shows that the segmentation task is the beneficial auxiliary objective, while reconstruction alone reduces AUC relative to the single-task Swin Transformer; only when combined with segmentation does reconstruction add a small gain on unseen hospitals.","pith_inferences":["Editorial inference: if segmentation is the main driver, this multitask recipe may transfer to other lesion types or imaging modalities wherever pixel-level annotations exist, even if only for a fraction of the data.","Editorial inference: the same-patient test split is at slice level, not patient level; a patient-level split would remove the strong within-patient correlation and could change the size of the reported F1 gap.","Editorial inference: the covariate-shift test subset was assembled by matching the positive ratio of the hospital-1 test set, but a more stable estimate of the AUC advantage would come from repeated random draws of such subsets with reported variance."],"forward_implications":["MTL-Swin-Unet can be deployed directly as a classifier even when segmentation masks are only available for a subset of training images, because the segmentation loss is averaged only over masked images.","Because the segmentation objective shifts the classifier's attention from skull and brain texture to the hematoma region, the resulting predictions may be more clinically trustworthy in settings where the training and test images come from different scanners or hospitals.","Adding reconstruction alone is not enough: the paper's comparison suggests that reconstruction should be paired with segmentation to help cross-hospital performance.","The method provides a practical middle ground between plain transfer learning and full domain adaptation for hospital-to-hospital generalization in medical imaging."],"supporting_citations":[{"why":"Supplies Swin-Unet, the U-shaped architecture that MTL-Swin-Unet extends with classification and reconstruction heads.","marker":"(Cao et al. 2022)"},{"why":"Supplies the Swin Transformer backbone and the main single-task classifier baseline that the proposed method must beat.","marker":"(Liu et al. 2021)"},{"why":"Frames the spurious-correlation problem that motivates adding segmentation and reconstruction objectives.","marker":"(Sagawa et al. 2020)"},{"why":"Provides Grad-CAM, the visualization method used to verify that the model focuses on hematomas rather than skull or brain texture.","marker":"(Selvaraju et al. 2016)"},{"why":"Defines the U-Net skip-connection decoder structure that Swin-Unet adapts and supplies the U-Net segmentation baseline.","marker":"(Ronneberger, Fischer, and Brox 2015)"},{"why":"Supplies ResNet152, the convolutional baseline for the classification comparison.","marker":"(He et al. 2015)"},{"why":"Establishes the clinical four-marker hemorrhage set that includes hypodensity, defining the classification target.","marker":"(Boulouis et al. 2016)"}],"fun_headline_variants":["Multitask Swin-Unet wins cross-hospital brain CT AUC","Segmentation plus reconstruction lifts Swin hematoma AUC","MTL-Swin-Unet beats Swin Transformer on held-out hospitals","Three-task Swin-Unet improves hematoma detection across sites","Multitask transformer boosts brain hypodensity classification"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline cross-hospital AUC result rests on the assumption that the 179 images selected from hospitals 5-11, matched to the positive ratio of the hospital-1 test set, are a fair and representative sample of the covariate shift; if this selection procedure biased the sample, the comparison would collapse.","fun_headline_variants_meta":{"raw":{"variants":["Multitask Swin-Unet wins cross-hospital brain CT AUC","Segmentation plus reconstruction lifts Swin hematoma AUC","MTL-Swin-Unet beats Swin Transformer on held-out hospitals","Three-task Swin-Unet improves hematoma detection across sites","Multitask transformer boosts brain hypodensity classification"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000196,"raw_usage":{"total_tokens":1284,"prompt_tokens":792,"completion_tokens":492,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":408,"completion_tokens_details":{"reasoning_tokens":407}},"tokens_in":408,"tokens_out":492,"duration_ms":5560,"temperature":1.0,"reasoning_tokens":407,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:47:09.100576+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the comparison with patient-level splits instead of slice-level splits: train on all patients from hospitals 1-4 and test on a fresh random draw of slices from hospitals 5-11, with the positive ratio matched by a documented procedure and multiple draws to estimate variance; if MTL-Swin-Unet no longer beats Swin Transformer in AUC, the reported advantage is an artifact of the chosen subset.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies Swin-Unet, the U-shaped architecture that MTL-Swin-Unet extends with classification and reconstruction heads."},{"cited_title":"B.; Charidimou, A.; Jessel, M","cited_arxiv_id":null,"evidence_quote":"Establishes the clinical four-marker hemorrhage set that includes hypodensity, defining the classification target."}],"review_version":1}