{"id":"f24cacb5-84a5-4072-855c-cbd56b1d6cff","arxiv_id":"1908.07362","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A residual CNN using RGB, HSV, and CIELAB channels reports 99.29% accuracy and AUROC 0.9996 for IDC classification on a 7,500-image subset of the Kaggle breast histopathology dataset.","lead":"This paper applies a deep residual neural network with seven color channels to classify breast tissue image patches as having invasive ductal carcinoma or not. On a small subset of a public benchmark, it reports 99.29% accuracy and an AUROC of 0.9996, but the evaluation has serious data-split concerns.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported 99.29% accuracy and 0.9996 AUROC are not trustworthy because evaluation splits are at patch level, not whole-slide or patient level, allowing patches from the same source WSI to appear in both training and test sets.","rationale":"I read the paper in good faith. The architecture and preprocessing are described specifically enough that the method is not inherently implausible, and the authors do report standard metrics rather than only loss curves. However, the manuscript's own dataset description creates a clear hierarchical structure: 277,524 patches from 162 whole-slide images, with a 7,500-patch subset used for experiments. The only split guarantee in Section IV-A is that no individual image overlaps between training, testing, and validation; there is no statement that whole-slide images or patients were separated. This is precisely the setting where patch-level train/test splitting is known to yield optimistic accuracy because patches from the same slide share staining and artifact distributions. Since the paper's entire contribution is the empirical performance claim, and since the independent sample size is the number of slides rather than the number of patches, this missing slide-level separation is a correctness risk, not a stylistic omission. The reader's weakest assumption identified the same issue, and I agree with that assessment. The concrete test of slide-disjoint partitioning would settle the concern; it is feasible from the public dataset's filenames and would directly show whether the near-perfect numbers survive a valid generalization evaluation. Because the concern undermines the central claim as reported, no change to the REJECT verdict is warranted.","tokens_in":7394,"tokens_out":3004,"duration_ms":35464,"concrete_test":"Re-run the same architecture on the same 7,500-patch subset, but partition the data at whole-slide level: use the slide identifiers encoded in the public Kaggle filenames to assign every patch from a given WSI to exactly one of train, validation, or test, with class-stratified folds. Train and evaluate under this split, reporting accuracy and AUROC across at least five folds. If the slide-disjoint accuracy is materially below 99.29% or the AUROC below 0.9996, the original evaluation is inflated and the central claim fails as stated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the proposed residual network achieves near-perfect IDC prediction. This claim rests entirely on the evaluation protocol. The dataset is hierarchical: 277,524 patches were cropped from only 162 whole-slide images, and the paper selects a 7,500-patch subset from this pool. Section IV-A states only that 'No images overlap with each other out of the training, testing and the validation set of images.' It does not say that patches from the same whole-slide image were kept in the same split. If patches from the same WSI appear in both training and test sets, the model can exploit slide-specific staining, tissue preparation artifacts, and other WSI-level confounders that are not present in unseen slides. Under those conditions, even a model that has memorized slide-level cues can report near-perfect accuracy. The effective number of independent samples is at most 162 WSIs, not 7,500 patches, so patch-level accuracy does not measure generalization to new patients or new slides. This concern is load-bearing because the Abstract and Section IV-C use the 99.29% accuracy and 0.9996 AUROC as the evidence for the 'one of a kind state-of-the-art-model' claim, and Table II compares those numbers directly with prior methods. Without slide-disjoint evaluation, the headline numbers cannot be interpreted as valid performance estimates.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a deep residual convolutional neural network for classifying invasive ductal carcinoma (IDC) in breast histopathology images. The authors take a 7,500-patch subset of the Kaggle Breast Histopathology Images dataset (originally 277,524 patches from 162 whole-slide images), preprocess the patches by resizing to 100×100, merging RGB with hue/saturation from HSV and l*/a* from CIELAB into a 7-channel input, applying Gaussian blur and CLAHE, and then train a network with four residual blocks and 4×4 kernels. The paper reports 99.29% accuracy and an AUROC of 0.9996 on a held-out test set, presents a confusion matrix and Grad-CAM visualizations, and claims state-of-the-art performance compared with earlier methods.","tokens_in":7609,"tokens_out":3703,"duration_ms":38583,"significance":"If the reported performance were obtained under a valid evaluation protocol, the method would represent a strong result for patch-level IDC classification and might be of practical interest for computer-aided diagnosis. The use of a multi-channel color representation, residual learning, and Grad-CAM visualization are sensible ingredients. However, the significance as presented is severely limited by the evaluation: the effective number of independent units is at most 162 whole-slide images, and the paper does not demonstrate that the train/test split is disjoint at the slide or patient level. Without such a split, the reported 99.29% accuracy and 0.9996 AUROC cannot be interpreted as estimates of generalization to new slides or patients, and the state-of-the-art claim is therefore not supported.","major_comments":[{"comment":"The evaluation protocol is the load-bearing issue. The dataset consists of 277,524 patches extracted from 162 whole-slide images, and the authors use a 7,500-patch subset. Section IV-A states only that \"No images overlap with each other out of the training, testing and the validation set of images,\" which is compatible with random patch-level splitting. If patches from the same whole-slide image appear in both training and test sets, the model can exploit slide-specific staining and preparation artifacts, inflating the reported 99.29% accuracy and 0.9996 AUROC. The authors must either perform a slide-level or patient-level split, or otherwise explicitly demonstrate that no two patches from the same source WSI are distributed across different splits, before the headline numbers can be accepted as valid performance estimates.","section":"Section IV-A"},{"comment":"All performance metrics are reported for a single random split, with no repeated runs, cross-validation, or confidence intervals. Given that the independent units are at most 162 whole-slide images, the point estimates in Table I (e.g., 0.9929 accuracy, 0.9996 AUROC) are not accompanied by any measure of variability. The authors should report mean and standard deviation over multiple slide-disjoint splits or use slide-level cross-validation to establish that the results are stable.","section":"Section IV-A and Table I"},{"comment":"The comparative analysis in Table II is not valid as presented. The cited prior works use different datasets, different patch-sampling strategies, different class balances, and different evaluation protocols. The claim that the proposed model \"outperforms\" these methods and is \"an one of a kind state-of-the-art-model\" cannot be sustained without a common benchmark and identical evaluation conditions. At minimum, the authors should clearly state the incomparability and restrict their claims to their own dataset and protocol.","section":"Table II and Section IV-C"},{"comment":"The process for selecting the 7,500-image subset is not described: it is not stated whether the subset was chosen randomly, in a stratified manner, or consecutively from particular whole-slide images. Additionally, the hyperparameter search (kernel dimension and number of residual blocks) is reported only as a final choice, and the authors state that models with 1–3 and 5 residual blocks \"produced poor results\" without reporting the actual values. This makes it impossible to assess overfitting or selection bias. The authors should describe the subset selection and report the full hyperparameter sweep, ideally on a validation set that is separate from the test set.","section":"Section III-A and Section IV"}],"minor_comments":[{"comment":"The phrase \"an one of a kind state-of-the-art-model\" should be corrected to \"a one-of-a-kind state-of-the-art model.\"","section":"Abstract and Section IV-C"},{"comment":"There is a typo: \"neagtive\" should be \"negative.\"","section":"Section III-A"},{"comment":"\"Kaming He\" should be \"Kaiming He\" in the reference to the residual network paper.","section":"Section III-C"},{"comment":"The phrase \"over the top accuracy\" is informal; please use a neutral formulation such as \"very high accuracy\" or simply report the numerical value.","section":"Section IV-B"},{"comment":"Figure 8 is referenced before Figure 6 in the text; please renumber or reorder the figures so that they are cited in numerical order.","section":"Section IV-C"},{"comment":"The architecture description does not specify the number of convolutional filters per layer, the stride, or the input channel dimension after the 7-channel merge. These details are necessary for reproducibility.","section":"Section III-C"},{"comment":"Reference [1] is cited in the abstract as the American Cancer Society, but the actual reference is to breastcancer.org; please align the citation with the intended source.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The patch-level leakage concern is real and central. If the authors cannot provide slide-disjoint evaluation results, the manuscript should not be accepted. The paper is also quite thin on experimental detail and the comparative table is not meaningful as presented. However, because the core method could, in principle, be re-evaluated with a proper split, I do not recommend outright rejection at this stage; the revision would need to be substantial, including new experiments and a rewritten results section."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing to know: this is a standard residual CNN applied to a well-known Kaggle IDC patch dataset, with a small twist of feeding a 7-channel input (RGB + HSV hue/saturation + CIELAB L*/a*). The reported 99.29% accuracy and 0.9996 AUROC are not credible as estimates of generalization, because the evaluation split is at patch level, not patient or whole-slide level, and the dataset has only 162 source WSIs. The paper explicitly says only that no images overlap between train/test/validation; it never says patches from the same WSI were kept together. The stress-test note is correct, and it's load-bearing.\n\nWhat's actually new: the 7-channel fusion is the only real novelty. No ablation isolates it. Everything else—residual blocks, ReLU/ELU, CLAHE, Gaussian blur, Grad-CAM—is off-the-shelf. The paper does cite the relevant prior work (He et al., Bejnordi et al., Reza & Ma), so the citation pattern is honest even if the comparisons in Table II are apples-to-oranges. The writing is clear enough, and the architecture description is reproducible in principle. The authors also disclose their subset selection (7,500 of 277,524 patches) and their hyperparameter tuning range, which is more than many similar preprints do.\n\nSoft spots, in proportion: the single-split evaluation with no error bars or repeated runs is a serious but fixable flaw. The absence of any patient-level analysis or code/data release makes the headline numbers unverifiable. The state-of-the-art comparison mixes patch-level and WSI-level tasks; the Bejnordi stacked-CNN result is a three-class WSI accuracy, not directly comparable. The \"one of a kind state-of-the-art-model\" phrase is hype, but that's minor. The core problem remains the split: with 162 slides, the effective sample size is at most 162, and any patch-level accuracy can be inflated by slide-specific staining artifacts. That alone justifies rejection as it stands.\n\nWho this is for: someone doing a quick literature sweep on IDC patch classification might skim it for the 7-channel idea, but the result should not be cited as evidence for a clinically useful system. It deserves a serious referee only if revised with a slide-level split, ablations, and confidence intervals. The current version would not survive peer review. My recommendation: desk reject now, but tell the authors what a revised version would need. It's not a fundamentally broken idea—it's an inadequately evaluated one.","headline":"A routine patch-level CNN paper with an untrustworthy evaluation protocol; the 99.29% accuracy is likely inflated by slide-level leakage and cannot be taken at face value.","tokens_in":8194,"tokens_out":653,"would_cite":false,"duration_ms":8543,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Seven-channel residual network learns IDC patches at 99.29% accuracy","keywords":["invasive ductal carcinoma","histopathology image classification","deep residual network","seven-channel color space","CIELAB","HSV","Grad-CAM","breast cancer detection"],"falsifier":"A patient-level or slide-level split—placing all patches from each of the 162 source slides entirely in train, validation, or test—would settle it: if accuracy or AUROC falls materially below 99.29% and 0.9996, the original numbers reflected patch-level leakage rather than true IDC detection. An additional RGB-only run would isolate whether the seven-channel input is responsible for the gain.","tokens_in":7131,"feed_emoji":"🔬","tokens_out":6674,"duration_ms":62460,"temperature":0.7,"pith_summary":"This paper claims that a deep residual convolutional network can classify invasive ductal carcinoma (IDC) in breast histopathology patches almost perfectly. On a 7,500-image subset of the Breast Histopathology Images benchmark, the authors convert each $50 \\times 50$ RGB patch into a seven-channel matrix—adding hue and saturation from HSV and $l^*$ and $a^*$ from CIELAB—then feed it through four residual blocks with $4 \\times 4$ kernels. The reported result is 99.29% test accuracy with an AUROC of 0.9996, and the authors argue this beats previously published CNN and feature-based detectors. If the claim holds, automated IDC screening on whole-slide images becomes substantially more reliable, and the seven-channel color preprocessing is a reusable recipe for stain-heavy microscopy.","feed_headline":"Seven-channel residual network learns IDC patches at 99.29%","feed_subtitle":"Adding HSV and CIELAB channels to RGB pushes AUROC to 0.9996 on a breast histopathology subset.","key_machinery":"The load-bearing mechanism is the seven-channel image matrix: RGB plus the hue and saturation channels of HSV plus the $l^*$ and $a^*$ channels of CIELAB, all denoised with a Gaussian blur and contrast-enhanced with CLAHE on the RGB channels only. The paper argues these extra color channels let the network see past stain variation. The network itself is a residual CNN with four residual blocks, each block stacking three convolutional layers and an unactivated shortcut connection whose outputs are added before ReLU; a global average pooling layer feeds a fully connected classifier.","core_discovery":"The central claim is that a 4-block deep residual network trained on a seven-channel image representation, rather than plain RGB, distinguishes IDC-positive from IDC-negative histopathology patches at essentially ceiling performance. The authors report per-class accuracy of 0.9929, precision and recall around 0.99 for both classes, and AUROC 0.9996, and they use Grad-CAM heatmaps to argue the network focuses on the affected regions. They frame this as a state-of-the-art result relative to the earlier CNN and machine-learning methods listed in their comparison table.","pith_inferences":["A patient-level or whole-slide-level split would be the real test of the 99.29% figure; because patches are drawn from only 162 source slides, random patch splitting likely places highly correlated neighbours in train and test, inflating the score.","An RGB-only ablation would quantify how much the four extra channels contribute; without it, the seven-channel claim is untested.","Applying the exact pipeline to the full 277,524-patch set or to an outside hospital's slides would show whether the near-perfect scores survive scale and stain shift."],"forward_implications":["The 99.29% accuracy and 0.9996 AUROC, if taken at face value, mean the model misclassifies only about one in 140 test patches, far below the 81–89% accuracies of the compared methods.","The seven-channel preprocessing is, on the paper's account, what lets the network handle stain variation and low-resolution $50 \\times 50$ patches.","Grad-CAM visualizations, as the authors use them, provide patch-level localization of the regions driving the IDC decision, which is useful for pathologist review.","The model's ceiling-level performance on this subset implies the remaining research question is generalization to larger and more varied histopathology cohorts, which the authors identify as future scope."],"supporting_citations":[{"why":"Supplies the Breast Histopathology Images benchmark from which the 7,500-patch training/test subset is drawn.","marker":"[2]"},{"why":"Introduces deep residual learning with shortcut connections that the four-block architecture is built on.","marker":"[10]"},{"why":"Earlier CNN on H&E breast specimens that achieved AUC 0.92, giving context for the claimed AUROC of 0.9996.","marker":"[17]"},{"why":"CNN with sampling methods for imbalanced histopathology data; comparison baseline at 85.48% in Table II.","marker":"[18]"},{"why":"Random-forest based IDC detector; comparison baseline at 88.7% in Table II.","marker":"[19]"},{"why":"Early CNN for IDC detection in whole-slide images; comparison baseline at 84.23% in Table II.","marker":"[20]"},{"why":"Stacked CNN for whole-slide breast carcinoma classification; comparison baseline at 81.3% in Table II and AUC 0.962 context.","marker":"[21]"}],"fun_headline_variants":["Deep residual net detects IDC at 99.29% accuracy","Seven-channel residual CNN achieves 99.29% on IDC","0.9996 AUROC for IDC detection with deep residual net","99.29% IDC classification via seven-channel deep learning","Residual network with HSV and CIELAB hits 99.29% IDC"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes randomly splitting patches into train and test does not leak information from the same whole-slide image into both sets; if that assumption fails, the reported near-perfect scores overstate how well the model would do on new patients.","fun_headline_variants_meta":{"raw":{"variants":["Deep residual net detects IDC at 99.29% accuracy","Seven-channel residual CNN achieves 99.29% on IDC","0.9996 AUROC for IDC detection with deep residual net","99.29% IDC classification via seven-channel deep learning","Residual network with HSV and CIELAB hits 99.29% IDC"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000658,"raw_usage":{"total_tokens":2975,"prompt_tokens":876,"completion_tokens":2099,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":492,"completion_tokens_details":{"reasoning_tokens":2004}},"tokens_in":492,"tokens_out":2099,"duration_ms":14099,"temperature":1.0,"reasoning_tokens":2004,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:20:00.785471+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A patient-level or slide-level split—placing all patches from each of the 162 source slides entirely in train, validation, or test—would settle it: if accuracy or AUROC falls materially below 99.29% and 0.9996, the original numbers reflected patch-level leakage rather than true IDC detection. An additional RGB-only run would isolate whether the seven-channel input is responsible for the gain.","supporting_citations":[{"cited_title":"Invasive ductal carcinoma dataset,","cited_arxiv_id":null,"evidence_quote":"Supplies the Breast Histopathology Images benchmark from which the 7,500-patch training/test subset is drawn."},{"cited_title":"Deep learning-based assessment of tumor-associated stroma for diagnosing breast cancer in histopathology images,","cited_arxiv_id":null,"evidence_quote":"Earlier CNN on H&E breast specimens that achieved AUC 0.92, giving context for the claimed AUROC of 0.9996."},{"cited_title":"Imbalanced histopathological breast cancer image classification with convolutional neural network,","cited_arxiv_id":null,"evidence_quote":"CNN with sampling methods for imbalanced histopathology data; comparison baseline at 85.48% in Table II."},{"cited_title":"Invasive ductal breast carcinoma detector that is robust to image magnification in whole digital slides,","cited_arxiv_id":null,"evidence_quote":"Random-forest based IDC detector; comparison baseline at 88.7% in Table II."},{"cited_title":"Automatic detection of invasive ductal carcinoma in whole slide images with convolutional neural networks,","cited_arxiv_id":null,"evidence_quote":"Early CNN for IDC detection in whole-slide images; comparison baseline at 84.23% in Table II."},{"cited_title":"Context- aware stacked convolutional neural networks for classification of breast carcinomas in whole-slide histopathology images,","cited_arxiv_id":null,"evidence_quote":"Stacked CNN for whole-slide breast carcinoma classification; comparison baseline at 81.3% in Table II and AUC 0.962 context."}],"review_version":1}