{"id":"614ae612-5b64-473d-80c0-baec7bb8fe78","arxiv_id":"1908.06943","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Pixel-wise explanation heatmaps can reveal and help remove hidden dataset biases in deep learning models for tumor tissue classification.","lead":"This paper uses pixel-level explanation heatmaps to show what deep learning models rely on when classifying tumor tissue in stained pathology images. It demonstrates that these heatmaps can expose hidden dataset biases, such as models focusing on image artifacts instead of cancer cells, and support cell-level model evaluation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim assumes LRP relevance faithfully reflects the model's decision evidence, but this faithfulness is never validated on the pathology models; both the cell-level ROC and the bias-detection experiments inherit this unverified premise.","rationale":"The paper provides a well-designed set of controlled experiments, and the class-correlated bias demonstration is visually compelling. However, the central claim is that explanation heatmaps serve as a reliable diagnostic instrument for what the model actually learned. That requires LRP to be faithful to the model's decision process. The authors choose a specific LRP rule, α=1, β=0 in Eq. (2), but never validate its fidelity for GoogLeNet on H&E images. The cell-level ROC compares LRP relevance with pathologist cell annotations, yet high overlap with human expectation is not a faithfulness check: it could reflect LRP's inductive biases or the human prior rather than the model's computation. The bias-detection experiments, especially the class-correlated artefact, offer a natural intervention test that is not performed: remove the artefact and see whether predictions change. Without such a test, the heatmap-based diagnosis remains plausible but not verified. This is exactly the weakness the reader identified, and the CONDITIONAL verdict is appropriate. I see no reason to change the verdict; the missing faithfulness check is addressable and does not by itself invalidate the proof-of-concept.","tokens_in":14130,"tokens_out":6695,"duration_ms":70354,"concrete_test":"For the class-correlated bias model, take 100 held-out cancer patches and overwrite the 5x5 px top-left artefact with the local background colour or with a random H&E-like patch, then record the softmax probability for class cancer. If the median probability does not drop substantially below the cancer/non-cancer decision threshold, the artefact was not the decision evidence and the LRP hotspot is not causal; if it does drop, LRP's attribution is corroborated. As a control, apply the same occlusion to 5x5 px regions in other corners or at random positions.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's diagnostic claim rests on the assumption that LRP relevance scores track what the model actually uses to decide. Section 'Explaining classifier decisions' defines the LRP rules in Eqs. (1) and (2), but no independent verification of faithfulness is provided for the GoogLeNet pathology models. Two quantitative uses of heatmaps inherit this assumption: (i) the cell-level ROC in 'Quantitative evaluation of explanation heatmaps' treats positive relevance as evidence that the classifier used a particular cancer cell, and (ii) the bias experiments in 'Uncovering biases' treat a heatmap hotspot as the feature the model relies on. The class-correlated bias experiment, where a 5x5 px artefact is inserted into all cancer patches, is the keystone demonstration: the model reaches 100% accuracy and LRP highlights the artefact. However, showing that LRP highlights a label-correlated pixel is not the same as showing the model causally uses that pixel. Without an intervention, such as removing or recolouring the artefact and observing the prediction change, or a second attribution method as a cross-check, the result is also consistent with LRP producing plausible but non-faithful attributions. The paper's Limitations section acknowledges that only relevance on cells, not surrounding tissue, is measured, but it never tests the faithfulness premise. Thus the central claim that heatmaps are a reliable diagnostic instrument is conditional on an unvalidated assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes using pixel-wise explanation heatmaps generated by Layer-wise Relevance Propagation (LRP) as a diagnostic instrument for deep learning models in digital histopathology. The authors fine-tune GoogLeNet classifiers for tumor versus non-tumor discrimination on H&E-stained TCGA patches from three entities (BRCA, SKCM, LUAD). They evaluate the heatmaps quantitatively at the cell level using ROC curves against independent pathologist annotations of cells, and they use heatmaps to investigate three types of dataset biases: a dataset-wide labeling bias (label determined by the patch center), a class-correlated bias (a 5x5 px artificial artefact inserted into all cancer patches), and a sampling bias (necrotic tissue excluded from training). They report that heatmaps reveal these biases, that the class-correlated bias is detectable on a single sample, and that counteracting the center bias with translation augmentation improves AUC by about 5%. The central claim is that pixel-wise heatmaps allow detecting and removing hidden biases and enable cell-level quantitative evaluation, offering advantages over patch-level metrics.","tokens_in":14397,"tokens_out":3359,"duration_ms":35678,"significance":"If the central claim holds, the paper demonstrates a practically useful role for explanation heatmaps in the development and deployment of pathology classifiers: cell-level evaluation without expensive detection models, and bias detection on single samples without large annotated test sets. The controlled class-correlated bias experiment (a known 5x5 px corruption) is clean and convincing as a proof of concept, and the cell-level ROC evaluation against board-certified pathologist annotations is a strong point; the authors also provide public code and data links and compare with Grad-CAM. However, the entire diagnostic claim rests on the assumption that LRP relevance scores faithfully reflect the features the model actually uses. This faithfulness is not independently validated for these models, and several supporting experiments are informal (no error bars, small numbers of tiles, a single acceptance radius). These gaps affect the strength of the paper's main conclusions and require further evidence before the claims can be accepted as established.","major_comments":[{"comment":"The load-bearing assumption that LRP relevance faithfully reflects the model's decision evidence is unvalidated for these GoogLeNet pathology models. Equations (1) and (2) define the LRP rules, but no independent verification of faithfulness is provided. Consequently, both the cell-level ROC in 'Quantitative evaluation of explanation heatmaps' and the bias-detection experiments in 'Uncovering biases' inherit an unverified premise: positive relevance is interpreted as 'the feature the model relies on.' The controlled corruption experiment demonstrates that LRP highlights a label-correlated 5x5 px artefact, but this is a correlation, not a causal demonstration: it could in principle occur even if the model primarily used other features. To secure the central claim, the authors should add an intervention test (e.g., removing or recolouring the artefact and measuring the prediction change) or a second attribution method as a cross-check, on at least the class-correlated and sampling-bias experiments.","section":"Explaining classifier decisions; Quantitative evaluation of explanation heatmaps; Uncovering biases"},{"comment":"The quantitative support for the bias-removal claims is thin. The reported 5% AUC improvement after counteracting the center bias is described as a 'trend' and is presented without confidence intervals or significance testing; the underlying comparison uses a single test set of seven tiles. The sampling-bias evaluation in 'Sampling bias' is based on only five tiles and 'half of the considered regions' showing positive relevance, with no error bars or statistical assessment. These are small-sample, single-instance results, and the paper generalizes from them to the statement that heatmaps help 'remove the effects of common hidden biases, which improves generalization within and across datasets.' I ask the authors to either provide error bars and significance tests or soften the general claim to what the current evidence strictly supports.","section":"Uncovering biases (Dataset bias; Sampling bias); Supplemental Fig. 7"},{"comment":"The claim that heatmaps reveal biases 'often without the necessity of labels' and 'on a single sample' is only demonstrated for biases that were already known to the experimenters: the center bias is known by construction, the class-correlated artefact is an artificial insertion, and the sampling bias is studied by comparing a model trained without necrosis against one trained with it. The paper does not provide a protocol for discovering an unknown bias in a blind setting, nor a quantitative criterion for what pattern in a heatmap constitutes evidence of bias. This limits the practical scope of the central claim. I suggest the authors either add a blind or semi-blind validation (e.g., presenting heatmaps without knowledge of the bias to an expert) or explicitly restrict the claim to 'known by construction' biases in the conclusion.","section":"Uncovering biases and Table 2"}],"minor_comments":[{"comment":"In the Conclusion, the sentence 'we can, both quantitatively and quantitatively, compare features' should read 'both qualitatively and quantitatively.'","section":"Conclusion"},{"comment":"The choice of a single acceptance radius of ~50 px around each point annotation is not tested for sensitivity; a short analysis of how the cell-level ROC changes with the radius would strengthen the quantitative evaluation.","section":"Quantitative evaluation of explanation heatmaps"},{"comment":"The dataset bias experiment uses a separate 2116-tile breast-cancer dataset, but this is not clearly distinguished from the TCGA-BRCA dataset described in Table 1; please clarify the relationship and avoid potential confusion.","section":"Uncovering biases (Dataset bias)"},{"comment":"The sentence 'For the training and test set patient cases were split 80/20, while keeping the ratio of healthy and diseased cases constant' would be clearer as 'keeping the ratio of healthy to diseased patients constant' or similar.","section":"Method (Convolutional neural network training)"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the journal and presents a potentially useful application of explainable AI in pathology. The main concern is the unvalidated faithfulness of LRP, which underpins the central claim. If the authors can add an intervention-based validation or a second attribution method, the work would be substantially strengthened. The informal quantitative claims (5% AUC trend, small sampling-bias evaluation) should also be tightened. I see no reason to suspect circularity or fabrication; the controlled bias experiment and the independent cell annotations are genuine strengths."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about arXiv:1908.06943. First, it is a genuinely useful proof-of-concept: LRP heatmaps at pixel resolution can expose latent biases in histopathology classifiers and shift evaluation from patch level to cell level. Second, its central premise—that LRP relevance tracks what the model actually uses—is never directly tested, and the paper would be stronger with an intervention or a second attribution method. That gap is real but not disqualifying.\n\nWhat is new: the authors apply pixel-wise LRP to three TCGA tumor entities and show three things. They reproduce the known \"Clever Hans\" effect under controlled conditions, inserting a 5x5 px artefact into every cancer patch; the model hits 100% accuracy and the heatmap points straight at the artefact. They build ROC curves over pathologist-annotated cells and show the heatmaps localize cancer cells well. They also show sampling strategies (class-balanced vs. oversampled) leave a visible trace in the heatmaps. Prior work used Grad-CAM or probability maps; using LRP to detect small biases and evaluate at cell level is a clear step forward.\n\nThe strongest evidence is the controlled class-correlated bias experiment. It is clean and convincing. The cell-level ROC evaluation is also a real contribution, especially the observation that tumor-infiltrating lymphocytes inflate false positives in SKCM—something patch-level metrics hide. The paper is honest about its limitations, including only scoring relevance on cells rather than surrounding tissue.\n\nThe soft spots are worth naming. The faithfulness assumption is the main one. Showing that LRP highlights a label-correlated artefact is not the same as showing the model relies on it. A simple ablation—removing or recoloring the artefact and watching predictions change—would close the gap, and would also let the authors claim causation rather than correlation. The dataset-bias experiment reports a 5% AUC improvement with no error bars or significance test; the cell-level analysis uses a fixed radius around point annotations. These are minor-to-moderate issues and none of them undercut the proof-of-concept.\n\nThis paper deserves a serious referee. It is aimed at researchers in explainable AI and digital pathology who want practical tools for debugging classifiers. I would send it out with a request for an intervention-based faithfulness check and proper uncertainty quantification. With those additions it could become a standard citation for LRP-based bias detection in medical imaging.","headline":"A solid proof-of-concept for pixel-level LRP heatmaps in histopathology bias detection and cell-level evaluation; the faithfulness assumption needs an ablation but the core contribution holds.","tokens_in":14939,"tokens_out":2417,"would_cite":true,"duration_ms":23118,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Pixel-wise heatmaps expose hidden biases in pathology AI","keywords":["explainable AI","layer-wise relevance propagation","digital pathology","histopathology","hidden dataset bias","cell-level evaluation","tumour classification","heatmaps"],"falsifier":"Take a fresh model trained on the same tumour-discrimination task, plant a known hidden marker such as a small colour square in every cancer patch, and check whether LRP on a single held-out patch localises the marker before any biological structure is highlighted. A quantitative version: compare LRP heatmaps with a perturbation test in which the highest-relevance pixels are occluded and the prediction drop is measured; if predictions do not degrade when top-relevance regions are removed, the relevance scores are not faithfully ranking decision-contributing pixels.","tokens_in":13951,"feed_emoji":"🔬","tokens_out":5905,"duration_ms":58122,"temperature":0.7,"pith_summary":"This paper argues that pixel-level explanation heatmaps—maps of which image regions drove a neural network's tumour-versus-normal decision—should be a standard diagnostic tool in deep-learning pathology. The authors show that common hidden dataset biases that patch-level accuracy misses are visible in single heatmaps and can be corrected. The same heatmaps let a model be evaluated at the level of individual cells, using ROC curves that compare heatmap relevance against a pathologist's cell annotations. If this is right, explanation heatmaps are not just for post-hoc understanding; they are practical instruments for both developing and deploying medical image classifiers.","feed_headline":"Heatmaps expose hidden bias in pathology AI","feed_subtitle":"Pixel-wise explanations catch what patch-level accuracy misses and let models be judged cell by cell.","key_machinery":"The load-bearing instrument is Layer-wise Relevance Propagation (LRP), which distributes the network's output score backward through the network so that every input pixel receives a relevance value indicating how strongly it speaks for or against the predicted class. The specific rules are the epsilon rule for the classifier layer, $$$R_i^{{(l)}}$ = \\sum_j \\frac{z_{ij}}{\\sum_{i'} z_{i'j} + \\varepsilon \\cdot \\operatorname{sign}(\\sum_{i'} z_{i'j})} $R_j^{{(l+1)}}$,$$ and the $\\alpha$-$\\beta$ rule for convolutional layers, $$$R_i^{{(l)}}$ = \\sum_j \\left(\\$\\alpha$ \\frac{z^+_{ij}}{\\sum_{i'} z^+_{i'j}} + \\$\\beta$ \\frac{z^-_{ij}}{\\sum_{i'} z^-_{i'j}}\\right) $R_j^{{(l+1)}}$,$$ with $\\alpha=1,\\beta=0$. These pixel-level maps carry the argument because they show where the model is looking; in the bias experiments the heatmaps, not the accuracy scores, are the evidence that the model exploited an artefact, the patch centre, or the absence of necrosis. The same maps feed the cell-level ROC evaluation, where each annotated cell receives the mean relevance in a circular neighbourhood.","core_discovery":"The central claim is that Layer-wise Relevance Propagation produces high-resolution heatmaps that make the classifier's decision process visible at cell resolution, and that this visibility resolves two practical problems in digital pathology. First, heatmaps reveal biases invisible to patch-level metrics: a model trained on patches labelled by the centre cell concentrates relevance at the patch centre, a model trained with a tiny 5x5-pixel corner artefact on every cancer patch learns the artefact while still scoring 100% patch accuracy, and a model trained without necrotic tissue assigns positive cancer relevance to necrosis at test time. In each case the bias is detectable on one or a few affected images without a large labelled hold-out set. Second, the heatmaps allow quantitative evaluation at the cellular level: ROC curves built from relevance at annotated cell locations beat zero-, one-, and random-heatmap baselines in all three tumour entities, and this granularity exposes behaviours patch-level F1 misses, such as positive relevance assigned to tumour-infiltrating lymphocytes in the SKCM model.","pith_inferences":["Editorial inference: the spatial concentration of relevance, such as how much absolute relevance falls near a patch centre, could serve as a cheap, label-free screening metric for dataset bias before any test set is evaluated.","Editorial inference: because the cell-level ROC protocol only scores relevance on annotated cells, it could be extended to measure relevance on surrounding stroma and lymphocytes, capturing the context features the paper acknowledges are important but does not quantify.","Editorial inference: the same heatmap-first audit could be applied to whole-slide scanning pipelines, where staining variation and tissue folds are frequent, by averaging heatmaps per batch or per contributing site.","Editorial inference: the faithfulness of LRP itself remains an open question; one could build a controlled benchmark with synthetic known features and compare different attribution methods, which the paper does not do."],"forward_implications":["A model that learns the wrong thing can be caught from a single heatmap, without collecting a new annotated test set, which lowers the cost of bias detection in small medical datasets.","Cell-level ROC can reveal failure modes invisible in patch-level F1; in the SKCM example, small positive relevance on tumour-infiltrating lymphocytes flattens the curve, and excluding those two tiles raises AUC to 94%.","The balance of positive and negative relevance in a heatmap tracks the precision/recall trade-off induced by class sampling ratios, letting deployment teams choose a ratio for sensitivity- or specificity-oriented applications.","Heatmaps support iterative dataset enrichment: samples where the model performs poorly on unlabelled data can be flagged and sent to experts for annotation, making the training distribution more comprehensive.","The pixel-resolution advantage is not tied to LRP specifically; the paper argues that any fine-grained heatmap method should offer the same bias-detection and cell-evaluation benefits."],"supporting_citations":[{"why":"Supplies the Layer-wise Relevance Propagation method that generates the pixel-wise heatmaps on which all experiments rest.","marker":"[14]"},{"why":"Supplies the combination of LRP rules, namely the epsilon rule and the alpha-beta rule, used in the backward pass.","marker":"[42]"},{"why":"Provides the parameter choices for the LRP redistribution rules used in this work.","marker":"[43]"},{"why":"Provides the breast-cancer dataset and the earlier computational histopathology approach that the dataset-bias experiment builds on.","marker":"[19]"},{"why":"Gives the Grad-CAM whole-slide heatmap evaluation that this paper contrasts with pixel-wise heatmaps.","marker":"[20]"},{"why":"Demonstrated in computer vision how spurious correlations can be uncovered by explanation, serving as the precedent for class-correlated bias.","marker":"[9]"},{"why":"Shows how explanation methods unmask Clever Hans predictors, underpinning the argument that heatmaps reveal biases invisible to aggregate accuracy.","marker":"[10]"},{"why":"Supplies the software implementation used to compute the relevance maps.","marker":"[44]"}],"fun_headline_variants":["LRP heatmaps reveal hidden biases in pathology AI","Cell-level heatmaps expose pathology AI biases","Heatmaps catch pathology AI biases that accuracy misses","Pixel-wise relevance maps flag pathology AI blind spots"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole approach rests on the heatmaps faithfully reflecting what the network actually used for its decision; if LRP attribution does not track the true decision process, the bias detections, the cell-level ROC evaluation, and the interpretation of heatmap differences all lose their basis.","fun_headline_variants_meta":{"raw":{"variants":["LRP heatmaps reveal hidden biases in pathology AI","Cell-level heatmaps expose pathology AI biases","Heatmaps catch pathology AI biases that accuracy misses","Pixel-wise relevance maps flag pathology AI blind spots"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000769,"raw_usage":{"total_tokens":3428,"prompt_tokens":985,"completion_tokens":2443,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":601,"completion_tokens_details":{"reasoning_tokens":2384}},"tokens_in":601,"tokens_out":2443,"duration_ms":16411,"temperature":1.0,"reasoning_tokens":2384,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:08:11.816448+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a fresh model trained on the same tumour-discrimination task, plant a known hidden marker such as a small colour square in every cancer patch, and check whether LRP on a single held-out patch localises the marker before any biological structure is highlighted. A quantitative version: compare LRP heatmaps with a perturbation test in which the highest-relevance pixels are occluded and the prediction drop is measured; if predictions do not degrade when top-relevance regions are removed, the relevance scores are not faithfully ranking decision-contributing pixels.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Layer-wise Relevance Propagation method that generates the pixel-wise heatmaps on which all experiments rest."},{"cited_title":"& Samek, W","cited_arxiv_id":null,"evidence_quote":"Supplies the combination of LRP rules, namely the epsilon rule and the alpha-beta rule, used in the backward pass."},{"cited_title":"& Müller, K.-R","cited_arxiv_id":null,"evidence_quote":"Provides the parameter choices for the LRP redistribution rules used in this work."},{"cited_title":"Towards computational fluorescence microscopy: Machine learning-based integrated prediction of morphological and molecular tumor profiles","cited_arxiv_id":"1805.11178","evidence_quote":"Provides the breast-cancer dataset and the earlier computational histopathology approach that the dataset-bias experiment builds on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the Grad-CAM whole-slide heatmap evaluation that this paper contrasts with pixel-wise heatmaps."},{"cited_title":"& Samek, W","cited_arxiv_id":null,"evidence_quote":"Demonstrated in computer vision how spurious correlations can be uncovered by explanation, serving as the precedent for class-correlated bias."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows how explanation methods unmask Clever Hans predictors, underpinning the argument that heatmaps reveal biases invisible to aggregate accuracy."},{"cited_title":"biased\" (left) and the","cited_arxiv_id":null,"evidence_quote":"Supplies the software implementation used to compute the relevance maps."}],"review_version":1}