{"id":"646a86dd-0052-48b8-94f1-03be95ebefa5","arxiv_id":"2412.19208","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"The paper proposes Augmented Concept Activation Vector (ACAV), which injects a visual concept into input images and measures the resulting activation shift to quantify that concept's influence on a classifier's decision.","lead":"This paper introduces a way to measure how strongly a trained image classifier relies on a human-understandable visual concept, by adding that concept into test images and watching how the model's internal activations shift. The approach could help medical teams verify that AI diagnosis tools base decisions on clinically meaningful patterns instead of image artifacts.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The isolation claim is untested: no control augmentation is run, so the measured activation shift cannot be attributed to the concept rather than to low-level pixel statistics of the inserted patch.","rationale":"The reader's weakest assumption correctly identifies the most load-bearing gap: the method assumes that augmentation changes only the high-level concept, but no control rules out low-level pixel-statistics confounds. My stress-test confirms this is the decisive point: even if the architecture were consistent and Eq. 4 were used, the measured activation shift would still be uninterpretable without a counterfactual control. The paper's own Section 3 asserts the benefit of preserving context but never verifies it. Since the central isolation claim fails on this missing control, the reader's REJECT verdict stands. The proposed concrete test would settle whether the concern lands: if a texture-matched random patch produces the same deviations, the method does not isolate concepts. Additional issues (inconsistent architecture description, unimplemented entropy metric, missing error bars) reinforce the rejection but are secondary to this logical gap.","tokens_in":8788,"tokens_out":3255,"duration_ms":34664,"concrete_test":"On the same 50 healthy fundus images, generate three augmentations: (a) the real cotton-wool patch as in the paper, (b) a control patch matched for size, mean intensity, edge energy, and local contrast but with randomized phase (or a random texture patch), and (c) no augmentation. Compute the activation deviation metric from Eq. 3 (or the angular deviations) for all three conditions. If the control deviation in (b) is statistically indistinguishable from the real patch in (a), the method cannot isolate the concept; if the control deviation is near zero, the concern is addressed. Also repeat with the real patch placed at random locations to test the 'independent of surrounding data' claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that ACAV can 'isolate the impact of a given high-level concept and accurately measure it quantitatively.' For this to hold, the observed activation deviations (Eq. 3; Tables 1-4) must be caused by the concept's semantic content and not by generic properties of the inserted patch. Section 3 asserts that augmenting a concept pattern 'preserving the context of the sample and avoiding external factors,' but no control condition is ever run. There is no random-patch augmentation, no phase-scrambled or texture-matched control, no brightness/contrast-matched patch, and no augmentation with a known irrelevant concept. Without such a control, the deviations in Tables 1-4 could be produced by any local image edit; the shift toward the diabetes class could reflect the patch's low-level statistics (edges, intensity, contrast, location) rather than the radiomic pattern the model allegedly uses. The conclusion 'can successfully isolate the impact of these patterns independently of the surrounding data' is therefore not established by the evidence. This is a missing control experiment, not a disagreement with consensus.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Augmented Concept Activation Vector (ACAV), a post-hoc explainability method that measures the impact of high-level visual concepts on a trained neural network's decisions. The method augments input images with concept patterns (e.g., cotton-wool spots, fatty dots, bleeding, tumor patches) and measures shifts in the network's internal activation vectors using Equation (3), with additional cosine-angle metrics in Tables 3-4. Experiments on RFMiD fundus images and a brain MRI dataset claim to isolate the impact of radiomic patterns and tumor size on classification. The central claims are that ACAV can 'isolate the impact of a given high-level concept and accurately measure it quantitatively' and can handle imbalanced data where rare patterns matter.","tokens_in":8983,"tokens_out":5903,"duration_ms":58015,"significance":"If validated, ACAV would be a lightweight, single-model alternative to TCAV-like methods for concept attribution in medical imaging, avoiding the need for a separately trained concept model and working directly on raw images rather than extracted features. The paper also targets a practically important problem: quantifying the influence of rare radiomic patterns in imbalanced datasets. These are genuine strengths of the proposal. However, as presented, the experimental evidence does not support the isolation and quantitative-accuracy claims: there is no control augmentation condition, no error bars or significance tests, and the reported metrics are not consistently tied to the equations that define them. The core idea is plausible, but the current manuscript is not a validated method.","major_comments":[{"comment":"The central claim that ACAV 'can isolate the impact of a given high-level concept and accurately measure it quantitatively' (Abstract) is not supported by the experiments because no control augmentation is ever run. Augmenting an image with a cotton-wool or fatty-dot patch changes low-level statistics (edges, intensity, contrast, location) as well as the semantic concept; Tables 1-4 therefore measure the combined effect of all these changes. Without a random-patch condition, a phase-scrambled or texture-matched control, or an augmentation with a known irrelevant concept, the observed activation deviations cannot be attributed uniquely to the concept. This is a missing control experiment, not a disagreement with consensus.","section":"Section 3, Eq. (3), Tables 1-4"},{"comment":"The quantitative claims are made without any estimate of variability or statistical significance. Each experiment is based on 50 images, yet the tables report only point values (e.g., average absolute deviations of 0.03-0.10 in Table 1 and angle differences of 3-60 degrees in Table 3) with no confidence intervals, standard deviations, significance tests, or multiple-seed runs. Given the small magnitudes of some deviations, these numbers could easily be within sampling noise; the conclusion that the model is 'sensitive' or 'insensitive' to a pattern is therefore not established. I request error bars, hypothesis tests, and comparison against the control condition from the previous comment.","section":"Section 4, Tables 1-4"},{"comment":"The metrics reported in the results do not match the definitions in Section 3. Equation (3) defines the average absolute deviation of activation vectors, and the text defines a first metric as the ratio of data points assigned to a different class after augmentation; neither quantity appears in the tables. Instead, Tables 1-2 report 'average norm vector' and 'average absolute deviation' (with no formula), and Tables 3-4 report cosine angles in degrees. The entropy metric of Eq. (4) is never used in the experiments. Because the reported numbers cannot be traced to the defined metrics, the paper does not provide an operational definition of the claimed quantitative measurement.","section":"Section 3, Eq. (3) vs Section 4, Tables 1-4"},{"comment":"The model description is internally contradictory. The text states that 'the last layer includes five neurons having soft-max activation function' and later states that 'the last layer includes a single neuron with softmax activation function,' with a 0.6/0.4 confidence threshold. These are incompatible descriptions of the output layer, and the threshold choice affects which images are counted as correctly classified and hence which activation vectors enter the averages in Tables 1-4. The architecture must be specified precisely for the quantitative results to be reproducible.","section":"Section 4, Figure 5 and softmax threshold"}],"minor_comments":[{"comment":"There are multiple typos and malformed notation: 'prposed', 'date points', and the set definition 'AC : {aC|aC = a L C, a∈ TN , C isaconceptpattern}' are garbled; the augmentation operator 'di L C' is never defined.","section":"Section 3"},{"comment":"Equation (3) writes f(di) for both the original and the augmented input; the augmented input should be f(dia) or equivalent, as written it suggests the same input is fed twice.","section":"Eq. (3)"},{"comment":"The column labels 'Average Norm Vector' are ambiguous: they do not state whether these are Euclidean norms, cosine similarities, or something else, and the unit of 'Average Absolute Deviation' in Table 1 is not defined.","section":"Tables 1-2"},{"comment":"The train/validation/test split, class balance, and preprocessing (e.g., image resolution for fundus images) are not described for either dataset, so the experiments cannot be reproduced.","section":"Section 4"},{"comment":"The statement that the results were 'validated by a domain expert' gives no protocol; please describe what was shown to the expert and what the confirmation consisted of.","section":"Section 4, final paragraph"},{"comment":"Several references contain formatting artifacts (e.g., '[PdSMBR VCdS19]', '[A VL+14]', '[LR VL+12]'); please clean the citation keys.","section":"References"},{"comment":"The second aim, 'determine the prevalence of frequent patterns' in imbalanced datasets, is not addressed by any reported prevalence measure or by use of Eq. (4).","section":"Abstract and Introduction"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is not ready for publication in its current form. The missing control condition is a standard requirement for causal attribution; without it, the method's central selling point is untested. I would encourage the authors to run the control experiments, add statistical estimates, and align the reported metrics with the definitions in Section 3. The relation to TCAV should also be clarified: ACAV is presented as avoiding a second model, but a direct empirical comparison with TCAV would strengthen the contribution. If the journal requires complete evidence at submission, rejection may be warranted, but the identified issues are addressable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper proposes a cheap, intuitive way to test whether a trained classifier relies on a clinical pattern: paste the pattern into healthy images and watch the activation vector move. That is a reasonable idea, and the qualitative trend—more patterns, larger shifts; small tumors, smaller shifts—is exactly what you would hope to see. It is a small, incremental extension of TCAV and perturbation-based attribution, not a new capability. The writing cites TCAV but skips the perturbation-attribution literature, which is a gap but not a fatal one.\n\nWhat it does well: single network, no retraining, uses raw images, and targets rare concepts in imbalanced medical data. The fundus and brain MRI demos show the procedure is easy to apply. The domain expert check, while informal, is a sensible sanity step.\n\nThe soft spots are significant. The central claim—that ACAV can \"isolate the impact of a given high-level concept and accurately measure it quantitatively\"—is not supported by the evidence. There is no control augmentation: no random patches, no phase-scrambled or texture-matched inserts. So the measured activation shift could be driven by low-level pixel statistics (edges, brightness, contrast) rather than by the concept itself. That is the load-bearing missing experiment. There are also no error bars or significance tests, n=50 per condition, and the reported deviations are 0.03–0.10 with no confidence intervals. The entropy metric in Eq. 4 never appears again. The architecture description is inconsistent (five softmax neurons in one place, a single neuron in another), and no code or data is released. TCAV/LIME/SHAP are discussed in prose but never run as baselines, so the \"quantitative measurement\" claim has no yardstick.\n\nThe paper itself admits the experiments are limited to a small number of augmented patterns and that location was not varied, yet it still asserts that the model \"can successfully isolate the impact of these patterns independently of the surrounding data.\" That assertion is exactly what the missing control condition would test.\n\nWho this is for: someone working on post-hoc explainability for medical imaging will find the idea worth a look, but the paper as written is a working note, not a finished validation. I would send it to peer review, because the core idea is concrete and the missing pieces are fixable with a moderate revision: control augmentations, error bars, a TCAV comparison, and released code.","headline":"Plausible TCAV-style idea with a useful clinical direction, but the isolation claim is unsupported: missing control augmentations, no error bars, and an unused entropy metric make the quantitative claims premature.","tokens_in":9547,"tokens_out":2109,"would_cite":false,"duration_ms":21798,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Measuring a concept's true pull on a model's decision","keywords":["concept activation vector","explainable AI","radiomics","fundus images","model interpretability","imbalanced data","medical imaging","post-hoc explanation"],"falsifier":"Augment healthy fundus images with a randomly textured patch matched for size, location, and low-level statistics but carrying no medical concept, then measure the activation deviation; if the deviation is comparable to that of the fatty-dot or cotton-wool pattern, the measured shift is not concept-specific and the isolation claim fails.","tokens_in":1288,"feed_emoji":"👁️","tokens_out":1586,"duration_ms":48885,"temperature":0.7,"pith_summary":"This paper tries to establish that a trained classifier's dependence on a human-understandable visual concept can be measured directly by adding that concept to an otherwise normal input and observing how the network's internal activation vector shifts. If true, it would give clinicians and model developers a way to ask, of a single already-trained model, how much a specific radiomic pattern or lesion shape actually drives a decision, without training a second model or collecting separate concept images. The authors apply the method to retinal fundus images augmented with diabetic-eye patterns and to brain MRI images augmented with differently sized tumors, reporting that activation vectors move toward the disease class as patterns are added. They conclude that ACAV isolates the impact of high-level patterns independently of the surrounding data.","feed_headline":"Measuring a concept's true pull on a model's decision","feed_subtitle":"Adding a single pattern to a healthy image shows how much it shifts the model's internal decision vector.","key_machinery":"The central object is the augmented activation vector: for a correctly classified input $d_i$ lacking concept $C$, the network's layer-$l$ activation $\\mathbf{V}_l = f(d_i)$ is compared with the activation of the same input after overlaying the concept, $\\mathbf{V}_l^a = f(d_i \\oplus C)$, and the mean absolute deviation $\\Delta V = \\frac{1}{n}\\sum_i |\\mathbf{V}_l^a - \\mathbf{V}_l|$ is taken as the concept's influence. The method also uses cosine-similarity angles between the shifted vector and average healthy-versus-disease reference vectors, and an entropy term $H = -\\sum_i p_i \\log p_i$ to account for how pattern frequency in imbalanced data affects the model's decisions. These measurements are made on a single trained network, which is what distinguishes ACAV from TCAV's two-dataset procedure.","core_discovery":"The paper proposes Augmented Concept Activation Vector (ACAV), a post-hoc explanation method that quantifies how much a high-level concept contributes to a model's classification by overlaying the concept pattern onto input images that lack it, then comparing the activation vector of the layer before the output before and after augmentation. The shift is summarized by the average absolute deviation $\\Delta V = \\frac{1}{n}\\sum_i |\\mathbf{V}_l^a - \\mathbf{V}_l|$, by the fraction of augmented samples that flip to the opposite class, and by an entropy-based imbalance metric $H = -\\sum_{i=1}^C p_i\\log p_i$ that weighs concept prevalence. In experiments on fundus images augmented with cotton-wool, fatty-dot, and bleeding patterns, and on brain MRI images augmented with small, medium, and large tumors, the measured cosine-similarity angles between the augmented activation vectors and the disease-class reference vectors shrink as more or larger patterns are added, which the authors interpret as evidence that the concept's impact has been isolated and quantitatively measured.","pith_inferences":["The isolation claim rests on the untested premise that overlaying a concept changes only the high-level concept; a control augmentation that disturbs low-level statistics without adding a concept would reveal whether the measured shift is genuinely concept-specific.","If ACAV is reliable, it could serve as a debugging tool for medical imaging models, flagging when a classifier over-weights a pattern that is correlated with the label but not causally related.","The entropy-based imbalance metric could be extended into a weighting scheme for explanations, so that 'rare but decisive' features are presented with a prevalence-adjusted confidence.","The method could be tested on a synthetic dataset with known ground-truth concept contributions to check whether the measured angles and class-flip rates recover the true ranking of concept influence."],"forward_implications":["A single trained model can be interrogated for the quantitative contribution of each visual concept, removing the need for a second concept-trained network.","Rare but clinically valued patterns can be evaluated even when they appear infrequently in the dataset, because the concept is artificially added rather than looked for in existing samples.","The results quantify not only whether a pattern matters but by how much, in degrees of activation-vector shift and in class-flip rates, which could be directly reported in a model audit.","The method is architecture-agnostic and could be applied to any neural network and any concept with a visual representation, including shape, size, intensity, and location variations.","Measured shifts that align with clinical expectations suggest ACAV could surface unexpected dependencies, such as a model leaning on a pattern practitioners believe is secondary."],"supporting_citations":[{"why":"Supplies the TCAV baseline that ACAV extends; the paper defines its contribution against TCAV's need for a separate concept dataset and a second model.","marker":"[KWG+18]"},{"why":"Defines radiomics and its feature-extraction challenges, motivating the use of high-level radiomic patterns as the concepts to be measured.","marker":"[KGB+12]"}],"fun_headline_variants":["Quantify a concept's true influence on AI decisions","Augmented activation vectors measure concept pull","New method isolates how much a pattern matters","Measuring concept impact via augmented activations","How much does a concept sway a model? ACAV tells"],"cache_read_input_tokens":11776,"weakest_assumption_plain":"Adding a concept pattern to an image is assumed to change only that high-level concept, leaving brightness, edges, texture, and local contrast otherwise untouched, so any activation shift can be attributed uniquely to the concept.","fun_headline_variants_meta":{"raw":{"variants":["Quantify a concept's true influence on AI decisions","Augmented activation vectors measure concept pull","New method isolates how much a pattern matters","Measuring concept impact via augmented activations","How much does a concept sway a model? ACAV tells"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000173,"raw_usage":{"total_tokens":1246,"prompt_tokens":877,"completion_tokens":369,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":493,"completion_tokens_details":{"reasoning_tokens":299}},"tokens_in":493,"tokens_out":369,"duration_ms":4031,"temperature":1.0,"reasoning_tokens":299,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T00:50:10.360278+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Augment healthy fundus images with a randomly textured patch matched for size, location, and low-level statistics but carrying no medical concept, then measure the activation deviation; if the deviation is comparable to that of the fatty-dot or cotton-wool pattern, the measured shift is not concept-specific and the isolation claim fails.","supporting_citations":[],"review_version":1}