{"id":"07ad34d4-e3f9-4990-a54c-3ce29ff6802e","arxiv_id":"2507.05810","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"BAGEL trains per-layer logistic-regression probes on CLIP-defined concepts and compares per-class concept probabilities with dataset-level concept frequencies, visualizing the alignment in a knowledge graph.","lead":"The paper introduces BAGEL, a toolbox that probes image classifiers layer by layer to see which high-level concepts (colors, textures, objects) each layer's activations encode. It compares those concept signals with biases present in the training data and renders the comparison as an interactive knowledge graph.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Recall metric is circular: both the top-biased-concept ground truth and BAGEL's logistic-regression training targets come from the same CLIP labels, so high recall may measure self-consistency rather than genuine bias detection.","rationale":"I agree with the reader's REJECT verdict but identify a more targeted and load-bearing flaw than the reader's stated weakest assumption. The reader emphasizes unvalidated CLIP accuracy as a secondary concern, but I argue the evaluation is circular: the same CLIP labels define both the ground-truth biased concepts and the training signal for BAGEL's probes. This circularity directly invalidates the quantitative comparison in Table 1, independent of whether CLIP is accurate. The paper's own Section 5.2 concession further undermines the mechanistic-interpretability framing, but even as a bias-auditing tool the reported recall numbers cannot be interpreted as evidence of recovering dataset biases in the model. The proposed test is feasible with existing annotated datasets and would settle whether the concern lands. If the test showed no drop, I would move to CONDITIONAL; given current evidence, REJECT is appropriate.","tokens_in":15094,"tokens_out":1283,"duration_ms":15069,"concrete_test":"Recompute Table 1 using an independent human-annotated concept ground truth (e.g., Derm7pt or MonuMAI expert labels) for both the top-biased-concept reference and the logistic-regression training targets, and separately measure CLIP's per-concept accuracy against those labels. If recall drops substantially or CLIP accuracy is poor for any concept set, the reported BAGEL advantage is an artifact of shared CLIP supervision rather than genuine bias detection.","verdict_should_be":"REJECT","load_bearing_attack":"The central evaluation claim is that BAGEL recovers dataset biases better than TCAV/SAE. However, the ground truth for Table 1 is the top-5 most biased concepts per class computed from p_dataset(c_k|y_i) (Eq. 5), which uses CLIP zero-shot concept labels. BAGEL's logistic-regression probes are trained on exactly the same CLIP labels as y_k in Eq. 6. Thus the reference set and the method being scored share the same error source: any systematic CLIP misclassification inflates recall without measuring anything about the DNN. The comparison is also asymmetric: TCAV and SAE are adapted without per-concept label supervision, while BAGEL uses those labels directly. The paper provides no validation of CLIP accuracy on the chosen concept sets (Section 3.2), and Section 5.2 concedes that concept detection does not confirm causal reliance. Consequently, the headline claim of revealing latent circuits and information flow is unsupported: the knowledge graph edges encode correlational linear-probe probabilities, not mechanisms or causal pathways.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces BAGEL, a post-hoc concept-based interpretability framework that quantifies dataset-level concept-class bias using CLIP zero-shot labels, trains per-layer logistic-regression probes on global-average-pooled activations to estimate concept presence, compares dataset and model probabilities with a binarized F1 score and JS divergence, and visualizes the results as an interactive knowledge graph. It is evaluated on five datasets and eight architectures, with recall-based comparison against TCAV and SAE. The authors claim the method bridges concept-based XAI and mechanistic interpretability by 'revealing latent circuits and information flow' that underlie model decisions.","tokens_in":15277,"tokens_out":6163,"duration_ms":70153,"significance":"If validated, a model-agnostic, scalable global bias-analysis tool with per-layer concept tracking and interactive visualization would be a useful complement to local XAI and neuron-level dissection. The paper has concrete strengths: it is model-agnostic, uses a unified GAP representation across layers, includes an interactive demo, and contains an honest caveat in Section 5.2 that concept detection does not confirm causal reliance. However, the central evaluation is circular, the probe quality is never measured, and the mechanistic claims go beyond what linear probes on pooled activations can establish, so the contribution as currently stated is not established.","major_comments":[{"comment":"The evaluation is circular: the ground-truth top biased concepts are computed from p_dataset(c_k|y_i), which is obtained from CLIP zero-shot labels, and the same labels serve as training targets y_k in Eq. (6) for the logistic regressions. High recall therefore measures BAGEL's consistency with CLIP rather than its ability to recover biases actually used by the DNN. The authors should validate CLIP concept detection on the human-annotated datasets (Derm7pt, MonuMAI) and either use independent ground truth or explicitly analyze the impact of CLIP label noise on the reported recall.","section":"Section 3.2/3.3, Eq. (5)-(7), Table 1"},{"comment":"No evidence is provided that the per-layer logistic probes are accurate; the paper never reports probe accuracy, AUC, or confidence intervals. On Derm7pt and Cats/Dogs, the BAGEL F1 'Best Block' is exactly 0.80 for every architecture in Table 2, which is suspiciously constant and suggests metric saturation or an artifact of the top-k/binarization procedure. The authors should report per-block probe performance with error bars over multiple seeds and statistical significance tests.","section":"Section 3.3, Table 2"},{"comment":"The evaluation reports only the best-performing block for each method, which cherry-picks the layer and undermines the paper's central 'across layers' claim. If BAGEL is meant to show how concepts emerge and propagate through the network, the recall metric must be reported per block for all blocks, not only the maximum; otherwise the comparison cannot distinguish a method that works at a single layer from one that tracks concepts consistently throughout the network.","section":"Section 5.2, Supp. A, Table 2"},{"comment":"The claims of revealing 'latent circuits and information flow' and explaining 'how and why' the model makes decisions are not supported by the method. BAGEL fits independent logistic regressions on global-average-pooled activations; this yields correlational, single-layer concept probabilities with no interaction terms, no connectivity, and no causal intervention. The paper itself concedes in Section 5.2 that concept detection does not confirm causal reliance. Unless causal validation is added, the mechanistic-interpretability framing should be removed or explicitly downgraded to correlational bias analysis.","section":"Abstract, Section 1, Section 5.2"},{"comment":"The TCAV and SAE baselines are adapted to the task, but the adaptations are underspecified: no concept sets, number of CAVs, SAE architecture, latent count, or layer-selection details are given. Without a shared, precisely specified protocol, the recall comparison in Tables 1 and 2 cannot be reproduced or interpreted fairly.","section":"Supp. A"}],"minor_comments":[{"comment":"The notation p_model^(ell)(c_k|y_i) averages image-level probabilities pk(x) over class y_i; please clarify that this is an empirical estimate of P(c_k|y_i) and not a model conditional probability in the strict sense.","section":"Eq. (7)"},{"comment":"The edge width formula uses 'n_j' and 'c_i' in place of the earlier 'c_k' and 'y_i'; align the notation with the rest of Section 3.","section":"Section 3.5"},{"comment":"The caption mentions ResNet50 while the experimental setup lists ResNet18 but not ResNet50; verify which architecture was actually used.","section":"Figure 3 caption"},{"comment":"Reference [26] is a blog post; replace it with a peer-reviewed sparse-autoencoder reference, and merge the duplicated CLIP entries [44] and [45].","section":"References"},{"comment":"The main-text table reports only aggregated recall without error bars; include per-seed variance or confidence intervals so the reader can assess stability.","section":"Tables 1-2"}],"recommendation":"reject","confidential_remarks":"The authors have built a usable tool and include a useful caveat about correlation versus causation, but the evaluation is circular and the mechanistic claims are not supported by the method. A resubmission that reframes the contribution as correlational bias analysis, validates CLIP labels against human annotations, and reports per-layer results with error bars would be worth considering."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"BAGEL is a useful bias-auditing tool, but the mechanistic-interpretability framing overreaches. The abstract promises 'latent circuits and information flow,' yet the method measures correlational probabilities from linear probes, and the paper's own Section 5.2 concedes that concept detection doesn't confirm causal reliance. The main recall metric is also partly circular for the three CLIP-labeled datasets, because the same CLIP labels define both the ground-truth 'biased' concepts and the training targets for BAGEL's logistic regressions.\n\nWhat's genuinely new is the combination: per-layer global average pooling with linear concept probes, then comparing dataset-level CLIP-derived concept-class probabilities against model-level probe-derived probabilities. That yields a layer-resolved view of concept bias alignment, and the interactive knowledge graph is a useful way to explore it. Running the pipeline across eight architectures and five datasets gives decent breadth, and the paper is honest about some limitations.\n\nThe soft spots, in increasing severity. First, no error bars or repeated seeds anywhere; the recall numbers are single points. Second, BAGEL F1 best-block scores are 0.80 for every architecture on Derm7pt and Cats/Dogs, which suggests the metric saturates and doesn't discriminate between models. Third, no probe accuracy is reported, so we can't judge whether the concept classifiers actually decode the concepts. Fourth, the baseline comparisons are asymmetric: TCAV and SAE are adapted without per-concept label supervision while BAGEL uses those labels directly, and adaptation details are thin. Fifth, the circularity concern is real for the three non-annotated datasets; it would be partially mitigated by validating CLIP's accuracy on the chosen concepts or by leading with the human-annotated datasets.\n\nThis paper is better read as a bias-auditing visualization tool than as mechanistic interpretability. That framing is defensible, and the demo plus experimental breadth make it worth a referee's time. I'd want major revisions: soften the abstract, add error bars and probe accuracies, address the CLIP circularity, and report sensitivity to tau and top-k. On the current version my own verdict is mixed-to-reject; the strongest claims don't hold up. But I would not desk-reject it.","headline":"BAGEL is a useful bias-auditing tool, but the mechanistic-interpretability framing overreaches and the main recall metric is partly circular on the CLIP-labeled datasets.","tokens_in":15847,"tokens_out":4824,"would_cite":false,"duration_ms":46535,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper introduces BAGEL, a model-agnostic framework that maps how semantic concepts surface and interact across a network's layers and compares them against dataset biases in an interactive knowledge graph.","keywords":["BAGEL","concept-based interpretability","mechanistic interpretability","knowledge graphs","dataset bias","spurious correlations","CLIP zero-shot labeling","model-agnostic analysis"],"falsifier":"Run BAGEL on a dataset where CLIP's concept labels are known to be wrong (for example, stylized or textured images that defeat its text-image matching) and check whether the red 'model-specific bias' edges concentrate on those mislabeled concepts; if they do, the claimed bias signal is partly an artifact of the labeler rather than the model's internal circuitry.","tokens_in":14869,"feed_emoji":"🧠","tokens_out":8295,"duration_ms":78247,"temperature":0.7,"pith_summary":"BAGEL is a model-agnostic framework that tries to move concept-based interpretability from single-prediction explanations to a global, layer-by-layer account of how a vision model uses high-level semantic concepts. It estimates, for each layer and class, how often a concept is present in internal activations, and compares those numbers against the concept-class frequencies in the training dataset. The comparison is rendered as an interactive knowledge graph whose edge colors mark concepts the dataset and model agree on, biases the model failed to learn, and biases the model invented on its own. If the framework works as claimed, it gives practitioners a direct way to see which spurious correlations a model has absorbed and where in the network they live.","feed_headline":"BAGEL graphs concept bias layer by layer across vision models","feed_subtitle":"Colored edges reveal which biases come from the data, which from the model, and which both share.","key_machinery":"The load-bearing object is the per-layer concept probability $p_{\\text{model}}^{(\\ell)}(c_k|y_i)$ (Eq. 7), built by applying global average pooling to each convolutional layer to get a vector $\\phi^\\ell(x)$ (Eq. 4), training a logistic regression $g_k^{(\\ell)}$ to predict concept presence from that vector (Eq. 6), and averaging its outputs over images of a class. This is compared with the empirical dataset bias $p_{\\text{dataset}}(c_k|y_i)=N_{y_i,c_k}/N_{y_i}$ (Eq. 5), where concept labels come from CLIP zero-shot classification. The comparison is summarized by weighted F1 and Jensen-Shannon divergence, and visualized in a directed labeled graph $G=(V,E,W)$ whose edge colors encode overlap, dataset-only, and model-only biases.","core_discovery":"The central claim is that a network's internal representations can be read as a layer-resolved probability distribution over human-interpretable concepts, and that comparing this distribution to the dataset's empirical concept-class distribution exposes the mechanisms behind biased decisions. For each class $y_i$, concept $c_k$, and layer $\\ell$, BAGEL computes $p_{\\text{model}}^{(\\ell)}(c_k|y_i)$ by training logistic-regression concept classifiers on global-average-pooled activations, and contrasts it with $p_{\\text{dataset}}(c_k|y_i)$ estimated from CLIP zero-shot labels. Weighted F1 and Jensen-Shannon divergence quantify alignment; the knowledge graph colors each class-concept edge green (both agree), blue (dataset bias the model does not carry), or red (model-specific bias absent from the data). The paper reports that BAGEL recovers top dataset-biased concepts with recall comparable to or better than TCAV and SAE across five datasets and eight architectures.","pith_inferences":["A testable extension: replace the CLIP concept labeler with human annotations or an independent detector and rerun the pipeline; if the red/blue edge patterns shift, part of what BAGEL labels as model bias is actually labeler bias.","A testable extension: use red edges that persist across many layers to prioritize data collection, because those concepts are ones the model relies on even though the dataset does not support them.","A practical extension: track each concept's layer-resolved probabilities over training checkpoints to see when a spurious correlation becomes linearly decodable, which could inform early stopping or targeted debiasing."],"forward_implications":["A user can adjust the threshold $\\tau$, navigate through layers, and visually separate dataset-driven biases from model-invented ones for any class-concept pair.","Because the pipeline only needs forward hooks to read activations, it applies to convolutional and fully connected networks without retraining.","Layer-wise concept probabilities show a consistent trend across architectures: low-level concepts such as colors and textures peak in early layers, while object parts become prominent in deeper layers.","On the reported benchmarks, BAGEL's F1-based recall reaches 0.8 for best-block performance on most datasets, and its JS-based variant matches or exceeds TCAV on MonuMAI and KitFox vs RedFox."],"supporting_citations":[{"why":"Supplies the CLIP zero-shot concept-presence labels that define the dataset-bias estimate and the training target for the per-layer concept classifiers.","marker":"[44]"},{"why":"Provides the unit-level and global-average-pooling view of activations that BAGEL generalizes to all layers.","marker":"[4]"},{"why":"Introduces the idea of training linear classifiers on internal activations to test for concepts, which BAGEL adapts per layer.","marker":"[31]"},{"why":"Serves as the TCAV baseline whose top-concept recall BAGEL is compared against.","marker":"[32]"},{"why":"Supplies the sparse-autoencoder baseline adapted for the biased-concept detection comparison.","marker":"[26]"},{"why":"Frames concepts as class-conditioned distributions, the same conditional-probability structure used in BAGEL's dataset bias.","marker":"[33]"}],"fun_headline_variants":["BAGEL exposes concept bias layer by layer in a graph","Knowledge graph shows if bias is from data or model","BAGEL dissects model bias with colored concept graph","Layer-resolved concept bias visualized in BAGEL graph","BAGEL graph reveals model vs dataset bias origins"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire pipeline assumes that CLIP's zero-shot concept labels correctly say which concepts appear in an image, because those same labels set the dataset-bias statistics and train the concept classifiers.","fun_headline_variants_meta":{"raw":{"variants":["BAGEL exposes concept bias layer by layer in a graph","Knowledge graph shows if bias is from data or model","BAGEL dissects model bias with colored concept graph","Layer-resolved concept bias visualized in BAGEL graph","BAGEL graph reveals model vs dataset bias origins"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000568,"raw_usage":{"total_tokens":2679,"prompt_tokens":927,"completion_tokens":1752,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":543,"completion_tokens_details":{"reasoning_tokens":1672}},"tokens_in":543,"tokens_out":1752,"duration_ms":13215,"temperature":1.0,"reasoning_tokens":1672,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:17:26.048174+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run BAGEL on a dataset where CLIP's concept labels are known to be wrong (for example, stylized or textured images that defeat its text-image matching) and check whether the red 'model-specific bias' edges concentrate on those mislabeled concepts; if they do, the claimed bias signal is partly an artifact of the labeler rather than the model's internal circuitry.","supporting_citations":[{"cited_title":"Learning transferable visual models from natural language supervision","cited_arxiv_id":null,"evidence_quote":"Supplies the CLIP zero-shot concept-presence labels that define the dataset-bias estimate and the training target for the per-layer concept classifiers."},{"cited_title":"Network dissection: Quantifying in- terpretability of deep visual representations","cited_arxiv_id":null,"evidence_quote":"Provides the unit-level and global-average-pooling view of activations that BAGEL generalizes to all layers."},{"cited_title":"Inter- pretability beyond feature attribution: Quantitative test- ing with concept activation vectors (tcav), 2018","cited_arxiv_id":null,"evidence_quote":"Serves as the TCAV baseline whose top-concept recall BAGEL is compared against."},{"cited_title":"An intuitive explanation of sparse autoencoders for llm interpretability","cited_arxiv_id":null,"evidence_quote":"Supplies the sparse-autoencoder baseline adapted for the biased-concept detection comparison."},{"cited_title":"Concept bottleneck models","cited_arxiv_id":null,"evidence_quote":"Frames concepts as class-conditioned distributions, the same conditional-probability structure used in BAGEL's dataset bias."}],"review_version":1}