Pith. sign in

REVIEW 6 major objections 5 minor 59 references

Neural Concept Verifier: Scaling Prover-Verifier Games via Concept Encodings

T0 review · 6 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper argues that moving the prover-verifier proof game from pixels to compact concept encodings makes verifiable, interpretable classification practical on high-dimensional images such as ImageNet, with completeness matching or…

desk verdict NCV is a sensible new combination of Merlin-Arthur games and concept encodings with honest limitations sections, but its headline real-world comparisons are confounded by representation choice and missing rejection rates. read the letter →

arxiv 2507.07532 v4 pith:SF6T4ACB submitted 2025-07-10 cs.LG cs.AI

Berkant Turan , Suhrab Asadulla , David Steinmann , Kristian Kersting , Wolfgang Stammer , Sebastian Pokutta This is my paper
classification cs.LGcs.AI
keywords prover-verifiergamesMerlin-Arthurclassifiersconceptbottleneckmodelsencodingsverifiableclassificationcompletenessandsoundnessshortcutlearninginterpretablemachine
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that formal verifiability for nonlinear classifiers can be made to work on high-dimensional images by moving the proof game from raw pixels to compact concept encodings. It introduces the Neural Concept Verifier, in which a cooperative prover selects a sparse set of interpretable concepts, an adversarial prover selects deliberately misleading ones, and a nonlinear verifier must classify from those selections alone, never seeing the full image and with the option to abstain. If correct, the approach gives classifiers that are accurate and can justify every decision with a short, human-readable list of concepts, while closing the accuracy gap that has kept concept-based models tied to constrained linear heads. The paper reports that the scheme matches or outperforms pixel-space verifier baselines and opaque ResNets in completeness on most benchmarks while keeping soundness near perfect, and that it shrinks reliance on spurious correlations when even a few clean samples are available.

What carries the argument

The carrying mechanism is a prover-verifier game relocated into concept space. The named pieces are Merlin, the cooperative prover, and Morgana, the adversarial prover — two neural networks that output sparse binary masks over the concept encoding — and Arthur, the nonlinear verifier, which sees only the masked concept values and predicts into $K$ classes plus a rejection class $\perp$. The two quantitative objects that carry the argument are completeness, $P(A(S)=y)$, and soundness, $P(A(\hat{S})\in\{y,\perp\})$, where $S$ is the masked concept subset produced by Merlin and $\hat{S}$ the one produced by Morgana. Training alternates between the provers and Arthur: the provers are updated through continuous soft masks so gradients flow, the masks are then rounded by a top-$m$ operator, and Arthur is trained on the hard-masked subsets with loss $(1-\gamma)L_{\text{Merlin}}+\gamma L_{\text{Morgana}}$, where Morgana's loss treats abstention as a sound outcome. The formal spine is a one-vs-rest instantiation of the Merlin-Arthur mutual-information bound: for each class $k$, the average precision of Merlin's concept features lower-bounds the mutual information $I(Y_k(y); M(x')\subseteq x)$ through the binary entropy function, and the observable completeness and soundness errors jointly lower-bound that precision whenever the asymmetric feature correlation $\kappa_k$, Morgana's relative success rate $\alpha_k$, and the class imbalance $B_k$ are well-behaved.

What would settle it

Count how often Merlin's selected concept subsets actually recur across a held-out set in the CLIP-based regime: if the 32-concept masks are almost all unique per image, the containment event $\{M(x')\subseteq x\}$ has near-zero empirical probability, so the average-precision and mutual-information quantities behind the faithfulness claim cannot be estimated from data and the certificate is vacuous there — the paper itself says the bound is most readily evaluated for discrete, low-dimensional encodings, not dense continuous ones such as CLIP. A second check is to search over concept subsets at inference time with an adversary that is not the trained Morgana; if soundness drops well below the reported ~99.9%, the guarantee is robustness against a specific trained adversary rather than protection against any misleading subset.

Watch

Extended reading notes

Core claim

The central discovery is that the Merlin-Arthur prover-verifier game, which previously stalled on high-dimensional inputs because it argued over pixels, can be played in concept space without losing its formal guarantees. A frozen concept extractor maps each image to an encoding — object slots from the Neural Concept Binder for CLEVR-style scenes, or CLIP similarity scores against a 10,000-word caption vocabulary for natural images. Two differentiable provers, Merlin and Morgana, emit sparse binary masks selecting a fixed small number of concepts, and Arthur, a nonlinear verifier (a Set Transformer or two-layer MLP), predicts the label from the masked concepts alone, with an explicit rejection class it may use when uncertain. The paper defines completeness as the probability that Arthur recovers the true label from Merlin's subset and soundness as the probability that Arthur, on Morgana's adversarial subset, either stays correct or abstains; it then instantiates the Merlin-Arthur theory class-wise, showing that under bounded feature-correlation, class-imbalance, and relative-adversary-strength assumptions, high completeness and soundness imply that Merlin's sparse selections carry near-maximal mutual information about the label in each one-vs-rest subproblem. Across CLEVR-Hans3/7, CIFAR-100, ImageNet-1k, and COCOLogic, the paper reports that NCV matches or beats all baselines in completeness in most settings, achieves soundness at or near 100%, and narrows or closes the interpretability-accuracy gap of linear concept bottleneck models.

Load-bearing premise

The load-bearing premise is that the frozen concept extractor's encodings contain enough task-relevant information, with consistent semantics across inputs, that a prover-verifier game played on top of them can both classify accurately and justify itself; if the concept space discards or entangles the information that separates classes, no game can recover it, and the paper's own Discussion concedes that effectiveness depends on the quality of the underlying concept extractor.

Editorial extensions

If this is right

  • Prover-verifier games leave the toy-data regime: the same game machinery runs on $224\times224$ images with 1,000 classes, at a training cost the paper estimates as one to three orders of magnitude below pixel-space provers.
  • Concept bottleneck models keep their interpretable concept interface while gaining nonlinear heads: NCV matches or beats linear CBMs and opaque ResNets on CIFAR-100 and COCOLogic, so the interpretability-accuracy gap is not intrinsic to concept bottlenecks.
  • Verified decisions come with human-readable certificates: a class explanation is a short concept list such as 'small and metal and sphere' rather than a diffuse pixel mask, which is what pixel-space provers return.
  • The adversarial branch acts as robustness training against spurious correlations: with only 1-5% clean samples, NCV's validation-to-test gap under shortcuts drops faster than CBM baselines, indicating reduced shortcut reliance.
  • Near-perfect soundness means a misleading concept subset makes Arthur abstain rather than commit to an error, giving the model a built-in 'I don't know' response under adversarial selection.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Soundness is reported against the specific Morgana network actually trained, so 'verifiable' here means robust to that adversary class; a natural stress test is an independent inference-time search over concept subsets, and the paper's own supplement concedes this dependence.
  • The mutual-information certificate needs selected concepts to recur across inputs, which holds for the discrete NCB encodings but not for dense CLIP similarities — so the strongest-sounding guarantee applies precisely in the regime where the explanation is already easiest to inspect by hand.
  • The shortcut results suggest Morgana's selections teach Arthur to distrust spurious attributes; one observable test is to record Morgana's masks early in training and check whether they preferentially mark shortcut concepts such as 'gray' on CLEVR-Hans.
  • The theory bounds each class separately and says nothing directly about the joint K-class argmax; deriving a genuine multiclass certificate, or testing whether the class-wise bounds compose, would be the next step toward an end-to-end guarantee.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 5 minor

Summary. The paper introduces the Neural Concept Verifier (NCV), which combines a pretrained concept extractor (NCB on CLEVR-Hans, CLIP-Sim on CIFAR-100, ImageNet-1k, and COCOLogic) with a Merlin-Arthur prover-verifier game played over sparse concept masks. A cooperative prover Merlin selects a small concept subset to support the true class, an adversarial prover Morgana selects misleading subsets, and a nonlinear verifier Arthur predicts from the masked concepts, with completeness and soundness as the headline metrics. The authors report that NCV scales prover-verifier games to high-dimensional images, narrows the interpretability-accuracy gap of linear concept bottleneck models, produces more interpretable explanations than pixel-level masks, and mitigates shortcut learning. The central empirical claim is that NCV matches or outperforms concept-based and pixel-based baselines in completeness on most real-world settings while retaining near-perfect soundness.

Significance. If the central claims were established, the contribution would be useful: NCV provides a clean way to enforce sparse, per-sample concept selection through an adversarial game while keeping a nonlinear classifier, and it is evaluated over a wide range of datasets (CLEVR-Hans3/7, CIFAR-100, ImageNet-1k, COCOLogic) with 10-20 seeds. The paper also ships public code, reports computational costs, includes ablations over mask size and the trade-off parameter gamma, and is unusually transparent about the limitations of the information-theoretic guarantees in App. A.5. These are genuine strengths. However, as submitted, the main empirical claims for the real-world datasets are not yet supported because the completeness comparisons conflate the prover-verifier game with a change of concept representation, and because the near-perfect soundness figures are reported without any information about how much of them is abstention.

major comments (6)
  1. [§4.1 and Table 1] Section 4.1 promises that within each dataset, NCV is compared against baselines that share the same concept extractor, but this is not what Table 1 does for CIFAR-100, ImageNet-1k, or COCOLogic. The CBM baselines use per-sample optimized sparse SpLiCE encodings (Suppl. C.3), while NCV uses CLIP-Sim, which Sec. 4.1 explicitly defines as retaining the full dense activation vector. The headline completeness comparison therefore conflates two changes: the Merlin-Arthur game and the choice of dense CLIP-Sim over sparse SpLiCE. To support the Q1/Q2 claims, the authors should add a nonlinear CBM trained on the identical dense CLIP-Sim features, or alternatively an NCV variant operating on SpLiCE encodings, so that the representation is held fixed.
  2. [Table 8 vs. Table 1] The claim that NCV matches or outperforms baselines in completeness on real-world datasets is undercut by the paper's own pretraining results. Table 8 shows that the pretrained verifier alone, without any prover, reaches 85.96% on CIFAR-100, 77.07% on ImageNet-1k, and 81.39% on COCOLogic, whereas the full NCV game achieves 83.32%, 67.04%, and 75.42%, respectively. Thus on every real-world dataset the prover-verifier game reduces completeness relative to its own pretrained verifier on the same CLIP-Sim features. The apparent advantage over the SpLiCE-based CBMs should therefore be attributed to the dense representation, not to the game, and the interpretability-accuracy gap reduction is not established without a nonlinear CBM on the same features.
  3. [§3.4, Eq. (2), and Suppl. A.4] Soundness as defined in Eq. (2) treats the rejection class as success, and the Morgana loss in App. A.4 switches the target to the rejection class whenever Arthur already prefers abstention over the true class, in addition to a softplus penalty on the gap between the true-class and rejection logits. Under this training objective, near-perfect soundness can be achieved largely by abstaining. The paper never reports rejection or abstention rates for any model or dataset, so a reader cannot tell whether the reported 99.9% soundness means that Morgana fails to elicit wrong predictions or that Arthur simply declines to answer. The authors should report soundness conditional on making a class prediction, together with the overall rejection rate.
  4. [§4.2 and Suppl. C.2] The Pixel-MAC comparison is not a controlled test of whether prover-verifier games scale to high-dimensional images. For CIFAR-100, Pixel-MAC is restricted to a mask of 32 pixels out of 32x32 pixels, and for ImageNet-1k and COCOLogic to 1000 pixels out of roughly 50k pixels, while NCV's 32 concepts are full-image CLIP similarity scores. The low Pixel-MAC completeness on these datasets is therefore confounded by an extremely small pixel budget and by the different information content of a pixel versus a global concept score. The authors should either give Pixel-MAC a comparable fraction of the input (e.g., by using a larger patch budget, or by reporting a Pixel-MAC variant with the same number of scalar features) or restrict the scaling claim to the CLEVR-Hans settings where the mask sizes are more comparable.
  5. [§3.5 and App. A.5] The theoretical section is presented as providing information-theoretic grounding, but App. A.5 explicitly states that the mutual-information bounds are not estimated numerically, that the assumptions on asymmetric feature correlation and Morgana's relative success rate are not verified, and that the precision is only a population quantity that is not meaningfully estimable for dense continuous encodings. This is an honest and appropriate caveat, but it should be reflected in the abstract and introduction: the current wording, especially formal verifiability and strong soundness guarantees, overstates what the experiments establish, since the reported completeness and soundness are empirical proxies rather than certified guarantees.
  6. [Table 1] Table 1's matches or outperforms summary is also inaccurate on ImageNet-1k, where NCV's completeness (67.04) is below both the linear CBM (68.59) and the nonlinear CBM (69.02). The paper's text does acknowledge this in §4.2, but the table caption and the Q2 discussion should state more precisely that the advantage is limited to CIFAR-100 and COCOLogic, and even there only against the SpLiCE-based CBMs.
minor comments (5)
  1. [Abstract and §6] The abstract's formal verifiability and the conclusion's verifibale explanations (typo) should be aligned with the limitations stated in App. A.5; I suggest replacing formal verifiability with something like verifiable, concept-level reasoning and correcting the typo.
  2. [§3.6] The text says the three modules are updated jointly and then describes a two-phase alternating scheme with continuous masks for prover updates and hard top-m masks for Arthur's update; this is clear in App. A.4, but the main text would benefit from a pointer to the exact loss formulations for Merlin's and Morgana's updates, since the footnote referring to Sec. A.4 is easy to miss.
  3. [§4.1 and Table 1] The table header layout with three column blocks is hard to parse because the dataset names are separated from their columns by the shared completeness/soundness headers; adding explicit column groups for each dataset would improve readability.
  4. [Suppl. E.5] The comparison of selected concepts between NCV and a linear CBM is helpfully honest about being illustrative, but the sentence the verifier is therefore not simply handed the answer is a bit misleading because the concept vocabulary deliberately contains phrases that are surface forms of class names; I would acknowledge this more concretely in the main text when discussing explanation quality.
  5. [§4.1 and Suppl. C.3] The term CLIP-Sim is defined only in passing in Sec. 4.1; since it is the central representation for the real-world experiments, a short formal definition (e.g., c_j = cosine similarity between image embedding and text embedding of concept j) would help the reader.

Circularity Check

0 steps flagged · score 2.0 of 10

No derivational circularity: headline metrics are measured benchmarks; Merlin-Arthur theory is imported but explicitly treated as motivation, and the self-defined adversary's limitation is disclosed.

full rationale

No circular step satisfies the evidence rule. Completeness and soundness (Eqs. 1-2) are empirical accuracies measured on trained provers and verifier, not quantities derived from fitted parameters. The information-theoretic bound in Eq. 3 is quoted from Wäldchen et al. (2024), and Sec. 3.5 states: 'we treat the theory as the motivation for our faithfulness criterion rather than as a quantity measured directly in our experiments.' Suppl. A likewise says 'We do not prove new theorems here; rather, we instantiate existing results,' and Suppl. A.5 openly limits soundness to 'robustness against this trained adversary class, rather than against an arbitrary worst-case adversary.' The import of the co-authored Merlin-Arthur theory is therefore disclosed and non-load-bearing for the reported numbers, and the near-perfect soundness values are a measured fit to a correct-or-abstain training target rather than an independent certificate. Two non-circular caveats remain: the 'same concept extractor' control promised in Sec. 4.1 is not honored in Table 1 for real-world datasets (CBM baselines use SpLiCE while NCV uses CLIP-Sim), and rejection rates are not reported, so soundness magnitudes should be read with caution. These are validity concerns, not derivation-by-construction, hence the low score.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim depends on hyperparameters such as mask size and gamma, plus several domain assumptions about the concept extractor and the adversary. No new physical or mathematical entities are postulated. The theory relies on assumptions kappa_k and alpha_k that the paper explicitly does not estimate.

free parameters (4)
  • Mask size m = 12 (NCB), 32 (CLIP)
    Number of concepts or slots the provers may select; directly controls accuracy/soundness trade-off and is chosen per dataset with ablations in Suppl. D/E.
  • Weight parameter gamma = 0.5 for main results
    Balances Merlin and Morgana losses in Arthur's objective; ablation in Suppl. E.4 shows soundness collapses at gamma=0.
  • L1 penalty on provers = 0.1 (CLIP instantiations)
    Added to incentivize sparse masks; interacts with mask size and is not derived from theory.
  • Concept vocabulary size = 10,000
    Top 10k one- and two-word phrases from LAION captions, following SpLiCE; vocabulary choice affects which concepts exist.
assumptions (4)
  • domain assumption Label y is determined by input x
    Used in Suppl. A.1/A.2 to apply the Merlin-Arthur precision bounds; the paper assumes imperfect precision reflects insufficient concepts rather than label noise.
  • domain assumption Concept extractor g produces encodings preserving sufficient task information
    Sec. 3.3 requires expressiveness; if encodings lose task-relevant information, no sparse concept selection can recover accuracy.
  • ad hoc to paper Asymmetric feature correlation kappa_k is bounded and Morgana's relative success rate alpha_k is not tiny
    Required for the precision bound in Eq. 7; the paper uses symmetric architectures as heuristic evidence but does not estimate alpha_k.
  • domain assumption Selected concepts recur across inputs for estimating average precision
    Suppl. A.2/A.5: precision is only practically estimable for discrete or low-dimensional encodings; for dense CLIP encodings the paper treats the bound as a qualitative lens.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Neural Concept Verifier: Scaling Prover-Verifier Games via Concept Encodings." pith.science (2026). https://pith.science/paper/SF6T4ACB

@misc{pith2026250707532,
  author       = {Pith},
  title        = {Pith review of: Neural Concept Verifier: Scaling Prover-Verifier Games via Concept Encodings},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SF6T4ACB}},
  note         = {Machine review of arXiv:2507.07532}
}
read the original abstract

While Prover-Verifier Games (PVGs) offer a promising path toward verifiability in nonlinear classification models, they have not yet been applied to complex inputs such as high-dimensional images. Conversely, expressive concept encodings effectively allow to translate such data into interpretable concepts but are often utilised in the context of low-capacity linear predictors. In this work, we push towards real-world verifiability by combining the strengths of both approaches. We introduce Neural Concept Verifier (NCV), a unified framework combining PVGs for formal verifiability with concept encodings to handle complex, high-dimensional inputs in an interpretable way. NCV achieves this by utilizing recent minimally supervised concept discovery models to extract structured concept encodings from raw inputs. A prover then selects a subset of these encodings, which a verifier, implemented as a nonlinear predictor, uses exclusively for decision-making. Our evaluations show that NCV outperforms classic concept-based models and pixel-based PVG classifier baselines on high-dimensional, logically complex datasets and helps mitigate shortcut behavior. Overall, we demonstrate NCV as a promising step toward concept-level, verifiable AI.

Figures

Figures reproduced from arXiv: 2507.07532 by the authors.

Figure 1
Figure 1. Challenges of Prover-Verifier Games (PVGs) in image classification: (i) It is non-trivial to scale up for high-dimensional data. (ii) Furthermore, the learned explanation masks on the pixel level remain difficult for humans to understand. stakes applications, it is critical that models provide veri￾fiable justifications for their decisions (Irving et al., 2018; Fok & Weld, 2023). Prover-Verifier Games (PVGs), intro￾… view at source ↗
Figure 2
Figure 2. Overview of the Neural Concept Verifier (NCV). The input image is first processed by a concept extractor to produce symbolic concept encodings. A prover–verifier game is then played over these encodings: a cooperative prover selects a sparse concept subset supporting the true class, while an adversarial prover selects misleading concepts. Finally, the nonlinear verifier makes a prediction based only on these selecte… view at source ↗
Figure 3
Figure 3. Comparison of explanations from NCV vs. Pixel-MAC. (top) Merlin–Arthur training on pixel space yields uninformative masks. (bottom) MAC on a concept-space via NCV translates into combinations of high-level concepts and, in turn, in an interpretable prediction. a total mask size of 32) across 32 samples for ImageNet-1k. All concepts are part of the CLIP/SpLiCE-based concept vo￾cabulary that is derived from LAION capt… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Comparison of explanations from NCV vs. Pixel-MAC for CLEVR-Hans3 images of all three classes. (a) Merlin–Arthur training on pixel space yields uninformative masks. (b) NCV provides clear explanations by highlighting object features corresponding to the class rule. The…
Figure 5
Figure 5. Figure 5: Effect of the weighting parameter γ on completeness and soundness for (a) CIFAR-100, (b) ImageNet-1k validation, and (c) COCOLogic validation (balanced metrics). All curves show means over 3 random seeds. E.4. Effect of the weighting parameter γ The weighting parameter…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

59 extracted references · 39 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    B., Goldgof, G

    Ahmed, K. B., Goldgof, G. M., Paul, R., Goldgof, D. B., and Hall, L. O. (2021). Discovery of a generalization gap of convolutional neural networks on covid-19 x-rays classification. Ieee Access , 9:72970--72979

    work page 2021

  3. [3]

    Amit, N., Goldwasser, S., Paradise, O., and Rothblum, G. (2024). Models that prove their own correctness. arXiv preprint arXiv:2405.15722

    arXiv 2024

  4. [4]

    Anil, C., Zhang, G., Wu, Y., and Grosse, R. (2021). Learning to give checkable answers with prover-verifier games. arXiv preprint arXiv:2108.12099

    arXiv 2021

  5. [5]

    Bhalla, U., Oesterling, A., Srinivas, S., Calmon, F., and Lakkaraju, H. (2024). Interpreting clip with sparse linear concept embeddings (splice). Advances in Neural Information Processing Systems , 37:84298--84328

    work page 2024

  6. [6]

    U., and Kahembwe, E

    Birhane, A., Prabhu, V. U., and Kahembwe, E. (2021). Multimodal datasets: misogyny, pornography, and malignant stereotypes. CoRR , abs/2110.01963

    arXiv 2021

  7. [7]

    Bortolotti, S., Marconato, E., Morettin, P., Passerini, A., and Teso, S. (2025). Shortcuts and identifiability in concept-based models from a neuro-symbolic lens. CoRR , abs/2502.11245

    arXiv 2025

  8. [8]

    Brown-Cohen, J., Irving, G., and Piliouras, G. (2024). Scalable ai safety via doubly-efficient debate. In Proceedings of the 41st International Conference on Machine Learning , ICML'24. JMLR.org

    work page 2024

Show all 59 references
  1. [9]

    and van der Schaar, M

    Crabb \'e , J. and van der Schaar, M. (2022). Concept activation regions: A generalized framework for concept-based explanations. Advances in Neural Information Processing Systems , pages 2590--2607

    2022

  2. [10]

    Debole, N., Barbiero, P., Giannini, F., Passeggini, A., Teso, S., and Marconato, E. (2025). If concept bottlenecks are the question, are foundation models the answer? CoRR , abs/2504.19774

    2025 arXiv

  3. [11]

    Delfosse, Q., Sztwiertnia, S., Rothermel, M., Stammer, W., and Kersting, K. (2024). Interpretable concept bottlenecks to align reinforcement learning agents. Advances in Neural Information Processing Systems (NeurIPS)

    2024

  4. [12]

    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 248--255

    2009

  5. [13]

    B., and Mordatch, I

    Du, Y., Li, S., Torralba, A., Tenenbaum, J. B., and Mordatch, I. (2024). Improving factuality and reasoning in language models through multiagent debate. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F., editors, Proceeding...

    2024

  6. [14]

    and Weld, D

    Fok, R. and Weld, D. S. (2023). In search of verifiability: Explanations rarely enable complementary performance in ai-advised decision making. AI Magazine

    2023

  7. [15]

    Friedrich, F., Stammer, W., Schramowski, P., and Kersting, K. (2023). A typology for exploring the mitigation of shortcut behaviour. Nature Machine Intelligence , 5(3):319--330

    2023

  8. [16]

    Gehman, S., Gururangan, S., Sap, M., Choi, Y., and Smith, N. A. (2020). Realtoxicityprompts: Evaluating neural toxic degeneration in language models. CoRR , abs/2009.11462

    2020 arXiv

  9. [17]

    Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., and Wichmann, F. A. (2020). Shortcut learning in deep neural networks. Nature Machine Intelligence , 2(11):665--673

    2020

  10. [18]

    Y., and Kim, B

    Ghorbani, A., Wexler, J., Zou, J. Y., and Kim, B. (2019). Towards automatic concept-based explanations. Advances in Neural Information Processing Systems (NeurIPS)

    2019

  11. [19]

    G., and Pokutta, S

    G uch, G., Turan, B., Nagarajan, S. G., and Pokutta, S. (2024). The G ood, the B ad and the U gly: Watermarks, T ransferable A ttacks and A dversarial D efenses. arXiv preprint arXiv:2410.08864

    2024

  12. [20]

    Goldwasser, S., Micali, S., and Rackoff, C. (1985). The knowledge complexity of interactive proof-systems. In Proceedings of the Seventeenth Annual ACM Symposium on Theory of Computing , STOC '85, page 291–304, New York, NY, USA. Association for Computing Machinery

    1985

  13. [21]

    Havasi, M., Parbhoo, S., and Doshi-Velez, F. (2022). Addressing leakage in concept bottleneck models. Advances in Neural Information Processing Systems , 35:23386--23397

    2022

  14. [22]

    He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2016

  15. [23]

    and Gimpel, K

    Hendrycks, D. and Gimpel, K. (2016). Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415

    2016 arXiv

  16. [24]

    Irving, G., Christiano, P., and Amodei, D. (2018). Ai safety via debate. In arXiv preprint arXiv:1805.00899

    2018 arXiv

  17. [25]

    Johnson, J., Hariharan, B., van der Maaten, L., Fei-Fei, L., Lawrence Zitnick, C., and Girshick, R. (2017). Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2017

  18. [26]

    Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., et al. (2018). Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International Conference on Machine Learning (ICML) , pages 2668--2677. PMLR

    2018

  19. [27]

    Kimura, M., Shimizu, R., Hirakawa, Y., Goto, R., and Saito, Y. (2024). On permutation-invariant neural networks. arXiv preprint arXiv:2403.17410

    2024 arXiv

  20. [28]

    H., Chen, Y., Edwards, H., Leike, J., McAleese, N., and Burda, Y

    Kirchner, J. H., Chen, Y., Edwards, H., Leike, J., McAleese, N., and Burda, Y. (2024). Prover-verifier games improve legibility of llm outputs. arXiv preprint arXiv:2407.13692

    2024 arXiv

  21. [29]

    W., Nguyen, T., Tang, Y

    Koh, P. W., Nguyen, T., Tang, Y. S., Mussmann, S., Pierson, E., Kim, B., and Liang, P. (2020). Concept bottleneck models. In International conference on machine learning , pages 5338--5348. PMLR

    2020

  22. [30]

    Krizhevsky, A. (2009). Learning multiple layers of features from tiny images. Technical report, University of Toronto

    2009

  23. [31]

    a ldchen, S., Binder, A., Montavon, G., Samek, W., and M \

    Lapuschkin, S., W \"a ldchen, S., Binder, A., Montavon, G., Samek, W., and M \"u ller, K.-R. (2019). Unmasking clever hans predictors and assessing what machines really learn. Nature Communications , 10(1):1--8

    2019

  24. [32]

    Lee, J., Lee, Y., Kim, J., Kosiorek, A., Choi, S., and Teh, Y. W. (2019). Set transformer: A framework for attention-based permutation-invariant neural networks. In Chaudhuri, K. and Salakhutdinov, R., editors, Proceedings of the 36th International Conference on Machine Learni...

    2019

  25. [33]

    H., Lanza, S., and Wermter, S

    Lee, J. H., Lanza, S., and Wermter, S. (2025). From neural activations to concepts: A survey on explaining concepts in neural networks. Neurosymbolic Artificial Intelligence , 1:NAI--240743

    2025

  26. [34]

    Mahinpei, A., Clark, J., Lage, I., Doshi-Velez, F., and Pan, W. (2021). Promises and pitfalls of black-box concept learning models. arXiv preprint arXiv:2106.13314

    2021 arXiv

  27. [35]

    Marconato, E., Teso, S., Vergari, A., and Passerini, A. (2023). Not all neuro-symbolic concepts are created equal: Analysis and mitigation of reasoning shortcuts. Advances in Neural Information Processing Systems , 36:72507--72539

    2023

  28. [36]

    Mescheder, L., Geiger, A., and Nowozin, S. (2018). Which training methods for GAN s do actually converge? In Dy, J. and Krause, A., editors, Proceedings of the 35th International Conference on Machine Learning , volume 80 of Proceedings of Machine Learning Research , pages 348...

    2018

  29. [37]

    and Kolter, J

    Nagarajan, V. and Kolter, J. Z. (2017). Gradient descent gan optimization is locally stable. In Proceedings of the 31st International Conference on Neural Information Processing Systems , NIPS'17, page 5591–5600, Red Hook, NY, USA. Curran Associates Inc

    2017

  30. [38]

    Nair, V., Schumacher, E., Tso, G., and Kannan, A. (2023). Dera: enhancing large language model completions with dialog-enabled resolving agents. arXiv preprint arXiv:2303.17071

    2023 arXiv

  31. [39]

    M., and Weng, T.-W

    Oikarinen, T., Das, S., Nguyen, L. M., and Weng, T.-W. (2023). Label-free concept bottleneck models. arXiv preprint arXiv:2304.06129

    2023 arXiv

  32. [40]

    P., Ienco, D., and Marcos, D

    Panousis, K. P., Ienco, D., and Marcos, D. (2024). Coarse-to-fine concept bottleneck models. In Globerson, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J., and Zhang, C., editors, Advances in Neural Information Processing Systems , volume 37, pages 105171--10519...

    2024

  33. [41]

    Poeta, E., Ciravegna, G., Pastor, E., Cerquitelli, T., and Baralis, E. (2023). Concept-based explainable artificial intelligence: A survey. CoRR , abs/2312.12936

    2023

  34. [42]

    B., Collins, M., Lipton, Z

    Pruthi, D., Bansal, R., Dhingra, B., Soares, L. B., Collins, M., Lipton, Z. C., Neubig, G., and Cohen, W. W. (2022). Evaluating explanations: How much do explanations from the teacher aid students? Transactions of the Association for Computational Linguistics , 10:359--375

    2022

  35. [43]

    W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al

    Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021). Learning transferable visual models from natural language supervision. In International conference on machine learning , pages 8748--8763. PmLR

    2021

  36. [44]

    Ronneberger, O., Fischer, P., and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention--MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part ...

    2015

  37. [45]

    S., Hughes, M

    Ross, A. S., Hughes, M. C., and Doshi-Velez, F. (2017). Right for the right reasons: Training differentiable models by constraining their explanations. In International Joint Conference on Artificial Intelligence (IJCAI) , pages 2662--2670

    2017

  38. [46]

    Rudin, C. (2019). Stop explaining black box models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence , 1:206--215

    2019

  39. [47]

    W., Hashimoto, T

    Sagawa, S., Koh, P. W., Hashimoto, T. B., and Liang, P. (2019). Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv:1911.08731

    2019 arXiv

  40. [48]

    and Nakamura, K

    Sawada, Y. and Nakamura, K. (2022). Concept bottleneck model with additional unsupervised concepts. IEEE Access , 10:41758--41765

    2022

  41. [49]

    and Vlachos, M

    Schneider, J. and Vlachos, M. (2024). Reflective-net: Learning from explanations. Data Mining and Knowledge Discovery , 38(5):2975--2996

    2024

  42. [50]

    Schramowski, P., Stammer, W., Teso, S., Brugger, A., Herbert, F., Shao, X., Luigs, H., Mahlein, A., and Kersting, K. (2020). Making deep neural networks right for the right scientific reasons by interacting with their explanations. Nature Machine Intelligence , 2(8):476--486

    2020

  43. [51]

    Schuhmann, C., Vencu, R., Beaumont, R., Kaczmarczyk, R., Mullis, C., Katta, A., Coombes, T., Jitsev, J., and Komatsuzaki, A. (2021). Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. arXiv preprint arXiv:2111.02114

    2021 arXiv

  44. [52]

    Schut, L., Toma s ev, N., McGrath, T., Hassabis, D., Paquet, U., and Kim, B. (2025). Bridging the human--ai knowledge gap through concept discovery and transfer in alphazero. Proceedings of the National Academy of Sciences , 122(13)

    2025

  45. [53]

    Shang, C., Zhou, S., Zhang, H., Ni, X., Yang, Y., and Wang, Y. (2024). Incremental residual concept bottleneck models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 11030--11040

    2024

  46. [54]

    Stammer, W., Friedrich, F., Steinmann, D., Brack, M., Shindo, H., and Kersting, K. (2024a). Learning by self-explaining. Transactions on Machine Learning Research

    2024

  47. [55]

    Stammer, W., Schramowski, P., and Kersting, K. (2021). Right for the right concept: Revising neuro-symbolic concepts by interacting with their explanations. In Conference on Computer Vision and Pattern Recognition CVPR , pages 3619--3629

    2021

  48. [56]

    Stammer, W., W\" u st, A., Steinmann, D., and Kersting, K. (2024b). Neural concept binder. In Globerson, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J., and Zhang, C., editors, Advances in Neural Information Processing Systems , volume 37, pages 71792--71830. C...

    2024

  49. [57]

    Steinmann, D., Divo, F., Kraus, M., W \"u st, A., Struppek, L., Friedrich, F., and Kersting, K. (2024). Navigating shortcuts, spurious correlations, and confounders: From origins via detection to mitigation. CoRR , abs/2412.05152

    2024 arXiv

  50. [58]

    W\" a ldchen, S., Sharma, K., Turan, B., Zimmer, M., and Pokutta, S. (2024). Interpretability guarantees with M erlin- A rthur classifiers. In Dasgupta, S., Mandt, S., and Li, Y., editors, Proceedings of The 27th International Conference on Artificial Intelligence and Statisti...

    2024

  51. [59]

    Yang, Y., Panagopoulou, A., Zhou, S., Jin, D., Callison-Burch, C., and Yatskar, M. (2023). Language in a bottle: Language model guided concept bottlenecks for interpretable image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogniti...

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.