Pith. sign in

REVIEW 5 major objections 4 minor 66 references

Model Science: getting serious about verification, explanation and control of AI systems

T0 review · 5 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proposes a new discipline, Model Science, in which the trained model—not the data—is the object of verification, explanation, control, and interface.

desk verdict A useful and well-organized position paper that names existing research streams under 'Model Science' and offers a concrete MEL taxonomy, but it under-specifies what 'the model' is and overclaims novelty. read the letter →

arxiv 2508.20040 v1 pith:QIDVXUML submitted 2025-08-27 cs.AI cs.LG

Przemyslaw Biecek , Wojciech Samek This is my paper
classification cs.AIcs.LG
keywords ModelSciencefoundationmodelsevaluationlevelsAIverificationexplainablealignmenthuman-AIinteractionmodel-centric
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Foundation models are being used in medicine, law, and code generation, yet the paper argues their evaluation still follows a data-centric habit: split a dataset, report a benchmark score. It proposes turning the trained model itself into the object of a new discipline, Model Science, and organizes that discipline around four pillars: verification, explanation, control, and interface. The concrete centerpiece is a five-level Model Evaluation hierarchy (MEL 0–5) that ranges from no evaluation to full adversarial access, and the paper shows through recent failure cases why high-stakes deployment should be pushed to the upper levels. A reader should care because the taxonomy gives regulators, auditors, and developers a common language for saying how rigorously a model has actually been checked.

What carries the argument

The central object is 'the model under investigation'—the trained artifact treated as the constant reference point across changing datasets. The load-bearing mechanism is the Model Evaluation Levels (MEL) taxonomy, a five-rung ladder (MEL 0: no evaluation; MEL 1: training-data evaluation; MEL 2: held-out data from the same distribution; MEL 3: disjoint out-of-time/out-of-region/out-of-domain data; MEL 4: adversarial probing without model access; MEL 5: full adversarial access). The ladder carries the argument by turning verification from a vague appeal to benchmarks into a graded standard, and by showing that most current practice clusters at the low end while high-stakes deployment demands

What would settle it

Send a fixed prompt to a deployed model, record responses and explanations, then repeat after a silent update; if outputs change while the 'same model' remains the stated object, the constant-element premise fails unless each version is treated as a separate object. A concrete disconfirming case would be a high-stakes failure detectable only through inaccessible training-data properties, such as a data-poisoning attack that shifts predictions on a narrow slice the tester never queries.

Watch

Extended reading notes

Core claim

The paper's claim is that the rise of foundation models makes data-centric data science insufficient, and that a new discipline—Model Science—should take the trained model itself as the persistent object of study. It argues that a model can be verified, explained, controlled, and interfaced with, and it makes this concrete through a five-level Model Evaluation hierarchy: from no evaluation (MEL 0), to training-data evaluation (MEL 1), held-out similar data (MEL 2), disjoint out-of-time/out-of-region data (MEL 3), adversarial probing by users without model access (MEL 4), and full adversarial access (MEL 5). The paper reviews evidence that state-of-the-art foundation models pass standard benc

Load-bearing premise

The paper assumes a trained model has a stable identity that persists while it is probed, updated, and redeployed; if models are continually versioned and accessed through APIs, the 'model under investigation' may not be one object.

Editorial extensions

If this is right

  • Verification would be reported as a MEL level rather than just an accuracy score, making the rigor of evaluation comparable across different AI systems.
  • A model that fails under MEL 4 or MEL 5 probing would feed directly into the explanation and control pillars: find the failure, understand it, then patch it via alignment or memory-editing techniques instead of full retraining.
  • Synthetic data would become a legitimate validation resource, since data is a variable in Model Science; generated counterfactuals and adversarial examples count as evidence about the model itself.
  • Interactive interfaces would be judged by whether they calibrate user trust, not by whether they are visually appealing; explanations that increase overconfidence would count as failures.
  • The explanatory study of models would become a form of basic science, with findings such as AlphaZero's chess concepts treated as transferable knowledge rather than engineering artifacts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the MEL ladder implies a public-access argument—a model whose weights are secret can never be evaluated at MEL 5, so frontier providers would have to offer reproducible access for full verification.
  • Editorial extension: one could make the taxonomy operational by scoring existing model cards and deployment reports on the MEL scale; a survey that found almost all high-stakes deployments at MEL 0–2 would quantify the gap the paper describes.
  • Editorial extension: if models are the objects of study, then explanation techniques function as measurement instruments, and their own reliability should be characterized—an analogy to metrology that the paper does not develop.
  • Editorial extension: treating 'the model' as constant is most plausible when each version is pinned; a practical test of the discipline's feasibility is whether reproducibility of explanation results survives model-version updates.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. This paper argues that the growing deployment of foundation models motivates a new analytic discipline, 'Model Science,' centered on the trained model rather than on data. It proposes four pillars—Verification, Explanation, Control, and Interface—and a five-level Model Evaluation Levels (MEL 0-5) taxonomy for grading evaluation rigor. Each pillar is illustrated with a review of recent work: auditing studies (Whisper hallucinations, legal hallucinations, Copilot security), XAI methods for LLMs and vision models, alignment methods (RLHF, DPO, ROME), and interactive/visual explanation tools. The paper's contribution is a conceptual framework and an invitation to formalize model-centered analysis; it does not introduce new algorithms or formal results.

Significance. The framework is timely and the literature synthesis is broad. The strongest concrete element is the MEL taxonomy, which gives practitioners a compact vocabulary for stating how a model was evaluated. The four-pillar decomposition is plausible but unproven as a complete taxonomy. The paper is honest about the provisional nature of verification findings and cites recent empirical evidence. It ships no code, proofs, or falsifiable predictions; its value is organizational and pedagogical. If the framework is adopted, it could shape research agendas and audit standards. However, the central definitions need to be tightened before the framework can be a rigorous foundation.

major comments (5)
  1. [Sections 1, 3, 7] Section 1 says 'The constant element is the model under investigation,' and the MEL taxonomy and the four pillars treat this as a fixed object. But Section 3 concedes that 'newer versions of these models will fix (some of) these errors over time, and new errors will appear in newer versions,' and Section 7 describes models 'continuously trained on diverse datasets, which are often no longer accessible afterwards.' For deployed foundation models the same name or API endpoint often corresponds to different weight snapshots, adapters, or versions. Without a model-identity criterion (weights, behavior, versioned artifact), a MEL 3-5 result is a certificate with no stable referent: it does not transfer across updates. Define what 'the model' means and how results accumulate across versions.
  2. [Model Evaluation Levels, Section 3] The MEL hierarchy mixes at least two orthogonal dimensions. Levels 1-3 are defined by data provenance relative to training (training data, random split, disjoint time/region); Levels 4-5 are defined by adversary knowledge/access (black-box vs full access). These are not naturally ordered: a MEL 5 audit on in-distribution data with full gradient access is not automatically more stringent than a MEL 3 out-of-time evaluation, and a black-box MEL 4 adversarial search may be more relevant for a particular threat model than MEL 5. The paper should state the ordering criterion or present the levels as a matrix, because Section 3 urges readers to move to 'higher MEL.'
  3. [Section 2] Section 2 asserts that the 'primary interests' of model analysis are verification, exploration, control, and communication, but gives no argument that these four are necessary, sufficient, or mutually exclusive. For instance, privacy, robustness, and monitoring are folded into the pillars without discussion, and the relationship between Control (Section 5) and Verification is left implicit. The paper should either present the pillars as an initial organization of existing work rather than a complete taxonomy, or provide explicit criteria for identifying pillars and demonstrate that the categories are not just relabelings of XAI, alignment, and HCI.
  4. [Sections 2 and 5] Section 2 states that Model Science concerns what can be done 'after the model has been developed,' yet Section 5's core examples (InstructGPT/RLHF, Constitutional AI, DPO, ROME/MEMIT, robust fine-tuning) are weight-modifying interventions. The text says control should happen 'preferably without complete retraining,' but the cited methods are training-time or fine-tuning methods. Specify whether Control covers only inference-time interventions on a fixed artifact, or includes model revision; if the latter, reconcile it with the 'already trained' scope.
  5. [Section 4] Section 4 presents Explanation as 'various approaches to explore of internal model operations,' but does not define a validity criterion for explanations, nor state how explanation claims are themselves verified. This matters because Section 3 cites work on adversarial attacks on explanations; the reader cannot tell whether explanation methods are objects of Model Science (to be verified) or tools of Model Science (to be used for verification). Clarify the status of explanation methods relative to the Verification pillar.
minor comments (4)
  1. [Throughout] Numerous typos and grammar issues: 'explore of internal model operations' (Abstract), 'Two landmark works shaped ... are' (Section 1), 'Tukey argue' (Section 1), 'frameworks is' (Section 3), 'stody' and 'colled'/'adress'/'may be use'/'fromjustifying' (Sections 3-4). Please copyedit.
  2. [References] Reference formatting is inconsistent: 'et. al.' appears with varying spacing and punctuation, and some entries have a stray space before the period (e.g., 'Bai, Y .'). The caption of Figure 4 appears to contain an unnumbered citation to Biecek & Burzykowski.
  3. [Section 3] The sentence 'This revision is intended to emphasize...' is unclear in a standalone paper; either delete or rephrase.
  4. [Throughout] The paper's evidence for the 'emerging discipline' includes a large share of citations to the authors' own groups; independent examples would strengthen the claim that this is a field-level trend rather than a local research program.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: a position paper whose self-citations are illustrative, not load-bearing.

full rationale

This paper is a conceptual/position proposal, not a derivation chain. It introduces a framework—four pillars (Verification, Explanation, Control, Interface) and a five-level Model Evaluation Levels (MEL) taxonomy—and surveys existing work under those headings. It does not claim to derive a predictive result from first principles, so there is no fitted-input-called-prediction and no equation-level self-definitional reduction. The self-citations (e.g., Biecek & Samek 2024 for RED/BLUE-XAI; Baniecki & Biecek 2024/2025 for adversarial XAI; Baniecki et al. 2023 for IEMA) are used as examples of ongoing research in the proposed pillars, not as load-bearing justification for the framework's validity. The central claim—that AI research should adopt a model-centric discipline—is a normative proposal that stands independently of these citations. The MEL taxonomy organizes existing evaluation practices (out-of-time validation, out-of-region validation, adversarial access) into a hierarchy; this is a conceptual contribution, not a renaming of a known result presented as a derivation. The paper does admit a genuine limitation: Section 1 asserts 'The constant element is the model under investigation,' while Sections 3 and 7 acknowledge that models change over time. This is a substantive assumption gap about model identity, but it is a weakness of the proposal, not circular reasoning. No circular step can be exhibited by quoting specific reductions to inputs, so the circularity score is low.

Assumptions & free parameters 0 free parameters · 3 assumptions · 2 invented entities

The framework rests on the assumption that a trained model is a stable object of study, and that the four pillars are the right organizing categories. Neither is derived from external evidence. 'Model Science' and 'Model Evaluation Levels' are newly named constructs with no independent handle.

assumptions (3)
  • domain assumption A trained model is a stable, persistent object that can be analyzed across operational contexts.
    Section 1: 'The constant element is the model under investigation.' This justifies the entire model-centric framing; if models are ephemeral or constantly updated, the discipline lacks a fixed object.
  • ad hoc to paper The four pillars (Verification, Explanation, Control, Interface) are the central areas of model analysis.
    Introduced in Section 2 without derivation, comparison to alternative taxonomies, or empirical justification for completeness.
  • ad hoc to paper The five Model Evaluation Levels form a useful and sufficient hierarchy for describing evaluation rigor.
    Defined in Section 3 as a proposal; not validated against existing evaluation standards or audited by practitioners.
invented entities (2)
  • Model Science as a discipline
    purpose: To name and legitimize a research agenda centered on trained models rather than data.
    A proposed framing with no external measurement or falsifiable handle; its existence depends on community adoption.
  • Model Evaluation Levels (MEL 0-5)
    purpose: Taxonomy for the depth and rigor of model validation.
    A novel classification introduced in Section 3; not derived from data and not shown to be more effective than existing validation standards.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Model Science: getting serious about verification, explanation and control of AI systems." pith.science (2026). https://pith.science/paper/QIDVXUML

@misc{pith2026250820040,
  author       = {Pith},
  title        = {Pith review of: Model Science: getting serious about verification, explanation and control of AI systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QIDVXUML}},
  note         = {Machine review of arXiv:2508.20040}
}
read the original abstract

The growing adoption of foundation models calls for a paradigm shift from Data Science to Model Science. Unlike data-centric approaches, Model Science places the trained model at the core of analysis, aiming to interact, verify, explain, and control its behavior across diverse operational contexts. This paper introduces a conceptual framework for a new discipline called Model Science, along with the proposal for its four key pillars: Verification, which requires strict, context-aware evaluation protocols; Explanation, which is understood as various approaches to explore of internal model operations; Control, which integrates alignment techniques to steer model behavior; and Interface, which develops interactive and visual explanation tools to improve human calibration and decision-making. The proposed framework aims to guide the development of credible, safe, and human-aligned AI systems.

Figures

Figures reproduced from arXiv: 2508.20040 by the authors.

Figure 1
Figure 1. Data Science and Model Science cover similar modeling areas but emphasize complementary perspectives. Adopting a triadic view, where to understand a phenomenon (World) we collect Data, build a Model based on it, and confront the model with World again, Data Science focuses primarily on data. Around a given resource—typically a single well-defined dataset—many models are created to fit the data. Central to this persp… view at source ↗
Figure 2
Figure 2. graphically summarizes the four main pillars of model analysis, which will be discussed in greater detail in the following sections. 3 Verification: does it work? We will begin this section by discussing recently discovered weak￾nesses in the most popular AI models currently available. Certainly, newer versions of these models will fix (some of) these errors over time, and new errors will appear in newer versions of… view at source ↗
Figure 3
Figure 3. Graphical summary of the five levels of model evaluation Model Evaluation Level 0 — no explicit model evaluation. It also includes the use of models for a new task for which they were not designed (unintended use) without any verification. Model Evaluation Level 1 — for model evaluation we use the same data on which we trained the model. This is a common prac￾tice in statistical modelling based on linear models or o… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

66 extracted references · 50 canonical work pages

  1. [1]

    Bai, Y. and et. al. (2022). Constitutional AI : Harmlessness from AI Feedback . arXiv. 10.48550/arXiv.2212.08073

    arXiv doi:10.48550/arxiv.2212.08073 2022

  2. [2]

    and Biecek, P

    Baniecki, H. and Biecek, P. (2024). Adversarial attacks and defenses in explainable artificial intelligence: A survey. Information Fusion , 107:102303

    work page 2024

  3. [3]

    and Biecek, P

    Baniecki, H. and Biecek, P. (2025). Birds look like cars: Adversarial analysis of intrinsically interpretable deep learning . arXiv. 10.48550/arXiv.2503.08636

    work page doi:10.48550/arxiv.2503.08636 2025

  4. [4]

    Baniecki, H., Parzych, D., and Biecek, P. (2023). The grammar of interactive explanatory model analysis. Data Mining and Knowledge Discovery

    work page 2023

  5. [5]

    T., and Weld, D

    Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., and Weld, D. (2021). Does the Whole Exceed its Parts ? The Effect of AI Explanations on Complementary Team Performance . In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems , pages 1--16. ACM

    work page 2021

  6. [6]

    P., and Lakkaraju, H

    Bhalla, U., Oesterling, A., Srinivas, S., Calmon, F. P., and Lakkaraju, H. (2024). Interpreting CLIP with Sparse Linear Concept Embeddings ( SpLiCE ) . arXiv. 10.48550/arXiv.2402.10376

    arXiv doi:10.48550/arxiv.2402.10376 2024

  7. [7]

    and Burzykowski, T

    Biecek, P. and Burzykowski, T. (2021). Explanatory Model Analysis : Explore , Explain and Examine Predictive Models . Chapman and Hall/CRC, 1 edition

    work page 2021

  8. [8]

    and Samek, W

    Biecek, P. and Samek, W. (2024). Position: Explain to question not to justify. volume 235 of Proceedings of Machine Learning Research , pages 3996--4006. PMLR

    work page 2024

Show all 66 references
  1. [9]

    Bommasani, R. and et. al. (2022). On the Opportunities and Risks of Foundation Models . arXiv. 10.48550/arXiv.2108.07258

    2022 doi arXiv

  2. [10]

    Bücker, M., Szepannek, G., Gosiewska, A., and Biecek, P. (2022). Transparency, auditability, and explainability of machine learning models in credit scoring. Journal of the Operational Research Society , 73(1):70--90

    2022

  3. [11]

    and Korolova, A

    Castleman, J. and Korolova, A. (2025). Adultification Bias in LLMs and Text -to- Image Models . In Proceedings of the 2025 ACM Conference on Fairness , Accountability , and Transparency , pages 2751--2767. ACM

    2025

  4. [12]

    and Liu, S

    Choo, J. and Liu, S. (2018). Visual Analytics for Explainable Deep Learning . arXiv. 10.48550/arXiv.1804.02527

    2018 doi arXiv

  5. [13]

    Cleveland, W. S. (2001). Data science: An action plan for expanding the technical areas of the field of statistics. International Statistical Review , 69(1):21--26

    2001

  6. [14]

    Dahl, M., Magesh, V., Suzgun, M., and Ho, D. E. (2024). Large legal fictions: Profiling legal hallucinations in large language models. Journal of Legal Analysis , 16(1):64--93

    2024

  7. [15]

    Delfosse, Q., Shindo, H., Dhami, D., and Kersting, K. (2023). Interpretable and Explainable Logical Policies via Neurally Guided Symbolic Abstraction . arXiv. 10.48550/arXiv.2306.01439

    2023 doi arXiv

  8. [16]

    F., Wang, J., and Berger, M

    DeRose, J. F., Wang, J., and Berger, M. (2020). Attention Flows : Analyzing and Comparing Attention Mechanisms in Language Models . arXiv. 10.48550/arXiv.2009.07053

    2020 doi arXiv

  9. [17]

    Dreyer, M., Hufe, L., Berend, J., Wiegand, T., Lapuschkin, S., and Samek, W. (2025). From What to How : Attributing CLIP 's Latent Components Reveals Unexpected Semantic Reliance . arXiv. 10.48550/arXiv.2505.20229

    2025 doi arXiv

  10. [18]

    Elena, M., Valentin, M., Rafaela, C., and Codrut, T. (2021). An IFRS 9 Framework for Model Validation . International Journal of Business and Management Invention , 10:2319--8028

    2021

  11. [19]

    A., and Steinhardt, J

    Gandelsman, Y., Efros, A. A., and Steinhardt, J. (2024). Interpreting CLIP 's Image Representation via Text - Based Decomposition . arXiv. 10.48550/arXiv.2310.05916

    2024 doi arXiv

  12. [20]

    Ganguli, D. and et. al. (2022). Red Teaming Language Models to Reduce Harms . arXiv. 10.48550/arXiv.2209.07858

    2022 doi arXiv

  13. [21]

    Gould, R., Ong, E., Ogden, G., and Conmy, A. (2023). Successor Heads : Recurring , Interpretable Attention Heads In The Wild . arXiv. 10.48550/arXiv.2312.09230

    2023 doi arXiv

  14. [22]

    Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., and Pedreschi, D. (2019). A Survey of Methods for Explaining Black Box Models . ACM Computing Surveys , 51(5):1--42

    2019

  15. [23]

    Y., Phua, K., Wong, L., and Bin Goh, W

    Ho, S. Y., Phua, K., Wong, L., and Bin Goh, W. W. (2020). Extensions of the External Validation for Checking Learned Model Interpretability and Generalizability . Patterns , 1(8):100129

    2020

  16. [24]

    Holzinger, A., Saranti, A., Molnar, C., Biecek, P., and Samek, W. (2022). Explainable AI Methods - A Brief Overview , pages 13--38. Springer International Publishing, Cham

    2022

  17. [25]

    Hryniewska, W., Bombiński, P., Szatkowski, P., Tomaszewska, P., Przelaskowski, A., and Biecek, P. (2021). Checklist for responsible deep learning modeling of medical images based on COVID -19 detection studies. Pattern Recognition , 118:108035

    2021

  18. [26]

    Hryniewska, W., Grudzień, A., and Biecek, P. (2022). LIMEcraft : handcrafted superpixel selection and inspection for Visual eXplanations . Machine Learning

    2022

  19. [27]

    Kauffmann, J., Dippel, J., Ruff, L., Samek, W., Müller, K.-R., and Montavon, G. (2025). Explainable AI reveals Clever Hans effects in unsupervised learning models. Nature Machine Intelligence , 7(3):412--422

    2025

  20. [28]

    Kaur, H., Nori, H., Jenkins, S., Caruana, R., Wallach, H., and Wortman Vaughan, J. (2020). Interpreting Interpretability : Understanding Data Scientists ' Use of Interpretability Tools for Machine Learning . In Proceedings of the 2020 CHI Conference on Human Factors in Computi...

    2020

  21. [29]

    D., Biecek, P., Russell, S., and Bengio, Y

    Knott, A., Pedreschi, D., Chatila, R., Chakraborti, T., Leavy, S., Baeza-Yates, R., Eyers, D., Trotman, A., Teal, P. D., Biecek, P., Russell, S., and Bengio, Y. (2023). Generative AI models should include detection mechanisms as a condition for public release. Ethics and Infor...

    2023

  22. [30]

    D., Basu, S., Haklidir, M., Morini, V., Russell, S., and Bengio, Y

    Knott, A., Pedreschi, D., Jitsuzumi, T., Leavy, S., Eyers, D., Chakraborti, T., Trotman, A., Sundareswaran, S., Baeza-Yates, R., Biecek, P., Weller, A., Teal, P. D., Basu, S., Haklidir, M., Morini, V., Russell, S., and Bengio, Y. (2024). AI content detection in the emerging in...

    2024

  23. [31]

    Koenecke, A., Choi, A. S. G., Mei, K. X., Schellmann, H., and Sloane, M. (2024). Careless Whisper : Speech -to- Text Hallucination Harms . In The 2024 ACM Conference on Fairness , Accountability , and Transparency , pages 1672--1681. ACM

    2024

  24. [32]

    Kulesza, T., Burnett, M., Wong, W.-K., and Stumpf, S. (2015). Principles of Explanatory Debugging to Personalize Interactive Machine Learning . In Proceedings of the 20th International Conference on Intelligent User Interfaces , pages 126--137. ACM

    2015

  25. [33]

    and Biecek, P

    Kuzba, M. and Biecek, P. (2020). What Would You Ask the Machine Learning Model ? Identification of User Needs for Model Explanations Based on Human - Model Conversations . In Communications in Computer and Information Science , pages 447--459

    2020

  26. [34]

    Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., and Müller, K.-R. (2019). Unmasking Clever Hans predictors and assessing what machines really learn. Nature Communications , 10(1)

    2019

  27. [35]

    Lee, H., Phatale, S., Mansoor, H., Mesnard, T., Ferret, J., Lu, K., Bishop, C., Hall, E., Carbune, V., Rastogi, A., and Prakash, S. (2024). RLAIF vs. RLHF : Scaling Reinforcement Learning from Human Feedback with AI Feedback . arXiv. 10.48550/arXiv.2309.00267

    2024 doi arXiv

  28. [36]

    M., Zheng, Y., Zhang, W., and Ma, K.-L

    Li, Y., Wang, J., Dai, X., Wang, L., Yeh, C.-C. M., Zheng, Y., Zhang, W., and Ma, K.-L. (2023). How Does Attention Work in Vision Transformers ? A Visual Analytics Attempt . IEEE Transactions on Visualization and Computer Graphics , 29(6):2888--2900

    2023

  29. [37]

    K., and Metzen, J

    Lovisotto, G., Finnie, N., Munoz, M., Mummadi, C. K., and Metzen, J. H. (2022). Give Me Your Attention : Dot - Product Attention Considered Harmful for Adversarial Patch Robustness . arXiv. 10.48550/arXiv.2203.13639

    2022 doi arXiv

  30. [38]

    Lupo, U., Sgarbossa, D., and Bitbol, A.-F. (2022). Protein language models trained on multiple sequence alignments learn phylogenetic relationships. Nature Communications , 13(1)

    2022

  31. [39]

    J., Rasheed, S., Moradidakhel, A., Tahir, A., and Khomh, F

    Majdinasab, V., Bishop, M. J., Rasheed, S., Moradidakhel, A., Tahir, A., and Khomh, F. (2024). Assessing the Security of GitHub Copilot 's Generated Code - A Targeted Replication Study . In 2024 IEEE International Conference on Software Analysis , Evolution and Reengineering (...

    2024

  32. [40]

    McGrath, T., Kapishnikov, A., Tomašev, N., Pearce, A., Wattenberg, M., Hassabis, D., Kim, B., Paquet, U., and Kramnik, V. (2022). Acquisition of chess knowledge in AlphaZero . Proceedings of the National Academy of Sciences , 119(47)

    2022

  33. [41]

    Meng, K., Bau, D., Andonian, A., and Belinkov, Y. (2023a). Locating and Editing Factual Associations in GPT . arXiv. 10.48550/arXiv.2202.05262

    doi arXiv

  34. [42]

    S., Andonian, A., Belinkov, Y., and Bau, D

    Meng, K., Sharma, A. S., Andonian, A., Belinkov, Y., and Bau, D. (2023b). Mass- Editing Memory in a Transformer . arXiv. 10.48550/arXiv.2210.07229

    doi arXiv

  35. [43]

    Mieleszczenko-Kowszewicz, W., Płudowski, D., Kołodziejczyk, F., Świstak, J., Sienkiewicz, J., and Biecek, P. (2024). The Dark Patterns of Personalized Persuasion in Large Language Models . arXiv. 10.48550/arXiv.2411.06008

    2024 doi arXiv

  36. [44]

    Miller, T. (2019). Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence , 267:1--38

    2019

  37. [45]

    Nam, A., Conklin, H., Yang, Y., Griffiths, T., Cohen, J., and Leslie, S.-J. (2025). Causal Head Gating : A Framework for Interpreting Roles of Attention Heads in Transformers . arXiv. 10.48550/arXiv.2505.13737

    2025 doi

  38. [46]

    Ngo, A., Gupta, S., Perrine, O., Reddy, R., Ershadi, S., and Remick, D. (2024). ChatGPT 3.5 fails to write appropriate multiple choice practice exam questions. Academic Pathology , 11(1):100099

    2024

  39. [47]

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., and Lowe, R. (2022). Training language mod...

    2022

  40. [48]

    and Björnsson, Y

    Pálsson, A. and Björnsson, Y. (2023). Unveiling Concepts Learned by a World - Class Chess - Playing Agent . In Proceedings of the Thirty - Second International Joint Conference on Artificial Intelligence , pages 4864--4872

    2023

  41. [49]

    V., Guidotti, R., and Biecek, P

    Płudowski, D., Spinnato, F., Wilczyński, P., Kotowski, K., Ntagiou, E. V., Guidotti, R., and Biecek, P. (2025). MASCOTS : Model - Agnostic Symbolic COunterfactual explanations for Time Series . arXiv. 10.48550/arXiv.2503.22389

    2025 doi arXiv

  42. [50]

    D., and Finn, C

    Rafailov, R., Sharma, A., Mitchell, E., Ermon, S., Manning, C. D., and Finn, C. (2024). Direct Preference Optimization : Your Language Model is Secretly a Reward Model . arXiv. 10.48550/arXiv.2305.18290

    2024 doi arXiv

  43. [51]

    Why Should I Trust You ?

    Ribeiro, M. T., Singh, S., and Guestrin, C. (2016). " Why Should I Trust You ?": Explaining the Predictions of Any Classifier . In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pages 1135--1144. ACM

    2016

  44. [52]

    R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D

    Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017). Grad- CAM : Visual Explanations from Deep Networks via Gradient - Based Localization . In 2017 IEEE International Conference on Computer Vision ( ICCV ) , pages 618--626. IEEE

    2017

  45. [53]

    and Biecek, P

    Sobieski, B. and Biecek, P. (2025). Global Counterfactual Directions . In Lecture Notes in Computer Science , pages 72--90. Springer Nature

    2025

  46. [54]

    Sobieski, B., Grzywaczewski, J., Sadlej, B., Tivnan, M., and Biecek, P. (2025). Rethinking visual counterfactual explanations through region constraint. In The Thirteenth International Conference on Learning Representations

    2025

  47. [55]

    Spyrison, N., Cook, D., and Biecek, P. (2025). Exploring local explanations of nonlinear models using animated linear projections. Computational Statistics , 40(2):1071--1095

    2025

  48. [56]

    Strobelt, H., Webson, A., Sanh, V., Hoover, B., Beyer, J., Pfister, H., and Rush, A. M. (2022). Interactive and Visual Prompt Engineering for Ad -hoc Task Adaptation With Large Language Models . IEEE Transactions on Visualization and Computer Graphics , pages 1--11

    2022

  49. [57]

    and Zhang, Y

    Tan, J. and Zhang, Y. (2023). ExplainableFold : Understanding AlphaFold Prediction with Explainable AI . arXiv. 10.48550/arXiv.2301.11765

    2023 doi arXiv

  50. [58]

    Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, A., Bhosale, S., et al. (2023). LLaMA 2: Open Foundation and Fine-Tuned Chat Models . 10.48550/arXiv.2307.09288

    2023 doi arXiv

  51. [59]

    Tukey, J. W. (1962). The future of data analysis. The Annals of Mathematical Statistics , 33(1):1--67

    1962

  52. [60]

    Wang, L., Wen, Z., Liu, S.-W., Zhang, L., Finley, C., Lee, H.-J., and Fan, H.-J. S. (2024). Overview of AlphaFold2 and breakthroughs in overcoming its limitations. Computers in Biology and Medicine , 176:108620

    2024

  53. [61]

    Wu, T., Jiang, E., Donsbach, A., Gray, J., Molina, A., Terry, M., and Cai, C. J. (2022). PromptChainer : Chaining Large Language Model Prompts through Visual Programming . In CHI Conference on Human Factors in Computing Systems Extended Abstracts , pages 1--10. ACM

    2022

  54. [62]

    S., Heer, J., and Ribeiro, M

    Wu, T., Shen, H., Weld, D. S., Heer, J., and Ribeiro, M. T. (2023). ScatterShot : Interactive In -context Example Curation for Text Transformation . pages 353--367

    2023

  55. [63]

    Yeh, C., Chen, Y., Wu, A., Chen, C., Viégas, F., and Wattenberg, M. (2023). AttentionViz : A Global View of Transformer Attention . arXiv. 10.48550/arXiv.2305.03210

    2023 doi arXiv

  56. [64]

    Zack, T. and et. al. (2024). Assessing the potential of GPT -4 to perpetuate racial and gender biases in health care: a model evaluation study. The Lancet Digital Health , 6(1):e12--e22

    2024

  57. [65]

    Zaigrajew, V., Baniecki, H., and Biecek, P. (2025). Interpreting CLIP with Hierarchical Sparse Autoencoders . arXiv. 10.48550/arXiv.2502.20578

    2025 doi arXiv

  58. [66]

    Zhou, Z., Hu, S., Li, M., Zhang, H., Zhang, Y., and Jin, H. (2023). AdvCLIP : Downstream -agnostic Adversarial Examples in Multimodal Contrastive Learning . arXiv. 10.48550/arXiv.2308.07026

    2023 doi arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.