Pith. sign in

REVIEW 5 major objections 4 minor 66 references

The paper proposes a new discipline, Model Science, in which the trained model—not the data—is the object of verification, explanation, control, and interface.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Proposes 'Model Science' as a model-centric paradigm for AI with four pillars: verification, explanation, control, and interface.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A useful and well-organized position paper that names existing research streams under 'Model Science' and offers a concrete MEL taxonomy, but it under-specifies what 'the model' is and overclaims novelty. the 5 major comments →

arxiv 2508.20040 v1 pith:QIDVXUML submitted 2025-08-27 cs.AI cs.LG

Model Science: getting serious about verification, explanation and control of AI systems

Przemyslaw Biecek , Wojciech Samek This is my paper
classification cs.AI cs.LG
keywords Model Sciencefoundation modelsmodel evaluation levelsAI verificationexplainable AIAI alignmenthuman-AI interactionmodel-centric AI
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Foundation models are being used in medicine, law, and code generation, yet the paper argues their evaluation still follows a data-centric habit: split a dataset, report a benchmark score. It proposes turning the trained model itself into the object of a new discipline, Model Science, and organizes that discipline around four pillars: verification, explanation, control, and interface. The concrete centerpiece is a five-level Model Evaluation hierarchy (MEL 0–5) that ranges from no evaluation to full adversarial access, and the paper shows through recent failure cases why high-stakes deployment should be pushed to the upper levels. A reader should care because the taxonomy gives regulators, auditors, and developers a common language for saying how rigorously a model has actually been checked.

Core claim

The paper's claim is that the rise of foundation models makes data-centric data science insufficient, and that a new discipline—Model Science—should take the trained model itself as the persistent object of study. It argues that a model can be verified, explained, controlled, and interfaced with, and it makes this concrete through a five-level Model Evaluation hierarchy: from no evaluation (MEL 0), to training-data evaluation (MEL 1), held-out similar data (MEL 2), disjoint out-of-time/out-of-region data (MEL 3), adversarial probing by users without model access (MEL 4), and full adversarial access (MEL 5). The paper reviews evidence that state-of-the-art foundation models pass standard benc

What carries the argument

The central object is 'the model under investigation'—the trained artifact treated as the constant reference point across changing datasets. The load-bearing mechanism is the Model Evaluation Levels (MEL) taxonomy, a five-rung ladder (MEL 0: no evaluation; MEL 1: training-data evaluation; MEL 2: held-out data from the same distribution; MEL 3: disjoint out-of-time/out-of-region/out-of-domain data; MEL 4: adversarial probing without model access; MEL 5: full adversarial access). The ladder carries the argument by turning verification from a vague appeal to benchmarks into a graded standard, and by showing that most current practice clusters at the low end while high-stakes deployment demands

Load-bearing premise

The paper assumes a trained model has a stable identity that persists while it is probed, updated, and redeployed; if models are continually versioned and accessed through APIs, the 'model under investigation' may not be one object.

What would settle it

Send a fixed prompt to a deployed model, record responses and explanations, then repeat after a silent update; if outputs change while the 'same model' remains the stated object, the constant-element premise fails unless each version is treated as a separate object. A concrete disconfirming case would be a high-stakes failure detectable only through inaccessible training-data properties, such as a data-poisoning attack that shifts predictions on a narrow slice the tester never queries.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Verification would be reported as a MEL level rather than just an accuracy score, making the rigor of evaluation comparable across different AI systems.
  • A model that fails under MEL 4 or MEL 5 probing would feed directly into the explanation and control pillars: find the failure, understand it, then patch it via alignment or memory-editing techniques instead of full retraining.
  • Synthetic data would become a legitimate validation resource, since data is a variable in Model Science; generated counterfactuals and adversarial examples count as evidence about the model itself.
  • Interactive interfaces would be judged by whether they calibrate user trust, not by whether they are visually appealing; explanations that increase overconfidence would count as failures.
  • The explanatory study of models would become a form of basic science, with findings such as AlphaZero's chess concepts treated as transferable knowledge rather than engineering artifacts.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the MEL ladder implies a public-access argument—a model whose weights are secret can never be evaluated at MEL 5, so frontier providers would have to offer reproducible access for full verification.
  • Editorial extension: one could make the taxonomy operational by scoring existing model cards and deployment reports on the MEL scale; a survey that found almost all high-stakes deployments at MEL 0–2 would quantify the gap the paper describes.
  • Editorial extension: if models are the objects of study, then explanation techniques function as measurement instruments, and their own reliability should be characterized—an analogy to metrology that the paper does not develop.
  • Editorial extension: treating 'the model' as constant is most plausible when each version is pinned; a practical test of the discipline's feasibility is whether reproducibility of explanation results survives model-version updates.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. This paper argues that the growing deployment of foundation models motivates a new analytic discipline, 'Model Science,' centered on the trained model rather than on data. It proposes four pillars—Verification, Explanation, Control, and Interface—and a five-level Model Evaluation Levels (MEL 0-5) taxonomy for grading evaluation rigor. Each pillar is illustrated with a review of recent work: auditing studies (Whisper hallucinations, legal hallucinations, Copilot security), XAI methods for LLMs and vision models, alignment methods (RLHF, DPO, ROME), and interactive/visual explanation tools. The paper's contribution is a conceptual framework and an invitation to formalize model-centered analysis; it does not introduce new algorithms or formal results.

Significance. The framework is timely and the literature synthesis is broad. The strongest concrete element is the MEL taxonomy, which gives practitioners a compact vocabulary for stating how a model was evaluated. The four-pillar decomposition is plausible but unproven as a complete taxonomy. The paper is honest about the provisional nature of verification findings and cites recent empirical evidence. It ships no code, proofs, or falsifiable predictions; its value is organizational and pedagogical. If the framework is adopted, it could shape research agendas and audit standards. However, the central definitions need to be tightened before the framework can be a rigorous foundation.

major comments (5)
  1. [Sections 1, 3, 7] Section 1 says 'The constant element is the model under investigation,' and the MEL taxonomy and the four pillars treat this as a fixed object. But Section 3 concedes that 'newer versions of these models will fix (some of) these errors over time, and new errors will appear in newer versions,' and Section 7 describes models 'continuously trained on diverse datasets, which are often no longer accessible afterwards.' For deployed foundation models the same name or API endpoint often corresponds to different weight snapshots, adapters, or versions. Without a model-identity criterion (weights, behavior, versioned artifact), a MEL 3-5 result is a certificate with no stable referent: it does not transfer across updates. Define what 'the model' means and how results accumulate across versions.
  2. [Model Evaluation Levels, Section 3] The MEL hierarchy mixes at least two orthogonal dimensions. Levels 1-3 are defined by data provenance relative to training (training data, random split, disjoint time/region); Levels 4-5 are defined by adversary knowledge/access (black-box vs full access). These are not naturally ordered: a MEL 5 audit on in-distribution data with full gradient access is not automatically more stringent than a MEL 3 out-of-time evaluation, and a black-box MEL 4 adversarial search may be more relevant for a particular threat model than MEL 5. The paper should state the ordering criterion or present the levels as a matrix, because Section 3 urges readers to move to 'higher MEL.'
  3. [Section 2] Section 2 asserts that the 'primary interests' of model analysis are verification, exploration, control, and communication, but gives no argument that these four are necessary, sufficient, or mutually exclusive. For instance, privacy, robustness, and monitoring are folded into the pillars without discussion, and the relationship between Control (Section 5) and Verification is left implicit. The paper should either present the pillars as an initial organization of existing work rather than a complete taxonomy, or provide explicit criteria for identifying pillars and demonstrate that the categories are not just relabelings of XAI, alignment, and HCI.
  4. [Sections 2 and 5] Section 2 states that Model Science concerns what can be done 'after the model has been developed,' yet Section 5's core examples (InstructGPT/RLHF, Constitutional AI, DPO, ROME/MEMIT, robust fine-tuning) are weight-modifying interventions. The text says control should happen 'preferably without complete retraining,' but the cited methods are training-time or fine-tuning methods. Specify whether Control covers only inference-time interventions on a fixed artifact, or includes model revision; if the latter, reconcile it with the 'already trained' scope.
  5. [Section 4] Section 4 presents Explanation as 'various approaches to explore of internal model operations,' but does not define a validity criterion for explanations, nor state how explanation claims are themselves verified. This matters because Section 3 cites work on adversarial attacks on explanations; the reader cannot tell whether explanation methods are objects of Model Science (to be verified) or tools of Model Science (to be used for verification). Clarify the status of explanation methods relative to the Verification pillar.
minor comments (4)
  1. [Throughout] Numerous typos and grammar issues: 'explore of internal model operations' (Abstract), 'Two landmark works shaped ... are' (Section 1), 'Tukey argue' (Section 1), 'frameworks is' (Section 3), 'stody' and 'colled'/'adress'/'may be use'/'fromjustifying' (Sections 3-4). Please copyedit.
  2. [References] Reference formatting is inconsistent: 'et. al.' appears with varying spacing and punctuation, and some entries have a stray space before the period (e.g., 'Bai, Y .'). The caption of Figure 4 appears to contain an unnumbered citation to Biecek & Burzykowski.
  3. [Section 3] The sentence 'This revision is intended to emphasize...' is unclear in a standalone paper; either delete or rephrase.
  4. [Throughout] The paper's evidence for the 'emerging discipline' includes a large share of citations to the authors' own groups; independent examples would strengthen the claim that this is a field-level trend rather than a local research program.

Circularity Check

0 steps flagged

No significant circularity: a position paper whose self-citations are illustrative, not load-bearing.

full rationale

This paper is a conceptual/position proposal, not a derivation chain. It introduces a framework—four pillars (Verification, Explanation, Control, Interface) and a five-level Model Evaluation Levels (MEL) taxonomy—and surveys existing work under those headings. It does not claim to derive a predictive result from first principles, so there is no fitted-input-called-prediction and no equation-level self-definitional reduction. The self-citations (e.g., Biecek & Samek 2024 for RED/BLUE-XAI; Baniecki & Biecek 2024/2025 for adversarial XAI; Baniecki et al. 2023 for IEMA) are used as examples of ongoing research in the proposed pillars, not as load-bearing justification for the framework's validity. The central claim—that AI research should adopt a model-centric discipline—is a normative proposal that stands independently of these citations. The MEL taxonomy organizes existing evaluation practices (out-of-time validation, out-of-region validation, adversarial access) into a hierarchy; this is a conceptual contribution, not a renaming of a known result presented as a derivation. The paper does admit a genuine limitation: Section 1 asserts 'The constant element is the model under investigation,' while Sections 3 and 7 acknowledge that models change over time. This is a substantive assumption gap about model identity, but it is a weakness of the proposal, not circular reasoning. No circular step can be exhibited by quoting specific reductions to inputs, so the circularity score is low.

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 2 invented entities

The framework rests on the assumption that a trained model is a stable object of study, and that the four pillars are the right organizing categories. Neither is derived from external evidence. 'Model Science' and 'Model Evaluation Levels' are newly named constructs with no independent handle.

axioms (3)
  • domain assumption A trained model is a stable, persistent object that can be analyzed across operational contexts.
    Section 1: 'The constant element is the model under investigation.' This justifies the entire model-centric framing; if models are ephemeral or constantly updated, the discipline lacks a fixed object.
  • ad hoc to paper The four pillars (Verification, Explanation, Control, Interface) are the central areas of model analysis.
    Introduced in Section 2 without derivation, comparison to alternative taxonomies, or empirical justification for completeness.
  • ad hoc to paper The five Model Evaluation Levels form a useful and sufficient hierarchy for describing evaluation rigor.
    Defined in Section 3 as a proposal; not validated against existing evaluation standards or audited by practitioners.
invented entities (2)
  • Model Science as a discipline no independent evidence
    purpose: To name and legitimize a research agenda centered on trained models rather than data.
    A proposed framing with no external measurement or falsifiable handle; its existence depends on community adoption.
  • Model Evaluation Levels (MEL 0-5) no independent evidence
    purpose: Taxonomy for the depth and rigor of model validation.
    A novel classification introduced in Section 3; not derived from data and not shown to be more effective than existing validation standards.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Model Science: getting serious about verification, explanation and control of AI systems." pith.science (2026). https://pith.science/paper/QIDVXUML

@misc{pith2026250820040,
  author       = {Pith},
  title        = {Pith review of: Model Science: getting serious about verification, explanation and control of AI systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QIDVXUML}},
  note         = {Machine review of arXiv:2508.20040}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

The growing adoption of foundation models calls for a paradigm shift from Data Science to Model Science. Unlike data-centric approaches, Model Science places the trained model at the core of analysis, aiming to interact, verify, explain, and control its behavior across diverse operational contexts. This paper introduces a conceptual framework for a new discipline called Model Science, along with the proposal for its four key pillars: Verification, which requires strict, context-aware evaluation protocols; Explanation, which is understood as various approaches to explore of internal model operations; Control, which integrates alignment techniques to steer model behavior; and Interface, which develops interactive and visual explanation tools to improve human calibration and decision-making. The proposed framework aims to guide the development of credible, safe, and human-aligned AI systems.

Figures

Figures reproduced from arXiv: 2508.20040 by Przemyslaw Biecek, Wojciech Samek.

Figure 1
Figure 1. Figure 1: Data Science and Model Science cover similar modeling areas but emphasize complementary perspectives. Adopting a triadic view, where to understand a phenomenon (World) we collect Data, build a Model based on it, and confront the model with World again, Data Science focuses primarily on data. Around a given resource—typically a single well-defined dataset—many models are created to fit the data. Central to … view at source ↗
Figure 2
Figure 2. Figure 2: graphically summarizes the four main pillars of model analysis, which will be discussed in greater detail in the following sections. 3 Verification: does it work? We will begin this section by discussing recently discovered weak￾nesses in the most popular AI models currently available. Certainly, newer versions of these models will fix (some of) these errors over time, and new errors will appear in newer v… view at source ↗
Figure 3
Figure 3. Figure 3: Graphical summary of the five levels of model evaluation Model Evaluation Level 0 — no explicit model evaluation. It also includes the use of models for a new task for which they were not designed (unintended use) without any verification. Model Evaluation Level 1 — for model evaluation we use the same data on which we trained the model. This is a common prac￾tice in statistical modelling based on linear m… view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

66 extracted references · 50 canonical work pages · 6 internal anchors

  1. [1]

    Bai, Y. and et. al. (2022). Constitutional AI : Harmlessness from AI Feedback . arXiv. 10.48550/arXiv.2212.08073

    Pith/arXiv arXiv doi:10.48550/arxiv.2212.08073 2022

  2. [2]

    and Biecek, P

    Baniecki, H. and Biecek, P. (2024). Adversarial attacks and defenses in explainable artificial intelligence: A survey. Information Fusion , 107:102303

    work page 2024

  3. [3]

    and Biecek, P

    Baniecki, H. and Biecek, P. (2025). Birds look like cars: Adversarial analysis of intrinsically interpretable deep learning . arXiv. 10.48550/arXiv.2503.08636

    work page doi:10.48550/arxiv.2503.08636 2025

  4. [4]

    Baniecki, H., Parzych, D., and Biecek, P. (2023). The grammar of interactive explanatory model analysis. Data Mining and Knowledge Discovery

    work page 2023

  5. [5]

    T., and Weld, D

    Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., and Weld, D. (2021). Does the Whole Exceed its Parts ? The Effect of AI Explanations on Complementary Team Performance . In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems , pages 1--16. ACM

    work page 2021

  6. [6]

    P., and Lakkaraju, H

    Bhalla, U., Oesterling, A., Srinivas, S., Calmon, F. P., and Lakkaraju, H. (2024). Interpreting CLIP with Sparse Linear Concept Embeddings ( SpLiCE ) . arXiv. 10.48550/arXiv.2402.10376

    Pith/arXiv arXiv doi:10.48550/arxiv.2402.10376 2024

  7. [7]

    and Burzykowski, T

    Biecek, P. and Burzykowski, T. (2021). Explanatory Model Analysis : Explore , Explain and Examine Predictive Models . Chapman and Hall/CRC, 1 edition

    work page 2021

  8. [8]

    and Samek, W

    Biecek, P. and Samek, W. (2024). Position: Explain to question not to justify. volume 235 of Proceedings of Machine Learning Research , pages 3996--4006. PMLR

    work page 2024

  9. [9]

    Bommasani, R. and et. al. (2022). On the Opportunities and Risks of Foundation Models . arXiv. 10.48550/arXiv.2108.07258

    Pith/arXiv arXiv doi:10.48550/arxiv.2108.07258 2022

  10. [10]

    Bücker, M., Szepannek, G., Gosiewska, A., and Biecek, P. (2022). Transparency, auditability, and explainability of machine learning models in credit scoring. Journal of the Operational Research Society , 73(1):70--90

    work page 2022

  11. [11]

    and Korolova, A

    Castleman, J. and Korolova, A. (2025). Adultification Bias in LLMs and Text -to- Image Models . In Proceedings of the 2025 ACM Conference on Fairness , Accountability , and Transparency , pages 2751--2767. ACM

    work page 2025

  12. [12]

    Visual Analytics for Explainable Deep Learning

    Choo, J. and Liu, S. (2018). Visual Analytics for Explainable Deep Learning . arXiv. 10.48550/arXiv.1804.02527

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.1804.02527 2018

  13. [13]

    Cleveland, W. S. (2001). Data science: An action plan for expanding the technical areas of the field of statistics. International Statistical Review , 69(1):21--26

    work page 2001

  14. [14]

    Dahl, M., Magesh, V., Suzgun, M., and Ho, D. E. (2024). Large legal fictions: Profiling legal hallucinations in large language models. Journal of Legal Analysis , 16(1):64--93

    work page 2024

  15. [15]

    Delfosse, Q., Shindo, H., Dhami, D., and Kersting, K. (2023). Interpretable and Explainable Logical Policies via Neurally Guided Symbolic Abstraction . arXiv. 10.48550/arXiv.2306.01439

    Pith/arXiv arXiv doi:10.48550/arxiv.2306.01439 2023

  16. [16]

    Attention Flows: Analyzing and Comparing Attention Mechanisms in Language Models

    DeRose, J. F., Wang, J., and Berger, M. (2020). Attention Flows : Analyzing and Comparing Attention Mechanisms in Language Models . arXiv. 10.48550/arXiv.2009.07053

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2009.07053 2020

  17. [17]

    Dreyer, M., Hufe, L., Berend, J., Wiegand, T., Lapuschkin, S., and Samek, W. (2025). From What to How : Attributing CLIP 's Latent Components Reveals Unexpected Semantic Reliance . arXiv. 10.48550/arXiv.2505.20229

    Pith/arXiv arXiv doi:10.48550/arxiv.2505.20229 2025

  18. [18]

    Elena, M., Valentin, M., Rafaela, C., and Codrut, T. (2021). An IFRS 9 Framework for Model Validation . International Journal of Business and Management Invention , 10:2319--8028

    work page 2021

  19. [19]

    A., and Steinhardt, J

    Gandelsman, Y., Efros, A. A., and Steinhardt, J. (2024). Interpreting CLIP 's Image Representation via Text - Based Decomposition . arXiv. 10.48550/arXiv.2310.05916

    Pith/arXiv arXiv doi:10.48550/arxiv.2310.05916 2024

  20. [20]

    Ganguli, D. and et. al. (2022). Red Teaming Language Models to Reduce Harms . arXiv. 10.48550/arXiv.2209.07858

    Pith/arXiv arXiv doi:10.48550/arxiv.2209.07858 2022

  21. [21]

    Gould, R., Ong, E., Ogden, G., and Conmy, A. (2023). Successor Heads : Recurring , Interpretable Attention Heads In The Wild . arXiv. 10.48550/arXiv.2312.09230

    Pith/arXiv arXiv doi:10.48550/arxiv.2312.09230 2023

  22. [22]

    Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., and Pedreschi, D. (2019). A Survey of Methods for Explaining Black Box Models . ACM Computing Surveys , 51(5):1--42

    work page 2019

  23. [23]

    Y., Phua, K., Wong, L., and Bin Goh, W

    Ho, S. Y., Phua, K., Wong, L., and Bin Goh, W. W. (2020). Extensions of the External Validation for Checking Learned Model Interpretability and Generalizability . Patterns , 1(8):100129

    work page 2020

  24. [24]

    Holzinger, A., Saranti, A., Molnar, C., Biecek, P., and Samek, W. (2022). Explainable AI Methods - A Brief Overview , pages 13--38. Springer International Publishing, Cham

    work page 2022

  25. [25]

    Hryniewska, W., Bombiński, P., Szatkowski, P., Tomaszewska, P., Przelaskowski, A., and Biecek, P. (2021). Checklist for responsible deep learning modeling of medical images based on COVID -19 detection studies. Pattern Recognition , 118:108035

    work page 2021

  26. [26]

    Hryniewska, W., Grudzień, A., and Biecek, P. (2022). LIMEcraft : handcrafted superpixel selection and inspection for Visual eXplanations . Machine Learning

    work page 2022

  27. [27]

    Kauffmann, J., Dippel, J., Ruff, L., Samek, W., Müller, K.-R., and Montavon, G. (2025). Explainable AI reveals Clever Hans effects in unsupervised learning models. Nature Machine Intelligence , 7(3):412--422

    work page 2025

  28. [28]

    Kaur, H., Nori, H., Jenkins, S., Caruana, R., Wallach, H., and Wortman Vaughan, J. (2020). Interpreting Interpretability : Understanding Data Scientists ' Use of Interpretability Tools for Machine Learning . In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems , pages 1--14. ACM

    work page 2020

  29. [29]

    D., Biecek, P., Russell, S., and Bengio, Y

    Knott, A., Pedreschi, D., Chatila, R., Chakraborti, T., Leavy, S., Baeza-Yates, R., Eyers, D., Trotman, A., Teal, P. D., Biecek, P., Russell, S., and Bengio, Y. (2023). Generative AI models should include detection mechanisms as a condition for public release. Ethics and Information Technology , 25(4)

    work page 2023

  30. [30]

    D., Basu, S., Haklidir, M., Morini, V., Russell, S., and Bengio, Y

    Knott, A., Pedreschi, D., Jitsuzumi, T., Leavy, S., Eyers, D., Chakraborti, T., Trotman, A., Sundareswaran, S., Baeza-Yates, R., Biecek, P., Weller, A., Teal, P. D., Basu, S., Haklidir, M., Morini, V., Russell, S., and Bengio, Y. (2024). AI content detection in the emerging information ecosystem: new obligations for media and tech companies. Ethics and In...

    work page 2024

  31. [31]

    Koenecke, A., Choi, A. S. G., Mei, K. X., Schellmann, H., and Sloane, M. (2024). Careless Whisper : Speech -to- Text Hallucination Harms . In The 2024 ACM Conference on Fairness , Accountability , and Transparency , pages 1672--1681. ACM

    work page 2024

  32. [32]

    Kulesza, T., Burnett, M., Wong, W.-K., and Stumpf, S. (2015). Principles of Explanatory Debugging to Personalize Interactive Machine Learning . In Proceedings of the 20th International Conference on Intelligent User Interfaces , pages 126--137. ACM

    work page 2015

  33. [33]

    and Biecek, P

    Kuzba, M. and Biecek, P. (2020). What Would You Ask the Machine Learning Model ? Identification of User Needs for Model Explanations Based on Human - Model Conversations . In Communications in Computer and Information Science , pages 447--459

    work page 2020

  34. [34]

    Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., and Müller, K.-R. (2019). Unmasking Clever Hans predictors and assessing what machines really learn. Nature Communications , 10(1)

    work page 2019

  35. [35]

    Lee, H., Phatale, S., Mansoor, H., Mesnard, T., Ferret, J., Lu, K., Bishop, C., Hall, E., Carbune, V., Rastogi, A., and Prakash, S. (2024). RLAIF vs. RLHF : Scaling Reinforcement Learning from Human Feedback with AI Feedback . arXiv. 10.48550/arXiv.2309.00267

    Pith/arXiv arXiv doi:10.48550/arxiv.2309.00267 2024

  36. [36]

    M., Zheng, Y., Zhang, W., and Ma, K.-L

    Li, Y., Wang, J., Dai, X., Wang, L., Yeh, C.-C. M., Zheng, Y., Zhang, W., and Ma, K.-L. (2023). How Does Attention Work in Vision Transformers ? A Visual Analytics Attempt . IEEE Transactions on Visualization and Computer Graphics , 29(6):2888--2900

    work page 2023

  37. [37]

    Give Me Your Attention: Dot-Product Attention Considered Harmful for Adversarial Patch Robustness

    Lovisotto, G., Finnie, N., Munoz, M., Mummadi, C. K., and Metzen, J. H. (2022). Give Me Your Attention : Dot - Product Attention Considered Harmful for Adversarial Patch Robustness . arXiv. 10.48550/arXiv.2203.13639

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2203.13639 2022

  38. [38]

    Lupo, U., Sgarbossa, D., and Bitbol, A.-F. (2022). Protein language models trained on multiple sequence alignments learn phylogenetic relationships. Nature Communications , 13(1)

    work page 2022

  39. [39]

    J., Rasheed, S., Moradidakhel, A., Tahir, A., and Khomh, F

    Majdinasab, V., Bishop, M. J., Rasheed, S., Moradidakhel, A., Tahir, A., and Khomh, F. (2024). Assessing the Security of GitHub Copilot 's Generated Code - A Targeted Replication Study . In 2024 IEEE International Conference on Software Analysis , Evolution and Reengineering ( SANER ) , pages 435--444. IEEE

    work page 2024

  40. [40]

    McGrath, T., Kapishnikov, A., Tomašev, N., Pearce, A., Wattenberg, M., Hassabis, D., Kim, B., Paquet, U., and Kramnik, V. (2022). Acquisition of chess knowledge in AlphaZero . Proceedings of the National Academy of Sciences , 119(47)

    work page 2022

  41. [41]

    Meng, K., Bau, D., Andonian, A., and Belinkov, Y. (2023a). Locating and Editing Factual Associations in GPT . arXiv. 10.48550/arXiv.2202.05262

    Pith/arXiv arXiv doi:10.48550/arxiv.2202.05262

  42. [42]

    S., Andonian, A., Belinkov, Y., and Bau, D

    Meng, K., Sharma, A. S., Andonian, A., Belinkov, Y., and Bau, D. (2023b). Mass- Editing Memory in a Transformer . arXiv. 10.48550/arXiv.2210.07229

    Pith/arXiv arXiv doi:10.48550/arxiv.2210.07229

  43. [43]

    Mieleszczenko-Kowszewicz, W., Płudowski, D., Kołodziejczyk, F., Świstak, J., Sienkiewicz, J., and Biecek, P. (2024). The Dark Patterns of Personalized Persuasion in Large Language Models . arXiv. 10.48550/arXiv.2411.06008

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2411.06008 2024

  44. [44]

    Miller, T. (2019). Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence , 267:1--38

    work page 2019

  45. [45]

    Nam, A., Conklin, H., Yang, Y., Griffiths, T., Cohen, J., and Leslie, S.-J. (2025). Causal Head Gating : A Framework for Interpreting Roles of Attention Heads in Transformers . arXiv. 10.48550/arXiv.2505.13737

    work page doi:10.48550/arxiv.2505.13737 2025

  46. [46]

    Ngo, A., Gupta, S., Perrine, O., Reddy, R., Ershadi, S., and Remick, D. (2024). ChatGPT 3.5 fails to write appropriate multiple choice practice exam questions. Academic Pathology , 11(1):100099

    work page 2024

  47. [47]

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., and Lowe, R. (2022). Training language models to follow instructions with human feedback. In Proceedings of the 36th Inter...

    work page 2022

  48. [48]

    and Björnsson, Y

    Pálsson, A. and Björnsson, Y. (2023). Unveiling Concepts Learned by a World - Class Chess - Playing Agent . In Proceedings of the Thirty - Second International Joint Conference on Artificial Intelligence , pages 4864--4872

    work page 2023

  49. [49]

    MASCOTS: Model-Agnostic Symbolic COunterfactual explanations for Time Series

    Płudowski, D., Spinnato, F., Wilczyński, P., Kotowski, K., Ntagiou, E. V., Guidotti, R., and Biecek, P. (2025). MASCOTS : Model - Agnostic Symbolic COunterfactual explanations for Time Series . arXiv. 10.48550/arXiv.2503.22389

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2503.22389 2025

  50. [50]

    D., and Finn, C

    Rafailov, R., Sharma, A., Mitchell, E., Ermon, S., Manning, C. D., and Finn, C. (2024). Direct Preference Optimization : Your Language Model is Secretly a Reward Model . arXiv. 10.48550/arXiv.2305.18290

    Pith/arXiv arXiv doi:10.48550/arxiv.2305.18290 2024

  51. [51]

    Why Should I Trust You ?

    Ribeiro, M. T., Singh, S., and Guestrin, C. (2016). " Why Should I Trust You ?": Explaining the Predictions of Any Classifier . In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pages 1135--1144. ACM

    work page 2016

  52. [52]

    R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D

    Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017). Grad- CAM : Visual Explanations from Deep Networks via Gradient - Based Localization . In 2017 IEEE International Conference on Computer Vision ( ICCV ) , pages 618--626. IEEE

    work page 2017

  53. [53]

    and Biecek, P

    Sobieski, B. and Biecek, P. (2025). Global Counterfactual Directions . In Lecture Notes in Computer Science , pages 72--90. Springer Nature

    work page 2025

  54. [54]

    Sobieski, B., Grzywaczewski, J., Sadlej, B., Tivnan, M., and Biecek, P. (2025). Rethinking visual counterfactual explanations through region constraint. In The Thirteenth International Conference on Learning Representations

    work page 2025

  55. [55]

    Spyrison, N., Cook, D., and Biecek, P. (2025). Exploring local explanations of nonlinear models using animated linear projections. Computational Statistics , 40(2):1071--1095

    work page 2025

  56. [56]

    Strobelt, H., Webson, A., Sanh, V., Hoover, B., Beyer, J., Pfister, H., and Rush, A. M. (2022). Interactive and Visual Prompt Engineering for Ad -hoc Task Adaptation With Large Language Models . IEEE Transactions on Visualization and Computer Graphics , pages 1--11

    work page 2022

  57. [57]

    ExplainableFold: Understanding AlphaFold Prediction with Explainable AI

    Tan, J. and Zhang, Y. (2023). ExplainableFold : Understanding AlphaFold Prediction with Explainable AI . arXiv. 10.48550/arXiv.2301.11765

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2301.11765 2023

  58. [58]

    Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, A., Bhosale, S., et al. (2023). LLaMA 2: Open Foundation and Fine-Tuned Chat Models . 10.48550/arXiv.2307.09288

    Pith/arXiv arXiv doi:10.48550/arxiv.2307.09288 2023

  59. [59]

    Tukey, J. W. (1962). The future of data analysis. The Annals of Mathematical Statistics , 33(1):1--67

    work page 1962

  60. [60]

    Wang, L., Wen, Z., Liu, S.-W., Zhang, L., Finley, C., Lee, H.-J., and Fan, H.-J. S. (2024). Overview of AlphaFold2 and breakthroughs in overcoming its limitations. Computers in Biology and Medicine , 176:108620

    work page 2024

  61. [61]

    Wu, T., Jiang, E., Donsbach, A., Gray, J., Molina, A., Terry, M., and Cai, C. J. (2022). PromptChainer : Chaining Large Language Model Prompts through Visual Programming . In CHI Conference on Human Factors in Computing Systems Extended Abstracts , pages 1--10. ACM

    work page 2022

  62. [62]

    S., Heer, J., and Ribeiro, M

    Wu, T., Shen, H., Weld, D. S., Heer, J., and Ribeiro, M. T. (2023). ScatterShot : Interactive In -context Example Curation for Text Transformation . pages 353--367

    work page 2023

  63. [63]

    Yeh, C., Chen, Y., Wu, A., Chen, C., Viégas, F., and Wattenberg, M. (2023). AttentionViz : A Global View of Transformer Attention . arXiv. 10.48550/arXiv.2305.03210

    Pith/arXiv arXiv doi:10.48550/arxiv.2305.03210 2023

  64. [64]

    Zack, T. and et. al. (2024). Assessing the potential of GPT -4 to perpetuate racial and gender biases in health care: a model evaluation study. The Lancet Digital Health , 6(1):e12--e22

    work page 2024

  65. [65]

    Zaigrajew, V., Baniecki, H., and Biecek, P. (2025). Interpreting CLIP with Hierarchical Sparse Autoencoders . arXiv. 10.48550/arXiv.2502.20578

    Pith/arXiv arXiv doi:10.48550/arxiv.2502.20578 2025

  66. [66]

    Zhou, Z., Hu, S., Li, M., Zhang, H., Zhang, Y., and Jin, H. (2023). AdvCLIP : Downstream -agnostic Adversarial Examples in Multimodal Contrastive Learning . arXiv. 10.48550/arXiv.2308.07026

    Pith/arXiv arXiv doi:10.48550/arxiv.2308.07026 2023

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.