Pith. sign in

REVIEW 3 major objections 3 minor 18 references

AI Forensics Across White-, Grey-, and Black-Box Access: A Process Model and Research Agenda for Post-Incident Investigation of AI Systems

T0 review · 3 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Investigator access, not system type, should be the organizing axis of AI forensics, and a phase-by-access matrix makes the fragmented field cohere while exposing its gaps.

desk verdict A useful, honest scaffold for AI forensics, but the access-primacy claim is asserted more than shown and the matrix leans heavily on LLM/agentic examples. read the letter →

arxiv 2608.03520 v1 pith:6AC4JEYW submitted 2026-08-04 cs.CR cs.AI

Ali Dehghantanha , Sajad Homayoun This is my paper · ORCID
classification cs.CRcs.AI
keywords AIforensicspost-incidentinvestigationwhite-boxaccessgrey-boxblack-boxorderofvolatilitymodelversionattestationchaincustody
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the field of AI forensics—investigating an AI system after an incident—should be organized by how much of the system an investigator can reach, not by the type of system involved. It defines three access regimes: white box (weights and training data), grey box (deployment logs, prompts, retrieval state), and black box (inputs and outputs only). On this axis it builds a process model matrix spanning collection, preservation, analysis, and reporting, and an order of volatility unique to AI systems that tells investigators what to capture first. The payoff is a unified map where fragmented prior work finds its place and the empty cells—black-box preservation, model version attestation, uncertainty for surrogate analysis—become a concrete research agenda. The authors do not claim the framework is validated; they offer it as a scaffold to test and refine.

What carries the argument

The process model matrix (Table 1): rows are the four forensic phases (collection, preservation, analysis, reporting) and columns are the three access regimes (white, grey, black box), each cell stating what is achievable, what binds, and which prior work occupies it. The access regimes are defined purely by artifact possession: white-box holds model internals and provenance, grey-box holds the deployment surface, black-box holds only I/O. The matrix carries the argument by converting the abstract 'access matters' claim into a concrete grid whose empty cells become research problems. Supporting machinery is the order of volatility, inspired by RFC 3227, which sets collection priority and exp

What would settle it

Collect a corpus of real AI-incident investigation reports and code each by artifacts actually held, system type, and the confidence of causal conclusions. If two investigations with the same access posture but different system types show systematically different forensic capability, or if most real cases have such fragmented access that no regime cleanly applies, the central organizing axis fails. A sharper test: document a black-box investigation that reached high-confidence causal attribution of a model-resident defect without provider cooperation or attestation—that single case would refut

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is that the forensic character of an AI investigation is determined first by what the investigator can reach. Access—defined by the artifacts obtainable, not by institutional role—sets what can be collected, preserved, analyzed, and reported with what confidence. The paper's contributions are the access taxonomy, the four-phase by three-regime process model matrix (Table 1), an AI-specific order of volatility (accelerator memory and context window most volatile, training lineage most stable, with a 'volatility-reachability inversion' where the most probative evidence is often least reachable), and a derived access-conditioned examination workflow.

Load-bearing premise

The framework assumes that the white/grey/black-box trichotomy, defined by which artifacts an investigator possesses, is a faithful and decisive description of real investigations—more decisive than system type or tooling—and the authors concede the workflow has not yet been tested on a corpus of real investigations.

Editorial extensions

If this is right

  • If adopted, new investigations can be scoped by asking 'what artifacts can we obtain?' first, and the appropriate process follows from that answer.
  • Existing techniques like influence functions, log correlation, or model extraction find their place as access-level tools; the field can stop fragmenting by system type.
  • Black-box investigations can name their epistemic ceiling: behavioral findings with explicit uncertainty, unless provider cooperation or model version attestation raises the floor.
  • The order of volatility gives a defensible collection sequence and motivates forensic readiness: logging, attestation, and preserved context windows become the high-value intervention points.
  • The research agenda becomes concrete: model version attestation, uncertainty quantification for surrogate analysis, and cross-boundary chain of custody are the gaps to fund.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The access trichotomy predicts a confidence gradient that is testable: if real case corpora show grey-box investigations routinely achieving high-confidence causal attribution, the gradient as stated is too pessimistic; conversely, if black-box investigations routinely fail without provider cooperation, the case for mandated disclosure strengthens.
  • The 'volatility-reachability inversion' suggests a policy lever the paper leaves implicit: regulators could compel providers to expose an attestable snapshot of the model version behind each logged inference, converting black-box cases into grey-box ones at the source.
  • The matrix could extend to a fourth dimension—time since incident—since preservation feasibility likely decays along the volatility order; a longitudinal study could quantify how quickly each evidence class becomes unreachable.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper proposes a framework for post-incident forensic investigation of AI systems organized primarily by investigator access. It defines three access regimes—white box, grey box, and black box—based on the artifacts an investigator can obtain, and maps these onto a four-phase process model: collection, preservation, analysis, and reporting (Table 1). It introduces an order of volatility for AI evidence (Section 5), derives an access-conditioned examination workflow (Section 8), and identifies open problems such as black-box preservation, model version attestation, and uncertainty quantification for surrogate-based analysis. The motivating example of a financial advisory agent is carried through each phase to illustrate how the same incident is investigated differently across access levels. The paper explicitly positions itself as a position paper and research agenda rather than an empirically validated standard.

Significance. If the access-first thesis holds, the paper would provide a useful unifying axis for a fragmented literature spanning model forensics, agentic system investigations, provenance, and black-box extraction. The authors deserve credit for clearly defining the access regimes, constructing a concrete matrix that situates prior work, articulating an order of volatility with a sharp 'volatility–reachability inversion' insight, connecting the framework to the EU AI Act, and candidly flagging that the workflow has not been evaluated on real investigations. The framework is internally consistent and the definitions are transparent. However, the central organizing claim—that access, not system type, is the primary determinant of forensic process—is not actually established by the paper's operational content, because the matrix and volatility order are populated almost entirely from an LLM/agentic scenario.

major comments (3)
  1. [Section 4, Table 1 and Section 3.3] The matrix's operational content is almost entirely drawn from the motivating agentic-LLM/RAG scenario: grey-box cells list prompts, RAG stores, and tool-call traces; black-box cells list surrogate modeling and reanimation; analysis cells mention prompt-injection reconstruction and drift analysis; the volatility order includes a 'context window' and 'RAG/vector store state.' For non-LLM AI systems—tabular classifiers, vision models, speech systems—these artifacts and techniques do not exist, and the corresponding grey-box/black-box processes would be different (e.g., feature/service logs, prediction explanations, a different API extraction surface, a different volatility ordering). Thus Table 1 does not demonstrate that access alone organizes the field; system type appears to do the organizational work. The authors need to either populate the matrix for at least one non-LLM system type t
  2. [Section 5] The proposed order of volatility is presented as 'AI specific' and is used to 'dictate collection sequencing' for the whole discipline, yet items 1, 2, 6, and 7—accelerator memory, assembled context window, RAG/vector store state, and agent/tool-call traces—are artifacts of one system type. A tabular credit-scoring model or an image classifier has no context window, no KV cache, and no RAG store; its volatility ordering would look different, and the 'volatility–reachability inversion' would need to be re-derived. This reinforces the concern in my first comment: the paper's substantive content is system-type-specific even though the stated thesis is that access is the primary axis. A general order of volatility should be stated at the level of evidence classes (e.g., transient inference state, request/response logs, persistent model artifacts, training lineage) with system-type-specific i
  3. [Section 8, final paragraph] The paper states: 'It has not yet been evaluated across a corpus of real investigations, and some steps, especially black box preservation, depend on capabilities that the field has not yet matured.' This is a load-bearing limitation because the central claim is an organizational/empirical one: that access, rather than system type, determines the forensic process. As written, the framework is a plausible hypothesis, but the only concrete instantiation is the LLM/agentic case, and no retrospective test against published case studies (e.g., the OpenClaw investigation cited as [7]) is provided. The authors should either add retrospective application to one or more published investigations spanning different system types, or explicitly frame the access-first thesis as a falsifiable prediction with proposed tests, so that the paper's contribution as a research agenda is not overstated.
minor comments (3)
  1. [Sections 4.1–4.4] The recurring heading 'Case beat.' appears to be a typo; likely intended as 'Case in point' or 'Case study.' Please fix throughout.
  2. [Section 4, Table 1] The table caption in the text renderings shows 'T able 1' with a stray space. Also, the table uses 'Gap' as a stylistic marker, but some cells marked as gaps (e.g., 'model-version attestation') are also discussed in the main text; consider adding cross-references to the corresponding sections.
  3. [Section 4.3] The root-cause taxonomy is introduced with examples but no citation for the enumerated categories (data poisoning, backdoor, prompt injection, RAG poisoning, jailbreak, drift, supply-chain compromise). Please cite relevant surveys or taxonomies to help readers locate the broader attack literature.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the access-based taxonomy is definitional but transparent, with no fitted inputs, self-citation, or hidden reduction.

full rationale

The paper's central claim—that investigator access determines forensic capability—is introduced as an organizing definition (Sec. 3.1: white-box = possession of internals, black-box = I/O only), and the process matrix (Table 1) unpacks that definition rather than predicting empirical outcomes from hidden assumptions. No parameter is fitted, no related quantity is predicted from a subset, and the reference list contains no self-citations by the authors. The framework even acknowledges its own limitation (Sec. 8: 'has not yet been evaluated across a corpus of real investigations'). The only potentially definitional move is that collection cells restate the access definitions; this is explicit and framed as 'idealized access postures rather than rigid categories' (Sec. 3.1), so it is not concealed circular reasoning. If the matrix is over-fit to LLM/agentic artifacts, that is a correctness/completeness concern, not a circularity one. Score 1 reflects the mild definitional character, not a circular derivation.

Assumptions & free parameters 0 free parameters · 4 assumptions · 1 invented entities

The paper rests on domain assumptions about the usefulness of its access taxonomy and the transferability of traditional forensic phases to AI systems. It introduces one new postulated capability, model version attestation, with no independent evidence or implementation. There are no free parameters or fitted constants because the paper makes no quantitative claims.

assumptions (4)
  • domain assumption The white, grey, and black box trichotomy, defined by artifact possession, is a usable approximation of real investigator reach.
    Section 3.1 defines the regimes as idealized postures and admits access is often fragmented; the taxonomy is assumed rather than validated against real cases.
  • domain assumption Access, rather than system type, is the primary determinant of forensic capability.
    Section 3.2 argues this through a single motivating incident and does not compare against organization by system type or by tooling maturity.
  • domain assumption The traditional forensic phases of collection, preservation, analysis, and reporting transfer to AI systems.
    Section 2.1 asserts that DFRWS, NIST SP 800 86, ISO 27037, and ISO 27043 provide foundations, with AI-specific extensions needed; no empirical validation is offered.
  • domain assumption RFC 3227 volatility reasoning can be generalized to AI pipeline stages.
    Section 5 proposes an AI order of volatility based on qualitative decay rates, without measurements of how quickly each evidence class actually decays.
invented entities (1)
  • Model version attestation
    purpose: A proposed verifiable and timestamped binding between an observed behavior and the model version, configuration, retrieval state, and serving context that produced it, intended to make black box preservation possible.
    Section 4.2 and Section 9 name this as a missing forensic primitive, but the paper provides no concrete protocol, implementation, or test. It is a needed capability, not an existing artifact.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AI Forensics Across White-, Grey-, and Black-Box Access: A Process Model and Research Agenda for Post-Incident Investigation of AI Systems." pith.science (2026). https://pith.science/paper/6AC4JEYW

@misc{pith2026260803520,
  author       = {Pith},
  title        = {Pith review of: AI Forensics Across White-, Grey-, and Black-Box Access: A Process Model and Research Agenda for Post-Incident Investigation of AI Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6AC4JEYW}},
  note         = {Machine review of arXiv:2608.03520}
}
read the original abstract

AI systems are increasingly involved in decisions and actions that may later require investigation. When an AI related incident occurs, investigators need to reconstruct what the system did, why it behaved that way, and which part of the system or supply chain contributed to the outcome. Existing work on AI forensics remains fragmented, often focusing on a specific system type, artifact, or analysis technique. This paper argues that investigator access is a useful starting point for organizing the field. We distinguish white box, grey box, and black box access and show how each access level changes what can be collected, preserved, analyzed, and reported. Based on this distinction, we propose a process model matrix for AI forensics across four phases: collection, preservation, analysis, and reporting. We also introduce an order of volatility for AI systems, covering runtime state, context windows, logs, retrieval stores, model artifacts, and training lineage. From this matrix, we derive an access conditioned examination framework and identify open research problems, including black box preservation, model version attestation, uncertainty quantification for surrogate based analysis, and chain of custody for mutable AI artifacts.

Figures

Figures reproduced from arXiv: 2608.03520 by the authors.

Figure 1
Figure 1. Access conditioned workflow for AI forensic investigation. internal evidence, including training data attribution, weight comparison, acti￾vation inspection, and backdoor checks. The report can support stronger causal claims, but should still explain the methods and their limits in terms that a non specialist audience can assess. Grey box. In a grey box investigation, the priority is to preserve the deployment surfa… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 15 canonical work pages

  1. [7]

    Foundations for Agentic AI Investigations from the Forensic Analysis of OpenClaw

    Gruber, J., Hilgert, J.N.: Foundations for Agentic AI Investigations from the Forensic Analysis of OpenClaw. arXiv preprint arXiv:2604.05589 (2026),https: //arxiv.org/abs/2604.05589

    work page Pith review arXiv 2026

  2. [1]

    White paper (2025),https://www.alvarezandmarsal.com/insights/ artificial-intelligence-and-machine-learning-model-forensics

    Alvarez & Marsal: Artificial intelligence and machine learning model forensics. White paper (2025),https://www.alvarezandmarsal.com/insights/ artificial-intelligence-and-machine-learning-model-forensics

    work page 2025

  3. [2]

    Founding The Domain of AI Forensics

    Baggili, I., Behzadan, V.: Founding the domain of AI forensics. In: Proceed- ings of the AAAI Workshop on Artificial Intelligence Safety (SafeAI) (2020), arXiv:1912.06497

    work page Pith review arXiv 2020

  4. [3]

    Information Fusion58, 82–115 (2020).https://doi.org/10.1016/j.inffus.2019.12.012,http://dx.doi.org/ 10.1016/j.inffus.2019.12.012

    Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., Chatila, R., Herrera, F.: Explainable artificial intelligence (xai): Concepts, taxonomies, op- portunities and challenges toward responsible ai. Information Fusion58, 82–115 (2020).https://doi.org/10.1016/j.inff...

    doi:10.1016/j.inffus.2019.12.012 2020

  5. [4]

    RFC 3227, Internet Engineering Task Force (IETF) (2002)

    Brezinski, D., Killalea, T.: Guidelines for evidence collection and archiving. RFC 3227, Internet Engineering Task Force (IETF) (2002)

    work page 2002

  6. [5]

    Forensic Science International: Digital Investiga- tion48, 301675 (Mar 2024).https://doi.org/10.1016/j.fsidi.2023.301675, http://dx.doi.org/10.1016/j.fsidi.2023.301675

    Dunsin, D., Ghanem, M.C., Ouazzane, K., Vassilev, V.: A comprehensive anal- ysis of the role of artificial intelligence and machine learning in modern digital forensics and incident response. Forensic Science International: Digital Investiga- tion48, 301675 (Mar 2024).https://doi.org/10.1016/j.fsidi.2023.301675, http://dx.doi.org/10.1016/j.fsidi.2023.301675

    arXiv 2024

  7. [6]

    laying down harmonised rules on artificial intelligence (artificial in- telligence act)

    European Parliament and Council of the European Union: Regulation (EU) 2024/1689 ... laying down harmonised rules on artificial intelligence (artificial in- telligence act). Official Journal of the European Union, OJ L, 2024/1689 (2024)

    work page 2024

  8. [8]

    International Organization for Standardization: ISO IEC 27037:2012 Information technology, Security techniques, Guidelines for identification, collection, acquisi- tion and preservation of digital evidence (2012),https://www.iso.org/standard/ 44381.html

    work page 2012

Show all 18 references
  1. [9]

    International Organization for Standardization: ISO IEC 27043:2015 Information technology, Security techniques, Incident investigation principles and processes (2015),https://www.iso.org/standard/44407.html

    2015

  2. [10]

    Kent, K., Chevalier, S., Grance, T., Dang, H.: Guide to Integrating Forensic Tech- niques into Incident Response. Tech. Rep. Special Publication 800 86, National Institute of Standards and Technology (2006),https://csrc.nist.gov/pubs/sp/ 800/86/final

    2006

  3. [11]

    In: Proceedings of the 34th International Conference on Machine Learning (ICML)

    Koh, P.W., Liang, P.: Understanding black-box predictions via influence functions. In: Proceedings of the 34th International Conference on Machine Learning (ICML). PMLR, vol. 70, pp. 1885–1894 (2017)

    2017

  4. [12]

    Mu, X., Wang, Y., Zhang, Y., Zhang, J., Wang, H., Xiang, Y., Yu, Y.: Model prove- nance via model dna (2023).https://doi.org/10.48550/ARXIV.2308.02121, https://arxiv.org/abs/2308.02121

    2023 doi arXiv

  5. [13]

    In: 33rd USENIX Security Symposium (USENIX Security 24)

    Oygenblik, D., Yagemann, C., Zhang, J., Mastali, A., Park, J., Saltaformaggio, B.: AI psychiatry: Forensic investigation of deep learning networks in memory images. In: 33rd USENIX Security Symposium (USENIX Security 24). pp. 1687–1704. USENIX Association (2024)

    2024

  6. [14]

    In: Proceedings of the First Digital Forensic Research Workshop

    Palmer, G.: A Road Map for Digital Forensic Research. In: Proceedings of the First Digital Forensic Research Workshop. Digital Forensic Research 18 A. Dehghantanha et al. Workshop (2001),https://dfrws.org/wp-content/uploads/2019/06/2001_USA_ a_road_map_for_digital_forensic_res...

    2001

  7. [15]

    Schneider, J., Breitinger, F.: Towards AI forensics: Did the artificial intelligence system do it? Journal of Information Security and Applications76, 103517 (2023)

    2023

  8. [16]

    Proceedings of the VLDB Endowment15(13), 4015–4022 (2022).https://doi.org/10.14778/3565838.3565853,http://dx.doi.org/10

    Shankar, S., Parameswaran, A.G.: Towards observability for production machine learning pipelines. Proceedings of the VLDB Endowment15(13), 4015–4022 (2022).https://doi.org/10.14778/3565838.3565853,http://dx.doi.org/10. 14778/3565838.3565853

    2022

  9. [17]

    Chapman & Hall/CRC, Boca Raton, FL (2026)

    Sremack, J.C.: AI Forensics: Investigation and Analysis of Artificial Intelligence Systems. Chapman & Hall/CRC, Boca Raton, FL (2026)

    2026

  10. [18]

    In: 25th USENIX Security Symposium (USENIX Security 16)

    Tramèr, F., Zhang, F., Juels, A., Reiter, M.K., Ristenpart, T.: Stealing ma- chine learning models via prediction APIs. In: 25th USENIX Security Symposium (USENIX Security 16). pp. 601–618. USENIX Association (2016)

    2016

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.