Pith. sign in

REVIEW 5 major objections 5 minor 89 references

This paper argues that runtime telemetry from AI agents can be turned directly into EU AI Act compliance evidence.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 02:27 UTC pith:UZWWHSAN

load-bearing objection A coherent architecture paper that usefully integrates OTel telemetry, guardrail verification, and EU AI Act mapping, but its central tamper-resistance and privacy claims are asserted rather than demonstrated. the 5 major comments →

arxiv 2607.14309 v1 pith:UZWWHSAN submitted 2026-07-15 cs.AI cs.CY

Traccia: An OpenTelemetry-Based Governance Platform for AI Systems

Nutan Kumar Naik , Aditya Kumar Saroj , Vijay Prasad Poudel , Saurav Samantray , Abhishek Patel This is my paper
classification cs.AI cs.CY
keywords AI governanceOpenTelemetrycompliance evidenceEU AI ActLLM observabilityruntime telemetryguardrailsAI agents
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Traccia's central thesis is that the 'last mile' of AI governance—translating high-level compliance rules into verified, machine-readable evidence—can be closed by making compliance artifacts a side effect of ordinary runtime execution. The paper proposes a seven-layer stack, built on OpenTelemetry, that intercepts large-language-model and agent workflows in-process, structures them as trace spans, applies passive safety guardrails and PII redaction, and hashes the results into a JSON Evidence Package. Each package is claimed to map automatically to specific EU AI Act obligations (record-keeping, human oversight, transparency, incident reporting), so auditors would receive execution-linked proof rather than self-reported documents. If correct, this would let regulated enterprises demonstrate continuous compliance for non-deterministic, agentic systems that static pre-deployment checks cannot cover. The claim's force is carried by a loan-approval example, where registration, guardrail triggers, inference hashes, human reviews, and a fundamental-rights impact assessment record all emerge from one instrumented run without separate compliance code.

Core claim

Traccia's core claim is that trace telemetry can be upgraded into a governance ledger by adding semantics and hashing at each stage. Runtime instrumentation turns agent executions into OpenTelemetry spans (prompts, tokens, guardrail outcomes); governance layers add registry data, human reviews, incidents; and the compliance layer bundles these into one JSON Evidence Package with a SHA-256 hash and 'tamper-resistant fingerprints.' Export metadata maps each record to EU AI Act Articles 12, 14, 19, 26(6), 50, 72, 73 and Annexure VIII. The demonstration: all of this happens automatically, with PII redacted before export, so evidence is continuous, execution-derived, and privacy-preserving.

What carries the argument

The load-bearing mechanism is the hashed trace ledger: a chain of OpenTelemetry spans starting inside the application, enriched into an execution graph, and ending in a cryptographically hashed export. Standard OpenTelemetry (OTel) semantic conventions are extended with AI attributes (prompts, tokens, provider signals); a three-tier guardrail detector runs over completed spans; and a missing-guardrail evaluator flags LLM calls without a safety wrapper. The compliance layer bundles policy configuration, run history, incidents, and human reviews into one JSON Evidence Package whose SHA-256 hash is meant to make later modification evident. OpenTelemetry is an open standard for distributed trace

Load-bearing premise

The load-bearing premise is that the SHA-256 hash appended by Traccia's own export process is enough to make evidence packages tamper-resistant; if that hash is not anchored somewhere the operator cannot rewrite, the integrity guarantee does not follow from the described architecture.

What would settle it

Edit one field in an exported Evidence Package, re-run the package generator, and ask a third-party verifier with no access to Traccia whether the modified package is still valid. If it validates, the tamper-resistance claim is refuted.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If Traccia is right, EU AI Act audits can run against execution-linked evidence packages instead of self-assessment documents, giving regulators machine-readable records of what a system actually did.
  • The loan-approval example implies that high-risk deployments can satisfy record-keeping, transparency, human-oversight, and incident-reporting duties without developers writing compliance code—governance artifacts emerge from normal instrumentation.
  • The missing-guardrail evaluator turns guardrail coverage into a detectable property: any LLM span with no surrounding safety span is flagged at runtime, letting enterprises find ungoverned agents as they run.
  • Because the instrumentation layer emits generic OTLP spans and does not require an agent daemon, swapping backends only means pointing an exporter at a new endpoint, so the telemetry layer itself does not create vendor lock-in.
  • Hashing inputs, outputs, and decision records at span level means that if a logged decision is later disputed, the evidence package can show whether the recorded interaction was altered.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's own mechanism does not yet show how an auditor can trust the hash: nothing in the described architecture anchors the SHA-256 digest outside Traccia's own export process, so the tamper-resistance guarantee depends on an external anchoring step (append-only store, trusted timestamp, or verifier) that is not specified.
  • A natural, testable next step is to formalize the eu_ai_act_article_mapping as a machine-verifiable link, so an auditor can check that every obligation attached to a span is actually satisfied by the recorded telemetry rather than by a metadata tag.
  • If runtime evidence becomes the expected form of AI compliance, the practical consequence is broader than one platform: self-reported governance would shift toward continuous assurance, and both regulators and vendors would need to agree on what counts as an anchored, verifiable trace.
  • The same instrumentation could be reused as a benchmark harness: by replaying known adversarial inputs (prompt injections, PII disclosures) through a governed agent and checking that every attack produces a guardrail span and redaction record, teams could measure coverage quantitatively.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces Traccia, an OpenTelemetry-based governance platform for AI systems. It argues that current LLM evaluation platforms, MLOps experiment trackers, and general observability suites are fragmented and fail to provide continuous, verifiable governance evidence for non-deterministic agentic systems. The core proposal is a seven-layer pipeline — from runtime instrumentation through telemetry intelligence, visualization, guardrail detection, a governance hub, and compliance export — that converts execution traces into structured compliance artifacts. The paper claims that Traccia automatically produces tamper-resistant compliance evidence packages with SHA-256 hashes mapped to EU AI Act articles (12, 14, 19, 26(6), 50, 72, 73, Annex VIII) without invading data privacy, and it grounds this in a loan-approval-assistant walkthrough (Section 4.8) plus a qualitative comparison with eight platforms (Section 5).

Significance. If the central claims were substantiated, Traccia would address a real and current gap: turning runtime telemetry into machine-readable, audit-ready compliance evidence for high-risk AI systems. The architectural description is concrete and unusually detailed for this space, and the choice to build on OpenTelemetry/W3C trace context is a genuine strength with vendor-lock-in implications. The EU AI Act article mapping, the governance hub with human-in-the-loop review, and the missing-guardrail evaluator are useful design contributions. However, the paper's most distinctive claims — tamper resistance, privacy preservation, and runtime guardrail verification — are asserted rather than demonstrated. There is no threat model, no benchmark, no overhead or accuracy measurement, and no independent verification mechanism. The paper is best read as a design/architecture proposal; its evaluative language exceeds the evidence provided.

major comments (5)
  1. [§4.7 / Abstract] The tamper-resistance claim is load-bearing and unsupported. The paper says Evidence Packages are 'exported as a single JSON file protected by a cryptographic integrity hash,' but the hash is computed and appended by Traccia's own export process. Without external anchoring — e.g., a trusted timestamping service, append-only ledger, or hardware root of trust — any party with write access to the platform or export code can recompute the hash after modifying evidence. No threat model or adversary analysis is provided. This is not a minor omission: the paper's differentiation from self-assessment platforms (Section 5.3) depends on Traccia evidence being independently verifiable, and the current mechanism does not provide that.
  2. [§4.8 / §5] The paper provides no empirical evaluation of the platform's core functionality. Section 4.8 is an illustrative walkthrough, not a benchmark; Section 5 is a qualitative feature comparison. There are no measurements of instrumentation overhead, guardrail detection accuracy, false-positive rates, redaction effectiveness, trace export scalability, or evidence-package verification. Claims such as 'resolves the last mile' and 'solid machine-readable base' are therefore unsupported. For a systems paper, at least basic experimental validation — or an explicit statement that the article is a design proposal with evaluation deferred — is needed before the central claims can be accepted.
  3. [§4.2 / §4.8] The privacy claim ('without invading any data privacy') is overstated. The mechanism described is regex-based PII redaction via init(redact_pii=True) and redact_string(). There is no formal privacy model, no data-flow analysis covering all span attributes, log fields, hashed inputs, or exported JSON, and no threat model for the telemetry path. Redaction before export is a reasonable start, but it does not establish a privacy guarantee. The paper should either weaken the claim to 'best-effort regex redaction' or provide a concrete privacy analysis.
  4. [§5.2 / §4.5] Tier A guardrail verification is annotation-based and may not support the paper's stronger claims. Tier A detects spans marked by guardrail_span() or @observe(as_type='guardrail'), i.e., self-declared guardrails present in the trace. This verifies the presence of an annotation, not that the guardrail actually enforced a policy. A span could be annotated but ineffective, bypassed, or incorrectly configured. The paper should distinguish 'guardrail presence in telemetry' from 'guardrail efficacy' and either add evidence that Tier A relates to actual enforcement or soften the §5.2 claim about 'runtime activation.'
  5. [§4.7] The EU AI Act mapping is asserted rather than validated. The paper states that telemetry may be associated with Articles 12, 14, 19, 26(6), 50, 72, 73, and Annex VIII, and that Evidence Packages provide a 'tamperproof chain of compliance documentation.' However, Section 4.7 also explicitly scopes out legal conformity and database registration. The article should clarify that the eu_ai_act_article_mapping metadata is a suggested correspondence, not legal evidence of compliance. Without this clarification, the abstract's phrase 'compliance evidence packages' overclaims.
minor comments (5)
  1. [Abstract / Throughout] The manuscript contains numerous typographical and grammatical errors (e.g., 'account ability,' 'Artificial Intelligent,' 'unbound state space agentic architecture'). A thorough language edit is needed.
  2. [Section 4] Figure 1 and Figure 2 are referenced as the architecture diagram and the loan-assistant trace visualization, but the rendered manuscript does not include the actual figures. Please ensure the figures are present and legible.
  3. [§4.2] The description of the SDK's automatic instrumentation uses 'monkey-patching' without discussing its limitations or compatibility risks across provider SDK versions. This is relevant to the robustness of the telemetry layer.
  4. [§5.2] The statement that 'none of the listed platforms feature a native tool for verifying the presence of guardrails and their runtime activation' is strong and would benefit from concrete citations or public documentation links for each platform.
  5. [§6] Minor typo: 'HIPPA' should be 'HIPAA'. Also, the future-work paragraph mentions 'runtime enforcement possibilities' while Section 4.8 describes guardrails in block mode; clarify the difference between current enforcement at the guardrail layer and future system-level enforcement.

Circularity Check

0 steps flagged

No significant circularity: Traccia is an architectural proposal with no fitted-parameter or self-citation derivation chain.

full rationale

The paper is an architecture/design description, not a derivation. It contains no equations, no fitted parameters, and no quantitative predictions. The strongest claim—that appending SHA-256 hashes yields tamper-resistant evidence packages (§4.7)—is an engineering assertion whose support would require an external anchoring mechanism and threat model, but that is an evidential gap, not circularity: the hash is not a quantity fitted to the compliance mapping, and the EU AI Act article mapping is an explicit enrichment metadata assignment ('enriches exported telemetry with eu_ai_act_article_mapping metadata that links each evidence record to the relevant provisions of the EU AI Act'), not a result derived from the hash. The paper also scopes out the legal conclusion ('legal conformations and database registration lie beyond the scope of the technical solution'). Tier A guardrail verification (§4.5) reads annotations produced by Traccia's own decorators, so it is self-referential as an operational signal, but the paper frames it as detection of developer-declared guardrails rather than as an independent proof of safety; no load-bearing 'prediction' is forced by construction. Existing-tool comparisons are external and are not used to justify Traccia's mechanisms. No self-citation chain or uniqueness theorem is invoked. Thus no circular step meets the quotation-and-reduction standard.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 0 invented entities

No free parameters or invented entities. The core claims rest on domain assumptions about telemetry completeness, hash-based integrity, and legal interpretation; all three are asserted rather than demonstrated.

axioms (4)
  • domain assumption OTel trace attributes capture complete and accurate AI execution semantics.
    Sections 4.2-4.3 rely on spans and attributes for prompts, tool calls, and guardrails; no validation of completeness or accuracy is provided.
  • ad hoc to paper A SHA-256 hash over the exported JSON is a tamper-resistant evidence mechanism.
    Section 4.7: 'single JSON file protected by a cryptographic integrity hash'; no external anchor or threat model is specified.
  • domain assumption The EU AI Act article mapping assigned by the Governance Enrichment Processor is legally sufficient.
    Sections 4.7-4.8 map Articles 12, 14, 19, 26(6), 50, 72, 73, and Annex VIII without legal validation.
  • domain assumption Monkey-patching and auto-instrumentation are passive and do not alter application behavior.
    Section 4.2 describes the framework as an observer 'not affecting the performance of the application'; no overhead measurements are reported.

pith-pipeline@v1.3.0-alltime-deepseek · 19540 in / 9940 out tokens · 90647 ms · 2026-08-02T02:27:29.668094+00:00 · methodology

0 comments
read the original abstract

The rapid development of Large Language Models (LLMs) and Artificial Intelligent (AI) powered autonomous agents has fundamentally changed the existing forms of software governance. In spite of the rigorous standards of transparency and account ability required according to the international frameworks such as the European Union's AI Act, there is a considerable gap between theory and reality. The present study discusses the inherent drawbacks of currently utilized platforms for LLM evaluation, machine learning workflow, and application performance monitoring in general. It has been shown that current disjointed solutions fail to protect unbound state space agentic architecture from serious threats such as alignment drift, SaaS security concerns, and unauthorized deployment of shadow AI systems. Moreover, a solution is proposed for overcoming the discussed challenges in form of a coherent multi-level AI governance stack Traccia built on the top of OpenTelemetry infrastructure platform. Traccia resolves the last mile for AI Alignment by adding the telemetry data, passive semantic guardrail assessment, and execution lineage into a hashed trace ledger. Traccia automatically creates compliance evidence packages by appending tamper-resistant fingerprints and SHA-256 content hash, that map to regulatory requirements (Articles 12, 14, 19, 26(6), and 50 of the EU AI Act) without invading any data privacy. By performing this evaluation in a methodical manner, a solid machine-readable base has been created for enterprise-wide management of autonomous AI systems.

Figures

Figures reproduced from arXiv: 2607.14309 by Abhishek Patel, Aditya Kumar Saroj, Nutan Kumar Naik, Saurav Samantray, Vijay Prasad Poudel.

Figure 1
Figure 1. Figure 1: 4.1 AI Systems The AI systems is the boundary layer of the external operational system, where the workloads creating the telemetry stream are running. The execution of LLMs, agentic workflows, multi-agent systems, and augmented tooling pipelines created on top of orchestrators [72, 4]. By processing user queries and handling them, the application creates a dynamic reasoning loop, querying vector database, … view at source ↗
Figure 1
Figure 1. Figure 1: The vertical architecture of Traccia showing the end-to-end transformation of AI applica [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Execution trace from Traccia platform for the AI Loan Approval Assistant (trace id - [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

89 extracted references · 43 canonical work pages · 5 internal anchors

  1. [1]

    Sapkota, K.I

    R. Sapkota, K.I. Roumeliotis, and M. Karkee. AI agents vs.Agentic AI: A Conceptual Taxonomy, Applications and Challenges. https://doi.org/10.70777/si.v2i3.15161, 2025

    work page doi:10.70777/si.v2i3.15161 2025

  2. [2]

    S. Tsuei. AI competency: Current state and challenges.https://doi.org/10.2196/86686, 2026

    work page doi:10.2196/86686 2026

  3. [3]

    S. Yang, Z. Yang, Y . Liu, and H. Wang. From single-agent to multi-agent: a comprehensive review of LLM-based legal agents.https://doi.org/10.20517/aiagent.2025.06, 2025

    work page doi:10.20517/aiagent.2025.06 2025

  4. [4]

    Yao and H

    Z. Yao and H. Yu. A survey on LLM-based multi-agent AI hospital. https://doi.org/10.31219/osf.io/bv5sg_v2, 2025

    work page doi:10.31219/osf.io/bv5sg_v2 2025

  5. [5]

    Patra, A.N

    D. Patra, A.N. Saha, and S.S. Mukherjee. Safety-constrained agentic AI for autism screening: A multimodal, clinician-guided architecture.https://doi.org/10.7759/cureus.108961, 2026

    work page doi:10.7759/cureus.108961 2026

  6. [6]

    Sengupta, P

    A. Sengupta, P. Seth, and V .K. Sankarapu. Interpretability as alignment: Making internal understanding a design principle.https://doi.org/10.48550/arXiv.2509.08592, 2025

    doi:10.48550/arxiv.2509.08592 2025

  7. [7]

    Hagendorff

    T. Hagendorff. The ethics of AI ethics: An evaluation of guidelines. https://doi.org/10.1007/s11023-020-09517-8, 2020

    doi:10.1007/s11023-020-09517-8 2020

  8. [8]

    E. Tabassi. Artificial intelligence risk management framework (AI RMF 1.0). https://doi.org/10.6028/nist.ai.100-1, 2023

    doi:10.6028/nist.ai.100-1 2023

  9. [9]

    Chowdhury, P.K

    S. Chowdhury, P.K. Dey, S. Joel-Edgar, S. Bhattacharya, O. Rodríguez-Espíndola, A. Abadie, and L. Truong. Unlocking the value of artificial intelligence in human resource management through AI capability framework.https://doi.org/10.1016/j.hrmr.2022.100899, 2022

    arXiv 2022

  10. [10]

    Chua, B.M

    D.Y . Chua, B.M. McMahon, and S. Kos. Shadow AI in consumer health: The case for safe adoption of national health AI assistants.https://doi.org/10.1016/j.mcpdig.2026.100352, 2026

    arXiv 2026

  11. [11]

    L.M. Wass, Z. Wu, J. Vizoso, J.T. Wu, and L. Lin. Global sentiment toward health AI at the dawn of the chatgpt era: Empirical analysis of twitter (x) discourse. https://doi.org/10.2196/80346, 2026

    work page doi:10.2196/80346 2026

  12. [12]

    W. Collins. Systemic governance risk in AI agent networks: Cross-institutional contagion and topological vulnerability.https://doi.org/10.2139/ssrn.6535599, 2026

    work page doi:10.2139/ssrn.6535599 2026

  13. [13]

    Westover

    J. Westover. When algorithms meet ethics: Systematic evidence of framing effects in LLM organizational decision-making.https://doi.org/10.20944/preprints202602.1103.v1, 2026. 21 Traccia: An OpenTelemetry-Based Governance Platform for AI Systems

    arXiv 2026

  14. [14]

    Demirhan and W

    H. Demirhan and W. Zadrozny. LLM-based control for simulated physical rea- soning: Modular evaluation in the neurips embodied agent interface challenge. https://doi.org/10.3390/ai7040131, 2026

    work page doi:10.3390/ai7040131 2026

  15. [15]

    Nguyen, K

    J. Nguyen, K. Hoang, C.L. Attubato, and F. Hofstätter. Probing and steering evaluation awareness of language models.https://doi.org/10.48550/arXiv.2507.01786, 2025

    Pith/arXiv arXiv doi:10.48550/arxiv.2507.01786 2025

  16. [16]

    Bandara, A

    E. Bandara, A. Gunaratna, R. Gore, A. Rahman, R. Mukkamala, S. Shetty, S. Rajapakse, I. Kularathna, P. Foytik, S.H. Bouk, X. Liang, A. Hass, N.W. Keong, and K.D. Zoysa. AI trust os – a continuous governance framework for autonomous AI observability and zero-trust compliance in enterprise environments.https://doi.org/10.48550/arXiv.2604.04749, 2026

    Pith/arXiv arXiv doi:10.48550/arxiv.2604.04749 2026

  17. [17]

    H. Morgan. AI governance control stack for operational stability: Achieving hardened governance in AI systems.https://doi.org/10.48550/arXiv.2604.03262, 2026

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2604.03262 2026

  18. [18]

    Rahman and S

    R. Rahman and S. Tajdari. Detecting hidden ML training with zero-overhead telemetry. 2026

    2026

  19. [19]

    Spadacini

    D. Spadacini. Clinical MLOps: A framework for responsible deploy- ment and observability of AI systems in cloud-native healthcare platforms. https://doi.org/10.20944/preprints202602.1951.v1, 2026

    arXiv 1951

  20. [20]

    Vigeant, A

    T. Vigeant, A. Tam, Q. Shi, J. Allison, M. Kim, J.-A. Sim, D.O. Won, D. Shin, D.M. McManus, J.J. Lee, J.Y . Yu, and A.H. Zai. A structured comparison of the coalition for health AI responsible AI guide and south korea’s trustworthy AI guideline for health care AI assurance: Comparative framework analysis.https://doi.org/10.2196/86220, 2026

    work page doi:10.2196/86220 2026

  21. [21]

    E. Meyman. Observability is not enforcement: A doctrinal and architectural framework for distinguishing compliance instrumentation from runtime authorization in AI governance. https://doi.org/10.2139/ssrn.6249979, 2026

    work page doi:10.2139/ssrn.6249979 2026

  22. [22]

    Ugarte, M.Á.P

    R.C. Ugarte, M.Á.P. Guisado, A.B. de Jesús, and J.M.M. López. Making AI compliance evidence machine-readable.https://doi.org/10.48550/arXiv.2604.13767, 2026

    Pith/arXiv arXiv doi:10.48550/arxiv.2604.13767 2026

  23. [23]

    Y . Yu. Backcasting the trust gap: A strategic road map for clinician adoption of AI diagnostics by 2040.https://doi.org/10.2196/94234, 2026

    work page doi:10.2196/94234 2040

  24. [24]

    Seth and V .K

    P. Seth and V .K. Sankarapu. Bridging the gap in XAI-why reliable metrics matter for explain- ability and compliance.https://doi.org/10.48550/arXiv.2502.04695, 2025

    doi:10.48550/arxiv.2502.04695 2025

  25. [25]

    False Sense of Security in Explainable Artificial Intelligence (XAI)

    N.C. Chung, Hongkyou Chung, H. Lee, L. Brocki, Hongbeom Chung, and G. Dyer. False sense of security in explainable artificial intelligence (XAI). https://doi.org/10.48550/arXiv.2405.03820, 2024

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2405.03820 2024

  26. [26]

    Siala and Y

    H. Siala and Y . Wang. Shifting artificial intelligence to be responsible in healthcare: A systematic review.https://doi.org/10.1016/j.socscimed.2022.114782, 2022

    arXiv 2022

  27. [27]

    Darena, and P.K

    Arpit Tripathi, Ankit Tripathi, F. Darena, and P.K. Mishra. Mapping the use of large language models in hiring decisions: a scoping review.https://doi.org/10.3389/frai.2026.1798519, 2026

    arXiv 2026

  28. [28]

    Varnosfaderani and M

    S.M. Varnosfaderani and M. Forouzanfar. The role of AI in hospitals and clinics: Transforming healthcare in the 21st century.https://doi.org/10.3390/bioengineering11040337, 2024

    doi:10.3390/bioengineering11040337 2024

  29. [29]

    DeShazer

    C. DeShazer. The governance gap no one is solving. https://doi.org/10.1016/j.jnma.2026.03.009, 2026. 22 Traccia: An OpenTelemetry-Based Governance Platform for AI Systems

    work page doi:10.1016/j.jnma.2026.03.009 2026

  30. [30]

    Morley, L

    J. Morley, L. Murphy, A. Mishra, I. Joshi, and K. Karpathakis. Governing data and artificial intelligence for health care: Developing an international understanding. https://doi.org/10.2196/31623, 2022

    work page doi:10.2196/31623 2022

  31. [31]

    E. Ferrara. Defeat devices in AI systems.https://doi.org/10.20944/preprints202606.0604.v1, 2026

    arXiv 2026

  32. [32]

    Sumner, Y .W

    J. Sumner, Y .W. Lim, T. Tan, A. Makmur, P. Forbes, J. Phua, L.S. Quek, and C.L.Y . Ngoh. The medivoice implementation journey: ambient artificial intelligence for clinical documentation. https://doi.org/10.3389/fdgth.2026.1764465, 2026

    arXiv 2026

  33. [33]

    Gökta¸ s and A

    P. Gökta¸ s and A. Grzybowski. Shaping the future of healthcare: Ethical clinical challenges and pathways to trustworthy AI.https://doi.org/10.3390/jcm14051605, 2025

    doi:10.3390/jcm14051605 2025

  34. [34]

    Idowu, A

    J. Idowu, A. Almasoud, and A. Alfahid. Mapping human anti-collusion mechanisms to multi-agent AI systems.https://doi.org/10.1016/j.knosys.2026.116067, 2026

    arXiv 2026

  35. [35]

    B. Sunu. Persistent identity in LLM multi-agent systems: A semantic consensus framework for heterogeneous AI swarms.https://doi.org/10.2139/ssrn.6423398, 2026

    work page doi:10.2139/ssrn.6423398 2026

  36. [36]

    J.Y . Kim, A. Hasan, S. Balu, and M. Sendak. People process technology and operations framework for establishing AI governance in healthcare organizations. https://doi.org/10.1038/s41746-026-02419-6, 2026

    work page doi:10.1038/s41746-026-02419-6 2026

  37. [37]

    Unver and L

    M.B. Unver and L. Roddeck. Ethics governance of AI for the legal sector: https://doi.org/10.21552/aire/2024/2/5. 2024

    work page doi:10.21552/aire/2024/2/5 2024

  38. [38]

    H.N. Cho, J. Wang, D. Hu, and K. Zheng. Large language model-based chatbots and agentic AI for mental health counseling: Systematic review of methodologies, evaluation frameworks, and ethical safeguards.https://doi.org/10.2196/80348, 2026

    work page doi:10.2196/80348 2026

  39. [39]

    F. Härer. Specification and evaluation of multi-agent LLM systems - prototype and cybersecu- rity applications.10.1109/cyber-ai66431.2025.11233474, 2025

    arXiv 2025

  40. [40]

    Williamson and V .R

    S. Williamson and V .R. Prybutok. Balancing privacy and progress: A review of pri- vacy challenges, systemic oversight, and patient perceptions in AI-driven healthcare. https://doi.org/10.3390/app14020675, 2024

    work page doi:10.3390/app14020675 2024

  41. [41]

    de Mello, M

    V .O. de Mello, M. Inzlicht, and J. Plaks. The effects of AI anthropomorphism on trust and responsibility.https://doi.org/10.1525/collabra.161757, 2026

    work page doi:10.1525/collabra.161757 2026

  42. [42]

    Y . Sharma. Maltopic: Multi-agent LLM topic modeling framework. https://doi.org/10.1109/aiiot65859.2025.11105319, 2025

    arXiv 2025

  43. [43]

    J. Tian, Z. Zhao, L. Tang, Y . Song, Y . Li, and N. Jiang. From pilot trap to institutional capacity: A governance framework for sustainable clinical AI implementation in health systems.https://doi.org/10.2196/92680, 2026

    work page doi:10.2196/92680 2026

  44. [44]

    Mökander, J

    J. Mökander, J. Schuett, H.R. Kirk, and L. Floridi. Auditing large language models: a three-layered approach.https://doi.org/10.2139/ssrn.4361607, 2023

    doi:10.2139/ssrn.4361607 2023

  45. [45]

    Q. Qian. Large language models in clinical trial recruitment: Sociotechnical and economic framework development study.https://doi.org/10.2196/95899, 2026

    work page doi:10.2196/95899 2026

  46. [46]

    Zahid, R

    A. Zahid, R. Sharma, and R. Ahmed. Public health responsible AI capability (ph- raic) framework: A conceptual model for integrating AI into public health agencies. https://doi.org/10.3390/healthcare14101364, 2026. 23 Traccia: An OpenTelemetry-Based Governance Platform for AI Systems

    work page doi:10.3390/healthcare14101364 2026

  47. [47]

    Zhang and A

    B. Zhang and A. Dafoe. Artificial intelligence: American attitudes and trends. https://doi.org/10.2139/ssrn.3312874, 2019

    doi:10.2139/ssrn.3312874 2019

  48. [48]

    Adapa and N

    K. Adapa and N. Jayakumar. Navigating AI governance for oncology nursing: Exist- ing models, implications, and a call for nurse-led oversight in asia pacific health systems. https://doi.org/10.1016/j.apjon.2026.100964, 2026

    arXiv 2026

  49. [49]

    Bateyko and K

    D. Bateyko and K. Levy. One bad nofo? AI governance in federal grantmaking. https://doi.org/10.1145/3715275.3732109, 2025

    arXiv 2025

  50. [51]

    Danks, B

    L. Danks, B. Semete-Makokotlela, C. Reynecke, S. Dijeng, B. Kgosietsile, J. van Tonder, C. van Graan, J. Hinch, A. Kruger, S. Walker, and S. Salek. Development and validation of an artificial intelligence proof-of-concept tool for risk-based quality assessment of generic medicines: a south african case study.https://doi.org/10.3389/fmed.2026.1811333, 2026

    arXiv 2026

  51. [52]

    Killoran and A

    J. Killoran and A. Park. Learn from the blame game when AI causes harm. https://doi.org/10.1073/pnas.2528408123, 2026

    work page doi:10.1073/pnas.2528408123 2026

  52. [53]

    K. Dadzie. Multi-layered alignment governance for AI models. https://doi.org/10.2139/ssrn.5345186, 2025

    work page doi:10.2139/ssrn.5345186 2025

  53. [54]

    S. Hatz. The nuclear analogy in AI governance research. https://doi.org/10.4337/9781035338580.00012, 2025

    arXiv 2025

  54. [55]

    Hazime and J

    J. Hazime and J. Farooq. Evaluation of LLM powered agentic AI for solving multi-arm bandit problems.https://doi.org/10.36227/techrxiv.175373072.24215212/v1, 2025

    arXiv 2025

  55. [57]

    O. Odubola. AI in social good: LLM-based multi-agent interventions in crisis management and disaster response.https://doi.org/10.51219/urforum.2024.oluwatimilehin-odubola, 2024

    work page doi:10.51219/urforum.2024.oluwatimilehin-odubola 2024

  56. [58]

    Bengio, G.E

    Y . Bengio, G.E. Hinton, A.C.-C. Yao, D. Song, P. Abbeel, T. Darrell, Y .N. Harari, Y .-Q. Zhang, X. Lan, S. Shalev-Shwartz, G.K. Hadfield, J. Clune, T. Maharaj, F. Hutter, A.G. Baydin, S.A. McIlraith, Q. Gao, A. Acharya, D. Krueger, A.D. Dragan, P.H.S. Torr, S. Russell, D. Kahneman, J. Brauner, and S. Mindermann. Managing extreme AI risks amid rapid prog...

    doi:10.1126/science.adn0117 2024

  57. [59]

    P. Yu, H. Xu, X. Hu, and C. Deng. Leveraging generative AI and large language models: A com- prehensive roadmap for healthcare integration.https://doi.org/10.3390/healthcare11202776, 2023

    work page doi:10.3390/healthcare11202776 2023

  58. [60]

    R. Binkyte. Interactional fairness in LLM multi-agent systems: An evaluation framework. https://doi.org/10.1609/aies.v8i1.36563, 2025

    work page doi:10.1609/aies.v8i1.36563 2025

  59. [61]

    B.-J. Kim, S. Jeong, B.-K. Cho, and J.-B. Chung. AI governance in the context of the eu AI act: A bibliometric and literature review approach.https://doi.org/10.1109/access.2025.3598023, 2025

    arXiv 2025

  60. [62]

    Akgün and M

    F.E. Akgün and M. Akgün. Governing generative AI in healthcare: A normative con- ceptual framework for epistemic authority, trust, and the architecture of responsibility. https://doi.org/10.3390/healthcare14081098, 2026. 24 Traccia: An OpenTelemetry-Based Governance Platform for AI Systems

    work page doi:10.3390/healthcare14081098 2026

  61. [63]

    Barak and I

    R. Barak and I. Wolf. The evolving physician-AI relationship: a five-tier frame- work for integrating intelligent systems into clinical practice and medical education. https://doi.org/10.1016/j.esmorw.2026.100719, 2026

    arXiv 2026

  62. [64]

    Dharmansyah, L

    D. Dharmansyah, L. Rahayuwati, I. Pramukti, and K. Mutyara. Privacy, security & gov- ernance frameworks for AI-powered wearable internet of health things in elderly care: A comprehensive review.https://doi.org/10.2147/rmhp.s606165, 2026

    work page doi:10.2147/rmhp.s606165 2026

  63. [65]

    Ramcharran, J.L

    D. Ramcharran, J.L. Painter, V . Kara, M. Glaser, M. Vanini, V .R. Chalamalasetti, C. Golds, A. Abdelkarim, A. Bate, and J.-U. Stegmann. Orchestrating gen- erative AI in pharmacovigilance: predicting and preempting the unpredictable. https://doi.org/10.1177/20420986251396023, 2025

    work page doi:10.1177/20420986251396023 2025

  64. [66]

    Minkkinen, A

    M. Minkkinen, A. Niukkanen, and M. Mäntymäki. What about investors? ESG analyses as tools for ethics-based AI auditing.https://doi.org/10.1007/s00146-022-01415-0, 2022

    work page doi:10.1007/s00146-022-01415-0 2022

  65. [67]

    Korinek and A

    A. Korinek and A. Balwit. Aligned with whom? direct and social goals for AI systems. https://doi.org/10.3386/w30017, 2022

    work page doi:10.3386/w30017 2022

  66. [68]

    Z. Xie, H. Wang, L. Dai, Z. Wang, H. Song, and J. Qian. Ethical issues in multi-agent AI systems for healthcare: a narrative review.https://doi.org/10.3389/fpubh.2026.1792627, 2026

    arXiv 2026

  67. [69]

    A Taxonomy for Design and Evaluation of Prompt-Based Natural Language Explanations

    I. Nejadgholi, M. Omidyeganeh, M.-A. Drouin, and J. Boisvert. A taxon- omy for design and evaluation of prompt-based natural language explanations. https://doi.org/10.48550/arXiv.2507.10585, 2025

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2507.10585 2025

  68. [70]

    Venugopal, P

    P. Venugopal, P. Garg, C. Prakash, S. Kumar, and N. Gupta. Evaluating AI adoption challenges in healthcare using a multi-criteria decision-making approach: implications for predictive risk analytics.https://doi.org/10.3389/frai.2026.1827586, 2026

    arXiv 2026

  69. [71]

    J. Truby. Governing artificial intelligence to benefit the un sustainable development goals. https://doi.org/10.1002/sd.2048, 2020

    work page doi:10.1002/sd.2048 2048

  70. [72]

    Wawer and J

    M. Wawer and J. Chudziak. Integrating traditional technical analysis with AI: A multi-agent LLM-based approach to stock market forecasting.https://doi.org/10.5220/0013191200003890, 2025

    work page doi:10.5220/0013191200003890 2025

  71. [73]

    Aggarwal

    N. Aggarwal. AI, fintech, and the evolving regulation of consumer financial privacy. https://doi.org/10.1093/oxfordhb/9780197579329.013.51, 2023

    arXiv 2023

  72. [74]

    Debelak, P

    K. Debelak, P. Pevcin, and R. Hržica. AI spring and its regulation discourse: a bibliometric study of trends in literature.https://doi.org/10.3389/frai.2026.1762748, 2026

    arXiv 2026

  73. [75]

    Ruschemeier

    H. Ruschemeier. Experimental regulation for AI governance with regulatory sandboxes. https://doi.org/10.1017/cfl.2025.10035, 2025

    arXiv 2025

  74. [76]

    T. Huang. Innovative research paradigms, AI models, and autonomous robots at the 2025 world artificial intelligence conference.https://doi.org/10.1016/j.xinn.2025.101082, 2025

    arXiv 2025

  75. [77]

    H. Yu, Z. Shen, C. Miao, C. Leung, V . Lesser, and Q. Yang. Building ethics into artificial intelligence.https://doi.org/10.24963/ijcai.2018/779, 2018

    work page doi:10.24963/ijcai.2018/779 2018

  76. [78]

    M. Chen. Evaluating large language models trained on code. https://doi.org/10.48550/arXiv.2107.03374, 2021

    Pith/arXiv arXiv doi:10.48550/arxiv.2107.03374 2021

  77. [79]

    Batool, D

    A. Batool, D. Zowghi, and M. Bano. AI governance: a systematic literature review. https://doi.org/10.1007/s43681-024-00653-w, 2025. 25 Traccia: An OpenTelemetry-Based Governance Platform for AI Systems

    doi:10.1007/s43681-024-00653-w 2025

  78. [80]

    N. Fabiano. The AI legal specialist: A juridically autonomous professional profile for AI governance.https://doi.org/10.48550/arXiv.2606.12415, 2026

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2606.12415 2026

  79. [81]

    Jarvers and O

    S. Jarvers and O. Papakyriakopoulos. Engaged AI governance: Addressing the last mile challenge through internal expert collaboration.https://doi.org/10.1145/3805689.3812341, 2026

    arXiv 2026

  80. [82]

    hÉigeartaigh, J

    S.Ó. hÉigeartaigh, J. Whittlestone, Y . Liu, Y . Zeng, and Z. Liu. Overcoming barriers to cross-cultural cooperation in AI ethics and governance. 2020

    2020

Showing first 80 references.