Pith. sign in

REVIEW 3 major objections 4 minor 23 references

Correct Is Not Governed: Provenance Integrity in Agentic Workflows

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Task correctness and governed execution are different properties, and a deterministic causal-state layer can enforce the governance side independently of model judgment.

desk verdict A bounded but honest case that correct outcomes and governed execution are different properties; worth refereeing, though external validity is still fixture-specific. read the letter →

arxiv 2608.12761 v1 pith:IY3CONSY submitted 2026-08-13 cs.AI cs.CR

Jesus Salas This is my paper · ORCID
classification cs.AIcs.CR
keywords provenancegovernedexecutionagenticworkflowsauditabilitydeterministicenforcementcompletionverificationdependencyscopingcontracttransfer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that evaluating agentic workflows by whether they reach the correct outcome is insufficient for institutional settings. A correct action may cite the wrong authority, claim completion without supporting evidence, or rely on work that a later change made stale. The author defines governed execution as work whose decisions, completion, and response to change carry explicit, independently checkable provenance, and presents Matrix, a deterministic layer that records authority and fact dependencies, verifies completion receipts against declared definitions of done, and selectively invalidates stale work. In controlled comparisons, governed and direct workflows reached the same outcomes, but only the governed path preserved governing evidence, refused unsupported closure, and limited recovery to dependent tasks. A transfer test failed: a deterministically enforced completeness contract over-blocked independently produced packets, showing that mechanism fidelity does not establish contract validity.

What carries the argument

The central object is Matrix, a deterministic causal-state layer built on an append-only event log with six first-class sets—versioned authority, accepted facts, admitted decisions, issued tasks, completion receipts, and supervisory verdicts—plus typed read/produce dependency edges. Models continue to interpret documents and propose plans; Matrix owns identity, transition validity, declared preconditions, receipt matching, bounded rework, and replay. Its lifecycle records applicable authority and facts, admits or rejects a proposed decision, issues tasks only after preconditions, attaches every attempt to a receipt, verifies receipts against definitions of done, emits supervisor verdicts, and on authority or fact change invalidates the dependent set and reissues only affected obligations. This machinery is what makes the three provenance forms measurable separately from outcome correctness.

What would settle it

Run the frozen completeness contract from the transfer experiment on a fresh, independently authored set of synthetic packets that was never exposed to the contract's authors; if specificity rises well above zero on reviewer-complete packets, the transfer failure was an artifact of the original packet set, whereas if it stays near zero, the claim that determinism does not confer contract validity holds.

Watch

Extended reading notes

Core claim

The paper's central claim is that task correctness and governed execution are different properties. A workflow is governed only when the authority behind its decisions, the evidence behind its completion, and the effects of later change are explicit and independently checkable. The empirical pattern is matched pairs: governed and direct workflows reach the same business outcome while differing on provenance—direct RAG cited non-governing records in six of fifteen runs that all chose the expected action; both recovery arms stayed current after an amendment but dependency-scoped recovery reissued three tasks versus eighteen; a generic self-review produced complete plans but the direct control self-closed on an unsupported completion claim. The paper also reports a controlling negative result: a completeness contract enforced with perfect sensitivity blocked every reviewer-complete packet in the transfer set, separating faithful mechanism from valid contract.

Load-bearing premise

The governance inputs—definitions of done, authority and applicability metadata, dependency declarations, and completeness contracts—are assumed to capture the real institutional requirement, because the deterministic layer can enforce a wrong contract with perfect fidelity.

Editorial extensions

If this is right

  • Evaluation of agentic workflows should report provenance properties (citation soundness, receipt-backed closure, dependency-scoped recovery) alongside outcome accuracy, because matched outcomes can still hide governance failures.
  • Deterministic receipt verification offers a practical answer to self-certified completion: an agent's own completion claim is never sufficient, and failed attempts remain in history.
  • Selective invalidation after policy change can preserve unaffected proof and reduce rework without leaving stale residue, compared with conservative full reruns.
  • Mechanism fidelity to a declared contract is not validity: a completeness contract can be perfectly enforced yet severely over-block inputs produced outside its authoring context.
  • The system's role is an institutional integrity layer—making work auditable and independently verifiable—not a general accuracy enhancer.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If correct-is-not-governed becomes an accepted evaluation standard, audit and compliance regimes that require traceability would naturally demand provenance layers of this kind, making governing evidence a first-class system output rather than a transcript byproduct.
  • The separation suggests a division of labor: models can remain fallible at semantic judgment while deterministic infrastructure owns identity, transitions, and evidence gates; a human or model verifier could supply the semantic judgments the contracts miss.
  • The transfer failure implies a testable engineering rule: completeness and applicability contracts should be authored by parties independent of packet production, and every invariant family should be evaluated on both incomplete and complete held-out cases to avoid over-blocking.
  • The dependency-based recovery mechanism could be extended to cost metrics: with accurate dependency declarations, the observed churn reduction (from eighteen to three reissued tasks) suggests potential savings in reexecution and review burden, but only where declarations are complete.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper argues that evaluating agentic workflows by business outcome alone is insufficient for institutional settings, and it defines 'governed execution' as the conjunction of decision provenance, execution provenance, and change provenance. It introduces Matrix, a deterministic causal-state layer that records versioned authority and facts, verifies completion receipts against definitions of done, and performs dependency-scoped invalidation after changes. The evaluation uses small, self-authored synthetic fixtures with repeated model seeds: Study 1 shows matched expected actions with diverging citation integrity; Study 2 shows self-closure versus receipt-verified closure; Study 3 shows tied currentness with different recovery scope; Study 4 shows a metadata-decidable refusal invariant; Study 5 composes the mechanisms; and Study 6 reports a controlling negative result in which a deterministically enforced completeness contract over-blocks on a role-separated transfer set. The paper's central claim is that task correctness and governed execution are different properties, and it explicitly disclaims that Matrix is a general accuracy enhancer or an external-validity guarantee.

Significance. If the framing holds, this is a useful conceptual contribution to the agent-auditability literature: it separates outcome correctness from the inspectability of decisions, completion evidence, and change propagation, and it does so with an implemented prototype and a matched-evaluation design. The paper is unusually transparent about limitations, including frozen claim-evidence maps, post-hoc specificity amendments, deterministic assertions rather than independent samples, and the failed contract-transfer result. The negative Study 6 result is a genuine strength: it distinguishes mechanism fidelity from contract validity and prevents overclaiming from the positive mechanism demonstrations. The contribution is modest but real, and the reproducibility-oriented reporting discipline is exemplary.

major comments (3)
  1. [Section 4.2, Table 3; Section 6.3] The Study 1 comparison is presented as evidence for the deterministic layer, but the governed condition differs from direct RAG not only by the presence of Matrix but also by an additional applicability adjudication call (21 vs 15 condition-specific calls for model-signaled governance), by prompt specialization in that adjudicator, and by a rewired action-revision pipeline. The paper discloses the call asymmetry and in Section 6.3 says the evidence cannot distinguish decomposition, prompt specialization, or persistent binding, but the abstract and conclusion still attribute the provenance preservation pattern to the governed path without this caveat. Please present Study 1 explicitly as a system-level comparison of direct RAG against governed adjudication-plus-binding, and place the causal attribution for the deterministic layer on Studies 2 and 3, where the controls are more tightly matched.
  2. [Section 3.5; Appendix A] The paper acknowledges that 'contract fidelity is partly by construction,' but several result sections—notably Section 5.3's description of 'refused unsupported closure' and Section 5.5's 'contract fidelity'—report the outcome without repeating this caveat. Because definitions of done, dependency edges, and completeness contracts are authored by the same team that designed Matrix, the positive enforcement results demonstrate that the code implements the declared rules, not that those rules capture an external institutional requirement. Study 6 shows exactly why this distinction matters. Please add an explicit sentence at the start of Sections 5.3 and 5.5 stating that all positive enforcement results are relative to author-supplied contracts, and that no claim of institutional validity is intended.
  3. [Section 4.8, Section 5.8, Abstract] The 'role-separated transfer challenge' is implemented as procedural information separation on a shared filesystem with fresh model-role reviewers, not as independent human authorship or review. The manuscript discloses this in Section 4.8, yet the abstract says packets 'produced outside its authoring context' and the conclusion says 'independently contextualized synthetic packets.' Please replace 'independently' with 'procedurally separated' in claim-bearing sentences, and state in Section 5.8 that the negative result establishes contract non-transfer under simulated role separation—stronger than no test, but weaker than independent human evaluation.
minor comments (4)
  1. [Section 4.1] The frozen claim-evidence map and reproducibility artifact are good practices, but the manuscript does not include a stable archive identifier; please add the repository or DOI to the camera-ready version so the frozen run reports can be verified.
  2. [Section 5.5, Table 6] The specificity value of 1.00 in Table 6 is from a post-hoc amendment on the same authored fixture, as the text explains; please add a footnote to the table itself clarifying that this is a degenerate-implementation check, not a population estimate.
  3. [Section 5.8] The phrase 'reviewer-complete packets' should specify that the reviewers are fresh model contexts under procedural role separation, not human reviewers, to avoid overstatement in the main text.
  4. [General] There are several missing spaces around the 'Matrix' macro in the abstract and body text (e.g., 'We presentMatrix'), which should be fixed in the final formatting pass.

Circularity Check

2 steps flagged · score 4.0 of 10

Positive mechanism results in Studies 2 and 4 reduce to the paper's own operational definitions; the central correctness-versus-governance claim retains independent empirical content.

  1. self definitional [Section 2.3 and Table 4 (Study 2, C4–C5)]
    "The task is complete only when an independent verifier accepts Eτ against DoDτ under the current authority and fact state. The executing agent's statement that it completed τ is an input claim, not sufficient evidence."

    Table 4 reports 'Valid completion 0/3 after self-closing 3/3' for the direct control versus Matrix's 'independently verified it, and closed 3/3'. But 'valid completion' is defined in Section 2.3 as acceptance by an independent verifier, and Matrix is that verifier by design. The direct control is given no independent verifier, so the 0/3 vs 3/3 split on this metric is entailed by the operational definition rather than discovered. The empirical residue, that the same Phi plan after generic self-review self-closes 3/3, is a real model behavior and is not circular; the circularity is confined to the headline 'valid completion' framing.

  2. self definitional [Section 3.7 and Section 5.5 (Study 4, Experiment 6)]
    "If the visible candidate set lacks a required class, the kernel records the model proposal but blocks a confident ruling and makes escalation effective. ... The public completeness invariant blocked each confident ruling and made escalation effective in all 12."

    The 'structurally incomplete' controls are defined by the same public completeness contract whose rule the kernel mechanically enforces. Sensitivity of 12/12 therefore demonstrates that the code executes its own declared rule; it does not provide independent evidence that the contract captures the institutional completeness requirement. The paper explicitly concedes this boundary, noting that perfect fidelity does not show the contract is semantically correct, and offsets it with the independent Study 6 transfer failure, which is the non-circular, controlling negative result.

full rationale

The paper is unusually explicit about the limits of its mechanism demonstrations. It defines governed execution operationally in Section 2.5 as a conjunction of decision, execution, and change provenance, and then reports that Matrix, a system built to enforce exactly those provenance checks, satisfies them. The positive results in Study 2 ('valid completion') and Study 4 ('structurally incomplete' refusal) are partly by construction, and Appendix A admits this directly: 'Contract fidelity is partly by construction. Receipt gates, dependency traversal, and metadata-completeness checks are designed to implement declared contracts.' That is disclosed circularity in the mechanism claims, not a hidden prediction error. However, the paper's central claim, that task correctness and governed execution are different properties, does not reduce to this definition. Study 1 shows direct RAG tying expected action 15/15 while failing citation integrity 6/15 on an externally scored metric; Study 3 shows both conditions current but reexecution scope differing 18 vs 3; and Study 6 supplies an independent transfer failure where a deterministically enforced contract over-blocks. These are empirical separations, not definitional equivalences. There is no load-bearing self-citation chain, no imported uniqueness theorem, and no renamed known result. Because the positive mechanism results are partly definitional but the central conceptual separation and the transfer-failure control are independently grounded, a score of 4 reflects partial, disclosed circularity rather than a forced or fully circular derivation.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The positive demonstrations depend on hand-authored contracts and dependency declarations rather than measured or externally validated inputs. The paper acknowledges that contract fidelity is partly by construction and that missing dependency edges can leave stale residue (Appendix A). No new theoretical entities are introduced; Matrix is a software artifact described as implemented.

free parameters (4)
  • Definition-of-done contracts for receipt verification = Authored per fixture, e.g., DPA countersignature requirement in C4/C5
    Task closure depends entirely on the authored DoD. The paper notes Matrix can implement a contract but cannot establish that it captures every semantic requirement (Section 3.5).
  • Completeness contracts for structural refusal = Required authority classes in Experiment 6/7
    The public completeness contract is authored and is shown to over-block in the transfer test (specificity 0.00), so the specific requirements are a hand-set parameter that determines refusal behavior (Sections 4.5, 5.8).
  • Retry allowance / bounded rework counter = One retry in C5; escalation on exhaustion
    The bounded rework policy is a chosen parameter that determines when rework becomes escalation; it affects closure behavior (Sections 3.5, 5.3).
  • Retrieval top-K = 4
    The retrieval depth is a hand-set experimental constant; retrieval recall is complete on the applicable-policy runs, but the choice of K influences which records are available for citation (Section 4.2).
assumptions (4)
  • ad hoc to paper The operational definition of governed execution as the conjunction of decision, execution, and change provenance is a valid characterization of institutional governance.
    The paper states 'This is an operational definition, not a theorem' (Section 2.5); the entire evaluation presupposes that these three provenance properties are the ones that matter for institutional accountability.
  • ad hoc to paper The six first-class sets (authority, facts, decisions, tasks, receipts, verdicts) and typed dependency edges fully represent the governed-work state.
    Section 2.1 and Table 1 define the lifecycle model; the paper notes missing edges can leave stale residue and overly broad edges erase the churn benefit (Appendix A).
  • domain assumption Authority and applicability metadata on records is accurate and machine-readable.
    Study 1 relies on records carrying authority, status, and applicability tags; if this metadata is wrong, citation soundness fails regardless of the governance layer (Section 4.2).
  • domain assumption The evaluated models, mainly the quantized local Phi-4 checkpoint, are representative enough to demonstrate the proposed mechanisms.
    Appendix A notes the lifecycle ladders use one quantized local Phi-4 checkpoint and the paper does not estimate model-family interactions or a scaling curve.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Correct Is Not Governed: Provenance Integrity in Agentic Workflows." pith.science (2026). https://pith.science/paper/IY3CONSY

@misc{pith2026260812761,
  author       = {Pith},
  title        = {Pith review of: Correct Is Not Governed: Provenance Integrity in Agentic Workflows},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IY3CONSY}},
  note         = {Machine review of arXiv:2608.12761}
}
read the original abstract

Agentic workflows are commonly evaluated by whether they reach the correct outcome. That is insufficient in institutional settings, where a correct action may rely on the wrong authority, an unsupported completion claim, or work made stale by a later change. We define governed execution as work whose decisions, completion, and response to change are supported by inspectable provenance. We present Matrix, a deterministic causal-state layer that records authority and fact dependencies, verifies completion evidence, and selectively invalidates affected work. Across controlled comparisons, governed and direct workflows often reached the same outcomes, but only the governed path consistently preserved governing evidence, refused unsupported closure, and limited recovery to dependent tasks. A role-separated transfer challenge then failed: a deterministically enforced completeness contract severely over-blocked synthetic packets produced outside its authoring context. These results do not establish Matrix as a general accuracy enhancer; they support its primary role as an institutional integrity layer for making agentic work auditable and independently verifiable.

Figures

Figures reproduced from arXiv: 2608.12761 by the authors.

Figure 1
Figure 1. Implemented causal-state architecture. Model and human roles submit typed proposals; the [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Headline matched comparisons. Left: expected-action accuracy ties at 15/15, while runs without [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 11 canonical work pages

  1. [9]

    Accessed 2026-08-11

    URLhttps://docs.langchain.com/oss/python/langgraph/ persistence. Accessed 2026-08-11. Martin Leucker and Christian Schallhart. A brief account of runtime verification.The Journal of Logic and Algebraic Programming, 78(5):293–303,

    work page 2026

  2. [12]

    Enabling large language models to generate text with citations

    Tianyu Gao, Howard Yen, Jiatong Yu, and Danqi Chen. Enabling large language models to generate text with citations. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6465–6488. Association for Computational Linguistics,

    work page 2023

  3. [13]

    Auditing Provenance Sensitivity in LLM Agent Action Selection

    URLhttps://arxiv. org/abs/2607.20827. Yanming Liu, Xinyue Peng, Jiannan Cao, Xinyi Wang, Songhang Deng, Jintao Chen, Jianwei Yin, and Xuhong Zhang. ToolGate: Contract-grounded and verified tool execution for LLMs. InFindings of the Association for Computational Linguistics: ACL 2026, pages9653–9684.AssociationforComputationalLinguistics,

    work page Pith review arXiv 2026

  4. [14]

    URLhttps://aclanthology.org/2026.findings-acl.470/

    doi: 10.18653/v1/2026.findings-acl.470. URLhttps://aclanthology.org/2026.findings-acl.470/. Microsoft. Microsoft agent framework workflows,

    work page doi:10.18653/v1/2026.findings-acl.470 2026

  5. [15]

    Accessed 2026-08-11

    URL https://learn.microsoft.com/en-us/ agent-framework/workflows/. Accessed 2026-08-11. Luc Moreau and Paolo Missier. PROV-DM: The PROV data model. W3c recommendation, World Wide Web Consortium,

    work page 2026

  6. [16]

    OpenPolicyAgent

    URL https: //arxiv.org/abs/2604.05485. OpenPolicyAgent. Openpolicyagentdocumentation,

    arXiv

  7. [17]

    Accessed 2026-08-11

    URL https://www.openpolicyagent.org/docs. Accessed 2026-08-11. Josh Rosen and Seth Rosen. From agent loops to deterministic graphs: Execution lineage for reproducible AI-native work,

    work page 2026

  8. [18]

    Igor Santos-Grueiro

    URLhttps://arxiv.org/abs/2605.06365. Igor Santos-Grueiro. Temporary authority, permanent effects: Commit-time authorization for LLM agents,

    arXiv

Show all 23 references
  1. [19]

    Yining She, Yiliang Liang, and Eunsuk Kang

    URLhttps://arxiv.org/abs/2607.10487. Yining She, Yiliang Liang, and Eunsuk Kang. Safeguarding LLM agents from misalignment through provenance analysis,

    arXiv

  2. [20]

    Kaya Stechly, Karthik Valmeekam, and Subbarao Kambhampati

    URLhttps://arxiv.org/abs/2607.01236. Kaya Stechly, Karthik Valmeekam, and Subbarao Kambhampati. On the self-verification limita- tions of large language models on reasoning and planning tasks. InInternational Conference on Learning Representations,

    arXiv

  3. [21]

    URLhttps://doi.org/10.6028/NIST.AI. 100-1. Haoyu Wang, Christopher M. Poskitt, and Jun Sun. AgentSpec: Customizable runtime enforcement for safe and reliable LLM agents,

    doi

  4. [22]

    Ruoyu Xu, Gaoxiang Li, and Victor S

    URLhttps://arxiv.org/abs/2503.18666. Ruoyu Xu, Gaoxiang Li, and Victor S. Sheng. GAVEL: Evidence-contract debate with mechanized scrutiny for provenance-grounded fact-checking. InFindings of the Association for Computational Linguistics: ACL 2026, pages 35907–35920. Associatio...

    2026 arXiv

  5. [23]

    findings-acl.1789

    doi: 10.18653/v1/2026. findings-acl.1789. URLhttps://aclanthology.org/2026.findings-acl.1789/. A Detailed Evidence Boundaries Post-hoc paper framing.The experiments were not preregistered as one three-provenance study. The paper recomposes existing frozen artifacts after their...

    2026 doi

  6. [1979]

    European Union

    doi: 10.1016/ 0004-3702(79)90008-0. European Union. Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence,

    2024

  7. [1985]

    Johan de Kleer

    doi: 10.2307/2274239. Johan de Kleer. An assumption-based TMS.Artificial Intelligence, 28(2):127–162,

    doi

  8. [2007]

    Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou

    doi: 10.1145/1265530.1265535. Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet. InInternational Conference on Learning Representations,

  9. [2009]

    doi: 10.1016/j.jlap.2008.08.004. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval- augmented generation for knowledge-intensiv...

    2008 doi

  10. [2012]

    Maurits Kaptein, Vassilis-Javed Khan, and Andriy Podstavnychy

    doi: 10.1007/978-3-642-28108-2_19. Maurits Kaptein, Vassilis-Javed Khan, and Andriy Podstavnychy. Runtime governance for AI agents: Policies on paths,

    doi

  11. [2020]

    Junchi Liao

    URLhttps://proceedings.neurips.cc/paper/2020/hash/ 6b493230205f780e1bc26945df7481e5-Abstract.html. Junchi Liao. Auditing provenance sensitivity in LLM agent action selection,

    2020

  12. [2023]

    URLhttps://aclanthology.org/2023.emnlp-main.398/

    doi: 10.18653/v1/2023.emnlp-main.398. URLhttps://aclanthology.org/2023.emnlp-main.398/. Todd J. Green, Grigoris Karvounarakis, and Val Tannen. Provenance semirings. InProceedings of the Twenty-Sixth ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, pages 31...

    2023 doi

  13. [2024]

    URLhttps://eur-lex.europa.eu/eli/reg/2024/1689/oj. Article

    2024

  14. [2025]

    Elham Tabassi

    URLhttps://proceedings.iclr.cc/paper_files/paper/2025/file/ f3c5e56274140e0420baa3916c529210-Paper-Conference.pdf. Elham Tabassi. Artificial intelligence risk management framework (AI RMF 1.0). Technical Report NIST AI 100-1, National Institute of Standards and Technology,

    2025

  15. [2026]

    LangChain

    URLhttps://arxiv.org/abs/2603.16586. LangChain. LangGraph persistence,

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.