Pith. sign in

REVIEW 3 major objections 5 minor 21 references

Workflow as Knowledge: Semantic Persistence for LLM-Mediated Workflows

T0 review · 3 major / 5 minor · reviewed 2026-07-10 · glm-5.2

Pith's one-line read Treat the workflow itself as knowledge, not just a process that produces it

desk verdict Solid conceptual vocabulary for LLM workflow persistence, but the operational gap between 'semantic persistence' and provenance-plus-checkpointing is not demonstrated read the letter →

arxiv 2607.08740 v1 pith:76RVDHLW submitted 2026-07-09 cs.AI cs.PLcs.SE

classification cs.AIcs.PLcs.SE
keywords semanticpersistenceLLMworkflowsderive/inferdistinctionknowledgesubstrateworkflowprovenancehuman-in-the-loopagentharnessLisp-inspiredobjectmodel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that LLM-mediated workflows should be elevated from ephemeral execution scripts to first-class, persistent knowledge objects. It proposes a conceptual model—inspired by Lisp's symbolic traditions—where workflow definitions, running instances, LLM inference records, human approval events, context snapshots, and dependency links all inhabit a shared knowledge substrate with stable identity and queryability. The model's central architectural distinction is between derive (deterministic computation over known state) and infer (mediated LLM judgment under explicit context and executor-controlled capability policy). Under semantic persistence, a workflow does not merely run, produce outputs, and leave traces; the workflow itself—its definition, its live instance, its intermediate judgments—becomes inspectable, resumable, and reviewable knowledge that persists beyond any single execution run.

What carries the argument

The derive/infer boundary: derive is deterministic computation over available state; infer is mediated LLM judgment under declared context and executor-controlled capability policy. The executor mediates all model output, validating it before it can influence a declared branch or trigger an external effect. Semantic objects persist in a shared knowledge substrate with typed roles, stable identity, and queryability across execution boundaries.

What would settle it

Build two systems with identical workflows: one using the proposed semantic object model with pre-declared roles, and one using a standard checkpoint-and-trace system with post-hoc provenance export. If reviewers performing inspection, attribution, and audit tasks show no measurable improvement in accuracy or speed when using the semantic model, the core claim—that pre-declared semantic persistence adds value beyond execution persistence—is not supported.

Watch

Extended reading notes

Core claim

The paper introduces a semantic object schema in which workflow artifacts are assigned typed roles (workflow-definition, workflow-instance, inference-record, context-snapshot, approval-record, panel-record, dependency-link, and others) before execution begins, rather than being captured as after-the-fact provenance. The load-bearing mechanism is the derive/infer boundary: derive denotes deterministic, replayable computation over available workflow state, while infer denotes LLM-mediated judgment that requires declared context, a prompt, an expected return type, validation, persistence, and an explicit capability policy. The LLM fills declared infer slots but never directly controls workflow—

Load-bearing premise

The paper assumes that assigning declared semantic roles to workflow artifacts before execution provides meaningful benefits over provenance systems that capture the same information after the fact. If pre-declaration offers no practical advantage over post-hoc provenance, the model reduces to a vocabulary overlay on existing persistence mechanisms.

Editorial extensions

If this is right

  • If workflows persist as typed knowledge objects, then queries like 'show me every decision that depended on this document' or 'list all deferred deliberations' become substrate-level operations rather than custom provenance engineering.
  • The derive/infer split creates an auditable seam: every point where an LLM exercised judgment is explicitly marked, context-bounded, and linked to its downstream effects, making influence chains inspectable rather than hidden in execution traces.
  • Workflow instances as persistent objects enable semantic resumption: a prior panel decision or inference result can be reused when compatibility conditions are met, without re-running the entire workflow.
  • The model provides a vocabulary for distinguishing human authorization (approval) from structured deliberation (panel), giving review and escalation paths explicit semantic status rather than treating them as generic 'human steps.'

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the model were implemented, a natural test would be whether review tasks (attribution, audit, reproducibility) are measurably faster or more accurate when artifacts have pre-declared semantic roles versus when the same information is reconstructed from post-hoc provenance logs.
  • The model's treatment of the LLM as a 'mediated peripheral' rather than an autonomous agent suggests a design pattern where multiple LLM services could fill different infer slots in the same workflow under different capability policies, with the executor enforcing isolation and recording cross-inference dependencies.
  • The supersession-link concept (revised or disputed objects linked to their predecessors) could support a form of knowledge evolution tracking where changed premises trigger re-derivation of dependent objects while preserving the full history of judgment changes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a conceptual model in which LLM-mediated workflow definitions, running instances, inference records, context snapshots, and dependency relations are represented as persistent, typed knowledge objects in a shared 'knowledge substrate.' The central semantic distinction is between derive (deterministic computation over available state) and infer (mediated LLM judgment under declared context and executor-controlled capability policy). The model is Lisp-inspired but language-independent, drawing on live-image and symbolic-environment traditions. The paper defines a vocabulary of semantic object kinds (Table 1), operating primitives (Table 2), and provides a worked example (Appendix B) and a preliminary PROV-DM mapping (Appendix C). The authors are explicit that this is a conceptual proposal: there is no formal transition semantics, no implementation, and no empirical evaluation.

Significance. The paper addresses a genuine representational gap in current LLM workflow systems. While frameworks such as LangGraph, AgentSPEX, and DSPy make control flow explicit and provide checkpointing, they generally treat intermediate artifacts as execution traces rather than first-class knowledge objects with pre-declared semantic roles. The derive/infer distinction is a useful conceptual contribution: it forces workflow authors to make explicit which steps involve nondeterministic model judgment and which are deterministic, replayable computation. The panel/approval separation (Section 3.3, Table 1) is a thoughtful refinement that distinguishes authorization gates from structured deliberation. The preliminary PROV-DM mapping in Appendix C demonstrates provenance-compatibility. However, the paper's significance is currently limited by the absence of a concrete demonstration that the proposed semantic object model enables capabilities beyond what structured provenance plus checkpointing can achieve. The paper is honest about this limitation but does not resolve it.

major comments (3)
  1. [3.6] The paper's central differentiating claim is that 'semantic persistence' (pre-declaring semantic roles for workflow artifacts before execution) is meaningfully different from 'execution persistence' (checkpoints, traces, logs captured during or after execution). Section 3.6 states: 'the model assigns declared semantic roles to workflow definitions, workflow instances, inference records, and related objects before execution begins treating them as stored data.' However, the paper does not identify any specific operation, query, or review task that requires this pre-declaration and cannot be achieved through execution persistence plus structured provenance. The three affordances listed in Section 5 (queryable history, precise review, authority clarification) are all achievable with a checkpoint system that stores typed records and a provenance store that links them. The paper's own Table 3
  2. [3.4] The derive/infer distinction is described as 'central' (Section 3.4) and as having operational consequences: 'A derivation may depend on an inference, but the inference should remain visible as a dependency rather than being absorbed into the derived value.' However, the paper does not specify what breaks operationally if an inference is absorbed into a derived value without a visible dependency link. What review task becomes impossible? What query fails? What correctness property is violated? Without at least one concrete scenario where the distinction has operational consequences that cannot be replicated by simply tagging model outputs in a provenance store, the derive/infer boundary risks being a naming convention rather than a semantic boundary with enforceable consequences. The paper should either (a) identify a concrete task that requires the distinction, or (b) reframe the claim从
  3. [3.5] The scoped reuse rule in Section 3.5 — where the executor queries the knowledge substrate for a compatible prior panel-record and reuses its choice instead of re-presenting the deliberation — is presented as a feature of semantic persistence. But this is functionally equivalent to checkpoint-based resumption with compatibility checks, which LangGraph already supports (Section 2.6 describes LangGraph's 'time travel' and checkpoint-based persistence). The paper should clarify what the semantic object model adds beyond a typed interface over existing checkpoint mechanisms. If the answer is 'nothing operationally, but the model provides a cleaner conceptual framework,' that is a legitimate but weaker claim than the paper currently implies.
minor comments (5)
  1. [Section 4, Appendix D] The exploratory vocabulary scan of 77 artifacts is described as a 'vocabulary-design probe' and the authors are careful to state it is not empirical validation. However, the scan's methodology is underspecified: the scoring criteria (0-3 checklist) are not reproduced, and the selection criteria for the corpus are vague ('selected, qualitative, and heterogeneous'). A brief description of the scoring rubric would help readers assess whether the vocabulary refinements (e.g., separating approval from panel) are well-grounded.
  2. [Table 2] The 'capability / action' and 'handoff / promotion' primitives are marked as 'Candidate refinement' in the Associated object or relation column. This is confusing because it is unclear whether these are part of the proposed model or not. If they are candidates for future inclusion, they should be separated from the core vocabulary table or clearly annotated as provisional.
  3. [Section 3.4] The statement that 'Non-LLM nondeterminism, such as randomized algorithms or stochastic simulations, is not infer; it can be treated as derive when the seed, inputs, algorithm version, and replay policy are explicit' is reasonable but raises a boundary question: what about external API calls that return different results over time (e.g., a web search)? These are neither deterministic nor LLM-mediated judgment. The paper does not address this common case, and the derive/infer binary framing may be insufficient for it.
  4. [Appendix B] The pseudo-Lisp workflow definition is helpful as an illustration but would benefit from a brief comparison with how the same workflow would be expressed in an existing system (e.g., LangGraph). This would make the contribution more concrete and help readers understand what the semantic object model adds expressively.
  5. [Section 2.7] The term 'agent harness' is treated as practitioner framing rather than a settled technical concept. The paper cites O'Reilly and HumanLayer blog posts. While this is transparent, the term is used in the paper's own architecture description (Section 1). The paper should either define it more precisely or avoid relying on it as a technical term.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity detected: conceptual model with no fitted predictions, parameter-fitting claims, or self-citation chains.

full rationale

This paper is a conceptual model proposal that does not fit the circularity framework. It makes no fitted predictions, no parameter-fitting claims, and no derivations that reduce to inputs by construction. The model is defined axiomatically through its vocabulary (derive, infer, workflow-definition, workflow-instance, etc.) and its four claims are design commitments, not empirical or mathematical results. The paper explicitly states it is 'a conceptual model proposal, not an empirical study or a formal calculus' (Section 1). Self-citation is minimal: the authors cite external systems (LangGraph, AgentSPEX, DSPy, W3C PROV, Flows/aiFlows, WorkflowLLM) as prior art and do not invoke their own prior work as load-bearing justification. The Lisp-inspired notation is explicitly framed as an 'explanatory lens, not implementation commitment.' The derive/infer distinction is a definitional choice, not a derived result that could be circular. The PROV-DM mapping in Appendix C is comparative, not a claim of derivation from PROV. No step in the paper's argument chain reduces to its inputs by construction, and no self-citation chain is load-bearing. The derivation is self-contained against external benchmarks and prior work. Score: 0.

Assumptions & free parameters 1 free parameters · 4 assumptions · 3 invented entities

The paper is a conceptual model proposal. All axioms are design commitments argued for on conceptual grounds, not derived from prior results. The invented entities (knowledge substrate, panel-record, context-snapshot) are the paper's contributions but lack independent empirical or formal evidence. No free parameters exist because the model makes no quantitative claims.

free parameters (1)
  • None (conceptual model)
    The paper proposes a conceptual model with no fitted parameters, no empirical data, and no quantitative claims. The model's vocabulary is defined by design choice, not by fitting to data.
assumptions (4)
  • ad hoc to paper Workflow definitions, instances, and inference records can and should be represented as persistent typed knowledge objects in a shared substrate.
    Section 1, claim 1-2. This is the core design commitment the paper argues for, not a result derived from prior principles.
  • ad hoc to paper The derive/infer distinction is a meaningful and enforceable semantic boundary.
    Section 3.4. The paper acknowledges the boundary 'is not always obvious' and requires author discipline, but treats the distinction as axiomatic for the model.
  • ad hoc to paper Pre-declaring semantic roles for workflow artifacts before execution provides benefits over post-hoc provenance capture.
    Section 3.6. The paper contrasts its approach with provenance systems but does not formally prove or empirically demonstrate the advantage.
  • domain assumption Local structural flow (continue, repeat, stop, accept, reject, defer) is sufficient for the target class of workflows.
    Section 3.5. The DSL avoids general goto-like flow by assumption about the workflow domain.
invented entities (3)
  • Knowledge substrate
    purpose: Abstract semantic persistence layer giving workflow objects stable identity, typed roles, and queryability across execution boundaries.
    Section 3.3. The knowledge substrate is a conceptual abstraction; no implementation or empirical evidence of its properties is provided.
  • Panel-record
    purpose: Typed persistent object for structured human deliberation, distinct from simple approval gates.
    Section 3.3, Table 1. The panel-record is a new object kind introduced by the paper. The exploratory scan (Section 4) provides vocabulary motivation but not independent validation.
  • Context-snapshot
    purpose: Bounded material visible for an inference, approval, or decision, recorded for later review.
    Section 3.3, Table 1. Introduced as a semantic object kind; no implementation demonstrates its utility.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Workflow as Knowledge: Semantic Persistence for LLM-Mediated Workflows." pith.science (2026). https://pith.science/paper/76RVDHLW

@misc{pith2026260708740,
  author       = {Pith},
  title        = {Pith review of: Workflow as Knowledge: Semantic Persistence for LLM-Mediated Workflows},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/76RVDHLW}},
  note         = {Machine review of arXiv:2607.08740}
}
read the original abstract

Large language model (LLM) applications increasingly use explicit workflows for tool use, retrieval, branching, checkpointing, and human approval. Existing workflow systems already address many execution concerns. This paper proposes a Lisp-inspired but language-independent conceptual model: symbolic forms, object identity, and live-image thinking are used as explanatory lenses, not implementation commitments. In this model, workflow definitions, workflow instances, inference records, context snapshots, and dependency relations are represented as persistent knowledge objects in a shared knowledge substrate. Its central semantic distinction is between derive and infer: derive is deterministic computation over available state; infer is mediated LLM judgment under declared context and executor-controlled capability policy. The result is a preliminary conceptual account of semantic persistence: workflows do not merely produce knowledge and leave traces, but can themselves be represented as inspectable, resumable, and reviewable knowledge objects, while formal transition semantics remain future work.

Figures

Figures reproduced from arXiv: 2607.08740 by the authors.

Figure 1
Figure 1. summarizes the three conceptual layers and the mediation boundary between semantic objects, the DSL-machine control layer, and runtime services [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Execution persistence retains runnable state, checkpoints, logs, traces, and outputs; semantic persistence treats workflow definitions, workflow instances, inference records, and context snapshots as first￾class knowledge objects. 2.7 Agent Harnesses Recent practitioner discussions of agent harnesses make the same boundary visible from an engi￾neering direction: the system around the model shapes context, tool use, … view at source ↗
Figure 3
Figure 3. derive computes over available state; infer requests mediated LLM judgment whose recorded value may influence an executor-applied declared branch. 10 [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: The executor as controller of workflow instantiation, policy checks, runtime-mediated resources, transitions, and knowledge-substrate persistence. 12 [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

21 extracted references · 21 canonical work pages

  1. [1]

    A Comparative Study of DSL Code Generation: Fine-Tuning vs. Optimized Retrieval Augmentation

    https://arxiv.org/abs/2407.02742. AgentSPEX: An Agent SPecification and EXecution Language

  2. [2]

    A Provenance Model for Control-Flow Driven Scientific Workflows

    “A Provenance Model for Control-Flow Driven Scientific Workflows. ”Data & Knowledge Engineering 131–132: 101877. https://doi.org/10.1016/j.data k.2021.101877. Cabot, Jordi

  3. [3]

    UCB/EECS-2019-72

    Jupyter’s Archive: Searchable Output Histories for Computational Notebooks. UCB/EECS-2019-72. EECS Department, University of California, Berkeley. https: //www2.eecs.berkeley.edu/Pubs/TechRpts/2019/EECS-2019-72.html. Davidson, Susan B., and Juliana Freire

  4. [4]

    Provenance and Scientific Workflows: Challenges and Opportunities

    “Provenance and Scientific Workflows: Challenges and Opportunities. ” Proceedings of the 2008 ACM SIGMOD International Conference on Man- agement of Data . https://doi.org/10.1145/1376616.1376772. Ding, X. et al

  5. [5]

    https://doi.org/10.1145/3652620. 3687782. 37 Workflow as Knowledge: Semantic Persistence for LLM-Mediated Workflows Gabriel, Richard P., Jon L. White, and Daniel G. Bobrow

  6. [6]

    The Dexter Hypertext Reference Model

    “The Dexter Hypertext Reference Model. ” Communi- cations of the ACM 37 (2): 30–39. https://doi.org/10.1145/175235.175237. HumanLayer

  7. [7]

    Academic Research Skills for Claude Code . V. v3.13.0. Released. https: //doi.org/10.5281/zenodo.20696614. Josifoski, Martin, Lars Klein, Maxime Peyrard, et al

  8. [8]

    Flows: Building Blocks of Reasoning and Collaborating AI

    Flows: Building Blocks of Reasoning and Collaborating AI . https://arxiv.org/abs/2308.01285. Khattab, Omar et al

Show all 21 references
  1. [9]

    https://arxiv.org/abs/2310.03714

    DSPy: Compiling Declarative Language Model Calls into Self- Improving Pipelines. https://arxiv.org/abs/2310.03714. Kiczales, Gregor, Jim des Rivieres, and Daniel G. Bobrow

  2. [10]

    MIT Press

    The Art of the Metaobject Protocol. MIT Press. https://mitpress.mit.edu/9780262610742/the-art-of-the-metaobject-protocol/ . LangGraph

  3. [11]

    https: //arxiv.org/abs/2505.13453

    Pel: A Programming Language for Orchestrating AI Agents . https: //arxiv.org/abs/2505.13453. Mosqueira-Rey, Eduardo, Elena Hernández-Pereira, David Alonso-Ríos, José Bobes-Bascarán, and Ángel Fernández-Leal

  4. [12]

    Human-in-the-Loop Machine Learning: A State of the Art

    “Human-in-the-Loop Machine Learning: A State of the Art. ” Artificial Intelligence Review 56: 3005–54. https://doi.org/10.1007/s10462-022-10246-w . Nelson, Theodor H

  5. [13]

    Complex Information Processing: A File Structure for the Complex, the Changing and the Indeterminate

    “Complex Information Processing: A File Structure for the Complex, the Changing and the Indeterminate. ” Proceedings of the ACM 20th National Conference , 84–100. https://doi.org/10.1145/800197.806036. O’Reilly. 2026a. Agent Harness Engineering . O’Reilly Radar. https://www.or...

  6. [14]

    Recursive Multi-Agent Systems

    https://arxiv.org/abs/2512.24601. Recursive Multi-Agent Systems

  7. [15]

    Samuel, Sheeba, and Birgitta König-Ries

    https://arxiv.org/abs/2604.25917. Samuel, Sheeba, and Birgitta König-Ries

  8. [16]

    https://doi.org/10.1109/ACCESS.2018.2887

  9. [17]

    LLM Agents for Interactive Workflow Provenance: Reference Archi- tecture and Evaluation Methodology

    Souza, Renan et al. 2025a. “LLM Agents for Interactive Workflow Provenance: Reference Archi- tecture and Evaluation Methodology. ” WORKS at ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis . https://doi.org/10.1145/37 31599.376...

  10. [18]

    https://arxiv.org/abs/2506.10021

    From Tool Calling to Symbolic Thinking: LLMs in a Persistent Lisp Metaprogramming Loop. https://arxiv.org/abs/2506.10021. W3C

  11. [19]

    W3C Recommendation

    PROV-DM: The PROV Data Model . W3C Recommendation. https://www.w3.org/T R/2013/REC-prov-dm-20130430/ . Walker, Janet H., David A. Moon, Daniel L. Weinreb, and Mike McMahon

  12. [20]

    Yao, Shunyu, Jeffrey Zhao, Dian Yu, et al

    https://arxiv.org/abs/2411.05451. Yao, Shunyu, Jeffrey Zhao, Dian Yu, et al

  13. [21]

    https://arxiv.org/abs/2210.03629

    ReAct: Synergizing Reasoning and Acting in Language Models. https://arxiv.org/abs/2210.03629. 39

Pith tools

Reviewed July 10, 2026 · model on record in the stance chip above.