{"id":"ed458a4e-1426-4a48-9a75-4495fecf9f08","arxiv_id":"2608.07167","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Niyam-AI binds agent permissions with SHA-256 and adds zk-SNARK proofs for a small Judge model's safety decisions, reporting F1 88.5% on Agent-SafetyBench, though the classifier is benchmark-adapted and the proof applies only to a separate toy model.","lead":"A team proposes Niyam-AI, a framework that hashes an agent's allowed actions into an Intent Contract and generates a zero-knowledge proof for each tool call approved by a small Judge model. The idea is to let third parties verify that a safety check actually ran, but the accuracy claims rely on a classifier trained on the same benchmark used for evaluation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 88.5% F1 classifier is not the model that gets ZK-proved; the ZK-proved model's accuracy is never reported, so 'highly accurate and mathematically verifiable' is not demonstrated for any single system.","rationale":"The paper is honest about the ASB-adaptation in IV.C and provides a full open-science appendix, which is commendable. The architecture is a plausible ZKML application. However, the central claim in the abstract requires a single Judge model that is both accurate and proof-carrying. The reported experiments split this across two models: the scikit-learn logistic regression (88.5% F1, not ZK-proved) and a tiny PyTorch NN (ZK-proved, no accuracy reported). This is not a question of consensus but of internal consistency: Table I explicitly labels the ZK model as a 'demo' and the primary classifier as 'N/A' for ZK. The ablation table's unchanged F1 with the ZK layer is unexplained. Since the proof is generated per approved action, in a production system the Judge must be the ZK-compiled model; its accuracy is unknown. The reader's stated weakest assumption (Judge correctness) is real but secondary: even a perfect Judge would not rescue the claim if the proof is generated for a different model. The proposed test settles the matter by measuring both properties on one model.","tokens_in":14749,"tokens_out":4747,"duration_ms":39921,"concrete_test":"Run the 2,000-scenario Agent-SafetyBench evaluation using the exact PyTorch 11→8→2 model from Table III (the one compiled to EZKL) and report its F1/accuracy with the same out-of-fold protocol. Separately, attempt to compile the scikit-learn TF-IDF logistic regression used for Tables IV/V into an EZKL circuit and measure proof generation/verification time. If the small NN's F1 is materially below 88.5%, or if the logistic regression cannot be compiled into a viable circuit, the 'both highly accurate and mathematically verifiable' claim is refuted; if both succeed, the central claim can be provisionally accepted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table I defines two distinct Judge models: the 'Primary Classifier' (scikit-learn TF-IDF 3000-dim + Logistic Regression) used for the 88.5% F1 result in Tables IV/V, and a 'Secondary ZK-Provable Model' (PyTorch feedforward NN, 11-dim → 8 → 2) used only for the EZKL proof demonstration in Table III. The primary classifier's export format is listed as 'N/A (not ZK-proved directly)'; the ZK model's purpose is labeled 'ZK circuit compilation demo' and has no accuracy evaluation. The abstract and Section IV.C claim the guardrail is 'both highly accurate and mathematically verifiable,' but the accurate model is never proved and the proved model is never shown to be accurate. The ablation's 'Full Niyam AI' row reports identical F1 to 'Gate + Judge' (88.5%), which would only be justified if the ZK layer proves the same Judge's output, yet the paper never specifies which Judge is used in that row or how a 3000-dim logistic regression is compiled into a 431-row EZKL circuit. Without a single configuration where the ZK proof attests to the exact model whose F1 is measured, the central claim lacks evidentiary support.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Niyam-AI, a guardrail architecture for LLM agents in which an Intent Contract is committed via SHA-256, tool calls are intercepted and classified by a lightweight Judge model, and a zk-SNARK proof of the Judge's Safe decision is generated with EZKL before execution. The authors evaluate the approach on 2,000 Agent-SafetyBench scenarios, report a 5-fold out-of-fold F1 of 88.5% with a 1.1% false-positive rate, compare against NeMo Guardrails, Llama Prompt Guard 2, and GPT-OSS-Safeguard, and report proof-generation and verification latencies of about 2.26 s and 53 ms. The paper includes an explicit methodological caveat in Section IV.C that the Judge is adapted to Agent-SafetyBench while the baselines are zero-shot, and an open-science appendix with code and evaluation artifacts.","tokens_in":15020,"tokens_out":10396,"duration_ms":101237,"significance":"The core idea—binding an agent's tool-use permissions to a cryptographic commitment and proving the execution of a small safety classifier with a zk-SNARK—is timely and potentially useful as a practical ZKML application. The paper also has genuine strengths: out-of-fold cross-validation reduces the risk of per-scenario leakage, bootstrap confidence intervals and McNemar's exact test are appropriate, the adversarial red-team section reports discovered vulnerabilities and fixes, and Appendix A promises reproducible artifacts. However, the manuscript does not currently demonstrate that the system that is accurate is the same system that is ZK-verifiable, and the formal protocol does not connect cleanly to the classifiers actually evaluated. The headline claim 'highly accurate and mathematically verifiable' is therefore not supported for any single configuration. The comparison against zero-shot baselines is honestly disclosed but still makes the reported accuracy margin difficult to interpret as evidence of architectural superiority.","major_comments":[{"comment":"The formal protocol defines the Judge input as the 512-bit tuple (IntentHash, ActionHash), but the classifier whose F1 is reported in Tables IV and V is a 3000-dim TF-IDF logistic regression, while the model compiled into the EZKL circuit in Table III is an 11-dim → 8 → 2 PyTorch network. The paper never specifies how the hash tuple is mapped to either feature representation, nor does it explain which Judge is used in the 'Full Niyam AI' row of Table V. Since that row reports metrics identical to 'Gate + Judge', the paper implicitly assumes the ZK layer proves the same classifier whose accuracy is measured, but Table I states the primary classifier's export format is 'N/A (not ZK-proved directly)'. The central claim that Niyam-AI is both highly accurate and mathematically verifiable is therefore not established for any single system.","section":"Section III.D, Table I, Table V"},{"comment":"The empirical comparison is between a classifier trained on Agent-SafetyBench's distribution (using out-of-fold predictions) and baselines that are evaluated fully zero-shot. The authors disclose this asymmetry in Section IV.C, and the abstract contains a partial caveat, but Section VI's conclusion restates the result as 'outperformed three established safety systems' without that caveat. Under the reported design, the F1 margin is expected to be large because the Judge has learned the benchmark's vocabulary and phrasing, so the result does not support a claim of inherent architectural superiority. The conclusion and abstract should either be restricted to 'a domain-adapted classifier evaluated against zero-shot baselines' or supported by an additional out-of-distribution evaluation.","section":"Section IV.C, Section VI, Table IV"},{"comment":"Theorem 1 proves only that no adversary can cause execution of an action with J(x,w)=0; it does not prove that J(x,w)=1 implies the action is actually safe. The equivalence J(x,w)=1 ⟺ action satisfies intent constraints is an assumption about the Judge's correctness, not a proven property. The Introduction's statement that 'the proof only certifies that a lightweight Judge model made the right call' is therefore overclaimed: the zk-SNARK certifies the Judge's computation, not the semantic correctness of the decision. This limitation should be stated formally as an assumption, and the paper's language should distinguish between 'the Judge classified the action as Safe' and 'the action is Safe'.","section":"Section III.D, Theorem 1, Section 1"},{"comment":"Several quantitative results are internally inconsistent. In Section IV.F, the text says adding the Judge raises F1 from 0% to 91.6% and that the full pipeline also yields 91.6%, but Table V reports Gate Only F1 as 74.0%, Gate + Judge as 88.5%, and Full Niyam AI as 88.5%. The text also attributes 295.6 ms of additional latency to the ZK layer, while Table V reports 2,260.6 ms. In Section V.D, the text describes 12 adversarial vectors across five classes with 9/12 protected before fixes and 12/12 after, but Table X lists six classes and 18 vectors with 15/18 before and 18/18 after. These contradictions make the ablation and adversarial-robustness claims unreliable as reported and must be corrected or reconciled.","section":"Section IV.F, Table V, Section V.D, Table X"},{"comment":"There is a mismatch between the described execution path and the implemented classifiers. Section III.C says the interception module feeds the (IntentHash, ActionHash) tuple into a Judge Neural Network implemented in PyTorch, and Section IV.A repeats that the Judge is implemented in PyTorch and exported to ONNX. Table I, however, identifies the primary classifier used for the headline results as a scikit-learn TF-IDF logistic regression, with the PyTorch model serving only as a ZK circuit compilation demo. The paper needs to specify which model is actually in the execution path, how its features are derived from the cryptographic commitments, and how the ZK circuit binds to those features. Without this, the cryptographic binding between the Intent Contract and the evaluated classifier is not demonstrated.","section":"Section III.C, Section III.D, Section IV.A, Table I"}],"minor_comments":[{"comment":"The abstract reports McNemar win counts of 390/20, 115/13, and 384/19, while Table VII reports 387/20, 112/13, and 381/19; these numbers should be harmonized.","section":"Abstract, Table VII"},{"comment":"Proof-generation and verification times differ between the abstract (2260.6 ± 218.4 ms and 53.1 ± 11.8 ms) and Table III (2160.4 ± 510.6 ms and 65.3 ± 18.8 ms); the paper should state which measurement is the headline result and why the values differ.","section":"Abstract, Table III"},{"comment":"Appendix A refers to the bootstrap table as Table VIII and the McNemar table as Table IX, but these correspond to Tables VI and VII in the main text; the cross-references should be corrected.","section":"Appendix A"},{"comment":"The standard deviation for the Full Niyam AI latency is printed as '2196.1029' without units and with an implausible number of decimal places; the associated text also gives a different value of 295.6 ms, so the latency reporting needs to be cleaned up.","section":"Table V"},{"comment":"The text says the allowlist alone achieves an F1 of 0%, but Table V reports 74.0% for Gate Only; if this is a typo, it should be fixed because it obscures the ablation's message.","section":"Section IV.F"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reads as a promising proof-of-concept, but the load-bearing evidence is currently split across two different models: the accurate classifier is not the proved classifier, and the proved classifier has no reported accuracy. A revision that integrates the ZK-proved model into the accuracy evaluation, or that honestly narrows the claims to a separately validated classifier and a separate ZK demonstration, would address the main concern. The internal inconsistencies in Tables V and X also need to be resolved. If the authors cannot provide a single configuration in which the same model is both evaluated for accuracy and ZK-verified, I would not recommend acceptance; the paper would then be a systems proposal rather than a validated guardrail."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe thing to know about this paper: it is a genuinely reasonable application of ZKML to agent safety, with a believable architecture and an unusually candid limitations section. But the headline claim — 'highly accurate and mathematically verifiable' — is not actually demonstrated for any single system, because the accurate model and the ZK-proved model are different.\n\nWhat's new: the intent-binding protocol (SHA-256 commitment to a structured contract) plus a zk-SNARK (via EZKL) that proves a small judge model computed a Safe decision before a tool call executes. Each component is known, but the integration is coherent and could be useful for compliance and tamper-evidence use cases. The paper also does several things right: it discloses in Section IV.C that the 88.5% F1 comes from a classifier adapted to Agent-SafetyBench's distribution against zero-shot baselines; it reports bootstrap CIs and McNemar's test; it includes an open-science appendix with code and baseline prediction CSVs; and the red-team section found two real implementation vulnerabilities and fixed them. That is more than most preprints do.\n\nThe soft spots, in order of size. First and load-bearing: Table I shows the 88.5% F1 classifier is a 3000-dim TF-IDF logistic regression that is explicitly 'not ZK-proved directly.' The ZK demo uses a separate 11-8-2 feedforward network with no accuracy reported. The ablation's 'Full Niyam AI' row repeats the same F1 as 'Gate + Judge' with no explanation of which judge is proved or how the LR model would compile to a 431-constraint circuit. So the abstract's conjunction is unsupported: you either have accuracy without a proof, or a proof without measured accuracy. This should be fixable — evaluate the small NN on the benchmark, or prove the LR — but until then the central claim is an aspiration.\n\nSecond: Theorem 1 only proves that execution follows from a valid proof of J(x)=1; it does not prove J is a correct safety oracle. The paper acknowledges the judge is imperfect (92% accuracy on one category), so the formal guarantee is really 'if the judge says unsafe, no execution; if the judge says safe, a proof exists' — not safety. That is a meaningful gap between the prose and the math.\n\nThird, smaller: internal numbers don't match across the abstract, main text, and tables (390 vs 387 wins; 12 vs 18 attack vectors). Minor, but it suggests the tables weren't checked against the text.\n\nWho is this for? Researchers working on verifiable guardrails or ZKML applied to agents; it is a useful case study in the gap between ML evaluation and cryptographic proof. It deserves a serious referee — the architecture direction is sound and the open artifacts lower the cost of review — but the authors need to close the two-model gap and align the abstract with what is actually verified.\n\nRecommendation: send it out. A good referee will focus the revision on the model-evidence mismatch.","headline":"The architecture is plausible and the paper is unusually honest about its benchmark adaptation, but the ZK proof is demonstrated on a different, unmeasured model than the one that gets the 88.5% F1, so the central claim is not supported as written.","tokens_in":15588,"tokens_out":2810,"would_cite":false,"duration_ms":24933,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that AI-agent tool calls can be gated by a zero-knowledge proof, so that external parties can verify a safety check happened before any action executes, without trusting or seeing the Judge model's weights.","keywords":["AI agent safety","zero-knowledge proofs","zk-SNARK","intent contract","tool call guardrail","verifiable machine learning","prompt injection defense","auditability"],"falsifier":"Locate any scenario from the benchmark's 'Produce unsafe information/misinformation' category that the Judge labels Safe but that genuinely violates the declared intent—the paper's own category-level accuracy of 92.0% implies such cases exist—and run it through the pipeline. If the tool executes with a valid proof, then the end-to-end guarantee is shown to certify only the Judge's computation, not the action's actual safety.","tokens_in":14502,"feed_emoji":"🔐","tokens_out":7512,"duration_ms":68343,"temperature":0.7,"pith_summary":"Niyam-AI aims to replace 'trust us' in AI-agent safety with a cryptographic receipt. It seals the agent's permitted tools and constraints into an immutable SHA-256-based Intent Contract at session start, then intercepts every proposed tool call and runs it through a small deterministic Judge model. Only when the Judge says Safe does the system generate a zero-knowledge proof, and the tool executes only after that proof verifies. On 2,000 real-world agent scenarios, the paper reports an F1 score of 88.5% with a 1.1% false-positive rate, beating three existing guardrail tools, with proof verification taking roughly 53 ms. The authors are explicit that this reflects a classifier adapted to the benchmark's distribution compared with fully zero-shot baselines.","feed_headline":"Zero-knowledge proof gates every approved AI tool call","feed_subtitle":"Niyam-AI seals agent intent in a hash and runs a tool only after a proof that its Judge model approved the action.","key_machinery":"The central mechanism is the Intent-Bound Execution protocol: an immutable Intent Contract hashed to H_I, an ActionHash H_a, a lightweight deterministic Judge model as the binary safety oracle, and a zk-SNARK compiled from the Judge's exported network into a verifiable circuit. The load-bearing design choice is proving only the small Judge model, not the full language model, which keeps proof generation near 2.26 seconds while verification takes about 53 ms and requires no access to model weights. The proof is what carries the argument: it converts a software check that could be bypassed or silently skipped into a publicly verifiable computation.","core_discovery":"The central claim is that safety enforcement for tool-calling agents can be made both accurate and independently checkable by separating the safety decision from the agent itself. Concretely, the system defines a session Intent Contract I, commits it as H_I = SHA-256(I), computes an ActionHash H_a = SHA-256(a) for each proposed action, and has a deterministic Judge network J map the pair (H_I, H_a) to Safe or Unsafe. A zk-SNARK proves knowledge of the Judge's weights and inference trace such that J(x, w) = 1 for the public tuple x = (H_I, H_a), and the execution rule is Execute(a) if and only if Verify(vk, π, x) = 1. Theorem 1 states that, with collision-resistant SHA-256 and a knowledge-sound zk-SNARK, no adversary can cause an action to execute when J(x, w) = 0. The contribution is therefore not a claim that the Judge is infallible—the paper reports imperfect recall—but that enforcement of the Judge's decision is cryptographically auditable after the fact.","pith_inferences":["If the Judge's binary verdict is the real bottleneck, the same architecture could be extended to attest richer policy outputs—such as a reason code or confidence score—without changing the proving layer; the correctness question would then shift from cryptography to the specification and training of the Judge.","A direct stress test of the semantic layer would be to evaluate the Judge on an out-of-distribution corpus with none of the benchmark's phrasing conventions; a sharp accuracy drop would show that the verifiable guarantee is computation integrity, not general safety.","Third-party verifiability does not by itself tell a verifier which Judge weights were used; two different Judges could issue opposite verdicts for the same hash pair, so the audit trail attests to a particular policy's execution, not to the policy's correctness.","One could instrument the pipeline to emit the Judge's full decision trace as public auxiliary data, allowing post-hoc analysis of which input features drove a Safe verdict and giving external auditors a way to probe for learned keyword shortcuts."],"forward_implications":["Every executed tool call in a Niyam-AI deployment is accompanied by a proof that a third party can verify in milliseconds, so compliance and audit trails no longer depend on trusting the host machine.","An append-only log of hashes and proofs makes tampering detectable: altering the Intent Contract mid-session changes H_I and causes proof verification to fail.","The verifier can run locally or be exposed as a smart contract, enabling decentralized or cross-organization verification of agent behavior.","Because proof generation adds about 2.26 seconds per approved action, the framework suits high-stakes, low-frequency actions such as irreversible system changes, while latency-sensitive workloads would require batching or hardware acceleration.","Token-level constrained decoding and this proof-verified semantic gate are complementary: one enforces syntactic form, the other verifies that a fully formed tool call honors the session's declared intent."],"supporting_citations":[{"why":"Supplies the ZK proving toolchain that compiles the Judge's exported network into a verifiable circuit, making proof generation for a small model practical.","marker":"[11]"},{"why":"Provides the Groth16 zk-SNARK scheme whose knowledge soundness Theorem 1 relies on to prevent forged proofs.","marker":"[8]"},{"why":"Defines SHA-256, the collision-resistant hash used to commit the Intent Contract and each proposed action.","marker":"[10]"},{"why":"Establishes the generalized verifiable-evaluation approach that justifies proving inference of a small model rather than the entire language model.","marker":"[1]"},{"why":"Documents prompt injection as the threat that motivates intercepting tool calls rather than trusting system prompts.","marker":"[15]"},{"why":"Shows that adaptive jailbreak attacks can bypass software guardrails, motivating the need for cryptographic enforcement.","marker":"[16]"},{"why":"Introduces the structured tool-call format that the interception layer parses into ActionHashes before judging.","marker":"[13]"},{"why":"Represents the orchestration-layer 'Trust the Admin' execution gate that Niyam-AI's proof-verified gate is designed to replace.","marker":"[12]"}],"fun_headline_variants":["AI tool calls now require zero-knowledge proof of safe intent","Guardrail proofs: AI agents act only after zk-SNARK verification","Niyam-AI: cryptographically verifiable guardrails for LLM agents","Zero-knowledge proofs make AI safety enforcement auditable","Provably safe AI: ZK-proof gate for every tool invocation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the Judge model's Safe verdict is a correct and complete test of whether the action truly honors the intent contract; the cryptography only proves that the Judge made that call, not that the call was right.","fun_headline_variants_meta":{"raw":{"variants":["AI tool calls now require zero-knowledge proof of safe intent","Guardrail proofs: AI agents act only after zk-SNARK verification","Niyam-AI: cryptographically verifiable guardrails for LLM agents","Zero-knowledge proofs make AI safety enforcement auditable","Provably safe AI: ZK-proof gate for every tool invocation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000324,"raw_usage":{"total_tokens":1934,"prompt_tokens":1180,"completion_tokens":754,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":796,"completion_tokens_details":{"reasoning_tokens":662}},"tokens_in":796,"tokens_out":754,"duration_ms":6778,"temperature":1.0,"reasoning_tokens":662,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T13:39:53.811989+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Locate any scenario from the benchmark's 'Produce unsafe information/misinformation' category that the Judge labels Safe but that genuinely violates the declared intent—the paper's own category-level accuracy of 92.0% implies such cases exist—and run it through the pipeline. If the tool executes with a valid proof, then the end-to-end guarantee is shown to certify only the Judge's computation, not the action's actual safety.","supporting_citations":[{"cited_title":"The EZKL System,","cited_arxiv_id":null,"evidence_quote":"Supplies the ZK proving toolchain that compiles the Judge's exported network into a verifiable circuit, making proof generation for a small model practical."},{"cited_title":"LangChain: Building applications with LLMs through composability,","cited_arxiv_id":null,"evidence_quote":"Represents the orchestration-layer 'Trust the Admin' execution gate that Niyam-AI's proof-verified gate is designed to replace."}],"review_version":1}