{"id":"365fe0f3-0a04-4e13-99dc-4c99e07388d2","arxiv_id":"2507.10324","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A consolidated description of a tool suite for specifying, verifying, and implementing multiagent systems through information protocols, with no new research result beyond the authors' prior tool papers.","lead":"This paper describes a set of software tools for building multiagent systems from communication protocols: Tango verifies protocols, Kiko helps program agents, and Mandrake handles lost messages. It is a useful consolidated tour of a mature research toolchain, though the tools themselves were published in earlier papers and no new evaluation is provided.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Tango's unreproved canonical reduction theorem is the load-bearing point: both the 'verified protocol' guarantee and the 'model you verify is the model you implement' claim rest on it, and this paper cites [70] without proof or independent check.","rationale":"The reader identified the Tango reduction theorem as the weakest assumption, and I agree. The paper is honestly a tool overview with public code, and I do not question the authors' integrity. But the central assertion of Section 3.5 inherits its force from Tango's verification outputs, and those outputs inherit their force from the reduction theorem. The paper's own Listing 2 makes the point vivid: 12 maximal paths are reduced to 1 for safety and liveness checking, yet Listing 4 shows that safety violations in this setting arise from concurrent binding of a parameter by two roles. Any reduction that compresses concurrent enactments must provably preserve such violations. Section 3.2 only cites [70] for this result; no proof, counterexample analysis, or independent test is provided. The absence is load-bearing precisely because the paper claims a formal guarantee, not merely a heuristic tool. My recommendation is unchanged from the reader's CONDITIONAL: accept the tool description with the caveat that the central verification guarantee remains contingent on an unverified reduction theorem and on a conformance claim for Kiko/Mandrake that is likewise asserted rather than demonstrated in this paper.","tokens_in":15240,"tokens_out":8550,"duration_ms":115535,"concrete_test":"Run an exhaustive comparison for the two sample protocols: use `bspl verify all_paths` to enumerate all maximal paths, then independently evaluate the BSPL safety and liveness predicates on each enumerated path and compare the aggregate result with `bspl verify safety` and `bspl verify liveness`. If the canonical one-path result differs from the exhaustive result for Flexible Purchase or Buggy Flexible Purchase, the reduction theorem is refuted and the central verification guarantee collapses. If it matches, additionally re-derive the reduction theorem from the BSPL transition semantics for a class containing protocols with concurrent out-parameter bindings, since the buggy example confirms that safety can depend on two sends racing to bind the same parameter.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2's efficiency claim and Section 3.5's central 'model you verify is the model you implement' claim both depend on the unproved reduction theorem that all enactments of a BSPL protocol can be compressed to a canonical set while preserving safety and liveness. In the paper's own example, 12 maximal paths are reduced to 1 for the safety and liveness checks, and the Buggy Flexible Purchase counterexample shows that safety can fail because two roles concurrently bind the same parameter. A reduction that erases concurrency or merges paths could easily erase exactly this kind of counterexample. The paper gives no proof, no formal statement, and no independent validation of this reduction; it only cites [70]. Given that the tool's verdicts are the sole basis for the claim that a protocol is formally verified, an unsound reduction would invalidate the headline guarantee even if Kiko and Mandrake are perfectly compliant. This is not an allegation of fraud: [70] may well contain a valid proof. But this paper presents the reduction as a load-bearing result and does not make it checkable here.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper describes a suite of tools for Interaction-Oriented Programming (IOP): Tango, a verifier for BSPL information protocols checking safety and liveness; Kiko, an event-driven agent programming model with an information protocol adapter; Mandrake, a model for fault-tolerant agent policies; and Orpheus, Cupid, and Azorus for cognition and commitments. It illustrates the tools on a Flexible Purchase protocol, showing verification outputs in Listing 2 and a buggy variant in Listing 4, and argues that the approach supports loose coupling, asynchronous messaging over UDP, and decentralized multiagent systems. The central claim is that the approach is formal and that 'the model you verify is the model you implement' (Section 3.5).","tokens_in":15404,"tokens_out":6305,"duration_ms":68126,"significance":"If the central claim holds, the tool suite offers a concrete, publicly available path from formal protocol verification to agent implementation without reliance on ordered or reliable message delivery. The paper's strengths include the public GitLab code, reproducible command-line examples with actual outputs, and the demonstration that a simple flexible protocol is beyond the expressiveness of traditional approaches. The main risk is that the verification guarantee rests on Tango's canonical reduction theorem and Kiko's compliance guarantee, both of which are cited to prior work rather than stated and justified here. The 'vastly improved' performance claim in Section 3.2 lacks benchmarks. These issues are addressable and do not, on the basis of this paper, invalidate the approach.","major_comments":[{"comment":"The claim that the canonical reduction 'leads to vastly improved verification performance compared to earlier verification approaches' is unsupported by any comparative benchmark; the only evidence provided is sub-millisecond elapsed times on a single toy protocol with no baseline. Because this reduction is the mechanism that makes the verification outputs in Listing 2 ('maximal paths: 1') tractable, the paper should either provide a reproducible benchmark against the earlier approach in [67] or substantially weaken the performance claim.","section":"Section 3.2, Listing 2"},{"comment":"The paper's headline guarantee, 'the model you verify is the model you implement' (Section 3.5), depends on Tango's reduction from all enactments to a canonical set preserving safety and liveness, yet the reduction theorem is only cited to [70] and never stated. The Buggy Flexible Purchase example in Listing 4 shows that safety failures arise from concurrent bindings of a parameter (paid), so a reader needs to know exactly which enactments are merged by the canonical reduction and why the merge preserves such failures. Please state the theorem, its side conditions, and a pointer to the proof in [70].","section":"Section 3.2, Section 3.5"},{"comment":"The statement that 'Kiko guarantees an agent's compliance with the protocol—statically, in the case of sequential agents' is load-bearing for the verified-protocol claim, but this paper gives no formal property, proof, or precise reference to where in [26] the guarantee is established. Listing 6's 'check' step is not specified: the paper should state what invariant the adapter enforces (e.g., only enabled forms are emitted, no duplicate bindings, key integrity) and provide a soundness argument or an exact theorem reference.","section":"Section 6, 'Types' paragraph; Section 3.3, Listing 6"},{"comment":"The Mandrake reminder policy in Listing 7 is presented as an agent-level retransmission mechanism, but the paper does not show that such policies preserve protocol safety and liveness. Since the seller's retransmitted Shipment messages are additional emissions, the paper should either prove (or cite a proof) that Mandrake policies are protocol-compliant by construction, or clarify that Mandrake's fault handling operates outside the verified protocol guarantee.","section":"Section 3.4, Listing 7"}],"minor_comments":[{"comment":"The phrase 'if a fan skeets' appears to be a typo (likely 'tweets'); please correct it.","section":"Section 2.1"},{"comment":"The comment formatting in Listing 6 is garbled; the rendering should be fixed so that 'ti: di represents a decision maker di with trigger ti' is readable.","section":"Section 3.3, Listing 6"},{"comment":"The code comment '00 17 * * *' and the text '1700 hours every day' should be aligned, and the cron syntax should be briefly explained for readers unfamiliar with cron.","section":"Section 3.3"},{"comment":"The 'Stable' status for Tango, Kiko, and Mandrake is not accompanied by version numbers or a test suite; adding release versions and commit hashes would strengthen the reproducibility statement.","section":"Table 1"},{"comment":"The reproducibility section says the 'entire codebase referenced in this paper' is available online, but the specific branches or commits for Kiko and Mandrake are not pinned; consider providing a reproducibility artifact with exact commit hashes.","section":"Section 7"}],"recommendation":"major_revision","confidential_remarks":"The paper is largely a synthesis and demonstration of the authors' own prior work; the self-citations are appropriate for a tools paper, but the novelty is limited and the central 'verified protocol' guarantee is inherited from [26,70] rather than demonstrated here. The editor may wish to consider whether the venue's scope includes tool/demonstration papers without new formal results, and whether the missing theorem statements and benchmarks are required for acceptance. I do not suspect any misconduct; the concerns are about completeness and substantiation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this is not a new research result. Every substantive tool (Tango, Kiko, Mandrake, Orpheus, Cupid, Azorus) was previously published by the same group, and the paper is a consolidated description built around a small purchase example. That is fine, provided you treat it as a tool overview. What is genuinely useful is the clarity of the exposition. The information protocol idea is well explained, the running examples with real tool output make the semantics concrete, and the code is public and appears to run. The 'model you verify is the model you implement' claim is the right way to think about the value of the suite, and the paper shows a credible path from BSPL specification to agent code via Kiko and Mandrake without assuming ordered or reliable delivery.\n\nThe soft spots are the unsupported performance and compliance claims. Section 3.2 says the canonical reduction 'vastly improves' verification performance compared to earlier approaches, but no benchmarks are given. Section 6 asserts that Kiko statically guarantees protocol compliance for sequential agents, but the proof or even a precise statement is not in this paper. Most importantly, the soundness of Tango's verification depends on a reduction theorem about canonical enactments that is cited to an earlier IJCAI paper and not stated here. The stress-test worry about that reduction is fair in that it is load-bearing, but I don't consider it fatal: the theorem is presumably proved in [70], and a tool paper can cite that. Still, the manuscript should state the theorem or at least make the dependency explicit, and soften the 'vastly improved' performance language without benchmarks. The heavy self-citation is expected here, since the tools are theirs, and the reproducibility section is a plus.\n\nFor whom is this? Practitioners and newcomers to IOP who want a starting point, and readers who want to see the suite in one place. It is not going to change anyone's research program, and the lack of independent evaluation means the 'verified' label is only as strong as the prior work's proofs plus the compliance argument, which is not fully spelled out.\n\nI would send this to review. It is a legitimate, honest description of a real software suite with reproducible examples, and a good referee could push the authors to either substantiate the performance claim or soften it, and to make the dependency on [70] explicit. The paper would be a fine fit for a systems or tool track. It would not survive as a main-track research contribution, but it does not pretend to be one. Recommendation: serious review, likely revise.","headline":"A clear, honest tool overview that consolidates the authors' prior IOP work; the main guarantees rest on earlier papers, so treat it as integration and push back on the unbenchmarked performance claim.","tokens_in":15978,"tokens_out":4223,"would_cite":false,"duration_ms":46013,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A protocol-first toolkit claims to let developers verify a multiagent interaction model and then implement agents from that same model, with no ordered or reliable delivery required.","keywords":["Interaction-Oriented Programming","information protocols","BSPL","multiagent systems","protocol verification","liveness","safety","fault-tolerant agents"],"falsifier":"Run Tango's liveness and safety checks and an exhaustive enumeration of all maximal enactments on a small two-role, three-message protocol; if any non-canonical enactment deadlocks or double-binds a parameter while the reduced check reports live and safe, the verified-protocol guarantee is false.","tokens_in":14990,"feed_emoji":"🤝","tokens_out":6407,"duration_ms":64635,"temperature":0.7,"pith_summary":"This paper presents a software suite and argues that it makes Interaction-Oriented Programming practical: a developer specifies a multiagent system as an interaction protocol, verifies that protocol for liveness and safety, and then implements role-playing agents against that same verified protocol. The key idea is that information protocols constrain what a sender may send based on information dependencies rather than message ordering, so agents can interact flexibly over unreliable, unordered networks. The paper's strongest assertion is that the model you verify is the model you implement, meaning the verification guarantees carry through to the running system.","feed_headline":"The model you verify is the model you implement","feed_subtitle":"A multiagent toolkit checks safety and liveness of protocols, then supports compliant agents that run over UDP.","key_machinery":"The load-bearing mechanism is the information protocol itself, written in BSPL. Each message declares parameters with adornments: in means the sender's local state must already contain a binding for that parameter, out means the sender must not already have one and sending creates it, and nil means either. The protocol therefore constrains information causality and integrity, not message order. On top of this sits Tango's verification, which reduces the possibly exponential set of enactments to a small set of canonical enactments for checking safety (no parameter gets more than one binding) and liveness (every enactment can extend to completion), and Kiko's adapter, which exposes to the programmer only the locally enabled message forms and handles sending, receiving, and duplicate-checking. Mandrake extends the same idea with application-level retransmission policies for lost messages.","core_discovery":"The paper's central claim is that the combination of information protocols, the Tango verifier, and the Kiko and Mandrake programming models realizes Interaction-Oriented Programming end to end. A developer specifies a protocol in BSPL as information causality and integrity constraints, verifies it with Tango for liveness and safety, and then implements agents using middleware that keeps agents compliant with that protocol. The demonstration uses a purchase scenario where the buyer and seller may send Payment and Shipment concurrently or in either order; Tango reports it live and safe, and a buggy variant is caught with concrete counterexamples. The stated principle is that the model you verify is the model you implement, without assuming ordered or reliable message delivery.","pith_inferences":["The paper leaves implicit that the same canonical-enactment reduction could be applied to higher-level protocol languages such as Langshaw, where protocols are compiled down to BSPL.","The paper leaves implicit that commitment specifications could automate fault tolerance: once an agent knows which messages its commitments make important, retransmission policies need not be written by hand.","A testable extension would be to combine the protocol adapter with learned decision making: an agent that learns which enabled forms to complete would still be protocol-compliant by construction."],"forward_implications":["If the central claim holds, a developer verifies a protocol once with Tango and can then trust that any Kiko or Mandrake agent playing its roles will stay within the verified protocol.","If the central claim holds, communication can run over unordered, lossy transport such as UDP without violating the protocol's safety or liveness.","If the central claim holds, application-level retransmission policies cover faults that lower-level reliable transports cannot, such as an agent that simply never sends an expected message.","If the central claim holds, protocols that allow concurrent or alternative message orders are expressible and verifiable, whereas traditional message-ordering protocol languages cannot express them."],"supporting_citations":[{"why":"Supplies the Tango verification method and the canonical-enactment reduction that the paper relies on for efficient liveness and safety checking.","marker":"[70]"},{"why":"Introduces Kiko, the enablement-based programming model whose adapter is the central implementation mechanism.","marker":"[26]"},{"why":"Introduces Mandrake, which provides the application-level retransmission policies for fault-tolerant agents.","marker":"[25]"},{"why":"Establishes the semantics of information-based protocols that the paper's verification and programming tools build on.","marker":"[67]"},{"why":"Introduces BSPL, the protocol language used in all the paper's examples and tooling.","marker":"[65]"},{"why":"Provides the end-to-end principle that justifies placing reliability in agent-level policies rather than the transport layer.","marker":"[57]"},{"why":"Supplies the actor-model precedent for asynchronous, order-free message passing that Kiko's adapter adopts.","marker":"[38]"}],"fun_headline_variants":["Toolsuite verifies protocols, builds compliant agents","Verify protocol, then implement agents","Safety and liveness checked, then agents run","One protocol, verified and implemented","End-to-end IOP: from protocol to agents"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole verification pipeline rests on an unproved reduction theorem cited to an earlier paper: that checking a small set of canonical enactments is enough to catch every deadlock or double-binding that could occur in any enactment of the protocol.","fun_headline_variants_meta":{"raw":{"variants":["Toolsuite verifies protocols, builds compliant agents","Verify protocol, then implement agents","Safety and liveness checked, then agents run","One protocol, verified and implemented","End-to-end IOP: from protocol to agents"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0004,"raw_usage":{"total_tokens":1995,"prompt_tokens":754,"completion_tokens":1241,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":370,"completion_tokens_details":{"reasoning_tokens":1175}},"tokens_in":370,"tokens_out":1241,"duration_ms":13041,"temperature":1.0,"reasoning_tokens":1175,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T17:33:33.077927+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Tango's liveness and safety checks and an exhaustive enumeration of all maximal enactments on a small two-role, three-message protocol; if any non-canonical enactment deadlocks or double-binds a parameter while the reduced check reports live and safe, the verified-protocol guarantee is false.","supporting_citations":[{"cited_title":"In: Proceedings of the 11th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS)","cited_arxiv_id":null,"evidence_quote":"Establishes the semantics of information-based protocols that the paper's verification and programming tools build on."}],"review_version":1}