Pith. sign in

REVIEW 4 major objections 5 minor 33 references

ADL: A Declarative Language for Agent-Based Chatbots

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read ADL claims that describing chatbot agents declaratively—what they are and how they interact, not how they run—makes them easier to maintain and debug.

desk verdict A real declarative agent language and interpreter, but the paper's strongest evaluation claim—LLM debugging—rests on a confounded comparison. read the letter →

arxiv 2504.14787 v2 pith:EYWYIHVE submitted 2025-04-21 cs.MA

Sirui Zeng , Xifeng Yan This is my paper · ORCID
classification cs.MA
keywords declarativeagentlanguagemulti-agentchatbotsnaturalprogrammingtask-orienteddialogueflowagentsensembleLLMorchestrationchatbotdebugging
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces ADL, a YAML-based declarative language for building customer-service chatbots as multi-agent systems. The central claim is that defining what agents are and how they interact, rather than how they are implemented, makes chatbot programs easier to maintain and debug. ADL puts natural language at the core: conditions such as "the user claims ..." and agent policies are written as free text that an LLM evaluates, and dialogue logic can be expressed either as natural-language prompts or as explicit flow steps. The authors present MICA, an interpreter and executor for ADL programs, compare five orchestration strategies for token cost and latency, and show case studies where a small business-logic change and an infinite-loop bug are handled more easily in ADL than in a Python-coupled approach.

What carries the argument

The load-bearing object is the ADL step_block: a small YAML instruction set of user, bot, label, next, call, set, if/else-if/else, and return statements, in which conditional expressions are free text such as "the user claims ..." that an LLM evaluates against the conversation. This block is shared by flow agents, ensemble-agent initialization, and the main entry point, so every control-flow decision in a chatbot is written in the same declarative vocabulary. A second piece is the ensemble agent, which holds the contains list and a natural-language policy for choosing among agents; because routing logic is declarative, the runtime can substitute orchestration methods such as merging, first-success, best-of-N, proactive, and autonomous without editing the program. MICA is the interpreter that executes ADL files and translates tool calls into the format expected by the LLM.

What would settle it

Run the same customer-service bot written in ADL and in an equivalent Python-coupled implementation, hide the same set of dialogue-logic bugs such as misroutes and infinite loops in both, and measure how quickly human developers or an LLM spot them; if ADL does not yield faster or more accurate detection, the maintainability and debugging claim fails. A second targeted check is to feed a test suite of paraphrased user utterances to a flow agent whose conditions are natural-language statements and count misrouted turns, since a high misclassification rate would falsify the assumption that natural-language conditions are production-safe.

Watch

Extended reading notes

Core claim

On its own terms, ADL claims that a chatbot can be specified as a collection of four agent types: knowledge-base agents that retrieve answers from sources and FAQs, LLM agents that follow natural-language prompts, flow agents that give explicit step-by-step control with labels, branches, and subflows, and ensemble agents that route user requests among the others. Every agent shares a header with a description, arguments, a fallback policy, and an exit policy, and the whole program requires a main agent as an entry point. The paper's discovery claim is that this abstraction boundary—separating task specification from runtime execution—makes maintenance and debugging easier than the Python-interleaved declarations used by general agent frameworks. A secondary empirical claim is that the same ADL program can be run under different orchestration strategies, with merging reaching 540 tokens and 2.34 seconds per turn versus 1732 tokens and 10.39 seconds for first-success.

Load-bearing premise

The load-bearing premise is that an LLM will interpret free-text conditions like "the user claims ..." and natural-language policies accurately enough that conversations are routed correctly in production; the paper does not measure that accuracy or specify fallback behavior when it fails.

Editorial extensions

If this is right

  • Adding a new intent branch, such as a discount inquiry, becomes a few lines inside an existing ADL flow rather than an update to an intent classifier and new internal state variables.
  • The same ADL program can be executed under different orchestration strategies, so the underlying runtime can choose among cost, latency, and output-quality trade-offs without rewriting the chatbot logic.
  • LLM-based static analysis can spot an infinite loop from the ADL code alone, whereas an equivalent Python-coupled implementation led the same LLM to report a false-positive loop.
  • Because conditions and policies are natural language, improvements in LLM capabilities translate into better behavior of existing ADL programs without code changes.
  • Descriptions, conditions, and policies written in natural language make chatbot designs readable by non-programmers, lowering the barrier to customer-service bot customization.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The infinite-loop debugging result is based on one injected bug; the claim that ADL is more debuggable by LLMs would be stronger if tested across many injected bugs and several declarative or structured frameworks.
  • If natural-language conditions prove reliable enough, an ADL program could serve simultaneously as the executable system and as a contract that business analysts can read and review, since the spec and the runtime program are the same artifact.
  • The orchestration measurements sketch a trade-off surface among merging, first-success, best-of-N, proactive, and autonomous methods; a standardized benchmark of customer-service dialogue workloads could turn this into a routine optimization problem for multi-agent runtimes.
  • The unmeasured risk is condition misclassification; a test suite of adversarial user paraphrases would show whether free-text conditions such as "the user claims ..." are robust enough for production banking and retail flows.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces ADL (Agent Declarative Language), a YAML-based declarative language for specifying customer-service chatbot multi-agent systems, together with MICA, an open-source interpreter/executor. ADL defines four agent types (KB, LLM, flow, and ensemble), supports natural-language conditions and policies, and separates agent declarations from Python implementations and runtime optimization. The paper reports three forms of evaluation: a comparison of five orchestration strategies on token cost and latency, a maintainability case study based on code-snippet comparison, and a debugging case study in which GPT-4o is asked to detect an infinite loop in ADL and Swarm implementations of a retail banking bot. The central claim is that defining what agents are and how they interact, rather than how they are implemented, makes such chatbots easier to maintain and debug.

Significance. If the claims were established, this would be a useful contribution to the agent-framework design space: ADL is small, readable, and the MICA artifact is publicly available, which is a genuine strength. The paper also provides a detailed grammar (Appendix A), realistic worked examples, and a careful comparison with Rasa-CALM in Appendix F, showing awareness of adjacent declarative efforts. The evaluation is not circular: Table 1 measures external LLM API behavior and Table 2 compares against Swarm. However, the two headline advantages (maintainability and debugging) are currently supported only by anecdotal or confounded evidence: Section 5 has no measurement, Section 6 uses a single infinite-loop example in which the ADL and Swarm programs are not behaviorally equivalent, and Table 1 reports five-run averages without variance. The core artifact is plausible, but the paper needs substantial experimental strengthening before the central claim can be accepted.

major comments (4)
  1. [Section 6 / Table 2 / Appendix D.2] The headline debugging experiment is not a controlled comparison. In the ADL program, transfer_money's prompt instructs entering add_payee and add_payee's prompt instructs entering transfer_money, so the described loop is actually reachable; in the Swarm implementation, transfer_money lists only transfer_to_triage among its functions and add_payee lists only transfer_to_triage and action_add_payee, so the same textual instructions cannot produce a direct agent-to-agent loop. Table 2 itself marks the Swarm loop as absent (✗). The LLM's false positive on Swarm therefore does not demonstrate that ADL is easier to debug; the comparison confounds language semantics with missing wiring in the baseline, and it is further affected by unrelated quirks such as transfer_to_block_card() calling block_card() instead of returning an agent. To support the claim, the authors should compare programs with the same reachable bug, verify that the bug is actually present in both implementations, and ideally repeat the detection over multiple runs and programs.
  2. [Section 5] The maintainability claim rests on a qualitative snippet comparison rather than on measurement. The discount-flow insertion in Figure 1 is an existence proof that ADL can express the change, but it does not show that maintenance is easier; there is no task-based metric, user study, or controlled comparison with Swarm or Rasa-CALM modification effort. The statement that Swarm 'requires more extensive modification if step-by-step control is needed' is not quantified, and the Python sketch on page 5 is not a real baseline. I recommend either tempering the claim to a design rationale or reporting controlled modification experiments (e.g., time, correctness, or number of edited lines for a set of change tasks).
  3. [Appendix B.4.5 / Section 3.2] The language's core relies on an LLM evaluating natural-language conditions such as 'the user claims ...', but the paper provides no accuracy measurement, no fallback behavior, and no failure-mode discussion for condition misclassification. Since flow agents and ensemble policies branch on these conditions, the runtime correctness of ADL programs is uncharacterized. The authors should report condition-evaluation accuracy on representative held-out user utterances, or specify a deterministic fallback/confidence threshold, so that a reader can assess when the language is safe to use in production.
  4. [Section 4 / Table 1] The orchestration comparison in Table 1 is reported as five-run averages with no standard deviations, and it uses a single scripted dialogue sequence with GPT-4o-mini. This is not sufficient to support the conclusions about cost/latency trade-offs or the claimed advantage of the merging method. Report per-run values or confidence intervals, use several distinct dialogues, and ideally include output-quality metrics, since the text states that Best-of-N has the highest output quality but does not measure it.
minor comments (5)
  1. [Appendix D.2] The check_balance agent declares functions=[actio_ask_account, action_check_balance, ...]; 'actio_ask_account' appears to be a typo for 'action_ask_account', which would break runtime behavior as written.
  2. [Section 6.2] The text refers to 'GPT-4o' for loop detection while Section 4 uses GPT-4o-mini; state the exact model, temperature, and number of runs for the loop-detection experiment, and clarify whether the chain-of-thought in Table 3 was elicited or observed.
  3. [Section 3.1] The AutoGen comparison contrasts a Python while-loop with ADL flow steps, but the claim that 'it is not very clear where is the user input from' is stylistic rather than a technical limitation; the example could be strengthened by pointing to a concrete maintenance or readability consequence.
  4. [Limitations] The Limitations section is candid, but the phrase 'due to space constraints' sits oddly in a paper of this length; given the appendix space available, the missing variance information and multi-scenario tests could reasonably have been included.
  5. [Figure 1] In Figure 1, the indentation of the 'else if : the user asks for any discount' branch and the placement of the 'else:' clause are hard to read in the two-column format; reformatting these lines would improve legibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper's claims are design demonstrations with external API measurements; the non-equivalent Swarm baseline is an experimental confound, not a derivation-level circularity.

full rationale

I find no circularity in the derivation chain. The paper's central contribution is a language design and an interpreter; it makes no quantitative prediction derived from fitted parameters, and no equation is defined in terms of another claim. Table 1 measures external LLM API token counts and latencies across ADL orchestration strategies, and although the merging method's low cost is partly by construction because it combines consecutive agents, the paper reports it as an illustrative systems trade-off rather than as a derived law. Table 2's debugging experiment asks GPT-4o to inspect ADL and Swarm code for an infinite loop; the skeptic's concern that the two implementations are not behaviorally equivalent is a threat to experimental validity, not circularity. The ADL program genuinely contains the loop ('Actual Presence of Loop' is marked ✓) and the Swarm program does not (marked ✗), so the LLM's false positive on Swarm is an empirical observation about non-equivalent artifacts, not a quantity whose definition already includes the conclusion. The Limitations section explicitly concedes that the debugging study covers only infinite loops and that orchestration evaluation is limited to token cost and latency, which further shows the claims are not being forced by construction. No load-bearing self-citation, imported uniqueness theorem, or ansatz smuggled via citation appears; the only self-references are artifact announcements and documentation pointers for MICA. The main risk in the paper is the unstated reliability assumption that LLMs classify natural-language conditions such as 'the user claims ...' accurately enough for production use, but that is a correctness and robustness concern, not circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper has no fitted constants and introduces no speculative physical or conceptual entities. Its claims rest on four domain assumptions about LLM reliability, maintainability transfer, LLM debugging advantage, and representativeness of the simulated dialogue. The language and interpreter are released as open source, which gives independent parties a way to test these assumptions.

assumptions (4)
  • domain assumption Natural-language conditions and policies can be evaluated by an LLM with enough reliability for correct dialogue branching.
    Appendix B.4.5 allows natural-language condition expressions and offers no accuracy threshold, error analysis, or fallback for misclassification.
  • domain assumption Declarative separation from implementation reduces maintenance and debugging effort.
    Section 5 argues by code-snippet comparison and informal feedback from a few novices; no user study or maintainability metric is provided.
  • domain assumption LLM static analysis is more accurate on natural-language ADL code than on Python-plus-prompt Swarm code.
    Section 6.2 demonstrates one positive and one negative example, which is insufficient to generalize this advantage.
  • domain assumption The simulated dialogue used for orchestration comparison is representative of customer-service conversations.
    Section 4 says each turn shifts user intent, but the exact dialogue, prompt set, and evaluation harness are not published.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ADL: A Declarative Language for Agent-Based Chatbots." pith.science (2026). https://pith.science/paper/EYWYIHVE

@misc{pith2026250414787,
  author       = {Pith},
  title        = {Pith review of: ADL: A Declarative Language for Agent-Based Chatbots},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EYWYIHVE}},
  note         = {Machine review of arXiv:2504.14787}
}
read the original abstract

There are numerous frameworks capable of creating and orchestrating agents to address complex tasks. However, most of them highly coupled Python programming with agent declaration, making it hard for maintenance and runtime optimization. In this work, we introduce ADL, an agent declarative language for customer service chatbots. ADL abstracts away implementation details, offering a declarative way to define agents and their interactions, which could ease maintenance and debugging. It also incorporates natural language programming at its core to simplify the specification and communication of chatbot designs. ADL includes four basic types of agents and supports integration with custom functions, tool use, and third-party agents. MICA, a multi-agent system designed to interpret and execute ADL programs, has been developed and is now available as an open-source project at https://github.com/Mica-labs/MICA. Its documentation can be found at https://mica-labs.github.io/.

Figures

Figures reproduced from arXiv: 2504.14787 by the authors.

Figure 1
Figure 1. A bookstore chatbot written in ADL performance of ADL programs can grow with the advances of LLMs without revising their code. We have released MICA, a multi-agent sys￾tem designed to interpret and execute ADL programs at github https://github.com/ Mica-labs/MICA and received feedback from a few novices that it is much faster to learn and program. In the following sections, we will first introduce the main highlight… view at source ↗
Figure 2
Figure 2. Body of different types of agents and which function to call. The args field is included when certain states or arguments must be collected or recorded. If the agent determines that it cannot respond adequately based on what is declared in its prompt, it will deactivate itself and transfer control back to other agent. An LLM agent might include a few initialization steps in the step block which will be executed befo… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 31 canonical work pages

  1. [2]

    action_ask_account_from

    Ask for the username. Based on the username, call the function "action_ask_account_from" to display the account name and balance. Ask which account to use and fill in account_from with the corresponding number (do not show the account number to the user, just populate account_from directly). ,→ ,→ ,→ ,→ 15

    work page

  2. [3]

    kb agent,

    Automl-agent: A multi-agent llm frame- work for full-pipeline automl. arXiv preprint arXiv:2410.02958. Google Workflow. 2021. https://cloud. google.com/workflows/docs/ reference/syntax. Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, Ahmed Hassan Awadallah, Ryen W White, Doug Burge...

    arXiv 2021

  3. [7]

    now", confirm whether to proceed with the immediate transfer. - If confirmed, call

    Collect the transfer timing timing. 8a. If timing is "now", confirm whether to proceed with the immediate transfer. - If confirmed, call "action_process_immediate_payment", then output "Transfer successful", and exit this agent.,→ - If not proceeding immediately, output "Transfer canceled", and exit this agent.,→ 8b. If timing is not "now", ask for the sp...

    work page

  4. [9]

    action_validate_payment_date

    Call "action_validate_payment_date". - If the date is a future date, confirm whether to schedule the transfer. - If confirmed, call "action_schedule_payment", output "Transfer scheduled", and exit this agent.,→ - If not scheduling, output "Transfer canceled", and exit this agent. args: - username - account_from - payee_name - amount - timing - payment_dat...

    work page

  5. [10]

    action_ask_account

    You need to ask for the username. Based on the username, call the function "action_ask_account" to display the account information to the user and ask which account name they want to inquire about. ,→ ,→

    work page

  6. [11]

    account" with the corresponding account_number, then call

    Based on the user's response, fill the variable "account" with the corresponding account_number, then call "action_check_balance" to inform the user of the balance for the requested account. ,→ ,→ args: - username - account uses: - action_ask_account - action_check_balance block_card: type: flow agent description: Block or freeze a user's debit or credit ...

    work page

  7. [12]

    Prompt the user sequentially for payee_name, account_number, payee_type (person/business), and reference (a short note to identify the payee or purpose). ,→ ,→

    work page

  8. [14]

    {payee_name} has been successfully added to your list of authorized payees

    If the action is successful, respond with "{payee_name} has been successfully added to your list of authorized payees." Otherwise, respond with "I'm sorry, but there was an error adding {payee_name}. Please try again later or contact Customer Support. ,→ ,→ ,→

    work page

Show all 33 references
  1. [15]

    Mary Brown

    Ask the user if they need to make a transfer; if so, enter the transfer_money agent.,→ args: - username - payee_name - account_number - payee_type - reference uses: - action_add_payee meta: type: ensemble agent contains: - what_can_you_do - check_balance: args: username: ref u...

  2. [16]

    action_add_payee

    Confirm whether all the provided information is correct. If correct, call "action_add_payee".,→

  3. [17]

    {payee_name} has been successfully added to your list of authorized payees

    If the action is successful, respond with " {payee_name} has been successfully added to your list of authorized payees." Otherwise, respond with "I'm sorry, but there was an error adding {payee_name}. Please try again later or contact Customer Support. ,→ ,→ ,→

  4. [18]

    ,→ """ GET_FEEDBACK_POLICY =

    Ask the user if they need to make a transfer; if so, enter the transfer_money agent.",→ """ GET_FEEDBACK_POLICY = """Ask the user if they are satisfied with the service. voting from 1 to 5. If it is less than 3, set satisfied = True; otherwise, set satisfied = False""" ,→ ,→ C...

  5. [19]

    account" with the corresponding account_number, then call

    Based on the user's response, fill the variable "account" with the corresponding account_number, then call "action_check_balance" to inform the user of the balance for the requested account.""" ,→ ,→ LIST_PAYEES_POLICY = """After knowing "username", call function "action_list_...

  6. [20]

    If satisfied == False, express your apology and jump to step 2

    Check satisfied value. If satisfied == False, express your apology and jump to step 2. Otherwise, proceed to step 1.,→

  7. [21]

    First, confirm that you understand the user wants to transfer money

  8. [22]

    action_ask_account_from

    Ask for the username. Based on the username, call the function "action_ask_account_from" to display the account name and balance. Ask which account to use and fill in account_from with the corresponding number (do not show the account number to the user, just populate account_...

  9. [23]

    add payee

    Prompt the user to enter the "add payee" agent and exit this agent

  10. [24]

    action_check_sufficient_funds

    Once all required details are collected, call "action_check_sufficient_funds".,→ 6a. If there are sufficient funds, proceed to step 7. 6b. If there are insufficient funds, terminate this agent immediately

  11. [25]

    now", confirm whether to proceed with the immediate transfer. 8b. If timing is not

    Collect the transfer timing timing. 8a. If timing is "now", confirm whether to proceed with the immediate transfer. 8b. If timing is not "now", ask for the specific transfer date, ensuring it is formatted as "DD/MM/YYYY". Proceed to step 9.,→

  12. [26]

    action_validate_payment_date

    Call "action_validate_payment_date". 9a. If the date is a future date, confirm whether to schedule the transfer. 9b. If confirmed, call "action_schedule_payment", output "Transfer scheduled", and exit this agent.,→ 9c. If not scheduling, output "Transfer canceled", and exit th...

  13. [27]

    list_payees_agent

    call "list_payees_agent" to get the current_payee_number. Set it as previous_payee_number.,→

  14. [28]

    action_remove_payee

    call "action_remove_payee"

  15. [29]

    list_payees_agent

    call "list_payees_agent" again to get the current_payee_number

  16. [30]

    Failed to removed!

    if current_payee_number - previous_payee_number == 0, tell that"Failed to removed!"; otherwise, tell that"Succeed removed!""",→ greeting = Agent( name="greeting", instructions=GREETING_PROMPT, functions=[transfer_to_triage] ) 21 add_payee = Agent( name="Add Payee", instruction...

  17. [31]

    Ask the user if they have a preferred book genre

  18. [32]

    query_book_genre

    If the user has a favorite book, call the "query_book_genre" function based on their response to obtain the "genre"

  19. [33]

    find_best sellers

    Using the genre and other attributes, call the "find_best sellers" function to recommend relevant books to the user. uses: - query_book_genre - find_bestsellers steps: - bot: "I'll place the order for you.",→ - label: confirm_books - bot: "You have selected these books so far:...

    2017

  20. [34]

    action_ask_card

    Based on the username, directly call "action_ask_card" to retrieve the user's available cards. Ask the user: "Select the card you require assistance with:" and fill in the "card" field

  21. [35]

    reason_for_blocking

    Ask the user for the reason they want to block the card (reason_for_blocking):,→ 28 - The reason should be described as lost, damaged, stolen, suspected of fraud, malfunctioning, or expired. - The user may also say they are traveling or moving, or that they want to temporarily...

  22. [36]

    action_update_card_status

    Inform the user that their card will be blocked due to the specified reason. - Confirm whether they want to **issue a new card** or if they prefer to visit the bank themselves. - If the user confirms they want to issue a new card, proceed to Step 5. - If the user declines, cal...

  23. [37]

    physical_address

    Confirm the user's "physical_address". - If the user confirms the address is correct, inform them: "The new card will be sent to this address." - If the user indicates the address is incorrect, go back to Step 3d. uses: - action_ask_card - action_update_card_status The Flow ag...

  24. [2023]

    In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 431–445, Singapore

    NeMo guardrails: A toolkit for controllable and safe LLM applications with programmable rails. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 431–445, Singapore. Associa- tion for Computational Linguistic...

    2023

  25. [2024]

    arXiv preprint arXiv:2405.11106

    Llm-based multi-agent reinforcement learn- ing: Current and future directions. arXiv preprint arXiv:2405.11106. 7 Yashar Talebirad and Amirhossein Nadiri. 2023. Multi- agent collaboration: Harnessing the power of intelli- gent llm agents. arXiv preprint arXiv:2306.03314. Patar...

    2023 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.