REVIEW 5 major objections 5 minor 29 references
Adaptive Arena-based Contestable Argumentative Network-of-Experts for Open-Ended Care Plan Coordination
T0 review · 5 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read CANOE builds care plans as contestable argument graphs that clinicians can edit, with deterministic recomputation after each change.
desk verdict Well-built framework, but the headline claim about contestability and faithfulness is not tested by the experiments. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central machinery is the Arena-based Quantitative Bipolar Argumentation Framework (A-QBAF). For each candidate intervention, arguments from role-specialized agents carry intrinsic strengths $\tau(\alpha)$; opposing pairs whose strengths differ by less than a threshold $\delta$ are sent to an arena adjudicator that computes a win rate $w(\alpha)$ and recalibrates strengths as $\tilde{\tau}(\alpha) = \mathrm{clip}(\tau(\alpha) + \beta(2w(\alpha)-1),0,1)$; then a gradual semantics $\sigma(x) = \tilde{\tau}(x) + (1-\tilde{\tau}(x))h(E(x)) - \tilde{\tau}(x)h(-E(x))$ with $h(z) = \max\{z,0\}^2/(1+\max\{z,0\}^2)$ propagates net support-minus-attack energy $E(x)$, and the intervention's converged score becomes its acceptability $F(o_i)$. Because the propagation is deterministic, any human edit to an argument or relation yields a reproducible recomputation of the plan.
What would settle it
A concrete test: take a set of discharge cases, have clinicians independently rank the candidate interventions, run CANOE, and check whether its acceptability scores $F(o_i)$ match the clinicians' ranking; if the correlation is weak, or if removing a safety-critical argument fails to change the plan's ranking, the central claim that the scores encode clinically meaningful acceptability fails.
Extended reading notes
Core claim
CANOE's central claim is that open-ended care plan coordination can be restructured as a contestable argumentative computation. Role-based LLM agents propose supporting and attacking arguments for candidate interventions, an Arena-based Quantitative Bipolar Argumentation Framework resolves near-tied conflicts through pairwise adjudication and propagates acceptability scores through support and attack edges, and a human care planner can edit the argument graph with deterministic recomputation of the final plan. The paper asserts that this structure provides faithful explanation—each recommendation linked to the roles, evidence, and arguments that survived—and human contestability, directly addressing the transparency and safety limits of monolithic LLM pipelines. Evaluated on Discharge Me! and MedicalRAG with three backbones, the medically fine-tuned MedGemma 1.5 achieves the best AlignScore, MEDCON F1, and LLM-judge safety and validity scores, while the larger GPT-OSS leads on ROUGE-L and coherence.
Load-bearing premise
The load-bearing premise is that the LLM-generated argument strengths, arena win rates, and QBAF propagation produce acceptability scores that are clinically meaningful without any calibration against real clinical outcomes or clinician judgment.
Editorial extensions
If this is right
- Every recommendation in a CANOE care plan carries a traceable chain: which roles argued for it, which evidence sources support it, and which arguments survived human contestation.
- Human edits modify the formal reasoning object, and the deterministic propagation guarantees that a rejected or weakened argument changes the plan in a reproducible way.
- Medically fine-tuned backbones are the safer choice for care-plan generation, since MedGemma 1.5 led on factual consistency, clinical concept coverage, and the judge's safety and validity scores.
- Larger general-purpose models trade clinical grounding for coherence and completeness, so backbone selection becomes a safety-versus-style decision.
- The same argumentative machinery can be lifted to other open-ended, multi-stakeholder coordination tasks where monolithic generation is considered unsafe.
Reading between the lines
- A direct head-to-head comparison against a monolithic LLM pipeline—which the paper does not run—would test whether the argumentative structure itself is what improves safety and transparency over a single generation pass.
- The acceptability scores are uncalibrated against clinician judgments or patient outcomes; a prospective study correlating $F(o_i)$ with readmission or adverse-event rates would test whether the framework's safety claims are empirical rather than formal.
- The arena clash-resolution step could be ablated by replacing it with plain averaging of near-tied arguments; if final rankings do not change, the added adjudication complexity may not be earning its place.
- Supporting the four human actions with direct strength overrides would test whether the propagation semantics respect qualitative clinical edits in the intended direction.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes CANOE, a five-module multi-agent neuro-symbolic framework for care plan coordination. Patient information P and retrieved evidence D are used to propose candidate interventions O, recruit role-specialized agents A, and generate supporting and attacking arguments whose strengths τ(α) are calibrated by an arena-based clash resolution mechanism and then propagated through a quantitative bipolar argumentation graph (A-QBAF). Human planners can accept, reject, edit, or add arguments, after which the framework deterministically recomputes option acceptability scores F^H(oi). A coordinator LLM Π then synthesizes the final care plan Y from the revised argumentative structure. The paper evaluates CANOE on Discharge Me! and MedicalRAG using three LLM backbones, reporting ROUGE-L, AlignScore, MEDCON F1, FKGL, and an LLM-as-a-judge protocol, and concludes that medically fine-tuned models give the strongest clinical correctness and safety and that CANOE's argumentative structure provides faithful explanation and human contestability.
Significance. If the central claims were validated, CANOE would be a meaningful step toward contestable, explainable multi-agent clinical decision support: the formal QBAF machinery in Section 4.3 is clearly specified, the deterministic recomputation after human edits is a concrete operationalization of contestability, and the evaluation metrics are all external to the framework, avoiding equation-level circularity. The framework is also fairly general and could be reused outside care planning. However, the significance is currently limited because the experiments only compare LLM backbones inside a fixed CANOE pipeline; they do not isolate the contribution of the argumentative machinery and do not measure explanation faithfulness or contestability in use. As a result, the paper's headline claims about transparency, safety, and the superiority of argumentative structure over monolithic pipelines are not empirically supported by the data presented.
major comments (5)
- [Abstract; §§5.4–5.5] The central claim that CANOE's argumentative structure provides faithful explanation, human contestability, and addresses the limitations of monolithic LLM pipelines is not tested. All experiments vary only the LLM backbone while holding the CANOE skeleton fixed, so Tables 1 and 2 are attributable to backbone capability rather than to the A-QBAF layer, arena clash resolution, or contestation module. There is no monolithic-pipeline baseline, no ablation that disables the arena mechanism (Eqs. 7–8) or propagation (Eqs. 10–12), and no user study or simulation that exercises the accept/reject/edit/add actions of Module 4. A reader cannot determine from the reported results whether the argumentative computation changes the output at all.
- [Algorithm 1, lines 31–32; §4.5] The claimed 'faithful explanation' is not verified. The coordinator Π is an LLM that receives P, D, O, ΓH, and {F^H(oi)} and generates Y; nothing in the experiments checks that the attached explanation trace (roles, evidence, survived arguments, scores) corresponds to the computation that produced each y_t. Since Π could in principle generate text from P, D, and O while ignoring the acceptability scores, the trace may be as post-hoc as the chain-of-thought rationalizations the paper criticizes in its introduction. A faithfulness metric or a direct intervention study is needed before the explanation claim can be accepted.
- [Algorithm 1, lines 14–17; §4.3, Eqs. (7)–(8)] The conflict threshold δ and calibration factor β are never assigned values in the experimental setup of Section 5.2. These parameters directly control the arena-based calibration step, so the reported results are not reproducible as stated, and the 'deterministic recompute' promised for human edits is not fully specified. The paper should either state the values used for all experiments or provide a sensitivity analysis showing that the conclusions are robust across a range of δ and β.
- [Tables 1 and 2] All reported metrics are point estimates without error bars, confidence intervals, significance tests, or per-example variance, and the dataset preprocessing does not state the number of test examples used. Claims such as MedGemma 1.5 providing the 'strongest clinical correctness and safety' rest on small differences (e.g., AlignScore 0.451 vs. 0.403 on Discharge Me!; judge average 3.85 vs. 3.75) that could easily change with a different sample. Without a measure of variability, the comparisons between backbones are not statistically grounded.
- [§5.1; §5.2] The evaluation benchmarks only weakly instantiate the claimed multidisciplinary, open-ended care-planning setting. The Discharge Me! adaptation uses discharge-instruction sections as ground truth, which is related to but not identical to care plan coordination across multiple disciplines. The MedicalRAG experiments filter to the 'Medical Practitioners' role only, so they do not exercise the role-diverse argumentative comparisons that are central to the framework's motivation. A small-scale clinician or care-coordinator evaluation, or at least a benchmark that includes multiple professional perspectives, would be needed to substantiate the multidisciplinary claims.
minor comments (5)
- [§5.1] There is a typo in 'Brief Hospitalßl' (should be 'Brief Hospital Course').
- [§5.2] The description 'all models are evaluated in a zero-shot prompted setting' is ambiguous about which LLM calls use the backbone: does each of the five modules (complexity assessment, option proposal, argument generation, strength scoring, adjudication, coordination) run the same backbone, or is the backbone used only for the final coordinator Π? Please clarify the exact model assignment per module.
- [Template 1, §5.5] The paper states that full scoring rubrics 'will be released with the code upon acceptance'; for reproducibility, the rubrics and the judge model version should be included in the paper itself or in an available supplement.
- [References] Reference [8] is cited as the source of the MedicalRAG dataset, but the reference title is 'Controlnet: A firewall for rag-based llm system', which appears to be a different work. Please correct the citation for the MedicalRAG dataset.
- [§4.3, Eq. (11)] The impact function h(z) in Eq. (12) is nonnegative by construction, so the term h(−Ei(x)) is nonzero only when the net energy is negative. A brief explanation of how this implements bidirectional attack/support would help readers not already familiar with this specific QBAF semantics.
Circularity Check
No significant circularity: the argumentative scoring chain is fully specified in the paper and the reported metrics are external; the untested transparency and contestability claims are an evaluation gap, not a circular derivation.
full rationale
The core computation is defined in Sections 4.3-4.5. Argument strength tau(alpha) is an LLM-generated input; arena clash resolution (Eqs. 7-8) modifies tau using pairwise adjudicator win rates; propagation (Eqs. 10-12) deterministically produces F(oi); and the final plan is synthesized by coordinator Pi from P, D, O, Gamma_H, and the scores. None of these steps fits a parameter to the evaluation targets (ROUGE-L, AlignScore, MEDCON F1, FKGL, or the LLM judge), so no reported metric reduces by construction to an input or fitted value. The paper cites self-authored prior work [4,15] for the Arena-based QBAF mechanism, but Section 4.3 restates the full equations rather than importing an unverified black-box result, so the self-citation is not load-bearing in a circular way. The central claims that CANOE provides 'faithful explanation and human contestability' are not adequately tested by the backbone-only comparison in Sections 5.4-5.5, and the unreported values of delta and beta prevent full reproducibility; however, these are evidence and validation gaps, not equation-level circularity. Under the hard rule that circularity requires a quoted reduction of a prediction to its input, no such step is present.
Assumptions & free parameters
free parameters (3)
- conflict threshold delta
- calibration factor beta
- intervention node prior =
0.5
assumptions (5)
- standard math QBAF gradual semantics (Eqs. 10-12) from Baroni et al. yield valid acceptability scores for clinical decision support.
- domain assumption LLM-assigned intrinsic strengths tau(alpha) measure clinically meaningful argument quality.
- domain assumption The arena win rate w(alpha) from LLM adjudication identifies the clinically stronger argument in near-tied pairs.
- domain assumption Complexity classifier C and recruitment operator S(.) improve plan quality.
- domain assumption Synthesis planner Pi converts acceptability scores into clinically safe and coherent plans.
Cite this review
Pith. "Pith review of Adaptive Arena-based Contestable Argumentative Network-of-Experts for Open-Ended Care Plan Coordination." pith.science (2026). https://pith.science/paper/CTRCRHJ3
@misc{pith2026260805391,
author = {Pith},
title = {Pith review of: Adaptive Arena-based Contestable Argumentative Network-of-Experts for Open-Ended Care Plan Coordination},
year = {2026},
howpublished = {\url{https://pith.science/paper/CTRCRHJ3}},
note = {Machine review of arXiv:2608.05391}
}
read the original abstract
Care plan coordination demands synthesizing heterogeneous clinical, functional, and psychosocial information across multiple professional disciplines, where monolithic LLM pipelines cannot perform in a transparent or safe manner. We present CANOE (Contestable Argumentative Network-of-Experts), a multi-agent neuro-symbolic framework that addresses these limitations through five modules: complexity assessment, adaptive team recruitment, role-based argumentative computation via an Arena-based Quantitative Bipolar Argumentation Framework (A-QBAF), human-in-the-loop contestation, and care-plan synthesis. Role-specialized agents generate supporting and attacking arguments for candidate interventions; conflicts are resolved through arena-based clash resolution before acceptability scores propagate across the argumentation graph. Care planners may accept, reject, edit, or add arguments, and the framework will deterministically recompute the final plan. Evaluation on Discharge Me! and MedicalRAG using ROUGE-L, AlignScore, MEDCON F1, FKGL, and LLM-as-a-judge shows that medically fine-tuned models achieve the strongest clinical correctness and safety, while CANOE's argumentative structure provides faithful explanation and human contestability.
Figures
Reference graph
Works this paper leans on
-
[1]
arXiv preprint arXiv:2508.10925 (2025)
Agarwal, S., et al.: gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925 (2025)
arXiv 2025
-
[2]
Minds and Machines33(4), 613–639 (2023), publisher: Springer
Alfrink, K., Keller, I., Kortuem, G., Doorn, N.: Contestable AI by design: Towards a framework. Minds and Machines33(4), 613–639 (2023), publisher: Springer
2023
-
[3]
International Journal of Approximate Reasoning 105(2019)
Baroni, P., et al.: From fine-grained properties to broad principles for gradual argu- mentation: A principled spectrum. International Journal of Approximate Reasoning 105(2019)
work page 2019
-
[4]
In: The 39th Canadian Conference on Artificial Intelligence
Cao, H.L., et al.: Neuro-symbolic adaptive collaboration of arena-based argumen- tative llms for contestable legal reasoning. In: The 39th Canadian Conference on Artificial Intelligence. pp. 895–902. PMLR (2026)
work page 2026
-
[5]
Damm, H., et al.: Wispermed at “discharge me!”: Advancing text generation in healthcare with large language models, dynamic expert selection, and priming techniques on mimic-iv. In: Proceedings of the 23rd Workshop on Biomedical Natural Language Processing. pp. 105–121 (2024)
work page 2024
-
[6]
In: Proceedings of the AAAI Conference on Artificial Intelligence
Freedman, G., et al.: Argumentative Large Language Models for Explainable and Contestable Claim Verification. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 39, pp. 14930–14939 (2025), issue: 14
work page 2025
-
[7]
In: 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)
Hong, S., et al.: Argmed-agents: Explainable clinical decision reasoning with llm disscusion via argumentation schemes. In: 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). pp. 5486–5493. IEEE (2024)
work page 2024
-
[8]
arXiv preprint arXiv:2504.09593 (2025)
Hongwei, Y., et al.: Controlnet: A firewall for rag-based llm system. arXiv preprint arXiv:2504.09593 (2025)
arXiv 2025
Show all 29 references
-
[9]
In: ITM Web of Conferences
Huang, S.: Hallucinations of large language models in medical environments: A systematic review of risks, detection, and mitigation. In: ITM Web of Conferences. vol. 84, p. 03005. EDP Sciences (2026)
2026
-
[10]
The Journal for Nurse Practitioners18(8), 833–836 (2022)
Ju, H.H.: Improving care coordination of patients with chronic diseases. The Journal for Nurse Practitioners18(8), 833–836 (2022)
2022
-
[11]
Advances in Neural Information Processing Systems37, 79410–79452 (2024) Contestable Argumentative Network-of-Experts for Care Coordination 17
Kim, Y., et al.: Mdagents: An adaptive collaboration of llms for medical decision- making. Advances in Neural Information Processing Systems37, 79410–79452 (2024) Contestable Argumentative Network-of-Experts for Care Coordination 17
2024
-
[12]
In: Proceed- ings of the 21st International Conference on Principles of Knowledge Representation and Reasoning
Leofante, F., et al.: Contestable ai needs computational argumentation. In: Proceed- ings of the 21st International Conference on Principles of Knowledge Representation and Reasoning. pp. 888–896 (2024)
2024
-
[13]
npj Digital Medicine (2026)
Liu, Q., Hu, Z., Huang, T., Niu, Y., Zhang, X., Ma, S., Lin, C., Huat, G.K., Kwon, H.E., Gao, F., et al.: Evomdt: a self-evolving multi-agent system for structured clinical decision-making in multi-cancer. npj Digital Medicine (2026)
2026
-
[14]
ACM Trans
Nguyen, H., Rahimi, A., Whitford, V., Fournier, H., Kondratova, I., Richard, R., Cao, H.: Heart2mind: Human-centered contestable psychiatric disorder prediction system using wearable ecg monitors. ACM Trans. Comput. Healthcare (2026)
2026
-
[15]
In: Proceedings of the 2026 International Conference on Multimedia Retrieval
Nguyen, H.T.T., et al.: Contestable multi-agent debate with arena-based argu- mentative computation for multimedia verification. In: Proceedings of the 2026 International Conference on Multimedia Retrieval. pp. 2887–2891 (2026)
2026
-
[16]
In: Proceedings of 9th International Symposium on Chatbots and Human-centred AI (2025)
Nguyen, L.P.T., et al.: Motion2Meaning: A Clinician-Centered Framework for Contestable LLM in Parkinson’s Disease Gait Interpretation. In: Proceedings of 9th International Symposium on Chatbots and Human-centred AI (2025)
2025
-
[17]
In: International Workshop on Health Intelligence
Nguyen, T.T.H., et al.: Towards trust of explainable ai in thyroid nodule diagnosis. In: International Workshop on Health Intelligence. pp. 11–26. Springer (2023)
2023
-
[18]
Artificial intelligence in medicine107(2020)
Ploug, T., Holm, S.: The four dimensions of contestable AI diagnostics-A patient- centric approach to explainable AI. Artificial intelligence in medicine107(2020)
2020
-
[19]
In: Proceedings of the International Conference on Principles of Knowledge Representation and Reasoning
Rago, A., Li, H., Toni, F.: Interactive explanations by conflict resolution via argu- mentative exchanges. In: Proceedings of the International Conference on Principles of Knowledge Representation and Reasoning. vol. 19, pp. 582–592 (2023)
2023
-
[20]
arXiv preprint arXiv:2604.05081 (2026)
Sellergren, A., et al.: Medgemma 1.5 technical report. arXiv preprint arXiv:2604.05081 (2026)
2026 arXiv
-
[21]
Annals of internal medicine165(11), 753–760 (2016)
Sinsky, C., et al.: Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties. Annals of internal medicine165(11), 753–760 (2016)
2016
-
[22]
In: Findings of the Association for Computational Linguistics: ACL 2024 (2024)
Tang, X., et al.: Medagents: Large language models as collaborators for zero-shot medical reasoning. In: Findings of the Association for Computational Linguistics: ACL 2024 (2024)
2024
-
[23]
Advances in neural information processing systems35, 24824–24837 (2022)
Wei, J., et al.: Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems35, 24824–24837 (2022)
2022
-
[24]
In: Healthcare
Xiao, L., Greer, D.: Linked argumentation graphs for multidisciplinary decision support. In: Healthcare. vol. 11, p. 585. MDPI (2023)
2023
-
[25]
discharge me!
Xu, J., et al.: Overview of the first shared task on clinical text generation: Rrg24 and “discharge me!”. In: Proceedings of the 23rd Workshop on Biomedical Natural Language Processing. pp. 85–98 (2024)
2024
-
[26]
Scientific data10(1), 586 (2023)
Yim, W.w., et al.: Aci-bench: a novel ambient clinical intelligence dataset for benchmarking automatic visit note generation. Scientific data10(1), 586 (2023)
2023
-
[27]
In: Proceedings of the 15th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics
Yue, L., et al.: Clinicalagent: Clinical trial multi-agent system with large language model-based reasoning. In: Proceedings of the 15th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics. pp. 1–10 (2024)
2024
-
[28]
In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Zha, Y., et al.: Alignscore: Evaluating factual consistency with a unified align- ment function. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 11328–11348 (2023)
2023
-
[29]
In: Conference on Neurosymbolic Learning and Reasoning
Zhu, Y., et al.: Argrag: Explainable retrieval augmented generation using quan- titative bipolar argumentation. In: Conference on Neurosymbolic Learning and Reasoning. pp. 697–718. PMLR (2025)
2025
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.