pith. sign in

arxiv: 2605.20098 · v1 · pith:DIBSKAPCnew · submitted 2026-05-19 · 💻 cs.AI

Neurosymbolic Learning for Inference-Time Argumentation

Gabriel Freedman , Adam Dejl , Adam Gould , Mansi , Lihu Chen , Jianqi Jiang , Francesca Toni This is my paper

Pith reviewed 2026-05-20 05:07 UTC · model grok-4.3

classification 💻 cs.AI
keywords neurosymbolic learninginference-time argumentationternary claim verificationformal argumentation semanticsfaithful explanationslarge language modelsargument generationbase scores
0
0 comments X

The pith

A neurosymbolic framework trains language models to generate arguments whose formal evaluation yields faithful ternary claim verifications.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents inference-time argumentation, a framework that integrates formal argumentation semantics into the training and inference of large language models for ternary claim verification. During training, the semantics guides the model to produce arguments and assign base scores in a way that optimizes the quality of the resulting predictions. At inference, the same semantics computes the final true, false, or uncertain verdict directly from the generated arguments and scores. This setup ensures that predictions are always faithful to explicit, inspectable argumentative structures rather than opaque reasoning traces. Such faithfulness matters in high-stakes domains where understanding the basis for a verdict on incomplete or conflicting information is essential.

Core claim

Inference-time argumentation is a trainable neurosymbolic framework for ternary claim verification in which a formal argumentation semantics giving the strength of claims is used both to guide LLM training as models learn to generate arguments and assign them base scores and to compute ternary predictions from generated, scored arguments, making the final prediction faithful by construction to the arguments and scores.

What carries the argument

Inference-time argumentation, which uses formal argumentation semantics both to supervise the generation of arguments with base scores during training and to deterministically derive ternary verdicts from those outputs at inference time.

If this is right

  • ITA improves upon argumentative baselines on two ternary claim verification datasets.
  • ITA performs competitively against non-argumentative direct-prediction baselines.
  • Verdicts are computed deterministically from explicit, inspectable argumentative structures.
  • Argument generation and scoring are optimised during training according to the quality of the induced predictions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The method suggests a general pattern for making neural reasoning steps symbolically verifiable in other tasks beyond claim verification.
  • Explicit argumentative structures could enable post-hoc auditing or correction by domain experts in high-stakes applications.
  • Combining neural generation with symbolic evaluation might reduce reliance on post-hoc explanation techniques in language models.

Load-bearing premise

The formal argumentation semantics accurately reflects the quality of the generated arguments and produces reliable ternary predictions from the base scores.

What would settle it

Comparing the ternary predictions derived from the argumentation semantics against ground-truth labels on the two claim verification datasets, and checking whether accuracy drops when the semantics is removed from the training objective.

Figures

Figures reproduced from arXiv: 2605.20098 by Adam Dejl, Adam Gould, Francesca Toni, Gabriel Freedman, Jianqi Jiang, Lihu Chen, Mansi.

Figure 1
Figure 1. Figure 1: An example argumentative structure for a claim adapted from DEBATunE [Li et al., 2024a]. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of inference-time argumentation. Given a claim, the argument generator produces [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
read the original abstract

Claim verification is an important problem in high-stakes settings, including health and finance. When information underpinning claims is incomplete or conflicting, uncertain answers may be more appropriate than binary true or false classifications. In all cases, faithful explanations of the considerations determining the final verdict are crucial. We introduce inference-time argumentation (ITA), a trainable neurosymbolic framework for ternary claim verification in which a formal argumentation semantics giving the strength of claims is used both (i) to guide LLM training as models learn to generate arguments and assign them base scores (representing intrinsic strengths) and (ii) to compute ternary (true/false/uncertain) predictions from generated, scored arguments. As a result, at training time, argument generation and scoring can be optimised according to the quality of the induced argumentative predictions. Moreover, at inference time, the final prediction is faithful, by construction, to the arguments and scores determining the verdict, rather than being justified by a potentially unfaithful post-hoc reasoning trace as in conventional reasoning models. We finally show that, on two datasets for ternary claim verification, ITA improves upon argumentative baselines and can perform competitively against non-argumentative direct-prediction baselines, while providing verdicts that are computed deterministically from explicit, inspectable argumentative structures.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper introduces Inference-Time Argumentation (ITA), a trainable neurosymbolic framework for ternary claim verification. An LLM is trained to generate arguments and assign base scores; a formal argumentation semantics then uses these to both supervise training (by optimizing for the quality of the induced predictions) and to deterministically compute the final true/false/uncertain verdict at inference time. The authors claim that this yields predictions that are faithful by construction to explicit, inspectable argumentative structures, and report that ITA improves on argumentative baselines while remaining competitive with direct-prediction models on two datasets.

Significance. If the mapping from generated arguments and scores to ternary verdicts is shown to be reliable, ITA would offer a concrete route to neurosymbolic claim verification that is both optimizable end-to-end and guaranteed to be consistent with an explicit argumentative structure. This addresses a recognized weakness of post-hoc explanation methods in high-stakes domains and could serve as a template for other neurosymbolic integrations of formal reasoning with LLMs.

major comments (2)
  1. [Abstract and §4 (results)] The headline claim that predictions are 'faithful by construction' (abstract) rests on the premise that the chosen formal argumentation semantics correctly converts LLM-generated arguments and base scores into reliable ternary labels. No section, table, or experiment validates this mapping against human uncertainty judgments or shows that the semantics does not systematically over- or under-estimate argument strength; without such evidence the faithfulness property is formal but not necessarily substantive.
  2. [Abstract and experimental results] The abstract states that ITA 'improves upon argumentative baselines and can perform competitively' on two datasets, yet supplies no dataset sizes, error bars, ablation studies, or statistical tests. This absence makes it impossible to determine whether the reported gains are robust or whether they depend on particular choices of semantics or training objective.
minor comments (2)
  1. [§3 (method)] Specify the exact weighted or gradual argumentation semantics employed and the precise functional form by which base scores are combined with argument structure to produce claim strength.
  2. [§3] Add a clear statement of the training objective (loss) that links argument generation and scoring to the quality of the final ternary prediction.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive and insightful comments on our manuscript introducing Inference-Time Argumentation (ITA). We address each major comment below, providing clarifications and outlining specific revisions to improve the paper's rigor and transparency.

read point-by-point responses

  1. Referee: [Abstract and §4 (results)] The headline claim that predictions are 'faithful by construction' (abstract) rests on the premise that the chosen formal argumentation semantics correctly converts LLM-generated arguments and base scores into reliable ternary labels. No section, table, or experiment validates this mapping against human uncertainty judgments or shows that the semantics does not systematically over- or under-estimate argument strength; without such evidence the faithfulness property is formal but not necessarily substantive.

    Authors: We thank the referee for this important observation. The claim of faithfulness 'by construction' specifically denotes that the ternary verdict is obtained via a deterministic computation from the model's explicitly generated arguments and base scores using the formal semantics; this guarantees that the output is consistent with and inspectable via the argumentative structure, as opposed to post-hoc rationalizations common in direct LLM predictions. It does not assert that the semantics is the optimal or human-aligned model of uncertainty. We agree that the manuscript would benefit from greater clarity on this distinction and from some empirical grounding. In the revised version we will (i) rephrase the abstract and §1 to emphasize the formal, construction-based nature of the faithfulness guarantee and (ii) add a short discussion subsection that acknowledges the semantics' assumptions and reports a preliminary comparison of ITA predictions against available human uncertainty annotations in the datasets. A full-scale human validation study lies beyond the scope of the current work but will be noted as valuable future research. revision: partial

  2. Referee: [Abstract and experimental results] The abstract states that ITA 'improves upon argumentative baselines and can perform competitively' on two datasets, yet supplies no dataset sizes, error bars, ablation studies, or statistical tests. This absence makes it impossible to determine whether the reported gains are robust or whether they depend on particular choices of semantics or training objective.

    Authors: The referee correctly notes that the abstract omits these quantitative details. The full experimental section (§4) already contains dataset cardinalities, results averaged over multiple random seeds with standard deviations, and comparisons across semantics variants; however, we acknowledge that ablations on the training objective and formal statistical tests (e.g., paired t-tests or Wilcoxon tests) are not presented with sufficient prominence. We will revise the abstract to include concise statements of dataset scale and performance variability, and we will expand §4 with additional ablation tables and significance tests. These changes will make the robustness of the reported improvements explicit without altering the core experimental narrative. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation self-contained

full rationale

The paper defines ITA as a neurosymbolic loop in which an external formal argumentation semantics computes claim strengths from LLM-generated arguments and base scores, then derives ternary predictions from those strengths. Training optimizes the LLM so that the semantics-induced predictions match dataset labels, while inference uses the same deterministic mapping. This structure does not reduce any claimed result to a definitional tautology, a fitted parameter renamed as prediction, or a self-citation chain; the semantics is invoked as a pre-existing formal tool rather than derived from the model's outputs or prior self-work. Empirical gains are shown via direct comparison to baselines on held-out data, and the 'faithful by construction' property follows directly from the explicit separation of generation, scoring, and symbolic evaluation steps. No load-bearing step collapses to its own inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The central claim rests on the assumption that formal argumentation semantics can be used both as a training signal and as a deterministic inference procedure; no free parameters, axioms, or invented entities are explicitly listed in the abstract.

pith-pipeline@v0.9.0 · 5758 in / 1144 out tokens · 24691 ms · 2026-05-20T05:07:50.644374+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 24 canonical work pages · 7 internal anchors

  1. [1]

    Zero-shot scientific claim verification us- ing LLMs and citation text

    Carlos Alvarez, Maxwell Bennett, and Lucy Wang. Zero-shot scientific claim verification us- ing LLMs and citation text. In Tirthankar Ghosal, Amanpreet Singh, Anita Waard, Philipp Mayr, Aakanksha Naik, Orion Weller, Yoonjoo Lee, Shannon Shen, and Yanxia Qin, editors, Proceedings of the Fourth Workshop on Scholarly Document Processing (SDP 2024), pages 269...

    work page 2024

  2. [2]

    URL https://aclanthology.org/2024.sdp-1.25/

    Association for Computational Linguistics. URL https://aclanthology.org/2024.sdp-1.25/. Alfonso Amayuelas, Kyle Wong, Liangming Pan, Wenhu Chen, and William Yang Wang. Knowledge of knowledge: Exploring known-unknowns uncertainty with large language models. InFindings of the Association for Computational Linguistics ACL 2024, pages 6416–6432,

    work page 2024

  3. [3]

    doi: 10.1609/aimag.v38i3.2704

    ISSN 0738-4602. doi: 10.1609/aimag.v38i3.2704. URL https://doi.org/10.1609/aimag.v38i3.2704. Pietro Baroni, Antonio Rago, and Francesca Toni. From fine-grained properties to broad principles for gradual argumentation: A principled spectrum.International Journal of Approximate Reasoning, 105:252–286,

    work page doi:10.1609/aimag.v38i3.2704

  4. [4]

    doi: https://doi.org/10.1016/j.ijar.2018.11.019

    ISSN 0888-613X. doi: https://doi.org/10.1016/j.ijar.2018.11.019. URL https://www.sciencedirect.com/science/article/pii/S0888613X18304651. Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schulman, Arushi Somani, Peter Hase, Misha Wagner, Fabien Roger, Vlad Mikulik, Samuel R. Bowman, Jan Leike, Jared Kaplan, and Ethan Perez...

    work page doi:10.1016/j.ijar.2018.11.019 2018

  5. [5]

    Reasoning Models Don't Always Say What They Think

    URLhttps://arxiv.org/abs/2505.05410. Roi Cohen, Konstantin Dobler, Eden Biran, and Gerard de Melo. I don’t know: Explicit modeling of uncertainty with an [IDK] token.Advances in Neural Information Processing Systems, 37: 10935–10958,

    work page internal anchor Pith review Pith/arXiv arXiv

  6. [6]

    Chrisanna Cornish and Anna Rogers

    URLhttps://www.proceedings.com/079017-0349.html. Chrisanna Cornish and Anna Rogers. Examining the faithfulness of deepseek R1’s chain-of-thought reasoning. In Aman Sinha, Raúl Vázquez, Timothee Mickus, Rohit Agarwal, Ioana Buhnila, Patrícia Schmidtová, Federica Gamba, Dilip K. Prasad, and Jörg Tiedemann, editors,Proceedings of the 1st Workshop on Confabul...

    work page 2025

  7. [7]

    Argumentation for Explainable and Globally Contestable Decision Support with LLMs

    Association for Computational Linguistics. ISBN 979-8-89176-308-1. doi: 10.18653/v1/2025.chomps-main.2. URLhttps://aclanthology.org/2025.chomps-main.2/. 10 Adam Dejl, Matthew Williams, and Francesca Toni. Argumentation for explainable and globally contestable decision support with LLMs.CoRR, abs/2603.14643,

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.18653/v1/2025.chomps-main.2 2025

  8. [9]

    From scale to speed: Adaptive test-time scaling for image editing.CoRR, abs/2603.00141, 2026

    doi: 10.48550/ARXIV .2408.14317. URLhttps://doi.org/10.48550/arXiv.2408.14317. Gabriel Freedman and Francesca Toni. Exploring the potential for large language models to demon- strate rational probabilistic beliefs. InThe International FLAIRS Conference Proceedings,

    work page internal anchor Pith review doi:10.48550/arxiv

  9. [10]

    doi: 10.1609/aaai.v39i14.33637

    ISBN 978-1-57735-897-8. doi: 10.1609/aaai.v39i14.33637. URLhttps://doi.org/10.1609/aaai.v39i14.33637. Yang Gao and Francesca Toni. Argumentation accelerated reinforcement learning for coopera- tive multi-agent systems. In Torsten Schaub, Gerhard Friedrich, and Barry O’Sullivan, editors, ECAI 2014 - 21st European Conference on Artificial Intelligence, 18-2...

    work page doi:10.1609/aaai.v39i14.33637 2014

  10. [11]

    URLhttps://doi.org/10.3233/978-1-61499-419-0-333

    doi: 10.3233/ 978-1-61499-419-0-333. URLhttps://doi.org/10.3233/978-1-61499-419-0-333. Adam Gould and Francesca Toni. Neuro-argumentative learning with case-based reasoning. In Leilani H. Gilpin, Eleonora Giunchiglia, Pascal Hitzler, and Emile van Krieken, editors,Proceed- ings of The 19th International Conference on Neurosymbolic Learning and Reasoning (...

    work page doi:10.3233/978-1-61499-419-0-333 2025

  11. [12]

    doi: 10.1038/s41586-025-09422-z

    ISSN 1476-4687. doi: 10.1038/ s41586-025-09422-z. URLhttp://dx.doi.org/10.1038/s41586-025-09422-z. Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. InInternational Conference on Learning Representations,

    work page doi:10.1038/s41586-025-09422-z

  12. [13]

    Can LLMs speak for diverse people? Tuning LLMs via debate to generate controllable controversial statements

    Ming Li, Jiuhai Chen, Lichang Chen, and Tianyi Zhou. Can LLMs speak for diverse people? Tuning LLMs via debate to generate controllable controversial statements. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Findings of the Association for Computational Linguistics ACL 2024, pages 16160–16176, Bangkok, Thailand and virtual meeting, August 2024...

    work page arXiv 2024

  13. [14]

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

    URLhttps://arxiv.org/abs/2402.03300. Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, et al. OpenAI GPT-5 system card.arXiv preprint arXiv:2601.03267,

    work page internal anchor Pith review Pith/arXiv arXiv

  14. [15]

    FEVER: a large-scale dataset for fact extraction and VERification

    James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. FEVER: a large-scale dataset for fact extraction and VERification. InProceedings of the 2018 Confer- ence of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 809–819, New Orleans, Louisiana, June

    work page 2018

  15. [16]

    FEVER : a Large-scale Dataset for Fact Extraction and VER ification

    Association for Computational Linguistics. doi: 10.18653/v1/N18-1074. URL https://aclanthology.org/N18-1074/. Petros Vasileiadis, Emanuele De Angelis, Maurizio Proietti, and Francesca Toni. Neuro-argumentative learning with legal text. InProceedings of the 1st International Workshop on Advanced Neuro- Symbolic Applications (ANSyA 2025), co-located with th...

    work page internal anchor Pith review doi:10.18653/v1/n18-1074 2025

  16. [17]

    Juraj Vladika, Ivana Hacajová, and Florian Matthes

    URL https: //ceur-ws.org/Vol-4125/paper_22.pdf. Juraj Vladika, Ivana Hacajová, and Florian Matthes. Step-by-step fact verification system for medical claims with explainable reasoning. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors, Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguist...

    work page 2025

  17. [18]

    URL https: //doi.org/10.18653/v1/2025.naacl-short.68

    doi: 10.18653/V1/2025.NAACL-SHORT.68. URL https: //doi.org/10.18653/v1/2025.naacl-short.68. Francis Rhys Ward, Francesco Belardinelli, and Francesca Toni. Argumentative reward learning: Reasoning about human preferences. InICML 2022 Workshop on Human-Machine Collaboration and Teaming,

    work page doi:10.18653/v1/2025.naacl-short.68 2025

  18. [19]

    Qwen3 Technical Report

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    work page internal anchor Pith review Pith/arXiv arXiv

  19. [20]

    Outcome Rewards Do Not Guarantee Verifiable or Causally Important Reasoning

    URL https://arxiv.org/abs/ 2604.22074. Yilun Zhao, Yitao Long, Tintin Jiang, Chengye Wang, Weiyuan Chen, Hongjun Liu, Xiangru Tang, Yiming Zhang, Chen Zhao, and Arman Cohan. FinDVer: Explainable claim verification over long and hybrid-content financial documents. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pag...

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  20. [21]

    Eval- uating uncertainty quantification methods in argumentative large language models

    12 Kevin Zhou, Adam Dejl, Gabriel Freedman, Lihu Chen, Antonio Rago, and Francesca Toni. Eval- uating uncertainty quantification methods in argumentative large language models. In Chris- tos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Find- ings of the Association for Computational Linguistics: EMNLP 2025, Suzhou, China,...

    work page 2025

  21. [22]

    Yuqicheng Zhu, Nico Potyka, Daniel Hernández, Yuan He, Zifeng Ding, Bo Xiong, Dongzhuoran Zhou, Evgeny Kharlamov, and Steffen Staab

    URL https://aclanthology.org/2025.findings-emnlp.1184/. Yuqicheng Zhu, Nico Potyka, Daniel Hernández, Yuan He, Zifeng Ding, Bo Xiong, Dongzhuoran Zhou, Evgeny Kharlamov, and Steffen Staab. ArgRAG: Explainable retrieval augmented generation using quantitative bipolar argumentation. In Leilani H. Gilpin, Eleonora Giunchiglia, Pascal Hitzler, and Emile van K...

    work page 2025

  22. [23]

    {claim}”. Output: { ’support’: [ “<SUPPORT ARGUMENT 1>

    URL https: //proceedings.mlr.press/v284/zhu25a.html. A Prompts The prompts shown below are used for all variations of the respective components. A.1 Argument Generation Please provide a set of short arguments supporting and attacking the following claim. Construct the arguments so they refer to the truthfulness of the claim. The arguments should be short ...

    work page 2025

  23. [24]

    Ranking is measured as rank agreement with the expected ordering of argument strengths, so higher values indicate better sensitivity to relative strength

    Inconsistency is measured as the variance of the scores assigned to semantically equivalent arguments, averaged across paraphrase groups, so lower values indicate greater stability. Ranking is measured as rank agreement with the expected ordering of argument strengths, so higher values indicate better sensitivity to relative strength. Both the training re...

    work page 2025

  24. [25]

    The argumentative reward signal is derived from the gradual semantics score of the model’s output argument

    Each GRPO step samples 2 completions per prompt, with a maximum prompt length of 512 tokens and a maximum completion length of2,048tokens. The argumentative reward signal is derived from the gradual semantics score of the model’s output argument. When using a learned BSM, the base score for each claim is computed by the pre- trained BSM regression head pr...

    work page 2025