Pith. sign in

REVIEW 4 major objections 4 minor 73 references

Agent-Based Detection and Resolution of Incompleteness and Ambiguity in Interactions with Large Language Models

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Adding an agent that rewrites unclear questions boosts LLM answer accuracy on 5 of 6 datasets.

desk verdict The agent-based QA wrapper idea is plausible and worth discussing, but the reported accuracy gains are not yet reliable: the evaluation inserts the resolver's explanation into the responder's context, creating a likely answer-leakage channel, plus the statistics are thin. read the letter →

arxiv 2507.03726 v1 pith:VGVXTPC6 submitted 2025-07-04 cs.AI cs.CLcs.IR

Riya Naik (1) , Ashwin Srinivasan (1) , Swati Agarwal (2) , Estrid He (3) ((1) BITS Pilani K K Birla Goa Campus , (2) PandaByte Innovations Pvt Ltd , (3) RMIT University) This is my paper · ORCID
classification cs.AIcs.CLcs.IR
keywords questionansweringLLMagentsReActincompletenessambiguitytransducerzero-shotclassificationinteractivedialogue
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tests a simple idea: before an LLM answers a question, an agent-based 'question-transducer' checks whether the question is incomplete or ambiguous, rewrites it if it is, and only then lets the LLM answer. The authors argue that this classify-resolve-answer loop improves interactive question-answering performance, shortens human clarification turns, and makes the resolution of deficiencies explainable. Across six benchmark datasets with varying levels of incompleteness and ambiguity, the transducer improved answer accuracy on five of them, with gains ranging from 3 to 40 percentage points by the last turn tabulated. The approach does not help when questions already carry enough context, because the classifier then tends to alter normal questions. The costs are additional LLM invocations and, in some cases, higher latency, which the paper judges acceptable except in the already-well-specified case.

What carries the argument

The question-transducer is a context-transducer for questions: a function that maps a context ending in a question to a new context ending in a rewritten question. It is implemented as two zero-shot ReAct LLM agents: Classify labels the question as incomplete, ambiguous, or normal, and Resolve rewrites deficient questions into clear and complete ones or returns a clarifying question. The rewritten question, together with the explanation produced by Resolve, is inserted into the context that the responder LLM receives, and the responder then answers the resolved form. The ReAct loop lets the agent alternate reasoning and action until its classification or resolution goal is achieved.

What would settle it

Re-run the evaluation but remove the explanation from the transduced context before the responder answers, keeping only the rewritten question; if accuracy falls substantially, the explanation is leaking the answer. A complementary check is to measure token overlap between the resolver's explanation and the ground-truth answer on the resolved questions.

Watch

Extended reading notes

Core claim

The central claim is that the conjecture holds: the inclusion of a question-transducer to detect and resolve incomplete or ambiguous questions improves the interactive question-answering performance of an LLM. Accuracy is measured after each turn against human-labeled ground truth on 600 interactions from SQuAD, NQ-open, AmbigNQ, MedDialog, MultiWOZ, and ShARC, using GPT-3.5-Turbo as the responder and either GPT-3.5-Turbo or Llama-4-Scout as the transducer LLM. The paper reports that for 5 of the 6 datasets, introducing the transducer increases accuracy even from the first turn, and that the gain ranges from 3% to 40% by the last turn. It also reports that the GPT-based transducer outperforms the Llama-based one on 3 datasets, and vice versa. The exception is SQuAD, where accuracy drops because the Classify function mislabels many normal questions, so the paper concludes the transducer is not useful when questions already contain sufficient context.

Load-bearing premise

The evaluation assumes that the explanation inserted by the resolver into the responder's context does not already contain the answer; if it does, the measured accuracy gain could be a copying effect rather than evidence that detection and resolution improve question answering.

Editorial extensions

If this is right

  • On five of the six datasets tested, adding the transducer improves answer accuracy, with gains of 3 to 40 percentage points by the third turn.
  • The transducer shortens interactions by automatically resolving incompleteness and ambiguity, reducing the number of human clarification turns needed.
  • The approach costs extra LLM invocations and added latency, so it is best applied selectively rather than on every query.
  • When questions already contain sufficient context, the transducer can reduce accuracy, so a reliable Classify function is essential to avoid rewriting normal questions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The protocol inserts the resolver's explanation into the responder's context, and ReAct explanations often state the answer before the final response; the paper does not test whether the accuracy gain is partly answer leakage rather than genuine resolution.
  • A simple ablation—handing the responder only the rewritten question, without the explanation—would isolate whether the gain comes from the resolution or from copying the explanation.
  • The Classify component's unpredictable behavior suggests a small trained classifier on human labels, with the agent-based Resolve retained, might be a more reliable and cheaper architecture.
  • The same transducer pattern could transfer to other LLM-facing tasks where a deficient prompt blocks a deterministic downstream step, such as code generation or database queries.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a context-transducer for interactive question-answering with LLMs. The transducer is implemented as zero-shot ReAct LLM agents that first classify a question as incomplete, ambiguous, or normal and then, if needed, rewrite the question before the responder-LLM answers. The authors evaluate on six QA datasets (SQuAD, NQ-open, AmbigNQ, MedDialog, MultiWOZ, ShARC) using GPT-3.5-Turbo as the responder and either GPT-3.5-Turbo or Llama-4-Scout as the transducer-LLM. They report accuracy gains in five of the six datasets by the third turn, claiming improvements from 3% to 40%, as well as shortened interactions, at the cost of extra LLM calls and latency. The central claim is that including the question-transducer improves interactive QA performance.

Significance. If the reported gains are reliable, the paper describes a lightweight, zero-shot wrapper that could make LLM-based QA systems more robust to incomplete and ambiguous questions. The formal oracle-based definitions in Section 2 are a useful conceptual contribution, and it is a genuine strength that the transducer's Classify and Resolve components are implemented zero-shot with no parameter fitted to the test labels; the dataset characterization in Appendix A is explicitly retroactive and is not available to the transducer, so circularity is avoided. The paper also honestly lists several limitations, including the unpredictability of the Classify function and the need for diverse human verifiers. However, the current evaluation does not yet establish the central claim because of a plausible answer-leakage channel and the absence of statistical controls.

major comments (4)
  1. [Section 4.3, Procedure 2] The evaluation does not control for answer leakage from the resolver's explanation into the responder's context. Procedure 2 (line 4) constructs c' = (m1,...,mk, e, !α(q')), placing the resolver's explanation e (and, as printed, an answer message) directly in front of the responder-LLM. Since Λr is implemented as a zero-shot ReAct agent, e may state or imply the final answer, as the worked example in Table 6 indeed shows the agent discussing 'John Powell' before the final answer. The accuracy gains in Table 1 could therefore reflect the responder copying the answer from e rather than an improved question. I request an ablation that removes e from c' (or masks any answer spans), and a measurement of substring/n-gram overlap between e and the ground-truth answer, for all datasets. Without such a control, the central claim of Section 4.4 is not supported.
  2. [Section 4.3] The reported accuracies lack measures of uncertainty. The method states that there are about 600 (c,r) entries overall (Section 4.3, last bullet), so each dataset has roughly 100 instances per turn. Differences such as NQ-open turn 1 (0.81 vs 0.83) and AmbigNQ turn 1 (0.63 vs 0.65) are well within sampling error. The authors should report confidence intervals (e.g., bootstrap) or significance tests for the with- versus without-transducer comparisons in Table 1, and should state the exact number of instances per dataset. In addition, the manual AGREES verification was performed by a single human; an inter-annotator agreement measure on a subset is needed.
  3. [Section 4.1, Section 4.3] The design does not include a control for the extra inference step itself. The 'without transducer' condition is a single LLM call answering the original question; the 'with transducer' condition is a multi-step ReAct loop that produces a rewritten question and explanation. To attribute gains to detection and resolution of incompleteness/ambiguity, the authors should add a baseline in which the question is rewritten by a simple LLM prompt (e.g., 'rewrite this question to be complete and unambiguous before answering') without the classify-resolve machinery, and a condition in which the responder receives the rewritten question without the explanation e. Without these controls, the gains could be caused by the additional compute or by the general benefit of reformulation, not by the claimed agent-based detection.
  4. [Section 4.4, Table 3] The behavior of Classify is inconsistent with the dataset characterisation used to motivate the evaluation. For AmbigNQ, Table 2 estimates 0.36 of questions are possibly ambiguous, yet the GPT-based transducer labels only 0.04 as ambiguous at turn 1 (Table 3); for MedDialog, 0.92 possibly incomplete is roughly matched (0.99), but for SQuAD, 0.00 possibly incomplete is labeled 0.73 incomplete at turn 1. Since the manuscript's Limitations section itself calls the Classify function 'quite unpredictable', the reader cannot tell which component (Classify, Resolve, or the responder's reaction to the added text) drives the Table 1 gains. The authors should report accuracy stratified by the Classify label and by whether Resolve actually rewrote the question, for each dataset and turn.
minor comments (4)
  1. [Procedure 2, line 4] The message is printed as '!α(q′)' but the resolver's output is defined as a rewritten question '?α(q′)'. Please correct the typo and, in the revision, explicitly state whether the responder sees only the rewritten question or also the resolver's answer message; this is directly related to the leakage concern.
  2. [Section 4.4, Table 1] The dataset name 'SQuaAD' appears with an extra 'a' in the text; also, the table caption 'Role of increasing context on the proportions of correct answers' is unclear because the table actually compares accuracy with and without the transducer.
  3. [Section 4.3] The notation in the set definitions Cd,k and CT,d,k is confusing: the predicate uses 'a' both for the ground-truth answer and inside AGREES(a, aλ). Please clarify the notation, for example by denoting the ground-truth answer as a* and the LLM output as a_λ.
  4. [References] The reference list contains several duplicated entries (e.g., [11] and [12] are the same paper in preprint and EMNLP versions, and [45] and [46] are similarly duplicate AmbigQA entries). Please consolidate these to avoid citation clutter.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation found: the transducer claim is an empirical comparison with zero-shot agents, and the oracle-based definitions are motivational rather than load-bearing.

full rationale

The paper's central claim (Section 4.1) is an empirical comparison: does adding a question-transducer improve interactive QA accuracy? The comparison is made on six standard datasets, with the transducer implemented as zero-shot ReAct agents (Procedures 1-3) and with no parameter fitted to the test labels. The oracle-based definitions of incomplete and ambiguous questions (Definitions 4 and 5) are used only to motivate the transducer design; they are not part of the evaluation. Appendix A explicitly states that its retrospective rules for dataset characterization 'cannot be used prospectively during an interaction to classify questions,' so the dataset categories cannot be smuggled into the transducer. The reported results are not forced by construction: the transducer actually hurts on SQuAD at turns 1 and 2 and on ShARC at turn 2, which shows the claim is falsifiable and not equivalent to its inputs. The only self-citations ([49] and [61]) appear in Related Work and concern multi-turn mediated systems and an intelligibility protocol; neither is used as a load-bearing premise for the transducer conclusion. A reviewer concern about answer leakage through the inserted explanation e in Procedure 2 is a potential threat to internal validity, not a circularity, because it concerns information flow in the experimental protocol rather than a definitional identity, a fitted parameter renamed as a prediction, or a self-citation chain. Overall, the derivation is self-contained, with one or two peripheral self-citations that do not carry the central claim.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The paper's formal contribution is small: the message and transducer definitions are notation rather than axioms with mathematical content. The load-bearing assumptions are empirical: the dataset ground truth is valid, the evaluation protocol measures the intended effect, and the inserted explanation does not leak answers. Only one hand-chosen numerical parameter (K=3) affects the reported results.

free parameters (1)
  • Turn budget K = 3
    The evaluation only runs interactions to K=3 turns; the reported accuracy and the 'shortening' claim depend on this hand-chosen cutoff (Section 4.3).
assumptions (3)
  • ad hoc to paper Questions are either incomplete or ambiguous, but not both at once.
    Stated in Section 2 after Definitions 4 and 5; the three-label Classify function cannot represent a question with both deficiencies.
  • domain assumption The oracle-based definitions of incompleteness and ambiguity are a faithful operationalization of the phenomena in the benchmark datasets.
    Section 2 defines incompleteness and ambiguity via an oracle, and Section 4.2 assumes these categories capture the deficiencies in SQuAD, AmbigNQ, MedDialog, MultiWOZ, and ShARC.
  • domain assumption Human-labeled answers in the six datasets serve as correct oracle answers, and the manual AGREES check is a valid accuracy measure.
    Section 4.3 uses human-labeled ground truth and manual verification of up to 1800 responses, with no inter-annotator agreement reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Agent-Based Detection and Resolution of Incompleteness and Ambiguity in Interactions with Large Language Models." pith.science (2026). https://pith.science/paper/VGVXTPC6

@misc{pith2026250703726,
  author       = {Pith},
  title        = {Pith review of: Agent-Based Detection and Resolution of Incompleteness and Ambiguity in Interactions with Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VGVXTPC6}},
  note         = {Machine review of arXiv:2507.03726}
}
read the original abstract

Many of us now treat LLMs as modern-day oracles asking it almost any kind of question. However, consulting an LLM does not have to be a single turn activity. But long multi-turn interactions can get tedious if it is simply to clarify contextual information that can be arrived at through reasoning. In this paper, we examine the use of agent-based architecture to bolster LLM-based Question-Answering systems with additional reasoning capabilities. We examine the automatic resolution of potential incompleteness or ambiguities in questions by transducers implemented using LLM-based agents. We focus on several benchmark datasets that are known to contain questions with these deficiencies to varying degrees. We equip different LLMs (GPT-3.5-Turbo and Llama-4-Scout) with agents that act as specialists in detecting and resolving deficiencies of incompleteness and ambiguity. The agents are implemented as zero-shot ReAct agents. Rather than producing an answer in a single step, the model now decides between 3 actions a) classify b) resolve c) answer. Action a) decides if the question is incomplete, ambiguous, or normal. Action b) determines if any deficiencies identified can be resolved. Action c) answers the resolved form of the question. We compare the use of LLMs with and without the use of agents with these components. Our results show benefits of agents with transducer 1) A shortening of the length of interactions with human 2) An improvement in the answer quality and 3) Explainable resolution of deficiencies in the question. On the negative side we find while it may result in additional LLM invocations and in some cases, increased latency. But on tested datasets, the benefits outweigh the costs except when questions already have sufficient context. Suggesting the agent-based approach could be a useful mechanism to harness the power of LLMs to develop more robust QA systems.

Figures

Figures reproduced from arXiv: 2507.03726 by the authors.

Figure 1
Figure 1. Human-LLM question-answering: (a) Without a question-transducer; and (b) With a question-transducer. For [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

73 extracted references · 60 canonical work pages

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and others. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    Understanding and answering incomplete questions

    Angus Addlesee and Marco Damonte. Understanding and answering incomplete questions. In Proceedings of the 5th International Conference on Conversational User Interfaces, 1–9. 2023

    work page 2023

  3. [3]

    Detecting ambiguity in conversational systems

    Avik Banerjee. Detecting ambiguity in conversational systems. Master’s thesis, Unknown Institution, 2021

    work page 2021

  4. [4]

    Artificial intelligence—a modern approach by stuart russell and peter norvig, prentice hall

    Gerd Brewka. Artificial intelligence—a modern approach by stuart russell and peter norvig, prentice hall. series in artificial intelligence, englewood cliffs, nj. The Knowledge Engineering Review, 11(1):78–79, 1996

    work page 1996

  5. [5]

    Multiwoz-a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling

    Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gasic. Multiwoz-a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 5016–5026. 2018

    work page 2018

  6. [6]

    MultiWOZ - a large-scale multi-domain Wizard-of-Oz dataset for task-oriented dialogue modelling

    Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gaši´c. MultiWOZ - a large-scale multi-domain Wizard-of-Oz dataset for task-oriented dialogue modelling. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii, editors, EMNLP:2018:1, 5016–5026. Brussels, Belgium, October-November ...

    doi:10.18653/v1/d18-1547 2018

  7. [7]

    Preferences for single-turn vs

    Peter Burggräf, Moritz Beyer, Jan-Philip Ganser, Tobias Adlon, Katharina Müller, Constantin Riess, Kaspar Zollner, Till Saßmannshausen, and Vincent Kammerer. Preferences for single-turn vs. multiturn voice dialogs in automotive use cases—results of an interactive user survey in germany. IEEE Access, 10:55020–55033, 2022

    work page 2022

  8. [8]

    An empirical study of in-context learning in LLMs for machine transla- tion

    Pranjal Chitale, Jay Gala, and Raj Dabre. An empirical study of in-context learning in LLMs for machine transla- tion. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,FINDINGS:2024:acl, 7384–7406. Bangkok, Thailand, August 2024. acl. URL: anth#{2024.findings-acl.440/}, doi:10.18653/v1/2024.findings-acl.440

    work page doi:10.18653/v1/2024.findings-acl.440 2024

Show all 73 references
  1. [9]

    An empirical study of in-context learning in llms for machine translation

    Pranjal Chitale, Jay Gala, and Raj Dabre. An empirical study of in-context learning in llms for machine translation. In Findings of the Association for Computational Linguistics ACL 2024, 7384–7406. 2024

    2024

  2. [10]

    Llm contextbridge: a hybrid approach for intent and dialogue understanding in ivsr

    Changwoo Chun, Daniel Rim, and Juhee Park. Llm contextbridge: a hybrid approach for intent and dialogue understanding in ivsr. In Proceedings of the 31st International Conference on Computational Linguistics: Industry Track, 794–806. 2025

    2025

  3. [11]

    Selectively answering ambiguous questions

    Jeremy Cole, Michael Zhang, Dan Gillick, Julian Eisenschlos, Bhuwan Dhingra, and Jacob Eisenstein. Selectively answering ambiguous questions. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 530–543. 2023

    2023

  4. [12]

    Selectively answering ambiguous questions

    Jeremy Cole, Michael Zhang, Daniel Gillick, Julian Eisenschlos, Bhuwan Dhingra, and Jacob Eisenstein. Selectively answering ambiguous questions. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, EMNLP:2023:main, 530–543. Singapore, December 2023. acl. URL: anth#{2023.emnl...

    2023 doi

  5. [13]

    A short survey on end-to-end simple question answering systems

    José Wellington Franco da Silva, Amanda Drielly Pires Venceslau, Juliano Efson Sales, José Gilvan Rodrigues Maia, Vládia Célia Monteiro Pinheiro, and Vânia Maria Ponte Vidal. A short survey on end-to-end simple question answering systems. Artificial Intelligence Review, 53(7):...

    2020

  6. [14]

    Conversate: supporting reflective learning in interview practice through interactive simulation and dialogic feedback

    Taufiq Daryanto, Xiaohan Ding, Lance T Wilhelm, Sophia Stil, Kirk McInnis Knutsen, and Eugenia H Rho. Conversate: supporting reflective learning in interview practice through interactive simulation and dialogic feedback. Proceedings of the ACM on Human-Computer Interaction, 9(...

    2025

  7. [15]

    A survey on in-context learning

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, and others. A survey on in-context learning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 1107–1128. 2024

    2024

  8. [16]

    A survey on in-context learning

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, Xu Sun, Lei Li, and Zhifang Sui. A survey on in-context learning. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors,EMNLP:2024:main, 1107–1128. Miami,...

    2024 doi

  9. [17]

    AI hospital: bench- marking large language models in a multi-agent medical interaction simulator

    Zhihao Fan, Lai Wei, Jialong Tang, Wei Chen, Wang Siyuan, Zhongyu Wei, and Fei Huang. AI hospital: bench- marking large language models in a multi-agent medical interaction simulator. In Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al-Khalifa, Barbara Di Eugenio, and Ste...

    2025

  10. [18]

    Learning towards conversational ai: a survey

    Tingchen Fu, Shen Gao, Xueliang Zhao, Ji-rong Wen, and Rui Yan. Learning towards conversational ai: a survey. AI Open, 3:14–28, 2022

    2022

  11. [19]

    Multi-turn dialogue agent as sales’ assistant in telemarketing

    Wanting Gao, Xinyi Gao, and Yin Tang. Multi-turn dialogue agent as sales’ assistant in telemarketing. In 2023 International Joint Conference on Neural Networks (IJCNN), 1–9. IEEE, 2023

    2023

  12. [20]

    Answering ambiguous questions through generative evidence fusion and round-trip prediction

    Yifan Gao, Henghui Zhu, Patrick Ng, Cicero dos Santos, Zhiguo Wang, Feng Nan, Dejiao Zhang, Ramesh Nallapati, Andrew O Arnold, and Bing Xiang. Answering ambiguous questions through generative evidence fusion and round-trip prediction. In Proceedings of the 59th Annual Meeting ...

    2021

  13. [21]

    Arnold, and Bing Xiang

    Yifan Gao, Henghui Zhu, Patrick Ng, Cicero Nogueira dos Santos, Zhiguo Wang, Feng Nan, Dejiao Zhang, Ramesh Nallapati, Andrew O. Arnold, and Bing Xiang. Answering ambiguous questions through genera- tive evidence fusion and round-trip prediction. In Chengqing Zong, Fei Xia, We...

    2021 doi

  14. [22]

    Dr3: ask large language models not to give off-topic answers in open domain multi-hop question answering

    Yuan Gao, Yiheng Zhu, Yuanbin Cao, Yinzhi Zhou, Zhen Wu, Yujie Chen, Shenglan Wu, Haoyuan Hu, and Xinyu Dai. Dr3: ask large language models not to give off-topic answers in open domain multi-hop question answering. In Proceedings of the 2024 Joint International Conference on C...

    2024

  15. [23]

    Estimate and replace: a novel approach to integrating deep neural networks with existing applications

    Guy Hadash, Einat Kermany, Boaz Carmeli, Ofer Lavi, George Kour, and Alon Jacovi. Estimate and replace: a novel approach to integrating deep neural networks with existing applications. arXiv preprint arXiv:1804.09028, 2018

    2018 arXiv

  16. [24]

    Retrieving supporting evidence for generative question answering

    Siqing Huo, Negar Arabzadeh, and Charles Clarke. Retrieving supporting evidence for generative question answering. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region, 11–20. 2023

    2023

  17. [25]

    Paraqa: a question answering dataset with paraphrase responses for single-turn conversation

    Endri Kacupaj, Barshana Banerjee, Kuldeep Singh, and Jens Lehmann. Paraqa: a question answering dataset with paraphrase responses for single-turn conversation. In European semantic web conference, 598–613. Springer, 2021

    2021

  18. [26]

    Scope ambiguities in large language models

    Gaurav Kamath, Sebastian Schuster, Sowmya Vajjala, and Siva Reddy. Scope ambiguities in large language models. Transactions of the Association for Computational Linguistics, 12:738–754, 2024

    2024

  19. [27]

    Scope ambiguities in large language models

    Gaurav Kamath, Sebastian Schuster, Sowmya Vajjala, and Siva Reddy. Scope ambiguities in large language models. Transactions of the Association for Computational Linguistics, 12:738–754, 2024. URL: anth#{2024. tacl-1.41/}, doi:10.1162/tacl_a_00670

    2024 doi

  20. [28]

    Do llms understand ambiguity in text? a case study in open-world question answering

    Aryan Keluskar, Amrita Bhattacharjee, and Huan Liu. Do llms understand ambiguity in text? a case study in open-world question answering. In 2024 IEEE International Conference on Big Data (BigData) , 7485–7490. IEEE, 2024. 10 NAIK , S RINIVASAN , AGARWAL AND HE

    2024

  21. [29]

    Bert: pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. Bert: pre-training of deep bidirectional transformers for language understanding. In Proceedings of naacL-HLT, volume 1, 2. Minneapolis, Minnesota, 2019

    2019

  22. [30]

    Aligning language models to explicitly handle ambiguity

    Hyuhng Joon Kim, Youna Kim, Cheonbok Park, Junyeob Kim, Choonghyun Park, Kang Min Yoo, Sang-goo Lee, and Taeuk Kim. Aligning language models to explicitly handle ambiguity. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 1989–2007. 2024

    2024

  23. [31]

    Aligning language models to explicitly handle ambiguity

    Hyuhng Joon Kim, Youna Kim, Cheonbok Park, Junyeob Kim, Choonghyun Park, Kang Min Yoo, Sang-goo Lee, and Taeuk Kim. Aligning language models to explicitly handle ambiguity. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, EMNLP:2024:main, 1989–2007. Miami, Florid...

    2024 doi

  24. [32]

    Fast classification of handwritten on-line arabic characters

    George Kour and Raid Saabne. Fast classification of handwritten on-line arabic characters. In Soft Computing and Pattern Recognition (SoCPaR), 2014 6th International Conference of, 312–318. IEEE, 2014

    2014

  25. [33]

    Real-time segmentation of on-line handwritten arabic script

    George Kour and Raid Saabne. Real-time segmentation of on-line handwritten arabic script. In Frontiers in Handwriting Recognition (ICFHR), 2014 14th International Conference on, 417–422. IEEE, 2014

    2014

  26. [34]

    Fundamental limitations of generative llms

    Andrei Kucharavy. Fundamental limitations of generative llms. In Large Language Models in Cybersecurity: Threats, Exposure and Mitigation, pages 55–64. Springer, 2024

    2024

  27. [35]

    Conver- sational ai: an overview of methodologies, applications & future scope

    Pradnya Kulkarni, Ameya Mahabaleshwarkar, Mrunalini Kulkarni, Nachiket Sirsikar, and Kunal Gadgil. Conver- sational ai: an overview of methodologies, applications & future scope. In 2019 5th International conference on computing, communication, control and automation (ICCUBEA)...

    2019

  28. [36]

    Incomplete follow-up question resolution using retrieval based sequence to sequence learning

    Vineet Kumar and Sachindra Joshi. Incomplete follow-up question resolution using retrieval based sequence to sequence learning. In Proceedings of the 40th international acm sigir conference on research and development in information retrieval, 705–714. 2017

    2017

  29. [37]

    Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav...

    2019 doi

  30. [38]

    S2m: converting single-turn to multi-turn datasets for conversational question answering

    Baokui Li, Sen Zhang, Wangshu Zhang, Yicheng Chen, Changlin Yang, Sen Hu, Teng Xu, Siye Liu, and Jiwei Li. S2m: converting single-turn to multi-turn datasets for conversational question answering. In ECAI 2023, pages 1365–1372. IOS Press, 2023

    2023

  31. [39]

    Deep context modeling for multi-turn response selection in dialogue systems

    Lu Li, Chenliang Li, and Donghong Ji. Deep context modeling for multi-turn response selection in dialogue systems. Information Processing & Management, 58(1):102415, 2021

    2021

  32. [40]

    A review of prominent paradigms for llm-based agents: tool use, planning (including rag), and feedback learning

    Xinzhe Li. A review of prominent paradigms for llm-based agents: tool use, planning (including rag), and feedback learning. In Proceedings of the 31st International Conference on Computational Linguistics, 9760–9779. 2025

    2025

  33. [41]

    A review of prominent paradigms for LLM-based agents: tool use, planning (including RAG), and feedback learning

    Xinzhe Li. A review of prominent paradigms for LLM-based agents: tool use, planning (including RAG), and feedback learning. In Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al-Khalifa, Barbara Di Eugenio, and Steven Schockaert, editors, COLING:2025:main, 9760–9779. Abu Dh...

    2025

  34. [42]

    Dailydialog: a manually labelled multi-turn dialogue dataset

    Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. Dailydialog: a manually labelled multi-turn dialogue dataset. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 986–995. 2017

    2017

  35. [43]

    Qa-rag: exploring llm reliance on external knowledge

    Aigerim Mansurova, Aiganym Mansurova, and Aliya Nugumanova. Qa-rag: exploring llm reliance on external knowledge. Big Data and Cognitive Computing, 8(9):115, 2024

    2024

  36. [44]

    Detecting ambiguities to guide query rewrite for robust conversations in enterprise ai assistants

    Md Mehrab Tanjim, Xiang Chen, Victor S Bursztyn, Uttaran Bhattacharya, Tung Mai, Vaishnavi Muppala, Akash Maharaj, Saayan Mitra, Eunyee Koh, Yunyao Li, and others. Detecting ambiguities to guide query rewrite for robust conversations in enterprise ai assistants. arXiv e-prints...

    2025

  37. [45]

    Ambigqa: answering ambiguous open-domain questions

    Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. Ambigqa: answering ambiguous open-domain questions. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 5783–5797. 2020

    2020

  38. [46]

    AmbigQA: answering am- biguous open-domain questions

    Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. AmbigQA: answering am- biguous open-domain questions. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, EMNLP:2020:main, 5783–5797. Online, November 2020. acl. URL: anth#{2020.emnlp-main.466/},...

    2020 doi

  39. [47]

    Insurance question answering via single-turn dialogue modeling

    Seon-Ok Na, Young-Min Kim, and Seung-Hwan Cho. Insurance question answering via single-turn dialogue modeling. In Proceedings of the Second Workshop on When Creative AI Meets Conversational AI, 35–41. 2022

    2022

  40. [48]

    Insurance question answering via single-turn dialogue modeling

    Seon-Ok Na, Young-Min Kim, and Seung-Hwan Cho. Insurance question answering via single-turn dialogue modeling. In Xianchao Wu, Peiying Ruan, Sheng Li, and Yi Dong, editors, CAI:2022:1, 35–41. Gyeongju, Republic of Korea, October 2022. acl. URL: anth#{2022.cai-1.5/}

    2022

  41. [49]

    Multi-turn mediated solutions for conversational artificial intelligent systems leveraging graph-based techniques

    Riya Naik. Multi-turn mediated solutions for conversational artificial intelligent systems leveraging graph-based techniques. In Companion Proceedings of the ACM Web Conference 2023, 586–590. 2023

    2023

  42. [50]

    Evaluating ambiguous questions in semantic parsing

    Simone Papicchio, Paolo Papotti, and Luca Cagliero. Evaluating ambiguous questions in semantic parsing. In 2024 IEEE 40th International Conference on Data Engineering Workshops (ICDEW), 338–342. IEEE, 2024

    2024

  43. [51]

    Is temperature the creativity parameter of large language models? CoRR, 2024

    Max Peeperkorn, Tom Kouwenhoven, Daniel Brown, and Anna Jordanous. Is temperature the creativity parameter of large language models? CoRR, 2024

    2024

  44. [52]

    Reasoning with language model prompting: a survey

    Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang, and Huajun Chen. Reasoning with language model prompting: a survey. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pap...

    2023

  45. [53]

    Improving language understanding by generative pre-training

    Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, and others. Improving language understanding by generative pre-training. mikecaptain.com, 2018

    2018

  46. [54]

    Know what you don‘t know: unanswerable questions for SQuAD

    Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don‘t know: unanswerable questions for SQuAD. In Iryna Gurevych and Yusuke Miyao, editors, ACL:2018:2, 784–789. Melbourne, Australia, July 2018. acl. URL: anth#{P18-2124/}, doi:10.18653/v1/P18-2124

    2018 doi

  47. [55]

    Know what you don’t know: unanswerable questions for squad

    Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don’t know: unanswerable questions for squad. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 784–789. 2018

    2018

  48. [56]

    Broad-Domain Quantifier Scoping With RoBERTa

    Nathan Ellis Rasmussen. Broad-Domain Quantifier Scoping With RoBERTa. The Ohio State University, 2022

    2022

  49. [57]

    Interpretation of natural language rules in conversational machine reading

    Marzieh Saeidi, Max Bartolo, Patrick Lewis, Sameer Singh, Tim Rocktäschel, Mike Sheldon, Guillaume Bouchard, and Sebastian Riedel. Interpretation of natural language rules in conversational machine reading. In Proceedings of the 2018 Conference on Empirical Methods in Natural ...

    2018

  50. [58]

    Interpretation of natural language rules in conversational machine reading

    Marzieh Saeidi, Max Bartolo, Patrick Lewis, Sameer Singh, Tim Rocktäschel, Mike Sheldon, Guillaume Bouchard, and Sebastian Riedel. Interpretation of natural language rules in conversational machine reading. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii,...

    2018 doi

  51. [59]

    Multi-turn dialog system on single-turn data in medical domain

    Nazib Sorathiya, Chuan-An Lin, Daniel Chen Daniel Xiong, Scott Zin, Yi Zhang, He Sarina Yang, and Sharon Xiaolei Huang. Multi-turn dialog system on single-turn data in medical domain. arXiv e-prints, pages arXiv–2105, 2021

    2021

  52. [60]

    Simulating cooperative prosocial behavior with multi-agent llms: evidence and mechanisms for ai agents to inform policy decisions

    Karthik Sreedhar, Alice Cai, Jenny Ma, Jeffrey V Nickerson, and Lydia B Chilton. Simulating cooperative prosocial behavior with multi-agent llms: evidence and mechanisms for ai agents to inform policy decisions. In Proceedings of the 30th International Conference on Intelligen...

    2025

  53. [61]

    Implementation and application of an intelligibility protocol for interaction with an llm

    Ashwin Srinivasan, Karan Bania, Harshvardhan Mestha, Sidong Liu, and others. Implementation and application of an intelligibility protocol for interaction with an llm. arXiv e-prints, pages arXiv–2410, 2024

    2024

  54. [62]

    Exploring llm prompting strategies for joint essay scoring and feedback generation

    Maja Stahl, Leon Biermann, Andreas Nehring, and Henning Wachsmuth. Exploring llm prompting strategies for joint essay scoring and feedback generation. In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024), 283–298. 2024

    2024

  55. [63]

    Exploring LLM prompting strategies for joint essay scoring and feedback generation

    Maja Stahl, Leon Biermann, Andreas Nehring, and Henning Wachsmuth. Exploring LLM prompting strategies for joint essay scoring and feedback generation. In Ekaterina Kochmar, Marie Bexte, Jill Burstein, Andrea Horbach, Ronja Laarmann-Quante, Anaïs Tack, Victoria Yaneva, and Zhen...

    2024

  56. [64]

    LLM- friendly knowledge representation for customer support

    Hanchen Su, Wei Luo, Yashar Mehdad, Wei Han, Elaine Liu, Wayne Zhang, Mia Zhao, and Joy Zhang. LLM- friendly knowledge representation for customer support. In Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al-Khalifa, Barbara Di Eugenio, Steven Schockaert, Kareem Darwish, ...

    2025

  57. [65]

    Can chatgpt replace traditional kbqa models? an in-depth analysis of the question answering performance of the gpt llm family

    Yiming Tan, Dehai Min, Yu Li, Wenbo Li, Nan Hu, Yongrui Chen, and Guilin Qi. Can chatgpt replace traditional kbqa models? an in-depth analysis of the question answering performance of the gpt llm family. In International Semantic Web Conference, 348–367. Springer, 2023. 12 NAI...

    2023

  58. [66]

    Creating large language model applications utilizing langchain: a primer on developing llm apps fast

    Oguzhan Topsakal and Tahir Cetin Akinci. Creating large language model applications utilizing langchain: a primer on developing llm apps fast. In International Conference on Applied Engineering and Natural Sciences, volume 1, 1050–1056. 2023

    2023

  59. [67]

    Faq-based question answering via word alignment

    Zhiguo Wang and Abraham Ittycheriah. Faq-based question answering via word alignment. arXiv e-prints, pages arXiv–1507, 2015

    2015

  60. [68]

    Rethinking the evaluation of in-context learning for llms

    Guoxin Yu, Lemao Liu, Mo Yu, Yue Yu, and Xiang Ao. Rethinking the evaluation of in-context learning for llms. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 14068–14082. 2024

    2024

  61. [69]

    Conversational question answering: a survey

    Munazza Zaib, Wei Emma Zhang, Quan Z Sheng, Adnan Mahmood, and Yang Zhang. Conversational question answering: a survey. Knowledge and Information Systems, 64(12):3151–3195, 2022

    2022

  62. [70]

    MedDia- log: large-scale medical dialogue datasets

    Guangtao Zeng, Wenmian Yang, Zeqian Ju, Yue Yang, Sicheng Wang, Ruisi Zhang, Meng Zhou, Jiaqi Zeng, Xiangyu Dong, Ruoyu Zhang, Hongchao Fang, Penghui Zhu, Shu Chen, and Pengtao Xie. MedDia- log: large-scale medical dialogue datasets. In Bonnie Webber, Trevor Cohn, Yulan He, an...

    2020 doi

  63. [71]

    Meddialog: large-scale medical dialogue datasets

    Guangtao Zeng, Wenmian Yang, Zeqian Ju, Yue Yang, Sicheng Wang, Ruisi Zhang, Meng Zhou, Jiaqi Zeng, Xiangyu Dong, Ruoyu Zhang, and others. Meddialog: large-scale medical dialogue datasets. In Proceedings of the 2020 conference on empirical methods in natural language processin...

    2020

  64. [72]

    Multi-turn video question answering via multi-stream hierarchical attention context network

    Zhou Zhao, Xinghua Jiang, Deng Cai, Jun Xiao, Xiaofei He, and Shiliang Pu. Multi-turn video question answering via multi-stream hierarchical attention context network. In IJCAI, volume 2018, 27th. 2018

    2018

  65. [73]

    Normal”, “Incomplete

    Xiangyang Zhou, Lu Li, Daxiang Dong, Yi Liu, Ying Chen, Wayne Xin Zhao, Dianhai Yu, and Hua Wu. Multi-turn response selection for chatbots with deep attention matching network. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1...

    2018

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.