Pith. sign in

REVIEW 3 major objections 5 minor 59 references

ProEvent benchmark shows LLM agents keep a user's timetable right only 26.7% of the time, versus 90.5% for humans.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 17:13 UTC pith:AROKU2YT

load-bearing objection Novel benchmark and careful evaluation, but the DELETE instruction contradicts the ground truth on implicit cancellations, so the headline performance gap is likely overstated. the 3 major comments →

arxiv 2607.17701 v1 pith:AROKU2YT submitted 2026-07-20 cs.AI

ProEvent: An Event-centric Benchmark for Proactive Agents

Guanzhen Li , Liangming Pan , Leye Wang This is my paper
classification cs.AI
keywords proactive agentsevent trackingtimetable maintenancebenchmarkLLM evaluationinstant messagingdialogue understandingagent evaluation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper introduces ProEvent, a benchmark that tests whether proactive AI agents can maintain a user's calendar by reading ongoing instant-messaging chats. The authors formalize the task as updating a timetable through Insert, Update, and Delete operations, and evaluate both when actions are triggered and whether they are correct. Testing eight LLMs and pipelines, they find a systematic overreaction bias: agents act when they should stay quiet, and they frequently delete events without sufficient evidence. Even the strongest tested model only produces a fully correct timetable in 26.7% of multi-step scenarios, while humans succeed 90.5% of the time. The paper argues that current agents lack key abilities: detecting implicitly stated events, reasoning from the user's first-person perspective, and handling cancellations.

Core claim

The central claim is that proactive event tracking from chat is a measurable, unsolved capability. ProEvent provides 1,249 synthesized chats with ground-truth scheduling trajectories, requiring agents to output explicit timetable operations at each time window. Across eight models and pipelines, no system approaches human performance; the best multi-step success is 27.2% correct timetables. The authors identify two systematic deficiencies: models overreact (FDR far exceeds MNR) and struggle with cancellations and implicit instructions. Ablations show that making implicit instructions explicit dramatically improves recall, suggesting the bottleneck is perception of action requirements in natu

What carries the argument

The central object is the timetable-maintenance formalism: at each discrete time step the agent maps the previous timetable and newly received chats to a set of operations (Insert, Update, Delete). Ground truth comes from scheduling trajectories that drive chat synthesis, allowing objective correctness via Event Success Rate and Timetable Success Rate, alongside timing metrics (False Detection Rate, Missed Need Rate). The benchmark's construction pipeline—contact profiles, LLM-generated trajectories, chat skeletons, noise injection, and concurrent-thread merging—is what makes the task concrete and evaluable.

Load-bearing premise

The load-bearing premise is that the LLM-synthesized chat dialogues are realistic enough that model failures on ProEvent reflect genuine proactive-assistant limitations rather than artifacts of the generator's style.

What would settle it

If a study applied ProEvent to real-world chat logs (or to human-written rewrites of the same scheduling trajectories) and found that the same models scored near-human success rates, the benchmark's synthetic style would be the main driver of failure. Conversely, a real-chat test reproducing the overreaction and cancellation errors would validate the benchmark.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If ProEvent is valid, no current LLM agent can be trusted to maintain a user's timetable autonomously; human oversight is still required.
  • The systematic overreaction bias (high FDR) suggests proactive agents should incorporate explicit 'need assessment' steps; one necessity-reasoning pipeline cut a model's FDR by 70%.
  • Performance degrades with longer negotiation threads and more concurrent chats, so scaling to real-world multi-party coordination will be hard for current architectures.
  • Reasoning improvements help distinguish genuine cancellations from ambiguity: the best delete-operation precision reached 99.5%.
  • The benchmark's success-rate metrics resolve single-step ambiguities (e.g., Update vs. Delete+Insert) and provide a stable target for future agents.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The operation-based formalism could transfer to other asynchronous contexts like email or calendar feeds, making this a general schedule-sync task beyond chat.
  • The overreaction finding suggests a calibration problem: agents might be improved by tuning a confidence threshold for triggering actions, not just by prompting.
  • Because the chats are synthetically generated and the paper notes they are 'structured and repetitive,' the absolute success rates may not transfer directly to messy human chats; the benchmark is best read as a lower-bound difficulty probe.
  • The first-person perspective failure hints that fine-tuning on egocentric narratives or adding explicit 'act as the user' grounding could be a promising intervention.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces ProEvent, a synthetic event-centric benchmark for evaluating whether LLM-based agents can proactively maintain a user's timetable by monitoring instant-messaging chats. Chat scenarios are generated from hand-designed scheduling trajectories, annotated with Insert/Update/Delete timetable operations, and scored with timing, single-step, and multi-step metrics (FDR/MNR, precision/recall, ESR/TSR). Experiments on eight LLMs and pipelines report uniformly low performance, e.g., GPT-5.1 reaches only 26.7% Timetable Success Rate versus 90.5% for humans, and qualitative analysis identifies categories such as implicit-event misses and first-person perspective misalignment. The paper concludes that current LLMs are far from dependable proactive event trackers.

Significance. The task formulation is timely and the evaluation design is more objective than prior semantic-similarity or LLM-judge-only protocols: operation-level ground truth permits exact scoring of most fields, and the authors include a human performance baseline. The paper also contributes a useful diagnosis of over-triggering (FDR) and a demonstration that a deliberative 'ProCoT' prompting strategy reduces it. If the benchmark is valid and the numbers are trustworthy, the main claim—that strong LLMs still cannot reliably maintain user timetables from chat—is significant for the proactive-agent community. However, validity is currently undercut by an instruction/ground-truth mismatch in the cancellation definition and by the absence of a released dataset, so the strength of the empirical conclusion is not yet established.

major comments (3)
  1. [Appendix I, §5.1, Fig. 6(a)] The evaluation prompt states: 'DELETE an event when it is explicitly cancelled.' Section 5.1, however, says cancellations are often expressed implicitly, and Figure 6(a) shows ground truth Delete for a hedged 'Maybe I can't join you this time!' while GPT-5.1 outputs Update. The model is following the written instruction; labeling this as 'perspective misalignment' or a 'fundamental limitation' is not justified. This mismatch directly deflates Delete recall, single-step metrics, and TSR, and confounds the headline 26.7% result. Please align the prompt with the ground-truth policy (define cancellation to include implicit expressions and give examples) or restrict ground truth to explicit cancellations, then rerun all models and reclassify the qualitative error cases.
  2. [§4.3, Appendix C, Limitations] The realism/validity evidence is weak and partly self-undermining. The pairwise realism check relies on GPT-5.4 as a judge, with no human realism validation. The rewrite control uses only 50 cases and reports 6/50 output changes, but repeated runs on unchanged synthetic data already yield 8/50 inconsistencies; 6/50 is therefore within run-to-run noise and cannot support the claim that synthetic style has negligible impact. The Limitations section concedes the dialogues 'tend to be structured and repetitive' and all negotiation for an event occurs in a single chat. A larger human-based realism or consistency study, or inclusion of human chat data, is needed before model failures can be attributed to genuine proactive-assistant limitations rather than generator artifacts.
  3. [General (data availability)] No URL, repository, or release plan is provided for the ProEvent dataset, prompts, or evaluation harness. Since the paper's contribution is a benchmark, the central artifact must be accessible for reproduction and adoption. Please provide a public link and, ideally, a data card with usage terms.
minor comments (5)
  1. [Fig. 11] Typo: 'Inert(time=...)' should be 'INSERT(time=...)'.
  2. [Table 2, Fig. 6] The model is referred to as both 'GPT-5.1' and 'GPT-o5.1'; please standardize.
  3. [Table 1 caption] The highlight criterion is stated as 'FDR and MNR >90%, all other metrics <10%', but several highlighted entries (e.g., 96.5% FDR) are not paired with >90% MNR. Clarify the exact highlighting rule.
  4. [§3.2, §5] Section 3.2 says ESR/TSR resolve ambiguities such as Update versus Delete+Insert, but §5 says no such ambiguity was observed. Reconcile the two statements.
  5. [§2, References] The related work cites 'ProAgent (Yang et al., 2025b)', but the reference list entry is titled 'Contextagent'. Please align the citation and reference.

Circularity Check

0 steps flagged

No significant circularity: benchmark construction and evaluation do not reduce to a fitted input or self-citation.

full rationale

PROEVENT's pipeline generates scheduling trajectories, uses them to guide GPT-OSS-120B chat synthesis, and then derives ground-truth operations by comparing timetable states across time windows. This is a data-generation protocol, not a derivation in which a claimed prediction is equivalent to a fitted parameter or to the definition of the predictor. The evaluation compares model outputs against these independently validated ground-truth labels; the authors do not fit any parameter to model outputs and then report it as a prediction. The human validation (94.4% event success, 90.5% timetable success) and the rewritten-dialogue ablation provide external reference points. The only notable issue is an internal evaluation-prompt inconsistency: Appendix I instructs models to 'DELETE an event when it is explicitly cancelled,' while Section 5.1 states cancellations are 'often expressed implicitly,' and ground truth labels include implicit cancellations (e.g., Figure 6(a)). This is a benchmark-validity and prompt-alignment concern, not a circularity pattern, because it does not make the model's score equivalent by construction to the benchmark's inputs. No load-bearing self-citations or imported uniqueness theorems appear. Thus no circular step meets the stated evidentiary bar.

Axiom & Free-Parameter Ledger

0 free parameters · 5 axioms · 0 invented entities

The benchmark rests on design choices rather than fitted parameters: the synthetic-corpus realism, the sufficiency of three operations, and the reliability of LLM/human judging. These are reasonable for a benchmark but are assumptions introduced by the authors.

axioms (5)
  • domain assumption The discrete-time formulation, where all chats in a fixed interval are processed jointly and operations update the timetable, is an adequate model of proactive agent contexts like GPT-Pulse and MineContext.
    Invoked in Section 3.1; if real deployments are asynchronous or require cross-chat reasoning, the benchmark's timing metrics may not match practice.
  • domain assumption Instant-messaging chats contain enough unambiguous information to recover the ground-truth timetable operations.
    The task assumes every event change is expressed in the chats; ambiguity or missing context would make ground truth underdetermined. Sections 3.1 and 4.1.
  • ad hoc to paper LLM-generated dialogues guided by scheduling trajectories are realistic enough that model failures on ProEvent reflect genuine proactive-assistant limitations.
    Section 4.1 Step two uses GPT-OSS-120B; the Limitations section concedes dialogues are structured and repetitive. This is the central validity assumption.
  • domain assumption The three operation types Insert, Update, and Delete are sufficient to represent all timetable changes in the evaluated scenarios.
    Defined in Section 3.1; if event changes can occur outside this set, the scoring is incomplete.
  • domain assumption LLM-as-judge (GPT-5.1) for location equivalence is reliable.
    Appendix G; validated on 100 samples with 100% human agreement, but only for one attribute and one judge.

pith-pipeline@v1.3.0-alltime-deepseek · 15037 in / 11476 out tokens · 121611 ms · 2026-08-01T17:13:10.808378+00:00 · methodology

0 comments
read the original abstract

Proactive agents are expected to anticipate user needs and provide autonomous assistance by perceiving environmental context without explicit instructions. A fundamental capability of such agents is to identify and track users' upcoming events, enabling continuous and event-specific assistance. For example, by recording the time and location of a planned hike, an agent can deliver weather reminders in advance or provide navigation support before departure. However, existing works on proactive agents largely overlook event-centric assistance, and the open-ended nature of proactive assistance poses challenges for reliable evaluation. To bridge these gaps, we introduce ProEvent, the first event-centric benchmark designed to assess an agent's ability to proactively maintain a user's timetable based on ongoing instant messaging chats. ProEvent provides synthesized yet realistic chats that consider the dynamic interaction among users, concurrent chat threads, and noise in the real world, and evaluates proactive agents on response timing, single-step response correctness, and multi-step response correctness. Experiments on eight LLMs and pipelines reveal that current agents frequently overact and struggle with event cancellation. Notably, even GPT-5.1 only reacts correctly in 26.7% of scenarios. Further qualitative analysis reveals fundamental limitations of current LLMs as proactive agents, particularly in detecting implicit events and reasoning from the user's first-person perspective.

Figures

Figures reproduced from arXiv: 2607.17701 by Guanzhen Li, Leye Wang, Liangming Pan.

Figure 1
Figure 1. Figure 1: Illustration of the timetable maintenance task. The proactive agent needs to maintain the timetable on the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: PROEVENT construction pipeline. Left: Single chat synthesis. A contact is selected from the contact pool, and a scheduling trajectory is generated to update an attribute of the event. This trajectory then guides the chat generation, producing a single chat. Right: To simulate real-world concurrent chat threads, multiple chats are combined, noise is injected, and the chats are segmented into fixed time wind… view at source ↗
Figure 3
Figure 3. Figure 3: Dataset Statistics. (a) illustrates the diversity of events and their corresponding chat topics. (b) presents the [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Error distributions of GPT-5.1 and Qwen-3. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 4
Figure 4. Figure 4: Effects of dynamic interactions ((a), left), [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: Error cases. LLMs make mistakes when reasoning from the user’s first-person perspective, updating one [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Pairwise realism results between ProEvent and realistic datasets. [PITH_FULL_IMAGE:figures/full_fig_p011_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Pairwise realism results between ProEvent and realistic datasets. [PITH_FULL_IMAGE:figures/full_fig_p012_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: A case for demonstrating the effect of explicit expression and the update prompt. [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: The case for temporal reasoning mistakes and tentative events. [PITH_FULL_IMAGE:figures/full_fig_p014_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: A specific case of our chat synthesis pipeline. [PITH_FULL_IMAGE:figures/full_fig_p015_11.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

59 extracted references · 22 linked inside Pith

  1. [2]

    Findings of the Association for Computational Linguistics: NAACL 2025 , pages=

    Knowagent: Knowledge-augmented planning for llm-based agents , author=. Findings of the Association for Computational Linguistics: NAACL 2025 , pages=

    2025

  2. [3]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Webpilot: A versatile and autonomous multi-agent system for web task execution with strategic exploration , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  3. [4]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Os-genesis: Automating gui agent trajectory construction via reverse task synthesis , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  4. [10]

    Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    From role-play to drama-interaction: An LLM solution , author=. Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    2024

  5. [14]

    Proceedings of the 36th annual acm symposium on user interface software and technology , pages=

    Generative agents: Interactive simulacra of human behavior , author=. Proceedings of the 36th annual acm symposium on user interface software and technology , pages=

  6. [15]

    Findings of the Association for Computational Linguistics: NAACL 2025 , pages=

    Diasynth: Synthetic dialogue generation framework for low resource dialogue applications , author=. Findings of the Association for Computational Linguistics: NAACL 2025 , pages=

    2025

  7. [16]

    arXiv preprint arXiv:2305.10142 , year=

    Improving language model negotiation with self-play and in-context learning from ai feedback , author=. arXiv preprint arXiv:2305.10142 , year=

    Pith/arXiv arXiv

  8. [17]

    Science China Technological Sciences , volume=

    Recent advances and challenges in task-oriented dialog systems , author=. Science China Technological Sciences , volume=. 2020 , publisher=

    2020

  9. [18]

    Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Deep dyna-q: Integrating planning for task-completion dialogue policy learning , author=. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  10. [19]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Switch-based active deep dyna-q: Efficient adaptive planning for task-completion dialogue policy learning , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  11. [21]

    IEEE Access , volume=

    Dialogue logic aware and key utterance decoupling model for multi-party dialogue reading comprehension , author=. IEEE Access , volume=. 2023 , publisher=

    2023

  12. [22]

    arXiv preprint arXiv:2306.13063 , year=

    Can llms express their uncertainty? an empirical evaluation of confidence elicitation in llms , author=. arXiv preprint arXiv:2306.13063 , year=

    Pith/arXiv arXiv

  13. [23]

    arXiv preprint arXiv:2310.10158 , year=

    Character-llm: A trainable agent for role-playing , author=. arXiv preprint arXiv:2310.10158 , year=

    Pith/arXiv arXiv

  14. [24]

    Nature , volume=

    Role play with large language models , author=. Nature , volume=. 2023 , publisher=

    2023

  15. [25]

    Proceedings of the Third Workshop on Natural Language Generation, Evaluation, and Metrics (GEM) , pages=

    Synthetic dialogue dataset generation using llm agents , author=. Proceedings of the Third Workshop on Natural Language Generation, Evaluation, and Metrics (GEM) , pages=

  16. [26]

    Toolflow: Boosting llm tool-calling through natural and coherent dialogue synthesis , author=. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages=

    2025

  17. [34]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Egothink: Evaluating first-person perspective thinking capability of vision-language models , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  18. [36]

    arXiv preprint arXiv:2410.04345 , year=

    MVP-Bench: Can Large Vision--Language Models Conduct Multi-level Visual Perception Like Humans? , author=. arXiv preprint arXiv:2410.04345 , year=

    Pith/arXiv arXiv

  19. [37]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track , year=

    Mme: A comprehensive evaluation benchmark for multimodal large language models , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track , year=

  20. [38]

    Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies , volume=

    Can large language models be good companions? An LLM-based eyewear system with conversational common ground , author=. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies , volume=. 2024 , publisher=

    2024

  21. [40]

    arXiv preprint arXiv:2403.12393 , year=

    Dr3: Ask large language models not to give off-topic answers in open domain multi-hop question answering , author=. arXiv preprint arXiv:2403.12393 , year=

    Pith/arXiv arXiv

  22. [42]

    Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue , pages=

    Integrated taxonomy of errors in chat-oriented dialogue systems , author=. Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue , pages=

  23. [43]

    Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue , pages=

    Multiwoz 2.4: A multi-domain task-oriented dialogue dataset with essential annotation corrections to improve state tracking evaluation , author=. Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue , pages=

  24. [44]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Towards scalable multi-domain conversational agents: The schema-guided dialogue dataset , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  25. [45]

    Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages=

    RiSAWOZ: A large-scale multi-domain Wizard-of-Oz dataset with rich semantic annotations for task-oriented dialogue modeling , author=. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages=

    2020

  26. [46]

    Proceedings of the 16th annual meeting of the special interest group on discourse and dialogue , pages=

    The ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems , author=. Proceedings of the 16th annual meeting of the special interest group on discourse and dialogue , pages=

  27. [47]

    Sijie Cheng, Zhicheng Guo, Jingwen Wu, Kechen Fang, Peng Li, Huaping Liu, and Yang Liu. 2024. Egothink: Evaluating first-person perspective thinking capability of vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14291--14302

    2024

  28. [48]

    Yang Deng, Lizi Liao, Liang Chen, Hongru Wang, Wenqiang Lei, and Tat-Seng Chua. 2023. Prompting and evaluating large language models for proactive dialogues: Clarification, target-guided, and non-collaboration. arXiv preprint arXiv:2305.13626

    Pith/arXiv arXiv 2023

  29. [49]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    Pith/arXiv arXiv 2025

  30. [50]

    Ryuichiro Higashinaka, Masahiro Araki, Hiroshi Tsukahara, and Masahiro Mizukami. 2021. Integrated taxonomy of errors in chat-oriented dialogue systems. In Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 89--98

    2021

  31. [51]

    Guiyang Hou, Wenqi Zhang, Yongliang Shen, Zeqi Tan, Sihao Shen, and Weiming Lu. 2024. Egosocialarena: Benchmarking the social intelligence of large language models from a first-person perspective. arXiv preprint arXiv:2410.06195

    Pith/arXiv arXiv 2024

  32. [52]

    Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. 2023. Api-bank: A comprehensive benchmark for tool-augmented llms. arXiv preprint arXiv:2304.08244

    Pith/arXiv arXiv 2023

  33. [53]

    Zuoou Li, Weitong Zhang, Jingyuan Wang, Shuyuan Zhang, Wenjia Bai, Bernhard Kainz, and Mengyun Qiao. 2025. Towards effective mllm jailbreaking through balanced on-topicness and ood-intensity. arXiv preprint arXiv:2508.09218

    Pith/arXiv arXiv 2025

  34. [54]

    Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, and 1 others. 2025. Deepseek-v3. 2: Pushing the frontier of open large language models. arXiv preprint arXiv:2512.02556

    Pith/arXiv arXiv 2025

  35. [55]

    Ryan Lowe, Nissan Pow, Iulian Vlad Serban, and Joelle Pineau. 2015. The ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems. In Proceedings of the 16th annual meeting of the special interest group on discourse and dialogue, pages 285--294

    2015

  36. [56]

    Yaxi Lu, Shenzhi Yang, Cheng Qian, Guirong Chen, Qinyu Luo, Yesai Wu, Huadong Wang, Xin Cong, Zhong Zhang, Yankai Lin, and 1 others. 2024. Proactive agent: Shifting llm agents from reactive responses to active assistance. arXiv preprint arXiv:2410.12361

    Pith/arXiv arXiv 2024

  37. [57]

    sometimes you need facts, and sometimes a hug

    Niharika Mathur, Tamara Zubatiy, Agata Rozga, Jodi Forlizzi, and Elizabeth Mynatt. 2025. " sometimes you need facts, and sometimes a hug": Understanding older adults' preferences for explanations in llm-based conversational ai systems. arXiv preprint arXiv:2510.06697

    arXiv 2025

  38. [58]

    Aaron Parisi, Yao Zhao, and Noah Fiedel. 2022. Talm: Tool augmented language models. arXiv preprint arXiv:2205.12255

    Pith/arXiv arXiv 2022

  39. [59]

    Joon Sung Park, Joseph O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pages 1--22

    2023

  40. [60]

    Gil Pasternak, Dheeraj Rajagopal, Julia White, Dhruv Atreja, Matthew Thomas, George Hurn-Maloney, and Ash Lewis. 2025. Beyond reactivity: Measuring proactive problem solving in llm agents. arXiv preprint arXiv:2510.19771

    Pith/arXiv arXiv 2025

  41. [61]

    Baolin Peng, Xiujun Li, Jianfeng Gao, Jingjing Liu, and Kam-Fai Wong. 2018. Deep dyna-q: Integrating planning for task-completion dialogue policy learning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2182--2192

    2018

  42. [62]

    Huachuan Qiu and Zhenzhong Lan. 2024. Interactive agents: Simulating counselor-client psychological counseling via role-playing llm-to-llm interactions. arXiv preprint arXiv:2408.15787

    Pith/arXiv arXiv 2024

  43. [63]

    Jun Quan, Shian Zhang, Qian Cao, Zizhong Li, and Deyi Xiong. 2020. Risawoz: A large-scale multi-domain wizard-of-oz dataset with rich semantic annotations for task-oriented dialogue modeling. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 930--940

    2020

  44. [64]

    Abhinav Rastogi, Xiaoxue Zang, Srinivas Sunkara, Raghav Gupta, and Pranav Khaitan. 2020. Towards scalable multi-domain conversational agents: The schema-guided dialogue dataset. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 8689--8696

    2020

  45. [65]

    Sagar Sapkota, Mohammad Saqib Hasan, Mubarak Shah, and Santu Karmaker. 2025. Multi-party conversational agents: A survey. arXiv preprint arXiv:2505.18845

    Pith/arXiv arXiv 2025

  46. [66]

    Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291

    Pith/arXiv arXiv 2023

  47. [67]

    Cheng-Kuang Wu, Wei-Lin Chen, and Hsin-Hsi Chen. 2023. Large language models perform diagnostic reasoning. arXiv preprint arXiv:2307.08922

    Pith/arXiv arXiv 2023

  48. [68]

    Yuexin Wu, Xiujun Li, Jingjing Liu, Jianfeng Gao, and Yiming Yang. 2019. Switch-based active deep dyna-q: Efficient adaptive planning for task-completion dialogue policy learning. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 7289--7296

    2019

  49. [69]

    Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, and Yu Su. 2024. Travelplanner: A benchmark for real-world planning with language agents. arXiv preprint arXiv:2402.01622

    Pith/arXiv arXiv 2024

  50. [70]

    Zhenyu Xu, Hailin Xu, Zhouyang Lu, Yingying Zhao, Rui Zhu, Yujiang Wang, Mingzhi Dong, Yuhu Chang, Qin Lv, Robert P Dick, and 1 others. 2024. Can large language models be good companions? an llm-based eyewear system with conversational common ground. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 8(2):1--41

    2024

  51. [71]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025 a . Qwen3 technical report. arXiv preprint arXiv:2505.09388

    Pith/arXiv arXiv 2025

  52. [72]

    Bufang Yang, Lilin Xu, Liekang Zeng, Kaiwei Liu, Siyang Jiang, Wenrui Lu, Hongkai Chen, Xiaofan Jiang, Guoliang Xing, and Zhenyu Yan. 2025 b . Contextagent: Context-aware proactive llm agents with open-world sensory perceptions. arXiv preprint arXiv:2505.14668

    arXiv 2025

  53. [73]

    Qinglong Yang, Haoming Li, Haotian Zhao, Xiaokai Yan, Jingtao Ding, Fengli Xu, and Yong Li. 2025 c . Fingertip 20k: A benchmark for proactive and personalized mobile llm agents. arXiv preprint arXiv:2507.21071

    arXiv 2025

  54. [74]

    Tianqing Yang, Tao Wu, Song Gao, and Jingzong Yang. 2023. Dialogue logic aware and key utterance decoupling model for multi-party dialogue reading comprehension. IEEE Access, 11:10985--10994

    2023

  55. [75]

    Fanghua Ye, Jarana Manotumruksa, and Emine Yilmaz. 2022. Multiwoz 2.4: A multi-domain task-oriented dialogue dataset with essential annotation corrections to improve state tracking evaluation. In Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 351--360

    2022

  56. [76]

    Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? arXiv preprint arXiv:1801.07243

    Pith/arXiv arXiv 2018

  57. [77]

    Yao Zhang, Zijian Ma, Yunpu Ma, Zhen Han, Yu Wu, and Volker Tresp. 2025. Webpilot: A versatile and autonomous multi-agent system for web task execution with strategic exploration. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 23378--23386

    2025

  58. [78]

    Zheng Zhang, Ryuichi Takanobu, Qi Zhu, MinLie Huang, and XiaoYan Zhu. 2020. Recent advances and challenges in task-oriented dialog systems. Science China Technological Sciences, 63(10):2011--2027

    2020

  59. [79]

    Yuheng Zhao, Xueli Shu, Liwen Fan, Lin Gao, Yu Zhang, and Siming Chen. 2025. Proactiveva: Proactive visual analytics with llm-based ui agent. arXiv preprint arXiv:2507.18165

    Pith/arXiv arXiv 2025