{"id":"32758174-907f-4c06-a9c2-714e7a661a12","arxiv_id":"2505.17795","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"DialogXpert combines a frozen LLM action proposer with a lightweight online Q-network and emotion tracking, achieving sub-3-turn dialogue success rates above 94 percent in LLM-simulated benchmarks.","lead":"DialogXpert uses a frozen LLM to propose a short list of next actions in a conversation, then a small Q-network trained online picks the best action while tracking the user's emotion. In simulated negotiation, support, and tutoring dialogues, it finishes in under three turns with success above 94 percent.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Success rates are computed by the same LLM critic that supplies training rewards; without independent human terminal-state labels, the 94–97% SR and <3-turn claims are not established for real users.","rationale":"The reader's conditional verdict hinges on the validity of the LLM critic and simulated user. I agree that this is the load-bearing assumption. The architecture is coherent and the release of code and prompts is a real strength; the human pairwise evaluation is encouraging but tiny and does not measure absolute success or turn count. However, if the same critic provides both training rewards and success labels, the policy can be optimized to make the critic say 'completed,' and high SR then measures policy-critic agreement, not task success. The paper's own Limitations admit critic inconsistency, so this is more than a theoretical concern. The misreported CIMA ablation numbers in Section 5.1 (the text repeats the ESConv values) are a secondary data-quality issue, not the central one. A human gold-label check on sampled transcripts would settle the matter. Since the reader already conditions acceptance on external validation, my concern does not change the verdict; the paper should remain conditional pending that check.","tokens_in":25671,"tokens_out":6017,"duration_ms":52953,"concrete_test":"Sample 100 self-play transcripts per benchmark (CraigslistBargain, ESConv, CIMA) from the released code using the Qwen 2.5 14B configuration. Have human annotators independently label the terminal outcome and the turn at which the goal is met, using task-specific rubrics: deal price reached; complete Italian translation produced; patient's emotional issue resolved. Compute human-labeled SR and AT, plus Cohen's kappa between critic and human terminal judgments. If human-labeled SR is below 90% or kappa is below about 0.6, the central claim should be reported as a critic-relative result rather than an absolute success rate.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that DialogXpert completes dialogues in under three turns with >94% success (SR >97% with Qwen 2.5 14B). The load-bearing condition is that the LLM critic's 'Completed' judgment is a valid measure of task success. This condition is insecure. Section 3.4 trains the Q-network on scalar rewards produced by the Critic LLM, and Section 5.2 states that the same LLM is 'shared across system, user, and critic roles during self-play.' Success rate is then measured by the same critic that the policy is trained to satisfy, so high SR can reflect policy-critic agreement rather than genuine goal attainment. The paper's own Limitations admit that the critic 'can behave inconsistently, sometimes terminating too early (e.g., in ESConv) or failing to end dialogues when goals are met (e.g., in CIMA).' The human evaluation in Appendix B only reports pairwise win/tie/loss on 20 dialogues; it does not validate the absolute SR or AT numbers. Thus the headline numbers are not yet evidence about real users.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DialogXpert, a dialogue planning framework that combines a frozen LLM action prior (top-k candidate actions per turn) with a lightweight Q-network trained via online temporal-difference learning, and an emotion-tracking module that feeds inferred user emotions into the state representation. The method is evaluated with self-play on negotiation (CraigslistBargain), emotional support (ESConv, ExTES, P4G), and tutoring (CIMA) benchmarks, reporting average turns below 3, success rates above 94%, and improved sale-to-list ratios. The paper also provides ablations of the emotion tracker, top-k size, and epsilon-greedy exploration, plus a cross-dataset generalization experiment from ExTES to ESConv.","tokens_in":81,"tokens_out":3056,"duration_ms":44898,"significance":"If the reported results are valid, DialogXpert offers a computationally attractive alternative to MCTS-based planners, using only a handful of LLM calls per turn and a small trainable Q-network. The paper includes several strengths: the method is clearly described, the code is released, and the ablation study (e.g., Table 1 rows for -w/o Emotion, -w/o LLM-Prior) helps identify the contribution of each component. However, the headline claims hinge on the validity of an LLM-based critic that simultaneously provides training rewards and success-rate labels in a closed self-play loop. The paper's own Limitations section concedes that this critic is inconsistent. Without independent terminal-state labels or a human-validated success metric, the absolute SR and AT numbers remain unverified for real users, and the claimed superiority over baselines is therefore conditional on the self-play protocol being a faithful proxy for true task success.","major_comments":[{"comment":"The training reward and the reported success rate come from the same Critic LLM prompt family that also supplies the system and user roles during self-play. Section 5.2 explicitly states the LLM is 'shared across system, user, and critic roles.' This means the Q-network is trained to maximize rewards from this critic, and success is then measured by that same critic. Such circularity can inflate SR: the policy may learn to satisfy the critic's termination criterion rather than achieve genuine task goals. The paper's Limitations section admits the critic 'can behave inconsistently, sometimes terminating too early (e.g., in ESConv) or failing to end dialogues when goals are met (e.g., in CIMA).' The human evaluation in Appendix B only reports pairwise win/tie/loss on 20 dialogues and does not validate absolute SR or AT values. To support the abstract's claims, the authors should provide an independent ground-truth evaluation (e.g., human judges or rule-based terminal-state checks) for at least a subset of dialogues, and report SR and AT on those instances.","section":"§3.4, §5.2, Appendix B"},{"comment":"No error bars, confidence intervals, or multiple-seed results are reported anywhere in Tables 1-4 or Figure 2. All numbers are point estimates from a single run (or an unreported number of runs). Given the stochasticity of LLM sampling, the simulated user, and the epsilon-greedy exploration (epsilon=0.5 at evaluation), the differences between configurations—e.g., SR 0.9876 vs 0.9805 for Qwen-2.5-14B vs Qwen-1.8B on ESConv—may be within noise. The paper should report means and variances over at least 3 seeds and state the number of evaluation episodes used for AT/SR/SL.","section":"§5.1, Table 1"},{"comment":"The test-time epsilon is fixed at 0.5, meaning half of the actions are chosen uniformly at random among the top-4 candidates (unless the footnote implies epsilon applies only during training; the text is ambiguous). If 50% random actions still yield SR above 0.97, this raises questions about the sensitivity of the success metric to action quality. The paper should clarify whether epsilon is applied during evaluation, and should report results under pure exploitation (epsilon=0). If epsilon is applied at evaluation, the reported 'success' is not the success of the learned policy, but of a stochastic mixture that includes random actions.","section":"§5.1, Figure 2, Table 1 footnote"},{"comment":"The claim that the LLM prior's top-k set is sufficient for good performance is not verified. The paper approximates pproj(a|s_t) via beam search and extracts the top-k (k=4) actions, but never measures how often the optimal action (e.g., the action selected by the full prior or by an oracle) falls outside this candidate set. Without such a coverage analysis, the strong results only show that the Q-network selects well within the top-4, not that the top-4 always contains a good action. A concrete test would be to run an ablation with k equal to the full action set and compare SR/AT; this would separate the prior's coverage from the Q-network's selection ability.","section":"§3.2, Appendix A"}],"minor_comments":[{"comment":"The phrase 'solving dialogues in under 3 turns' is imprecise because the measured metric is average turns, not the maximum or median; consider phrasing 'average dialogue length below 3 turns.'","section":"Abstract/§1"},{"comment":"Typo: 'V oyager' should be 'Voyager' (also in the reference list).","section":"§2, Related Works"},{"comment":"The reward mappings in Appendix E are not derived from any user study or calibration; the claim that the mapping is 'consistent with prior works' is not supported by direct citations to those mappings. Please provide the exact prompts and mapping tables used for PPDPP/DPDP/LDPP for comparability.","section":"§4.3 / Appendix E"},{"comment":"The table reports LDPP results on ExTES (SR 0.903, AT 4.132) but the text says 'LDPP relies heavily on RoBERTa-based encoders/decoders'; it would be helpful to clarify the backbone used for LDPP in the table, since the original paper uses Qwen1-1.8B as the policy backbone.","section":"§5.1, Table 2"},{"comment":"The text says 'four expert annotators' and that they were given '40 dialogue contexts in total' (20 ESConv and 20 CIMA), but the results section only reports ESConv in the main text; please include the CIMA human-evaluation results in the main paper or move them to a separate appendix with clear interpretation.","section":"Appendix B"},{"comment":"The generalization result (ExTES-trained model evaluated on ESConv) is reported without any statistical significance test or comparison of the same critic distribution shift; please report the number of test dialogues and include a baseline of the critic's own agreement on the same dialogues.","section":"§5.1, Generalization Test"}],"recommendation":"major_revision","confidential_remarks":"The paper is well-written and the proposed architecture is plausible, but the core evaluation is compromised by the closed-loop LLM critic that both shapes the policy and measures success. I would recommend asking for an independent terminal-state evaluation (even on a subset) and multi-seed error bars before the claims can be taken at face value. The epsilon=0.5 evaluation choice is also unusual and should be clarified."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"DialogXpert is a clean engineering combination: a frozen LLM proposes top-k dialogue actions, a small BERT-based Q-network scores them via TD learning, and an LLM emotion tracker feeds the state. The components are known, but the packaging is new and the code and prompts are released, which is real. The paper is honest enough to list limitations, including that the critic LLM can terminate early or miss goal attainment.\n\nThe soft spot is the evaluation. Success rate and average turns are measured in a closed self-play loop where the same LLM family acts as policy, user, and critic. The critic supplies the training reward and also the terminal 'Completed' judgment, so the reported 94–97% SR may mostly reflect policy-critic agreement, not real task success. The paper's own limitations concede the critic is inconsistent. The human evaluation only compares 20 dialogues against PPDPP on win/tie/loss; it does not validate the absolute numbers. There are also reporting errors: the 'Impact of Emotions' text gives CIMA numbers that don't match Table 1, and the 'Impact of LLM Prior' section cites the -w/o RL row as the -w/o LLM-Prior effect. No error bars, no seeds, and the test-time epsilon of 0.5 (50% random actions) is an unusual protocol that needs justification.\n\nThe central architecture is worth engaging with; the evaluation protocol needs external validation with human users or at least a held-out, independently labeled terminal state. The paper deserves peer review but with major revisions expected on the evaluation and reporting. I'd bring it to a reading group as a case study in self-play eval pitfalls.","headline":"A clean and honest architecture for LLM-prior dialogue planning, but the headline 94–97% success rates are unvalidated because the same LLM critic that supplies training rewards also judges task success.","tokens_in":26435,"tokens_out":2164,"would_cite":true,"duration_ms":22381,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DialogXpert claims that a frozen LLM proposing top-k actions, scored by a small Q-network trained online with emotion tracking, makes goal-driven dialogues finish in under three turns with over 94% success, and that scaling the LLM prior…","keywords":["dialogue planning","large language models","reinforcement learning","Q-learning","emotion-aware dialogue","LLM priors","self-play","goal-driven conversation"],"falsifier":"Run the same DialogXpert policy against human users on negotiation and emotional support, counting actual task completion and user-rated satisfaction; if the sub-3-turn, above-94% success pattern does not appear with real users while the self-play critic still reports success, the reward and user-simulation chain is the point of failure. A cheaper probe is to change the CIMA reward mapping, for example correcting the 0.5 reward for partial translation, and check whether reported success changes materially.","tokens_in":25454,"feed_emoji":"💬","tokens_out":9599,"duration_ms":79046,"temperature":0.7,"pith_summary":"DialogXpert claims that proactive dialogue does not need expensive full-tree search: a frozen LLM proposes a small set of candidate actions each turn, and a compact Q-network trained with temporal-difference learning selects among them, with the user's inferred emotion folded into the state. Across negotiation, emotional support, and tutoring benchmarks, the paper reports average conversations ending in under three turns with success rates above 94%, and above 97% when the LLM prior is scaled up. This matters because LLM agents are otherwise reactive or myopic, and prior planners either act greedily or pay the latency of Monte Carlo tree search. If true, the framework offers real-time, goal-driven, emotionally aware conversation planning at scale.","feed_headline":"Sub-3-turn dialogues, 94% success: frozen LLM + tiny Q-network","feed_subtitle":"Goal-driven chats finish in under three turns when a frozen LLM proposes actions and a tiny Q-network chooses.","key_machinery":"The central object is the LLM Action Prior, a frozen LLM that takes the serialized case information, conversation history, and emotion history together with the full action inventory and returns a free-form continuation that is deterministically projected onto a valid action, implicitly defining a normalized prior over the action set. Beam search approximates this prior and yields the top-k candidate actions for the turn. A lightweight Q-network, built from a fixed BERT encoder followed by a three-layer MLP, scores those candidates, and softmax over the Q-values selects the executed action. The Q-network is updated online through temporal-difference Q-learning with Bellman backups; all LLM components and BERT remain frozen, so only the small value network changes during training.","core_discovery":"The central claim is that the usual trade-off between myopic generation and deep search can be broken by making a frozen LLM the action proposer rather than the decision-maker. DialogXpert prompts the LLM with the case, history, and accumulated user emotions, extracts a top-k set of semantically coherent actions via a free-form plus projection prior, and lets a BERT-embedded Q-network pick the best one using Q-learning with an epsilon-greedy policy. An LLM critic supplies scalar rewards for each transition, and an emotion tracker injects the inferred user feeling into the state representation. The paper shows that removing either the LLM prior or the emotion tracker degrades every metric, and that the Q-network trained on ExTES transfers to ESConv without fine-tuning, which argues the learned policy captures reusable dialogue skill rather than dataset memorization.","pith_inferences":["The paper's own limitations section concedes the critic \"can behave inconsistently, sometimes terminating too early (e.g., in ESConv) or failing to end dialogues when goals are met (e.g., in CIMA),\" so the reported turn counts and success rates partly inherit the critic's termination behavior rather than measuring a fixed task objective.","The same \"frozen LLM proposes, small Q-network selects\" pattern could be applied to other finite-action sequential decision problems, such as web navigation or tool use, wherever top-k sampling is cheap.","The emotion tracker's open-ended emotion space is an implementation choice; a lightweight classifier over a fixed emotion set would likely be more stable and cheaper, as the paper's own limitation discussion hints.","The reported four LLM calls per turn may understate runtime cost since the paper notes its dynamic state-action space prevents caching, so real-world latency depends on the full per-turn prompt cost."],"forward_implications":["Under the reported protocol, planning needs only four LLM calls per turn instead of roughly thirty from MCTS rollouts, so the method is fast enough for real-time deployment.","Scaling the frozen LLM prior from 13B to 14B raises success from above 94% to above 97% and improves negotiation outcomes, making prior-model scale a direct lever.","Ablations show the LLM prior and emotion tracker are each load-bearing: dropping the prior raises average turns and lowers success, while dropping emotion tracking lowers success and, in negotiation, cuts the sale-to-list ratio from 0.4389 to 0.3156.","The ExTES-trained Q-network transfers to ESConv without fine-tuning, suggesting the learned state-action values generalize across similar task environments.","Human raters prefer DialogXpert over a supervised planner on emotional support and tutoring, so the efficiency gains are accompanied by perceived quality gains."],"supporting_citations":[{"why":"Supplies the free-form plus projection LLM-prior technique that DialogXpert adapts to build its top-k candidate set.","marker":"(Yan et al., 2024)"},{"why":"Establishes Q-learning, the temporal-difference update at the core of the Q-network training.","marker":"(Watkins and Dayan, 1992)"},{"why":"Introduces the PPDPP planner and the self-play protocol with LLM user and reward simulators that DialogXpert follows and improves upon.","marker":"(Deng et al., 2024)"},{"why":"Provides the DPDP dual-process and MCTS baseline that DialogXpert compares against on efficiency and success.","marker":"(He et al., 2024)"},{"why":"Provides the LDPP baseline and the ExTES-to-ESConv generalization evaluation protocol.","marker":"(He et al., 2025a)"},{"why":"Provides the ESConv emotional-support dataset and the human evaluation criteria.","marker":"(Liu et al., 2021)"},{"why":"Provides the CraigslistBargain negotiation dataset and the sale-to-list-ratio metric.","marker":"(He et al., 2018)"},{"why":"Provides the CIMA tutoring dataset used in the main experiments.","marker":"(Stasaski et al., 2020)"}],"fun_headline_variants":["Frozen LLM proposes, tiny Q-network picks: 94% success, sub-3-turn chats","Emotion-aware Q-learning with LLM priors: 97% success in under 3 turns","DialogXpert: LLM as proposer, Q-network as decider, 94% success, <3 turns","LLM proposes, emotion-aware Q-network chooses: 3-turn dialogues, 94% success"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the LLM critic's scalar rewards and the LLM-simulated user provide a valid and stable measurement of dialogue success and user satisfaction, so if either is biased the reported success rates may not transfer to real users.","fun_headline_variants_meta":{"raw":{"variants":["Frozen LLM proposes, tiny Q-network picks: 94% success, sub-3-turn chats","Emotion-aware Q-learning with LLM priors: 97% success in under 3 turns","DialogXpert: LLM as proposer, Q-network as decider, 94% success, <3 turns","LLM proposes, emotion-aware Q-network chooses: 3-turn dialogues, 94% success"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001297,"raw_usage":{"total_tokens":5262,"prompt_tokens":883,"completion_tokens":4379,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":499,"completion_tokens_details":{"reasoning_tokens":4272}},"tokens_in":499,"tokens_out":4379,"duration_ms":24780,"temperature":1.0,"reasoning_tokens":4272,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:40:07.004574+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same DialogXpert policy against human users on negotiation and emotional support, counting actual task completion and user-rated satisfaction; if the sub-3-turn, above-94% success pattern does not appear with real users while the self-play critic still reports success, the reward and user-simulation chain is the point of failure. A cheaper probe is to change the CIMA reward mapping, for example correcting the 0.5 reward for partial translation, and check whether reported success changes materially.","supporting_citations":[],"review_version":1}