{"id":"42e0772f-3b4d-4aae-8475-37c843f1a6d6","arxiv_id":"2506.05947","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Adds explicit supporter intention inference to emotional support dialogue generation; claims improved response quality, but the evaluation is small-scale, partially confounded, and the full chain does not always help.","lead":"This paper introduces IntentionESC, a framework that gives emotional support chatbots explicit intentions, such as clarifying, comforting, or giving hope, and a chain-of-thought that reasons from the user's emotional state to the supporter's intention to the response strategy. The authors claim this improves supportive response quality over prior dialogue systems, but the evaluation is small-scale and partly confounded.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Intention labels in §4.3 are generated from ESConv's strategy annotations through the Figure 2 mapping, so ICECoT's gains may be an artifact of added strategy-conditioned tokens rather than an independent intention construct; the Limitations admit this bias.","rationale":"The paper's central claim is that explicitly modeling the supporter's intention improves emotional support responses. For that claim to hold, the 'intention' in the training data must be an independent, meaningful variable rather than a restatement of the strategy label. The pipeline in §4.3 violates this: intention generation receives the refined strategy annotation and the candidate intentions derived from it via Figure 2. GPT-4 then writes a natural-language intention that includes specific dialogue content, but the candidate set is already constrained by the strategy. Hence the intention step carries little information beyond the strategy label. The paper's own Limitations explicitly acknowledge this bias. The same-backbone ablation in Table 2 does not resolve the issue, because every ICECoT condition uses the same pipeline-derived intentions; the comparison only varies whether the intention text is generated during inference. A model trained with pipeline intentions may do better on Empathy, Informativeness, and whole-conversation metrics simply because the additional reasoning text makes the response more elaborate or more closely aligned with the ESConv ground-truth strategies. The reader's conditional verdict is therefore warranted: the central conceptual claim requires independent intention labels, and the current evidence does not distinguish genuine intention-driven support from strategy-conditioned generation. The proposed re-annotation test would settle this. If direct labels reproduce the gains, the concern is resolved; if not, the reported superiority is likely an artifact of the annotation pipeline. No additional adjustment to the reader's verdict is needed, since the reader already conditioned acceptance on the need for more reliable and unbiased evaluation.","tokens_in":19103,"tokens_out":7988,"duration_ms":80147,"concrete_test":"Re-annotate a random sample of 200 ESConv turns with intention labels directly from the dialogue history, using three trained annotators who are blind to both the ESConv strategy annotations and the Figure 2 mapping; measure inter-annotator agreement (e.g., Cohen's kappa) and the agreement between direct labels and the pipeline-generated intentions. Then fine-tune ICECoT on a directly labeled dataset of at least a few hundred turns (or on the full ESConv train split re-annotated in this manner) and compare its evaluation scores against the current pipeline-trained model. If the direct-label model matches or exceeds the pipeline model on the same metrics, the concern is not confirmed; if it underperforms substantially, the pipeline's strategy-derived intentions are the source of the reported gains.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that explicitly modeling the supporter's intention improves emotional support responses. That requires the intention variable in the training data to be an independent, meaningful construct, not a restatement of the strategy label. The pipeline in §4.3 violates this: Strategy Refinement converts the existing ESConv strategy annotation into fine-grained strategies; Intention Estimation then derives candidate intentions from those strategies using the predefined intention–strategy mapping in Figure 2; Intention Generation finally asks GPT-4 to write a natural-language intention based on the dialogue and those candidates. The resulting intention is therefore a paraphrase of a deterministic function of the very strategy label the framework is meant to justify. The paper's own Limitations state that the pre-existing strategy annotation distribution may disproportionately influence intention annotations, admitting the bias. Moreover, §5.5 reports that response–strategy mismatches may stem from 'inconsistencies or errors in the original ESConv strategy annotations,' so the intention labels inherit ESConv label noise. The same-backbone ablation in Table 2 does not resolve this, because every ICECoT condition uses the same pipeline-derived intentions; the ablation only varies whether the intention text is emitted during inference. If intentions are nearly deterministic given strategies, the observed gains on Empathy, Informativeness, and whole-conversation metrics could come from adding an intermediate token that is correlated with the training target, not from intention-driven reasoning. The pipeline therefore does not provide evidence that the intention construct itself adds causal value.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes IntentionESC, a framework that defines 12 supporter intentions in emotional support conversations, identifies four emotional-state aspects for inferring them, and maps intentions to support strategies. Building on this, the authors introduce ICECoT, a chain-of-thought mechanism in which a model first analyzes the seeker's emotional state, then infers the supporter's intention, then selects a strategy, and finally generates a response. They describe an automated GPT-4-based annotation pipeline that enriches the ESConv dataset with emotional-state and intention labels, train Llama3.1-8B-Instruct on this data, and evaluate the resulting model against BlenderBot, MultiESC, and ESCoT using a mix of GPT-4 ranking and human annotation on single-response and whole-conversation dimensions. The paper claims that ICECoT outperforms all baselines across all evaluated metrics.","tokens_in":19410,"tokens_out":5956,"duration_ms":53746,"significance":"If the central claim were fully supported, the paper would make a useful contribution: explicitly modeling supporter intentions in emotional support dialogue is a plausible route to more interpretable and effective systems, and the released data, code, and evaluation scheme would be reusable assets for the community. The authors also deserve credit for being transparent about the main limitation of their annotation pipeline, for proposing a new evaluation framework that includes safety and consistency dimensions, and for making their data and code available. However, the current evidence is not yet sufficient to establish the claimed superiority of the intention-centered approach: the main baseline comparison is confounded by different base models, the intention labels are derived from the very strategy annotations they are meant to explain, and the reliability of the human evaluation is low on several key dimensions.","major_comments":[{"comment":"The headline comparison in Table 1 is confounded by backbone. ICECoT is trained on Llama3.1-8B-Instruct, while ESCoT is the released Llama2-7B-chat model, and BlenderBot and MultiESC are smaller architectures. Because the main claimed advantage over ESCoT may be explained by the newer/larger instruction-tuned base model, the comparison does not isolate the effect of intention modeling. Please add a same-backbone ESCoT run, or an equivalent Llama3.1 baseline without ICECoT, before claiming that ICECoT outperforms prior methods.","section":"§5.3, Table 1, Appendix C"},{"comment":"The intention labels are not an independent construct. The pipeline first refines ESConv strategy annotations, then generates candidate intentions using the predefined intention–strategy mapping in Figure 2, and then asks GPT-4 to produce a natural-language intention from those candidates. The resulting intention is therefore effectively a paraphrase of a deterministic function of the strategy label that ICECoT is supposed to explain. The Limitations section itself concedes that the pre-existing strategy distribution may bias the intention annotations. To support the central claim, please provide evidence that the learned intention variable carries information beyond the strategy label, for example human intention annotations collected without exposing annotators to the strategy label, or a distributional analysis showing that the emitted intentions are not nearly deterministic given the strategy.","section":"§4.3, Figure 5, Limitations"},{"comment":"The sentence 'Our ICECoT outperforms other compared methods on all metrics' overstates the results. Only four of the seven dimensions carry a ‡ (p<0.05), and the reported p-values appear to come from a single sign test over all models rather than pairwise tests against each baseline. Please report pairwise p-values against each compared method, apply a multiple-comparison correction if needed, and restrict the superiority claim to the dimensions that are actually significant.","section":"§5.3, Table 1"},{"comment":"The full ICECoT chain underperforms direct generation on Base Quality, Empathy, Informativeness, and Comforting. The final best system is ICECoT plus an additional 'strategy-and-response-only' training set D_SA, which was introduced after observing this underperformance. As presented, the central claim that ICECoT itself improves emotional support is therefore contingent on a post-hoc repair. Please justify D_SA as a principled component of the method, report whether its introduction was pre-specified, and provide significance tests comparing the final model with the direct-generation setting on each dimension.","section":"§5.4, Table 2, Appendix E"},{"comment":"The inter-annotator agreement is very low for several load-bearing dimensions: in Table 1, Identification κ=0.12 and Comforting κ=0.17; in Table 2, Comforting κ=0.07 and Identification κ=0.21. With agreement at these levels, the ranking data may be too noisy to support fine-grained cross-model conclusions. Please report the number of annotators per item, the adjudication procedure, and consider pooling dimensions or using a more reliable scale, or interpret results with appropriate caution.","section":"§5.2, Tables 1 and 2"}],"minor_comments":[{"comment":"There is a typo: 'acomplish' should be 'accomplish'.","section":"§3.1"},{"comment":"The strategy numbers under each intention are difficult to decode; please add a legend that explicitly maps the numbers to the strategy list.","section":"Figure 2"},{"comment":"The κ values are listed in a single dense line in the captions; please report them per dimension in the table itself so that reliability is transparent and easy to compare.","section":"Tables 1 and 2 captions"},{"comment":"The composite 'Base Quality' dimension is formed from Fluency, Coherence, Safety, and Consistency, which are evaluated by GPT-4 ranking, while Empathy and Informativeness are evaluated by human annotators; please clarify exactly how the composite ranking was aggregated (e.g., average ranks, majority vote, or another rule).","section":"§5.2"},{"comment":"The prompt text contains a typo: 'evluation' should be 'evaluation'.","section":"Figure 10"},{"comment":"The claim of being 'the first to focus on supporters' intentions' should be softened: related work such as GLHG already models a local intention of the current speaker, so the novelty should be positioned as supporter-intention learning and intention-driven strategy selection rather than intention modeling per se.","section":"§1 and §2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within scope for a computational linguistics journal, and the authors are commendably transparent about the limitations of their annotation pipeline. My main concern is whether the central claim survives a same-backbone comparison and an independent validation of the intention construct; I would like the revision to include those experiments before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The real contribution here is conceptual: inserting an explicit supporter-intention stage between emotional state and strategy selection, with a 12-intention taxonomy adapted from Hill and O'Grady. That is a legitimate extension of ESCoT, not a rehash, and it gives the subfield a cleaner way to talk about why a supporter chooses a strategy. The paper also ships data, code, and an interpretable ICECoT chain, and the authors are transparent about the main weakness in their Limitations section. Credit where due: this is a serious, readable piece of work.\n\nThe soft spots are real but not all equally soft. The baseline comparison is the biggest problem: ICECoT runs on Llama3.1-8B while ESCoT uses Llama2-7B, so the headline “outperforms all baselines” is not a controlled comparison. The human evaluation is also thin—kappas as low as 0.07 on Comforting, significance on only four of seven dimensions—so the reported gains are suggestive, not conclusive. The ablation is more troubling: the full ICECoT chain actually lowers Base Quality compared to direct strategy+response generation, and only the mixed-training variant recovers. That is an honest result, but it weakens the claim that the chain itself is what helps.\n\nThe stress-test note about intention-label circularity lands. The intention annotations are produced from the ESConv strategy labels through the paper's own intention–strategy mapping, so the intention variable is partly a paraphrase of the strategy label. The authors admit this bias in Limitations. That does not make the framework useless—the intention text still adds explicit goals that may improve generation—but it means the experiments do not establish that the intention construct itself causes the gains. The gains could come from extra intermediate tokens correlated with the training target.\n\nWho gets value from this paper? Anyone working on emotional support dialogue or explainable response generation. It is a useful framework paper with an honest evaluation, but the empirical evidence is not yet strong enough to claim a decisive advance. A serious referee should ask for a same-backbone comparison against ESCoT, larger and more reliable human evaluation, and some sensitivity analysis on the intention–strategy mapping. I would send it to review, with a clear request for those revisions. It deserves referee time, not a desk reject.","headline":"IntentionESC adds a therapy-grounded intention stage to ESC, but the central causal claim is undercut by derived intention labels and uncontrolled baselines.","tokens_in":19938,"tokens_out":912,"would_cite":false,"duration_ms":17232,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Explicit supporter intention improves emotional support responses in dialogue systems.","keywords":["emotional support conversation","intention inference","chain-of-thought","large language models","strategy selection","ESConv","human evaluation","emotional state analysis"],"falsifier":"Run a controlled human evaluation in which the ICECoT model is compared with an identical model whose intention-inference step is replaced by a random intention drawn from the same training distribution, keeping data and generation otherwise unchanged; if scores do not drop significantly, the intention step is not carrying the improvement. A second check is to evaluate ICECoT on emotional support conversations outside ESConv, where the strategy-label distribution that generates the intention annotations is absent.","tokens_in":18901,"feed_emoji":"💬","tokens_out":7254,"duration_ms":68892,"temperature":0.7,"pith_summary":"The paper asks whether emotional support dialogue systems respond better when the supporter's intention is made explicit before a strategy is chosen. It argues that unclear intentions lead supporters to impose their own expectations or solutions, and that a clear intention should drive strategy selection. To test this, the paper defines IntentionESC—a framework with 12 supporter intentions adapted from counseling practice, four aspects of the seeker's emotional state that support intention inference, and a mapping from intentions to strategies—and introduces ICECoT, a chain-of-thought that walks a model through emotional state analysis, intention inference, strategy selection, and response generation. The authors report that a Llama-3.1-8B model trained on auto-annotated ICECoT chains outperforms previous emotional support systems on all of their single-response and whole-conversation evaluation metrics.","feed_headline":"Intention-first reasoning wins every emotional support metric","feed_subtitle":"A new chain-of-thought makes chatbots analyze the seeker's emotions, infer the supporter's goal, then choose a strategy.","key_machinery":"The load-bearing mechanism is the ICECoT reasoning chain together with the intention–strategy mapping from IntentionESC. The mapping connects 12 intentions (e.g., Clarify, Cathart, Insight, Support, Hope, Give information) to the support strategies used in ESConv, such as Open Questions and Probes, Affirmation and Reassurance, and Providing Suggestions. The ICECoT chain makes the reasoning explicit in four steps: emotional state analysis over four aspects, intention inference, strategy selection, and response generation. An automated pipeline built on GPT-4 produces the training chains by refining the original strategy labels and using the intention–strategy mapping to propose intention categories, and the model is trained to reproduce both the chain and the final response.","core_discovery":"On its own terms, the paper's central claim is that intention is the missing link between understanding the seeker's emotional state and selecting an effective support strategy. The authors define a supporter's intention as the goal, motivation, and expectation behind a response, and they derive 12 such intentions from the therapy literature. They claim that when a model first analyzes the seeker's main issue and causes, current emotions, needs, and relationship dynamics, then infers an intention, then picks a strategy, the resulting responses are more effective than responses generated from emotional state or strategy alone. In their experiments, ICECoT trained on ESConv with automatically annotated reasoning chains receives the best human and GPT-4-based scores on all single-response and entire-conversation dimensions compared to BlenderBot, MultiESC, and ESCoT.","pith_inferences":["The 12-intention taxonomy could be validated as a complete ontology by collecting open-domain support dialogues and checking for intentions that fall outside the list; the paper's own limitation about limited scenarios points in this direction.","A control condition that replaces the inferred intention with a paraphrase of the selected strategy would reveal whether the intention step adds signal beyond the strategy label, since the annotation pipeline derives intention candidates from strategy annotations.","The intention-to-strategy mapping could be exposed to users as a control knob, letting a person choose the kind of support they want (e.g., clarifying questions rather than advice) before the model writes a response."],"forward_implications":["Future emotional support systems can condition generation on an inferred intention instead of only on emotion labels or strategies.","The four-aspect emotional state checklist provides a reusable annotation scheme for building larger emotional support datasets.","ICECoT makes the model's reasoning visible, so users and evaluators can check whether the chosen strategy follows from an intention that matches the seeker's state.","Training on mixed data—full reasoning chains plus direct strategy-response pairs—can compensate for fluency losses that long chains cause, a design choice the paper's ablations support."],"supporting_citations":[{"why":"Supplies the ESConv dataset, its strategy labels, and the three-stage ESC framework that the paper augments with intention inference.","marker":"(Liu et al., 2021)"},{"why":"Provides the original therapist intention list that the paper adapts into its 12 emotional support intentions.","marker":"(Hill and O'Grady, 2001)"},{"why":"Provides Helping Skills Theory and the intention-to-strategy correspondence that grounds IntentionESC and ICECoT.","marker":"(Hill, 2009)"},{"why":"Defines ESCoT, the emotion-focused chain-of-thought baseline that ICECoT extends by inserting intention inference between emotional state analysis and strategy selection.","marker":"(Zhang et al., 2024a)"},{"why":"Defines MultiESC, a lookahead strategy-planning baseline that ICECoT is compared against on whole-conversation evaluation.","marker":"(Cheng et al., 2022)"},{"why":"Defines MISC, a mixed strategy-aware baseline whose experimental setup the paper follows for data splits and comparisons.","marker":"(Tu et al., 2022)"},{"why":"Releases the Llama 3.1-8B-Instruct base model that ICECoT is trained on and evaluated with.","marker":"(Dubey et al., 2024)"},{"why":"Earlier work modeling the seeker's local intention in emotional support conversations, which the paper contrasts with its focus on the supporter's intention.","marker":"(Peng et al., 2022)"}],"fun_headline_variants":["Infer intentions first to boost support scores","Intention reasoning improves chatbot empathy","ICECoT: plan support by reading intentions first","Emotional support bots gain from intention inference"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a supporter's intention can be reliably read off from a seeker's emotional state and, in the training data, from the existing strategy labels; if those strategy labels are noisy or the 12 therapy-derived intentions do not fit everyday support conversations, the intention annotations inherit that bias and the reported gains may not transfer.","fun_headline_variants_meta":{"raw":{"variants":["Infer intentions first to boost support scores","Intention reasoning improves chatbot empathy","ICECoT: plan support by reading intentions first","Emotional support bots gain from intention inference"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000171,"raw_usage":{"total_tokens":1269,"prompt_tokens":940,"completion_tokens":329,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":556,"completion_tokens_details":{"reasoning_tokens":274}},"tokens_in":556,"tokens_out":329,"duration_ms":4157,"temperature":1.0,"reasoning_tokens":274,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T10:12:53.853561+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a controlled human evaluation in which the ICECoT model is compared with an identical model whose intention-inference step is replaced by a random intention drawn from the same training distribution, keeping data and generation otherwise unchanged; if scores do not drop significantly, the intention step is not carrying the improvement. A second check is to evaluate ICECoT on emotional support conversations outside ESConv, where the strategy-label distribution that generates the intention annotations is absent.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the original therapist intention list that the paper adapts into its 12 emotional support intentions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides Helping Skills Theory and the intention-to-strategy correspondence that grounds IntentionESC and ICECoT."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Earlier work modeling the seeker's local intention in emotional support conversations, which the paper contrasts with its focus on the supporter's intention."}],"review_version":1}