Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

Towards Open-Ended Emotional Support Conversations in LLMs via Reinforcement Learning with Future-Oriented Rewards

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A compact 7B model trained on simulated future dialogue resolves emotional support goals more often than GPT-4o in the same evaluation.

desk verdict A plausible RL training recipe for emotional support that deserves a serious referee, but the headline goal-completion numbers are measured entirely inside an LLM user/critic loop and are not yet anchored to real users. read the letter →

arxiv 2508.12935 v1 pith:MZAV3XGL submitted 2025-08-18 cs.AI

Ting Yang , Li Chen , Huimin Wang This is my paper · ORCID
classification cs.AI
keywords emotionalsupportconversationreinforcementlearningGRPOfuture-orientedrewardmulti-agentdialoguesimulationmodelLLMalignmentopen-endedgeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper attempts to show that an emotional-support chatbot can be trained to aim at the user's long-term emotional state rather than only at the quality of the next reply. It proposes RLFF-ESC, which uses LLM-simulated future conversations to score each response by whether the user's problem eventually gets resolved, then optimizes a small LLM with reinforcement learning against that score. On two public datasets, the trained 7B model resolves more support goals than much larger models in matched evaluation, and human annotators prefer its responses on empathy, identification, suggestion, and overall quality. The intended payoff is that compact models can deliver open-ended, adaptive support without being restricted to predefined strategy menus.

What carries the argument

The load-bearing object is the future-oriented reward model, a binary classifier built from a frozen LLaMA-3.2-1B plus a linear head that predicts whether a response resolves the user's emotional problem. It is trained on $(\text{context}, \text{response}, r)$ tuples produced by a three-agent simulation: the policy model acts as supporter, a second LLM plays the user, and a critic LLM grades whether the emotional issue is resolved over simulated future turns; the scalar reward combines the terminal critic rating with the number of turns to resolution. During RL, GRPO samples groups of candidate outputs, normalizes the reward $R_{\mathrm{rlff}} = R_{\mathrm{fut}} + \alpha R_{\mathrm{thk\text{-}fmt}}$, and updates the policy to favor responses whose simulated futures end better. The format reward $R_{\mathrm{thk\text{-}fmt}}$ forces output into $\langle\text{think}\rangle \ldots \langle/\text{think}\rangle$ and $\langle\text{response}\rangle \ldots \langle/\text{response}\rangle$, which is how the explicit reasoning process is elicited. This design steers the model from immediate-token likelihood toward outcome-level consequences.

What would settle it

Run RLFF-ESC and a strong baseline against real users and measure their emotional state before and after each conversation with a standard self-report instrument. If users of the RLFF-ESC system show no greater improvement in emotional state than users of the baseline, even though the LLM critic reports higher resolution rates, then the central claim that the future-oriented reward tracks enduring emotional recovery is refuted.

Watch

Extended reading notes

Core claim

RLFF-ESC claims that reinforcement learning from future-oriented rewards converts a general instruction-tuned LLM into an emotional supporter that reasons explicitly and resolves distress in fewer turns. The paper's central result is the success-rate advantage: Qwen2.5-7B with RLFF-ESC reaches 41.5% on ESConv versus 30.8% for GPT-4o, 27.7% for Qwen2.5-72B, and 23.9% for LLaMA-3.1-405B under the same zero-shot evaluation, while also improving average turns to goal. Pairwise human evaluation on 100 dialogues reports wins over every baseline on fluency, empathy, identification, suggestion, and overall quality. The authors interpret this as evidence that optimizing for estimated enduring emotional impact, rather than for matching ground-truth next utterances or for selecting predefined strategies, yields responses that are both more effective and more human-preferred.

Load-bearing premise

The load-bearing premise is that the score an LLM critic gives to simulated future conversations is a valid proxy for real improvement in a real user's emotional state; the paper explicitly did not measure end users' emotional intensity.

Editorial extensions

If this is right

  • If RLFF-ESC is correct, a 7B-class model can outperform frontier API models on goal completion in emotional support under matched zero-shot prompting.
  • Training on simulated future trajectories can substitute for costly human annotation of long-term conversational outcomes, making outcome-driven RL scalable.
  • Explicit reasoning before responding improves downstream resolution, so interpretable intermediate reasoning can serve as a practical training target for dialogue systems.
  • The method transfers across backbones (LLaMA-3.1-8B and Qwen2.5-7B) and across datasets (ESConv and ExTES), suggesting that the future-oriented reward signal, not a single model, drives the gain.
  • A classification-based future reward beats ranking-based and random reward designs in the ablation, so a simple binary 'resolved' prediction is an effective training signal for GRPO in this setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: because the reward model and the evaluation success metric are both LLM-based, the reported success-rate gains may partly reflect the policy learning to satisfy the LLM critic rather than to change a real human's emotional state; a user study with direct emotional measures would separate those explanations.
  • Beyond the paper: the same simulation-reward-GRPO pipeline could be applied to other long-horizon dialogue goals, such as tutoring, behaviour change, or negotiation, wherever the outcome is defined by the downstream conversation state rather than by a single turn.
  • Beyond the paper: the reward formula's explicit averaging over turns penalizes long conversations, so a testable extension is whether this efficiency pressure ever trades away depth for premature closure in real users who need more time.
  • Beyond the paper: the threshold analysis in the paper suggests that the definition of 'success' is sensitive to critic calibration; a natural extension is to train the reward model against graded emotional-outcome data rather than a binary resolved/not-resolved label.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces RLFF-ESC, a reinforcement-learning framework for open-ended emotional support conversations. The method has three stages: (1) a multi-agent LLM simulation samples future dialogue trajectories for candidate system responses and produces scalar future-oriented rewards; (2) a LLaMA-based reward model is trained on these simulated labels; (3) GRPO optimizes the policy with a reward composed of the future-oriented reward and a format reward that enforces <think>/<response> structure. Experiments on ESConv and ExTES report that RLFF-ESC improves Success Rate and Average Turn over baseline methods, including GPT-4o and LLaMA-3.1-405B on ESConv, and wins pairwise human comparisons on response quality. The paper also includes ablations of the reward model design and an analysis of the evaluation threshold.

Significance. The framework is timely and potentially impactful: if the reported numbers are trustworthy, a 7B policy optimized with a future-oriented reward can outperform much larger models on a conversational goal-completion metric, and the use of simulated future trajectories as a reward signal is a clean idea that could transfer to other long-horizon dialogue tasks. The paper's strengths include a clear three-stage pipeline, an ablation study (Table 3) isolating the contribution of the reward model type, and an explicit limitations section that acknowledges the lack of end-user emotional assessment. The main caveat is that the headline goal-completion gains are measured with an LLM-as-judge loop that is structurally similar to the training signal, so the external validity of the central claim is not yet established.

major comments (3)
  1. [§4.2.1, Appendix B, Figure 3] The SR/AT metrics are computed by GPT-4o role-playing both the user simulator and the critic, with 'goal completed' defined as the critic reward exceeding 0.5, while the training reward model in §3.2 was trained on trajectories simulated by Qwen-2.5-72B in the same role-play setup. Because the policy is optimized to maximize that reward model, the reported goal-completion advantage (e.g., 41.5% vs 30.8% for GPT-4o on ESConv) is measured inside the same LLM-simulation paradigm that generated the training signal. Appendix D.3 validates the GPT-4o critic against humans only on 100 existing human-human ESConv dialogues, not on the model-generated evaluation dialogues, and the limitations state that end-user emotional intensity was not assessed. This is a load-bearing validity gap for the 'goal completion' half of the abstract's claim. Please add a human evaluation of goal completion on model-generated dialogues, or demonstrate that the SR/AT gains persist under an independent evaluation protocol (e.g., a different LLM judge family or a held-out judge with a different prompt), and validate the user simulator's realism.
  2. [Algorithm 1, §3.2, §3.3.1] The formula for the future-oriented reward is not correctly typeset: Algorithm 1 line 14 reads 'rsys t,j = rt+k,j + 1 AvgT 2', which is not a well-defined expression. Since this scalar is thresholded to create the binary training labels for the reward model, the training-data construction is not reproducible as printed. Please provide the explicit equation for rsys in terms of r_{t+k,j} and AvgT, and report the threshold δ used to binarize the labels.
  3. [§3.3.1 Eq. (3) vs. Appendix B] Equation (3) describes the reward model as 'a frozen LLaMA model followed by a linear layer', but Appendix B states 'We fully fine-tune LLaMA-3.2-1B as an LLM-based classifier'. These statements are contradictory. Please clarify which components of the reward model are trainable, since this is essential for reproducing the reward model and for interpreting its capacity.
minor comments (6)
  1. [Appendix A.2] The prompt for the user simulator says it is 'the prompt proposed by' but the citation is missing; please add the reference.
  2. [Limitations] The Limitations section contains a duplicated phrase: 'For automatic evaluation, we utilize LLMs, For automatic evaluation, we utilize LLMs because...' Please remove the duplication.
  3. [§4.2.1 Human Evaluation] Table 2 reports pairwise win/lose percentages for five quality dimensions, but no inter-annotator agreement is reported for these judgments; please add agreement statistics (e.g., Krippendorff's alpha) for the human evaluation of response quality.
  4. [§4.2.1, Figure 3] The SR/AT comparisons with larger-scale LLMs are reported as point estimates without confidence intervals or significance tests; please report variance across evaluation runs or a statistical test.
  5. [§3.3.1] There is a typo 'thiking format reward' in the paragraph introducing Rthk-fmt; it should be 'thinking format reward'.
  6. [Figure 6] The axis legend in Figure 6 labels the proposed method as 'RLSF-ESC'; this should be 'RLFF-ESC'.

Circularity Check

1 steps flagged · score 4.0 of 10

Goal-completion gains are measured by the same LLM-critic construct used to train the policy, leaving the headline SR/AT advantage inside a closed LLM-judge loop.

  1. self definitional [Section 3.2/Algorithm 1, Section 3.3.1 (Eq. 5), Section 4.1 Evaluation Details]
    "The critic determines whether the user’s emotional issue has been resolved and assigns an estimated reward accordingly. ... the label yi is derived from the scalar reward r_sys_i using a predefined threshold δ. The label is set to one if r_sys_i is greater than δ; otherwise, it is set to zero. ... The dialogue goal is considered complete if the reward exceeds 0.5."

    The binary label used to train the future-oriented reward model is a thresholded LLM-critic reward (y_i=1 iff r_sys_i>δ), and the evaluation success event (SR) is defined by the same construct: the LLM critic's reward exceeding 0.5. GRPO maximizes the reward model's predicted probability of y_i=1 (Eq. 8), so the policy is optimized toward the very judgment that defines 'goal completion' in the automatic evaluation. Although training simulation uses Qwen-2.5-72B and evaluation uses GPT-4o, both are the same prompt and role-play protocol, so the reported SR/AT advantage is an optimization outcome inside the LLM user-simulator/critic loop rather than independent evidence of enduring emotional improvement.

full rationale

The paper's central derivation (RLFF-ESC = GRPO over a future-oriented reward model trained on LLM-critic labels) is internally consistent and is not a tautology: the reported SR improvement over baselines is an empirical result, and the human preference study (Table 2), ablation study, and agreement analysis provide independent content. However, the automatic goal-completion metric is definitionally tied to the training signal: the reward model's binary label is a thresholded LLM-critic reward, and SR is defined by the same LLM-critic reward exceeding 0.5. The policy is optimized to maximize the predicted probability of that label, so the headline 41.5% versus 30.8% SR advantage over GPT-4o is an optimization outcome inside the LLM user-simulator/critic loop rather than externally validated goal completion. The paper's own Limitations explicitly acknowledge possible LLM evaluation bias and the absence of any assessment of end users' emotional intensity. Because human evaluation addresses response quality but not real-user emotional resolution, the circularity is partial: the relative ranking may be real, but the 'goal completion' label is not independently established. Overall score 4 reflects one load-bearing closed-loop definition while recognizing the substantial independent empirical content in the human judgments and ablations.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The framework does not introduce new physical or conceptual entities. Its load-bearing assumptions are about the validity of LLM-based simulation and LLM-as-judge signals, plus several hand-chosen thresholds and weights that the reported results depend on. The fact that the evaluation success threshold is not a part of the training pipeline, but strongly changes the reported SR, makes it a free parameter in the claim's assessment.

free parameters (4)
  • Evaluation success threshold = reward > 0.5
    Section 4 and Appendix E; changing it to >= 0.5 raises SR from 35.5% to 57.7% on ESConv for LLaMA3.1, so the headline numbers are sensitive to this hand-chosen cutoff.
  • Reward model label threshold delta = not specified
    Section 3.3.1 binarizes the simulated reward into a label using an undefined delta; this threshold affects the reward model's training distribution and is never reported.
  • Future reward composition weights = unclear (terminal reward and 1/AvgT combined)
    Algorithm 1 line 14's formula for rsys_t,j is garbled; the intended mixture of terminal reward and average turns to goal is a hand-designed scalarization that is not reproducible as written.
  • Format reward weight alpha = 0.5
    Appendix B sets alpha = 0.5 in Rrlff = Rfut + alpha * Rthk-fmt; no sensitivity analysis is provided, and this weight shapes the RL objective.
assumptions (3)
  • domain assumption LLM-simulated future dialogues and LLM critic judgments are valid proxies for a real user's enduring emotional improvement.
    Section 3.2 builds the entire reward pipeline on this; Limitations states the study did not assess end users' emotional intensity, so this premise is explicitly unvalidated.
  • domain assumption The binary resolved/not-resolved judgment of the LLM critic agrees well enough with human judgment to serve as a training and evaluation signal.
    Appendix D.3 gives one agreement check (alpha 0.786) on a 100-dialogue sample, but this checks the critic used in evaluation, not the Qwen-72B critic used to generate training labels, and not the trained reward model itself.
  • domain assumption GRPO optimization against a learned scalar reward will improve genuine support quality rather than reward-hack the judge.
    The paper never discusses reward hacking or distribution shift of the LLM judge; the training reward is a classifier trained on a different model's simulations, and the policy could exploit classifier blind spots.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Open-Ended Emotional Support Conversations in LLMs via Reinforcement Learning with Future-Oriented Rewards." pith.science (2026). https://pith.science/paper/MZAV3XGL

@misc{pith2026250812935,
  author       = {Pith},
  title        = {Pith review of: Towards Open-Ended Emotional Support Conversations in LLMs via Reinforcement Learning with Future-Oriented Rewards},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MZAV3XGL}},
  note         = {Machine review of arXiv:2508.12935}
}
read the original abstract

Emotional Support Conversation (ESC) systems aim to alleviate users' emotional difficulties and provide long-term, systematic support for emotional well-being. However, most large language model (LLM)-based ESC systems rely on predefined strategies, which limits their effectiveness in complex, real-life scenarios. To enable flexible responses to diverse emotional problem scenarios, this paper introduces a novel end-to-end framework (RLFF-ESC) that directly learns enduring emotionally supportive response skills using reinforcement learning. For sustained emotional support, we first employ an LLM-based multi-agent mechanism to simulate future dialogue trajectories and collect future-oriented rewards. We then train a future-oriented reward model, which is subsequently used to train the emotional support policy model. Additionally, we incorporate an explicit reasoning process during response generation to further enhance the quality, relevance, and contextual appropriateness of the system's responses. We evaluate the backbone policy model on Qwen2.5-7B-Instruct-1M and LLaMA3.1-8B-Instruct models, testing the proposed RLFF-ESC framework across two public ESC datasets. Experimental results demonstrate that RLFF-ESC consistently outperforms existing baselines in terms of goal completion and response quality.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MICA: Multi-granularity Intertemporal Credit Assignment for Long-Horizon Emotional Support Dialogue

    cs.CL 2026-03 unverdicted novelty 6.0 of 10

    MICA mixes per-turn and whole-trajectory normalized reward signals to train emotional-support chatbots, outperforming GRPO and REINFORCE++ on EMPA, EQ-Bench, and EmoBench.

Reference graph

Works this paper leans on

44 extracted references · 9 canonical work pages · cited by 1 Pith paper

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  2. [2]

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, and 1 others. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862

    arXiv 2022

  3. [3]

    Brant R Burleson. 2003. Emotional support skills. In Handbook of Communication and Social Interaction Skills, pages 569--612. Routledge

    work page 2003

  4. [4]

    Brant R Burleson. 2013. Comforting messages: Features, functions, and outcomes. In Strategic interpersonal communication, pages 135--161. Routledge

    work page 2013

  5. [5]

    Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, and 1 others. 2024. A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology, 15(3):1--45

    2024

  6. [6]

    Maximillian Chen, Xiao Yu, Weiyan Shi, Urvi Awasthi, and Zhou Yu. 2023. https://doi.org/10.18653/v1/2023.acl-short.82 Controllable mixed-initiative dialogue generation through prompting . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 951--966, Toronto, Canada. Association for Com...

    doi:10.18653/v1/2023.acl-short.82 2023

  7. [7]

    Zhuang Chen, Yaru Cao, Guanqun Bi, Jincenzi Wu, Jinfeng Zhou, Xiyao Xiao, Si Chen, Hongning Wang, and Minlie Huang. 2025. Socialsim: Towards socialized simulation of emotional support conversation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 1274--1282

    work page 2025

  8. [8]

    Jiale Cheng, Sahand Sabour, Hao Sun, Zhuang Chen, and Minlie Huang. 2023. https://doi.org/10.18653/v1/2023.findings-acl.34 PAL : Persona-augmented emotional support conversation generation . In Findings of the Association for Computational Linguistics: ACL 2023, pages 535--554, Toronto, Canada. Association for Computational Linguistics

    doi:10.18653/v1/2023.findings-acl.34 2023

Show all 44 references
  1. [9]

    Yi Cheng, Wenge Liu, Wenjie Li, Jiashuo Wang, Ruihui Zhao, Bang Liu, Xiaodan Liang, and Yefeng Zheng. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.195 Improving multi-turn emotional support dialogue generation with lookahead strategy planning . In Proceedings of the 2022 ...

    2022 doi

  2. [10]

    Yang Deng, Wenxuan Zhang, Wai Lam, See-Kiong Ng, and Tat-Seng Chua. 2024. https://openreview.net/forum?id=MCNqgUFTHI Plug-and-play policy planner for large language model powered dialogue agents . In The Twelfth International Conference on Learning Representations

    2024

  3. [11]

    Yang Deng, Wenxuan Zhang, Yifei Yuan, and Wai Lam. 2023. Knowledge-enhanced mixed-initiative dialogue system for emotional support conversations. arXiv preprint arXiv:2305.10172

    2023 arXiv

  4. [12]

    Yao Fu, Hao Peng, Tushar Khot, and Mirella Lapata. 2023. https://arxiv.org/abs/2305.10142 Improving language model negotiation with self-play and in-context learning from ai feedback . Preprint, arXiv:2305.10142

    2023 arXiv

  5. [13]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  6. [14]

    Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, and 1 others. 2024. A survey on llm-as-a-judge. arXiv preprint arXiv:2411.15594

    2024 arXiv

  7. [15]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    2025 arXiv

  8. [16]

    Tao He, Lizi Liao, Yixin Cao, Yuanxing Liu, Ming Liu, Zerui Chen, and Bing Qin. 2024. https://doi.org/10.18653/v1/2024.acl-long.262 Planning like human: A dual-process framework for dialogue planning . In Proceedings of the 62nd Annual Meeting of the Association for Computatio...

    2024 doi

  9. [17]

    Catherine A Heaney and Barbara A Israel. 2008. Social networks and social support. Health Behavior and Health Education: Theory, Research, and Practice, 4(1):189--210

    2008

  10. [18]

    Mengzhao Jia, Qianglong Chen, Liqiang Jing, Dawei Fu, and Renyu Li. 2023. Knowledge-enhanced memory model for emotional support conversation. arXiv preprint arXiv:2310.07700

    2023 arXiv

  11. [19]

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Dong Wang, Hamed Zamani, and Jiawei Han. 2025. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516

    2025 arXiv

  12. [20]

    Dongjin Kang, Sunghwan Kim, Taeyoon Kwon, Seungjun Moon, Hyunsouk Cho, Youngjae Yu, Dongha Lee, and Jinyoung Yeo. 2024. https://doi.org/10.18653/v1/2024.acl-long.813 Can large language models be good emotional supporter? mitigating preference bias on emotional support conversa...

    2024 doi

  13. [21]

    Siyang Liu, Chujie Zheng, Orianna Demasi, Sahand Sabour, Yu Li, Zhou Yu, Yong Jiang, and Minlie Huang. 2021. https://doi.org/10.18653/v1/2021.acl-long.269 Towards emotional support dialog systems . In Proceedings of the 59th Annual Meeting of the Association for Computational ...

    2021 doi

  14. [22]

    Crystal L Park, Laura D Kubzansky, Sandra M Chafouleas, Richard J Davidson, Dacher Keltner, Parisa Parsafar, Yeates Conwell, Michelle Y Martin, Janel Hanmer, and Kuan Hong Wang. 2023. Emotional well-being: What it is and why it matters. Affective Science, 4(1):10--20

    2023

  15. [23]

    Wei Peng, Yue Hu, Luxi Xing, Yuqiang Xie, Yajing Sun, and Yunpeng Li. 2022. Control globally, understand locally: A global-to-local hierarchical graph network for emotional support conversation. arXiv preprint arXiv:2204.12749

    2022 arXiv

  16. [24]

    Huachuan Qiu, Hongliang He, Shuai Zhang, Anqi Li, and Zhenzhong Lan. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.34 SMILE : Single-turn to multi-turn inclusive language expansion via C hat GPT for mental health support . In Findings of the Association for Computation...

    2024 doi

  17. [25]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728--53741

    2023

  18. [26]

    Tazeek Bin Abdur Rakib, Ambuj Mehrish, Lay-Ki Soon, Wern Han Lim, and Soujanya Poria. 2025. Dialogxpert: Driving intelligent and emotion-aware conversations through online value-based reinforcement learning with llm priors. arXiv preprint arXiv:2505.17795

    2025 arXiv

  19. [27]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347

    2017 arXiv

  20. [28]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300

    2024 arXiv

  21. [29]

    Quan Tu, Yanran Li, Jianwei Cui, Bin Wang, Ji-Rong Wen, and Rui Yan. 2022. https://doi.org/10.18653/v1/2022.acl-long.25 MISC : A mixed strategy-aware model integrating COMET for emotional support conversation . In Proceedings of the 60th Annual Meeting of the Association for C...

    2022 doi

  22. [30]

    Chenwei Wan, Matthieu Labeau, and Chloé Clavel. 2025. https://arxiv.org/abs/2408.08782 Emodynamix: Emotional support dialogue strategy prediction by modelling mixed emotions and discourse dynamics . Preprint, arXiv:2408.08782

    2025

  23. [31]

    Xiaoyu Wang, Yue Zhao, Qingqing Gu, Zhonglin Jiang, Xiaokai Chen, Yong Chen, and Luo Ji. 2025. https://arxiv.org/abs/2505.06987 Convert language model into a value-based strategic planner . Preprint, arXiv:2505.06987

    2025 arXiv

  24. [32]

    Tian Xie, Zitian Gao, Qingnan Ren, Haoming Luo, Yuqian Hong, Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, and Chong Luo. 2025. Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning. arXiv preprint arXiv:2502.14768

    2025 arXiv

  25. [33]

    An Yang, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoyan Huang, Jiandong Jiang, Jianhong Tu, Jianwei Zhang, Jingren Zhou, Junyang Lin, Kai Dang, Kexin Yang, Le Yu, Mei Li, Minmin Sun, Qin Zhu, Rui Men, Tao He, and 9 others. 2025. Qwen2.5-1m technical report. arXiv prep...

    2025 arXiv

  26. [34]

    Zihao Yi, Jiarui Ouyang, Yuwen Liu, Tianhao Liao, Zhe Xu, and Ying Shen. 2024. A survey on recent advances in llm-based multi-turn dialogue systems. arXiv preprint arXiv:2402.18013

    2024 arXiv

  27. [35]

    Xiao Yu, Maximillian Chen, and Zhou Yu. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.439 Prompt-based M onte- C arlo tree search for goal-oriented dialogue policy planning . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7...

    2023 doi

  28. [36]

    Hongyi Yuan, Zheng Yuan, Chuanqi Tan, Wei Wang, Songfang Huang, and Fei Huang. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/file/23e6f78bdec844a9f7b6c957de2aae91-Paper-Conference.pdf Rrhf: Rank responses to align language models with human feedback . In Advances...

    2023

  29. [37]

    Qiang Zhang, Jason Naradowsky, and Yusuke Miyao. 2023. https://doi.org/10.18653/v1/2023.findings-acl.417 Ask an expert: Leveraging language models to improve strategic reasoning in goal-oriented dialogue models . In Findings of the Association for Computational Linguistics: AC...

    2023 doi

  30. [38]

    Tenggan Zhang, Xinjie Zhang, Jinming Zhao, Li Zhou, and Qin Jin. 2024. https://doi.org/10.18653/v1/2024.acl-long.723 ESC o T : Towards interpretable emotional support dialogue systems . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics ...

    2024 doi

  31. [39]

    Weixiang Zhao, Xingyu Sui, Xinyang Han, Yang Deng, Yulin Hu, Jiahe Guo, Libo Qin, Qianyun Du, Shijin Wang, Yanyan Zhao, and 1 others. 2025. Chain of strategy optimization makes large language models better emotional supporter. arXiv preprint arXiv:2503.05362

    2025

  32. [40]

    Weixiang Zhao, Yanyan Zhao, Xin Lu, Shilong Wang, Yanpeng Tong, and Bing Qin. 2023 a . Is chatgpt equipped with emotional dialogue capabilities? arXiv preprint arXiv:2304.09582

    2023 arXiv

  33. [41]

    Weixiang Zhao, Yanyan Zhao, Shilong Wang, and Bing Qin. 2023 b . https://doi.org/10.18653/v1/2023.findings-acl.420 T rans ESC : Smoothing emotional support conversation via turn-level state transition . In Findings of the Association for Computational Linguistics: ACL 2023, pa...

    2023 doi

  34. [42]

    Zhonghua Zheng, Lizi Liao, Yang Deng, and Liqiang Nie. 2023. Building emotional support chatbots in the era of llms. arXiv preprint arXiv:2308.11584

    2023 arXiv

  35. [43]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  36. [44]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.