pith. sign in

arxiv: 2502.19731 · v2 · submitted 2025-02-27 · 💻 cs.CL

Preference Learning Unlocks LLMs' Psycho-Counseling Skills

Mian Zhang , Shaun M. Eack , Zhiyu Zoey Chen This is my paper

Pith reviewed 2026-05-23 03:05 UTC · model grok-4.3

classification 💻 cs.CL
keywords preference learningpsycho-counselinglarge language modelspreference datasetmental healthmodel alignmenttherapist evaluation
0
0 comments X

The pith

Preference learning on a 36k-pair dataset from therapist principles lets an 8B model outperform GPT-4o in psycho-counseling.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that LLMs acquire effective counseling skills when preference optimization is applied to a dataset of 36,000 comparison pairs. These pairs are constructed by applying a set of professional principles that rate how therapists should answer client statements. Real counseling transcripts are hard to obtain because of privacy rules and uneven quality, so the synthetic pairs supply a consistent training signal aligned with expert judgment. If the approach holds, it supplies a practical route to build AI systems that can fill gaps in mental-health support where human therapists are scarce.

Core claim

Defining a set of professional and comprehensive principles for evaluating therapists' responses to client speeches, then using those principles to build the PsychoCounsel-Preference dataset of 36k high-quality comparison pairs, allows preference learning to equip LLMs with the skills needed for psycho-counseling; the resulting PsychoCounsel-Llama3-8B model records an 87 percent win rate against GPT-4o.

What carries the argument

The PsychoCounsel-Preference dataset of 36k high-quality preference comparison pairs generated from a set of professional principles that evaluate therapist responses to client speeches.

If this is right

  • Reward modeling and preference learning on the dataset let LLMs acquire the core skills for generating appropriate replies during counseling sessions.
  • The aligned 8B model reaches an 87 percent win rate over GPT-4o on preference evaluations.
  • The released dataset, reward model, and aligned model supply reusable resources for further psycho-counseling research with LLMs.
  • Training on these pairs produces responses that better handle the variable quality seen in actual therapist transcripts.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same principle-driven preference construction could be repeated in other privacy-sensitive domains where direct high-quality data is unavailable.
  • The evaluation principles themselves might be reused as a checklist for auditing any AI system intended for therapeutic dialogue.
  • Real-world deployment trials could measure whether the learned skills produce measurable changes in client engagement or session outcomes.

Load-bearing premise

The proposed professional principles accurately reflect what practicing psychotherapists prefer, so that models optimized on the resulting pairs actually produce more helpful responses in real counseling sessions.

What would settle it

A blind rating study in which licensed psychotherapists or clients compare model and GPT-4o replies inside live counseling sessions and find no reliable preference for the trained model.

Figures

Figures reproduced from arXiv: 2502.19731 by Mian Zhang, Shaun M. Eack, Zhiyu Zoey Chen.

Figure 1
Figure 1. Figure 1: PsychoCounsel-Preference Construction Pipeline. 1) We first collect over 26k client speeches covering a wide range of topics from various sources, applying necessary data cleaning. 2) 20 popular LLMs are sampled and prompted to roleplay as psychother￾apists and give responses to these client speeches. 3) GPT-4o is instructed to evaluate the responses based on our proposed PsychoCounsel Principles, and pref… view at source ↗
Figure 2
Figure 2. Figure 2: Experts’ Comparison between GPT-4o and PychoChat-Llama3-8B in Two Settings [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Comparison of Training Online or Offline [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Chosen Model Distribution Baichuan2-7B-Chat Orion-14B-Chat OLMo-7B-0724-Instruct-hf Baichuan2-13B-Chat deepseek-llm-67b-chat MiniCPM3-4B o1-mini Phi-3.5-mini-instruct Cohere-command-r-08-2024 Ministral-8B-Instruct-2410 AI21-Jamba-1.5-Mini Qwen2.5-7B-Instruct Mistral-Nemo-Instruct-2407 Llama-3.2-3B-Instruct Llama-3.1-8B-Instruct Llama-3.1-70B-Instruct Qwen2.5-72B-Instruct GPT-4o GPT-4o-mini gemma-2-9b-it 0 … view at source ↗
Figure 6
Figure 6. Figure 6: Rejected Model Distribution 18 [PITH_FULL_IMAGE:figures/full_fig_p018_6.png] view at source ↗
read the original abstract

Applying large language models (LLMs) to assist in psycho-counseling is an emerging and meaningful approach, driven by the significant gap between patient needs and the availability of mental health support. However, current LLMs struggle to consistently provide effective responses to client speeches, largely due to the lack of supervision from high-quality real psycho-counseling data, whose content is typically inaccessible due to client privacy concerns. Furthermore, the quality of therapists' responses in available sessions can vary significantly based on their professional training and experience. Assessing the quality of therapists' responses remains an open challenge. In this work, we address these challenges by first proposing a set of professional and comprehensive principles to evaluate therapists' responses to client speeches. Using these principles, we create a preference dataset, PsychoCounsel-Preference, which contains 36k high-quality preference comparison pairs. This dataset aligns with the preferences of professional psychotherapists, providing a robust foundation for evaluating and improving LLMs in psycho-counseling. Experiments on reward modeling and preference learning demonstrate that PsychoCounsel-Preference is an excellent resource for LLMs to acquire essential skills for responding to clients in a counseling session. Our best-aligned model, PsychoCounsel-Llama3-8B, achieves an impressive win rate of 87% against GPT-4o. We release PsychoCounsel-Preference, PsychoCounsel-Llama3-8B and the reward model PsychoCounsel Llama3-8B-Reward to facilitate the research of psycho-counseling with LLMs at: https://hf.co/Psychotherapy-LLM.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 3 minor

Summary. The paper proposes a set of author-defined principles for evaluating therapist responses to client speech, uses them to construct the 36k-pair PsychoCounsel-Preference dataset, trains a reward model and applies preference optimization to Llama-3-8B yielding PsychoCounsel-Llama3-8B, and claims this model achieves an 87% win rate against GPT-4o. The authors release the dataset, aligned model, and reward model.

Significance. If the win-rate evaluation is shown to be independent of the training rubric and the principles are validated against practicing therapists, the work would provide a useful public resource for preference-based alignment in a high-stakes domain and demonstrate that preference learning can improve LLM counseling responses. The explicit release of the 36k-pair dataset and models is a concrete strength that enables reproducibility and follow-on research.

major comments (3)
  1. [Abstract / Experiments] Abstract and Experiments section: the headline 87% win-rate result against GPT-4o is reported without any description of the evaluation protocol (judge model or human raters, prompt template, whether the judge is prompted with the same author-proposed principles used to build the training pairs, inter-rater agreement, or comparison to the reward model itself). This directly loads on the central claim and leaves open the possibility that the metric is circular.
  2. [Principles / Dataset Construction] Dataset construction / Principles section: no validation is reported that the proposed principles match the preferences of actual professional psychotherapists (e.g., no inter-rater study with licensed clinicians, no comparison to existing counseling rubrics). The entire pipeline (preference pairs, reward model, and downstream win-rate) rests on the untested assumption that these principles are authoritative.
  3. [Experiments] Experiments section: the paper provides no details on the preference-optimization hyperparameters, the exact training split, or any ablation that isolates the contribution of the new dataset versus standard RLHF techniques, making it impossible to assess whether the reported improvement is attributable to the claimed resource.
minor comments (3)
  1. [Reward Modeling] Notation for the reward model (PsychoCounsel Llama3-8B-Reward) is introduced without an explicit equation or training objective in the main text.
  2. [Abstract] The abstract states the dataset 'aligns with the preferences of professional psychotherapists' without a supporting citation or empirical check in the body.
  3. [Results] Figure or table presenting the 87% win rate should include confidence intervals and the exact number of evaluation instances.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback. We address each major comment below and will revise the manuscript to improve transparency and address the identified gaps where feasible.

read point-by-point responses

  1. Referee: [Abstract / Experiments] Abstract and Experiments section: the headline 87% win-rate result against GPT-4o is reported without any description of the evaluation protocol (judge model or human raters, prompt template, whether the judge is prompted with the same author-proposed principles used to build the training pairs, inter-rater agreement, or comparison to the reward model itself). This directly loads on the central claim and leaves open the possibility that the metric is circular.

    Authors: We agree the evaluation protocol requires fuller description. The 87% win rate was computed with GPT-4 as judge using a prompt template that does not reference the author-defined principles. In revision we will add explicit protocol details (judge model, template, independence from training rubric, and any agreement metrics) to the Experiments section and abstract to eliminate ambiguity about circularity. revision: yes

  2. Referee: [Principles / Dataset Construction] Dataset construction / Principles section: no validation is reported that the proposed principles match the preferences of actual professional psychotherapists (e.g., no inter-rater study with licensed clinicians, no comparison to existing counseling rubrics). The entire pipeline (preference pairs, reward model, and downstream win-rate) rests on the untested assumption that these principles are authoritative.

    Authors: The principles were synthesized from established counseling literature; however, we did not perform a new inter-rater validation study with licensed clinicians. We will add an explicit limitations paragraph acknowledging this gap and noting that the public release of the 36k-pair dataset enables independent validation by the community. We view this as a substantive limitation rather than a fatal flaw given the grounding in prior work. revision: partial

  3. Referee: [Experiments] Experiments section: the paper provides no details on the preference-optimization hyperparameters, the exact training split, or any ablation that isolates the contribution of the new dataset versus standard RLHF techniques, making it impossible to assess whether the reported improvement is attributable to the claimed resource.

    Authors: We will expand the Experiments section with the requested hyperparameter values, exact training/validation splits, and additional ablation experiments that compare the new dataset against standard RLHF baselines. This will allow readers to isolate the contribution of PsychoCounsel-Preference. revision: yes

Circularity Check

0 steps flagged

No significant circularity in derivation chain

full rationale

The paper proposes a set of principles, constructs a preference dataset from them, performs reward modeling and preference optimization, and reports an 87% win rate for the resulting model against GPT-4o. No equations, self-citations, or evaluation descriptions in the provided text reduce the win-rate result to the input principles by construction, nor do any steps match the enumerated circularity patterns (no fitted parameter renamed as prediction, no load-bearing self-citation, no imported uniqueness theorem). The central claim remains an empirical outcome from separate experiments rather than a definitional or fitted tautology, making the derivation self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

2 free parameters · 2 axioms · 0 invented entities

Central claim rests on the domain assumption that the custom evaluation principles are valid and that preference optimization on the resulting pairs produces genuinely better counseling behavior. No new physical or mathematical entities are postulated. Free parameters are limited to standard training choices whose specific values are not detailed in the abstract.

free parameters (2)
  • Preference dataset size (36k pairs)
    Chosen scale of constructed data; not derived from first principles.
  • RLHF / preference optimization hyperparameters
    Standard training knobs selected to produce the reported alignment.
axioms (2)
  • domain assumption The proposed principles for evaluating therapist responses are valid and comprehensive.
    Directly invoked when constructing the 36k preference pairs from client speeches.
  • domain assumption Preference learning on pairs derived from these principles transfers to improved real-world counseling performance.
    Required for the claim that the dataset is an excellent resource for acquiring counseling skills.

pith-pipeline@v0.9.0 · 5822 in / 1510 out tokens · 44519 ms · 2026-05-23T03:05:23.191347+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

46 extracted references · 46 canonical work pages · 1 internal anchor

  1. [1]

    Phi-3 technical report: A highly capable language model locally on your phone

    Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Qin Cai, Vishrav Chaudhary, Dong Chen, Dongdong Chen, Weizhu Chen, Yen-Chun Chen, Yi-Ling Chen, Hao Cheng, Parul Chopra, Xiyang Dai, Matt...

    work page 2024

  2. [2]

    Training a helpful and harmless assistant with reinforcement learning from human feedback

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, ...

    work page 2022

  3. [3]

    The use of the area under the ROC curve in the evaluation of machine learning algorithms

    Andrew P Bradley. The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognit., 30 0 (7): 0 1145--1159, July 1997

    work page 1997

  4. [4]

    Orion- 14B : Open-source multilingual large language models

    Du Chen, Yi Huang, Xiaopu Li, Yongqiang Li, Yongqiang Liu, Haihui Pan, Leichao Xu, Dacheng Zhang, Zhipeng Zhang, and Kun Han. Orion- 14B : Open-source multilingual large language models. arXiv [cs.CL], January 2024

    work page 2024

  5. [5]

    LLM -empowered chatbots for psychiatrist and patient simulation: Application and evaluation

    Siyuan Chen, Mengyue Wu, Kenny Q Zhu, Kunyao Lan, Zhiling Zhang, and Lyuchun Cui. LLM -empowered chatbots for psychiatrist and patient simulation: Application and evaluation. arXiv [cs.CL], May 2023 a

    work page 2023

  6. [6]

    Empowering psychotherapy with large language models: Cognitive distortion detection through diagnosis of thought prompting

    Zhiyu Chen, Yujie Lu, and William Yang Wang. Empowering psychotherapy with large language models: Cognitive distortion detection through diagnosis of thought prompting. arXiv [cs.CL], October 2023 b

    work page 2023

  7. [7]

    Challenges of large language models for mental health counseling

    Neo Christopher Chung, George Dyer, and Lennart Brocki. Challenges of large language models for mental health counseling. arXiv [cs.CL], November 2023

    work page 2023

  8. [8]

    UltraFeedback : Boosting language models with high-quality feedback

    Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Wei Zhu, Yuan Ni, Guotong Xie, Zhiyuan Liu, and Maosong Sun. UltraFeedback : Boosting language models with high-quality feedback. arXiv [cs.CL], October 2023

    work page 2023

  9. [9]

    DeepSeek LLM : Scaling open-source language models with longtermism

    DeepSeek-AI , Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, Huazuo Gao, Kaige Gao, Wenjun Gao, Ruiqi Ge, Kang Guan, Daya Guo, Jianzhong Guo, Guangbo Hao, Zhewen Hao, Ying He, Wenjie Hu, Panpan Huang, Erhang Li, Guowei Li, Jiashi Li, Yao Li, Y K Li, Wenfeng Liang, Fangyun Lin, A X Liu...

    work page 2024

  10. [10]

    DeepSeek - R1 : Incentivizing reasoning capability in LLMs via reinforcement learning

    DeepSeek-AI , Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z F Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Da...

    work page 2025

  11. [11]

    Gemma 2: Improving open language models at a practical size

    Gemma Team . Gemma 2: Improving open language models at a practical size. arXiv [cs.CL], July 2024

    work page 2024

  12. [12]

    MiniCPM : Unveiling the potential of small language models with scalable training strategies

    Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Weilin Zhao, Xinrong Zhang, Zheng Leng Thai, Kaihuo Zhang, Chongyi Wang, Yuan Yao, Chenyang Zhao, Jie Zhou, Jie Cai, Zhongwu Zhai, Ning Ding, Chao Jia, Guoyang Zeng, Dahai Li, Zhiyuan Liu, and Maosong Sun. MiniCPM : Unveiling the potential of small lang...

    work page 2024

  13. [13]

    Jamba-1.5: Hybrid transformer-mamba models at scale

    Jamba Team , Barak Lenz, Alan Arazi, Amir Bergman, Avshalom Manevich, Barak Peleg, Ben Aviram, Chen Almagor, Clara Fridman, Dan Padnos, Daniel Gissin, Daniel Jannai, Dor Muhlgay, Dor Zimberg, Edden M Gerber, Elad Dolev, Eran Krakovsky, Erez Safahi, Erez Schwartz, Gal Cohen, Gal Shachaf, Haim Rozenblum, Hofit Bata, Ido Blass, Inbal Magar, Itay Dalmedigos, ...

    work page 2024

  14. [14]

    RewardBench : Evaluating reward models for language modeling

    Nathan Lambert, Valentina Pyatkin, Jacob Morrison, L J Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A Smith, and Hannaneh Hajishirzi. RewardBench : Evaluating reward models for language modeling. arXiv [cs.LG], March 2024

    work page 2024

  15. [15]

    MentalAgora : A gateway to advanced personalized care in mental health through multi-agent debating and attribute control

    Yeonji Lee, Sangjun Park, Kyunghyun Cho, and Jinyeong Bak. MentalAgora : A gateway to advanced personalized care in mental health through multi-agent debating and attribute control. arXiv [cs.CL], July 2024

    work page 2024

  16. [16]

    Skywork-reward: Bag of tricks for reward modeling in LLMs

    Chris Yuhao Liu, Liang Zeng, Jiacai Liu, Rui Yan, Jujie He, Chaojie Wang, Shuicheng Yan, Yang Liu, and Yahui Zhou. Skywork-reward: Bag of tricks for reward modeling in LLMs . arXiv [cs.AI], October 2024

    work page 2024

  17. [17]

    ChatCounselor : A large language models for mental health support

    June M Liu, Donghao Li, He Cao, Tianhe Ren, Zeyi Liao, and Jiamin Wu. ChatCounselor : A large language models for mental health support. arXiv [cs.CL], September 2023

    work page 2023

  18. [18]

    The llama 3 herd of models

    Llama Team . The llama 3 herd of models. arXiv [cs.AI], July 2024

    work page 2024

  19. [19]

    Training models to generate, recognize, and reframe unhelpful thoughts

    Mounica Maddela, Megan Ung, Jing Xu, Andrea Madotto, Heather Foran, and Y-Lan Boureau. Training models to generate, recognize, and reframe unhelpful thoughts. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Stroudsburg, PA, USA, 2023. Association for Computational Linguistics

    work page 2023

  20. [20]

    Beyond training objectives: Interpreting reward model divergence in large language models

    Luke Marks, Amir Abdullah, Clement Neo, Rauno Arike, Philip Torr, and Fazl Barez. Beyond training objectives: Interpreting reward model divergence in large language models. arXiv [cs.LG], October 2023

    work page 2023

  21. [21]

    Motivational interviewing: Helping people change

    Miller and William Stephen Rollnick. Motivational interviewing: Helping people change

    work page

  22. [22]

    OLMoE : Open mixture-of-experts language models

    Niklas Muennighoff, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Jacob Morrison, Sewon Min, Weijia Shi, Pete Walsh, Oyvind Tafjord, Nathan Lambert, Yuling Gu, Shane Arora, Akshita Bhagia, Dustin Schwenk, David Wadden, Alexander Wettig, Binyuan Hui, Tim Dettmers, Douwe Kiela, Ali Farhadi, Noah A Smith, Pang Wei Koh, Amanpreet Singh, and Hannaneh Hajishirzi. OL...

    work page 2024

  23. [23]

    A survey of large language models in psychotherapy: Current landscape and future directions

    Hongbin Na, Yining Hua, Zimu Wang, Tao Shen, Beibei Yu, Lilin Wang, Wei Wang, John Torous, and Ling Chen. A survey of large language models in psychotherapy: Current landscape and future directions. arXiv [cs.CL], February 2025

    work page 2025

  24. [24]

    Obtaining well calibrated probabilities using bayesian binning

    Mahdi Pakdaman Naeini, Gregory F Cooper, and Milos Hauskrecht. Obtaining well calibrated probabilities using bayesian binning. Proc. Conf. AAAI Artif. Intell., 2015: 0 2901--2907, January 2015

    work page 2015

  25. [25]

    GPT - 4o system card

    OpenAI . GPT - 4o system card. arXiv [cs.CL], October 2024

    work page 2024

  26. [26]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke E Miller, Maddie Simens, Amanda Askell, P Welinder, P Christiano, J Leike, and Ryan J Lowe. Training language models to follow instructions with human feedback. Adv. ...

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  27. [27]

    Iterative reasoning preference optimization

    Richard Yuanzhe Pang, Weizhe Yuan, Kyunghyun Cho, He He, Sainbayar Sukhbaatar, and Jason Weston. Iterative reasoning preference optimization. arXiv [cs.CL], April 2024

    work page 2024

  28. [28]

    Qwen2 .5 technical report

    Qwen , An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, T...

    work page 2024

  29. [29]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. arXiv [cs.LG], May 2023

    work page 2023

  30. [30]

    Scaling laws for reward model overoptimization in direct alignment algorithms, 2024

    Rafael Rafailov, Yaswanth Chittepu, Ryan Park, Harshit Sikchi, Joey Hejna, Bradley Knox, Chelsea Finn, and Scott Niekum. Scaling laws for reward model overoptimization in direct alignment algorithms, 2024

    work page 2024

  31. [31]

    Key factors in psychotherapy training: an analysis of trainers’, trainees’ and psychotherapists’ points of view

    Diego Rocco, Alessandro Gennaro, Lorena Filugelli, Patrizia Squarcina, and Elena Antonelli. Key factors in psychotherapy training: an analysis of trainers’, trainees’ and psychotherapists’ points of view. Res. Psychother. Psychopathol. Process Outcome, 22 0 (3), December 2019

    work page 2019

  32. [32]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv [cs.LG], July 2017

    work page 2017

  33. [33]

    Facilitating self-guided mental health interventions through human-language model interaction: A case study of cognitive restructuring

    Ashish Sharma, Kevin Rushton, Inna Wanyin Lin, Theresa Nguyen, and Tim Althoff. Facilitating self-guided mental health interventions through human-language model interaction: A case study of cognitive restructuring. In Proceedings of the CHI Conference on Human Factors in Computing Systems, volume 21, pp.\ 1--29, New York, NY, USA, May 2024. ACM

    work page 2024

  34. [34]

    Detecting cognitive distortions from patient-therapist interactions

    Sagarika Shreevastava and Peter Foltz. Detecting cognitive distortions from patient-therapist interactions. In Proceedings of the Seventh Workshop on Computational Linguistics and Clinical Psychology: Improving Access, Stroudsburg, PA, USA, 2021. Association for Computational Linguistics

    work page 2021

  35. [35]

    Large language models could change the future of behavioral healthcare: a proposal for responsible development and evaluation

    Elizabeth C Stade, Shannon Wiltsey Stirman, Lyle H Ungar, Cody L Boland, H Andrew Schwartz, David B Yaden, João Sedoc, Robert J DeRubeis, Robb Willer, and Johannes C Eichstaedt. Large language models could change the future of behavioral healthcare: a proposal for responsible development and evaluation. Npj Ment Health Res, 3 0 (1): 0 12, April 2024

    work page 2024

  36. [36]

    Understanding the performance gap between online and offline alignment algorithms

    Yunhao Tang, Daniel Zhaohan Guo, Zeyu Zheng, Daniele Calandriello, Yuan Cao, Eugene Tarassov, Rémi Munos, Bernardo Ávila Pires, Michal Valko, Yong Cheng, and Will Dabney. Understanding the performance gap between online and offline alignment algorithms. arXiv [cs.LG], May 2024

    work page 2024

  37. [37]

    HelpSteer2 -preference: Complementing ratings with preferences

    Zhilin Wang, Alexander Bukharin, Olivier Delalleau, Daniel Egert, Gerald Shen, Jiaqi Zeng, Oleksii Kuchaiev, and Yi Dong. HelpSteer2 -preference: Complementing ratings with preferences. arXiv [cs.LG], October 2024

    work page 2024

  38. [38]

    Is DPO superior to PPO for LLM alignment? a comprehensive study

    Shusheng Xu, Wei Fu, Jiaxuan Gao, Wenjie Ye, Weilin Liu, Zhiyu Mei, Guangju Wang, Chao Yu, and Yi Wu. Is DPO superior to PPO for LLM alignment? a comprehensive study. arXiv [cs.CL], April 2024

    work page 2024

  39. [39]

    Baichuan 2: Open large-scale language models

    Aiyuan Yang, Bin Xiao, Bingning Wang, Borong Zhang, Ce Bian, Chao Yin, Chenxu Lv, Da Pan, Dian Wang, Dong Yan, Fan Yang, Fei Deng, Feng Wang, Feng Liu, Guangwei Ai, Guosheng Dong, Haizhou Zhao, Hang Xu, Haoze Sun, Hongda Zhang, Hui Liu, Jiaming Ji, Jian Xie, Juntao Dai, Kun Fang, Lei Su, Liang Song, Lifeng Liu, Liyun Ru, Luyao Ma, Mang Wang, Mickel Liu, M...

    work page 2023

  40. [40]

    Qwen2 technical report

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei...

    work page 2024

  41. [41]

    CBT -bench: Evaluating large language models on assisting cognitive behavior therapy

    Mian Zhang, Xianjun Yang, Xinlu Zhang, Travis Labrum, Jamie C Chiu, Shaun M Eack, Fei Fang, William Yang Wang, and Zhiyu Zoey Chen. CBT -bench: Evaluating large language models on assisting cognitive behavior therapy. arXiv [cs.CL], October 2024

    work page 2024

  42. [42]

    Judging LLM -as-a-judge with MT -bench and chatbot arena

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P Xing, Hao Zhang, Joseph E Gonzalez, and Ion Stoica. Judging LLM -as-a-judge with MT -bench and chatbot arena. arXiv [cs.CL], June 2023

    work page 2023

  43. [43]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

    work page

  44. [44]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

    work page

  45. [45]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

    work page

  46. [46]

    0362 #1 ^H 2

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

    work page arXiv 2025