Pith. sign in

REVIEW 4 major objections 5 minor 14 references

Psychological Counseling Cannot Be Achieved Overnight: Automated Psychological Counseling Through Multi-Session Conversations

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A multi-session dataset and memory-guided model let an LLM counselor track client progress and produce sustained mood improvement over six simulated sessions.

desk verdict Useful new multi-session counseling dataset and three-task model, but the headline 'long-term improvement' claim rests entirely on a closed GPT-4o loop and needs human outcome validation before it can be cited as a counseling effect. read the letter →

arxiv 2506.06626 v1 pith:P2VYJXQ4 submitted 2025-06-07 cs.CL

Junzhe Wang , Bichen Wang , Xing Fu , Yixin Sun , Yanyan Zhao , Bing Qin This is my paper
classification cs.CL
keywords multi-sessionpsychologicalcounselingcognitivebehavioraltherapyLLMdatasettherapeuticmemorygoalplanningworkingalliancePANASsimulatedclientevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Psychological counseling is a process that unfolds over multiple sessions, and this paper argues automated systems should be built and evaluated that way. The authors create MusPsy-Dataset, a collection of 1,400 client cases averaging 6.17 sessions each, generated from real case-report profiles and CBT goals, and train MusPsy-Model with session-level memory and next-session goal planning. In six-session simulated counseling, the model lowers the client's negative affect and raises positive affect over time, while single-session baselines stall or repeat. If the result holds, it shifts the bar for automated counseling from one-shot empathy to longitudinal tracking of client progress.

What carries the argument

The load-bearing mechanism is a structured external memory and the three-task loop built around it. After each session, a Memory Extraction task condenses the dialogue into client profiles, counseling notes (counselor observations, session goals, assignments), and a counseling summary; a Goal Planning task then proposes the next session's CBT goal from that memory; and the Counseling Generation task produces the next dialogue conditioned on the memory and goal. On the data side, the enabling construction step is 'seed conversations': the authors generate short coherent multi-session dialogues in one pass and then expand each seed into a full session, which is what keeps the generated corpus coherent across sessions. This memory-plus-goal loop is what distinguishes MusPsy-Model from single-session baselines and is claimed to carry the longitudinal improvement.

What would settle it

Run the same six-session protocol with a simulated client explicitly instructed to resist the counselor's reframing and to report emotions honestly; if PANAS and WAI still climb steadily, the gains are likely artifacts of role-play. A complementary check is to have blinded human raters score transcripts for whether the client's reported improvement is grounded in concrete behavioral changes mentioned in the dialogue.

Watch

Extended reading notes

Core claim

The paper's central claim is that the multi-session setting is not just an add-on but the defining condition of realistic automated counseling, and that a model explicitly built for it outperforms single-session systems over time. The authors construct MusPsy-Dataset from real client profiles and session goals in public CBT case reports, generating 1,400 clients with an average of 6.17 sessions each, then train MusPsy-Model to extract session memories, plan next-session goals, and generate counseling conditioned on both. In six-session simulations where an LLM plays the client, the model reduces negative affect and raises positive affect and working-alliance scores across sessions, whereas single-session baselines become repetitive after two sessions. The authors also show that the dataset alone contributes part of the gain: a version of the model using only generation, without memory and goal planning, still beats some baselines but cannot match the full multi-session system.

Load-bearing premise

The effectiveness result stands on the assumption that an LLM prompted to act as a client and then rate its own affect, together with GPT-4o's alliance ratings, captures genuine therapeutic change rather than prompt-driven compliance or stylistic matching to the training data; the paper offers no human-client outcome measure for the model.

Editorial extensions

If this is right

  • Multi-session counseling models can be trained to make working alliance grow over sessions instead of flatlining, mirroring the expected therapy trajectory.
  • A counselor LLM that keeps session-level memory and plans next-session goals can continue addressing a client's issues past session two, where single-session baselines start repeating themselves.
  • Evaluation of automated counseling should include longitudinal measures such as PANAS and WAI over multiple simulated sessions, not only single-turn quality.
  • The ablation shows that dataset quality and architecture both matter: the multi-session data alone lifts performance above some baselines, but the memory-and-goal design is needed for sustained long-term gains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same memory-extraction and goal-planning loop could serve as a general longitudinal case-management pattern for other helping conversations—tutoring, coaching, chronic-care follow-up—where the system must remember progress and re-plan each encounter.
  • Because the simulated-client PANAS protocol is cheap, it could be turned into a screening test for counseling models before human trials, provided it is first validated against human-client outcomes.
  • The case-report-to-dataset pipeline suggests that anonymized session-level case documentation from other therapeutic schools could generate multi-session corpora without full transcripts, broadening beyond CBT.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a multi-session paradigm for automated psychological counseling. It introduces MusPsy-Dataset, constructed from public case reports by extracting client profiles and counseling goals, generating short seed conversations, expanding them into complete sessions, and producing structured memories, and MusPsy-Model, a Llama-3-8B-Instruct model fine-tuned jointly for memory extraction, goal planning, and counseling generation. The central empirical claim is that the model outperforms single-session baselines across sessions and produces long-term improvement in simulated clients' positive and negative affect, measured by GPT-4o-rated WAI scores and an LLM-simulated client's PANAS ratings.

Significance. If validated, the dataset and modeling tasks address a genuine gap: most counseling datasets are single-session despite counseling in practice being longitudinal. The construction pipeline is thoughtful, with grounding in case reports, explicit session goals, and a memory module, and the dataset receives positive human expert ratings in Table 2. The weakness is that the model-effectiveness claim rests entirely on an unvalidated closed-loop LLM evaluation in which GPT-4o generates the data, plays the client, and judges the outputs, so the central contribution is not yet established. The dataset itself may still be a useful resource for the community.

major comments (4)
  1. [6.3, Appendix G (Figures 13, 18, 19)] The central claim that MusPsy-Model 'effectively reduces negative emotions and enhances positive emotions in the long term' is supported only by a closed GPT-4o loop. GPT-4o generates the training data, simulates the client, infers the client's PANAS, and rates WAI; no human evaluation of model-generated outputs is reported. Because the simulated client is prompted to update its internal dynamic state after each session, the positive PANAS trajectory may reflect the simulation protocol rather than counselor skill. The Limitations paragraph acknowledges LLM-evaluation bias, but the abstract and Section 6.3.1 still state the effectiveness result without this caveat. This is load-bearing: the model comparison and the abstract's 'better than baseline' claim depend on these measures.
  2. [6.3, Appendix G] The outcome measures are not validated for this setting. PANAS and WAI are psychometric instruments designed for human respondents and human raters; an LLM prompted to 'play the role of the user' and then asked to rate its own emotions (Figure 19) has no established validity, and no comparison against human PANAS or WAI ratings on the same transcripts is provided. A cross-family LLM judge or human ratings would be needed to rule out prompt-induced compliance and in-distribution style matching with the GPT-4o-generated data.
  3. [Figure 6, Section 6.3.1] The plots show mean trajectories over 100 test clients without error bars, confidence intervals, or significance tests, yet the text claims the model 'significantly alleviates' negative emotions and shows a 'long-term upward trend.' With 100 clients, inferential statistics are feasible, and without them the apparent separation from baselines may be driven by a few extreme clients or by the closed-loop protocol.
  4. [6.1, Figure 6, Appendix H] The baseline comparison is confounded by training data. Baselines are fine-tuned on SMILE, CACTUS, SimPsyDial, and CPsyCoun, which are single-session or multi-turn datasets but not multi-session counseling corpora, so their inferior multi-session performance may reflect dataset mismatch rather than the MusPsy-Model's multi-session design. Appendix H's MusPsy(single) control is a partial remedy, but it is evaluated with the same unvalidated loop and without statistical tests.
minor comments (5)
  1. [4.3] The qualifications of the 'two psychology experts' are described only in the Ethical Statement as a bachelor's degree and two years of relevant experience; the main text should state their credentials and report inter-annotator agreement.
  2. [Figure 6] The PANAS panels use a session axis starting at 0 while the WAI panels start at 1; the axes should be made consistent or the difference should be explained.
  3. [Table 3] Task 1 and Task 2 report BLEU and F1 scores without any baseline, so values such as BLEU-1 42.7 for memory extraction are difficult to interpret; adding a simple baseline or reference model would help.
  4. [Introduction, References] The introduction cites 'Organization, 2024' for rising mental-health burden, but the reference list entry is 'Global tuberculosis report 2024'; the intended WHO mental health report should be cited.
  5. [Throughout] There are typos and inconsistent terminology throughout, e.g., 'counseling' versus 'counseling,' 'The construction of our dataset involves' with an initial capital mid-sentence, and inconsistent capitalization of MusPsy.

Circularity Check

1 steps flagged · score 7.0 of 10

Central 'long-term improvement' claim rests on a closed GPT-4o loop: the training data generator, the simulated client, and the PANAS/WAI judges are the same model family, so the measured effect is a self-consistency check rather than an independent outcome.

  1. other [Section 6.3 and Appendix G (Figures 18-19)]
    "In these simulations, the LLM acts as a client interacting with the counselor model. ... After the session, we ask it to evaluate its PANAS score to analyze the counseling model's effectiveness. It updates its own state and prepares for the next counseling conversation. ... For these evaluations, we utilize GPT-4o."

    The outcome used to support the claim that the model 'effectively reduces negative emotions and enhances positive emotions in the long term' is produced by the same model family that generated the training dialogues and that plays the client in the simulation. The counselor model is fine-tuned on GPT-4o-generated dialogues; the 'client' is a GPT-4o prompt instructed to update its internal state after each session; the PANAS score is GPT-4o's inference about that simulated client; the WAI scores are also GPT-4o judgments. The measured improvement is therefore a closed-loop consistency check between the trained model and the generator/judge, not an externally validated counseling outcome.

full rationale

The dataset construction itself is not circular: MusPsy-Dataset is assembled from public case reports with human validation, and the dataset-quality comparisons use human expert ratings in addition to GPT-4o. The central problem is that the model's behavioral claim—the abstract's 'better than baseline models across multiple sessions' and Section 6.3.1's 'effectively reduces negative emotions and enhances positive emotions in the long term'—is evaluated entirely inside a GPT-4o loop. GPT-4o generates the training dialogues from case reports whose dynamic states already document improvement; GPT-4o plays the client and updates its own internal state after each session; GPT-4o scores the resulting PANAS trajectory and WAI ratings. Thus the trained Llama model is rewarded for imitating the GPT-4o-generated counseling style, and the evaluation judges that imitation using the same model's priors. The observed upward PANAS/WAI trends are therefore a self-consistency check, not independent evidence of therapeutic effectiveness. No self-citation chain or imported uniqueness theorem is present; the circularity is in the evaluation design. Because this loop is the load-bearing evidence for the central performance claim, the paper receives a score of 7 rather than a lower score for minor self-citation.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central result depends on four domain assumptions, two of which are ad hoc to the paper's evaluation. No free parameters or invented entities are involved; the contribution rests on a chain of synthetic-data validity assumptions.

assumptions (4)
  • domain assumption CBT counseling is inherently multi-session and progress across sessions is a valid target for automation.
    Intro and §2.1 ground the work in the multi-session nature of CBT; if single-session sufficed, the dataset would be unnecessary.
  • domain assumption Public psychological case reports, though lacking full transcripts, contain enough session-by-session state and goals to reconstruct realistic counseling.
    §3.1: profiles and goals are extracted from case reports with GPT-4o; the paper acknowledges reports exclude full transcripts.
  • ad hoc to paper LLM-generated expansions of seed conversations faithfully represent real counseling practice and client responses.
    §3.3 generates full sessions from seeds; the authors' Limitations section notes realism concerns.
  • ad hoc to paper An LLM prompted to role-play a client can provide a valid self-report of emotional state (PANAS) and the LLM judge's WAI scores reflect the true working alliance.
    §6.3 and Appendix G: improvement is measured via LLM client self-report and GPT-4o ratings, without human validation of model outputs.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Psychological Counseling Cannot Be Achieved Overnight: Automated Psychological Counseling Through Multi-Session Conversations." pith.science (2026). https://pith.science/paper/P2VYJXQ4

@misc{pith2026250606626,
  author       = {Pith},
  title        = {Pith review of: Psychological Counseling Cannot Be Achieved Overnight: Automated Psychological Counseling Through Multi-Session Conversations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P2VYJXQ4}},
  note         = {Machine review of arXiv:2506.06626}
}
read the original abstract

In recent years, Large Language Models (LLMs) have made significant progress in automated psychological counseling. However, current research focuses on single-session counseling, which doesn't represent real-world scenarios. In practice, psychological counseling is a process, not a one-time event, requiring sustained, multi-session engagement to progressively address clients' issues. To overcome this limitation, we introduce a dataset for Multi-Session Psychological Counseling Conversation Dataset (MusPsy-Dataset). Our MusPsy-Dataset is constructed using real client profiles from publicly available psychological case reports. It captures the dynamic arc of counseling, encompassing multiple progressive counseling conversations from the same client across different sessions. Leveraging our dataset, we also developed our MusPsy-Model, which aims to track client progress and adapt its counseling direction over time. Experiments show that our model performs better than baseline models across multiple sessions.

Figures

Figures reproduced from arXiv: 2506.06626 by the authors.

Figure 1
Figure 1. An real example of multi-session counseling [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. This figure illustrates the process of constructing the MusPsy-Dataset. It includes four parts: Client [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Multi-session CBT generally follows a paradigm in which the counselor typically adheres to this process, integrating it with the client’s actual situa￾tion to set more specific and actionable counseling goals and use counseling techniques (Beck et al., 2011). At the end of each counseling session, we sum￾marize the client’s conversation into this memory structure. This approach allows the model to up￾date and retain… view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Evaluation of WAI scores for each counsel [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Overview of Our Three Tasks: Illustrating the [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Emotional changes of the LLM client and performance changes of the LLM counselor across multiple [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 8
Figure 8. Figure 8: Prompt for constructing seed conversations [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 7
Figure 7. Figure 7: Prompt for extracting client profiles and coun [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 11
Figure 11. Figure 11: Prompt for constructing counselor’s memory [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]
Figure 10
Figure 10. Figure 10: Prompt for constructing complete counsel [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 12
Figure 12. Figure 12: The prompt used for to evaluate score. The following counseling session reflects a dialogue between the client and the counselor. Please evaluate how the client might respond to the questions below after the session by selecting the appropriate scores from the provide…
Figure 15
Figure 15. Figure 15: The prompt used for task 2 (Goal Planning). [PITH_FULL_IMAGE:figures/full_fig_p013_15.png]
Figure 16
Figure 16. Figure 16: The prompt used for task 3 (Counseling Generation). G LLM Client Construction To ensure a fair and direct comparison of the coun￾seling models, we maintain a consistent simulated client from the initial session. Recognizing that the language environment of the model c…
Figure 17
Figure 17. Figure 17: Emotional changes of the LLM client and performance changes of the LLM counselor across multiple [PITH_FULL_IMAGE:figures/full_fig_p014_17.png]
Figure 19
Figure 19. Figure 19: The prompt used to evaluate PANAS score. [PITH_FULL_IMAGE:figures/full_fig_p014_19.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

14 extracted references · 12 canonical work pages

  1. [1]

    As a result of these sessions I am clearer as to how I might be able to change

    work page

  2. [2]

    What I am doing in therapy gives me new ways of looking at my problem

    work page

  3. [3]

    I believe counselor likes me

    work page

  4. [4]

    counselor and I collaborate on setting goals for my therapy

    work page

  5. [5]

    counselor and I respect each other

    work page

  6. [6]

    counselor and I are working towards mutually agreed upon goals

    work page

  7. [7]

    I feel that counselor appreciates me

    work page

  8. [8]

    counselor and I agree on what is important for me to work on

    work page

Show all 14 references
  1. [9]

    I feel counselor cares about me even when I do things that he/she does not approve of

  2. [10]

    I feel that the things I do in therapy will help me to accomplish the changes that I want

  3. [11]

    counselor and I have established a good understanding of the kind of changes that would be good for me

  4. [12]

    I believe the way we are working with my problem is correct. [Rating Scale] 1: Seldom 2: Sometimes 3: Fairly Often 4: Very Often 5: Always [Response Format] Question number: Score [Below is the history of the counseling dialogue] {diag} Prompt Figure 13: The prompt used for to...

  5. [2014]

    client-centered

    Digital cognitive behaviour therapy (cbt) in canada: Ethical considerations.Canadian Psychol- ogy/Psychologie canadienne, 55(4):231. Suyeon Lee, Sunghwan Mac Kim, Minju Kim, Dongjin Kang, Dongil Yang, Harim Kim, Minseok Kang, Dayi Jung, Min Kim, Seungbeen Lee, et al. 2024. Cac...

    2024 arXiv

  6. [2024]

    Judith S Beck

    Content and attainment of individual treatment goals in cbt.Psychotherapy Research, 34(1):111– 123. Judith S Beck. 2020.Cognitive behavior therapy: Ba- sics and beyond. Guilford Publications. Judith S Beck, AT Beck, and JS Beck. 2011. Cognitive behavior therapy: basics and bey...

    2020 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.