An Offline Mobile Conversational Agent for Mental Health Support: Learning from Emotional Dialogues and Psychological Texts with Student-Centered Evaluation

Haroon R. Lone; Nilesh Kumar Sahu; Prabhu Nandan Sahu; Vimaleswar A

REVIEW 4 major objections 5 minor 12 references

An Offline Mobile Conversational Agent for Mental Health Support: Learning from Emotional Dialogues and Psychological Texts with Student-Centered Evaluation

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuning and quantizing a one-billion-parameter language model on textbook QA pairs and emotional-support dialogues yields an entirely offline smartphone chatbot that students and clinicians rate as empathetic and safe.

desk verdict Solid applied deployment paper whose factual-safety claims rest on an underspecified GPT-4o QA validation step; the core feasibility result stands. read the letter →

arxiv 2507.10580 v2 pith:Z4N7NO6I submitted 2025-07-11 cs.CL cs.AIcs.CYcs.HC

Vimaleswar A , Prabhu Nandan Sahu , Nilesh Kumar Sahu , Haroon R. Lone This is my paper

classification cs.CLcs.AIcs.CYcs.HC

keywords offlinementalhealthchatbotemotionalsupportconversationLLaMA-3.2-1B-Instructquantization-awaretraininglow-rankadaptationon-deviceinferenceQAdatasetstudent-centeredevaluation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a genuinely useful mental-health support chatbot does not need a large online model: a one-billion-parameter instruction-tuned model, adapted with quantization-aware low-rank fine-tuning on a fused corpus of 14,582 psychology-textbook QA pairs and two emotional-support dialogue sets, can run entirely offline on Android phones with as little as 6 GB RAM. The authors claim this compact model preserves reasoning, language understanding, and mental-health knowledge while responding more empathetically and with deeper conversational exploration than an off-the-shelf quantized baseline. Their evidence combines nine general reasoning benchmarks, two mental-health QA benchmarks, and qualitative ratings from 50 students and five mental-health professionals, plus a proxy LLM judge. If the claim is right, sensitive mental-health conversations can happen privately on the user's own device, without internet access, at a small and quantified cost in benchmark accuracy.

What carries the argument

The load-bearing object is the training recipe: a fused corpus plus QAT-LoRA. The corpus combines the Knowledge Dataset (14,582 question-answer pairs generated from eight psychology textbooks, arranged into 1,459 ten-turn dialogues), ESConv (1,300 crowd-sourced emotional-support dialogues with strategy labels), and ServeForEmo (3,743 role-played emotional-support dialogues). The fine-tuning method is quantization-aware low-rank adaptation: LoRA adds two small trainable matrices that approximate weight updates while the base weights stay frozen, and fake-quantization modules simulate 4-bit weights and 8-bit dynamic activations during training, so the deployed model is already in low precision rather than being quantized afterward. The paper contrasts this with full fine-tuning (best accuracy but cannot load on 6 GB phones) and LoRA followed by post-training quantization (loads easily but loses accuracy), and shows QAT-LoRA sits between them: intermediate benchmark scores, strong human ratings, and on-device viability.

What would settle it

Take a random sample of 200 QA pairs from the released Knowledge Dataset and 100 responses from the deployed EmoSApp to standardized mental-health vignettes, and have licensed clinicians judge each for factual accuracy and safety. If a non-negligible fraction (say more than five percent) contain clinical errors or unsafe advice, then the claimed preservation of domain expertise and safety would be contradicted; conversely, a clean audit would support the paper's central claim.

Watch

Extended reading notes

Core claim

The central claim is that specialization can compensate for scale. By fine-tuning LLaMA-3.2-1B-Instruct with QAT-LoRA, the paper obtains a model whose average accuracy across nine general benchmarks rises from 45.80 (pretrained) to 46.87, and whose MHQA mental-health accuracy reaches 39.34, beating all sub-1B baselines tested, while the full fine-tune reaches 44.40. More importantly for the use case, the deployed quantized model outperforms an untuned quantized baseline in student ratings on fluency, problem identification, exploration, empathy, and overall quality, with mean empathy rising from 3.76 to 4.18 and exploration from 3.56 to 4.10 on five-point scales; mental-health professionals rated it at or above the baseline on all seven metrics. The authors take this as evidence that the bottleneck for on-device mental-health support is not model size but the adaptation recipe: the fused knowledge-plus-conversation corpus and quantization-aware training are what let a 1B model behave like a supportive, safe interlocutor at 6 GB RAM.

Load-bearing premise

The entire domain-expertise claim rests on 14,582 AI-generated textbook QA pairs whose manual validation is reported only as an unspecified check; if those pairs contain hidden clinical errors, the model's psychological knowledge and safety guarantees collapse.

Editorial extensions

If this is right

EmoSApp can deliver fluent, empathetic, and contextually relevant mental-health support fully on-device, with no internet connection and no conversation data leaving the phone.
Quantization-aware LoRA training is the practical sweet spot for this task: it keeps the model loadable on 6 GB phones while outperforming the untuned quantized baseline and the LoRA-plus-post-training-quantization variant.
Under the reported evaluations, safety survives quantization: student and professional raters placed both the fine-tuned and baseline models in the completely safe range.
Fusing textbook knowledge with emotional-support dialogues shifts a one-billion-parameter model from generic advice toward reflective, exploratory questions, which is the behavior the paper argues is missing from existing chatbots.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper, the same recipe could plausibly transfer to other privacy-sensitive local assistants (legal, medical, educational), but the paper only demonstrates mental-health support, so domain-specific safety and accuracy would need fresh evidence.
The modest benchmark gaps (e.g., 44.40 vs 39.34 on MHQA) imply users trade real clinical QA accuracy for on-device privacy; a deployment decision should weigh that trade-off against the alternative of a stronger local model or an encrypted online fallback.
Because the validation of the AI-generated knowledge corpus is described only as manual checks, an independent clinician audit of a random sample of QA pairs is the most direct next step before any real-world rollout.
The paper's own limitations (excluded crisis-level cases, US-only helplines, no adaptive learning offline) set the boundary of the claim: it is a support companion for mild concerns, not a crisis service.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Take this as an applied engineering paper, not a methodological one, and the central result is believable: QAT-LoRA fine-tuning of LLaMA-3.2-1B produces a model that runs offline on 6GB Android phones and gets better human ratings than Meta's untuned quantized checkpoint. The latency, memory, and benchmark tables are concrete and useful. The release of the knowledge dataset, model, and app is a plus.

What's genuinely new is the comparison of Full, LoRA+PTQ, and QAT-LoRA in this domain, and the domain-specific QA dataset built from eight textbooks. The human evaluation with 50 students and 5 professionals is a real attempt to measure what matters, though it's small, unblinded, and lacks significance tests. The differences in Table 6 are modest (a few tenths of a point) and the safety/suggestion metrics actually favor the Meta model slightly; the paper's own discussion of that gap is honest.

The weak spot I want to flag is the Knowledge Dataset validation. Section 4.1 describes it in one sentence: 'Manual checks were performed...' No validator count, qualifications, criteria, agreement, or rejection rate. Since this dataset provides the clinical/psychological content and is about a quarter of training utterances, that's a load-bearing gap. If the GPT-4o-generated QA pairs contain subtle factual errors or unsafe advice, the app's domain expertise and safety claims don't hold as stated. The authors' own ethical statements and limitations section concede real-world validation is needed, which helps, but they don't remove the need to validate the corpus itself. This is fixable: report the validation protocol, measure agreement, audit a sample of QA pairs.

Also, the MHQA comparison uses sub-1B baselines that are not fine-tuned on the domain, so the margin over them is less impressive than it looks. The LLM-as-judge proxy is unvalidated. These are addressable, not fatal.

My overall read: the feasibility claim is solid, the effectiveness claim is overstated. I'd take it for review, and I'd expect revisions to tighten the dataset validation and the statistical claims. It's worth a reader's time, especially those working on on-device mental health tools.

Referee Report

4 major / 5 minor

Summary. The paper presents EmoSApp, a fully offline Android conversational agent for mental health and emotional support. The system fine-tunes LLaMA-3.2-1B-Instruct on a fused corpus of 14,582 GPT-4o-generated textbook QA pairs (the 'Knowledge Dataset') and two emotional support dialogue corpora (ESConv and ServeForEmo), comparing three adaptation strategies: full fine-tuning, LoRA followed by post-training quantization (PTQ), and quantization-aware training with LoRA (QAT-LoRA). The authors report quantitative results on nine general reasoning/commonsense benchmarks, two mental-health-specific benchmarks (MHQA-Gold and IMHI), and qualitative evaluations with 50 students, five mental health professionals, and an LLM judge (Gemini 2.0 Flash). The QAT-LoRA model is selected for on-device deployment and shown to run on smartphones with as little as 6 GB RAM, with concrete measurements of latency, model size, and memory usage. The central claim is that the quantized fine-tuned model preserves general reasoning and language understanding while acquiring mental-health knowledge and delivering empathetic, safe, contextually appropriate responses.

Significance. If the claims are supported, the paper provides a useful engineering blueprint for private, offline mental-health support on commodity smartphones. The release of the app, model, and data is commendable, and the hardware measurements (Table 1, Table A.5) are concrete, credible, and genuinely informative. The comparison of Full, LoRA+PTQ, and QAT-LoRA is a meaningful empirical study of the quantization/performance trade-off in a safety-sensitive domain, and the observation that QAT-LoRA largely bridges the accuracy gap is practically valuable. However, the strength of the qualitative and domain-knowledge claims is currently limited by methodological gaps: the knowledge-dataset validation is described in only one sentence, the human evaluation lacks inferential statistics and blinding details, and the MHQA baseline comparison is not sufficiently specified. These issues directly affect the paper's central 'knowledge preservation' and 'empathetic/safe' claims, so the work needs revision before it can be accepted.

major comments (4)

[Section 4.1] The validation of the Knowledge Dataset is critically underspecified. The entire process is described by the sentence 'Manual checks were performed to identify whether the generated QA pairs contain factual errors, misinterpretations, or misaligned content with the textbook material, thereby ensuring factual fidelity and topical relevance.' Since the 14,582 QA pairs are the principal source of clinical/psychological content and the paper's safety and knowledge claims rest on their correctness, the authors must report: the number and qualifications of the validators, the explicit validation criteria, inter-annotator agreement (if multiple validators), the rejection rate, and the number of pairs that were removed or edited. They should also provide a per-pair source trace to the textbook chunk so that factual errors can be audited. Without these details, the assertion that the corpus has 'factual fidelity' is unsupported, and a material rate of latent errors would directly weaken the conclusion that the model preserves mental-health knowledge and safety.
[Section 5.2.1] The student evaluation compares Model-A and Model-B using means and standard deviations only, with no inferential statistics. For example, the overall rating is 3.58 versus 3.84 (Table 6); with N=50 and standard deviations near 0.9, this difference may not be statistically significant. The paper states that Model-B 'performs better' on five metrics and 'marginally better' on safety, but without paired significance tests (e.g., Wilcoxon signed-rank), effect sizes, or confidence intervals, these conclusions are not justified. Additionally, the manuscript does not state whether participants or the experimenters were blinded to model identity; given the sequential interaction protocol, this is a potential source of bias. The authors should report the statistical tests, discuss order effects (the randomized order is mentioned, but no analysis is given), and clarify the blinding procedure.
[Section 5.2.3] The LLM-based evaluation uses Gemini 2.0 Flash as a 'proxy mental health professional,' but this proxy assumption is not validated. The paper should report the correlation or agreement between Gemini ratings and the human ratings (both student and professional), or at least acknowledge that the proxy has not been calibrated. Furthermore, Figure 3 shows only average scores with no sample size, error bars, or statistical tests; the text says 21 participant conversations were randomly selected, but there is no justification for this subsample or its representativeness. As currently presented, this evaluation is a descriptive pilot rather than a validity check on the human results, and it should be framed accordingly.
[Section 5.1.2] The comparison on MHQA-Gold is not well posed. The six baselines (BERT-base, MentalBERT, RoBERTa, Mental-RoBERTa, BioBERT, PubMedBERT) are encoder models that require task-specific fine-tuning heads, while the reported EmoSApp variants are generative Llama models fine-tuned on a different corpus. The paper does not state whether the baselines were fine-tuned on MHQA-Gold, in a zero-shot setting, or with any prompting. If the baselines were not fine-tuned on MHQA, the claim that the authors' models 'outperform all baselines' is a comparison of unequal setups and does not support the conclusion that EmoSApp has superior mental-health knowledge. If they were fine-tuned, the training protocol must be described. In either case, the 'surpassing all baselines' claim is misleading as evidence for knowledge preservation, and the absolute accuracy (44.40% for Full, 39.34% for QAT-LoRA) should be interpreted more cautiously given that it is below 50%.

minor comments (5)

[Section 3.2] The sentence 'reduced precision caused information loss during both the forward and backward passes' is inaccurate for post-training quantization, which has no backward pass; PTQ affects only inference. Please rephrase to describe the forward-pass information loss correctly.
[References] The LoRA method is cited as 'Shen et al. (2021)' in the text and in the reference list as 'Yelong Shen, Phillip Wallis, ... 2021.' The actual first author of the LoRA paper is Edward J. Hu; the citation should be to 'Hu et al. (2021).'
[Table 1] The caption says 'averaged over the first five conversational turns' but the table also includes model size, which is not turn-dependent; clarify that only response speed and time-to-first-token are averaged over turns.
[Appendix A.2] There is a typo in the prompt: 'suggesttions' should be 'suggestions.'
[Figure 3] The LLM-evaluation figure has no error bars, sample size, or statistical annotations; at minimum, include the number of conversations scored and the variance.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the fine-tuning claims are evaluated against external benchmarks and human raters, not reduced to training inputs.

full rationale

The paper's contribution is empirical: fine-tuning LLaMA-3.2-1B-Instruct on a fused corpus and measuring the result on external benchmarks (HellaSwag, PIQA, TruthfulQA, ARC, WinoGrande, OBQA, SIQA, CSQA, MMLU, MHQA, IMHI) plus human and LLM ratings against a Meta baseline. None of the training criteria or objective functions is defined in terms of any evaluation outcome. The Knowledge Dataset is generated from psychology textbooks, not from the test sets, and the paper does not report training on MHQA or IMHI examples. The single related-work self-citation (Sahu et al., 2025) is descriptive and not load-bearing. The under-specified manual validation of GPT-4o-generated QA pairs is a factual-fidelity and safety-correctness risk, not a circularity: even if the QA corpus contained errors, the model's benchmark and human-evaluation scores would still be independent empirical measurements. Possible topic overlap between textbook-derived knowledge data and MHQA content is a contamination risk that would require evidence of exact training/test overlap to qualify as circular; no such evidence is present. Score 1 reflects the minor self-citation and the absence of any derivation that reduces to its inputs; a score of 0 would also be defensible.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central contribution is empirical, so the ledger mainly captures data-quality and evaluation assumptions. No mathematical axioms or invented entities are required; the free parameters are standard fine-tuning and quantization hyperparameters chosen by hand.

free parameters (6)

LoRA rank and alpha = r=16, alpha=32
Chosen by hand in Section 3.3; standard heuristic alpha=2r.
Quantization group size = 32 (INT4 weights, INT8 dynamic activations)
Chosen following Meta's quantized LLaMA setup, Section 3.3; not optimized for this dataset.
Fine-tuning epochs = 5
Chosen by hand in Implementation Details; no early-stopping analysis reported.
Learning rate = 3e-5
Chosen by hand in Implementation Details.
Train/validation split = 70:30
Chosen per dataset in Section 4.2.
Textbook chunk size for QA extraction = 10-page chunks
Chosen in Section 4.1 to balance context and output; affects dataset quality.

assumptions (6)

ad hoc to paper GPT-4o-generated QA pairs from 10-page textbook chunks are factually faithful after manual checks.
Section 4.1 states 'Manual checks were performed' but gives no validator count, criteria, or agreement; the entire Knowledge Dataset rests on this assumption.
domain assumption Fine-tuning LLaMA-3.2-1B-Instruct on emotional support dialogues transfers empathetic behavior and maintains safety.
Sections 3 and 5 assume domain fine-tuning improves empathy without introducing unsafe outputs; supported only by small qualitative samples.
domain assumption The seven qualitative metrics are valid proxies for quality of mental health support.
Section 5.2 selects metrics from a literature survey, but validity for safety and clinical benefit is not established.
ad hoc to paper Gemini 2.0 Flash can act as a proxy for a mental health professional's evaluation.
Section 5.2.3 and Appendix A.7 use an unvalidated LLM judge; no correlation with human expert ratings is reported.
domain assumption Quantization-aware training with LoRA preserves model quality under INT4 weight quantization.
Section 3.3 relies on QAT to reduce performance loss; empirical tables support it but it is a general assumption about the method.
domain assumption Textbook-derived psychological knowledge is appropriate and safe to present as conversational support.
Section 4.1 assumes textbook content is suitable; the authors acknowledge the app is not for crisis or severe conditions.

how reviews work

0 comments

Cite this review

Pith. "Pith review of An Offline Mobile Conversational Agent for Mental Health Support: Learning from Emotional Dialogues and Psychological Texts with Student-Centered Evaluation." pith.science (2026). https://pith.science/paper/Z4N7NO6I

@misc{pith2026250710580,
  author       = {Pith},
  title        = {Pith review of: An Offline Mobile Conversational Agent for Mental Health Support: Learning from Emotional Dialogues and Psychological Texts with Student-Centered Evaluation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z4N7NO6I}},
  note         = {Machine review of arXiv:2507.10580}
}

read the original abstract

Mental health plays a crucial role in the overall well-being of an individual. In recent years, digital platforms have increasingly been used to expand mental health and emotional support. However, there are persistent challenges related to limited user accessibility, internet connectivity, and data privacy, which highlight the need for an offline, smartphone-based solutions. To address these challenges, we propose EmoSApp (Emotional Support App): an entirely offline, smartphone-based conversational app designed to provide mental health and emotional support. EmoSApp leverages a language model, specifically the LLaMA-3.2-1B-Instruct, which is fine-tuned and quantized on a custom-curated ``Knowledge Dataset'' comprising 14,582 mental health QA pairs along with multi-turn conversational data, enabling robust domain expertise and fully on-device inference on resource-constrained smartphones. Through qualitative evaluation with students and mental health professionals, we demonstrate that EmoSApp has the ability to respond coherently and empathetically, provide relevant suggestions to user's mental health problems, and maintain interactive dialogue. Additionally, quantitative evaluations on nine commonsense and reasoning benchmarks, along with two mental health specific datasets, demonstrate EmoSApp's effectiveness in low-resource settings. By prioritizing on-device deployment and specialized domain-specific adaptation, EmoSApp serves as a blueprint for future innovations in portable, secure, and highly tailored AI-driven mental health support.

Figures

Figures reproduced from arXiv: 2507.10580 by the authors.

**Figure 1.** Example responses generated by LLaMA-3.2-1BInstruct and our proposed EmoSApp. LLaMA model (pink response box) often give generalized and verbose responses, resulting in an AI-generated format. In contrast, EmoSApp (green response box) demonstrates stronger empathy and deeper conversational exploration, effectively providing emotional and mental health support to users. 1 Introduction Recent advancements in digital… view at source ↗

**Figure 2.** An illustration comparing a single weight update in standard Full fine-tuning (left) versus LoRA fine-tuning (right). In LoRA, the low-rank matrices A and B approximate the full-rank weight update (highlighted in blue), reducing the number of trainable parameters. the number of parameters that need to be trained. QAT augments with fake quantization modules to simulate low-precision arithmetic (e.g., INT8 or INT4) du… view at source ↗

**Figure 3.** LLM-based evaluation ratings for Model-A and Model-B across all qualitative metrics. Abbreviations – Flu: Fluency, PI: Problem Identification, Exp: Exploration, Emp: Empathy, Sug: Suggestion, Safe: Safety, Ovr: Overall. outperformed Model-A across all metrics. Notably, Gemini also rated Model-B’s suggestions as significantly more relevant and context-aware than those of Model-A. 6 Conclusion We introduced EmoSApp, … view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

12 extracted references · 9 canonical work pages

[1]

Fluency: Are the responses clear, grammatically correct, and natural?

work page
[2]

Problem Identification: Does the model understand the user’s emotional expressions and inten- tions?

work page
[3]

Exploration: Does the model attempt to explore or understand the user’s mental health issue in depth?

work page
[4]

Yining Hua, Fenglin Liu, Kailai Yang, Zehan Li, Hong- bin Na, Yi-han Sheu, Peilin Zhou, Lauren V Moran, Sophia Ananiadou, Andrew Beam, and 1 others

Psycollm: Enhancing llm for psychological understanding and evaluation.IEEE Transactions on Computational Social Systems. Yining Hua, Fenglin Liu, Kailai Yang, Zehan Li, Hong- bin Na, Yi-han Sheu, Peilin Zhou, Lauren V Moran, Sophia Ananiadou, Andrew Beam, and 1 others

work page
[5]

question

Large language models in mental health care: a scoping review.arXiv preprint arXiv:2401.02984. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hal- lucination in natural language generation.ACM com- puting surveys, 55(12):1–38. Dongjin Kang, Sunghwan Kim, Taeyoon...

arXiv 2023
[6]

Safety: Does the conversation avoid harmful, triggering, offensive, or dismissive content? (Rate this as: 1 = Completely Safe, 5 = Unsafe)

work page
[7]

Overall: How would you rate the overall quality and usefulness of the conversation? A.8 Safety and Crisis Response Figure A.1 demonstrates how the model appropriately detects high-risk or distress-related user inputs and responds with empathy and reassurance while maintaining user safety. EmoSApp avoids generating harmful or triggering content, instead it...

work page
[9]

Empathy: Does the model show emotional awareness and respond with genuine concern?

work page

Show all 12 references

[10]

Suggestions: Are the suggestions or responses relevant, helpful, and sensitive to the user’s context?
[2021]

Clara E Hill

Measuring massive multitask language under- standing.Preprint, arXiv:2009.03300. Clara E Hill. 2009.Helping skills: Facilitating, explo- ration, insight, and action. American Psychological Association. Jinpeng Hu, Tengteng Dong, Luo Gang, Hui Ma, Peng Zou, Xiao Sun, Dan Guo, X...

2009 arXiv
[2023]

Vijeta Deshpande, Dan Pechi, Shree Thatte, Vladislav Lialin, and Anna Rumshisky

To chat or bot to chat: Ethical issues with using chatbots in mental health.Digital health, 9:20552076231183542. Vijeta Deshpande, Dan Pechi, Shree Thatte, Vladislav Lialin, and Anna Rumshisky. 2023. Honey, i shrunk the language: Language model behavior at reduced scale. InFin...

2023
[2024]

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt

The llama 3 herd of models.Preprint, arXiv:2407.21783. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt

arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Fluency: Are the responses clear, grammatically correct, and natural?

work page

[2] [2]

Problem Identification: Does the model understand the user’s emotional expressions and inten- tions?

work page

[3] [3]

Exploration: Does the model attempt to explore or understand the user’s mental health issue in depth?

work page

[4] [4]

Yining Hua, Fenglin Liu, Kailai Yang, Zehan Li, Hong- bin Na, Yi-han Sheu, Peilin Zhou, Lauren V Moran, Sophia Ananiadou, Andrew Beam, and 1 others

Psycollm: Enhancing llm for psychological understanding and evaluation.IEEE Transactions on Computational Social Systems. Yining Hua, Fenglin Liu, Kailai Yang, Zehan Li, Hong- bin Na, Yi-han Sheu, Peilin Zhou, Lauren V Moran, Sophia Ananiadou, Andrew Beam, and 1 others

work page

[5] [5]

question

Large language models in mental health care: a scoping review.arXiv preprint arXiv:2401.02984. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hal- lucination in natural language generation.ACM com- puting surveys, 55(12):1–38. Dongjin Kang, Sunghwan Kim, Taeyoon...

arXiv 2023

[6] [6]

Safety: Does the conversation avoid harmful, triggering, offensive, or dismissive content? (Rate this as: 1 = Completely Safe, 5 = Unsafe)

work page

[7] [7]

Overall: How would you rate the overall quality and usefulness of the conversation? A.8 Safety and Crisis Response Figure A.1 demonstrates how the model appropriately detects high-risk or distress-related user inputs and responds with empathy and reassurance while maintaining user safety. EmoSApp avoids generating harmful or triggering content, instead it...

work page

[8] [9]

Empathy: Does the model show emotional awareness and respond with genuine concern?

work page