{"id":"fc8c0dcb-b72f-4d0d-a811-a3ddb4ef3035","arxiv_id":"2607.23648","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A seeker-emotion-trajectory framework with schemas and EFT counselor control yields a 1,114-dialogue corpus and a fine-tuned model that score higher on emotional richness and empathy than prior counseling datasets and bots.","lead":"EmoTrace builds multi-turn counseling dialogues by steering a simulated seeker’s emotional path through three stages, emotional schemas, and an EFT-style counselor. It aims to fix flat, overly compliant synthetic seekers so fine-tuned models handle unstable emotion better.","discovery_kind":"new_method","skeptic_critique":{"model":"moonshotai/kimi-k3","headline":"The headline \"emotional richness\" gains are scored on a rubric (EmoTrace-E) whose dimensions are exactly the properties the ETC controller forces and the corpus filter selects for, so Table 2's advantage is partly tautological rather than independently demonstrated.","rationale":"The reader's weakest_assumption identifies the same soft spot — synthetic generation steered and filtered toward trajectory properties, then ranked by judges on those same properties — so I agree with their diagnosis. I sharpen it in two ways the reader only gestured at: (1) the circularity is not just judge/generator overlap but a direct pipeline-to-rubric identity (filter criterion in §3.3 = EmoTrace-E dimensions in Fig. 15; ETC forced stage rules = the VA fluctuation pattern in Figs. 3–4), and (2) the model evaluation seeker in §5.2 is in-distribution relative to training, weakening the \"handles unstable seekers\" claim. These are correctness-risk issues, not mere consensus deviation, and they are exactly why the reader's CONDITIONAL verdict with its requested remedies (independent judging, uncertainty estimates, artifact release) is appropriate. I do not see grounds to move to REJECT: the framework is a genuine methodological contribution, the ablations (Table 4) show the components do measurable work, human evaluation exists (Fig. 5, with significance markers), and the paper is candid about limitations. But the strongest quantitative claims should be read as \"outperforms on metrics aligned with the design target\" until a blind, rubric-independent human check is run, which keeps the verdict at CONDITIONAL rather than ACCEPT. Hence UNCHANGED relative to the reader.","tokens_in":32035,"tokens_out":1797,"duration_ms":55706,"concrete_test":"Run a blind human evaluation decoupled from the enforced trajectory: have the 10 psychology raters (or independent licensed counselors) score 100 dialogues each from EmoTrace-D and PsyDTCorpus, with dataset identity and rubric provenance hidden, on perceived emotional realism and whether they would expect this dialogue from a real client — without showing them the EmoTrace-E rubric. Report bootstrap CIs on the means. If EmoTrace-D's advantage over PsyDTCorpus shrinks to within the confidence intervals (as the 2.85 vs 2.80 PsyDT Emotional Empathy result suggests it may), the headline richness gain is a metric-design artifact; if it persists, the circularity concern does not land.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that trajectory-centered generation yields dialogues with superior emotional richness and empathy quality. For that claim to hold, the metrics showing the gain must measure something the pipeline did not directly manufacture. They largely do not. §3.3 states the corpus is filtered to retain only dialogues with \"full coverage of the seeker's three emotional stages and reasonable stage duration.\" The ETC module (Fig. 11 prompt) mandates stage progression (max 3 turns in Initial Impact, max 5 in Turbulence, forced guidance toward Integration after turn 8). EmoTrace-E (Fig. 15) then scores Emotional Variation (layering, progression from surface to deep emotion), Autonomy (self-observation, reflection), and Growth Motivation (willingness to change) — i.e., precisely the stage-trajectory signature that was enforced pre-generation and selected for post-generation. EmoTrace-D's 4.74 vs PsyDTCorpus's 4.21 on Emotional Changes is therefore close to re-measuring the filter criterion. The same circularity appears in the VA trajectories (Figs. 3–4): the \"largest peak and smallest valley\" is what the three-zone controller was built to produce, so it cannot serve as independent evidence of authenticity. The model-level claim has a parallel problem: §5.2 evaluates EmoTrace-M against a GPT-4.1-mini seeker that \"retains emotional schemas and the schema activation mechanism\" — the same seeker distribution the model was trained on — so the \"complex emotional scenarios\" test is in-distribution. Meanwhile the two non-self-designed matrices show the margin collapsing: on PsyDT Emotional Empathy, EmoTrace-D scores 2.85 vs PsyDTCorpus 2.80, and CpsyCoun Comprehensiveness/Professionalism go to PsyDTCorpus — with 100 samples, no error bars, and no significance testing, the cross-framework picture is roughly a tie except on the self-targeted metrics. None of this shows the dialogues are bad; it shows the evidence for \"substantially outperforms\" is not independent of the co","agreement_with_reader":"agree"},"referee_report":{"model":"moonshotai/kimi-k3","summary":"The paper proposes EmoTrace, a pipeline for synthesizing multi-turn psychological support dialogues in which generation is organized around the seeker's emotional trajectory rather than the counselor's strategy. A persona-profile constructor (with eight \"emotional schemas\" drawn from CBT/Leahy), a schema-activation mechanism with frequency constraints, an Emotion Trajectory Control (ETC) module enforcing a three-stage progression (Initial Impact / Turbulence / Integration), and an EFT-grounded counselor module jointly drive GPT-4.1-mini role-play. The resulting corpus (EmoTrace-D, 1,114 dialogues) is filtered for full stage coverage, and Qwen3-8B is fine-tuned on it (EmoTrace-M). The authors evaluate the corpus against four prior datasets using two external rubrics (CpsyCoun, PsyDT) plus a new in-house rubric (EmoTrace-E), all scored by averaging two LLM judges; they evaluate the model against six baselines using LLM-simulated seekers plus a small human evaluation (4 experts, 6 graduate students); an ablation removes ETC and the schema mechanism. Reported results favor EmoTrace-D and EmoTrace-M on nearly all dimensions, with the largest margins on the in-house EmoTrace-E metrics.","tokens_in":32476,"tokens_out":4448,"duration_ms":219518,"significance":"If the results hold, the contribution is meaningful for the counseling-dialogue subfield: the reframing of corpus construction around controllable seeker emotional trajectories (rather than counselor strategy optimization) is a genuinely different design axis; the framework is specified to an unusual level of detail (full prompts in Figs. 7–11, pseudocode in Algorithm 1, stage definitions in Table 6), which aids reproducibility; the paper ships an ablation isolating ETC and schema contributions (Table 4), dual LLM judges to mitigate single-judge bias, and a human evaluation with ten psychology-trained raters (Fig. 5), and commits to releasing the dataset and model. The components that are independently checkable — the external-matrix comparisons and the human study — are the right kind of evidence, but they currently show parity-to-modest gains rather than the large improvements the abstract advertises. The work is publishable in principle, but the strength of the central claim must be matched to evidence that is not partially manufactured by the generation pipeline itself.","major_comments":[{"comment":"The headline claim of superior 'emotional richness' rests on EmoTrace-E, a rubric whose dimensions are largely the properties the pipeline manufactures and selects for. §3.3 states the corpus is filtered to retain only dialogues with 'full coverage of the seeker's three emotional stages and reasonable stage duration'; the ETC prompt (Fig. 11) mandates stage progression (max 3 turns Initial Impact, max 5 Turbulence, forced guidance to Integration after turn 8); EmoTrace-E (Fig. 15) then scores Emotional Layering, Depth of Exploration, Self-Observation, and Motivation for Change — i.e., the stage-trajectory signature. EmoTrace-D's 4.74 vs PsyDTCorpus's 4.21 on Emotional Changes is therefore close to re-measuring the filter criterion. Tellingly, on the two external frameworks EmoTrace-D is only at parity with PsyDTCorpus and in fact loses on Comprehensiveness (1.90 vs 1.95), Professionalism","section":"§4, Table 2; §3.3; Fig. 11; Fig. 15"},{"comment":"The model comparison evaluates each counselor model against a GPT-4.1-mini seeker that 'retains emotional schemas and the schema activation mechanism' — the same seeker generator and mechanism used to construct EmoTrace-D. EmoTrace-M is fine-tuned on exactly this seeker distribution, so the 'complex emotional scenarios' test is in-distribution for the proposed model and out-of-distribution for the baselines; the large margins in Table 3 (e.g., Emo. 4.51 vs 3.59 for the best open baseline) are confounded by this match. A defensible fix is to repeat the evaluation with a different seeker generator (different LLM and/or profiles not derived from PsyDT seeds) and, ideally, a small human-seeker or expert role-play condition; at minimum the confound must be acknowledged and the claim tempered.","section":"§5.2, Table 3"},{"comment":"The human evaluation is the main evidence independent of LLM judges, but it is under-specified to the point of being unverifiable: no statement on whether raters were blind to model identity, no inter-annotator agreement statistic, no report of how many dialogues each rater scored (50 profiles, but per-model N is unclear), and the significance markers in Figure 5 do not name the test. Only two baselines were carried into human eval, chosen post hoc as the automatic-eval leaders, which introduces selection bias. Please report blinding, per-dimension agreement (e.g., Krippendorff's alpha), the statistical test and effect sizes, and include at least one baseline not selected via the LLM-judge leaderboard.","section":"§5.2–5.3, Figure 5"},{"comment":"No uncertainty estimates accompany any quantitative result. Table 2 uses n=100 dialogues per dataset and Table 3 uses 50 simulated dialogues per model, reported as point scores to two decimals; several claimed advantages on external matrices (e.g., Emotional Empathy 2.85 vs 2.80; State and Attitude 2.89 vs 2.81) are plausibly within sampling noise of the dual-judge average. Table 4 is a single run per ablation variant. Confidence intervals and significance tests (or bootstrap over dialogues) are needed to support 'outperforms' / 'significant decline' language, and the training is single-seed, so model-level variance is unknown.","section":"§4, Table 2; §5.3, Table 3; §5.4, Table 4"}],"minor_comments":[{"comment":"Safety metrics saturate (all datasets score 1.0 or near-1.0 on CpsyCoun/PsyDT safety; 1.94/2 for EmoTrace-M in Table 3), making these columns uninformative; either use a safety probe set designed to discriminate or drop the columns.","section":"Table 2, Table 3"},{"comment":"Model-evaluation dialogues are capped at 10 turns (§5.2), but EmoTrace-D averages 12.2 turns (Table 1) and the ETC design pushes Integration after turn 8; the truncation may differentially affect models. Please justify the cap or report sensitivity to it.","section":"§5.2"},{"comment":"Table 4: the bold/underline formatting is inconsistent with the 'best/runner-up' convention (e.g., 'w/o Schema' shows 4.71 for Empathy Quality, above 'all' at 4.61, yet the text only discusses this in prose). Please re-check the highlighting and add the Cognitive Structure metric definition reference.","section":"Table 4"},{"comment":"The claim that EmoTrace-D's arousal trajectory has 'the largest fluctuation amplitude without being overly erratic' introduces an undefined criterion ('overly erratic'); either operationalize it (e.g., turn-to-turn variance bound) or remove it.","section":"§4, Figure 4"},{"comment":"The schema-activation thresholds (two consecutive activations or five total, Algorithm 1) and the 'at least two of four conditions' activation criterion (Fig. 8) are heuristic; a brief ablation or sensitivity note on these constants would strengthen the design.","section":"§3.2.3, Algorithm 1"},{"comment":"Numerous copyediting issues: 'we proposeEmoTrace' and 'we construct' (capitalization) in the abstract; 'These issues result in LLM that lack'; 'such ascore beliefs'; 'which are essential' (agreement); 'better captures' (§3.2.2); 'its scores lower than' (§4); 'occasionall', 'pasively', 'felings' in Fig. 16–17 prompts; 'Narate' in Fig. 7. The MindChat reference ('Dong Xue* Xin Yan') is malformed.","section":"Abstract, §3.2.2, §4, Appendix B"},{"comment":"Dataset and model release is conditioned on acceptance ('will be made publicly available once the paper is accepted'); an anonymized sample at submission would materially help reviewers assess data quality.","section":"Abstract / Ethical Statement"}],"recommendation":"major_revision","confidential_remarks":"The dataset and model are withheld until acceptance, so I could not verify data quality or reproduce any numbers; the editor may want to require at least an anonymized sample at submission. The reliance on LLM-generated data scored by LLM judges is increasingly common in this subfield, but here it is compounded by the rubric/controller alignment detailed in my first major comment, which is why I am asking for human-anchored validation rather than accepting the current evidence."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"Punchline: this is a competent systems paper that fixes a real failure mode in synthetic counseling data—flat, over-compliant seekers—by putting trajectory control and Leahy-style schemas on the seeker side. The gain is useful for people building counseling LLMs. It is not independent proof that the dialogues are more authentic in a clinical sense.\n\nWhat is actually new is the control stack, not “therapy-aware synthesis” in general. SmileChat, CpsyCounD, PsyDT, and CBT-style lines already inject theory on the counselor side. EmoTrace’s move is three-zone ETC, schema activation with frequency caps (Algorithm 1), and an EFT analysis→plan→generate counselor split, plus the released-intent artifacts EmoTrace-D/M. That is a clear recipe shift and worth reading if you care about multi-turn seeker dynamics.\n\nWhat they do well: the problem statement matches what people actually see in synthetic corpora; the module split is inspectable; ablations (ETC vs schemas) move the right knobs; they report external matrices (CpsyCoun, PsyDT) not only their own; human ratings line up with the automatic counselor eval; limitations and ethics are written like adults. VA plots and case studies make the intended behavior easy to see.\n\nSoft spots, in proportion. The stress-test on circularity mostly lands for the self-designed headline. §3.3 filters for full three-stage coverage; the ETC prompt caps Impact/Turbulence and forces Integration guidance; EmoTrace-E then scores layering, progression, autonomy, and growth motivation—the signature they enforced. So 4.74 vs 4.21 on Emotional Changes is not clean external evidence. On PsyDT/CpsyCoun the margin shrinks or flips (Professionalism/Comprehensiveness favor PsyDTCorpus), with n=100, no error bars, dual LLM judges, and unreleased weights. Model eval also keeps the same schema-activated GPT seeker, so “complex emotion” is partly in-distribution. None of that makes the dialogues bad; it means “substantially outperforms on emotional richness” is oversold relative to the independent matrices.\n\nWho it is for: NLP people building counseling corpora/models who want a seeker-side control pattern. Not for clinical efficacy claims—the authors do not pretend otherwise.\n\nI would send it to referees. Ask for uncertainty estimates, less generator/judge overlap, held-out seeker protocols, and artifact release. Engage if you work this stack; treat EmoTrace-E wins as design verification more than independent discovery.","headline":"Solid seeker-side corpus recipe with real engineering value; headline “emotional richness” gains are partly scored on properties the controller and filter were built to produce.","tokens_in":33517,"tokens_out":639,"would_cite":true,"duration_ms":21006,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Centering counseling-dialogue generation on the seeker’s controllable emotional trajectory produces richer emotions and stronger empathy than counselor-first methods.","keywords":["psychological support dialogue","emotional trajectory","emotional schemas","Emotion-Focused Therapy","synthetic counseling corpus","empathy","multi-turn dialogue generation","LLM fine-tuning"],"falsifier":"Hold out real multi-turn counseling transcripts with annotated seeker valence/arousal and schema-like defenses; train or evaluate the same base model on EmoTrace-D versus prior corpora and test whether gains in emotional richness and targeted empathy still hold against human counselors and unstable human seekers, not only LLM judges.","tokens_in":33176,"feed_emoji":"💬","tokens_out":910,"duration_ms":22698,"temperature":0.7,"pith_summary":"Prior synthetic counseling corpora mostly keep the seeker emotionally flat, compliant, and problem-focused, so models trained on them struggle when real seekers are unstable. EmoTrace flips the design: it builds a seeker persona with emotional schemas, activates those schemas under frequency limits, steers the seeker’s feelings through a three-stage trajectory (initial impact, turbulence, integration), and has the counselor plan responses with Emotion-Focused Therapy stages rather than pure problem-solving. From that process the authors release a multi-turn dataset and a fine-tuned open model. On both automatic dual-model scoring and a small expert panel, the dataset and model beat existing counseling corpora and systems on emotional richness, intensity, and targeted empathy while staying competitive on safety and professionalism. The practical stake is simple: better training data for handling emotionally volatile seekers, not just tidy advice-giving dialogues.","feed_headline":"Seeker emotion trajectories beat counselor-first counseling data","feed_subtitle":"Three-stage control plus emotional schemas yield richer dialogues and a stronger empathy model","key_machinery":"EmoTrace: interactive role-play among a seeker module (persona + emotional-schema activation with consecutive/total frequency caps), a counselor module (EFT analysis–planning then response generation), and an Emotional Trajectory Control (ETC) module that assigns the seeker to Initial Impact / Turbulence / Integration zones and issues next-turn expression guidance without hard-coding the words.","core_discovery":"The paper establishes that a seeker-emotion-trajectory-centered generation pipeline—persona plus emotional schemas with activation constraints, a three-stage emotional trajectory controller, and an EFT-grounded counselor planner—yields multi-turn psychological support dialogues and a fine-tuned model that outperform prior counselor-centered corpora and systems on emotional richness and empathy quality.","pith_inferences":["If stage caps and schema frequency rules are doing most of the work, simpler trajectory schedules without full EFT planning might retain much of the richness at lower generation cost—the paper’s own scalability limitation points here.","The slight empathy-score rise when schemas are ablated suggests evaluation rubrics may still reward easy shallow empathy; future benchmarks may need adversarial, schema-defended seekers by design.","The same trajectory-plus-schema idea could transfer to other long affective dialogues (crisis hotlines, coaching) where the user’s internal state, not the helper’s script, is the scarce modeling target."],"forward_implications":["Counseling LLMs trained this way should handle non-compliant, fluctuating seekers better than models trained on flat, compliant synthetic seekers.","Emotion-focused planning (awareness → deepening → transformation) can be operationalized as structured intermediate plans inside generation pipelines, not only as free-form style.","Valence–arousal trajectory plots and seeker-side metrics become first-class dataset quality checks alongside counselor professionalism scores.","Releasing EmoTrace-D and EmoTrace-M would give a concrete baseline for seeker-centered rather than counselor-centered corpus construction."],"fun_headline_variants":["Seeker emotion trajectories beat counselor-first counseling data","EmoTrace centers seeker emotion paths for richer support dialogues","Emotional schemas plus trajectory control lift empathy quality","Seeker-first emotion trajectories outperform prior counseling corpora","Three-stage emotion control yields stronger psychological support models"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"That scripted LLM role-play under hand-set stage caps, schema frequency rules, and EFT prompts, judged mainly by other LLMs plus a small expert panel, faithfully captures real emotionally unstable counseling dynamics.","fun_headline_variants_meta":{"raw":{"variants":["Seeker emotion trajectories beat counselor-first counseling data","EmoTrace centers seeker emotion paths for richer support dialogues","Emotional schemas plus trajectory control lift empathy quality","Seeker-first emotion trajectories outperform prior counseling corpora","Three-stage emotion control yields stronger psychological support models"]},"model":"grok-4.5","effort":"low","cost_usd":0.004206,"raw_usage":{"total_tokens":1232,"prompt_tokens":742,"num_sources_used":0,"completion_tokens":59,"cost_in_usd_ticks":42064000,"prompt_tokens_details":{"text_tokens":742,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":431,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":742,"tokens_out":59,"duration_ms":7656,"temperature":1.0,"reasoning_tokens":431,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-30T16:41:27.163031+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Hold out real multi-turn counseling transcripts with annotated seeker valence/arousal and schema-like defenses; train or evaluate the same base model on EmoTrace-D versus prior corpora and test whether gains in emotional richness and targeted empathy still hold against human counselors and unstable human seekers, not only LLM judges.","supporting_citations":[],"review_version":1}