REVIEW 4 major objections 6 minor 47 references
The Language of Attachment: Modeling Attachment Dynamics in Psychotherapy
T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read The paper demonstrates that patient attachment style — the relational pattern at the center of PACS-based psychotherapy assessment — can be recovered from therapy transcripts by NLP classifiers, with the best configuration reaching a mean…
desk verdict A well-motivated first attempt at PACS attachment classification, but the headline accuracy is below the majority baseline and the only above-baseline result is an untested ensemble on 12 sessions, so the feasibility claim is not established. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism that carries the argument is the minimum-input-length ablation: consecutive patient turns from the same session are concatenated until they reach a threshold — 0, 50, 100, 150, or 250 words — and each resulting window is labeled with the PACS attachment style of its source document. The classifiers are RoBERTa-base, RoBERTa-large, and MentalRoBERTa, optionally further pre-trained on unlabeled counseling transcripts; the best-performing configuration uses RoBERTa-large at the 150-word threshold. By comparing accuracy across thresholds, the paper tests whether attachment is expressed in discourse-level patterns rather than in isolated short utterances; the upward trend in accuracy with longer input is the load-bearing evidence for that claim.
What would settle it
A decisive check: train the same RoBERTa-large 150-word setup on the same turns but with session-level attachment labels randomly permuted across training documents; if test accuracy stays clearly above chance, the reported signal is a confound, not attachment. The paper's own wide split-to-split spread (51.69% to 68.54%) already warns that the result is sensitive to which patients land in the test fold.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that automatic turn-level classification can distinguish the three PACS attachment categories — avoidant, secure, and preoccupied — from patient speech alone, and does so better when short turns are merged into longer passages. In the test condition, each instance is a patient speech window of at least 150 words labeled with the attachment style of the whole session; the five fine-tuned RoBERTa-large models average 59.55% accuracy, the majority vote reaches 67.42%, and the confusion matrices show a persistent overprediction of 'preoccupied.' The paper interprets these results as evidence that attachment-relevant signal is present in patient language, while acknowledging that the class skew, the small corpus, and the variance across splits keep the finding from being deployment-ready.
Load-bearing premise
The evaluation assumes that every patient speech turn carries the attachment style of the whole session, so each turn can be judged as an independent instance; if attachment is expressed only across longer discourse patterns, or if the model latches onto speaker identity, topic, or session content rather than attachment-relevant language, the reported accuracy does not measure what it claims.
Editorial extensions
If this is right
- If the central claim holds, automatic attachment screening is feasible enough to justify building larger PACS-annotated corpora, which the paper identifies as the main bottleneck.
- Longer input windows — up to entire session transcripts, and possibly including therapist speech — become the obvious next step, since accuracy rose as the minimum input length grew.
- Repeated automated measurement of attachment throughout treatment becomes practical, whereas manual PACS annotation takes about 90 minutes per session and needs roughly 30 hours of coder training.
- Because the models overpredict 'preoccupied' and miss most 'avoidant' patients, any applied use would require calibration against the clinical cost of each confusion — especially the preoccupied/avoidant swap, which the paper argues is the most damaging.
- The one generative attempt (Llama 3.1 70B at a 150-word minimum) scored only 29% accuracy, suggesting that fine-tuned discriminative encoders, not out-of-the-box generative models, are the near-term route.
Reading between the lines
- A model that sees only patient turns cannot exploit turn-taking itself; extending the same ablation to therapist-patient exchanges might sharpen attachment detection, since PACS codes how patients manage emotional proximity with the therapist.
- The document-level labeling assumption implies that turn accuracy may overstate the model's grasp of attachment: an analysis of which turns the model gets right, or an attention-based feature study, could reveal whether it keys on content topics rather than relational language.
- The majority-vote gain the paper reports suggests the five training splits learn different, partly complementary signals; ensembling across more splits or across model architectures is a cheap extension that could be tested immediately.
- A direct next experiment is to measure accuracy per therapist and per therapy modality (Brief Relational Therapy vs. CBT), since the corpus mixes both and the paper's document-level split may not balance them.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents an exploratory study on automatically classifying patient attachment style (avoidant, secure, preoccupied) from psychotherapy transcripts. The authors extract patient speech turns from 78 therapy sessions annotated with the Patient Attachment Coding System (PACS), label each turn with the source document's attachment label, and train several RoBERTa-based classifiers. They report a mean test accuracy of 59.55% for their best setup (RoBERTa-large, minimum input length 150 words), with a majority vote across five models reaching 67.42%. They also analyze the effect of input length, domain-adaptive pre-training, and discuss clinical and ethical implications.
Significance. If the reported classification accuracy were reliable, the paper would represent a valuable first step toward automating attachment assessment, with potential implications for personalized psychotherapy and research. However, the central evidence is not statistically established: the individual models do not exceed the majority-class baseline, and the only above-baseline result is an ensemble with no significance testing and an effective sample size of 12 sessions. The paper does provide a useful exploratory analysis and a thoughtful discussion of error patterns and clinical considerations, and it honestly reports that domain adaptation did not help.
major comments (4)
- [Section 5.2, Table 2] The mean test accuracy of the five RoBERTa-large models is 59.55%, which is below the 61.80% majority-class baseline that the authors themselves report in Section 6.2; individual models therefore do not outperform a trivial majority classifier. The only above-baseline result is the majority vote (67.42%), but this is a post-hoc ensemble with no confidence interval or significance test, and with only 12 test sessions the difference is not shown to be reliable. Please provide a session-level significance test (e.g., permutation or bootstrap) and report confidence intervals.
- [Section 3.1] All 7,255 speech turns are labeled with the document-level PACS label, and the evaluation treats each turn as an independent instance. Because all turns from a session share the same label and are likely correlated, the effective sample size is 12 test sessions, not the number of turns, which inflates the apparent reliability of the accuracy estimates. Please report a session-level evaluation (e.g., majority vote within each session or a mixed-effects model) and discuss the implications for the conclusions.
- [Section 4.3 and Table 1] The conclusion that increasing input length improves performance is based on cross-validation accuracy across different thresholds, but no statistical significance is reported, and the class distribution changes with concatenation (at minimum length 150, the test set majority becomes 61.80%). The improvement may be driven by a more skewed class prior rather than better classification. Please compare each length condition against the corresponding majority baseline and test the trend with appropriate error bars.
- [Abstract and Introduction] The statement 'Our best results achieve an average accuracy of almost 60%' is presented without the crucial context that this accuracy is below the majority-class baseline of the test set. This framing overstates the evidence and should be corrected by reporting the baseline comparison and the ensemble result with appropriate caveats.
minor comments (6)
- [Section 3.1] The description of the split says 'stratification according to class balance,' but the test set appears highly skewed toward preoccupied (61.80%); please clarify the stratification procedure and report the per-class session counts in train and test.
- [Section 5.1] The table header says 'Std dev' but the text refers to 'relative standard deviation'; please clarify which quantity is reported.
- [Section 6.2] There is a typo in the sentence 'most of the models do not rely only on the majority label that they ignore the avoidant and secure labels' ('that' should likely be 'and').
- [Figure 3] The error bars represent standard deviation, but confidence intervals would be more informative given the small test set.
- [Limitations] The limitation section does not address the statistical non-independence of turns or the fact that individual models do not beat the majority baseline; please add a discussion of these points.
- [Table 1 / Section 5] The paper does not report the number of test instances for each minimum length condition; please add this to Table 1 or the text.
Circularity Check
No circularity: the classification is a supervised empirical evaluation against an external PACS label set; the reported weaknesses are statistical validity concerns, not circular reasoning.
full rationale
The paper's central claim is an empirical supervised-learning result: models are trained on patient speech turns and evaluated on held-out psychotherapy sessions whose labels come from the Patient Attachment Coding System (PACS; Talia et al., 2017). PACS is an external, manually administered coding instrument, and the paper does not define the target label in terms of the model outputs or derive the label from the model. The document-level split is explicit and intended to avoid leakage: 'We split at the document level to avoid data leakage and ensure that the model is not learning to recognize individuals but rather generalizable speech patterns according to PACS classification.' The reported 'almost 60%' mean accuracy and the input-length trend are measured outcomes, not fitted parameters renamed as predictions, and no equation in the paper reduces one claimed result to another by construction. The fact that the 59.55% mean test accuracy is below the 61.80% majority baseline, and that turn-level instances inherit document labels with only 12 held-out sessions, are substantive empirical and statistical limitations, but they are not circularity. There is also no load-bearing self-citation: the cited prior work on PACS, RoBERTa, MentalRoBERTa, domain-adaptive pre-training, and MaChAmp provides external methods and data whose assumptions do not include the target result. The paper itself flags its small dataset and resource constraints in the Limitations section, which further supports treating the findings as preliminary empirical evidence rather than as a derivation. Because no specific circular step can be exhibited, the honest finding is score 0.
Assumptions & free parameters
free parameters (2)
- Minimum input length thresholds =
50, 100, 150, 250 words
- Learning rate =
1e-5
assumptions (5)
- domain assumption PACS labels are valid and reliable ground-truth attachment classifications.
- domain assumption Attachment style is expressed in the patient's speech at the turn level, so labeling each turn with its document's attachment class is meaningful.
- domain assumption Concatenating consecutive turns preserves the attachment class of the source document.
- domain assumption The three PACS attachment categories (secure, preoccupied, avoidant) are exhaustive for this population.
- domain assumption Splitting at document level prevents data leakage of speaker identity.
Cite this review
Pith. "Pith review of The Language of Attachment: Modeling Attachment Dynamics in Psychotherapy." pith.science (2026). https://pith.science/paper/I2DN4UBK
@misc{pith2026250416271,
author = {Pith},
title = {Pith review of: The Language of Attachment: Modeling Attachment Dynamics in Psychotherapy},
year = {2026},
howpublished = {\url{https://pith.science/paper/I2DN4UBK}},
note = {Machine review of arXiv:2504.16271}
}
read the original abstract
The delivery of mental healthcare through psychotherapy stands to benefit immensely from developments within Natural Language Processing (NLP), in particular through the automatic identification of patient specific qualities, such as attachment style. Currently, the assessment of attachment style is performed manually using the Patient Attachment Coding System (PACS; Talia et al., 2017), which is complex, resource-consuming and requires extensive training. To enable wide and scalable adoption of attachment informed treatment and research, we propose the first exploratory analysis into automatically assessing patient attachment style from psychotherapy transcripts using NLP classification models. We further analyze the results and discuss the implications of using automated tools for this purpose -- e.g., confusing `preoccupied' patients with `avoidant' likely has a more negative impact on therapy outcomes with respect to other mislabeling. Our work opens an avenue of research enabling more personalized psychotherapy and more targeted research into the mechanisms of psychotherapy through advancements in NLP.
Figures
Reference graph
Works this paper leans on
-
[1]
Katie Aafjes-van Doorn, C \'e line Kamsteeg, Jordan Bate, and Marc Aafjes. 2021. A scoping review of machine learning in psychotherapy research. Psychotherapy Research, 31(1):92--116
work page 2021
- [2]
-
[3]
Allison L. Baier, Alexander C. Kline, and Norah C. Feeny. 2020. https://doi.org/10.1016/j.cpr.2020.101921 Therapeutic alliance as a mediator of change: A systematic review and evaluation of research
arXiv 2020
-
[4]
Bakermans-Kranenburg and Marinus H
Marian J. Bakermans-Kranenburg and Marinus H. van IJzendoorn. 2009. https://doi.org/10.1080/14616730902814762 The first 10,000 adult attachment interviews: Distributions of adult attachment representations in clinical and non-clinical groups . Attachment and Human Development, 11:223--263
-
[5]
Judith S. Beck. 2011. Cognitive behavior therapy: Basics and beyond, 2nd edition. Guilford Press
work page 2011
-
[6]
Reuben Binns. 2022. Human judgment in algorithmic loops: Individual justice and automated decision-making. Regulation & Governance, 16(1):197--211
work page 2022
-
[7]
John. Bowlby. 1988. A secure base: parent-child attachment and healthy human development. Basic Books
work page 1988
-
[8]
J. Cassidy, J. D. Jones, and P. R. Shaver. 2013. https://doi.org/10.1017/S0954579413000692 Contributions of attachment theory and research: a framework for future research, translation, and policy . Developmental Psychopathology, 25(4 Pt 2):1415--1434
Show all 47 references
-
[9]
Pim Cuijpers, Mirjam Reijnders, and Marcus JH Huibers. 2019. The role of common factors in psychotherapy outcomes. Annual review of clinical psychology, 15(1):207--231
2019
-
[10]
Daly and Brent Mallinckrodt
Katherine D. Daly and Brent Mallinckrodt. 2009. https://doi.org/10.1037/a0016695.supp Experienced therapists’ approach to psychotherapy for adults with attachment avoidance or attachment anxiety . Journal of Counseling Psychology, 56:549--563
2009 doi
-
[11]
Yeager, Christopher J
Dorottya Demszky, Diyi Yang, David S. Yeager, Christopher J. Bryan, Margarett Clapper, Susannah Chandhok, Johannes C. Eichstaedt, Cameron Hecht, Jeremy Jamieson, Meghann Johnson, Michaela Jones, Danielle Krettek-Cobb, Leslie Lai, Nirel JonesMitchell, Desmond C. Ong, Carol S. D...
2023 doi
-
[12]
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zha...
2024 arXiv
-
[13]
Vicky Eames and Anthony Roth. 2000. https://doi.org/10.1093/ptr/10.4.421 Patient attachment orientation and the early working alliance—a study of patient and therapist reports of alliance quality and ruptures . Psychotherapy Research, 10:421--434
2000 doi
-
[14]
Judith A Feeney and Susan M Ryan. 1994. Attachment style and affect regulation: Relationships with health behavior and family experiences of illness in a student sample. Health Psychology, 13:334--345
1994
-
[15]
Carol George, Nancy Kaplan, and Mary Main. 1985. Adult Attachment Interview Protocol . Unpublished manuscript
1985
-
[16]
Jonathan Gratch, Ron Artstein, Gale Lucas, Giota Stratou, Stefan Scherer, Angela Nazarian, Rachel Wood, Jill Boberg, David DeVault, Stacy Marsella, David Traum, Skip Rizzo, and Louis-Philippe Morency. 2014. The distress analysis interview corpus of human and computer interview...
2014
-
[17]
Gregory, L
M. Gregory, L. Kannis-Dymand, and R. Sharman. 2020. https://doi.org/10.1111/ajpy.12270 A review of attachment-based parenting interventions: Recent advances and future considerations . Australian Journal of Psychology, 72(2):109--122
2020 doi
-
[18]
Suchin Gururangan, Ana Marasovíc, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A Smith. 2020. https://github.com/allenai/ Don't stop pretraining: Adapt language models to domains and tasks . pages 8342--8360
2020
-
[19]
John D Haltigan, Emily M Leerkes, Maria S Wong, Kristin Fortuna, Gilberte I Roisman, Anne J Supple, Michael O'Brien, Susan D Calkins, and Angela Plamondon. 2014. https://doi.org/10.1111/cdev.12180 Adult attachment states of mind: Measurement invariance across ethnicity and ass...
2014 doi
-
[20]
Jennifer A Hoffmann, Megan M Attridge, Michael S Carroll, Norma-Jean E Simon, Andrew F Beck, and Elizabeth R Alpern. 2023. Association of youth suicides and county-level mental health professional shortage areas in the us. JAMA pediatrics, 177(1):71--80
2023
-
[21]
Shaoxiong Ji, Tianlin Zhang, Luna Ansari, Jie Fu, Prayag Tiwari, and Erik Cambria. 2022. Mentalbert: Publicly available pretrained language models for mental healthcare. pages 7184--7190
2022
-
[22]
Katri Kanninen, Jari Salo, and Raija Leena Punamäki. 2000. https://doi.org/10.1093/ptr/10.4.435 Attachment patterns and working alliance in trauma therapy for victims of political violence . Psychotherapy Research, 10:435--449
2000 doi
-
[23]
Karantzas and Si \^a n F
Gery C. Karantzas and Si \^a n F. Cole. 2011. https://api.semanticscholar.org/CorpusID:145084238 Arthritis and support seeking tendencies: The role of attachment . Journal of Social and Clinical Psychology, 30:404--440
2011
-
[24]
Levy, William D
Kenneth N. Levy, William D. Ellison, Lori N. Scott, and Samantha L. Bernecker. 2011. https://doi.org/10.1002/jclp.20756 Attachment style . Journal of Clinical Psychology, 67:193--203
2011 doi
-
[25]
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. https://arxiv.org/abs/1907.11692 Roberta: A robustly optimized BERT pretraining approach . CoRR, abs/1907.11692
2019 arXiv
-
[26]
Matteo Malgaroli, Thomas D Hull, James M Zech, and Tim Althoff. 2023. Natural language processing for mental health interventions: a systematic review and research framework. Translational Psychiatry, 13(1):309
2023
-
[27]
Ganeshan Malhotra, Abdul Waheed, Aseem Srivastava, Md Shad Akhtar, and Tanmoy Chakraborty. 2022. https://doi.org/10.1145/3488560.3498509 Speaker and time-aware joint contextual learning for dialogue-act classification in counselling conversations . pages 735--745. Association ...
2022
-
[28]
Mario Mikulincer and Phillip R. Shaver. 2007. Attachment in Adulthood: Structure, Dynamics, and Change. Guilford Press, New York
2007
-
[29]
Shaver, and Ety Berant
Mario Mikulincer, Phillip R. Shaver, and Ety Berant. 2013. https://doi.org/10.1111/j.1467-6494.2012.00806.x An attachment perspective on therapeutic processes and outcomes . Journal of Personality, 81:606--616
2013
-
[30]
Safran, and J
Madeleine Miller-Bottome, Alessandro Talia, Jeremy D. Safran, and J. Christopher Muran. 2018. https://doi.org/10.1037/pap0000152 Resolving alliance ruptures from an attachment-informed perspective . Psychoanalytic Psychology, 35:175--183
2018 doi
-
[31]
Rodgers, Laurent Cailhol, Elsa Bui, Raymond Klein, Ludovic Schmitt, and Herve Chabrol
Richard F. Rodgers, Laurent Cailhol, Elsa Bui, Raymond Klein, Ludovic Schmitt, and Herve Chabrol. 2010. https://doi.org/10.1016/j.encep.2010.02.005 L'alliance th \'e rapeutique en psychoth \'e rapie: apports de la recherche empirique [therapeutic alliance in psychotherapy: the...
2010 doi
-
[32]
Rosenthal and Roger Kobak
Natalie L. Rosenthal and Roger Kobak. 2010. https://doi.org/10.1111/j.1532-7795.2010.00655.x Assessing adolescents' attachment hierarchies: Differences across developmental periods and associations with individual adaptation . Journal of Research on Adolescence, 20:678--706
2010
-
[33]
Rowe and Katherine B
Angela C. Rowe and Katherine B. Carnelley. 2005. https://doi.org/10.1111/j.1475-6811.2005.00128.x Preliminary support for the use of a hierarchical mapping technique to examine attachment networks . Personal Relationships, 12:499--519
2005
-
[34]
Safran and J
Jeremy D. Safran and J. Christopher Muran. 2000. Negotiating the therapeutic alliance: A relational treatment guide. Guilford Press
2000
-
[35]
Shorey and C
Hal S. Shorey and C. R. Snyder. 2006. https://doi.org/10.1037/1089-2680.10.1.1 The role of adult attachment styles in psychopathology and psychotherapy outcomes . Review of General Psychology, 10:1--20
2006 doi
-
[36]
Arietta Slade. 2016. Attachment and Adult Psychotherapy: Theory, Research, and Practice, 3 edition, pages 759--779. Guilford Publications
2016
-
[37]
Arietta Slade and Jeremy Holmes. 2019. https://doi.org/10.1016/j.copsyc.2018.06.008 Attachment and psychotherapy . Current Opinion in Psychology, 25:152--156
2019 doi
-
[38]
Brennan M. R. Spiegel, Omer Liran, Allistair Clark, Jamil S. Samaan, Carine Khalil, Robert Chernoff, Kavya Reddy, and Muskaan Mehra. 2024. https://doi.org/10.1038/s41746-024-01011-0 Feasibility of combining spatial computing and ai for mental health support in anxiety and depr...
2024 doi
-
[39]
Elizabeth C Stade, Shannon Wiltsey Stirman, Lyle H Ungar, Cody L Boland, H Andrew Schwartz, David B Yaden, Jo \ a o Sedoc, Robert J DeRubeis, Robb Willer, and Johannes C Eichstaedt. 2024. Large language models could change the future of behavioral healthcare: a proposal for re...
2024
-
[40]
move to the level of the relation
Alessandro Talia, Sarah I.F. Daniel, Madeleine Miller-Bottome, Daniela Brambilla, Denise Miccoli, Jeremy D. Safran, and Vittorio Lingiardi. 2014. https://doi.org/10.1080/14616734.2013.859161 Aai predicts patients' in-session interpersonal behavior and discourse: A "move to the...
2014
-
[41]
Alessandro Talia, Madeleine Miller-Bottome, and Sarah I.F. Daniel. 2017. https://doi.org/10.1002/cpp.1990 Assessing attachment in psychotherapy: Validation of the patient attachment coding system (pacs) . Clinical Psychology and Psychotherapy, 24:149--161
2017 doi
-
[42]
Tancredy and R
Caroline M. Tancredy and R. Chris Fraley. 2006. https://doi.org/10.1037/0022-3514.90.1.78 The nature of adult twin relationships: An attachment-theoretical perspective . Journal of Personality and Social Psychology, 90:78--93
2006 doi
-
[43]
Rob van der Goot, Ahmet \"U st \"u n, Alan Ramponi, Ibrahim Sharaf, and Barbara Plank. 2021. https://doi.org/10.18653/v1/2021.eacl-demos.22 Massive choice, ample tasks ( M a C h A mp): A toolkit for multi-task learning in NLP . In Proceedings of the 16th Conference of the Euro...
2021 doi
-
[44]
Marinus H van IJzendoorn. 1995. Adult attachment representations, parental responsiveness, and infant attachment: A meta-analysis on the predictive validity of the adult attachment interview. Psychological Bulletin, 117:387--403
1995
-
[45]
Zixiu Wu, Simone Balloccu, Vivek Kumar, Rim Helaoui, Ehud Reiter, Diego Reforgiato Recupero, and Daniele Riboni. 2022. https://doi.org/10.1109/ICASSP43922.2022.9746035 Anno-mi: A dataset of expert-annotated counselling dialogues . volume 2022-May, pages 6177--6181. Institute o...
2022
-
[46]
online" 'onlinestring :=
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...
-
[47]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.