REVIEW 4 major objections 4 minor 1 cited by
Textual forma mentis networks bridge language structure, emotional content and psychopathology levels in adolescents
T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Textual forma mentis networks predict adolescent psychopathology levels from emotionally charged interview transcripts.
desk verdict Interesting exploratory associations between language-network features and adolescent psychopathology, but the 'prediction' claim is unsupported by an analysis that explicitly avoids out-of-sample testing. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the textual forma mentis network (TFMN), a cognitive network in which nodes are words or concepts from a transcript and edges connect words that are syntactically dependent within a sentence, extended by synonym links and annotated with emotion scores. Edges are drawn between non-stopwords whose syntactic distance in the dependency parse is at most k = 4, which captures about 94% of short-distance syntactic relations in these transcripts. Network measures such as degree, core structure, clustering, path lengths, centrality, modularity, global and local efficiency, together with emotion z-scores, become the feature set; Random Forest and gradient-boosting regressions model three latent psychopathology factors, and SHAP values, a game-theoretic explanation method, attribute each prediction to individual features.
What would settle it
Retrain the same regression pipeline on the 232 interviews and evaluate it on a held-out sample of new adolescent interviews with the same protocol, or on participants excluded from the original selection: if the correlation between predicted and observed factor scores drops to zero or becomes negative, the central claim that TFMN features predict psychopathology is unsupported. A cheaper check is nested cross-validation with feature selection applied only inside each training fold; the r ≈ 0.33–0.37 values would need to survive that re-evaluation to support the predictive reading.
Extended reading notes
Core claim
Textual forma mentis networks can successfully predict psychometric levels of psychopathology in adolescents based on transcripts of emotionally charged interviews. The central finding is that combinations of semantic/syntactic network features and emotional profiles outperform either alone, and that the model's decisions are interpretable: higher modularity and a pronounced core–periphery structure predict higher social maladjustment; higher betweenness centrality with stronger disgust predicts higher internalising scores; lower local efficiency predicts higher neurodevelopmental risk. A permutation test with shuffled target variables removes the correlations, supporting the claim that real structure, not chance, drives the associations. Because the evaluation uses four-fold cross-validation on the same 232-participant dataset, the paper presents the result as evidence about structure in these data rather than as a validated predictor for new individuals.
Load-bearing premise
The load-bearing premise is that correlations found by fitting models to the same 232 interviews, with choices like the syntactic-distance cutoff made from that same data, reflect a real link between language and psychopathology rather than patterns specific to this one group, an assumption the authors explicitly did not test since they say they had no interest in predicting unseen data.
Editorial extensions
If this is right
- The reproducibility of TFMN features makes language a candidate transdiagnostic marker for adolescent mental-health screening.
- The SHAP patterns generate concrete, testable hypotheses: tightly clustered conversation topics go with social maladjustment, repeated bridge concepts and disgust go with internalising tendencies, and weakly integrated concepts go with neurodevelopmental risk.
- Combined structural and emotional features should be preferred over word-count or emotion-only baselines in future language-based mental-health studies.
- The same pipeline can be reapplied to larger interview datasets, where nested cross-validation would test true out-of-sample prediction.
Reading between the lines
- A direct test would be whether the same k = 4 TFMN features trained on one interview protocol transfer to a different protocol or age band; the paper's own reading suggests this is unknown.
- Because the authors state they had no interest in predicting unseen data, the reported r values are likely upper bounds; with feature selection inside the loop, more conservative estimates would probably be lower.
- If disgust-driven betweenness is truly a rumination signature, then interventions that shift conversational focus should lower both disgust expression and the betweenness bottleneck in follow-up transcripts.
- Since modularity reflects topic separation, the social-maladjustment result could be checked against human topic annotations, linking network modularity to independently rated conversational coherence.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes a framework based on textual forma mentis networks (TFMNs) to predict three psychopathology factor scores—social maladjustment, specific internalising behaviors, and neurodevelopmental risk—in 232 adolescents from the Healthy Brain Network. Network features and emotion z-scores are extracted from transcripts of emotionally charged interviews and fed into Random Forest and Gradient Boosting regressions, evaluated with four-fold cross-validation. The authors report significant correlations (r = 0.37, 0.33, 0.34) and use SHAP values to interpret feature contributions. The central conclusion is that TFMNs 'can successfully predict psychometric levels of psychopathology in adolescents based on transcripts of emotionally charged interviews.'
Significance. If the predictive claim were supported, the paper would offer a novel, interpretable language-based marker for adolescent mental health, with concrete network features (modularity, core-periphery structure, centrality) and emotion scores linked to distinct psychopathology dimensions. The use of independent parent-report questionnaires as targets is a clear strength, and the SHAP analysis provides falsifiable, feature-level associations. However, the reported evidence does not establish out-of-sample prediction, and the authors explicitly disclaim interest in predicting unseen data. As currently presented, the contribution is best characterized as an exploratory association study, which is substantially weaker than the stated predictive claim.
major comments (4)
- [Conclusions; Discussion (Limitations)] The Conclusions state that 'TFMNs can successfully predict psychometric levels of psychopathology in adolescents,' but the Limitations section states that cross-validation was used on the whole dataset and that the authors had 'a general lack of interest in predictability of unseen data.' These statements are directly contradictory. Four-fold cross-validation on the same 232 transcripts, with no held-out sample or nested procedure, does not support a claim of successful prediction. This is the central claim of the paper, and it is unsupported by the reported analysis.
- [Methods - Machine Learning Models; Methods - Explainable AI and feature contribution analysis] Feature selection and hyperparameter tuning are not nested within the cross-validation procedure. The Methods describe features ranked by SHAP and discarded if reshuffling does not deteriorate performance 'during training/validation,' and grid search is used for hyperparameters, but no nested loop or independent validation set is described. If feature selection or hyperparameter selection uses any of the data that also contribute to the cross-validated r values in Table 2, those correlations are optimistically biased. The authors must clarify the exact procedure and, if necessary, re-run the analysis with fully nested cross-validation or an external hold-out set.
- [Results - Table 5] The 'permutation test' reports a single random shuffle of the target variable, not a null distribution over many shuffles. The p-values in Table 5 (e.g., p = 0.25, p = 0.10) are not permutation-based p-values; they appear to be ordinary significance tests applied to one shuffled dataset. A valid permutation test requires repeating the shuffle many times and comparing the observed performance to the resulting null distribution. As reported, this analysis does not support the claim that the observed results are 'unlikely due to chance.'
- [Methods - Transforming texts in textual forma mentis networks] The syntactic distance parameter k=4 is selected from the cumulative distribution function of syntactic distances in the same interview transcripts. This data-dependent choice, combined with feature selection and hyperparameter tuning on the same corpus, means that the p-values reported alongside the cross-validated correlations are not valid frequentist inferences for generalizable prediction. The authors should either provide an honest out-of-sample evaluation or explicitly reframe the claims as descriptive associations and remove 'predict' language from the title, abstract, and conclusions.
minor comments (4)
- [Abstract; Results - Machine learning models] The abstract and the Results use 'XGBoost' while the Methods consistently refer to a Gradient Boosting Machine (GBM); please standardize the terminology throughout.
- [Methods - Emotional Profiling and Emotion Quantification] The text states that Ze > 1.96 indicates both over- and under-represented emotions; the under-represented case should be Ze < -1.96.
- [Results - Explainable AI for Social Maladjustment] The text contains the typo 'Modalurity' where 'Modularity' is intended.
- [Results - Network features] Table 1 mentions 'Reciprocity (RC)' as an excluded feature, but Reciprocity is not defined in the Methods feature list; please either define it or remove the reference.
Circularity Check
No significant circularity: the reported associations are empirical fits against independent psychometric targets, and the data-dependent choices are overfitting risks rather than definitional reductions.
full rationale
The derivation chain uses TFMN features (syntactic/semantic edges via EmoAtlas, WordNet enrichment, EmoLex emotional z-scores) as predictors and latent psychopathology factor scores from parent-report questionnaires (RCADS, Conners-3, SDQ) as targets. No target score enters the construction of the TFMN features, and no equation defines a target in terms of a predictor, so the reported r values are not true by construction. The syntactic-distance parameter k = 4 is chosen from the cumulative distribution of syntactic distances in the transcripts, not from the outcome scores; SHAP-based feature selection and grid-search hyperparameter tuning are model-selection procedures that can inflate cross-validated estimates, but they do not make the result circular in the sense of a fitted parameter being renamed as a prediction. The paper explicitly disclaims out-of-sample predictive intent: 'cross-validation being used on the whole dataset... a general lack of interest in predictability of unseen data.' That is a limitation on generalizability and the strength of the word 'predict' in the Conclusions, but it is a statistical validity concern, not a circularity concern. Self-citations to prior TFMN/EmoAtlas work are methodological references to independently published tools, and no uniqueness theorem or ansatz is imported from those citations to force the reported associations. Because no circular step can be exhibited from the paper's text, the circularity score is 0.
Assumptions & free parameters
free parameters (4)
- k (syntactic distance cutoff) =
4
- Number of latent factors =
3
- Feature selection threshold =
|r| > 0.1
- Emotion z-score null sampling size N
assumptions (4)
- domain assumption Syntactic dependency parsing by EmoAtlas produces accurate syntactic trees for interview transcripts.
- domain assumption WordNet synonym relations and EmoLex emotion associations are valid proxies for semantic and emotional associations in the mental lexicon.
- domain assumption The three-factor p-factor model derived from parent-report questionnaires yields valid latent dimensions of psychopathology.
- domain assumption Network features such as modularity, core structure and betweenness reflect cognitive and linguistic organization (e.g., modularity indicates topics, high betweenness indicates rumination).
Cite this review
Pith. "Pith review of Textual forma mentis networks bridge language structure, emotional content and psychopathology levels in adolescents." pith.science (2026). https://pith.science/paper/VGT6O425
@misc{pith2026250506387,
author = {Pith},
title = {Pith review of: Textual forma mentis networks bridge language structure, emotional content and psychopathology levels in adolescents},
year = {2026},
howpublished = {\url{https://pith.science/paper/VGT6O425}},
note = {Machine review of arXiv:2505.06387}
}
read the original abstract
We introduce a network-based AI framework for predicting dimensions of psychopathology in adolescents using natural language. We focused on data capturing psychometric scores of social maladjustment, internalizing behaviors, and neurodevelopmental risk, assessed in 232 adolescents from the Healthy Brain Network. This dataset included structured interviews in which adolescents discussed a common emotion-inducing topic. To model conceptual associations within these interviews, we applied textual forma mentis networks (TFMNs)-a cognitive/AI approach integrating syntactic, semantic, and emotional word-word associations in language. From TFMNs, we extracted network features (semantic/syntactic structure) and emotional profiles to serve as predictors of latent psychopathology factor scores. Using Random Forest and XGBoost regression models, we found significant associations between language-derived features and clinical scores: social maladjustment (r = 0.37, p < .01), specific internalizing behaviors (r = 0.33, p < .05), and neurodevelopmental risk (r = 0.34, p < .05). Explainable AI analysis using SHAP values revealed that higher modularity and a pronounced core-periphery network structure-reflecting clustered conceptual organization in language-predicted increased social maladjustment. Internalizing scores were positively associated with higher betweenness centrality and stronger expressions of disgust, suggesting a linguistic signature of rumination. In contrast, neurodevelopmental risk was inversely related to local efficiency in syntactic/semantic networks, indicating disrupted conceptual integration. These findings demonstrated the potential of cognitive network approaches to capture meaningful links between psychopathology and language use in adolescents.
Figures
Forward citations
Cited by 1 Pith paper
-
Natural Language Processing Psychometrics
LLM persona questionnaire responses yield interpretable text features that transfer, with modest accuracy, to classifying depression in real human speech.
Reference graph
Works this paper leans on
-
[1]
K. Abramski, L. Ciringione, G. Rossetti, and M. Stella. V oices of rape: Cognitive networks link passive voice usage to psychological distress in online narratives. Computers in Human Behavior, 158:108266, 2024
work page 2024
-
[2]
S. Aeschbach, R. Mata, and D. U. Wulff. Mapping mental representations with free associations: A tutorial using the r package associator. Journal of Cognition, 8(1):3, 2025
work page 2025
- [3]
-
[4]
M. Al-Mosaiwi and T. Johnstone. In an absolute state: Elevated use of absolutist words is a marker specific to anxiety, depression, and suicidal ideation. Clinical psychological science, 6(4):529–542, 2018
work page 2018
-
[5]
L. M. Alexander, J. Escalera, L. Ai, C. Andreotti, K. Febre, A. Mangone, N. Vega-Potler, N. Langer, A. Alexander, M. Kovacs, S. Litke, B. O’Hagan, J. Andersen, B. Bronstein, A. Bui, M. Bushey, H. Butler, V . Castagna, N. Camacho, E. Chan, D. Citera, J. Clucas, S. Cohen, S. Dufek, M. Eaves, B. Fradera, J. Gardner, N. Grant-Villegas, G. Green, C. Gregory, E...
work page 2017
-
[6]
E. Ariyazangane, M. R. Borna, and R. Johari Fard. Relation of anger rumination and self- criticism with social maladjustment with the mediating role of psychological flexibility in adolescent boys and girls. Caspian Journal of Health Research, 7(1):5–14, 2022
work page 2022
-
[7]
A. Caspi, R. M. Houts, D. W. Belsky, S. J. Goldman-Mellor, H. Harrington, S. Israel, M. H. Meier, S. Ramrakha, I. Shalev, R. Poulton, et al. The p factor: one general psychopathology factor in the structure of psychiatric disorders? Clinical psychological science, 2(2):119–137, 2014
work page 2014
-
[8]
N. Castro and C. S. Siew. Contributions of modern network science to the cognitive sciences: Revisiting research spirals of representation and process. Proceedings of the Royal Society A, 476(2238):20190825, 2020
work page 2020
Show all 52 references
-
[9]
C. K. Conners, G. Sitarenios, J. D. Parker, and J. N. Epstein. The revised conners’ parent rating scale (cprs-r): factor structure, reliability, and criterion validity. Journal of abnormal child psychology, 26:257–268, 1998
1998
-
[10]
Crespo and V
M. Crespo and V . Fern’andez-Lansac. Linguistic indicators of post-traumatic stress disorder. Clinical Psychology Review, 43:69–81, 2016
2016
-
[11]
Cutler and D
A. Cutler and D. M. Condon. Deep lexical hypothesis: Identifying personality structure in natural language. Journal of Personality and Social Psychology, 125(1):173, 2023
2023
-
[12]
De Choudhury, M
M. De Choudhury, M. Gamon, S. Counts, and E. Horvitz. Predicting depression via social media. ICWSM, 13:128–137, 2013
2013
-
[13]
R. L. de Ross, E. Gullone, and B. F. Chorpita. The revised child anxiety and depression scale: a psychometric investigation with australian youth. Behaviour Change, 19(2):90–101, 2002
2002
-
[14]
B. Dóczi. An overview of conceptual models and theories of lexical representation in the mental lexicon. The Routledge handbook of vocabulary studies, pages 46–65, 2019
2019
-
[15]
Edwards and N
T. Edwards and N. S. Holtzman. A meta-analysis of correlations between depression and first-person singular pronoun use. Journal of Research in Personality, 68:63–68, 2017
2017
-
[16]
Fatima, Y
A. Fatima, Y . Li, T. T. Hills, and M. Stella. Dasentimental: Detecting depression, anxiety, and stress in texts via emotional recall, cognitive networks, and machine learning. Big data and cognitive computing, 5(4):77, 2021
2021
-
[17]
E. M. Geronimi and J. Woodruff-Borden. The language of generalized anxiety disorder. Cognitive Therapy and Research, 39:115–122, 2015
2015
-
[18]
Goodman and R
A. Goodman and R. Goodman. Strengths and difficulties questionnaire as a dimensional measure of child mental health. Journal of the American Academy of Child & Adolescent Psychiatry, 48(4):400–403, 2009
2009
-
[19]
S. Harnad. To cognize is to categorize: cognition is categorization. In Handbook of categoriza- tion in cognitive science, pages 21–54. Elsevier, 2017
2017
-
[20]
T. T. Hills and Y . N. Kenett. Is the mind a network? maps, vehicles, and skyhooks in cognitive network science. Topics in Cognitive Science, 14(1):189–208, 2022
2022
-
[21]
Holmes, S
J. Holmes, S. Mareva, M. P. Bennett, M. J. Black, and J. Guy. Higher-order dimensions of psychopathology in a neurodevelopmental transdiagnostic sample. Journal of Abnormal Psychology, 130(8):909–922, 2021
2021
-
[22]
Y . N. Kenett, O. Levy, D. Y . Kenett, H. E. Stanley, M. Faust, and S. Havlin. Flexibility of thought in high creative individuals represented by percolation analysis. Proceedings of the National Academy of Sciences, 115(5):867–872, 2018
2018
-
[23]
O. M. Laceulle, W. A. V ollebergh, and J. Ormel. The structure of psychopathology in adoles- cence: Replication of a general psychopathology factor in the trails study.Clinical Psychological Science, 3(6):850–860, 2015. 20
2015
-
[24]
B. B. Lahey, B. Applegate, J. K. Hakes, D. H. Zald, A. R. Hariri, and P. J. Rathouz. Is there a general factor of prevalent psychopathology during adulthood? Journal of abnormal psychology, 121(4):971, 2012
2012
-
[25]
Latora and M
V . Latora and M. Marchiori. Efficient behavior of small-world networks. Phys. Rev. Lett., 87:198701, Oct 2001
2001
-
[26]
E. J. Lyons and M. R. Mehl. Disordered eating and internet use: A qualitative investigation. Journal of Health Psychology, 24(7):972–982, 2019
2019
-
[27]
G. A. Miller. Wordnet: a lexical database for english. Commun. ACM, 38(11):39–41, Nov. 1995
1995
-
[28]
S. M. Mohammad and P. D. Turney. Crowdsourcing a word–emotion association lexicon. Computational intelligence, 29(3):436–465, 2013
2013
-
[29]
S. E. Morgan, K. Diederen, P. E. Vértes, S. H. Ip, B. Wang, B. Thompson, A. Demjaha, A. De Micheli, D. Oliver, M. Liakata, et al. Natural language processing markers in first episode psychosis and people at clinical high-risk. Translational psychiatry, 11(1):630, 2021
2021
-
[30]
N. B. Mota, M. Copelli, and S. Ribeiro. Thought disorder measured as random speech structure classifies negative symptoms and schizophrenia diagnosis 6 months in advance. npj Schizophre- nia, 3(1):18, 2017
2017
-
[31]
N. B. Mota, N. A. Vasconcelos, N. Lemos, A. C. Pieretti, O. Kinouchi, G. A. Cecchi, M. Copelli, and S. Ribeiro. Speech graphs provide a quantitative measure of thought disorder in psychosis. PloS one, 7(4):e34928, 2012
2012
-
[32]
M. Newman. Networks: An Introduction. Oxford University Press, London (UK), 2018
2018
-
[33]
J. Olah, N. Cummins, M. Arribas, T. Gibbs-Dean, E. Molina, D. Sethi, M. J. Kempton, S. Mor- gan, T. Spencer, and K. Diederen. Towards a scalable approach to assess speech organization across the psychosis-spectrum-online assessment in conjunction with automated transcription a...
2024
-
[34]
J. W. Pennebaker, M. R. Mehl, and K. G. Niederhoffer. Psychological aspects of natural language use: Our words, our selves. Annual review of psychology, 54(1):547–577, 2003
2003
-
[35]
Pugach, S
C. Pugach, S. Nester, and B. Wisco. Uncovering the temporal dynamics of negative thought in posttraumatic stress disorder. Cognitive Therapy and Research, pages 1–15, 2025
2025
-
[36]
Richards and C
A. Richards and C. C. French. Anxiety-related bias in the classification of emotionally ambigu- ous words. Cognition and Emotion, 6(5):479–491, 1992
1992
-
[37]
C. Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature machine intelligence, 1(5):206–215, 2019
2019
-
[38]
Sekuli’c, M
S. Sekuli’c, M. Gjurkovi’c, and J. Šnajder. Automated detection of bipolar disorder in text. Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic, pages 148–157, 2018
2018
-
[39]
Semeraro, S
A. Semeraro, S. Vilella, R. Improta, E. S. De Duro, S. M. Mohammad, G. Ruffo, and M. Stella. Emoatlas: An emotional network analyzer of texts that merges psychological lexicons, artificial intelligence, and network science. Behavior Research Methods, 57(2):77, 2025
2025
-
[40]
Semeraro, S
A. Semeraro, S. Vilella, G. Ruffo, and M. Stella. Emotional profiling and cognitive networks un- ravel how mainstream and alternative press framed astrazeneca, pfizer and covid-19 vaccination campaigns. Scientific Reports, 12(1):14445, Aug 2022
2022
-
[41]
C. S. Q. Siew, D. U. Wulff, N. M. Beckage, and Y . N. Kenett. Cognitive network science: A review of research on cognition through the lens of network representations, processes, and dynamics. Complexity, 2019(1):2108423, 2019. 21
2019
-
[42]
Sorella, A
S. Sorella, A. Grecucci, L. Piretti, and R. Job. Do anger perception and the experience of anger share common neural mechanisms? coordinate-based meta-analytic evidence of similar and different mechanisms from functional neuroimaging studies. NeuroImage, 230:117777, 2021
2021
-
[43]
E. C. Stade, L. Ungar, J. C. Eichstaedt, G. Sherman, and A. M. Ruscio. Depression and anxiety have distinct and overlapping language patterns: Results from a clinical interview. Journal of psychopathology and clinical science, 132(8):972, 2023
2023
-
[44]
M. Stella. Text-mining forma mentis networks reconstruct public perception of the stem gender gap in social media. PeerJ Computer Science, 6:e295, 2020
2020
-
[45]
M. Stella. Cognitive network science for understanding online social cognitions: A brief review. Topics in Cognitive Science, 14(1):143–162, 2022
2022
-
[46]
Stella, S
M. Stella, S. Citraro, G. Rossetti, D. Marinazzo, Y . N. Kenett, and M. S. Vitevitch. Cognitive modelling of concepts in the mental lexicon with multilayer networks: Insights, advancements, and future challenges. Psychonomic Bulletin & Review, pages 1–24, 2024
2024
-
[47]
Y . R. Tausczik and J. W. Pennebaker. The psychological meaning of words: Liwc and com- puterized text analysis methods. Journal of language and social psychology , 29(1):24–54, 2010
2010
-
[48]
M. S. Vitevitch. Network Science in Cognitive Psychology. Routledge, 2019
2019
-
[49]
M. S. Vitevitch. What can network science tell us about phonology and language processing? Topics in Cognitive Science, 14(1):127–142, 2022
2022
-
[50]
E. R. Watkins and H. Roberts. Reflecting on rumination: Consequences, causes, mechanisms and treatment of rumination. Behaviour research and therapy, 127:103573, 2020
2020
-
[51]
F. Xu, H. Uszkoreit, Y . Du, W. Fan, D. Zhao, and J. Zhu. Explainable ai: A brief survey on history, research areas, approaches and challenges. In Natural language processing and Chinese computing: 8th cCF international conference, NLPCC 2019, dunhuang, China, October 9–14, 20...
2019
-
[52]
Zhang and Y
C. Zhang and Y . Ma. Ensemble machine learning, volume 144. Springer, 2012. 22 A Supplementary Information This Section contains additional information for the random forest classifier and explainable AI. A.1 RFR SHAP Values for Social Maladjustment This Section contains SHAP ...
2012
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.