Pith. sign in

REVIEW 3 major objections 6 minor 92 references

Natural Language Processing Psychometrics

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Language-to-score models trained purely on AI-generated questionnaire answers separate clinically depressed speakers from controls in real transcribed speech, with AUC up to 0.780.

desk verdict Novel framework, but the headline human-transfer result is undercut by a length confound the authors themselves identify. read the letter →

arxiv 2608.07316 v1 pith:454KAAJ4 submitted 2026-08-07 cs.CL cs.AIcs.SI

Edoardo Sebastiano De Duro , Emma Franchino , Massimo Stella This is my paper · ORCID
classification cs.CLcs.AIcs.SI
keywords NLPpsychometricscognitivedigitalshadowstextualformamentisnetworksdepressiondetectionlargelanguagemodelsrandomforestsSHAPtransferlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that psychometric scores — life satisfaction, depression, anxiety, and stress — can be read off the structure and emotional content of free text, and that the mapping can be learned from synthetic data and then applied to real human language. Nine LLMs, prompted to impersonate personas that vary in sociodemographics, personality, and mental-health profile, answered standard questionnaires (SWLS, PHQ-9, DASS-21) and wrote a short explanation for each item. Random-forest regressors trained on the emotion and network features of those explanations separate clinically depressed speakers from controls in the Androids Corpus transcripts without retraining, with an AUC of 0.780 for DASS-21 depression and 0.695 for PHQ-9. The framework's point is that psychometric prediction from language can be made interpretable — via syntactic network topology and emotional profiles, not opaque embeddings — and that synthetic personas can expose what LLMs encode, provided the mapping is validated on human data.

What carries the argument

The carrying machinery is the textual forma mentis network (TFMN), a syntactic-semantic network built from each respondent's item explanations: content words are nodes, edges encode syntactic dependencies and synonymy relations, and the network yields eight structural features (nodes, edges, components, clustering, average shortest path, diameter, degree assortativity, valence assortativity) plus eight emotion z-scores from EmoAtlas. These features, combined with Big Five persona traits and sociodemographics, are fed into ablated random-forest regressors; SHAP attributes each prediction to specific features. The training signal comes from cognitive digital shadows: nine LLMs prompted to impersonate randomly generated personas and complete standardized questionnaires, so each text carries a known psychometric score. Transfer from synthetic to human text is the decisive test: the same random forests, untouched, are applied to diary entries and to transcribed Androids Corpus interviews.

What would settle it

Re-run the human-transfer analysis with N nodes, N edges, and N components either removed or divided by word count, or with transcripts truncated to a common length; if the DASS-21 depression AUC of 0.780 collapses toward chance, the claimed human transfer is largely a verbosity artifact.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that a text-to-score mapping trained exclusively on LLM-generated questionnaire explanations carries over to authentic human depressive language. Using only network and emotion features — textual forma mentis network descriptors (nodes, edges, components, clustering, path length, diameter, degree and valence assortativity) and eight Plutchik emotion z-scores — a random forest trained on Mistral Small's DASS-21 depression explanations separates 63 clinically depressed speakers from 52 controls with an AUC of 0.780 and 68% classification accuracy, while the analogous PHQ-9 model reaches 0.695 and 62%. The same models transfer to LLM-generated diaries with rank correlations up to 0.9. The paper reads the topological signature — low degree assortativity, hub-centred star-like networks with many syntactic edges — as a linguistic trace of rumination, and notes that the direction reverses for anxiety. It also reports that sociodemographics alone explain essentially no variance in depression, anxiety, or stress, while emotion features and income drive life-satisfaction predictions.

Load-bearing premise

The load-bearing premise is that the network features separating depressed from control speakers reflect linguistic content rather than the amount of speech, because three of the eight network features grow mechanically with text length and the clinical and control interviews were analyzed without length matching.

Editorial extensions

If this is right

  • If the transfer result is right, psychometric scoring of unseen text no longer requires a matched corpus of human questionnaire-plus-explanation responses; synthetic persona data can bootstrap models for depression-related constructs.
  • The framework gives an interpretable audit channel for LLMs: SHAP attributions show whether a model's psychometric answers are driven by persona metadata (income, neuroticism), emotional content, or discourse topology, exposing model-specific biases such as the abliterated GPT-OSS variant carrying almost no psychometric signal.
  • Depression-specific topology (hub-centred, low-assortativity discourse) and anxiety-specific topology (integrated, distributed discourse) are claimed as distinct signatures; if confirmed, the direction of degree assortativity becomes a candidate marker separating rumination from worry.
  • Cross-domain transfer is uneven — anxiety transfers poorly to diaries — so any application of a text-to-score mapping must be validated register by register and construct by construct.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A sharper test of the human transfer would drop or length-normalize the three size-sensitive network features (N nodes, N edges, N components); because the clinical and control interviews were not matched for duration or word count, part of the reported AUC could be a verbosity effect rather than a psycholinguistic one, a point the paper itself flags in its limitations.
  • The reversed assortativity signal between depression and anxiety, if it survives verbosity controls, gives a concrete topological hypothesis that could be tested directly on human corpora with matched-length transcripts.
  • The synthetic-to-human pipeline suggests a low-cost way to generate training data for other constructs, such as burnout or PTSD, where no paired text-score corpus yet exists — provided a length-control protocol is built in from the start.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces NLP Psychometrics, a framework that trains random-forest regressors on LLM-generated questionnaire explanations (produced under controlled 'cognitive digital shadow' personas) to predict psychometric scores (SWLS, PHQ-9, DASS-21) from interpretable network and emotion features extracted from text. The authors report strong in-sample R² values, SHAP-based feature attributions, and two transfer tests: (i) to LLM-generated diary entries without retraining, and (ii) to real transcribed clinical speech from the Androids Corpus (63 depressed vs. 52 control participants). The central claim is that the language-to-score mapping learned from synthetic data overlaps with genuine markers of depressive language in humans, as evidenced by AUCs of 0.695 (PHQ-9) and 0.780 (DASS-21 Depression) and classification accuracies of 62% and 68% in the human transfer.

Significance. If the human-transfer result is correct, the paper provides a notable demonstration that synthetic LLM data can be used to learn interpretable, text-based markers of depression that generalize to authentic human speech. The framework is methodologically transparent: it uses established psychometric instruments, an ablation design across four feature families, SHAP for interpretability, and it openly reports model failures (e.g., GPT-OSS-Uncensored) and inconsistent transfer (Qwen-4B-Instruct in Appendix B). The paper's explicit discussion of limitations and its framing of the framework as an auditing tool rather than a validated clinical measure are strengths. However, the human-transfer result, which is load-bearing for the entire enterprise, is currently threatened by a plausible length confound, as the authors themselves acknowledge in Section 4.1. The central claim is therefore defensible but not yet established.

major comments (3)
  1. The human-transfer analysis uses only the Network+Emotion random forest, whose feature set includes N nodes, N edges, and N components—features the paper explicitly states in Section 4.1 'scale with raw text length rather than purely with discourse structure.' The Androids Corpus analysis does not control for word count, speaking duration, or transcript length, and the depressed (N=63) and control (N=52) groups are not reported as length-matched. If the two groups differ in verbosity, the AUCs of 0.695 and 0.780 could partly or entirely reflect a length proxy rather than depressive content. This is load-bearing because the human transfer is the only evidence that the synthetic mapping captures genuine human depressive markers. The authors' own Section 4.1 concludes that 'requires explicit verbosity controls before clinical interpretation,' but no such controls are applied to the human-transfer test. Please rerun the human-transfer analysis with length normalization (e.g., dividing N nodes and N edges by word count, or including transcript length as a covariate) and/or demonstrate that the groups do not differ in length and that results survive a length-matched subsample analysis.
  2. The main-text transfer analyses use Mistral Small, selected after the authors observed that Qwen-4B-Instruct failed to transfer PHQ-9 to diaries under the reference condition (Appendix B). Because the model choice is informed by transfer performance, the reported effect sizes (e.g., r=0.907 for PHQ-9 diary transfer) are at risk of selection bias. The paper does report the Qwen results in an appendix, which is good, but it should additionally state clearly that the human-transfer results are only for the model that succeeded in the diary transfer, and discuss how this selection affects the generalizability of the human-transfer claim. Ideally, report human-transfer results for all LLMs for which data are available, or justify why the selected model is representative.
  3. The in-sample R² values are computed on data where the persona prompt explicitly includes a mental-health level, and the LLM generates both the score and the explanation. High R² may therefore reflect the LLM's internal consistency between the assigned persona and the generated text—a form of circularity—rather than a general language-to-score law. The human transfer is the only test that breaks this circularity. This makes the length confound in the human transfer (first major comment) even more consequential: if the human result is confounded, the paper's central claim rests on the self-referential in-sample performance, which is not independently informative. The authors should either provide an additional out-of-sample test on human data with proper controls or temper the claim that the mapping 'transfers to humans' until such controls are provided.
minor comments (6)
  1. The symbol 'r' is used for the Spearman correlation in Tables 2-4 and for the Mann-Whitney effect size in Tables 5-9. The captions should explicitly distinguish these two statistics to avoid confusion.
  2. The phrase 'r up to 0.91' refers to the rank-biserial effect size from the Mann-Whitney test, not a Pearson or Spearman correlation. The abstract should clarify this to avoid misinterpretation.
  3. The AUC formula '1−U/(n_low·n_high)' is correct for the Mann-Whitney U statistic, but the direction convention (whether higher U corresponds to higher predicted scores in the depressed group) should be stated explicitly.
  4. Several URLs in Table 1 are broken across lines and some are incomplete (e.g., the Mistral Small entry). These should be formatted as proper hyperlinks or footnotes.
  5. The persona mental-health distribution is a simplified mutually exclusive approximation ('no symptoms' 50%, depression 20%, anxiety 20%, stress 10%). The paper notes this is 'indicative rather than epidemiologically exact,' but it would be helpful to state the sensitivity of the main RF results to these weights, since they directly shape the training labels.
  6. The comparison between the PHQ-9 and DASS-21 Depression transfer effects uses median score differences on different scales (0-27 vs. 0-21). The AUC values are the more appropriate comparison; the text should avoid implying that the raw median differences are directly comparable.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the in-sample fit is presented as an LLM audit, and the central claim rests on out-of-sample transfer to diaries and real clinical speech.

full rationale

The paper's derivation chain is: (i) LLM personas generate questionnaire scores together with item-level explanations; (ii) random forests learn a mapping from text-derived features to those scores; (iii) the fitted mapping is applied, without retraining, to LLM-generated diaries and to an independent human corpus with binary clinical labels. No step collapses into its own input by construction. The high in-sample R2 values are not presented as external validation of a human psychometric law; the paper frames them as an audit of how LLMs encode psychometric constraints into language (Sections 1.3, 2.2, and the Discussion). The transfer tests are out-of-sample in genre and source, and the human-data AUCs (0.695, 0.780) come from a model that never saw those transcripts during training. The threshold used for classification is estimated on the same human data, but the paper applies leave-one-out cross-validation and notes that the threshold does not affect the AUC (Section 2.2.5), so this is not a fitted input that is then called a prediction. The main validity threat is verbosity: the network features N nodes, N edges, and N components scale with text length, and the human-transfer analysis does not control for word count. The paper explicitly acknowledges this in Section 4.1, stating these features 'grow mechanically with longer texts' and that the rumination signature 'requires explicit verbosity controls before clinical interpretation.' That is an honest limitation and a potential confound, not a circular reduction of the conclusion to its inputs. Self-citations to cognitive digital shadows and EmoAtlas introduce the tools and framing; they are not invoked as external proofs of the transfer result, which is instead tested against real clinical data. Accordingly, no specific circular step can be exhibited, and the circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or theoretical entities. The framework relies on prior constructs (cognitive digital shadows, TFMN, EmoAtlas) and on hand-set parameters for persona generation and analysis choices. The main unverified assumptions are the validity of LLM personas as psychological simulators and the reliability of the external clinical labels.

free parameters (3)
  • Persona mental-health prevalence weights = 50% no symptoms, 20% depression, 20% anxiety, 10% stress
    Hand-set to approximate pandemic-era Italian prevalence; shapes the distribution of assigned scores and thus the regression targets.
  • R2 tolerance for parsimonious model selection = 0.01
    Used to select the most parsimonious ablated model within 0.01 R2 of the best; affects which model is interpreted in SHAP analysis.
  • Diary word-length constraints = 100-150, 150-200, 250-300 words
    Prompt constraints for diary generation; may affect network features in transfer.
assumptions (4)
  • domain assumption Deep Lexical Hypothesis: psychologically meaningful variation is partially structured within language
    Theoretical grounding for treating network features as psychometric signals (Section 1.2).
  • domain assumption Cognitive digital shadows produce text whose psychometric signal reflects assigned persona profiles
    The rationale for training on LLM-generated questionnaire explanations (Section 2.2).
  • domain assumption EmoAtlas and EmoLex provide valid Italian emotion and TFMN extraction
    All text features rely on this toolkit and lexicon; validation cited in references [10, 43].
  • domain assumption Androids Corpus transcripts and binary clinical labels are reliable ground truth for depression
    External dataset used for human transfer; no questionnaire scores available (Section 2.2.5).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Natural Language Processing Psychometrics." pith.science (2026). https://pith.science/paper/454KAAJ4

@misc{pith2026260807316,
  author       = {Pith},
  title        = {Pith review of: Natural Language Processing Psychometrics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/454KAAJ4}},
  note         = {Machine review of arXiv:2608.07316}
}
abstract

Natural Language Processing (NLP) models predicting mental health outcomes rarely specify what they measure: contextual knowledge, emotional content, or syntactic structure. NLP Psychometrics treats psychological prediction from text as a psychometric problem, linking scores to interpretable linguistic evidence and testing beyond the training text format. Nine LLMs, conditioned on controlled personas (cognitive digital shadows), completed psychometric questionnaires with textual explanations per item. We extracted emotional profiles and syntactic-semantic structure via textual forma mentis networks, combined with personality and sociodemographic variables in ablated random forest (RF) regressors, using SHAP to identify which features drove performance and in which direction. Full RF models explained up to 70.8% of variance in life satisfaction (SWLS), 55.7% in depression (PHQ-9), and, for DASS-21, 68.5% depression, 76.0% anxiety, 72.4% stress. Sociodemographics alone explained no meaningful variance in depression, anxiety, or stress, but did so for life satisfaction, where emotion features and income were the strongest predictors; neuroticism and network topology instead dominated depression and anxiety, reversing direction between them. Without retraining, RF models separated diaries from low- and high-score personas ($r$ up to 0.91) and, using only network/emotion features, classified clinical from control participants in real transcripts with up to 68% accuracy. These results show the promise and limits of synthetic data: LLM personas can expose model biases, recover patterns consistent with clinical rumination, and support psychometric prediction from human text without a matched questionnaire, but cannot substitute for human validation. NLP Psychometrics makes these distinctions explicit, measurable, and testable through interpretable AI and network/emotional features.

Figures

Figures reproduced from arXiv: 2608.07316 by the authors.

Figure 1
Figure 1. NLP Psychometrics: an interpretable pipeline from language to psychometric scores. [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Random Forest prediction performance (R2 ) on SWLS scores for each of the nine feature-set configurations (see legend), grouped by LLM. Bars within each group correspond to the same feature sets across all LLMs, allowing direct comparison of how predictive performance varies both across LLMs (x-axis) and across feature combinations (bar colour). PHQ-9: network and emotion features carry great predictive power. The P… view at source ↗
Figure 3
Figure 3. Random Forest prediction performance (R2 ) on PHQ-9 scores for each of the nine feature-set configurations (see legend), grouped by LLM. Bars within each group correspond to the same feature sets across all LLMs, allowing direct comparison of how predictive performance varies both across LLMs (x-axis) and across feature combinations (bar colour) [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Random Forest prediction performance (R2 ) on DASS-21 scores for Mistral Small, across each of the nine feature-set configurations (bar colour, see legend) and each factor subscale (Depression, Anxiety, Stress) [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: SHAP summary plots for the four best-performing LLMs on SWLS, each showing the best Random Forest [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: SHAP summary plots for the four best-performing LLMs on PHQ-9, each showing the best Random Forest [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]
Figure 7
Figure 7. Figure 7: SHAP summary plots for Mistral Small on the three DASS-21 factor subscales, each showing the best Random [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

92 extracted references · 55 canonical work pages

  1. [1]

    Enrico Perinelli. Substantive individual differences in social desirability: Stability, change, and associations with personality traits and job satisfaction in a large-scale longitudinal survey.Journal of Research in Personality, page 104726, 2026

    2026

  2. [2]

    Self-efficacy and locus of control when facing listening challenges: Validation of the listening challenges attitude scale (licas)

    Elena Giovanelli, Enrico Perinelli, Chiara Valzolgher, Elena Gessa, and Francesco Pavani. Self-efficacy and locus of control when facing listening challenges: Validation of the listening challenges attitude scale (licas). International Journal of Listening, 40(2):110–125, 2026

    2026

  3. [3]

    Exploratory factor analysis: Current use, methodological developments and recommendations for good practice.Current psychology, 40(7):3510–3521, 2021

    David Goretzko, Trang Thien Huong Pham, and Markus Bühner. Exploratory factor analysis: Current use, methodological developments and recommendations for good practice.Current psychology, 40(7):3510–3521, 2021

    2021

  4. [4]

    Dasentimental: Detecting depression, anxiety, and stress in texts via emotional recall, cognitive networks, and machine learning.Big data and cognitive computing, 5(4):77, 2021

    Asra Fatima, Ying Li, Thomas Trenholm Hills, and Massimo Stella. Dasentimental: Detecting depression, anxiety, and stress in texts via emotional recall, cognitive networks, and machine learning.Big data and cognitive computing, 5(4):77, 2021

    2021

  5. [5]

    Llms can persuade only psychologically susceptible humans on societal issues, via trust in ai and emotional appeals, amid logical fallacies.arXiv preprint arXiv:2604.16935, 2026

    Alexis Carrillo, Salvatore Citraro, Ali Aghazhadeh Ardebili, Enrique Taietta, Giulio Rossetti, Emilio Ferrara, Giuseppe Alessandro Veltri, and Massimo Stella. Llms can persuade only psychologically susceptible humans on societal issues, via trust in ai and emotional appeals, amid logical fallacies.arXiv preprint arXiv:2604.16935, 2026

    arXiv 2026

  6. [6]

    Examining linguistic differences in electronic health records for diverse patients with diabetes: natural language processing analysis.JMIR Medical Informatics, 12(1):e50428, 2024

    Isabel Bilotta, Scott Tonidandel, Winston R Liaw, Eden King, Diana N Carvajal, Ayana Taylor, Julie Thamby, Yang Xiang, Cui Tao, and Michael Hansen. Examining linguistic differences in electronic health records for diverse patients with diabetes: natural language processing analysis.JMIR Medical Informatics, 12(1):e50428, 2024

    2024

  7. [7]

    Language is primarily a tool for communication rather than thought.Nature, 630(8017):575–586, 2024

    Evelina Fedorenko, Steven T Piantadosi, and Edward AF Gibson. Language is primarily a tool for communication rather than thought.Nature, 630(8017):575–586, 2024

    2024

  8. [8]

    John Wiley & Sons, 2012

    Jean Aitchison.Words in the mind: An introduction to the mental lexicon. John Wiley & Sons, 2012

    2012

Show all 92 references
  1. [9]

    Deep lexical hypothesis: Identifying personality structure in natural language.Journal of Personality and Social Psychology, 125(1):173, 2023

    Andrew Cutler and David M Condon. Deep lexical hypothesis: Identifying personality structure in natural language.Journal of Personality and Social Psychology, 125(1):173, 2023

    2023

  2. [10]

    Emoatlas: An emotional network analyzer of texts that merges psychological lexicons, artificial intelligence, and network science.Behavior Research Methods, 57(2):77, 2025

    Alfonso Semeraro, Salvatore Vilella, Riccardo Improta, Edoardo Sebastiano De Duro, Saif M Mohammad, Giancarlo Ruffo, and Massimo Stella. Emoatlas: An emotional network analyzer of texts that merges psychological lexicons, artificial intelligence, and network science.Behavior R...

    2025

  3. [11]

    Using complex networks to understand the mental lexicon

    Michael S Vitevitch, Rutherford Goldstein, Cynthia SQ Siew, and Nichol Castro. Using complex networks to understand the mental lexicon. InYearbook of the Poznan Linguistic Meeting, volume 1, pages 119–138, 2014

    2014

  4. [12]

    Structure and flexibility: Inves- tigating the relation between the structure of the mental lexicon, fluid intelligence, and creative achievement

    Yoed N Kenett, Roger E Beaty, Paul J Silvia, David Anaki, and Miriam Faust. Structure and flexibility: Inves- tigating the relation between the structure of the mental lexicon, fluid intelligence, and creative achievement. Psychology of Aesthetics, Creativity, and the Arts, 10...

    2016

  5. [13]

    Cognitive modelling of concepts in the mental lexicon with multilayer networks: Insights, advancements, and future challenges.Psychonomic Bulletin & Review, 31(5):1981–2004, 2024

    Massimo Stella, Salvatore Citraro, Giulio Rossetti, Daniele Marinazzo, Yoed N Kenett, and Michael S Vitevitch. Cognitive modelling of concepts in the mental lexicon with multilayer networks: Insights, advancements, and future challenges.Psychonomic Bulletin & Review, 31(5):198...

    1981

  6. [14]

    In an absolute state: Elevated use of absolutist words is a marker specific to anxiety, depression, and suicidal ideation.Clinical psychological science, 6(4):529–542, 2018

    Mohammed Al-Mosaiwi and Tom Johnstone. In an absolute state: Elevated use of absolutist words is a marker specific to anxiety, depression, and suicidal ideation.Clinical psychological science, 6(4):529–542, 2018

    2018

  7. [15]

    How do users of a mental health app conceptualise digital therapeutic alliance? a qualitative study using the framework approach

    Theresa Taylor, Simon D’Alfonso, Maria João Tralhão Dolan, Jenny Yiend, and Pamela Jacobsen. How do users of a mental health app conceptualise digital therapeutic alliance? a qualitative study using the framework approach. BMC Public Health, 25(1):2450, 2025

    2025

  8. [16]

    Detecting and measuring depression on social media using a machine learning approach: systematic review.JMIR Mental Health, 9(3): e27244, 2022

    Danxia Liu, Xing Lin Feng, Farooq Ahmed, Muhammad Shahid, Jing Guo, et al. Detecting and measuring depression on social media using a machine learning approach: systematic review.JMIR Mental Health, 9(3): e27244, 2022

    2022

  9. [17]

    Predicting depression via social media

    Munmun De Choudhury, Michael Gamon, Scott Counts, and Eric Horvitz. Predicting depression via social media. InProceedings of the international AAAI conference on web and social media, volume 7, pages 128–137, 2013

    2013

  10. [18]

    Digital shadows in mental health map how llms simulate depression, anxiety, and stress through language and psychometrics.PsyArXiv,

    Emma Franchino, Rodolfo Rizzi, Edoardo Sebastiano De Duro, and Massimo Stella. Digital shadows in mental health map how llms simulate depression, anxiety, and stress through language and psychometrics.PsyArXiv,

  11. [19]

    Harvard University, 2024

    Daniel Mark Low.Speech and text psychometrics: Identifying suicide risk factors with large language models and acoustic networks. Harvard University, 2024. 25

    2024

  12. [20]

    Text psychometrics: Assessing psychological constructs in text using natural language processing, 2026

    Daniel Low, Patrick Mair, Matthew Nock, and Satrajit Ghosh. Text psychometrics: Assessing psychological constructs in text using natural language processing, 2026

    2026

  13. [21]

    Peter F Lovibond and Sydney H Lovibond. The structure of negative emotional states: Comparison of the depression anxiety stress scales (dass) with the beck depression and anxiety inventories.Behaviour research and therapy, 33(3):335–343, 1995

    1995

  14. [22]

    The phq-9: validity of a brief depression severity measure.Journal of general internal medicine, 16(9):606–613, 2001

    Kurt Kroenke, Robert L Spitzer, and Janet BW Williams. The phq-9: validity of a brief depression severity measure.Journal of general internal medicine, 16(9):606–613, 2001

    2001

  15. [23]

    Mapping how llms debate societal issues when shadowing human personality traits, sociodemographics and social media behavior.arXiv preprint arXiv:2604.27624, 2026

    Ali Aghazadeh Ardebili and Massimo Stella. Mapping how llms debate societal issues when shadowing human personality traits, sociodemographics and social media behavior.arXiv preprint arXiv:2604.27624, 2026. URL https://arxiv.org/abs/2604.27624

    2026 arXiv

  16. [24]

    Large language model based multi-agents: A survey of progress and challenges.arXiv preprint arXiv:2402.01680, 2024

    Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang. Large language model based multi-agents: A survey of progress and challenges.arXiv preprint arXiv:2402.01680, 2024

    2024 arXiv

  17. [25]

    Edoardo Sebastiano De Duro, Riccardo Improta, and Massimo Stella. Introducing counsellme: A dataset of simulated mental health dialogues for comparing llms like haiku, llamantino and chatgpt against humans.Emerging Trends in Drugs, Addictions, and Health, 5:100170, 2025

    2025

  18. [26]

    Escaping the jingle-jangle jungle: Increasing conceptual clarity in psychology using large language models.Current Directions in Psychological Science, 35(2):59–65, 2026

    Dirk U Wulff and Rui Mata. Escaping the jingle-jangle jungle: Increasing conceptual clarity in psychology using large language models.Current Directions in Psychological Science, 35(2):59–65, 2026

    2026

  19. [27]

    Lost in the middle: How language models use long contexts.Transactions of the Association for Computational Linguistics, 12:157–173, 2024

    Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts.Transactions of the Association for Computational Linguistics, 12:157–173, 2024

    2024

  20. [28]

    Large language models accurately identify decision reasons in verbal reports.Proceedings of the National Academy of Sciences, 123(27):e2526798123, 2026

    Kamil Fuławka, Ralph Hertwig, and Dirk U Wulff. Large language models accurately identify decision reasons in verbal reports.Proceedings of the National Academy of Sciences, 123(27):e2526798123, 2026

    2026

  21. [29]

    Evaluating llms for synthetic personas generation: A comparative analysis of personality representation and censorship effects

    Luigi Casoria, Pietro Neroni, Luca Sabatucci, Agnese Augello, and Giuseppe Caggianese. Evaluating llms for synthetic personas generation: A comparative analysis of personality representation and censorship effects. In Proceedings of the 16th Biannual Conference of the Italian ...

    2025

  22. [30]

    Quantifying the persona effect in llm simulations.arXiv preprint arXiv:2402.10811, 2024

    Tiancheng Hu and Nigel Collier. Quantifying the persona effect in llm simulations.arXiv preprint arXiv:2402.10811, 2024

    2024 arXiv

  23. [31]

    Safety tax: Safety alignment makes your large reasoning models less reasonable.arXiv preprint arXiv:2503.00555, 2025

    Tiansheng Huang, Sihao Hu, Fatih Ilhan, Selim Furkan Tekin, Zachary Yahn, Yichang Xu, and Ling Liu. Safety tax: Safety alignment makes your large reasoning models less reasonable.arXiv preprint arXiv:2503.00555, 2025

    2025 arXiv

  24. [32]

    Large language models that replace human participants can harmfully misportray and flatten identity groups.Nature Machine Intelligence, 7(3):400–411, 2025

    Angelina Wang, Jamie Morgenstern, and John P Dickerson. Large language models that replace human participants can harmfully misportray and flatten identity groups.Nature Machine Intelligence, 7(3):400–411, 2025

    2025

  25. [33]

    Large language models are homogeneously creative.PNAS nexus, 5(3): pgag042, 2026

    Emily Wenger and Yoed N Kenett. Large language models are homogeneously creative.PNAS nexus, 5(3): pgag042, 2026

    2026

  26. [34]

    Math education digital shadows for facilitating learning with llms: Math performance, anxiety and confidence in simulated students and ais.arXiv preprint arXiv:2604.27618, 2026

    Naomi Esposito, Anthony Tricarico, Luisa Porzio, Ali Aghazadeh Ardebili, and Massimo Stella. Math education digital shadows for facilitating learning with llms: Math performance, anxiety and confidence in simulated students and ais.arXiv preprint arXiv:2604.27618, 2026. URLhtt...

    2026 arXiv

  27. [35]

    Annamaria Di Fabio and Alessio Gori. Measuring adolescent life satisfaction: psychometric properties of the satisfaction with life scale in a sample of italian adolescents and young adults.Journal of Psychoeducational Assessment, 34(5):501–506, 2016

    2016

  28. [36]

    Gioia Bottesi, Marta Ghisi, Gianmarco Altoè, Erica Conforti, Gabriele Melli, and Claudio Sica. The italian version of the depression anxiety stress scales-21: Factor structure and psychometric properties on community and clinical samples.Comprehensive psychiatry, 60:170–181, 2015

    2015

  29. [37]

    A diagnostic meta-analysis of the patient health questionnaire- 9 (phq-9) algorithm scoring method as a screen for depression.General hospital psychiatry, 37(1):67–75, 2015

    Laura Manea, Simon Gilbody, and Dean McMillan. A diagnostic meta-analysis of the patient health questionnaire- 9 (phq-9) algorithm scoring method as a screen for depression.General hospital psychiatry, 37(1):67–75, 2015

    2015

  30. [38]

    Using network science to analyze concept maps of psychology undergraduates.Applied Cognitive Psychology, 33(4):662–668, 2019

    Cynthia SQ Siew. Using network science to analyze concept maps of psychology undergraduates.Applied Cognitive Psychology, 33(4):662–668, 2019

    2019

  31. [39]

    Cognitive networks for knowledge modeling: A gentle introduction for data-and cognitive scientists.Wiley Interdisciplinary Reviews: Cognitive Science, 17(2):e70026, 2026

    Edith Haim and Massimo Stella. Cognitive networks for knowledge modeling: A gentle introduction for data-and cognitive scientists.Wiley Interdisciplinary Reviews: Cognitive Science, 17(2):e70026, 2026

    2026

  32. [40]

    Cognitive network science: A review of research on cognition through the lens of network representations, processes, and dynamics.Complexity, 2019(1): 2108423, 2019

    Cynthia SQ Siew, Dirk U Wulff, Nicole M Beckage, and Yoed N Kenett. Cognitive network science: A review of research on cognition through the lens of network representations, processes, and dynamics.Complexity, 2019(1): 2108423, 2019. 26

    2019

  33. [41]

    spreadr: An r package to simulate spreading activation in a network.Behavior Research Methods, 51(2):910–929, 2019

    Cynthia SQ Siew. spreadr: An r package to simulate spreading activation in a network.Behavior Research Methods, 51(2):910–929, 2019

    2019

  34. [42]

    Using network science in the language sciences and clinic.International journal of speech-language pathology, 17(1):13–25, 2015

    Michael S Vitevitch and Nichol Castro. Using network science in the language sciences and clinic.International journal of speech-language pathology, 17(1):13–25, 2015

    2015

  35. [43]

    Crowdsourcing a word–emotion association lexicon.Computational intelligence, 29(3):436–465, 2013

    Saif M Mohammad and Peter D Turney. Crowdsourcing a word–emotion association lexicon.Computational intelligence, 29(3):436–465, 2013

    2013

  36. [44]

    Affective biases in english are bi-dimensional.Cognition and Emotion, 29(7):1147–1167, 2015

    Amy Beth Warriner and Victor Kuperman. Affective biases in english are bi-dimensional.Cognition and Emotion, 29(7):1147–1167, 2015

    2015

  37. [45]

    A perspective on explainable artificial intelligence methods: Shap and lime.Advanced Intelligent Systems, 7(1):2400304, 2025

    Ahmed M Salih, Zahra Raisi-Estabragh, Ilaria Boscolo Galazzo, Petia Radeva, Steffen E Petersen, Karim Lekadir, and Gloria Menegaz. A perspective on explainable artificial intelligence methods: Shap and lime.Advanced Intelligent Systems, 7(1):2400304, 2025

    2025

  38. [46]

    Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.Nature machine intelligence, 1(5):206–215, 2019

    Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.Nature machine intelligence, 1(5):206–215, 2019

    2019

  39. [47]

    Personality traits in large language models.Prepritnt, 2023

    Gregory Serapio-García, Mustafa Safdari, Clément Crepy, Luning Sun, Stephen Fitz, Marwa Abdulhai, Aleksandra Faust, and Maja Matari´c. Personality traits in large language models.Prepritnt, 2023

    2023

  40. [48]

    The satisfaction with life scale.Journal of personality assessment, 49(1):71–75, 1985

    ED Diener, Robert A Emmons, Randy J Larsen, and Sharon Griffin. The satisfaction with life scale.Journal of personality assessment, 49(1):71–75, 1985

    1985

  41. [49]

    Screening for depressive disorders in patients with skin diseases: a comparison of three screeners.Acta dermato-venereologica, 85(5):414–419, 2005

    Angelo Picardi, David A Adler, Damiano Abeni, Hong Chang, Paolo Pasquini, William H Rogers, and Kathleen M Bungay. Screening for depressive disorders in patients with skin diseases: a comparison of three screeners.Acta dermato-venereologica, 85(5):414–419, 2005

    2005

  42. [50]

    Advanced natural-based interaction for the italian language: Llamantino-3-anita, 2024

    Marco Polignano, Pierpaolo Basile, and Giovanni Semeraro. Advanced natural-based interaction for the italian language: Llamantino-3-anita, 2024

    2024

  43. [51]

    Qwen3 technical report, 2025

    Team Qwen. Qwen3 technical report, 2025. URLhttps://arxiv.org/abs/2505.09388

    2025 arXiv

  44. [52]

    gpt-oss-120b & gpt-oss-20b model card, 2025

    OpenAI. gpt-oss-120b & gpt-oss-20b model card, 2025. URLhttps://arxiv.org/abs/2508.10925

    2025 arXiv

  45. [53]

    OLMo 3: Open language models.arXiv preprint arXiv:2512.13961, 2025

    AI2. OLMo 3: Open language models.arXiv preprint arXiv:2512.13961, 2025

    2025 arXiv

  46. [54]

    Nemotron 3 nano: Open, efficient mixture-of-experts hybrid mamba-transformer model for agentic reasoning.arXiv preprint arXiv:2512.20848, 2025

    NVIDIA. Nemotron 3 nano: Open, efficient mixture-of-experts hybrid mamba-transformer model for agentic reasoning.arXiv preprint arXiv:2512.20848, 2025

    2025

  47. [55]

    Testing theory of mind in large language models and humans.Nature Human Behaviour, 8(7):1285–1295, 2024

    James WA Strachan, Dalila Albergo, Giulia Borghini, Oriana Pansardi, Eugenio Scaliti, Saurabh Gupta, Krati Saxena, Alessandro Rufo, Stefano Panzeri, Guido Manzi, et al. Testing theory of mind in large language models and humans.Nature Human Behaviour, 8(7):1285–1295, 2024

    2024

  48. [56]

    Beliefs about beliefs: Representation and constraining function of wrong beliefs in young children’s understanding of deception.Cognition, 13(1):103–128, 1983

    Heinz Wimmer and Josef Perner. Beliefs about beliefs: Representation and constraining function of wrong beliefs in young children’s understanding of deception.Cognition, 13(1):103–128, 1983

    1983

  49. [57]

    Comparative performance of large language models in emotional safety classification across sizes and tasks.Frontiers in Artificial Intelligence, 8:1706090, 2025

    Edoardo Pinzuti, Oliver Tüscher, and André Ferreira Castro. Comparative performance of large language models in emotional safety classification across sizes and tasks.Frontiers in Artificial Intelligence, 8:1706090, 2025

    2025

  50. [58]

    The cost of thinking is similar between large reasoning models and humans.Proceedings of the National Academy of Sciences, 122(47):e2520077122, 2025

    Andrea Gregor de Varda, Ferdinando Pio D’Elia, Hope Kean, Andrew Lampinen, and Evelina Fedorenko. The cost of thinking is similar between large reasoning models and humans.Proceedings of the National Academy of Sciences, 122(47):e2520077122, 2025

    2025

  51. [59]

    Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting.Advances in Neural Information Processing Systems, 36:74952–74965, 2023

    Miles Turpin, Julian Michael, Ethan Perez, and Samuel Bowman. Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting.Advances in Neural Information Processing Systems, 36:74952–74965, 2023

    2023

  52. [60]

    Leveraging llm respondents for item evaluation: A psychometric analysis.British Journal of Educational Technology, 56(3):1028–1052, 2025

    Yunting Liu, Shreya Bhandari, and Zachary A Pardos. Leveraging llm respondents for item evaluation: A psychometric analysis.British Journal of Educational Technology, 56(3):1028–1052, 2025

    2025

  53. [61]

    Covid-19 pandemic and lockdown measures impact on mental health among the general population in italy.Frontiers in psychiatry, 11:550552, 2020

    Rodolfo Rossi, Valentina Socci, Dalila Talevi, Sonia Mensi, Cinzia Niolu, Francesca Pacitti, Antinisca Di Marco, Alessandro Rossi, Alberto Siracusano, and Giorgio Di Lorenzo. Covid-19 pandemic and lockdown measures impact on mental health among the general population in italy....

    2020

  54. [62]

    Psychological distress among italians during the 2019 coronavirus disease (covid-19) quarantine

    Maurizio Bonati, Rita Campi, Michele Zanetti, Massimo Cartabia, Francesca Scarpellini, Antonio Clavenna, and Giulia Segre. Psychological distress among italians during the 2019 coronavirus disease (covid-19) quarantine. BMC psychiatry, 21(1):20, 2021

    2019

  55. [63]

    Text-mining forma mentis networks reconstruct public perception of the stem gender gap in social media.PeerJ Computer Science, 6:e295, 2020

    Massimo Stella. Text-mining forma mentis networks reconstruct public perception of the stem gender gap in social media.PeerJ Computer Science, 6:e295, 2020. 27

    2020

  56. [64]

    Textual forma mentis networks bridge language structure, emotional content and psychopathology levels in adolescents.arXiv preprint arXiv:2505.06387, 2025

    Alexis Carrillo, Simon Friedrich Roske, Rebeca Ianov-Vitanov, Enrico Perinelli, Alessandro Grecucci, and Massimo Stella. Textual forma mentis networks bridge language structure, emotional content and psychopathology levels in adolescents.arXiv preprint arXiv:2505.06387, 2025

    2025 arXiv

  57. [65]

    Cognitive networks for knowledge modelling: A gentle tutorial for data-and cognitive scientists.PsyArXiv Preprints, 2023

    Edith Haim and Massimo Stella. Cognitive networks for knowledge modelling: A gentle tutorial for data-and cognitive scientists.PsyArXiv Preprints, 2023

    2023

  58. [66]

    Estimating the number of communities in a network.Physical review letters, 117(7):078301, 2016

    Mark EJ Newman and Gesine Reinert. Estimating the number of communities in a network.Physical review letters, 117(7):078301, 2016

    2016

  59. [67]

    Ysocial: An artificial intelligence powered social media virtual twin

    Giulio Rossetti, Massimo Stella, Rémy Cazabet, Katherine Abramski, Salvatore Citraro, Erica Cau, Andrea Failla, Virginia Morini, and Valentina Pansanella. Ysocial: An artificial intelligence powered social media virtual twin. Big Data & Society, 13(3):20539517261431576, 2026

    2026

  60. [68]

    Forma mentis networks quantify crucial differences in stem perception between students and experts.PloS one, 14(10):e0222870, 2019

    Massimo Stella, Sarah De Nigris, Aleksandra Aloric, and Cynthia SQ Siew. Forma mentis networks quantify crucial differences in stem perception between students and experts.PloS one, 14(10):e0222870, 2019

    2019

  61. [69]

    A general psychoevolutionary theory of emotion

    Robert Plutchik. A general psychoevolutionary theory of emotion. InTheories of emotion, pages 3–33. Elsevier, 1980

    1980

  62. [70]

    Random forests.Machine learning, 45(1):5–32, 2001

    Leo Breiman. Random forests.Machine learning, 45(1):5–32, 2001

    2001

  63. [71]

    Pedregosa, G

    F. Pedregosa, G. Varoquaux, A. Gramfort, V . Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V . Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python.Journal of Machine Learnin...

    2011

  64. [72]

    A unified approach to interpreting model predictions.Advances in neural information processing systems, 30, 2017

    Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions.Advances in neural information processing systems, 30, 2017

    2017

  65. [73]

    From local explanations to global understanding with explainable ai for trees.Nature machine intelligence, 2(1):56–67, 2020

    Scott M Lundberg, Gabriel Erion, Hugh Chen, Alex DeGrave, Jordan M Prutkin, Bala Nair, Ronit Katz, Jonathan Himmelfarb, Nisha Bansal, and Su-In Lee. From local explanations to global understanding with explainable ai for trees.Nature machine intelligence, 2(1):56–67, 2020

    2020

  66. [74]

    Do models of mental health based on social media data generalize? InFindings of the association for computational linguistics: EMNLP 2020, pages 3774–3788, 2020

    Keith Harrigian, Carlos Alejandro Aguirre, and Mark Dredze. Do models of mental health based on social media data generalize? InFindings of the association for computational linguistics: EMNLP 2020, pages 3774–3788, 2020

    2020

  67. [75]

    The androids corpus: A new publicly available benchmark for speech based depression detection.Depression, 47:11–9, 2023

    Fuxiang Tao, Anna Esposito, and Alessandro Vinciarelli. The androids corpus: A new publicly available benchmark for speech based depression detection.Depression, 47:11–9, 2023

    2023

  68. [76]

    Robust speech recognition via large-scale weak supervision

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. InInternational conference on machine learning, pages 28492–28518. PMLR, 2023

    2023

  69. [77]

    Modeling depressive patterns in italian discourse: Insights from natural language processing, 2025

    Annapia Borraccino. Modeling depressive patterns in italian discourse: Insights from natural language processing, 2025

    2025

  70. [78]

    Will money increase subjective well-being?Social indicators research, 57 (2):119–169, 2002

    Ed Diener and Robert Biswas-Diener. Will money increase subjective well-being?Social indicators research, 57 (2):119–169, 2002

    2002

  71. [79]

    High income improves evaluation of life but not emotional well-being

    Daniel Kahneman and Angus Deaton. High income improves evaluation of life but not emotional well-being. Proceedings of the national academy of sciences, 107(38):16489–16493, 2010

    2010

  72. [80]

    Happiness, income satiation and turning points around the world.Nature Human Behaviour, 2(1):33–38, 2018

    Andrew T Jebb, Louis Tay, Ed Diener, and Shigehiro Oishi. Happiness, income satiation and turning points around the world.Nature Human Behaviour, 2(1):33–38, 2018

    2018

  73. [81]

    Sadness as an integral part of depression.Dialogues in clinical neuroscience, 10(3):321–327, 2008

    Sabine Mouchet-Mages and Franck J Baylé. Sadness as an integral part of depression.Dialogues in clinical neuroscience, 10(3):321–327, 2008

    2008

  74. [82]

    American psychiatric association Washington, DC, 2013

    DSMTF American Psychiatric Association, D American Psychiatric Association, et al.Diagnostic and statistical manual of mental disorders: DSM-5, volume 5. American psychiatric association Washington, DC, 2013

    2013

  75. [83]

    Language use of depressed and depression-vulnerable college students.Cognition & Emotion, 18(8):1121–1133, 2004

    Stephanie Rude, Eva-Maria Gortner, and James Pennebaker. Language use of depressed and depression-vulnerable college students.Cognition & Emotion, 18(8):1121–1133, 2004

    2004

  76. [84]

    Facebook language predicts depression in medical records

    Johannes C Eichstaedt, Robert J Smith, Raina M Merchant, Lyle H Ungar, Patrick Crutchley, Daniel Preo¸ tiuc- Pietro, David A Asch, and H Andrew Schwartz. Facebook language predicts depression in medical records. Proceedings of the National Academy of Sciences, 115(44):11203–11...

    2018

  77. [85]

    Constructive and unconstructive repetitive thought.Psychological bulletin, 134(2):163, 2008

    Edward R Watkins. Constructive and unconstructive repetitive thought.Psychological bulletin, 134(2):163, 2008

    2008

  78. [86]

    Language-based personality: A new approach to personality in a digital world.Current opinion in behavioral sciences, 18:63–68, 2017

    Ryan L Boyd and James W Pennebaker. Language-based personality: A new approach to personality in a digital world.Current opinion in behavioral sciences, 18:63–68, 2017. 28

    2017

  79. [87]

    Forma mentis networks map how nursing and engineering students enhance their mindsets about innovation and health during professional growth.PeerJ Computer Science, 6:e255, 2020

    Massimo Stella and Anna Zaytseva. Forma mentis networks map how nursing and engineering students enhance their mindsets about innovation and health during professional growth.PeerJ Computer Science, 6:e255, 2020

    2020

  80. [88]

    Rethinking rumination.Perspectives on psychological science, 3(5):400–424, 2008

    Susan Nolen-Hoeksema, Blair E Wisco, and Sonja Lyubomirsky. Rethinking rumination.Perspectives on psychological science, 3(5):400–424, 2008

    2008

  81. [89]

    Linking “big” personality traits to anxiety, depressive, and substance use disorders: a meta-analysis.Psychological bulletin, 136(5):768, 2010

    Roman Kotov, Wakiza Gamez, Frank Schmidt, and David Watson. Linking “big” personality traits to anxiety, depressive, and substance use disorders: a meta-analysis.Psychological bulletin, 136(5):768, 2010

    2010

  82. [90]

    Worry: A cognitive phenomenon intimately linked to affective, physiological, and interpersonal behavioral processes.Cognitive therapy and research, 22(6):561–576, 1998

    Thomas D Borkovec, William J Ray, and Joachim Stober. Worry: A cognitive phenomenon intimately linked to affective, physiological, and interpersonal behavioral processes.Cognitive therapy and research, 22(6):561–576, 1998

    1998

  83. [91]

    Machine psychology.arXiv preprint arXiv:2303.13988, 2023

    Thilo Hagendorff, Ishita Dasgupta, Marcel Binz, Stephanie CY Chan, Andrew Lampinen, Jane X Wang, Zeynep Akata, and Eric Schulz. Machine psychology.arXiv preprint arXiv:2303.13988, 2023. 29 A Supplementary Results: Random Forest Table A.1: Random Forest performance on SWLS scor...

    2023 arXiv

  84. [2026]

    doi: 10.31234/osf.io/7zhvr_v1

    doi

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.