pith. sign in

arxiv: 2606.07714 · v1 · pith:IB4WKZJQnew · submitted 2026-06-05 · 💻 cs.LG · cs.AI· cs.HC

Beyond Accuracy: Interpreting Topic Representation in Suicide Ideation Detection Models

Hamideh Ghanadian , Isar Nejadgholi , Hussein Al Osman This is my paper

Pith reviewed 2026-06-27 22:28 UTC · model grok-4.3

classification 💻 cs.LG cs.AIcs.HC
keywords suicide ideation detectiontopic augmentationmodel interpretabilitypsychosocial risk factorsinternal representationsgeometric analysismental health AIdata augmentation
0
0 comments X

The pith

Topic-aware augmentation makes suicide ideation models represent risk factors like immigration and family issues more clearly and distinctly in their internal space.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Suicide ideation detection models are normally assessed only by overall accuracy, leaving open how they internally encode psychologically meaningful risk factors. The authors train models on both the original data and on datasets augmented with topic information, then apply visualization and geometric analysis to measure the coherence and separability of topic-related features. They report that the augmented versions produce greater clarity and distinctness for underrepresented factors such as immigration, family issues, and financial crisis. This matters in high-stakes mental health settings because more structured internal representations could support transparency and safer deployment.

Core claim

Models trained with topic-aware augmentation encode underrepresented psychosocial risk factors such as immigration, family issues, and financial crisis with greater clarity and distinctness in their internal representation space compared to models trained on the original dataset, as measured by visualization and geometric analysis.

What carries the argument

Topic-aware augmentation of the training dataset, applied before model training to increase the coherence and separability of topic-related features in the learned representation space.

If this is right

  • Augmentation improves not only accuracy but also the structured nature of internal representations.
  • Underrepresented psychosocial topics become more separable, potentially aiding human inspection of model behavior.
  • Clearer topic encoding could help identify which risk factors a model is actually using for its decisions.
  • The approach offers a route to more interpretable models without changing the underlying architecture.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the geometric measures align with clinical understanding of risk factors, the same pipeline could be applied to other mental-health classification tasks.
  • The findings suggest a possible way to diagnose and mitigate under-representation of certain demographic or situational topics during data preparation.
  • Models with more distinct topic clusters might generalize better when the distribution of risk factors shifts across populations or time periods.

Load-bearing premise

The chosen visualization and geometric analysis methods correctly isolate and measure psychologically meaningful risk factors rather than topic-modeling artifacts or dataset-specific patterns.

What would settle it

Re-running the geometric separability analysis after randomly shuffling the topic labels on the same model embeddings and obtaining comparable scores would indicate that the measures are not capturing meaningful risk-factor structure.

read the original abstract

Suicide ideation detection models are typically evaluated using aggregate performance metrics, yet little is known about how they internally represent psychologically meaningful risk factors. In high-stakes mental health applications, understanding these internal representations is essential for safety, transparency, and responsible deployment. In this work, we move beyond accuracy and analyze how suicide detection models trained on original and topic-augmented datasets encode psychological risk factors in their internal representation space. Using visualization and geometric analysis, we examine the coherence and separability of topic-related features. Our results show that topic-aware augmentation increases the clarity and distinctness of underrepresented psychosocial risk factors such as immigration, family issues, and financial crisis. These findings suggest that augmentation not only improves model performance but also leads to more structured and interpretable internal representations.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 0 minor

Summary. The manuscript claims that suicide ideation detection models trained on topic-augmented datasets encode underrepresented psychosocial risk factors (immigration, family issues, financial crisis) with greater clarity and distinctness in their internal representation space than models trained on the original data, as demonstrated through visualization and geometric analysis; this is presented as evidence that topic-aware augmentation improves not only performance but also the structure and interpretability of learned representations.

Significance. If the geometric analysis validly isolates psychologically meaningful structure rather than augmentation-induced artifacts, the work would contribute to the literature on interpretability for high-stakes mental-health models by shifting focus from aggregate accuracy to internal representation properties.

major comments (2)
  1. [geometric analysis and results] The central claim requires that observed increases in cluster distinctness reflect improved encoding of the named psychosocial constructs. No control is described that preserves the augmentation procedure while breaking its semantic alignment with the target risk factors (e.g., by substituting unrelated topics), leaving open the possibility that any measured separability is a mechanical consequence of the augmentation rather than evidence of psychological validity.
  2. [Abstract] No quantitative results, error bars, dataset sizes, specific geometric metrics (e.g., silhouette scores, inter-cluster distances), or method descriptions appear in the provided abstract; without these, the support for the stated claim cannot be evaluated.

Simulated Author's Rebuttal

2 responses · 0 unresolved

Thank you for the constructive feedback. We address each major comment below and indicate the planned revisions.

read point-by-point responses

  1. Referee: [geometric analysis and results] The central claim requires that observed increases in cluster distinctness reflect improved encoding of the named psychosocial constructs. No control is described that preserves the augmentation procedure while breaking its semantic alignment with the target risk factors (e.g., by substituting unrelated topics), leaving open the possibility that any measured separability is a mechanical consequence of the augmentation rather than evidence of psychological validity.

    Authors: We agree that a control preserving the augmentation mechanics but disrupting semantic alignment with the psychosocial factors would help rule out artifacts. In revision we will add such an experiment by substituting unrelated topics and recomputing the geometric metrics for comparison. revision: yes

  2. Referee: [Abstract] No quantitative results, error bars, dataset sizes, specific geometric metrics (e.g., silhouette scores, inter-cluster distances), or method descriptions appear in the provided abstract; without these, the support for the stated claim cannot be evaluated.

    Authors: We will revise the abstract to include the requested quantitative elements: dataset sizes, specific metrics such as silhouette scores and inter-cluster distances with error bars, and concise method descriptions. revision: yes

Circularity Check

0 steps flagged

No significant circularity; empirical comparison only

full rationale

The paper is an empirical study that trains models on original versus topic-augmented data and reports visualization/geometric metrics on the resulting embeddings. No equations, fitted parameters renamed as predictions, self-definitional constructs, or load-bearing self-citation chains appear in the text. Claims rest on experimental outcomes rather than any derivation that reduces to its own inputs by construction. This is the most common honest finding for non-mathematical ML papers.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract contains no explicit free parameters, axioms, or invented entities; the contribution is framed as an empirical observation.

pith-pipeline@v0.9.1-grok · 5665 in / 962 out tokens · 22432 ms · 2026-06-27T22:28:08.381192+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

79 extracted references · 12 canonical work pages · 4 internal anchors

  1. [1]

    arXiv preprint arXiv:2306.09390 , year=

    ChatGPT for Suicide Risk Assessment on Social Media: Quantitative Evaluation of Model Performance, Potentials and Limitations , author=. arXiv preprint arXiv:2306.09390 , year=

    work page arXiv

  2. [2]

    IEEE Access , year=

    Socially Aware Synthetic Data Generation for Suicidal Ideation Detection Using Large Language Models , author=. IEEE Access , year=

  3. [3]

    JMIR Formative Research , volume=

    Improving Suicidal Ideation Detection in Social Media Posts: Topic Modeling and Synthetic Data Augmentation Approach , author=. JMIR Formative Research , volume=. 2025 , publisher=

    2025

  4. [4]

    1831 , url =

    George Merriam, Charles Merriam , title =. 1831 , url =

  5. [5]

    Suicide and Life-Threatening Behavior , volume=

    Hopelessness as a predictor of suicide ideation in depressed male and female adolescent youth , author=. Suicide and Life-Threatening Behavior , volume=. 2019 , publisher=

    2019

  6. [6]

    BMC psychiatry , volume=

    Psychosocial, psychiatric and work-related risk factors associated with suicide in Ireland: optimised methodological approach of a case-control psychological autopsy study , author=. BMC psychiatry , volume=. 2019 , publisher=

    2019

  7. [7]

    Frontiers in public health , volume=

    A mixed-methods study protocol on factors contributing to suicide clusters among Native American youth in a northern plains reservation , author=. Frontiers in public health , volume=. 2024 , publisher=

    2024

  8. [8]

    The Lancet Psychiatry , volume=

    The psychology of suicidal behaviour , author=. The Lancet Psychiatry , volume=. 2014 , publisher=

    2014

  9. [9]

    Transactions of the Association for Computational Linguistics , volume=

    Anchored correlation explanation: Topic modeling with minimal domain knowledge , author=. Transactions of the Association for Computational Linguistics , volume=. 2017 , publisher=

    2017

  10. [10]

    International Symposium on Artificial Intelligence and Robotics , pages=

    Experimental comparison of three topic modeling methods with LDA, Top2Vec and BERTopic , author=. International Symposium on Artificial Intelligence and Robotics , pages=. 2023 , organization=

    2023

  11. [11]

    Archives of Suicide Research , volume=

    Long-term suicide risk in anxiety—the Lundby study 1947--2011 , author=. Archives of Suicide Research , volume=. 2016 , publisher=

    1947

  12. [12]

    Psychiatry research , volume=

    Risk-taking behaviors and stressors differentially predict suicidal preparation, non-fatal suicide attempts, and suicide deaths , author=. Psychiatry research , volume=. 2018 , publisher=

    2018

  13. [13]

    2024 , publisher=

    Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet , author=. 2024 , publisher=

    2024

  14. [14]

    Distill , volume=

    Zoom in: An introduction to circuits , author=. Distill , volume=

  15. [15]

    Toy Models of Superposition

    Toy models of superposition , author=. arXiv preprint arXiv:2209.10652 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  16. [16]

    Vision research , volume=

    Sparse coding with an overcomplete basis set: A strategy employed by V1? , author=. Vision research , volume=. 1997 , publisher=

    1997

  17. [17]

    IEEE Transactions on signal processing , volume=

    K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation , author=. IEEE Transactions on signal processing , volume=. 2006 , publisher=

    2006

  18. [18]

    Transformer Circuits Thread , year =

    Towards Monosemanticity: Decomposing Language Models With Dictionary Learning , author =. Transformer Circuits Thread , year =

  19. [19]

    Anthropic Research , year =

    Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet , author =. Anthropic Research , year =

  20. [20]

    2024 , journal =

    Sparse Autoencoders for Interpretability , author =. 2024 , journal =

    2024

  21. [21]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    On Concept-Based Explanations in Deep Neural Networks , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =

  22. [22]

    Proceedings of the 37th International Conference on Machine Learning (ICML) , year =

    Concept Bottleneck Models , author =. Proceedings of the 37th International Conference on Machine Learning (ICML) , year =

  23. [23]

    Proceedings of the 37th International Conference on Machine Learning (ICML) , year =

    Concept Whitening for Interpretable Image Recognition , author =. Proceedings of the 37th International Conference on Machine Learning (ICML) , year =

  24. [24]

    Nature machine intelligence , volume=

    Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead , author=. Nature machine intelligence , volume=. 2019 , publisher=

    2019

  25. [25]

    ACM Computing Surveys , year=

    Concept-based explainable artificial intelligence: A survey , author=. ACM Computing Surveys , year=

  26. [26]

    2023 , howpublished =

    Towards Monosemanticity: Decomposing Language Models With Dictionary Learning , author =. 2023 , howpublished =

    2023

  27. [27]

    arXiv preprint arXiv:2410.21331 , year=

    Beyond Interpretability: The Gains of Feature Monosemanticity on Model Robustness , author=. arXiv preprint arXiv:2410.21331 , year=

    work page arXiv

  28. [28]

    arXiv preprint arXiv:2410.08201 , year=

    Efficient dictionary learning with switch sparse autoencoders , author=. arXiv preprint arXiv:2410.08201 , year=

    work page arXiv

  29. [29]

    Explainable Depression Detection Using Large Language Models on Social Media Data

    Wang, Yuxi and Inkpen, Diana and Kirinde Gamaarachchige, Prasadith. Explainable Depression Detection Using Large Language Models on Social Media Data. Proceedings of the 9th Workshop on Computational Linguistics and Clinical Psychology (CLPsych 2024). 2024

    2024

  30. [30]

    Distill , volume=

    Feature visualization , author=. Distill , volume=

  31. [31]

    Distill , volume=

    The building blocks of interpretability , author=. Distill , volume=

  32. [32]

    Sparse Autoencoders Find Highly Interpretable Features in Language Models

    Sparse autoencoders find highly interpretable features in language models , author=. arXiv preprint arXiv:2309.08600 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  33. [33]

    International conference on machine learning , pages=

    Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav) , author=. International conference on machine learning , pages=. 2018 , organization=

    2018

  34. [34]

    arXiv preprint arXiv:2307.01900 , year=

    Concept-Based Explanations to Test for False Causal Relationships Learned by Abusive Language Classifiers , author=. arXiv preprint arXiv:2307.01900 , year=

    work page arXiv

  35. [35]

    Hua et al

    Large Language Models in Mental Health Care: a Scoping Review , author=. arXiv preprint arXiv:2401.02984 , year=

    work page arXiv

  36. [36]

    arXiv preprint arXiv:2206.03945 , year=

    Challenges in applying explainability methods to improve the fairness of NLP models , author=. arXiv preprint arXiv:2206.03945 , year=

    work page arXiv

  37. [37]

    Finetuned Language Models Are Zero-Shot Learners

    Finetuned language models are zero-shot learners , author=. arXiv preprint arXiv:2109.01652 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  38. [38]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Overlooked factors in concept-based explanations: Dataset choice, concept learnability, and human capability , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  39. [39]

    arXiv preprint arXiv:2205.03302 , year=

    Necessity and sufficiency for explaining text classifiers: A case study in hate speech detection , author=. arXiv preprint arXiv:2205.03302 , year=

    work page arXiv

  40. [40]

    arXiv preprint arXiv:2210.10689 , year=

    Towards Procedural Fairness: Uncovering Biases in How a Toxic Language Classifier Uses Sentiment Information , author=. arXiv preprint arXiv:2210.10689 , year=

    work page arXiv

  41. [41]

    Neural computation , volume=

    Training with noise is equivalent to Tikhonov regularization , author=. Neural computation , volume=. 1995 , publisher=

    1995

  42. [42]

    Proceedings of the VLDB endowment

    Snorkel: Rapid training data creation with weak supervision , author=. Proceedings of the VLDB endowment. International conference on very large data bases , volume=. 2017 , organization=

    2017

  43. [43]

    JBI Evidence Implementation , volume=

    Guidance for conducting systematic scoping reviews , author=. JBI Evidence Implementation , volume=. 2015 , publisher=

    2015

  44. [44]

    International journal of social research methodology , volume=

    Scoping studies: towards a methodological framework , author=. International journal of social research methodology , volume=. 2005 , publisher=

    2005

  45. [45]

    Journal of public health , volume=

    ‘Scoping the scope’of a cochrane review , author=. Journal of public health , volume=. 2011 , publisher=

    2011

  46. [46]

    BMC medical research methodology , volume=

    Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach , author=. BMC medical research methodology , volume=. 2018 , publisher=

    2018

  47. [47]

    Archives of general psychiatry , volume=

    Predictors of prospectively examined suicide attempts among youth with bipolar disorder , author=. Archives of general psychiatry , volume=. 2012 , publisher=

    2012

  48. [48]

    American family physician , volume=

    Schizophrenia: a review , author=. American family physician , volume=

  49. [49]

    The British Journal of Psychiatry , volume=

    Schizophrenia and suicide: systematic review of risk factors , author=. The British Journal of Psychiatry , volume=. 2005 , publisher=

    2005

  50. [50]

    Medicina , volume=

    Suicide in schizophrenia: an educational overview , author=. Medicina , volume=. 2019 , publisher=

    2019

  51. [51]

    Psychiatric services , volume=

    Chronic suicidality among patients with borderline personality disorder , author=. Psychiatric services , volume=. 2002 , publisher=

    2002

  52. [52]

    American Journal of Psychiatry , volume=

    Characteristics of borderline personality disorder associated with suicidal behavior , author=. American Journal of Psychiatry , volume=. 1997 , publisher=

    1997

  53. [53]

    Psychiatry research , volume=

    Predictors of suicidal ideation in a community sample: roles of anger, self-esteem, and depression , author=. Psychiatry research , volume=. 2014 , publisher=

    2014

  54. [54]

    Child abuse & neglect , volume=

    Sexual abuse: A sociological perspective , author=. Child abuse & neglect , volume=. 1982 , publisher=

    1982

  55. [55]

    BMC psychiatry , volume=

    The effect of sexual abuse and dissociation on suicide attempt , author=. BMC psychiatry , volume=. 2022 , publisher=

    2022

  56. [56]

    International journal of public health , volume=

    Sexual and physical abuse in childhood is associated with depression and anxiety over the life course: systematic review and meta-analysis , author=. International journal of public health , volume=. 2014 , publisher=

    2014

  57. [57]

    Comprehensive psychiatry , volume=

    Factors associated with suicide: Case-control study in South Tyrol , author=. Comprehensive psychiatry , volume=. 2018 , publisher=

    2018

  58. [58]

    Journal of affective disorders , volume=

    Clinical and course characteristics of depression and all-cause mortality: a prospective population-based study , author=. Journal of affective disorders , volume=. 2016 , publisher=

    2016

  59. [59]

    Journal of psychiatric research , volume=

    Estimating the risk of suicide associated with mental disorders: A systematic review and meta-regression analysis , author=. Journal of psychiatric research , volume=. 2021 , publisher=

    2021

  60. [60]

    BMC psychiatry , volume=

    Suicide in young adults: psychiatric and socio-economic factors from a case--control study , author=. BMC psychiatry , volume=. 2014 , publisher=

    2014

  61. [61]

    Comprehensive psychiatry , volume=

    Is impulsivity a link between childhood abuse and suicide? , author=. Comprehensive psychiatry , volume=. 2010 , publisher=

    2010

  62. [62]

    A meta-analysis , author=

    Suicide as an outcome for mental disorders. A meta-analysis , author=. British journal of psychiatry , volume=. 1997 , publisher=

    1997

  63. [63]

    The Lancet , volume=

    Borderline personality disorder , author=. The Lancet , volume=. 2004 , publisher=

    2004

  64. [64]

    American Journal of Psychiatry , volume=

    Prediction of the 10-year course of borderline personality disorder , author=. American Journal of Psychiatry , volume=. 2006 , publisher=

    2006

  65. [65]

    Nursing Clinics , volume=

    Risk factors for suicide in men , author=. Nursing Clinics , volume=. 2023 , publisher=

    2023

  66. [66]

    The Lancet Public Health , volume=

    Individual-level risk factors for suicide mortality in the general population: an umbrella review , author=. The Lancet Public Health , volume=. 2023 , publisher=

    2023

  67. [67]

    Psychological medicine , pages=

    Risk factors for suicide reattempt: a systematic review and meta-analysis , author=. Psychological medicine , pages=. 2024 , publisher=

    2024

  68. [68]

    Acta Psychiatrica Scandinavica , volume=

    Meta-analysis of clinical risk factors for suicide among people presenting to emergency departments and general hospitals with suicidal thoughts and behaviours , author=. Acta Psychiatrica Scandinavica , volume=. 2023 , publisher=

    2023

  69. [69]

    Progress in Neuro-Psychopharmacology and Biological Psychiatry , volume=

    Chronic pain and suicide risk: a comprehensive review , author=. Progress in Neuro-Psychopharmacology and Biological Psychiatry , volume=. 2018 , publisher=

    2018

  70. [70]

    Human Behavior and Emerging Technologies , volume=

    Identification of Risk Factors for Suicide and Insights for Developing Suicide Prevention Technologies: A Systematic Review and Meta-Analysis , author=. Human Behavior and Emerging Technologies , volume=. 2023 , publisher=

    2023

  71. [71]

    PloS one , volume=

    Suicide capability within the ideation-to-action framework: A systematic scoping review , author=. PloS one , volume=. 2022 , publisher=

    2022

  72. [72]

    2007 , publisher=

    Manic-depressive illness: bipolar disorders and recurrent depression , author=. 2007 , publisher=

    2007

  73. [73]

    Pediatrics , volume=

    Childhood sexual abuse and suicidal behavior: a meta-analysis , author=. Pediatrics , volume=. 2014 , publisher=

    2014

  74. [74]

    Clinical psychology review , volume=

    Linguistic features of suicidal thoughts and behaviors: A systematic review , author=. Clinical psychology review , volume=. 2022 , publisher=

    2022

  75. [75]

    Information Processing & Management , volume=

    Leveraging enhanced BERT models for detecting suicidal ideation in Thai social media content amidst COVID-19 , author=. Information Processing & Management , volume=. 2024 , publisher=

    2024

  76. [76]

    Proceedings of the Ninth Workshop on Computational Linguistics and Clinical Psychology, Association for Computational Linguistics , year=

    Extraction and Summarization of Suicidal Ideation Evidence in Social Media Content Using Large Language Models , author=. Proceedings of the Ninth Workshop on Computational Linguistics and Clinical Psychology, Association for Computational Linguistics , year=

  77. [77]

    Minneapolis

    Zirikly, Ayah and Resnik, Philip and Uzuner,. Proceedings of the Sixth Workshop on Computational Linguistics and Clinical Psychology , location="Minneapolis", month="June", day="6", year=

  78. [78]

    Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic , pages=

    Expert, crowdsourced, and machine assessment of suicide risk via online postings , author=. Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic , pages=

  79. [79]

    UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction

    Umap: Uniform manifold approximation and projection for dimension reduction , author=. arXiv preprint arXiv:1802.03426 , year=

    work page internal anchor Pith review Pith/arXiv arXiv