Pith. sign in

REVIEW 3 major objections 6 minor 16 references

Improving Mental Health Screening and Early Risk Detection in Spanish

T0 review · 3 major / 6 minor · reviewed 2026-07-31 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Spanish mental-health language models plus Incremental Context Expansion detect risk earlier on social-media timelines without sacrificing accuracy.

desk verdict Solid Spanish mental-health LMs plus a usable early-detection relabeling recipe; ICE gains are real on the tables but partly confounded with prefix multiplication and an unvalidated SVM onset detector. read the letter →

arxiv 2607.28476 v1 pith:S6NOM7IC submitted 2026-07-30 cs.CL

Andreu Casamayor-Segarra , Vicent Ahuir , Antonio Molina-Marco , Lluís-F. Hurtado This is my paper · ORCID
classification cs.CL
keywords datasetrelabelingearlydetectionIncrementalContextExpansion(ICE)mentalhealthSpanishlanguagemodelssocialmediaanalysistransformersdomain-adaptivepretraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Early risk detection of mental disorders from Spanish social media is held back by scarce domain-specific models and by user-level labels that do not say when symptoms first become visible. This paper builds three Spanish foundation models adapted to mental-health discourse and introduces Incremental Context Expansion (ICE), which automatically finds the first message prefix that is enough evidence of a disorder and turns each user’s history into a sequence of growing, correctly labeled contexts. Fine-tuning the specialized models on those ICE samples yields systems that wait only as long as needed, cut detection latency, and match or beat prior Spanish benchmarks while keeping high F1. The practical payoff is screening that can flag risk sooner from ordinary posts, without requiring a fixed word quota or full history up front.

What carries the argument

Incremental Context Expansion (ICE): an automatic relabeling method that trains a transition detector on user-level labels, locates the first cumulative prefix predicted positive, then emits every shorter prefix as non-symptomatic and every longer prefix as symptomatic, producing context-level training samples that teach both what to predict and when enough evidence has arrived.

What would settle it

Re-run the same early-detection benchmarks after replacing ICE’s automatic transition points with human-annotated onset posts (or with an intentionally delayed/noisy detector); if latency gains and F1 vanish or reverse while full-context accuracy stays flat, the central claim fails.

Watch

Extended reading notes

Core claim

Combining three Spanish mental-health foundation models (domain-adapted RoBERTa and Longformer variants) with training data relabeled by Incremental Context Expansion improves state-of-the-art early-risk detection on the three MentalRisk Spanish benchmarks: the models reduce latency (better ERDE5 and LTP) while preserving or raising classification quality (F1 and ERDE30), including under a no-threshold setting that needs no hand-chosen minimum context.

Load-bearing premise

The transition detector correctly marks the first moment a user’s messages contain enough evidence of the disorder, so earlier messages truly lack evidence and later ones truly have it.

Editorial extensions

If this is right

  • Spanish early-risk systems can operate without a fixed minimum-context threshold and still remain competitive on both speed and accuracy.
  • Publicly released domain-adapted Spanish models become stronger starting points for depression, anxiety, and eating-disorder screening from Telegram-style posts.
  • User-level mental-health corpora can be turned into larger, onset-aware training sets without new human labeling of every prefix.
  • Long-context architectures benefit most when paired with ICE, because almost every incremental sample fits their window.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the same ICE pipeline were applied to other low-resource languages that only have user-level labels, early-detection gains might transfer without new clinical annotation.
  • Diffuse depression language may be the hardest case for the current SVM transition detector; a joint end-to-end onset model could shrink the remaining performance gap on that task.
  • Deployed monitors could expose the inferred transition point as an explanation of “why now,” linking the latency improvement to a concrete evidence boundary.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper contributes (1) three Spanish mental-health foundation models (RoBERTa-es-mental-large, Longformer-es-mental-base/large) obtained by domain-adaptive pre-training of MarIA checkpoints on ~1.9M automatically translated Reddit posts (SWMH+RMHN); (2) Incremental Context Expansion (ICE), which trains a user-level SVM on the same labeled users, finds the first prefix that flips to the positive class (or the mean flip if none), and emits every expanding prefix with the corresponding non-/symptomatic label; and (3) fine-tuned early-detection models on the resulting context-level data. Evaluated on three MentalRisk IberLEF tasks (MR24-DD, MR23-ED, MR23-D) under the official sequential protocol with ERDE5/30, LTP and Macro-F1, the ICE models improve over non-ICE counterparts and several shared-task references, especially in no-threshold (NT) settings that avoid a fixed word minimum, while Longformer variants remain strong under full-context classification (Table 11).

Significance. Spanish mental-health NLP remains under-resourced relative to English; releasing three domain-adapted foundation models (including long-context Longformers) on Hugging Face is a concrete, reusable contribution. ICE is a practical recipe for turning user-level labels into early-detection supervision and, if the onset attribution holds, would be a useful general technique beyond this paper. Empirical gains on official MentalRisk protocols, public models, and transparent comparison to shared-task systems are strengths. The work is incremental rather than paradigm-shifting, but the combination of Spanish DAP models plus a relabeling method aimed at latency is of clear applied interest to the community.

major comments (3)
  1. [§5.1] §5.1 (steps 1–3) and Fig. 3: the transition detector is an SVM trained as a user-level classifier on the same training users whose expanding prefixes it then slices; when no flip is found it falls back to the dataset mean transition. There is no held-out detector accuracy, no human onset annotation, and no stability analysis. This is load-bearing for the claim that ICE “identifies the point at which cumulative messages provide enough evidence” (abstract; §5). On MR23-D, where the paper itself notes diffuse language (§6.4), the risk that targets encode the SVM’s decision surface rather than true symptom onset is material. At minimum, report detector agreement with a small human-annotated onset sample or a held-out detector, and discuss failure modes when the flip never occurs.
  2. [§5.2, Table 18] Table 18 vs §4.2.1/§5.2: ICE multiplies training size by roughly 20–40× (e.g., MR24-DD None 178→7415) by emitting every prefix, while non-ICE baselines use one full-context sample per user. Reported NT gains (LTP 3–4, improved F1/ERDE in Tables 20–25 and Figs. 5–7) therefore confound “correct onset labels” with ordinary multi-instance prefix augmentation and an implicit wait regularizer. Without an ablation that keeps prefix multiplicity but uses random, last-k, or full-history-only labels (or that freezes the detector and varies only the label assignment), the attribution of SOTA/latency gains specifically to ICE’s evidence-point identification is only partly supported. ED still loses to CIMAT-NLP-GTO on ERDE30 and F1 (Table 24).
  3. [§4–§6, Tables 15–25] Tables 11–25 and Figs. 5–7 report point estimates only: no confidence intervals, bootstrap, or significance tests on metric differences versus references or non-ICE ablations. Given small user counts (e.g., validation 37 users on MR23-ED/D) and the sequential protocol’s path dependence, several “improvements” may not be reliable. Add uncertainty estimates or paired tests on the official metrics before claiming state-of-the-art across the three benchmarks.
minor comments (6)
  1. [Abstract, §8] Abstract and §8 claim broad SOTA improvement; Table 24 shows CIMAT still best on ERDE30/F1 for ED. Soften the blanket claim to “improves over transformer references / improves latency–accuracy trade-off under NT.”
  2. [§1] Section order in the roadmap (§1) lists future work (Section 7) before conclusions (Section 8); swap for conventional order.
  3. [Throughout] Typos and naming inconsistencies: “metal-health” (§4.1.1), “Depresion” (Table 25 caption), “RoBERTa-es-m-large” vs “RoBERTa-es-mental-large”, mixed decimal commas/points in tables (e.g., Table 15).
  4. [§4.1.2, §6.4] DAP uses EasyNMT/OPUS-MT English→Spanish Reddit data; §6.4 notes possible affective drift but gives no qualitative check or comparison to native Spanish mental-health text. A short appendix sample would strengthen the foundation-model claim.
  5. [§4.3.1] Hyperparameters (Table 10) and the 100-word threshold are fixed without sensitivity beyond the reported grid; briefly justify why 100 words generalizes across tasks with different mean posts/user.
  6. [§2] Related work could more explicitly contrast ICE with eRisk-style early-alert training and with temporal embedding methods (Couto et al.) beyond the brief mention in §2.

Circularity Check

1 steps flagged · score 1.0 of 10

Empirical benchmark paper; ICE self-labels train prefixes via an SVM fit on the same user-level labels, but reported SOTA/latency claims are held-out test scores, not true-by-construction.

  1. other [§5.1 steps 1–3; Fig. 2–3; Table 18]
    "For the first step, to identify the boundary between non-symptomatic and symptomatic states, we decided to use a SVM-based model... The SVM was trained as a user-level classifier with the original dataset. Later, the classifier was used to identify the transition point by classifying the incrementally expanding partial contexts from a user's history until the prediction is changed from non-symptomatic to symptomatic. ... If the transition point is not found by the detector, we assign the mean value of the transition points of all samples with a detected transition point in the dataset."

    Mild train-side self-labeling only: the transition detector is fit on the same user-level train labels it then uses to mint context-level targets (and multiplies N ~20–40×). That couples ICE targets to the SVM decision surface rather than independent onset annotation. It is not full circularity of the paper’s claims: test metrics are on held-out MentalRisk users vs external systems and are not equal to the SVM fit by construction.

full rationale

This is an empirical Spanish NLP methods paper (domain-adaptive pretraining + ICE relabeling + fine-tuning), not a first-principles derivation. The load-bearing claims are Macro-F1 / ERDE5 / ERDE30 / LTP on the official MentalRisk test partitions against external shared-task reference systems (Tables 15–17, 23–25; §4.3, §5.3). Those test labels and competitor systems are independent of the authors’ training pipeline, so the headline “improves the state-of-the-art” is not forced by definition. ICE does fit a user-level SVM on the same training users whose chronological prefixes it then slices and relabels (§5.1 steps 1–3, Fig. 3; fallback to mean transition when no flip), which is mild self-labeling structure and a validity risk for the “true onset” story—but the paper never presents the SVM transition points themselves as an external scientific prediction, nor does any reported test metric reduce algebraically to that fit. Self-citations (e.g. Casamayor et al. 2024 as a reference system and for the SVM idea) are normal prior-work pointers and are not used as uniqueness theorems that forbid alternatives. No self-definitional loop, no fitted parameter renamed as a held-out prediction, no ansatz smuggled in as a theorem. Score 1 only for the minor train-side self-labeling; central results remain externally falsifiable benchmark numbers.

Assumptions & free parameters 4 free parameters · 5 assumptions · 2 invented entities

The empirical claim rests on standard transformer DAP practice, shared-task early-detection protocol definitions, the unproven quality of MT’d Reddit discourse as Spanish mental-health signal, and ICE’s SVM-defined onset as a proxy for true symptom emergence. Free choices include context thresholds and training hyperparameters; no new physical entities are postulated.

free parameters (4)
  • minimum_context_word_thresholds = 100 words emphasized; grid {0,50,100,150,200}
    NT/50/100/150/200 word gates are chosen and 100 words is promoted as balanced from validation ERDE30 (§4.3.1); not derived from first principles.
  • fine-tuning_hyperparameters = lr=5e-5, epochs=10, batch=16
    AdamW lr 5e-5, 10 epochs, batch 16, weight decay 0.01 fixed across models (Table 10) without reported search.
  • DAP_schedule = 20 epochs, bs=8 per GPU
    20 epochs, batch 8/GPU, eval every 10k steps on 4×A40 (§4.1.3) chosen by authors.
  • ICE_mean_transition_fallback = mean of detected transition indices
    If SVM never flips positive, transition point is set to the mean of detected transitions in the dataset (§5.1 step 1)—a dataset-level imputed constant.
assumptions (5)
  • domain assumption Domain-adaptive continued pretraining on translated mental-health social text yields Spanish representations useful for downstream risk classification.
    Invoked throughout §4; supported empirically in Table 11 but not guaranteed a priori, especially under MT.
  • ad hoc to paper A user-level SVM decision boundary on expanding prefixes identifies a meaningful non-symptomatic→symptomatic transition for relabeling.
    Core of ICE §5.1; justified by prior SVM competitiveness citation, not by human onset annotation.
  • domain assumption Official MentalRisk sequential protocol, Macro-F1, ERDE5/30, and LTP are appropriate measures of early screening quality.
    §3.2–3.3 adopts organizer metrics; ERDE’s latency budget interacts strongly with short posts (~14–15 tokens).
  • domain assumption Automatic OPUS-MT/EasyNMT translation preserves enough affective and clinical signal for DAP.
    §4.1.2; limitations §6.4 acknowledge possible semantic drift.
  • standard math Standard transformer fine-tuning and Longformer long-context attention behave as in prior literature.
    Architecture and training objectives taken as given from RoBERTa/Longformer/MarIA lines.
invented entities (2)
  • Incremental Context Expansion (ICE) relabeling procedure
    purpose: Turn user-level risk labels into ordered context-level training samples with an inferred onset.
    Methodological construct defined in §5; not a physical entity. Independent evidence is only downstream benchmark improvement, not external onset ground truth.
  • RoBERTa-es-mental-large / Longformer-es-mental-base / Longformer-es-mental-large independent evidence
    purpose: Spanish foundation checkpoints specialized to mental-health discourse via DAP.
    New model artifacts; evidence is public weights plus task metrics, not a new theoretical object.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Improving Mental Health Screening and Early Risk Detection in Spanish." pith.science (2026). https://pith.science/paper/S6NOM7IC

@misc{pith2026260728476,
  author       = {Pith},
  title        = {Pith review of: Improving Mental Health Screening and Early Risk Detection in Spanish},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/S6NOM7IC}},
  note         = {Machine review of arXiv:2607.28476}
}
read the original abstract

Early detection of mental health disorders is often limited by the lack of specialized resources in Spanish and the difficulty of analyzing long histories of social media posts. This paper addresses these challenges through three main contributions. First, we introduce three Spanish foundational models specifically adapted to the mental health domain through domain-specific pre-training. Second, we propose Incremental Context Expansion (ICE), an automatic relabeling methodology designed for early detection. ICE identifies the point at which cumulative messages provide enough evidence of a disorder, generating more informative training samples. Third, we provide a set of fine-tuned models using the samples generated with the ICE methodology for early risk detection tasks. Our results on three Spanish benchmarks show that combining these specialized models with ICE improves the state-of-the-art, reducing detection latency while maintaining high performance. All models are publicly available.

Figures

Figures reproduced from arXiv: 2607.28476 by the authors.

Figure 1
Figure 1. Workflow for adapting Spanish LMs to the mental health domain. [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 4
Figure 4. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_4.png] view at source ↗
Figure 4
Figure 4. Fine-tuning workflow process that includes the ICE relabeling methodology. [PITH_FULL_IMAGE:figures/full_fig_p017_4.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 2 linked inside Pith

  1. [1]

    E., and Cohan, A

    [Beltagy et al., 2020] Beltagy, I., Peters, M. E., and Cohan, A. (2020). Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150. [Bulgarelli et al., 2021] Bulgarelli, L., Tavares, A., Pires, F., Camargo, I., et al. (2021). EasyNMT: A Python Library for Easy Access to Multilingual Pretrained Models for Translation and Other NLP Tasks.ht...

    arXiv 2020

  2. [6]

    CEUR-WS.org

    co-located with the 40th International Conference of the Spanish Society for Natural Language Processing (SEPLN 2024), volume 3756 ofCEUR Workshop Proceedings, pages 13–23. CEUR-WS.org. [Gonz´alez-Silot et al., 2023] Gonz ´alez-Silot, S., Mart´ınez-C´amara, E., and L ´opez, L. A. U. (2023). SINAI at Men- talRisk: Using Emotions for Detecting Depression. I...

    2024

  3. [7]

    CEUR-WS.org

    co-located with the 39th Conference of the Spanish Society for Natural Language Processing (SE- PLN 2023), volume 3496 ofCEUR Workshop Proceedings, pages 109–118. CEUR-WS.org. [Gururangan et al., 2020] Gururangan, S., Marasovic, A., Swayamdipta, S., Lo, K., Beltagy, I., Downey, D., and Smith, N. A. (2020). Don’t Stop Pretraining: Adapt Language Models to ...

    arXiv 2023

  4. [10]

    A., and Valencia-Garc ´ıa, R

    [Pan et al., 2023] Pan, R., Garc ´ıa-D´ıaz, J. A., and Valencia-Garc ´ıa, R. (2023). UMUTeam at Mental- RiskES2023@IberLEF: Transformer and Ensemble Learning Models for Early Detection of Eating Disorders and Depression. InProceedings of the Iberian Languages Evaluation F orum (IberLEF

    2023

  5. [11]

    CEUR-WS.org

    co-located with the 39th Conference of the Spanish Society for Natural Language Processing (SEPLN 2023), volume 3496 ofCEUR Work- shop Proceedings, pages 89–98. CEUR-WS.org. [Parapar et al., 2021] Parapar, J., Mart ´ın-Rodilla, P., Losada, D. E., and Crestani, F. (2021). Overview of erisk 2021: Early risk prediction on the internet. InExperimental IR Meet...

    2023

  6. [12]

    and Errecalde, M

    [Thompson and Errecalde, 2023] Thompson, H. and Errecalde, M. (2023). Early Detection of Depression and Eating Disorders in Spanish: UNSL at MentalRiskES

    2023

  7. [13]

    CEUR-WS.org

    co-located with the 39th Conference of the Spanish Society for Natural Language Processing (SEPLN 2023), volume 3496 ofCEUR Workshop Proceedings, pages 99–108. CEUR-WS.org. [Tiedemann et al., 2023] Tiedemann, J., Aulamo, M., Bakshandaeva, D., Boggia, M., Gr ¨onroos, S.-A., Nieminen, T., Raganato A., Scherrer, Y ., Vazquez, R., and Virpioja, S. (2023). Dem...

    2023

  8. [14]

    CEUR-WS.org

    co-located with the 39th Conference of the Spanish Society for Natural Language Processing (SE- PLN 2023), volume 3496 ofCEUR Workshop Proceedings, pages 67–78. CEUR-WS.org. 28 Improving Mental Health Screening and Early Risk Detection in Spanish [Vaswani et al., 2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L...

    2023

Show all 16 references
  1. [15]

    [Wu and Dredze, 2020] Wu, S

    Curran Associates, Inc. [Wu and Dredze, 2020] Wu, S. and Dredze, M. (2020). Are All Languages Created Equal in Multilingual BERT? InProceedings of the 5th Workshop on Representation Learning for NLP, pages 120–130, Online. Association for Computational Linguistics. [Zhou et al...

    2020

  2. [16]

    A way of making Europe,

    co-located with the 40th Conference of the Spanish Society for Natural Language Processing (SEPLN 2024), volume 3756 ofCEUR Workshop Proceedings. CEUR-WS.org. Acknowledgment This work was partially supported by the Spanish Agencia Estatal de Investigaci ´on (AEI) under grant P...

    2024

  3. [30]

    M., Moreno-Mu ˜noz, A., Plaza-Del-Arco, F

    [M´armol Romero et al., 2024] M ´armol Romero, A. M., Moreno-Mu ˜noz, A., Plaza-Del-Arco, F. M., Molina- Gonz´alez, M. D., and Montejo-R ´aez, A. (2024). MentalRiskES: A New Corpus for Early Detection of Mental Disorders in Spanish. In Calzolari, N., Kan, M.-Y ., Hoste, V ., L...

    2024

  4. [2020]

    [Conneau et al., 2020] Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V ., Wenzek, G., Guzm´an, F., Grave, E., Ott, M., Zettlemoyer, L., and Stoyanov, V . (2020). Unsupervised Cross-lingual Representation Learning at Scale. In Jurafsky, D., Chai, J., Schluter, N., and Tetr...

    2020

  5. [2023]

    CEUR-WS.org

    co-located with the 39th Conference of the Spanish Society for Natural Language Processing (SEPLN 2023), volume 3496 ofCEUR Workshop Proceedings, pages 179–188. CEUR-WS.org. 26 Improving Mental Health Screening and Early Risk Detection in Spanish [European Commission, 2023] Eu...

    2023

  6. [2024]

    CEUR-WS.org

    co-located with 40th International Conference of the Spanish Soci- ety for Natural Language Processing (SEPLN 2024), volume 3756 ofCEUR Workshop Proceedings, pages 24–33. CEUR-WS.org. [Ca˜nete et al., 2020] Ca ˜nete, J., Chaperon, G., Fuentes, R., Ho, J.-H., Kang, H., and P ´e...

    2024

  7. [2025]

    [Fern´andez-Hern´andez et al., 2024] Fern ´andez-Hern´andez, J., Fabregat, H., Duque, A., Araujo, L., and Mart ´ınez- Romo, J. (2024). UNED-GELP at MentalRiskES 2024: Transformer-Based Encoders and Similarity Techniques for Early Risk Prediction of Mental Disorders. InProceedi...

    2024

  8. [2125]

    [Losada et al., 2019] Losada, D

    CEUR-WS. [Losada et al., 2019] Losada, D. E., Crestani, F., and Parapar, J. (2019). Overview of erisk 2019 early risk prediction on the internet. InExperimental IR Meets Multilinguality, Multimodality, and Interaction, Berlin, Heidelberg. Springer-Verlag. [Losada et al., 2020]...

    2019

Pith tools

Reviewed July 31, 2026 · model on record in the stance chip above.