Pith. sign in

REVIEW 3 major objections 41 references

LLMs know CBT theory but still default to validation; structured guidance barely moves them.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 06:23 UTC pith:BBK5Z4L3

load-bearing objection Solid diagnostic of the CBT knowledge–application gap plus a reusable (if uncalibrated) behavior metric; small synthetic N and low rater agreement keep it provisional but still worth referee time. the 3 major comments →

arxiv 2607.02885 v2 pith:BBK5Z4L3 submitted 2026-07-03 cs.CL cs.AIcs.HCcs.IR

Where do LLMs Fall Short in CBT-Guided Affective Reasoning?

Vaishnavi Sinha , Pooja Guttal , Pranay Deep Reddy Katike , Vishal Sinha , Gerald Ndawula , Lira Yoon , Andrea Kleinsmith , Manas Gaur This is my paper
classification cs.CL cs.AIcs.HCcs.IR
keywords cognitive behavioral therapylarge language modelsclinical knowledge groundingmental healthvalence–arousallinguistic entrainmentProtocol Leverage Forceaffective reasoning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Large language models score high on CBT licensing-exam questions yet collapse into validation and reflection when talking to users in distress. The authors treat therapeutic dialogue as controlled affective reasoning: they decompose a user’s story into Beck’s cognitive model, ground it in SNOMED CT concepts verified by natural-language inference, then force the model to choose among three CBT strategies via multiple chains of thought. They introduce Protocol Leverage Force (F) to measure how far any of this actually shifts the model’s reply away from its default. Across three open-weight models and fourteen case studies, F stays around one percent; every model remains biased toward validation. The paper therefore claims that knowing CBT is not the same as practicing it, and supplies a concrete instrument for measuring that gap.

Core claim

CBT knowledge alone does not ensure effective application. Even when user narratives are decomposed into Beck’s Cognitive Conceptualization structure, grounded in NLI-validated SNOMED CT concepts, and routed through Multiple Chain-of-Thought selection among Validation & Reflection, Socratic Questioning, and Alternative Perspectives, the Protocol Leverage Force F remains only ~1.2–1.3 percent and all models stay biased toward Validation & Reflection.

What carries the argument

Protocol Leverage Force (F): a centrifugal-force-inspired scalar that quantifies how far a CBT-guided response reorients the model away from its default baseline manifold and toward the patient’s surfaced cognitive state.

Load-bearing premise

Fourteen short synthetic dialogues whose later user turns are themselves generated by an LLM are a fair proxy for real therapeutic strategy selection and emotional trajectories.

What would settle it

Run the identical MCoT pipeline on a larger set of real, multi-session CBT transcripts with human clients and measure whether mean F rises substantially above 1.3 percent and the Validation bias disappears.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Simply stuffing CBT definitions into a single chain-of-thought prompt is insufficient to change model behavior.
  • Multiple-chain selection improves strategy choice yet still cannot overcome the models’ native preference for validation.
  • Affective-computing systems now have a behavior-level metric (F) that can diagnose whether guidance is actually moving the response manifold.
  • Surface linguistic entrainment can rise while deeper Socratic or alternative-perspective work remains rare.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same F metric could be reused to test whether fine-tuning or mechanistic interventions close the knowledge–application gap more effectively than prompting.
  • Persistent Validation bias may reflect training-data priors that reward agreeableness; F offers a way to audit those priors in other high-stakes dialogue domains.
  • If F remains near zero even after stronger grounding, the field may need new architectures that keep the cognitive model active at every decoding step rather than only at selection time.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 0 minor

Summary. The paper argues that LLMs possess strong theoretical CBT knowledge (up to 96% on CBT-Bench MCQs) yet default to Validation & Reflection in dialogue. It introduces a knowledge-guided pipeline: Beck PBCCD-style User Cognitive Model extraction, SNOMED CT retrieval filtered by NLI, and Multiple Chain-of-Thought selection among Validation & Reflection (V), Socratic Questioning (SQ), and Alternative Perspectives (AP). The central instrument is Protocol Leverage Force F (Eq. 2), a scalar that measures how far MCoT responses reorient from the model’s baseline manifold toward the patient’s embedding. Across Gemma3-12B, Mistral-7B and GPT-OSS-20B on 14 RealCBT-derived 10-turn synthetic cases, mean F remains ~1.18–1.34% (Table V); Wilcoxon tests (Table IV) show directional shifts for non-default protocols, yet all models stay V-biased. Supporting evidence includes nCLiD entrainment, valence–arousal trajectories versus RealCBT, and human ratings with low inter-rater agreement.

Significance. If the knowledge–application gap is real, the work supplies the affective-computing community with a concrete instrumentation package (PBCCD + NLI-grounded SNOMED + MCoT + F) rather than another accuracy number. The explicit separation of theoretical CBT-Bench scores from behavioral reorientation, the public code repository, and the multi-metric evaluation (entrainment, VAD arcs, expert preference) are genuine strengths. Even a modest, well-documented failure of prompting to overcome V-bias is useful for the field, provided the metric’s scale and the synthetic-user proxy are clarified.

major comments (3)
  1. Eq. (2) and Table V: F is reported as ~1.2–1.3% and treated as evidence that CBT knowledge fails to reorient models. No external calibration is supplied—no human-therapist upper bound, no random-protocol or null-intervention baseline, and no demonstration that F can ever exceed a few percent under any intervention. Without a scale, the numerical value is consistent both with “knowledge does not transfer” and with “F is simply insensitive.” A calibration experiment (or at least an explicit statement of the metric’s dynamic range) is load-bearing for the central claim.
  2. §III-A and Ethical Impact Statement: All subsequent user turns after the seed RealCBT utterance are generated by GPT-4o-mini (fixed 10-turn sessions, self-harm excluded). A simulated user that itself defaults to high-agreeableness language can keep the patient embedding near the therapist model’s native manifold, mechanically suppressing Δ and therefore F. The paper needs either a human-user control, an ablation that freezes user turns, or a quantitative argument that the synthetic-user bias does not systematically understate strategy shift.
  3. Table VII: Expert Keep rates are modest (22–57%) and Fleiss κ is near chance (0.024–0.123). The manuscript correctly notes subjectivity, yet still interprets low F as a clean failure of application. When human judges cannot agree on the preferred strategy, the small F cannot be read as decisive evidence that the models failed to apply CBT; the evaluation target itself may be under-specified. Strengthening the human protocol or reporting per-principle preference conditioned on rater confidence would tighten this link.

Circularity Check

0 steps flagged

No load-bearing circularity: F is an independent post-hoc behavior metric; knowledge-application gap is measured, not derived by construction from the inputs.

full rationale

The paper's central claim is empirical, not a first-principles derivation that reduces to its premises. CBT-Bench Q&A accuracies (Table I, up to 96%) are obtained on an external licensing-exam MCQ set and are independent of the dialogue experiments. Protocol Leverage Force F (Eq. 2) is defined from cosine distances, protocol weights, and a small baseline-perturbation term ε; it is computed after generation, not fitted to force a small value, and no external upper-bound calibration is claimed as a prediction. MCoT strategy selection, Beck CCD decomposition, and SNOMED+NLI grounding are procedural inputs whose effect is measured by F, entrainment, valence-arousal, and human ratings; the small observed F (~1.2-1.3%, Table V) and residual V bias are results, not tautologies. Mild non-circular concern exists that GPT-4o-mini generates the synthetic user turns while open-weight models act as therapists, which can compress Δ, but this is a proxy limitation, not a definitional loop or self-citation chain. No uniqueness theorem, ansatz, or renamed known result is load-bearing. Score 1 only for the ordinary use of related LLMs in the evaluation pipeline; the derivation chain itself is self-contained against the paper's own equations and external benchmarks.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 2 invented entities

The central claim rests on a small set of modeling and evaluation choices rather than free parameters fitted to maximize F. The invented metric F and the synthetic-user pipeline are the main novel constructs; clinical ontologies and Beck's CCD are taken as standard domain background.

free parameters (4)
  • top-k SNOMED retrieval = 5
    k=5 is chosen by hand for 'balancing relevance coverage with noise reduction'; no ablation is reported.
  • nCLiD look-ahead window k = 2
    k=2 is fixed following prior work; sensitivity not shown.
  • session length = 10
    Fixed at 10 turns because 'instruction-following degrades over extended interactions'; arbitrary relative to real 60-turn RealCBT sessions.
  • baseline-perturbation ε = 3 paraphrases
    Mean distance across three prompt paraphrases; the number of paraphrases and the paraphrase generator are free design choices that enter the denominator of F.
axioms (4)
  • domain assumption Beck's Cognitive Conceptualization Diagram (triggers, automatic thoughts, emotions, behaviors) is an adequate and complete decomposition for selecting among V/SQ/AP strategies.
    Invoked throughout Section III-B; the paper notes the missing Strength-Based CCD as a limitation but still treats PBCCD as sufficient for the main experiments.
  • domain assumption SNOMED CT mental-health subgraph plus NLI entailment/neutral/contradiction labels supply clinically valid grounding that improves affective reasoning.
    Section III-C; no external clinical validation of the retrieved concepts is provided beyond the NLI classifier.
  • domain assumption Cosine distance in sentence-embedding space is a valid proxy for both linguistic entrainment (nCLiD) and therapeutic reorientation (Δ inside F).
    Equations (1)–(2); the paper itself cites Delaherche et al. cautioning that surface metrics miss affective depth.
  • ad hoc to paper The three discrete CBT principles (Validation & Reflection, Socratic Questioning, Alternative Perspectives) exhaust the relevant intervention space for the studied dialogues.
    Section III-D; other CBT techniques (behavioral activation, exposure, etc.) are omitted by design.
invented entities (2)
  • Protocol Leverage Force (F) no independent evidence
    purpose: Scalar that quantifies how far a CBT-guided response moves a model away from its default baseline manifold toward the patient's cognitive state.
    Defined by analogy to Huygens centrifugal force (Eq. 2); no independent theoretical derivation or external validation outside this paper.
  • User Cognitive Model (PBCCD extraction) no independent evidence
    purpose: Structured intermediate representation (triggers/thoughts/emotions/behaviors) that is fed to the MCoT selector.
    Extracted by GPT-4o-mini with a prompt adapted from PATIENT-Ψ; treated as ground truth for downstream strategy choice without human clinical verification of the extractions.

pith-pipeline@v1.1.0-grok45 · 21688 in / 3335 out tokens · 28014 ms · 2026-07-12T06:23:02.573139+00:00 · methodology

0 comments
read the original abstract

Cognitive Behavioral Therapy (CBT) provides a structured framework for understanding a user's mental state by examining the interaction between cognitive and behavioral factors. However, out-of-the-box LLMs respond fluently and empathetically, yet collapse into validation & reflection, regardless of what the user actually needs. They know theoretical CBT (scoring up to 96% accuracy on licensing exam questions) but fail to apply it effectively. We explore this gap with a knowledge-guided framework that treats CBT dialogue as controlled affective reasoning: user narratives are decomposed into Beck's Cognitive Conceptualization structure, grounded in clinical SNOMED CT concepts validated via Natural Language Inference, and a Multiple Chain-of-Thought (MCoT) strategy selection between Validation & Reflection, Socratic Questioning, or Alternative Perspectives. To measure whether such guidance actually changes behavior, we introduce the Protocol Leverage Force (F), a behavior-level metric that captures how far an intervention shifts a model away from its default response. Across three open-weight LLMs and 14 RealCBT-derived case studies, evaluated with human experts, valence-arousal trajectories, and linguistic entrainment, F shows that simply introducing protocol definitions via single chain-of-thought prompting fails to change LLM behavior, while MCoT on these definitions guides strategy selection better. Still, the effect stays within 1% (approx. 1.2-1.3%), and all models remain biased toward Validation & Reflection. These results show CBT knowledge alone does not ensure effective application, giving the affective-computing community instrumentation to measure where LLMs fall short.

Figures

Figures reproduced from arXiv: 2607.02885 by Andrea Kleinsmith, Gerald Ndawula, Lira Yoon, Manas Gaur, Pooja Guttal, Pranay Deep Reddy Katike, Vaishnavi Sinha, Vishal Sinha.

Figure 1
Figure 1. Figure 1: Current response strategies of an out-of-the-box LLM, GPT-5.3 (top), and Ash—AI for Mental Health (bottom) given the same initial user query without prior conversational context. GPT-5.3 makes early assumptions (“It just means you’ve learned to protect others”) and moves towards premature problem-solving (“we can break this down together”). Whereas, Ash prioritizes questioning early on (“What makes you fee… view at source ↗
Figure 3
Figure 3. Figure 3: Illustrating how a user’s query is grounded in clinical termi￾nologies through top-5 semantic retrieval and validated via contrastive NLI-based reasoning to categorize clinical concepts as Entailment, Neutral, or Contradiction based on the user’s expressed narrative [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: MULTIPLE CHAIN￾OF-THOUGHT REASON￾ING. The model gener￾ates candidate responses for (V), (SQ), and (AP), scores each candidate, and selects the highest-scoring response before outputting to the user. Thought (MCoT) ( [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Centrifugal-force-inspired view of F, following Huygens’s F = mv2 /r [38]. The selected protocol is a point mass m orbiting the baseline centroid (the “gravity” of the model’s default behavior) at radius r = dcos(eMCoT, µbase) + ε. The tangential arrow v is the signed shift of the response toward the user’s utterance; the outward arrow is the resulting force F, which grows when the CBT-guided approach move… view at source ↗
Figure 6
Figure 6. Figure 6: Valence–Arousal trajectories of user language across CBT-MCoT synthetic sessions (purple) and RealCBT human transcripts (blue), scored using the NRC VAD Lexicon. Each point is a cumulative mean over turns, and each graph shows the average arc per model across 14 transcripts. RealCBT’s longer sessions reveal a more nuanced emotional journey (more oscillations). TABLE II: Comparing baseline and MCoT examples… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

41 extracted references · 4 linked inside Pith

  1. [1]

    Zao-Sanders

    M. Zao-Sanders. (2025, Apr. 9) How people are really using gen ai in 2025. [Online]. Available: https://hbr.org/2025/04/ how-people-are-really-using-gen-ai-in-2025

    2025

  2. [2]

    How people are using chatgpt,

    OpenAI, “How people are using chatgpt,” September 2025. [Online]. Available: https://openai.com/index/how-people-are-using-chatgpt/

    2025

  3. [3]

    How people use chatgpt,

    A. Chatterji, T. Cunningham, D. J. Deming, Z. Hitzig, C. Ong, C. Y . Shan, and K. Wadman, “How people use chatgpt,” National Bureau of Economic Research, Working Paper 34255, September 2025. [Online]. Available: http://www.nber.org/papers/w34255

    2025

  4. [4]

    Sam altman says chatgpt has hit 800m weekly active users,

    T. Harris, “Sam altman says chatgpt has hit 800m weekly active users,” Oct. 2025, news report citing OpenAI CEO remarks. [Online]. Available: https://techcrunch.com/2025/10/06/ sam-altman-says-chatgpt-has-hit-800m-weekly-active-users/

    2025

  5. [5]

    R. W. Picard,Affective computing. MIT press, 2000

    2000

  6. [6]

    Emotional intelligence of large language models,

    X. Wang, X. Li, Z. Yin, Y . Wu, and J. Liu, “Emotional intelligence of large language models,”Journal of Pacific Rim Psychology, vol. 17, p. 18344909231213958, 2023

    2023

  7. [7]

    Aware yet biased: Investigating emotional reasoning and appraisal bias in large language models,

    A. N. Tak, J. Gratch, and K. R. Scherer, “Aware yet biased: Investigating emotional reasoning and appraisal bias in large language models,”IEEE Transactions on Affective Computing, 2025

    2025

  8. [8]

    The efficacy of cognitive behavioral therapy: A review of meta-analyses,

    S. G. Hofmann, A. Asnaani, I. J. V onk, A. T. Sawyer, and A. Fang, “The efficacy of cognitive behavioral therapy: A review of meta-analyses,” Cognitive Therapy and Research, vol. 36, no. 5, pp. 427–440, 2012

    2012

  9. [9]

    The cognitive concep- tualization diagram,

    Beck Institute for Cognitive Behavior Therapy, “The cognitive concep- tualization diagram,” https://beckinstitute.org/wp-content/uploads/2021/ 08/Traditional-CCD.pdf, 2021

    2021

  10. [10]

    Using linguistic entrainment to evaluate large language models for use in cognitive behavioral therapy,

    M. Kian, K. Shrestha, K. Fischer, X. Zhu, J. Ong, A. Trehan, J. Wang, G. Chang, S. Arnold, and M. Mataric, “Using linguistic entrainment to evaluate large language models for use in cognitive behavioral therapy,” inFindings of the Association for Computational Linguistics: NAACL 2025, 2025, pp. 7724–7743

    2025

  11. [11]

    Feel the difference? a comparative analysis of emotional arcs in real and llm-generated cbt sessions,

    X. Wang, J. Zhang, G. Zhang, and H. Guo, “Feel the difference? a comparative analysis of emotional arcs in real and llm-generated cbt sessions,” inFindings of the Association for Computational Linguistics: EMNLP 2025, 2025

    2025

  12. [12]

    Cbt-bench: Evaluating large language models on assisting cognitive behavior therapy,

    M. Zhang, X. Yang, X. Zhang, T. Labrum, J. C. Chiu, S. M. Eack, F. Fang, W. Y . Wang, and Z. Chen, “Cbt-bench: Evaluating large language models on assisting cognitive behavior therapy,” arXiv preprint arXiv:2410.13218, 2024. [Online]. Available: https: //arxiv.org/pdf/2410.13218

    Pith/arXiv arXiv 2024

  13. [13]

    Patient-ψ: using large language models to simulate patients for training mental health professionals,

    R. Wang, S. Milani, J. C. Chiu, J. Zhi, S. M. Eack, T. Labrum, S. M. Murphy, N. Jones, K. V . Hardy, H. Shenet al., “Patient-ψ: using large language models to simulate patients for training mental health professionals,” inProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024, pp. 12 772–12 797

    2024

  14. [14]

    Crisp: Cognitive restructuring of negative thoughts through multi-turn supportive dialogues,

    J. Zhou, Y . Chen, J. Yin, Y . Huang, Y . Shi, X. Zhang, L. Peng, R. Zhang, T. Lv, Z. Hu, H. Wang, and M. Huang, “Crisp: Cognitive restructuring of negative thoughts through multi-turn supportive dialogues,” in Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, C. Christodoulopoulos, T. Chakraborty, C. Rose, and V . Pe...

    2025

  15. [15]

    A circumplex model of affect

    J. A. Russell, “A circumplex model of affect.”Journal of personality and social psychology, vol. 39, no. 6, p. 1161, 1980

    1980

  16. [16]

    Obtaining reliable human ratings of valence, arousal, and dominance for 20,000 English words,

    S. Mohammad, “Obtaining reliable human ratings of valence, arousal, and dominance for 20,000 English words,” inProceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), I. Gurevych and Y . Miyao, Eds. Melbourne, Australia: Association for Computational Linguistics, Jul. 2018, pp. 174–184. [Online]. Av...

    2018

  17. [17]

    Modeling interpersonal linguistic coordination in conversations using word mover’s distance,

    M. Nasir, S. N. Chakravarthula, B. Baucom, D. C. Atkins, P. Georgiou, and S. Narayanan, “Modeling interpersonal linguistic coordination in conversations using word mover’s distance,” inInterspeech, vol. 2019, 2019, p. 1423

    2019

  18. [18]

    Aligning large language models for cognitive behavioral therapy: a proof-of-concept study,

    Y . Kim, C.-H. Choi, S. Cho, J.-y. Sohn, and B.-H. Kim, “Aligning large language models for cognitive behavioral therapy: a proof-of-concept study,”Frontiers in Psychiatry, vol. 16, p. 1583739, 2025

    2025

  19. [19]

    Interpersonal synchrony: A survey of evaluation methods across disciplines,

    E. Delaherche, M. Chetouani, A. Mahdhaoui, C. Saint-Georges, S. Vi- aux, and D. Cohen, “Interpersonal synchrony: A survey of evaluation methods across disciplines,”IEEE Transactions on Affective Computing, vol. 3, no. 3, pp. 349–365, 2012

    2012

  20. [20]

    Language style matching in psychotherapy: An implicit aspect of alliance

    A.-v. Doorn, J. Porcerelli, L. C. M ¨uller-Frommeyeret al., “Language style matching in psychotherapy: An implicit aspect of alliance.”Journal of Counseling Psychology, vol. 67, no. 4, p. 509, 2020

    2020

  21. [21]

    From word embed- dings to document distances,

    M. Kusner, Y . Sun, N. Kolkin, and K. Weinberger, “From word embed- dings to document distances,” inInternational conference on machine learning. PMLR, 2015, pp. 957–966

    2015

  22. [22]

    One battle af- ter another: Probing llms’ limits on multi-turn instruction following with a benchmark evolving framework,

    Q. Jia, K. Zhang, X. Song, Y . Shen, X. Zhu, and G. Zhai, “One battle af- ter another: Probing llms’ limits on multi-turn instruction following with a benchmark evolving framework,”arXiv preprint arXiv:2511.03508, 2025

    arXiv 2025

  23. [23]

    R. S. Lazarus,Emotion and adaptation. Oxford University Press, 1991

    1991

  24. [24]

    Individual differences in two emotion regulation processes: implications for affect, relationships, and well- being

    J. J. Gross and O. P. John, “Individual differences in two emotion regulation processes: implications for affect, relationships, and well- being.”Journal of personality and social psychology, vol. 85, no. 2, p. 348, 2003

    2003

  25. [25]

    Steal and smile: Impact of emotion regulation and social values on smiles in social conflicts,

    M. Hladk ´y, B. Han, J. Gratch, P. Gebhard, and T. Schneeberger, “Steal and smile: Impact of emotion regulation and social values on smiles in social conflicts,” in2025 13th International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, 2025

    2025

  26. [26]

    Use of snomed ct in large language models: Scoping review,

    E. Chang and S. Sung, “Use of snomed ct in large language models: Scoping review,”JMIR Medical Informatics, vol. 12, no. 1, p. e62924, 2024

    2024

  27. [27]

    Neurosymbolic retrievers for retrieval- augmented generation,

    Y . Saxena and M. Gaur, “Neurosymbolic retrievers for retrieval- augmented generation,”IEEE Intelligent Systems, vol. 41, no. 1, pp. 96–104, 2026

    2026

  28. [28]

    Minds versus machines: Rethinking entailment verification with language models,

    S. Sanyal, T. Xiao, J. Liu, W. Wang, and X. Ren, “Minds versus machines: Rethinking entailment verification with language models,” arXiv preprint arXiv:2402.03686, 2024

    Pith/arXiv arXiv 2024

  29. [29]

    Snomed ct,

    SNOMED International, “Snomed ct,” https://www.snomed.org/ snomed-ct, 2023, accessed: 2026-04-24

    2023

  30. [30]

    Mpnet: Masked and permuted pre-training for language understanding,

    K. Song, X. Tan, T. Qin, J. Lu, and T.-Y . Liu, “Mpnet: Masked and permuted pre-training for language understanding,”Advances in neural information processing systems, vol. 33, pp. 16 857–16 867, 2020

    2020

  31. [31]

    Retrieval- augmented generation for knowledge-intensive nlp tasks,

    P. Lewis, E. Perez, A. Piktus, F. Petroni, V . Karpukhin, N. Goyal, H. K ¨uttler, M. Lewis, W.-t. Yih, T. Rockt ¨aschelet al., “Retrieval- augmented generation for knowledge-intensive nlp tasks,”Advances in neural information processing systems, vol. 33, pp. 9459–9474, 2020

    2020

  32. [32]

    Self- alignment pretraining for biomedical entity representations,

    F. Liu, E. Shareghi, Z. Meng, M. Basaldella, and N. Collier, “Self- alignment pretraining for biomedical entity representations,” inPro- ceedings of the 2021 conference of the North American chapter of the association for computational linguistics: human language technologies, 2021, pp. 4228–4238

    2021

  33. [33]

    Large language models are contrastive reasoners,

    L. Yao and Y . Yang, “Large language models are contrastive reasoners,” Expert Systems with Applications, p. 130407, 2025

    2025

  34. [34]

    Sentence-bert: Sentence embeddings using siamese bert-networks,

    N. Reimers and I. Gurevych, “Sentence-bert: Sentence embeddings using siamese bert-networks,” inProceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP- IJCNLP), 2019, pp. 3982–3992

    2019

  35. [35]

    Forecasting rare language model behaviors,

    E. Jones, M. Tong, J. Mu, M. Mahfoud, J. Leike, R. Grosse, J. Kaplan, W. Fithian, E. Perez, and M. Sharma, “Forecasting rare language model behaviors,”arXiv preprint arXiv:2502.16797, 2025

    Pith/arXiv arXiv 2025

  36. [36]

    Evaluating large language models for accuracy incentivizes hallucinations,

    A. T. Kalai, O. Nachum, S. S. Vempala, and E. Zhang, “Evaluating large language models for accuracy incentivizes hallucinations,”Nature, pp. 1–3, 2026

    2026

  37. [37]

    ICML 2024 Tutorial: Physics of Language Models,

    Z. Allen-Zhu, “ICML 2024 Tutorial: Physics of Language Models,” July 2024, project page: https://physics.allen-zhu.com/

    2024

  38. [38]

    Huygens,Horologium oscillatorium, 1980

    C. Huygens,Horologium oscillatorium, 1980

    1980

  39. [39]

    A threshold selection method from gray-level histograms,

    N. Otsu, “A threshold selection method from gray-level histograms,” IEEE Transactions on Systems, Man, and Cybernetics, vol. 9, no. 1, pp. 62–66, 1979

    1979

  40. [40]

    Moral sensitivity in llms: A tiered evaluation of contextual bias via behavioral profiling and mechanistic interpretability,

    Y . Aggarwal, A. Gorti, V . Jain, A. Chadha, K. Thirunarayan, and M. Gaur, “Moral sensitivity in llms: A tiered evaluation of contextual bias via behavioral profiling and mechanistic interpretability,”arXiv preprint arXiv:2605.03217, 2026

    Pith/arXiv arXiv 2026

  41. [41]

    Krippendorff,Content Analysis: An Introduction to Its Methodology, 2nd ed

    K. Krippendorff,Content Analysis: An Introduction to Its Methodology, 2nd ed. Thousand Oaks, CA: Sage Publications, 2004

    2004