Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Psycholinguistic Word Features: a New Approach for the Evaluation of LLMs Alignment with Humans

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that existing psycholinguistic word ratings provide a new way to measure whether LLMs and humans mean the same things by words, and that current models match well on emotional and conceptual features but poorly on sensory…

desk verdict Useful evaluation framework using psycholinguistic norms, but the headline sensory-deficit claim rests on an uncontrolled between-dataset comparison. read the letter →

arxiv 2506.22439 v1 pith:75IJPBAD submitted 2025-05-29 cs.CL cs.AI

Javier Conde , Miguel González , María Grandury , Gonzalo Martínez , Pedro Reviriego , Mar Brysbaert This is my paper
classification cs.CLcs.AI
keywords psycholinguisticnormsLLMevaluationhumanalignmentembodiedcognitionsensorimotorwordratingsGlasgowLancaster
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper turns decades of psycholinguistic word ratings into a test of how closely large language models match human judgments about word meaning. Eight language models were asked to rate thousands of English words on thirteen features drawn from two norm datasets, and their answers were correlated with the average ratings from human experiments. The models line up reasonably well with humans on affective and conceptual features such as arousal, valence, concreteness, imageability, familiarity, gender, and dominance, but far less well on sensory-perceptual features such as how strongly a word is experienced by taste, smell, touch, hearing, sight, or bodily signals. The authors take this gap as a sign that text-only training leaves models without the embodied, sensorimotor grounding that contributes to human word meaning, and as evidence that psycholinguistic norms are a useful evaluation bench for alignment.

What carries the argument

The load-bearing objects are the two norm datasets: the Glasgow norms give human ratings on arousal, valence, dominance, concreteness, imageability, familiarity, and gender for 5,553 words, and the Lancaster norms give human ratings on six perceptual modalities (touch, hearing, smell, taste, vision, interoception) for 39,707 words. Each LLM receives the same instruction humans received, plus a request to respond with a single number. Two machine estimates are extracted from each prompt: the token with the largest probability and the probability-weighted expectation over the rating scale, with the latter used for the headline results. Alignment is quantified by Pearson and Spearman correlations, each computed on original and integer-rounded values, so every reported comparison has four coefficient views.

What would settle it

Take the words that appear in both the Glasgow and Lancaster norm sets and compute the same correlations on that matched subset; if the affective-conceptual correlations and the sensory correlations become comparable once the word list and distributions are controlled, the paper's central contrast would not survive.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is a split in LLM-human alignment along a perceptual-affective axis. Correlations with human ratings on the Glasgow norms (seven scales, 5,553 words) are generally in a moderate-to-strong range for arousal, valence, concreteness, imageability, and familiarity, with weaker results for gender and dominance; the best models, GPT-4o and GPT-4o-mini, sometimes approach but rarely reach the 0.8-1.0 range the authors treat as close agreement. On the Lancaster norms (six perceptual modalities, 39,707 words), correlations drop sharply for every model, and including multimodal models such as Llama-3.2-11B and GPT-4o does not improve the visual-feature alignment. The paper argues this asymmetry reflects a genuine limitation of current LLMs, likely rooted in the absence of embodied cognition, and shows that existing human rating studies can be mined to quantify and monitor that limitation.

Load-bearing premise

The interpretation depends on the assumption that lower correlations on the Lancaster norms are caused by the sensory nature of those features, rather than by the fact that the two datasets contain different word sets with different frequencies and very skewed rating distributions.

Editorial extensions

If this is right

  • Psycholinguistic word norms can be applied at scale as an automated LLM check, without collecting new human judgments.
  • Current models systematically under-align with human sensory associations, while keeping closer alignment on affective and conceptually rich dimensions.
  • Adding vision or other modalities to a model does not, by itself, close the sensory-perceptual gap on these norms.
  • Reporting both Pearson and Spearman correlations, plus rounded values, guards against drawing model-quality conclusions from distributional artifacts.
  • A broader benchmark built on more norm datasets and languages would give leaderboards a psycholinguistic alignment axis.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: because the Glasgow and Lancaster datasets contain different words and the sensory ratings are strongly skewed, the affective-versus-sensory gap could partly reflect word-list properties rather than a deficit in sensory knowledge; a matched-word comparison would separate these explanations.
  • Beyond the paper: correlation coefficients reward rank agreement, so two models with different rating distributions could score identically; calibration-style metrics would sharpen what the norms measure about human-model agreement.
  • Beyond the paper: porting the same procedure to non-English psycholinguistic norms would test whether the sensory gap is a general property of text-only language learning or an artifact of English-centric training data.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a methodology for evaluating how well LLMs align with human psycholinguistic word norms. Using the Glasgow norms (seven features: arousal, valence, dominance, concreteness, imageability, familiarity, gender) and the Lancaster norms (six perceptual modalities: interoceptive, gustatory, olfactory, haptic, auditory, visual), the authors prompt a set of open and proprietary LLMs with the same instructions used in the original human studies, compute log-probability-based rating estimates, and measure agreement via Pearson and Spearman correlations (on original and rounded values). The main empirical claim is that alignment is generally better on the Glasgow affective/cognitive features than on the Lancaster sensory-perceptual features, which the authors interpret as evidence that current LLMs lack human-like embodied cognition. The paper also releases prompts, code, and results in a public repository.

Significance. If the central comparison were established, the paper would make a useful contribution by introducing an automated, reproducible psycholinguistic benchmark for LLM evaluation, covering a representative set of models and linking results to an active theoretical discussion (embodied cognition). The public release of prompts, code, and raw results is a concrete strength that supports reproducibility. However, the headline claim—that LLMs align better with Glasgow norms than with Lancaster sensory norms—currently rests on an uncontrolled between-dataset comparison, and no statistical uncertainty is reported. These issues are load-bearing because the abstract and Section 5 generalize from that comparison to a claim about a 'sensory deficit' in LLMs.

major comments (3)
  1. [§4.2] The central claim that alignment is better on Glasgow norms than on Lancaster norms is a between-dataset comparison, but the two datasets differ in almost every respect besides the semantic nature of the features. Glasgow covers 5,553 words rated on 1–9 scales; Lancaster covers 39,707 words rated on 0–5 scales, with strongly zero-inflated distributions for gustatory, olfactory, and haptic ratings. The lower Lancaster correlations could therefore be due to word frequency, word length, scale granularity, range restriction, or floor effects rather than to the sensory nature of the features. The manuscript reports no analysis on the overlapping word set, no matching on lexical variables, and no control for distributional differences. Since the abstract and Section 5 (embodied-cognition interpretation) depend on this comparison, the authors should provide an overlap-based or matched-word analysis (e.g., correlations computed only on the common nouns, or on words matched for frequency and length) and show that the Glasgow-vs-Lancaster gap persists under those controls.
  2. [§4.2] The text states that Lancaster correlations are 'significantly lower' but no significance tests, confidence intervals, or standard errors are reported anywhere in the paper. With eight models and thirteen features, the pattern is described qualitatively from the figures. The authors should quantify uncertainty, for example by bootstrapping the per-feature correlations or by fitting a mixed-effects model with random intercepts for words and models; this would also make it possible to test the interaction between dataset/feature type and correlation, which is the actual claim of interest.
  3. [§5] The interpretation that the lower Lancaster correlations reflect missing embodied cognition is not uniquely supported by the data. The authors themselves note that multimodal models (Llama-3.2-11B, GPT-4o, GPT-4o-mini) do not outperform text-only models on the Visual feature, which weakens a straightforward embodiment story. Moreover, because the comparison is confounded by word set and scale properties (see previous comments), the sensory-deficit conclusion is premature. I recommend reframing Section 5 as a hypothesis and providing a concrete falsifiable test, such as comparing sensory vs. non-sensory norms on the same word set, or testing whether a text-only model's performance on a particular feature tracks the lexical statistics of that feature's ratings rather than its sensory status.
minor comments (5)
  1. [Abstract, Figure 3] There is a typo: 'introceptive' should be 'interoceptive' in the abstract and in Figure 3.
  2. [§3.3] The description of the log-probability-based estimate could be more explicit: it should state how the probabilities of the rating tokens are normalized (e.g., whether the model's probability mass over the allowed numeric tokens is renormalized) and how the direct answer is obtained at temperature zero. This detail is important for reproducibility.
  3. [§4] The text says that only the log-probability estimate is presented 'as, in general, it achieves better alignment', but no results for the direct-answer estimate are shown or summarized. Please clarify whether the direct-answer results are available in the repository and briefly report the difference, or remove the 'in general' claim.
  4. [§3.4] Rounding the human and LLM values to the nearest integer before computing Pearson correlations is a reasonable robustness check, but its effect on Spearman correlations (which use ranks and are not affected by monotonic rounding in expectation) should be described explicitly, since the manuscript currently lumps the four variants together without explaining when each is most informative.
  5. [Limitations] The Limitations section acknowledges the small number of datasets and models but does not mention the between-dataset confound or the absence of inferential statistics; adding a sentence to that effect would be appropriate and would strengthen the paper's transparency.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: alignment is measured against external human norm datasets; the Martínez et al. 2025 self-citation is motivational only, and the Glasgow-vs-Lancaster confound is a validity issue, not a by-construction reduction.

full rationale

The paper's derivation chain is: take two external human norm datasets (Glasgow, Scott et al. 2019; Lancaster, Lynott et al. 2020), prompt LLMs with the original human-study instructions plus a request to answer with only a number, compute an LLM rating for each word (the argmax, or the log-probability expectation), and correlate those ratings with the human mean ratings using Pearson and Spearman coefficients on original and rounded values. No parameter is fit to the target conclusion, no human rating is fed back into the LLM, and the ground truth is external to the authors' prior work. The log-probability estimator is adopted from Ivanova et al. 2024, an external source. The two self-citations (Martínez et al. 2025 and Plaza et al. 2024) are contextual: Martínez et al. is invoked only to motivate the choice of Glasgow features ('previous works have shown good alignment of leading LLMs such as GPT-4'), and the paper then measures that alignment itself in Figures 1-3, so the conclusion does not rest on the citation; Plaza et al. appears only in the Limitations section. The central claim, that alignment is better on the Glasgow features than on the Lancaster sensory features, is an empirical measurement from the paper's own computed correlations rather than an identity or a fitted value. A genuine validity concern remains, which belongs under correctness risk rather than circularity: the between-dataset comparison is uncontrolled for word set (5,553 vs 39,707 words), scale granularity (1-9 vs 0-5), and the zero-inflated distributions of the Lancaster perceptual ratings, and the paper itself notes that Pearson and Spearman diverge most for the most skewed dimensions (gustatory and olfactory). That confound does not make the result equivalent to its inputs by construction, so the honest finding is no significant circularity, score 1.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters were fitted; the analysis uses pre-existing human norms as benchmarks and standard correlation metrics. The main assumptions concern the validity and comparability of the human rating datasets and the equivalence of prompts between human and LLM conditions.

assumptions (4)
  • domain assumption The human ratings in the Glasgow and Lancaster norms accurately reflect the psycholinguistic features being measured.
    These ratings are treated as ground truth for LLM alignment throughout the evaluation in Section 3 and Section 4.
  • domain assumption The Pearson and Spearman correlation coefficients are appropriate measures of human-LLM alignment.
    The metrics are adopted from psycholinguistics (Section 3.4) without validation for the LLM context or the skewed distributions of several norms.
  • domain assumption The prompts given to LLMs capture the same rating instructions as those used in human data collection.
    The prompts are based on the original studies, but the paper does not show that LLMs interpret the rating scales equivalently to humans.
  • domain assumption The word sets of the Glasgow and Lancaster norms are comparable for comparing alignment across features.
    This is the key assumption that enables the cross-dataset comparison in Section 4.2; it is not tested, and the two datasets contain different word lists.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Psycholinguistic Word Features: a New Approach for the Evaluation of LLMs Alignment with Humans." pith.science (2026). https://pith.science/paper/75IJPBAD

@misc{pith2026250622439,
  author       = {Pith},
  title        = {Pith review of: Psycholinguistic Word Features: a New Approach for the Evaluation of LLMs Alignment with Humans},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/75IJPBAD}},
  note         = {Machine review of arXiv:2506.22439}
}
read the original abstract

The evaluation of LLMs has so far focused primarily on how well they can perform different tasks such as reasoning, question-answering, paraphrasing, or translating. For most of these tasks, performance can be measured with objective metrics, such as the number of correct answers. However, other language features are not easily quantified. For example, arousal, concreteness, or gender associated with a given word, as well as the extent to which we experience words with senses and relate them to a specific sense. Those features have been studied for many years by psycholinguistics, conducting large-scale experiments with humans to produce ratings for thousands of words. This opens an opportunity to evaluate how well LLMs align with human ratings on these word features, taking advantage of existing studies that cover many different language features in a large number of words. In this paper, we evaluate the alignment of a representative group of LLMs with human ratings on two psycholinguistic datasets: the Glasgow and Lancaster norms. These datasets cover thirteen features over thousands of words. The results show that alignment is \textcolor{black}{generally} better in the Glasgow norms evaluated (arousal, valence, dominance, concreteness, imageability, familiarity, and gender) than on the Lancaster norms evaluated (introceptive, gustatory, olfactory, haptic, auditory, and visual). This suggests a potential limitation of current LLMs in aligning with human sensory associations for words, which may be due to their lack of embodied cognition present in humans and illustrates the usefulness of evaluating LLMs with psycholinguistic datasets.

Figures

Figures reproduced from arXiv: 2506.22439 by the authors.

Figure 1
Figure 1. Pearson and Spearman correlation coefficients (on original and rounded values) for the Glasgow norms [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Pearson and Spearman correlation coefficients (on original and rounded values) for the Glasgow norms [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Pearson and Spearman correlation coefficients (on original and rounded values) for the Lancaster norms [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. From Monolingual to Bilingual: Investigating Language Conditioning in Large Language Models for Psycholinguistic Tasks

    cs.CL 2025-08 conditional novelty 6.0 of 10

    Prompted language identity changes both the outputs and the internal layer representations of Llama-3.3-70B and Qwen2.5-72B on sound symbolism and word valence tasks.

Reference graph

Works this paper leans on

38 extracted references · 21 canonical work pages · cited by 1 Pith paper

  1. [1]

    01. AI, :, Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang, Guanwei Zhang, Heng Li, Jiangcheng Zhu, Jianqun Chen, Jing Chang, Kaidong Yu, Peng Liu, Qiang Liu, Shawn Yue, Senbin Yang, Shiming Yang, Tao Yu, Wen Xie, Wenhao Huang, Xiaohui Hu, Xiaoyi Ren, Xinyao Niu, Pengcheng Nie, Yuchi Xu, Yudong Liu, Yue Wang, Yuxuan Cai, Zhenyu Gu, Zhiyuan Liu, and...

    arXiv 2024

  2. [2]

    Eleftherios Avramidis, Annika Gr \"u tzner-Zahn, Manuel Brack, Patrick Schramowski, Pedro Ortiz Suarez, Malte Ostendorff, Fabio Barth, Shushen Manakhimova, Vivien Macketanz, Georg Rehm, et al. 2024. Occiglot at wmt24: European open-source large language models evaluated on translation. In Proceedings of the Ninth Conference on Machine Translation, pages 292--298

    work page 2024

  3. [3]

    Lawrence W Barsalou. 2008. Grounded cognition. Annu. Rev. Psychol., 59(1):617--645

    work page 2008

  4. [4]

    Anna M Borghi, Chiara De Livio, Angelo Mattia Gervasi, Francesco Mannella, Stefano Nolfi, and Luca Tummolini. 2024. Language as a cognitive and social tool at the time of large language models. Journal of Cultural Cognitive Science, pages 1--20

    work page 2024

  5. [5]

    Yuyan Chen, Hao Wang, Songzhou Yan, Sijia Liu, Yueze Li, Yi Zhao, and Yanghua Xiao. 2024. Emotionqueen: A benchmark for evaluating empathy of large language models. arXiv preprint arXiv:2409.13359

    arXiv 2024

  6. [6]

    Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Hao Zhang, Banghua Zhu, Michael Jordan, Joseph E Gonzalez, et al. 2024. Chatbot arena: An open platform for evaluating llms by human preference. arXiv preprint arXiv:2403.04132

    arXiv 2024

  7. [7]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    arXiv 2024

  8. [8]

    Clémentine Fourrier, Nathan Habib, Alina Lozovskaya, Konrad Szafer, and Thomas Wolf. 2024. Open llm leaderboard v2. https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard

    2024

Show all 38 references
  1. [9]

    Chuanji Gao, Svetlana V Shinkareva, and Rutvik H Desai. 2023. Scope: the south carolina psycholinguistic metabase. Behavior Research Methods, 55(6):2853--2884

    2023

  2. [10]

    Zishan Guo, Renren Jin, Chuang Liu, Yufei Huang, Dan Shi, Linhao Yu, Yan Liu, Jiaxuan Li, Bojian Xiong, Deyi Xiong, et al. 2023. Evaluating large language models: A comprehensive survey. arXiv preprint arXiv:2310.19736

    2023 arXiv

  3. [11]

    Stevan Harnad. 1990. The symbol grounding problem. Physica D: Nonlinear Phenomena, 42(1-3):335--346

    1990

  4. [12]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300

    2020 arXiv

  5. [13]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. https://arxiv.org/abs/2103.03874 Measuring mathematical problem solving with the math dataset . Preprint, arXiv:2103.03874

    2021 arXiv

  6. [14]

    Jen-tse Huang, Man Ho Lam, Eric John Li, Shujie Ren, Wenxuan Wang, Wenxiang Jiao, Zhaopeng Tu, and Michael Lyu. 2024. Apathetic or empathetic? evaluating llms' emotional alignments with humans. In The Thirty-eighth Annual Conference on Neural Information Processing Systems

    2024

  7. [15]

    Anna A Ivanova, Aalok Sathe, Benjamin Lipkin, Evelina Fedorenko, and Jacob Andreas. 2024. Log probability scores provide a closer match to human plausibility judgments than prompt-based evaluations. In South NLP Symposium

    2024

  8. [16]

    E. M. Moreno J. A. Hinojosa and P. Ferré. 2020. https://doi.org/10.1080/23273798.2019.1620957 Affective neurolinguistics: towards a framework for reconciling language and emotion . Language, Cognition and Neuroscience, 35(7):813--839

    2020

  9. [17]

    Casey Kennington. 2021. https://doi.org/10.18653/v1/2021.conll-1.11 Enriching language models with visually-grounded word vectors and the L ancaster sensorimotor norms . In Proceedings of the 25th Conference on Computational Natural Language Learning, pages 148--157, Online. A...

    2021 doi

  10. [18]

    Dermot Lynott, Louise Connell, Marc Brysbaert, James Brand, and James Carney. 2020. The lancaster sensorimotor norms: multidimensional measures of perceptual and action strength for 40,000 english words. Behavior research methods, 52:1271--1291

    2020

  11. [19]

    Gonzalo Mart \' nez, Juan Diego Molero, Sandra Gonz \'a lez, Javier Conde, Marc Brysbaert, and Pedro Reviriego. 2025. Using large language models to estimate features of multi-word expressions: Concreteness, valence, arousal. Behavior Research Methods, 57(1):1--11

    2025

  12. [20]

    Jerome L Myers, Arnold D Well, and Robert F Lorch Jr. 2013. Research design and statistical analysis. Routledge

    2013

  13. [21]

    Aidar Myrzakhan, Sondos Mahmoud Bsharat, and Zhiqiang Shen. 2024. Open-llm-leaderboard: From multi-choice to open-style questions for llms evaluation, benchmark, and arena. arXiv preprint arXiv:2406.07545

    2024 arXiv

  14. [22]

    Tahira Naseem, Guangxuan Xu, Sarathkrishna Swaminathan, Asaf Yehudai, Subhajit Chaudhury, Radu Florian, Ram \'o n Astudillo, and Asim Munawar. 2024. https://doi.org/10.18653/v1/2024.findings-acl.10 A grounded preference model for LLM alignment . In Findings of the Association ...

    2024 doi

  15. [23]

    OpenAI. 2023. https://arxiv.org/abs/2303.08774 Gpt-4 technical report . Preprint, arXiv:2303.08774

    2023 arXiv

  16. [24]

    Irene Plaza, Nina Melero, Cristina del Pozo, Javier Conde, Pedro Reviriego, Marina Mayor-Rocher, and Mar \' a Grandury. 2024. Spanish and llm benchmarks: is mmlu lost in translation? arXiv preprint arXiv:2406.17789

    2024 arXiv

  17. [25]

    Ragnar Rommetveit. 2014. Words, meaning, and messages: Theory and experiments in psycholinguistics. Academic Press

    2014

  18. [26]

    Graham G Scott, Anne Keitel, Marc Becirspahic, Bo Yao, and Sara C Sereno. 2019. The glasgow norms: Ratings of 5,500 words on nine scales. Behavior research methods, 51:1258--1270

    2019

  19. [27]

    Feifan Song, Bowen Yu, Minghao Li, Haiyang Yu, Fei Huang, Yongbin Li, and Houfeng Wang. 2024. Preference ranking optimization for human alignment. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 18990--18998

    2024

  20. [28]

    Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adri \`a Garriga-Alonso, et al. 2022. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv p...

    2022 arXiv

  21. [29]

    Pérez-Sánchez, Constance Imbault, and José Antonio Hinojosa

    Hans Stadthagen-González, Pilar Ferré, Miguel A. Pérez-Sánchez, Constance Imbault, and José Antonio Hinojosa. 2018. https://doi.org/10.3758/S13428-017-0962-Y Norms for 10,491 spanish words for five discrete emotions: Happiness, disgust, anger, fear, and sadness . Behavior Rese...

    2018 doi

  22. [30]

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, L \'e onard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram \'e , et al. 2024. Gemma 2: Improving open language models at a practical size. arXiv e-prints, pages arXiv--2408

    2024

  23. [31]

    Sean Trott. 2024. Can large language models help augment english psycholinguistic datasets? Behavior Research Methods, pages 1--19

    2024

  24. [32]

    Philippe Vincent-Lamarre, Alexandre Blondin Mass \'e , Marcos Lopes, M \'e lanie Lord, Odile Marcotte, and Stevan Harnad. 2016. The latent structure of dictionaries. Topics in cognitive science, 8(3):625--659

    2016

  25. [33]

    Amy Beth Warriner, Victor Kuperman, and Marc Brysbaert. 2013. Norms of valence, arousal, and dominance for 13,915 english lemmas. Behavior research methods, 45:1191--1207

    2013

  26. [34]

    Margaret Wilson. 2002. Six views of embodied cognition. Psychonomic bulletin & review, 9:625--636

    2002

  27. [35]

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? In Annual Meeting of the Association for Computational Linguistics

    2019

  28. [36]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2024. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36

    2024

  29. [37]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  30. [38]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.