Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Do Language Models Think Consistently? A Study of Value Preferences Across Varying Response Lengths

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLMs' value preferences inferred from short answers barely match those expressed in long-form responses, and alignment helps only modestly.

desk verdict A useful empirical caution on short-form value evaluations, with a long-form preference measure that is plausible but under-validated; worth a serious referee with targeted revisions. read the letter →

arxiv 2506.02481 v1 pith:2Z3TIFK4 submitted 2025-06-03 cs.CL cs.AI

Inderjeet Nair , Lu Wang This is my paper
classification cs.CLcs.AI
keywords valuepreferencesLLMconsistencyshort-formevaluationlong-formgenerationargumentorderingalignmentmoraldilemmasspecificityanddiversity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether the value preferences a language model shows in a short, forced-choice answer carry over to long-form, open-ended replies—the kind of output users actually receive. Comparing five 7–9B parameter model families before and after alignment, the authors find only weak correlations (typically 0.05–0.25) between value preferences inferred from short-form decisions and from the order of arguments in long-form responses. The weakness persists across argument counts of 5, 10, and 20, and alignment yields only modest and inconsistent gains. The paper also reports that more strongly preferred values are expressed with more diverse and less specific arguments. If correct, the result implies that current short-form value evaluations do not tell us how a model's values will show up in practical long-form use.

What carries the argument

The argument runs on two preference meters built from one value system of 301 values. On the short-form side, TrueSkill (a Bayesian rating algorithm) updates a Gaussian belief $N(\mu_v,\sigma_v^2)$ per value from each binary dilemma choice, and $\mu_v$ becomes the value's preference weight. On the long-form side, the model is instructed to order its $k$ arguments by its own value preferences; gpt-4o extracts the arguments and tags each with values from the same 301-value list, and a value's preference is the negative average normalized position of its first tagged argument. The consistency claim is the low Pearson correlation between these two preference vectors, supported by two attribute metrics: path-based specificity (longest root-to-leaf path in an argument tree) and compression ratio (gZip-based redundancy, the inverse of diversity).

What would settle it

Take a set of long-form responses and infer value preferences both from the original ordering and from the same arguments presented in a reversed or randomly shuffled order. If the inferred preference ranking stays essentially the same when the instruction says 'least preferred first,' then argument order is not carrying preference information; if preferences track the shuffled order, positional artifacts dominate. A second check is to replace gpt-4o value tagging with human annotations on a subset and see whether short-long correlations rise to a level that changes the conclusion.

Watch

Extended reading notes

Core claim

The central claim is that an LLM's value preferences are not a stable quantity that survives a change in response format: the same model ranks the same 301-value system differently depending on whether it is choosing an action or writing an ordered list of arguments. Short-form preferences are estimated with a TrueSkill-style Gaussian belief update over the values attached to chosen and unchosen actions in DAILYDILEMMAS; long-form preferences are estimated by prompting the model to generate $k \in \{5,10,20\}$ arguments in its own preferred order, splitting those arguments and tagging them with values using gpt-4o, and taking each value's preference as the negative average normalized position of its first occurrence. The two preference vectors correlate weakly across all five model families, and preferences from different values of $k$ also correlate weakly, especially between $k=5$ and the longer settings. Alignment improves the short-long correlation only modestly and does not reliably improve long-long consistency, while contentious OPINION QA topics show less consistency than everyday DAILYDILEMMAS dilemmas.

Load-bearing premise

The load-bearing premise is that the position of a value's first argument in a long-form response reflects the model's preference for that value; if models do not actually order arguments by preference, or if gpt-4o's argument splitting and value tagging are biased, the weak short-long correlation could be an artifact of the measurement rather than genuine inconsistency in values.

Editorial extensions

If this is right

  • Short-form value benchmarks will overstate the stability of a model's ethical stance in real use, because long-form outputs express different value rankings.
  • A model's expressed values shift with the requested number of arguments, so verbosity preferences in user interfaces are not value-neutral.
  • Alignment methods in current use do not reliably make value expression consistent across response modes or domains, so alignment gains on short tests may not transfer.
  • Everyday moral dilemmas produce more stable value rankings than contentious societal topics, so the choice of evaluation domain changes the consistency verdict.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A straightforward test of the paper's mechanism would be to infer preferences from the same arguments presented in reverse order; if rankings do not flip, the ordering instruction is not what drives the measured preference.
  • The specificity result may reflect discourse effort rather than value strength: less preferred values receive more supporting and counter-argument detail because they need more persuasion.
  • If value preferences are format-dependent, value alignment should be evaluated and optimized on long-form, ordered generations rather than on short-form surveys.
  • The pipeline could be extended to non-English outputs or to larger models to see whether value consistency scales with capability and language.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper investigates whether value preferences inferred from short-form ethical dilemma responses align with preferences inferred from long-form open-ended responses. Using DAILYDILEMMAS and OPINION QA, the authors elicit short-form choices and long-form argument lists of k=5, 10, or 20 arguments, then derive a 301-dimensional value-preference vector per model and generation mode: TrueSkill latent ratings for short-form and negative normalized first-argument position for long-form. They compute Pearson and Spearman correlations across five base and five instruct models and report weak short-long correlations, weak correlations between different long-form settings, modest alignment gains, and associations between preference strength and argument specificity and diversity. The paper contributes a new evaluation perspective and publicly releases code.

Significance. If the measurement pipeline were validated, the paper would address a real gap: short-form value tests may not transfer to long-form deployment, where verbosity varies. The study is systematic in covering multiple model families, alignment methods, argument counts, and two datasets, and it ships public code. However, the headline claims rest on an unvalidated long-form preference measure and on correlations that may be confounded by frequency and measurement noise, so the significance of the current results is conditional on additional validation.

major comments (4)
  1. [Section 2.3, Figure 9] The long-form preference vector is defined as the negative normalized position of the first argument tagged with a value, under the explicit prompt instruction that models should order arguments by their own value preferences. The paper provides no evidence that models comply with this instruction; base models in particular may order arguments by rhetorical flow, argument strength, or dataset artifacts rather than by value preference. If the order is not preference-driven, the weak short-long correlations (Figures 3a-b and 12) reflect instruction-following noise rather than genuine value inconsistency. I request a compliance check (e.g., measuring whether the first argument corresponds to the action the model states it agrees with, or comparing long-form order to an explicitly elicited ranking on a sample) and a permutation baseline showing that the observed correlations are above what a random ordering of the same arguments would produce.
  2. [Section 2.3, Figure 11, Limitations item 2] Value attribution for every extracted argument is performed by a single GPT-4o call that asks for the five closest values from the 301-value list, with no reported inter-annotator agreement, no human validation sample, and no alternative annotator; the Limitations section itself acknowledges that "model bias may lead to ... incorrect value attribution." Because the long-form preference vector is computed from these value assignments, annotation noise and systematic bias can attenuate or inflate the observed correlations with short-form preferences. I request a reliability analysis on a sample (e.g., 100 arguments annotated by a second model or by humans) and a sensitivity analysis restricted to values with high annotation confidence.
  3. [Sections 5.1-5.2, Figures 6-7] The claimed negative correlation between specificity and preference and positive correlation between diversity and preference are potentially confounded with value frequency. For diversity, the compression ratio is computed over the set of all arguments tagged with a value (Section 3.2); a value that appears in many scenarios will mechanically have a larger and more heterogeneous text set and hence a lower compression ratio (higher diversity). Meanwhile, the TrueSkill mu used as the preference measure (Section 2.2) is also driven by how often a value appears in chosen actions. Thus the diversity-preference correlation may largely reflect frequency. Similarly, specificity is averaged over whatever arguments contain the value, so less frequent values have noisier and possibly biased estimates. Please report partial correlations controlling for the number of arguments per value (or a frequency-matched analysis) and confirm the findings survive.
  4. [Figures 3-7 and Appendix B] Many reported correlations are in the 0.05-0.25 range; with 301 values these can reach p<0.05 while explaining under 6% of the variance. No confidence intervals are given, and no multiple-comparison correction is applied across the many models, k settings, and datasets. More importantly, claims such as "alignment improves consistency" (Section 4.1) are based on visual comparison of point estimates without a test of the difference between independent correlations. Please report confidence intervals for all correlations and, where pre/post alignment or k settings are compared, a proper test (e.g., Fisher's z or bootstrap) for the difference.
minor comments (5)
  1. [Abstract and Section 1] The text says 'five LLMs' but the study evaluates ten models (five base and five instruct variants); please rephrase to 'five model families' or list all ten models explicitly.
  2. [Section 4.2, Figures 13-17] Section 4.2 states that p-values are omitted and that results are significant for most models; please report the actual p-values (or confidence intervals) for Figure 4 and Figures 13-17.
  3. [Figure 17 caption] The caption says 'when k = 5' but the text in Appendix B.2 indicates Figures 15, 16, and 17 correspond to OPINION QA for k=5, 10, and 20; the caption should read k=20.
  4. [Section 3.2] The diversity metric is defined via compression ratio, but the link to 'representation across scenarios' is asserted rather than derived; please clarify how compression ratio operationalizes cross-scenario breadth.
  5. [Figure 10] The line 'Format: List supporting values: values that support the given argument' appears to belong to the value-assignment prompt of Figure 11 and is confusing in the argument-extraction prompt; please remove or relocate it.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the short/long-form comparison is an empirical correlation between independently elicited measures, and no load-bearing step reduces by construction to its own inputs.

full rationale

The paper's central claim is an empirical comparison between two independently elicited measures: short-form value preferences come from TrueSkill belief updates over binary dilemma decisions (Section 2.2), while long-form preferences come from the normalized position of a value's first argument in prompted outputs (Section 2.3). Neither quantity is fitted to the other, and the observed weak correlations could plausibly have been strong, so the headline result is not forced by construction. The long-form measure is defined as argument order, and the prompt instructs the model to order arguments by its own preferences; this is an operationalization choice rather than a circular derivation, because the short-form measure is obtained through a different procedure and the paper does not claim that order-based scores are validated independently of that instruction. The Limitations section candidly notes that gpt-4o-based value attribution may be biased, but annotation-bias risk concerns construct validity, not circularity; the paper does not use that limitation as evidence for its findings. The diversity and specificity analyses are correlations between separately computed attributes and preference scores, and while they share the same gpt-4o value assignments, there is no equation-level identity or fitted-parameter renaming that would make those correlations true by definition. Citations to prior work, such as Ye et al. (2025) for the vector representation and Chiu et al. (2024) for the dataset, are external sources rather than load-bearing self-citations by the present authors. Overall, the derivation chain is self-contained as an empirical study, and the appropriate finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The analysis depends on a set of modeling choices inherited from prior work (TrueSkill, the DAILYDILEMMAS value system) and on the reliability of GPT-4o as an annotator and judge. No new objects are invented, but the preference measurements rest on several untested premises.

free parameters (4)
  • TrueSkill initial mean and variance (mu=25, sigma=8.333) = mu=25, sigma=8.333
    Chosen initial values for the Gaussian belief distributions over value preferences; the analysis uses relative ranks, but absolute scale depends on these.
  • TrueSkill performance variance beta and dynamics tau = not specified
    The appendix states beta is a predefined constant parameter but does not give its value or tau; these control the magnitude of preference updates.
  • Sampling temperature for long-form generation = 0.9
    Temperature 0.9 is used for the temperature-sampling consistency experiment; it is an arbitrary choice affecting output variance.
  • GPT-4o model version and annotation prompt settings = not pinned
    The paper uses gpt-4o for argument extraction, value assignment, and specificity scoring, but does not specify the exact model snapshot or decoding parameters.
assumptions (5)
  • domain assumption The DAILYDILEMMAS action-value sets are accurate and exhaustive.
    Short-form preference inference assumes that each dilemma's two actions are correctly tagged with the full set of relevant values (Section 2.1.1).
  • domain assumption Value preferences can be modeled as Gaussian distributions and updated by TrueSkill.
    The TrueSkill belief update (Appendix A.1) assumes value preferences behave like skills in team games, which is a modeling choice inherited from prior work.
  • domain assumption Models order arguments according to their true value preferences when prompted.
    The long-form preference measure is valid only if models comply with the instruction to present arguments in preference order (Section 2.3, Figure 9).
  • domain assumption GPT-4o annotations of arguments and values are accurate.
    Argument extraction, value assignment, and specificity scoring all rely on gpt-4o; the limitations section acknowledges this as a potential source of bias.
  • domain assumption Compression ratio is a valid measure of diversity.
    The diversity metric uses gzip compression ratio as a proxy for redundancy, which is an established but imprecise approach (Section 3.2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Do Language Models Think Consistently? A Study of Value Preferences Across Varying Response Lengths." pith.science (2026). https://pith.science/paper/2Z3TIFK4

@misc{pith2026250602481,
  author       = {Pith},
  title        = {Pith review of: Do Language Models Think Consistently? A Study of Value Preferences Across Varying Response Lengths},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2Z3TIFK4}},
  note         = {Machine review of arXiv:2506.02481}
}
read the original abstract

Evaluations of LLMs' ethical risks and value inclinations often rely on short-form surveys and psychometric tests, yet real-world use involves long-form, open-ended responses -- leaving value-related risks and preferences in practical settings largely underexplored. In this work, we ask: Do value preferences inferred from short-form tests align with those expressed in long-form outputs? To address this question, we compare value preferences elicited from short-form reactions and long-form responses, varying the number of arguments in the latter to capture users' differing verbosity preferences. Analyzing five LLMs (llama3-8b, gemma2-9b, mistral-7b, qwen2-7b, and olmo-7b), we find (1) a weak correlation between value preferences inferred from short-form and long-form responses across varying argument counts, and (2) similarly weak correlation between preferences derived from any two distinct long-form generation settings. (3) Alignment yields only modest gains in the consistency of value expression. Further, we examine how long-form generation attributes relate to value preferences, finding that argument specificity negatively correlates with preference strength, while representation across scenarios shows a positive correlation. Our findings underscore the need for more robust methods to ensure consistent value expression across diverse applications.

Figures

Figures reproduced from arXiv: 2506.02481 by the authors.

Figure 1
Figure 1. Analysis Protocol Summary: Starting from a set of moral scenarios, we collect both short-form reactions and long-form responses. Note that while long-form responses may present both views, the order of arguments reflects the model’s explicit preferences. Value preferences are independently inferred from each format and their alignment is subsequently evaluated. Finally, the individual arguments within the long-form … view at source ↗
Figure 2
Figure 2. Value extraction from a long-form re￾sponse’s argument: An example of the values extracted by gpt-4o from a given value-laden argument invoked by one of the models in the above described scenario. calculate the average normalized position across all responses. The negative of the averaged normal￾ized position is considered as the preference value for vi . Taking the negative ensures that a higher preference value fo… view at source ↗
Figure 3
Figure 3. Consistency of value preferences estimated [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (25 more)
Figure 4
Figure 4. Figure 4: Consistency in value preferences from the [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Pairwise Pearson correlations between value preferences across different modes of long-form generations [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Pearson correlation between path-based speci [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Pearson correlation between compression ra [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Prompt for extracting short-form model re [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Prompt for extracting long-form responses [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Prompt for extracting arguments from long form responses [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Prompt for assigning values to the argument in the [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: Consistency (measured by Pearson correla [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 16
Figure 16. Figure 16: Consistency in value preferences is deter [PITH_FULL_IMAGE:figures/full_fig_p018_16.png]
Figure 17
Figure 17. Figure 17: Consistency in value preferences is deter [PITH_FULL_IMAGE:figures/full_fig_p018_17.png]
Figure 18
Figure 18. Figure 18: Pairwise Pearson correlations between value preferences across different modes of long-form generation [PITH_FULL_IMAGE:figures/full_fig_p019_18.png]
Figure 19
Figure 19. Figure 19: Pairwise Pearson correlations between value preferences across different modes of long-form generation [PITH_FULL_IMAGE:figures/full_fig_p019_19.png]
Figure 20
Figure 20. Figure 20: Consistency between implicit and explicit [PITH_FULL_IMAGE:figures/full_fig_p019_20.png]
Figure 21
Figure 21. Figure 21: Prompt for assessing path-based specificity for an input argument. Prompt for assessing attribute-based specificity Evaluate the specificity of the given input argument by analyzing its level of detail, precision, and clarity, then assign a specificity score from 1 to…
Figure 22
Figure 22. Figure 22: Prompt for assessing attribute-based specificity for an input argument [PITH_FULL_IMAGE:figures/full_fig_p020_22.png]
Figure 23
Figure 23. Figure 23: Prompt of standardizing the value using a list of values . [PITH_FULL_IMAGE:figures/full_fig_p021_23.png]
Figure 24
Figure 24. Figure 24: Path-based Specificity for the long-form responses over OPINIONQA and DAILYDILEMMAS [PITH_FULL_IMAGE:figures/full_fig_p022_24.png]
Figure 25
Figure 25. Figure 25: Pearson correlation between path-based specificity from DAILYDILEMMAS and value prefer￾ence when k = 5 llama3-8b gemma2-9b mistral-7b qwen2-7b olmo-7b Models 0.15 0.10 0.05 0.00 0.05 Pearson Correlation p >= 0.05 p < 0.05 p >= 0.05 p >= 0.05 p >= 0.05 p >= 0.05 p < 0.…
Figure 26
Figure 26. Figure 26: Pearson correlation between path-based specificity from DAILYDILEMMAS and value prefer￾ence when k = 20 llama3-8b gemma2-9b mistral-7b qwen2-7b olmo-7b Models 0.150 0.125 0.100 0.075 0.050 0.025 0.000 0.025 Pearson Correlation p >= 0.05 p >= 0.05 p >= 0.05 p >= 0.05 p…
Figure 29
Figure 29. Figure 29: Pearson correlation between path-based specificity from OPINIONQA and value preference when k = 20 [PITH_FULL_IMAGE:figures/full_fig_p023_29.png]
Figure 30
Figure 30. Figure 30: Compression ratio for the long-form responses over OPINIONQA and DAILYDILEMMAS [PITH_FULL_IMAGE:figures/full_fig_p024_30.png]
Figure 31
Figure 31. Figure 31: Pearson correlation between compression ration from DAILYDILEMMAS and value preference when k = 5 llama3-8b gemma2-9b mistral-7b qwen2-7b olmo-7b Models 0.25 0.20 0.15 0.10 0.05 0.00 Pearson Correlation p >= 0.05 p < 0.05 p < 0.05 p < 0.001 p >= 0.05 p < 0.05 p < 0.05…
Figure 32
Figure 32. Figure 32: Pearson correlation between compression ration from DAILYDILEMMAS and value preference when k = 20 llama3-8b gemma2-9b mistral-7b qwen2-7b olmo-7b Models 0.25 0.20 0.15 0.10 0.05 0.00 Pearson Correlation p >= 0.05 p >= 0.05 p >= 0.05 p < 0.05 p >= 0.05 p >= 0.05 p < 0…
Figure 35
Figure 35. Figure 35: Pearson correlation between compression ration from OPINIONQA and value preference when k = 20 [PITH_FULL_IMAGE:figures/full_fig_p025_35.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Belief-Sim: Towards Belief-Driven Simulation of Demographic Misinformation Susceptibility

    cs.CL 2026-03 conditional novelty 6.0 of 10

    Conditioning LLMs on survey-derived demographic belief profiles improves their ability to predict individuals' misinformation judgments, especially when belief modeling is decoupled from susceptibility prediction.

Reference graph

Works this paper leans on

16 extracted references · 12 canonical work pages · cited by 1 Pith paper

  1. [1]

    • Nodes are connected with directed edges, where an edge represents how one node supports another

    Understand the Directed Tree Structure: • Each sentence or part of the argument is a node. • Nodes are connected with directed edges, where an edge represents how one node supports another. • The root of the tree is the most general statement in the argument, while leaves are the most specific points

    work page

  2. [2]

    • This path determines the specificity of the argument

    Evaluate the Depth: • Identify the longest path in the tree from the root (the most general part of the argument) to any leaf (the most specific detail). • This path determines the specificity of the argument

    work page

  3. [3]

    Advances in Neu- ral Information Processing Systems, 36

    Direct preference optimization: Your language model is secretly a reward model. Advances in Neu- ral Information Processing Systems, 36. Alexandre Rame, Guillaume Couairon, Corentin Dancette, Jean-Baptiste Gaya, Mustafa Shukor, Laure Soulier, and Matthieu Cord. 2023. Rewarded soups: towards pareto-optimal alignment by inter- polating weights fine-tuned on...

    arXiv 2023

  4. [4]

    argument

    Evaluating the moral beliefs encoded in llms. Advances in Neural Information Processing Systems, 36. Chantal Shaib, Joe Barrow, Jiuding Sun, Alexa F Siu, Byron C Wallace, and Ani Nenkova. 2024. Stan- dardizing the measurement of text diversity: A tool and a comparative analysis of scores. arXiv preprint arXiv:2403.00553. Bangzhao Shu, Lechen Zhang, Minje ...

    arXiv 2024

  5. [5]

    The steps for assigning the score are provided below:

    Extremely specific; thorough, precise, and leaves little room for interpretation. The steps for assigning the score are provided below:

    work page

  6. [7]

    • 2: Somewhat detailed but still broad

    Determine Specificity Level • 1: Argument is shallow, with minimal levels of detail (most general). • 2: Somewhat detailed but still broad. • 3: Moderate depth with balanced detail. • 4: Detailed and well-supported. • 5: Highly specific with deep supporting details (most specific). Figure 21: Prompt for assessing path-based specificity for an input argume...

    work page

  7. [8]

    Very vague or ambiguous; lacks detail and context

    work page

  8. [9]

    Somewhat clear but missing essential details or specificity

    work page

Show all 16 references
  1. [10]

    Moderately specific; provides sufficient detail to understand the core meaning

  2. [11]

    Very specific; well-defined, with clear context and details

  3. [13]

    Read and understand the input argument

  4. [14]

    Analyze the argument based on the following criteria: • Clarity: How easy is it to understand the argument? • Detail: How specific and thorough is the information provided? • Context: Does the argument provide adequate background or supporting details?

  5. [15]

    Compare the input against the scoring definitions to assign a score from 1 to 5

  6. [16]

    score": [1-5],

    Provide a brief justification for the assigned score, using at least one or two of the criteria above to explain the rating. The output must be presented as a JSON object with the following structure: {"score": [1-5], "explanation": "Provide a brief explanation justifying the ...

  7. [2022]

    Advances in neural information processing systems, 35:28458–28473

    When to make exceptions: Exploring lan- guage models as accounts of human moral judgment. Advances in neural information processing systems, 35:28458–28473. Grgur Kovaˇc, Masataka Sawayama, Rémy Portelas, Cé- dric Colas, Peter Ford Dominey, and Pierre-Yves Oudeyer. 2023. Large...

    2023 arXiv

  8. [2024]

    arXiv preprint arXiv:2402.13887

    Beyond probabilities: Unveiling the misalign- ment in evaluating large language models. arXiv preprint arXiv:2402.13887. Justin K Miller and Wenjia Tang. 2025. Evaluating llm metrics through real-world capabilities. arXiv preprint arXiv:2505.08253. Jared Moore, Tanvi Deshpande...

    2025 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.