Pith. sign in

REVIEW 3 major objections 5 minor 3 cited by

Potemkin Understanding in Large Language Models

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Large language models routinely answer benchmark keystone questions correctly while holding interpretations no human would hold, so benchmark success does not certify conceptual understanding.

desk verdict Useful framework and benchmark, credible empirical gap, but the potemkin interpretation leans on an unmeasured human baseline that should be supplied before the strong claim is accepted. read the letter →

arxiv 2506.21521 v2 pith:FSBWGUE7 submitted 2025-06-26 cs.CL cs.AI

Marina Mancoridis , Bec Weeks , Keyon Vafa , Sendhil Mullainathan This is my paper
classification cs.CLcs.AI
keywords potemkinunderstandinglargelanguagemodelsbenchmarkvaliditykeystonequestionsconceptualself-consistencyevaluationcognitive
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that high scores on human-designed benchmark questions do not by themselves show that a large language model understands the underlying concept. It formalizes the conditions under which a curated set of questions can certify understanding, and shows that those conditions require the model's possible misunderstandings to mirror human misunderstandings. When that fails, a model can answer keystone questions correctly while its actual interpretation of the concept is wrong; the paper calls this 'potemkin understanding' and measures it with two procedures. Across seven models and three domains, correct definitions are frequently followed by failures to apply the same concept in classification, generation, and editing tasks. If the claim holds, human benchmark performance cannot be read as conceptual understanding without an additional assumption.

What carries the argument

The load-bearing object is the keystone set $S$: a minimal set of concept-relevant strings such that any human interpretation that agrees with the correct interpretation on $S$ must be the correct interpretation. Interpretations are defined as functions $f : X \to \{0,1\}$ over all concept-relevant strings $X$, with $F_h$ the space of human interpretations and $F_l$ the space of LLM interpretations. A potemkin is an $f \in F_l$ that matches $f^*$ on a keystone but not everywhere. The paper's two measurement procedures operationalize this: the benchmark compares correct definitions (the keystone) against classification, constrained generation, and editing; the automated procedure has the model answer a question, generate related questions, answer them, then grade those answers with the same model, treating disagreements as potemkins and noting this yields only a lower bound.

What would settle it

Present the study's classification, generation, and editing items to a group of people who give correct definitions of the same concepts; if those people fail at rates comparable to the models, the definition-application gap would not distinguish LLM understanding from human understanding.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that large language models exhibit potemkin understanding: an interpretation function $f$ that agrees with the correct interpretation $f^*$ on a keystone set $S$ yet differs from $f^*$ elsewhere, so the model can answer keystone questions without grasping the concept. The paper proves the formal claim that human-designed keystones are valid tests for an LLM only if the model's space of interpretations equals the human space, $F_l = F_h$. Empirically, the paper reports that models define the 32 tested concepts correctly 94.2% of the time, yet conditioned on a correct definition they solve classification, generation, and editing items at rates that imply potemkin rates of roughly 0.40 to 0.55; an automated self-grading procedure gives a lower bound on potemkin rate of 0.62 overall. The paper further reports that models often grade their own generated examples inconsistently, indicating that the failure reflects incoherent internal representations rather than a consistent alternative interpretation.

Load-bearing premise

The paper needs the premise that a person who can correctly define a concept must also be able to apply it in classification, generation, and editing; this 'keystone' property of definitions is asserted as a matter of faith and is not checked against human performance on the same use tasks.

Editorial extensions

If this is right

  • Benchmark scores computed from keystone-style questions cannot be taken as evidence of conceptual understanding unless the model is known to be free of potemkins.
  • The hold-out principle still guarantees performance on fresh questions drawn from the same distribution, so the invalidity claim is specifically about generalization beyond that distribution.
  • Expanding keystone sets to include multiple use examples yields only modest gains, so adding a few application questions to an exam will not repair the invalid inference.
  • The automated self-grading procedure provides a cheap, scalable screening tool for potemkin prevalence that requires only the model itself as a judge.
  • Detecting and reducing potemkins becomes a target for training and evaluation pipelines, distinct from fixing hallucination or factual errors.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If potemkins are a general phenomenon, then model evaluations that rely on benchmark accuracy alone will systematically overstate downstream competence on open-ended tasks, which may matter for high-stakes deployment decisions.
  • The keystone framework suggests a direct constructive extension: build benchmarks by explicitly enumerating interpretation spaces, rather than sampling questions, to make the no-potemkins assumption testable.
  • A natural test of the human-side assumption is to run the paper's classification, generation, and editing tasks on human participants who can define the concepts; if humans also fail at comparable rates, the measured gap would not establish non-human understanding.
  • The incoherence results imply that model editing or fine-tuning that targets single facts may not repair conceptual understanding, since the same model holds conflicting representations of one concept.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces a formal framework for when human-designed benchmarks are valid tests of conceptual understanding in LLMs, arguing that such benchmarks are valid only if LLMs misunderstand concepts in the same structured ways humans do. It defines a 'keystone' set of questions that, if answered correctly by a human, force the correct interpretation, and defines 'potemkin understanding' as an LLM answering a keystone correctly while having a non-human, incorrect interpretation. The authors construct a benchmark across 32 concepts in literary techniques, game theory, and psychological biases, measuring how often models that correctly define a concept fail to apply it in classification, generation, and editing tasks. They also propose an automated self-grading procedure that yields a claimed lower bound on potemkin prevalence. Across 7-9 models, they report high rates of definition-use gaps and incoherence, concluding that existing benchmarks cannot be read as evidence of conceptual understanding in LLMs.

Significance. If the central claim holds, the paper addresses a fundamental question in LLM evaluation: it offers a formal explanation for why high benchmark scores need not imply conceptual understanding and provides two empirical procedures for detecting such failures. The benchmark dataset, with 3,159 labeled data points spanning three domains and multiple tasks, plus the publicly released code and data, is a valuable resource. The paper also connects to a broader literature on benchmark validity and model consistency, and the proposed 'potemkin' construct is likely to influence future evaluation methodology. The main empirical pattern—that models can define concepts correctly yet fail to apply them—is credible and consistent across models and tasks. However, the interpretation of this pattern as evidence of non-human understanding rests on an untested assumption about human concept use, which currently limits the strength of the conclusions.

major comments (3)
  1. [Section 3.1 and Discussion (third paragraph)] The keystone assumption is load-bearing but unvalidated. Section 3.1 states 'we have faith that humans who can clearly define the concept of a haiku have understood haikus,' and the Discussion responds to the 'too hard for humans' concern only with qualitative examples in Appendix J. The formal definition of potemkin understanding in Definition 2.1 requires the keystone set S to force the correct interpretation for all human interpretations f in Fh; this is an empirical claim about human concept use, not a logical consequence of being able to recite a definition. The paper never measures P(use correct | definition correct) for humans on the same 32 concepts and tasks. If human conditional accuracy on the generation or editing tasks is materially below ceiling, the observed LLM gaps would be at least partly explained by task difficulty rather than by non-human understanding. Since the headline conclusion (Section 5) that high potemkin rates invalidate benchmarks as measures of understanding depends on this assumption, the paper needs either a human baseline on these tasks or a substantially weakened claim.
  2. [Section 3.1, 3.3 and Definition 2.1] The empirical 'potemkin rate' is not the same as the formally defined potemkin. In Section 3.1, the keystone is operationalized as a single definition question, and the potemkin rate is defined in Section 3.3 as the proportion of use questions answered incorrectly conditional on a correct definition. But Definition 2.1 defines potemkin understanding as correct answers on a keystone set S that is guaranteed, for all human interpretations, to imply f = f*. The paper never verifies that a single definition question satisfies this keystone property, nor that a definition remains a keystone when the use tasks are as demanding as those in the benchmark. Therefore the measured quantity is more accurately described as a definition-use gap; calling it a potemkin rate presupposes the very assumption the paper needs to establish. This conflation affects the interpretation of Tables 1 and 6 and the title of the paper.
  3. [Section 4.2] The automatic procedure in Section 4.2 is not a lower bound on potemkins as defined in Definition 2.1. The procedure measures how often a model's self-judge grades the model's own answer to a self-generated question as incorrect. The argument that this is a lower bound on potemkin rate assumes that any judge disagreement indicates conceptual misunderstanding, but it does not involve a keystone set, nor does it condition on correct keystone answers in the formal sense. A model could answer keystone questions correctly and still generate questions that are difficult for the judge to grade consistently, so the 'potemkin rate (lower bound)' in Table 2 conflates self-consistency with potemkin understanding. Additionally, the role of the 'slightly incorrect response' control is not explained, and no results from that control are reported, so the reader cannot assess whether the judge is calibrated.
minor comments (5)
  1. [Appendix L, Table 7] The incoherence score for Claude-3.5 in game theory is reported as 1.04 (0.10), which exceeds the stated upper bound of 1 for 'as-good-as-random' performance; the paper should clarify whether scores above 1 are possible and what they indicate.
  2. [Figure 1 caption] The caption is difficult to parse: the panels are not labeled, and the text 'No Does out rhyme with soft?' is confusing. Please label the panels and describe each step clearly.
  3. [Section 4.2] The description of the judge procedure should define the 'expected response' explicitly and explain how the model's original answer is compared with the judge's grade; the current text leaves the reader to infer the comparison.
  4. [Section 2] The assertion that 'the space of human misunderstandings is predictable and sparse' is presented as fact without citation or supporting evidence; at minimum it should be labeled as an assumption of the framework.
  5. [Table 1 and Section 3.3] For the generation and editing tasks, no chance-level baseline is defined, yet the text implies that a potemkin rate of 1 corresponds to chance-level performance for all tasks; please specify the chance model used for these tasks or avoid the implication.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the potemkin rate is a transparent operationalization of the paper's own definition, and the 'faith' keystone assumption is a validity threat rather than a circular reduction.

full rationale

The paper's derivation chain is conditional and internally consistent. Definition 2.1 defines potemkin understanding as correct answers on a keystone set S with f != f*. Section 3.1 operationalizes a keystone as a single correct definition ('we have faith that humans who can clearly define the concept of a haiku have understood haikus') and Table 1 measures the potemkin rate as the percent of use questions answered incorrectly conditional on a correct definition. This is a direct measurement of the defined construct, not a result that presupposes its conclusion; the empirical rate is not used to justify the keystone assumption. The paper explicitly flags the main threat to this operationalization in the Section 3 Discussion ('Another concern is whether the use tasks we consider in the benchmark are too hard such that humans would fail these as well'), addressing it only qualitatively via Appendix J, and no human baseline for P(use correct | definition correct) is provided. That missing baseline is a substantive validity threat to the inference from conditional error rates to non-human misunderstanding, but it is a missing empirical premise, not a circular step: the conclusion is not assumed within the measurement. No fitted parameter is relabeled as a prediction, and the self-citations (Vafa et al. 2024a,b) appear only as related work, not as load-bearing justification. The lower-bound procedure in Section 4.2 is also a transparent operationalization of self-consistency rather than a circular derivation.

Assumptions & free parameters 0 free parameters · 4 assumptions · 2 invented entities

The central claim rests on domain assumptions about human understanding (keystones from definitions), the structure of human misunderstandings, and the interpretation of LLM self-disagreement. No free parameters are fitted; the measurements are conditional error rates and self-consistency statistics. The invented entity 'potemkin understanding' is a definitional construct whose empirical detection depends on these assumptions.

assumptions (4)
  • domain assumption Definitions are keystones for human understanding: a human who correctly defines a concept must be able to apply it correctly.
    Invoked in Section 3.1: 'we have faith that humans who can clearly define the concept of a haiku have understood haikus.' This assumption is load-bearing for interpreting conditional error rates as potemkins, but it is not tested against human data.
  • domain assumption The space of human misunderstandings Fh is small and structured, so keystone sets exist.
    Stated in Section 2: 'people misunderstand concepts in structured ways... The space of human misunderstandings is predictable and sparse.' This is the theoretical premise for why exams work for humans and why Fl=Fh matters, but no empirical evidence is offered.
  • domain assumption An LLM's interpretation is representable as a function f: X -> {0,1} that can be elicited by prompting.
    Assumed in the formal framework (Section 2) and used in both procedures. Prompting is stochastic and prompt-sensitive, so the function representation is an idealization; temperature settings are not reported.
  • domain assumption Self-disagreement in the automatic procedure reveals conceptual misunderstanding.
    Section 4.2 argues that if the LLM judge says a response is incorrect, it must be due to misunderstanding in either the answerer or the judge. This assumes away the possibility of random sampling noise or ambiguous self-generated questions.
invented entities (2)
  • Potemkin understanding
    purpose: Labels the phenomenon where an LLM answers keystone questions correctly while holding a non-human, incorrect interpretation of a concept (Definition 2.1).
    The construct is defined by the authors and detected through operationalizations that depend on the keystone assumption. It is a useful organizing category but lacks an independent falsifiable handle outside the paper's own measurement choices.
  • Keystone set
    purpose: Formalizes the minimal set of questions that, if answered correctly by a human, force the correct interpretation; used to argue benchmarks are valid for humans and as the basis for the potemkin definition.
    Keystone sets are formal objects whose existence for real concepts is assumed, not demonstrated. The paper treats definitions as keystones without validating this via human experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Potemkin Understanding in Large Language Models." pith.science (2026). https://pith.science/paper/FSBWGUE7

@misc{pith2026250621521,
  author       = {Pith},
  title        = {Pith review of: Potemkin Understanding in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FSBWGUE7}},
  note         = {Machine review of arXiv:2506.21521}
}
read the original abstract

Large language models (LLMs) are regularly evaluated using benchmark datasets. But what justifies making inferences about an LLM's capabilities based on its answers to a curated set of questions? This paper first introduces a formal framework to address this question. The key is to note that the benchmarks used to test LLMs -- such as AP exams -- are also those used to test people. However, this raises an implication: these benchmarks are only valid tests if LLMs misunderstand concepts in ways that mirror human misunderstandings. Otherwise, success on benchmarks only demonstrates potemkin understanding: the illusion of understanding driven by answers irreconcilable with how any human would interpret a concept. We present two procedures for quantifying the existence of potemkins: one using a specially designed benchmark in three domains, the other using a general procedure that provides a lower-bound on their prevalence. We find that potemkins are ubiquitous across models, tasks, and domains. We also find that these failures reflect not just incorrect understanding, but deeper internal incoherence in concept representations.

Figures

Figures reproduced from arXiv: 2506.21521 by the authors.

Figure 1
Figure 1. Illustration of potemkin understanding in a large lan￾guage model. This example shows GPT-4o’s failure to apply its own conceptual explanation of an ABAB rhyming scheme. models, success was interpreted differently. For example, if a pathology classifier performs well on X-ray classification, it is not credited with an understanding of vision—we only draw inferences for its performance on specific distributions, infe… view at source ↗
Figure 2
Figure 2. A schematic representation of keystones and potemkins. Rows represent interpretations of a concept and columns represent questions. Questions can either be interpreted correctly or incorrectly. A keystone is a set of questions that can only be interpreted correctly by a human who has understood the concept. An LLM has potemkin understanding when it correctly interprets all questions in a keystone but does not unders… view at source ↗
Figure 3
Figure 3. Examples of potemkins. In each example, GPT-4o correctly explains a concept but fails to correctly use it. 0 1 2 3 4 Number of 'Use' Examples in Keystone Set (k) 0.0 0.2 0.4 0.6 0.8 1.0 Concept Understanding Concept Understanding by Keystone Set Size Individual models Average [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Impact of expanding keystone sets on concept under￾standing values. We extend the keystone from a single definition to include multiple classification (“use”) examples. “Concept under￾standing” measures whether a model answers 10 additional “use” questions after answer…
Figure 5
Figure 5. Figure 5: Illustration of the method for evaluating incoherence in models. In the first step, the model generates an instance or non￾instance of a given concept. In the second step, the model evaluates whether the instance it generated is a true or false example of the concept. …
Figure 6
Figure 6. Figure 6: Example of a potemkin documented by our automatic evaluation procedure. In this example, the language model fails to agree with its own answer for a question that it generated about the topic of second cousins. findings from Section 3. 5. Related Work Providing a metho…
Figure 7
Figure 7. Figure 7: provides a schematic representation of our experimental framework, with examples from each of our three domains. The yellow examples correspond to the concept of a haiku, in the literary techniques domain. The green examples correspond to the concept of a mixed strateg…
Figure 8
Figure 8. Figure 8: Example screen from the expert annotation survey used to label psychological biases. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Example screen from the survey about evaluating model performance on our constrained generation task. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]
Figure 10
Figure 10. Figure 10: Example screen from the survey about evaluating model performance on our edit task. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_10.png]
Figure 11
Figure 11. Figure 11: Selected qualitative examples of potemkins across models and domains. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_11.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Re:Form -- Reducing Human Annotations in Scalable Formal Software Verification with RL in LLMs: A Preliminary Study on Dafny

    cs.CL 2025-07 conditional novelty 6.0 of 10

    A minimal-prior pipeline with automated data curation and verifier-driven RL lets small LLMs generate verifiable Dafny specifications and beat larger proprietary models on a synthetic compositional benchmark.

  2. Faster AI, Uneven Frontier: Rapid Crossings, a Jagged Frontier, and the Repositioning of Human Judgment

    cs.HC 2026-07 accept novelty 4.0 of 10

    AI has rapidly crossed expert baselines on bounded tasks on a still-jagged frontier, so human work must shift from production to specification, verification, and oversight.

  3. The wall confronting large language models

    cs.AI 2025-07 conditional novelty 4.0 of 10

    LLM scaling exponents near 0.1 imply that reducing loss tenfold would need 10^10 more compute, making scientific-grade reliability unreachable by brute-force scaling.

Reference graph

Works this paper leans on

65 extracted references · 32 canonical work pages · cited by 3 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Gpt-4 can't reason

    Arkoudas, K. Gpt-4 can't reason. arXiv preprint arXiv:2308.03762, 2023

    arXiv 2023

  3. [3]

    and Bisk, Y

    Belinkov, Y. and Bisk, Y. Synthetic and natural noise both break neural machine translation. arXiv preprint arXiv:1711.02173, 2017

    arXiv 2017

  4. [4]

    a is b" fail to learn

    Berglund, L., Tong, M., Kaufmann, M., Balesni, M., Stickland, A. C., Korbak, T., and Evans, O. The reversal curse: Llms trained on" a is b" fail to learn" b is a". arXiv preprint arXiv:2309.12288, 2023

    arXiv 2023

  5. [5]

    Bowman, S. R. and Dahl, G. E. What will it take to fix benchmarking in natural language understanding? arXiv preprint arXiv:2104.02145, 2021

    arXiv 2021

  6. [6]

    R., Angeli, G., Potts, C., and Manning, C

    Bowman, S. R., Angeli, G., Potts, C., and Manning, C. D. A large annotated corpus for learning natural language inference. arXiv preprint arXiv:1508.05326, 2015

    arXiv 2015

  7. [7]

    T., Li, Y., Lundberg, S., et al

    Bubeck, S., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., et al. Sparks of artificial general intelligence: Early experiments with gpt-4, 2023

    work page 2023

  8. [8]

    With Little Power Comes Great Responsibility

    Card, D., Henderson, P., Khandelwal, U., Jia, R., Mahowald, K., and Jurafsky, D. With little power comes great responsibility. arXiv preprint arXiv:2010.06595, 2020

    work page Pith review arXiv 2010

Show all 65 references
  1. [9]

    Chang, S., Anderson, A., and Hofman, J. M. Chatbench: From static benchmarks to human-ai evaluation. arXiv preprint arXiv:2504.07114, 2025

    2025 arXiv

  2. [10]

    N., Li, T., Li, D., Zhu, B., Zhang, H., Jordan, M., Gonzalez, J

    Chiang, W.-L., Zheng, L., Sheng, Y., Angelopoulos, A. N., Li, T., Li, D., Zhu, B., Zhang, H., Jordan, M., Gonzalez, J. E., et al. Chatbot arena: An open platform for evaluating llms by human preference. In Forty-first International Conference on Machine Learning, 2024

    2024

  3. [11]

    Church, K. W. and Hestness, J. A survey of 25 years of evaluation. Natural Language Engineering, 2019

    2019

  4. [12]

    and Etzioni, O

    Clark, P. and Etzioni, O. My computer is an honor student—but how intelligent is it? standardized tests as a measure of ai. AI Magazine, 2016

    2016

  5. [13]

    Recognizing textual entailment: Rational, evaluation and approaches--erratum

    Dagan, I., Dolan, B., Magnini, B., and Roth, D. Recognizing textual entailment: Rational, evaluation and approaches--erratum. Natural Language Engineering, 2010

    2010

  6. [14]

    Testing ai on language comprehension tasks reveals insensitivity to underlying meaning

    Dentella, V., G \"u nther, F., Murphy, E., Marcus, G., and Leivada, E. Testing ai on language comprehension tasks reveals insensitivity to underlying meaning. Scientific Reports, 2024

    2024

  7. [15]

    and Meurers, D

    Dickinson, M. and Meurers, D. Detecting errors in part-of-speech annotation. In 10th conference of the European chapter of the association for computational linguistics, 2003

    2003

  8. [16]

    Measuring and improving consistency in pretrained language models

    Elazar, Y., Kassner, N., Ravfogel, S., Ravichander, A., Hovy, E., Sch \"u tze, H., and Goldberg, Y. Measuring and improving consistency in pretrained language models. Transactions of the Association for Computational Linguistics, 2021

    2021

  9. [17]

    Evaluating superhuman models with consistency checks

    Fluri, L., Paleka, D., and Tram \`e r, F. Evaluating superhuman models with consistency checks. In 2024 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). IEEE, 2024

    2024

  10. [18]

    W., Wallach, H., Iii, H

    Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Iii, H. D., and Crawford, K. Datasheets for datasets. Communications of the ACM, 2021

    2021

  11. [19]

    Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., and Wichmann, F. A. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2020

    2020

  12. [20]

    Transformer feed-forward layers are key-value memories

    Geva, M., Schuster, R., Berant, J., and Levy, O. Transformer feed-forward layers are key-value memories. arXiv preprint arXiv:2012.14913, 2020

    2012 arXiv

  13. [21]

    R., and Goldberg, Y

    Geva, M., Caciularu, A., Wang, K. R., and Goldberg, Y. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. arXiv preprint arXiv:2203.14680, 2022

    2022 arXiv

  14. [22]

    Dissecting recall of factual associations in auto-regressive language models

    Geva, M., Bastings, J., Filippova, K., and Globerson, A. Dissecting recall of factual associations in auto-regressive language models. arXiv preprint arXiv:2304.14767, 2023

    2023 arXiv

  15. [23]

    What can large language models do in chemistry? a comprehensive benchmark on eight tasks

    Guo, T., Nan, B., Liang, Z., Guo, Z., Chawla, N., Wiest, O., Zhang, X., et al. What can large language models do in chemistry? a comprehensive benchmark on eight tasks. Advances in Neural Information Processing Systems, 2023

    2023

  16. [24]

    Measuring massive multitask language understanding

    Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020

    2009 arXiv

  17. [25]

    D., Sordoni, A., and Courville, A

    Hosseini, A., Reddy, S., Bahdanau, D., Hjelm, R. D., Sordoni, A., and Courville, A. Understanding by understanding not: Modeling negation in language models. arXiv preprint arXiv:2105.03519, 2021

    2021 arXiv

  18. [26]

    A., Sathe, A., Lipkin, B., Kumar, U., Radkani, S., Clark, T

    Ivanova, A. A., Sathe, A., Lipkin, B., Kumar, U., Radkani, S., Clark, T. H., Kauf, C., Hu, J., Pramod, R., Grand, G., et al. Elements of world knowledge (ewok): A cognition-inspired framework for evaluating basic world knowledge in language models. arXiv preprint arXiv:2405.09...

    2024 arXiv

  19. [27]

    Adversarial example generation with syntactically controlled paraphrase networks

    Iyyer, M., Wieting, J., Gimpel, K., and Zettlemoyer, L. Adversarial example generation with syntactically controlled paraphrase networks. arXiv preprint arXiv:1804.06059, 2018

    2018 arXiv

  20. [28]

    S., and Lukasiewicz, T

    Jang, M., Kwon, D. S., and Lukasiewicz, T. Accurate, yet inconsistent? consistency analysis on language understanding models. arXiv preprint arXiv:2108.06665, 2021

    2021 arXiv

  21. [29]

    S., and Lukasiewicz, T

    Jang, M., Kwon, D. S., and Lukasiewicz, T. Becel: Benchmark for consistency evaluation of language models. In Proceedings of the 29th International Conference on Computational Linguistics, 2022

    2022

  22. [30]

    Jang, M. E. and Lukasiewicz, T. Consistency analysis of chatgpt. arXiv preprint arXiv:2303.06273, 2023

    2023 arXiv

  23. [31]

    What disease does this patient have? a large-scale open domain question answering dataset from medical exams

    Jin, D., Pan, E., Oufattole, N., Weng, W.-H., Fang, H., and Szolovits, P. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 2021

    2021

  24. [32]

    Dynabench: Rethinking benchmarking in nlp

    Kiela, D., Bartolo, M., Nie, Y., Kaushik, D., Geiger, A., Wu, Z., Vidgen, B., Prasad, G., Singh, A., Ringshia, P., et al. Dynabench: Rethinking benchmarking in nlp. arXiv preprint arXiv:2104.14337, 2021

    2021 arXiv

  25. [33]

    a ldchen, S., Binder, A., Montavon, G., Samek, W., and M \

    Lapuschkin, S., W \"a ldchen, S., Binder, A., Montavon, G., Samek, W., and M \"u ller, K.-R. Unmasking clever hans predictors and assessing what machines really learn. Nature communications, 2019

    2019

  26. [34]

    L., Shrivastava, V., Li, S., Hashimoto, T., and Liang, P

    Li, X. L., Shrivastava, V., Li, S., Hashimoto, T., and Liang, P. Benchmarking and improving generator-validator consistency of language models. arXiv preprint arXiv:2310.01846, 2023

    2023 arXiv

  27. [35]

    Holistic evaluation of language models

    Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasunaga, M., Zhang, Y., Narayanan, D., Wu, Y., Kumar, A., et al. Holistic evaluation of language models. arXiv preprint arXiv:2211.09110, 2022

    2022 arXiv

  28. [36]

    Truthfulqa: Measuring how models mimic human falsehoods

    Lin, S., Hilton, J., and Evans, O. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958, 2021

    2021 arXiv

  29. [37]

    Mathbench: Evaluating the theory and application proficiency of llms with a hierarchical mathematics benchmark

    Liu, H., Zheng, Z., Qiao, Y., Duan, H., Fei, Z., Zhou, F., Zhang, W., Zhang, S., Lin, D., and Chen, K. Mathbench: Evaluating the theory and application proficiency of llms with a hierarchical mathematics benchmark. arXiv preprint arXiv:2405.12209, 2024

    2024 arXiv

  30. [38]

    Sycophancy in large language models: Causes and mitigations

    Malmqvist, L. Sycophancy in large language models: Causes and mitigations. arXiv preprint arXiv:2411.15287, 2024

    2024 arXiv

  31. [39]

    Locating and editing factual associations in gpt

    Meng, K., Bau, D., Andonian, A., and Belinkov, Y. Locating and editing factual associations in gpt. Advances in Neural Information Processing Systems, 2022

    2022

  32. [40]

    Mitchell, E., Lin, C., Bosselut, A., Finn, C., and Manning, C. D. Fast model editing at scale. arXiv preprint arXiv:2110.11309, 2021

    2021 arXiv

  33. [41]

    Why ai is harder than we think

    Mitchell, M. Why ai is harder than we think. arXiv preprint arXiv:2104.12871, 2021

    2021 arXiv

  34. [42]

    D., Bender, E

    Paullada, A., Raji, I. D., Bender, E. M., Denton, E., and Hanna, A. Data and its (dis) contents: A survey of dataset development and use in machine learning research. Patterns, 2021

    2021

  35. [43]

    H., and Riedel, S

    Petroni, F., Rockt \"a schel, T., Lewis, P., Bakhtin, A., Wu, Y., Miller, A. H., and Riedel, S. Language models as knowledge bases? arXiv preprint arXiv:1909.01066, 2019

    1909 arXiv

  36. [44]

    A., and Lewis, M

    Press, O., Zhang, M., Min, S., Schmidt, L., Smith, N. A., and Lewis, M. Measuring and narrowing the compositionality gap in language models. arXiv preprint arXiv:2210.03350, 2022

    2022 arXiv

  37. [45]

    D., Bender, E

    Raji, I. D., Bender, E. M., Paullada, A., Denton, E., and Hanna, A. Ai and the everything in the whole wide world benchmark. arXiv preprint arXiv:2111.15366, 2021

    2021 arXiv

  38. [46]

    Do imagenet classifiers generalize to imagenet? In International conference on machine learning

    Recht, B., Roelofs, R., Schmidt, L., and Shankar, V. Do imagenet classifiers generalize to imagenet? In International conference on machine learning. PMLR, 2019

    2019

  39. [47]

    Reuel, A., Hardy, A., Smith, C., Lamparth, M., Hardy, M., and Kochenderfer, M. J. Betterbench: Assessing ai benchmarks, uncovering issues, and establishing best practices. arXiv preprint arXiv:2411.12990, 2024

    2024 arXiv

  40. [48]

    why should i trust you?

    Ribeiro, M. T., Singh, S., and Guestrin, C. " why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 2016

    2016

  41. [49]

    T., Singh, S., and Guestrin, C

    Ribeiro, M. T., Singh, S., and Guestrin, C. Semantically equivalent adversarial rules for debugging nlp models. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018

    2018

  42. [50]

    T., Guestrin, C., and Singh, S

    Ribeiro, M. T., Guestrin, C., and Singh, S. Are red roses red? evaluating consistency of question-answering models. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019

    2019

  43. [51]

    T., Wu, T., Guestrin, C., and Singh, S

    Ribeiro, M. T., Wu, T., Guestrin, C., and Singh, S. Beyond accuracy: Behavioral testing of nlp models with checklist. arXiv preprint arXiv:2005.04118, 2020

    2005 arXiv

  44. [52]

    Models in the wild: On corruption robustness of neural nlp systems

    Rychalska, B., Basaj, D., Gosiewska, A., and Biecek, P. Models in the wild: On corruption robustness of neural nlp systems. In Neural Information Processing. Springer, 2019

    2019

  45. [53]

    Saba, W. S. Llms' understanding of natural language revealed. arXiv preprint arXiv:2407.19630, 2024

    2024 arXiv

  46. [54]

    everyone wants to do the model work, not the data work

    Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., and Aroyo, L. M. “everyone wants to do the model work, not the data work”: Data cascades in high-stakes ai. In proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 2021

    2021

  47. [55]

    H., Sch \"a rli, N., and Zhou, D

    Shi, F., Chen, X., Misra, K., Scales, N., Dohan, D., Chi, E. H., Sch \"a rli, N., and Zhou, D. Large language models can be easily distracted by irrelevant context. In International Conference on Machine Learning. PMLR, 2023

    2023

  48. [56]

    and Choi, Y

    Shwartz, V. and Choi, Y. Do neural language models overcome reporting bias? In Proceedings of the 28th International Conference on Computational Linguistics, 2020

    2020

  49. [57]

    S., Wei, J., Chung, H

    Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. Large language models encode clinical knowledge. Nature, 2023

    2023

  50. [58]

    Evaluating the factual consistency of large language models through summarization

    Tam, D., Mascarenhas, A., Zhang, S., Kwan, S., Bansal, M., and Raffel, C. Evaluating the factual consistency of large language models through summarization. arXiv preprint arXiv:2211.08412, 2022

    2022 arXiv

  51. [59]

    Y., Kleinberg, J., Mullainathan, S., and Rambachan, A

    Vafa, K., Chen, J. Y., Kleinberg, J., Mullainathan, S., and Rambachan, A. Evaluating the world model implicit in a generative model. arXiv preprint arXiv:2406.03689, 2024 a

    2024 arXiv

  52. [60]

    Do large language models perform the way people expect? measuring the human generalization function

    Vafa, K., Rambachan, A., and Mullainathan, S. Do large language models perform the way people expect? measuring the human generalization function. arXiv preprint arXiv:2406.01382, 2024 b

    2024 arXiv

  53. [61]

    On the planning abilities of large language models-a critical investigation

    Valmeekam, K., Marquez, M., Sreedharan, S., and Kambhampati, S. On the planning abilities of large language models-a critical investigation. Advances in Neural Information Processing Systems, 2023

    2023

  54. [62]

    T., Heer, J., and Weld, D

    Wu, T., Ribeiro, M. T., Heer, J., and Weld, D. S. Errudite: Scalable, reproducible, and testable error analysis. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019

    2019

  55. [63]

    Kformer: Knowledge injection in transformer feed-forward layers

    Yao, Y., Huang, S., Dong, L., Wei, F., Chen, H., and Zhang, N. Kformer: Knowledge injection in transformer feed-forward layers. In CCF International Conference on Natural Language Processing and Chinese Computing. Springer, 2022

    2022

  56. [64]

    Wildchat: 1m chatgpt interaction logs in the wild

    Zhao, W., Ren, X., Hessel, J., Cardie, C., Choi, Y., and Deng, Y. Wildchat: 1m chatgpt interaction logs in the wild. arXiv preprint arXiv:2405.01470, 2024

    2024 arXiv

  57. [65]

    S., Zaheer, M., Bhojanapalli, S., Li, D., Yu, F., and Kumar, S

    Zhu, C., Rawat, A. S., Zaheer, M., Bhojanapalli, S., Li, D., Yu, F., and Kumar, S. Modifying memories in transformer models. arXiv preprint arXiv:2012.00363, 2020

    2012 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.