Pith. sign in

REVIEW 3 major objections 6 minor 45 references

Visual Large Language Models Exhibit Human-Level Cognitive Flexibility in the Wisconsin Card Sorting Test

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Under chain-of-thought prompting with text inputs, visual large language models achieve or surpass human-level set-shifting on the Wisconsin Card Sorting Test.

desk verdict Useful systematic WCST mapping for VLLMs, but the 'human-level' claim rests on an instruction asymmetry and no inferential statistics. read the letter →

arxiv 2505.22112 v1 pith:BFIWE5PO submitted 2025-05-28 cs.AI q-bio.NC

Guangfu Hao , Frederic Alexandre , Shan Yu This is my paper
classification cs.AIq-bio.NC
keywords cognitiveflexibilityWisconsinCardSortingTestvisuallargelanguagemodelsset-shiftingchain-of-thoughtpromptingruleexclusivityimpairmentsimulationprefrontalcortex
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether visual large language models can do what the Wisconsin Card Sorting Test measures: discover an unstated sorting rule, apply it consistently, and switch when the rule silently changes. The authors tested GPT-4o, Gemini-1.5 Pro, and Claude-3.5 Sonnet under four conditions that vary input type (image vs. text) and prompting strategy (direct vs. chain-of-thought). They report that with chain-of-thought prompting and text-based card descriptions, all three models complete as many or more sorting categories than a 30-person human baseline, with Claude-3.5 Sonnet reaching a perfect score in all repetitions. They also report that removing an explicit rule-exclusivity hint degrades performance, and that role-playing prompts can make the models mimic prefrontal-type deficits in goal maintenance, inhibition, and updating. If correct, the result suggests that a key component of human executive function can be approximated by current VLLMs under favorable conditions, with the important caveat that the human comparison may not be instruction-matched.

What carries the argument

The load-bearing instrument is the WCST-64, a short-form card-sorting test in which the correct rule is one of three features, color, shape, or number, and the rule switches silently after ten consecutive correct matches. Its power for this study is that its standard metrics, Categories Completed (CC), Perseverative Errors (PE), Non-Perseverative Errors (NPE), Trials to First Category (TFC), Conceptual Level Responses (CLR), and Failure to Maintain Set (FMS), separate rule discovery, set maintenance, and set shifting within a single procedure. The second key mechanism is chain-of-thought prompting, which asks each model to state observations, hypotheses, and choice justifications before selecting a card; the paper's central comparison is the jump from near-chance performance in the direct-answer visual condition to near-human performance in the CoT-TI condition. A third control point is the explicit rule-exclusivity sentence, 'There will be no combination of these characteristics to define the rule,' because removing it is the manipulation that most clearly degrades model performance. The ALIEN Task serves as a surface-replacement control, preserving the same logical structure while changing all terminology and stimuli.

What would settle it

Give a fresh set of healthy adults the same 64-trial text-based WCST with the exact instruction set the models received, including the explicit sentence that the rule is never a combination of features, and compare CC, PE, and FMS distributions with the CoT-TI model results. If the human mean climbs well above the reported baseline, the core claim rests on an instruction mismatch; if it does not, the human-level comparison is supported.

Watch

Extended reading notes

Core claim

The central claim is that VLLMs can exhibit human-level, and in one case superhuman, cognitive flexibility as measured by the WCST, provided they are prompted to reason step by step and given card information as text. In the best condition, CoT-TI, Claude-3.5 Sonnet completed every category in every repetition ($CC = 5.00$, $\sigma = 0.00$), at or above the human baseline; Gemini-1.5 Pro and GPT-4o also matched or approached the baseline. The same performance pattern held on the ALIEN Task, a re-skinned version with alien-themed terminology, which the authors use to argue that the behavior reflects genuine set-shifting rather than memorization of the classic test. They also find that the visual-versus-text gap stems less from poor low-level recognition, which is mostly accurate, than from cascading errors after occasional visual misperceptions, and that removing the explicit statement that the rule is not a combination of features lowers performance, especially for Gemini-1.5 Pro. Finally, role-playing instructions that simulate impaired goal maintenance, inhibitory control, or adaptive updating produce distinct performance-decrement patterns that the authors map onto neuropsychological profiles of prefrontal lesion patients.

Load-bearing premise

The load-bearing premise is that the human baseline is a fair comparison: the 30 human participants faced the same inferential difficulty as the models, even though the models received an explicit hint that the sorting rule is never a combination of features.

Editorial extensions

If this is right

  • If the central claim is correct, current VLLMs can serve as behavioral stand-ins for healthy set-shifting in structured rule-learning experiments, matching or exceeding a typical adult human sample.
  • The near-chance performance under direct answering and image inputs means benchmark scores on cognitive tests depend heavily on prompt and modality details, so claims of human-level ability must state those conditions.
  • The drop when the rule-exclusivity hint is removed implies that VLLMs' flexibility is partly scaffolded by explicit constraints, so real-world tasks with ambiguous rule spaces should be expected to be harder for them.
  • The ALIEN Task result, if accepted, implies that the behavior transfers to novel surface content, which is evidence against rote memorization of the WCST.
  • The impairment-simulation results, if accepted, offer a prompt-level method for generating distinct executive-dysfunction profiles without modifying model weights.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The fairness of the headline comparison is the natural thing to test next: the models were helped by an explicit sentence ruling out compound rules, while human instructions are only described as 'adapted to be more intuitive,' so a human comparison using the identical instruction text is needed before 'human-level' is taken literally.
  • The role-played impairments may reflect learned clinical stereotypes rather than authentic internal mechanisms; comparing model error patterns trial-by-trial with actual patient data would help distinguish simulation from mechanism.
  • A reverse experiment, giving healthy humans chain-of-thought-style instructions, would clarify how much of the CoT-TI advantage comes from the instruction format itself rather than from any model-specific reasoning ability.
  • The visual-input findings suggest a useful evaluation protocol: benchmark VLLMs on text-description versions of established cognitive tests to isolate reasoning from perception, then add images to measure how perceptual errors cascade into higher-level rule following.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper evaluates three state-of-the-art VLLMs (GPT-4o, Gemini-1.5 Pro, Claude-3.5 Sonnet) on a 64-trial Wisconsin Card Sorting Test under a 2x2 design (visual/textual input crossed with straight-to-answer/chain-of-thought prompting), compares model performance to 30 human participants, and reports additional experiments: a novel ALIEN Task to control for memorization, removal of an explicit rule-exclusivity constraint, and role-playing prompts intended to simulate goal-maintenance, inhibitory-control, and adaptive-updating impairments. The central claim is that VLLMs achieve or surpass human-level set-shifting under CoT prompting with text-based inputs, and that role-played impairments mimic prefrontal dysfunction patterns.

Significance. If the headline claim holds, this would be a notable result: a systematic neuropsychology-inspired evaluation of multimodal LLMs with a strong control task (ALIEN) and multiple standard WCST metrics. The ALIEN Task is a genuine strength, as it addresses memorization-based alternative explanations, and the paper makes code and data available. However, the headline 'human-level or surpass' conclusion depends on two conditions that are not currently met: a fair model-human instruction match and inferential statistics that support the comparison. The impairment-simulation interpretation is also substantially overreach relative to the evidence and is partly retracted by the paper's own caveat.

major comments (3)
  1. [§IV.D vs. §III.A] The model-human comparison is not demonstrably fair. Section IV.D states that all previous model results included the explicit rule-exclusivity statement 'There will be no combination of these characteristics to define the rule,' while Section III.A only says the human instructions were 'carefully adapted to be more intuitive' and does not specify whether this constraint was included. Standard WCST instructions do not normally provide this exclusivity hint. This matters because Table III shows that removing the constraint lowers CoT-TI categories completed for all three models (Gemini-1.5 Pro 4.8 to 2.6; GPT-4o 4.6 to 3.5; Claude-3.5 Sonnet 5.0 to 4.7). Unless the human protocol contained the same exclusivity information, the models faced a smaller hypothesis space, and the abstract's 'achieve or surpass human-level' claim is not established. The authors should state the full human instruction text and, ideally, compare the no-constraint model condition directly against human performance.
  2. [§IV.A, Table I, Figure 2] The paper reports no inferential statistics for the model-versus-human comparison. With n=10 model runs and n=30 human participants, the claim that Claude-3.5 Sonnet 'surpasses' the human baseline (CC 5.00 vs. 4.73, with zero variance for the model) is not tested against sampling variability, and the same applies to the 'near-human' characterizations for GPT-4o and Gemini-1.5 Pro. The authors should add confidence intervals, effect sizes, and appropriate two-sample tests or bootstrap intervals for at least the primary metric CC, and preferably for the secondary metrics as well. Given the number of models, conditions, and metrics, a multiple-comparison correction or an explicitly preregistered analysis plan should be reported.
  3. [§IV.E and Discussion] The impairment-simulation results do not support the claim that VLLMs 'may possess a cognitive architecture, at least regarding the ability of set-shifting, similar to the brain.' The role-playing prompts explicitly instruct the model to behave as if it had a given deficit (e.g., impaired goal maintenance), so observing the requested performance decrements is largely a check of instruction following, not evidence about the model's internal architecture. The paper itself acknowledges in the Discussion that such simulations 'may reflect stereotypical representations of clinical populations rather than authentic mechanisms.' The authors should either reframe Section IV.E as an exploration of prompt-conditioned behavior, or provide convergent evidence beyond the prompted role-play to support the architectural claim.
minor comments (6)
  1. [§III.A] 'We employs a standard version of the WCST-64' should be 'We employed' or 'We use.'
  2. [Table I] The human baseline row is labeled 'Human STA-VI,' but human participants did not receive a straight-to-answer or chain-of-thought prompt; a label such as 'Human (Visual)' would be clearer and would avoid implying a factorial condition that humans did not have.
  3. [Figure 2] The figure shows distributions without error bars, confidence intervals, or individual data points; adding within-condition standard errors or a strip plot would make the reported variability more interpretable.
  4. [§IV.D, Table III] The 'Decline' column reports raw mean differences without uncertainty; because the standard deviations increase substantially without the exclusivity constraint, the authors should report effect sizes or confidence intervals for these declines.
  5. [§I] 'The test's established validity make it a core benchmark' should be 'makes it a core benchmark.'
  6. [§IV.C] The statement that GPT-4o 'almost always misidentified 5 cards as 6 cards' would benefit from an example or a note in the supplementary material, since Table II reports Count accuracy as 0%.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the core result is an external benchmark of VLLM behavior against human participants, not a derivation from fitted inputs, and the role-play impairment component is disclosed as prompt-induced and potentially stereotypical.

full rationale

The paper's central claim is an empirical benchmark: WCST performance is measured across models and conditions and compared with a human baseline. No parameter is fitted to a subset of data and then renamed a prediction; no load-bearing argument relies on a self-citation; no uniqueness result is imported from the authors' prior work; and no equation sets the conclusion equal to its input by construction. The closest edge cases are protocol-fairness issues, not circularity. Section IV.D shows all reported model results used the explicit exclusivity constraint 'There will be no combination of these characteristics to define the rule', while Section III.A only says human instructions were 'carefully adapted to be more intuitive' without confirming whether that constraint was included; this is a potential confound in the human-level comparison, but the model performance is still measured against an external human sample rather than generated from the constraint. Section IV.E induces impairments by role-playing prompts, and the Discussion explicitly cautions that such simulations 'may reflect stereotypical representations of clinical populations rather than authentic mechanisms underlying the disorders themselves'; the observed decrements are a manipulation check, not an independent derivation of brain-like architecture. The ALIEN task is an explicitly labeled control variant, not a renamed result offered as a discovery. Thus no step in the claimed chain reduces to its own input.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper is an empirical benchmark, not a derivation, so it introduces no free parameters and no new physical or conceptual entities. Its claims rest on domain assumptions about test validity, instruction equivalence between models and humans, and reliable response parsing.

assumptions (3)
  • domain assumption WCST-64 is a valid and sensitive measure of cognitive flexibility.
    The paper relies on this to equate WCST performance with cognitive flexibility throughout Sections I and II.
  • domain assumption Human participants received task instructions of equivalent difficulty to the model prompts.
    Model instructions included an explicit rule-exclusivity constraint (Section IV.D); the human instruction adaptation (Section III.A) does not state whether humans received the same constraint, so comparing model CC with human CC assumes instruction equivalence.
  • domain assumption Model output parsing reliably recovered the intended card selection from free-text responses.
    Algorithm 1 parses the model's response to extract a selection, but no parse-failure analysis or validation of the parser is reported (Section III.A).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Visual Large Language Models Exhibit Human-Level Cognitive Flexibility in the Wisconsin Card Sorting Test." pith.science (2026). https://pith.science/paper/BFIWE5PO

@misc{pith2026250522112,
  author       = {Pith},
  title        = {Pith review of: Visual Large Language Models Exhibit Human-Level Cognitive Flexibility in the Wisconsin Card Sorting Test},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BFIWE5PO}},
  note         = {Machine review of arXiv:2505.22112}
}
read the original abstract

Cognitive flexibility has been extensively studied in human cognition but remains relatively unexplored in the context of Visual Large Language Models (VLLMs). This study assesses the cognitive flexibility of state-of-the-art VLLMs (GPT-4o, Gemini-1.5 Pro, and Claude-3.5 Sonnet) using the Wisconsin Card Sorting Test (WCST), a classic measure of set-shifting ability. Our results reveal that VLLMs achieve or surpass human-level set-shifting capabilities under chain-of-thought prompting with text-based inputs. However, their abilities are highly influenced by both input modality and prompting strategy. In addition, we find that through role-playing, VLLMs can simulate various functional deficits aligned with patients having impairments in cognitive flexibility, suggesting that VLLMs may possess a cognitive architecture, at least regarding the ability of set-shifting, similar to the brain. This study reveals the fact that VLLMs have already approached the human level on a key component underlying our higher cognition, and highlights the potential to use them to emulate complex brain processes.

Figures

Figures reproduced from arXiv: 2505.22112 by the authors.

Figure 1
Figure 1. WCST Procedure and Sample Stimuli. The WCST consists of matching response cards to four stimulus cards based on a sorting rule (color, shape, or number) that changes periodically. Participants receive feedback on the accuracy of each match but are not explicitly told the sorting rule or when it changes. error detection [18]. The fronto-parietal network, encompass￾ing these regions, dynamically reconfigures during fl… view at source ↗
Figure 2
Figure 2. WCST Task Performance Across Models and Conditions. The distribution of standardized Categories Completed (CC) scores for GPT-4o, Gemini￾1.5 Pro, and Claude-3.5 Sonnet under four experimental conditions: STA-VI (Straight to Answer - Visual Input), STA-TI (Straight to Answer - Textual Input), CoT-VI (Chain of Thought - Visual Input), and CoT-TI (Chain of Thought - Textual Input), with the human baseline performance. … view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 31 canonical work pages

  1. [1]

    Demystifying cognitive flexibility: Implications for clinical and developmental neuroscience,

    D. R. Dajani and L. Q. Uddin, “Demystifying cognitive flexibility: Implications for clinical and developmental neuroscience,”Trends in neurosciences, vol. 38, no. 9, pp. 571–578, 2015

    work page 2015

  2. [2]

    Cognitive and behavioural flexibility: neural mechanisms and clinical considerations,

    L. Q. Uddin, “Cognitive and behavioural flexibility: neural mechanisms and clinical considerations,”Nature Reviews Neuroscience, vol. 22, no. 3, pp. 167–179, 2021

    work page 2021

  3. [3]

    Exploring the nature of cognitive flexibility,

    T. Ionescu, “Exploring the nature of cognitive flexibility,”New ideas in psychology, vol. 30, no. 2, pp. 190–200, 2012

    2012

  4. [4]

    Prefrontal deep projection neurons enable cognitive flexibility via persistent feedback monitoring,

    T. Spellman, M. Svei, J. Kaminsky, G. Manzano-Nieves, and C. Liston, “Prefrontal deep projection neurons enable cognitive flexibility via persistent feedback monitoring,”Cell, vol. 184, no. 10, pp. 2750–2766, 2021

    work page 2021

  5. [5]

    Prefrontal cortex and neural mech- anisms of executive function,

    S. Funahashi and J. M. Andreau, “Prefrontal cortex and neural mech- anisms of executive function,”Journal of Physiology-Paris, vol. 107, no. 6, pp. 471–482, 2013

    work page 2013

  6. [6]

    Large language models are visual reasoning coordinators,

    L. Chen, B. Li, S. Shen, J. Yang, C. Li, K. Keutzer, T. Darrell, and Z. Liu, “Large language models are visual reasoning coordinators,”Advances in Neural Information Processing Systems, vol. 36, 2024

    work page 2024

  7. [7]

    Evaluating cognitive maps and planning in large language models with cogeval,

    I. Momennejad, H. Hasanbeig, F. Vieira Frujeri, H. Sharma, N. Jojic, H. Palangi, R. Ness, and J. Larson, “Evaluating cognitive maps and planning in large language models with cogeval,”Advances in Neural Information Processing Systems, vol. 36, 2024

    work page 2024

  8. [8]

    A survey on evaluation of large language models,

    Y . Chang, X. Wang, J. Wang, Y . Wu, L. Yang, K. Zhu, H. Chen, X. Yi, C. Wang, Y . Wanget al., “A survey on evaluation of large language models,”ACM Transactions on Intelligent Systems and Technology, vol. 15, no. 3, pp. 1–45, 2024

    2024

Show all 45 references
  1. [9]

    Integration of cognitive tasks into artificial general intelligence test for large models,

    Y . Qu, C. Wei, P. Du, W. Che, C. Zhang, W. Ouyang, Y . Bian, F. Xu, B. Hu, K. Duet al., “Integration of cognitive tasks into artificial general intelligence test for large models,”Iscience, vol. 27, no. 4, 2024

    2024

  2. [10]

    Hello gpt-4o,

    OpenAI, “Hello gpt-4o,” https://openai.com/index/hello-gpt-4o/, 2024, accessed: 2024-05-18

    2024

  3. [11]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context,

    M. Reid, N. Savinov, D. Teplyashin, D. Lepikhin, T. Lillicrap, J.-b. Alayrac, R. Soricut, A. Lazaridou, O. Firat, J. Schrittwieseret al., “Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context,”arXiv preprint arXiv:2403.05530, 2024

    2024 arXiv

  4. [12]

    Announcements: Claude 3.5 sonnet,

    Anthropic, “Announcements: Claude 3.5 sonnet,” https://www.anthropic. com/news/claude-3-5-sonnet, 2024, accessed: 2024-06-21

    2024

  5. [13]

    A simple objective technique for measuring flexibility in thinking,

    E. A. Berg, “A simple objective technique for measuring flexibility in thinking,”The Journal of general psychology, vol. 39, no. 1, pp. 15–22, 1948

    1948

  6. [14]

    Considerations for using the wisconsin card sorting test to assess cognitive flexibility,

    S. Miles, C. A. Howlett, C. Berryman, M. Nedeljkovic, G. L. Moseley, and A. Phillipou, “Considerations for using the wisconsin card sorting test to assess cognitive flexibility,”Behavior research methods, vol. 53, no. 5, pp. 2083–2091, 2021

    2021

  7. [15]

    The wcst-64: A standardized short-form of the wisconsin card sorting test,

    K. W. Greve, “The wcst-64: A standardized short-form of the wisconsin card sorting test,”The Clinical Neuropsychologist, vol. 15, no. 2, pp. 228–234, 2001

    2001

  8. [16]

    Wisconsin card sorting test as a measure of executive function impairments in stroke patients,

    K. Jodzio and D. Biechowska, “Wisconsin card sorting test as a measure of executive function impairments in stroke patients,”Applied neuropsychology, vol. 17, no. 4, pp. 267–277, 2010. 11

    2010

  9. [17]

    Using fmri to decompose the neural processes underlying the wisconsin card sorting test,

    C.-H. Lie, K. Specht, J. C. Marshall, and G. R. Fink, “Using fmri to decompose the neural processes underlying the wisconsin card sorting test,”Neuroimage, vol. 30, no. 3, pp. 1038–1049, 2006

    2006

  10. [18]

    Common and distinct mechanisms of cognitive flexibility in prefrontal cortex,

    C. Kim, N. F. Johnson, S. E. Cilles, and B. T. Gold, “Common and distinct mechanisms of cognitive flexibility in prefrontal cortex,”Journal of Neuroscience, vol. 31, no. 13, pp. 4771–4779, 2011

    2011

  11. [19]

    Flexible adjustment of the effective connectivity between the fronto-parietal and visual regions supports cognitive flexibility,

    L. Qiao, M. Xu, X. Luo, L. Zhang, H. Li, and A. Chen, “Flexible adjustment of the effective connectivity between the fronto-parietal and visual regions supports cognitive flexibility,”NeuroImage, vol. 220, p. 117158, 2020

    2020

  12. [20]

    Executive functions,

    A. Diamond, “Executive functions,”Annual Review of Psychology, vol. 64, no. 1, pp. 135–168, 2013

    2013

  13. [21]

    On the historical and conceptual background of the wisconsin card sorting test,

    P. Eling, K. Derckx, and R. Maes, “On the historical and conceptual background of the wisconsin card sorting test,”Brain and cognition, vol. 67, no. 3, pp. 247–253, 2008

    2008

  14. [22]

    Adult clinical neuropsychology: lessons from studies of the frontal lobes,

    D. T. Stuss and B. Levine, “Adult clinical neuropsychology: lessons from studies of the frontal lobes,”Annual review of psychology, vol. 53, no. 1, pp. 401–433, 2002

    2002

  15. [23]

    The dimensional change card sort (dccs): A method of assessing executive function in children,

    P. D. Zelazo, “The dimensional change card sort (dccs): A method of assessing executive function in children,”Nature protocols, vol. 1, no. 1, pp. 297–301, 2006

    2006

  16. [24]

    Emo- tional processing and executive functions in major depressive disorder: dorsal prefrontal activity correlates with performance in the intra–extra dimensional set shift,

    A. Heinzel, G. Northoff, H. Boeker, P. Boesiger, and S. Grimm, “Emo- tional processing and executive functions in major depressive disorder: dorsal prefrontal activity correlates with performance in the intra–extra dimensional set shift,”Acta neuropsychiatrica, vol. 22, no. 6,...

    2010

  17. [25]

    Testing theory of mind in large language models and humans,

    J. W. Strachan, D. Albergo, G. Borghini, O. Pansardi, E. Scaliti, S. Gupta, K. Saxena, A. Rufo, S. Panzeri, G. Manziet al., “Testing theory of mind in large language models and humans,”Nature Human Behaviour, pp. 1–11, 2024

    2024

  18. [26]

    An empirical investigation of the impact of chatgpt on creativity,

    B. C. Lee and J. Chung, “An empirical investigation of the impact of chatgpt on creativity,”Nature Human Behaviour, pp. 1–9, 2024

    2024

  19. [27]

    Test of time: A benchmark for eval- uating llms on temporal reasoning,

    B. Fatemi, M. Kazemi, A. Tsitsulin, K. Malkan, J. Yim, J. Palowitch, S. Seo, J. Halcrow, and B. Perozzi, “Test of time: A benchmark for eval- uating llms on temporal reasoning,”arXiv preprint arXiv:2406.09170, 2024

    2024 arXiv

  20. [28]

    Vision language models are blind,

    P. Rahmanzadehgervi, L. Bolton, M. R. Taesiri, and A. T. Nguyen, “Vision language models are blind,”arXiv preprint arXiv:2407.06581, 2024

    2024 arXiv

  21. [29]

    Chal- lenging chatgpt’intelligence’with human tools: a neuropsychological investigation on prefrontal functioning of a large language model,

    R. Loconte, G. Orru, M. Tribastone, P. Pietrini, and G. Sartori, “Chal- lenging chatgpt’intelligence’with human tools: a neuropsychological investigation on prefrontal functioning of a large language model,” Intelligence, 2023

    2023

  22. [30]

    Using large language models to simulate multiple humans and replicate human subject studies,

    G. V . Aher, R. I. Arriaga, and A. T. Kalai, “Using large language models to simulate multiple humans and replicate human subject studies,” in International Conference on Machine Learning. PMLR, 2023, pp. 337–371

    2023

  23. [31]

    Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language processing,

    P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, and G. Neubig, “Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language processing,”ACM Computing Surveys, vol. 55, no. 9, pp. 1–35, 2023

    2023

  24. [32]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V . Le, D. Zhouet al., “Chain-of-thought prompting elicits reasoning in large language models,”Advances in neural information processing systems, vol. 35, pp. 24 824–24 837, 2022

    2022

  25. [33]

    A survey of chain of thought reasoning: Advances, frontiers and future,

    Z. Chu, J. Chen, Q. Chen, W. Yu, T. He, H. Wang, W. Peng, M. Liu, B. Qin, and T. Liu, “A survey of chain of thought reasoning: Advances, frontiers and future,”arXiv preprint arXiv:2309.15402, 2023

    2023 arXiv

  26. [34]

    Large lan- guage models are zero-shot reasoners,

    T. Kojima, S. S. Gu, M. Reid, Y . Matsuo, and Y . Iwasawa, “Large lan- guage models are zero-shot reasoners,”Advances in neural information processing systems, vol. 35, pp. 22 199–22 213, 2022

    2022

  27. [35]

    Self-consistency improves chain of thought reasoning in language models,

    X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, S. Narang, A. Chowdh- ery, and D. Zhou, “Self-consistency improves chain of thought reasoning in language models,”arXiv preprint arXiv:2203.11171, 2022

    2022 arXiv

  28. [36]

    M ˆ3cot: A novel benchmark for multi-domain multi-step multi-modal chain-of- thought,

    Q. Chen, L. Qin, J. Zhang, Z. Chen, X. Xu, and W. Che, “M ˆ3cot: A novel benchmark for multi-domain multi-step multi-modal chain-of- thought,”arXiv preprint arXiv:2405.16473, 2024

    2024 arXiv

  29. [37]

    Least-to-most prompting enables complex reasoning in large language models,

    D. Zhou, N. Sch ¨arli, L. Hou, J. Wei, N. Scales, X. Wang, D. Schu- urmans, C. Cui, O. Bousquet, Q. Leet al., “Least-to-most prompting enables complex reasoning in large language models,”arXiv preprint arXiv:2205.10625, 2022

    2022 arXiv

  30. [38]

    Automatic prompt optimization with

    R. Pryzant, D. Iter, J. Li, Y . T. Lee, C. Zhu, and M. Zeng, “Automatic prompt optimization with” gradient descent” and beam search,”arXiv preprint arXiv:2305.03495, 2023

    2023 arXiv

  31. [39]

    Large language models are human-level prompt engineers,

    Y . Zhou, A. I. Muresanu, Z. Han, K. Paster, S. Pitis, H. Chan, and J. Ba, “Large language models are human-level prompt engineers,”arXiv preprint arXiv:2211.01910, 2022

    2022 arXiv

  32. [40]

    D. J. Schretlen,Modified Wisconsin Card Sorting Test®: M-WCST; Professional Manual. PAR, 2010

    2010

  33. [41]

    An integrative theory of prefrontal cortex function,

    E. K. Miller and J. D. Cohen, “An integrative theory of prefrontal cortex function,”Annual review of neuroscience, vol. 24, no. 1, pp. 167–202, 2001

    2001

  34. [42]

    Is there a dysexecutive syndrome?

    D. T. Stuss and M. P. Alexander, “Is there a dysexecutive syndrome?” Philosophical Transactions of the Royal Society B: Biological Sciences, vol. 362, no. 1481, pp. 901–915, 2007

    2007

  35. [43]

    The involvement of orbitofrontal cerebrum in cognitive tasks,

    D. T. Stuss, D. Benson, W. Weir, M. Naeser, I. Lieberman, and D. Ferrill, “The involvement of orbitofrontal cerebrum in cognitive tasks,”Neuropsychologia, vol. 21, no. 3, pp. 235–248, 1983

    1983

  36. [44]

    Wisconsin card sorting test performance in patients with focal frontal and posterior brain damage: effects of lesion location and test structure on separable cognitive processes,

    D. Stuss, B. Levine, M. Alexander, J. Hong, C. Palumbo, L. Hamer, K. Murphy, and D. Izukawa, “Wisconsin card sorting test performance in patients with focal frontal and posterior brain damage: effects of lesion location and test structure on separable cognitive processes,” Neu...

    2000

  37. [45]

    Effects of different brain lesions on card sorting: The role of the frontal lobes,

    B. Milner, “Effects of different brain lesions on card sorting: The role of the frontal lobes,”Archives of neurology, vol. 9, no. 1, pp. 90–100, 1963

    1963

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.