Pith. sign in

REVIEW 4 cited by

Can GPT models be Financial Analysts? An Evaluation of ChatGPT and GPT-4 on mock CFA Exams

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.08678 v1 pith:DMGG7DAH submitted 2023-10-12 cs.CL cs.AIq-fin.GN

Ethan Callanan , Amarachi Mbakwe , Antony Papadimitriou , Yulong Pei , Mathieu Sibue , Xiaodan Zhu , Zhiqiang Ma , Xiaomo Liu
show 1 more author
Sameena Shah
This is my paper
classification cs.CLcs.AIq-fin.GN
keywords financialllmsmodelsevaluationanalysischatgptexamsgpt-4
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large Language Models (LLMs) have demonstrated remarkable performance on a wide range of Natural Language Processing (NLP) tasks, often matching or even beating state-of-the-art task-specific models. This study aims at assessing the financial reasoning capabilities of LLMs. We leverage mock exam questions of the Chartered Financial Analyst (CFA) Program to conduct a comprehensive evaluation of ChatGPT and GPT-4 in financial analysis, considering Zero-Shot (ZS), Chain-of-Thought (CoT), and Few-Shot (FS) scenarios. We present an in-depth analysis of the models' performance and limitations, and estimate whether they would have a chance at passing the CFA exams. Finally, we outline insights into potential strategies and improvements to enhance the applicability of LLMs in finance. In this perspective, we hope this work paves the way for future studies to continue enhancing LLMs for financial reasoning through rigorous evaluation.

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Evaluating Large Language Models as Expert Annotators

    cs.CL 2025-08 unverdicted novelty 6.0 of 10

    Material Fingerprinting recovers the form and parameters of hyperelastic material models by nearest-neighbor matching of test data against a simulated fingerprint database: exact at zero noise, degrading under 5% noise.

  2. AI Analyst: Framework and Comprehensive Evaluation of Large Language Models for Financial Time Series Report Generation

    cs.CL 2025-07 conditional novelty 6.0 of 10

    LLMs such as GPT-4o can generate coherent financial reports from time series data, and a proposed highlighting system categorizes report segments by whether they stem from data, reasoning, or external knowledge.

  3. ACD-CLIP: Decoupling Representation and Dynamic Fusion for Zero-Shot Anomaly Detection

    cs.CV 2025-08 unverdicted novelty 5.0 of 10

    ACD-CLIP improves zero-shot anomaly detection by co-designing a convolutional low-rank adapter with a dynamic fusion gateway that modulates text prompts from visual context.

  4. Towards Automated Regulatory Compliance Verification in Financial Auditing with Large Language Models

    cs.CL 2025-07 conditional novelty 4.0 of 10

    In a small evaluation with PwC data, Llama-2-70b beats GPT models at the 'no compliance' class for IFRS reports, but the result is based on a single selected prompt and a 100-item sample, and the data/code are not released.

Pith tools