FinMaster introduces a simulator-driven benchmark with 183 financial tasks and finds LLM accuracy collapses from about 96% on basic literacy to below 40% on multi-step accounting, auditing, and consulting workflows.
SECQUE: A Benchmark for Evaluating Real-World Financial Analysis Capabilities
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
We introduce SECQUE, a comprehensive benchmark for evaluating large language models (LLMs) in financial analysis tasks. SECQUE comprises 565 expert-written questions covering SEC filings analysis across four key categories: comparison analysis, ratio calculation, risk assessment, and financial insight generation. To assess model performance, we develop SECQUE-Judge, an evaluation mechanism leveraging multiple LLM-based judges, which demonstrates strong alignment with human evaluations. Additionally, we provide an extensive analysis of various models' performance on our benchmark. By making SECQUE publicly available, we aim to facilitate further research and advancements in financial AI.
citation-role summary
dataset 1
citation-polarity summary
fields
cs.AI 1years
2025 1verdicts
CONDITIONAL 1roles
dataset 1polarities
baseline 1representative citing papers
citing papers explorer
-
FinMaster: A Holistic Benchmark for Mastering Full-Pipeline Financial Workflows with LLMs
FinMaster introduces a simulator-driven benchmark with 183 financial tasks and finds LLM accuracy collapses from about 96% on basic literacy to below 40% on multi-step accounting, auditing, and consulting workflows.