The reviewed record of science sign in
Pith

arxiv: 2308.09975 · v2 · pith:STOYC5PU · submitted 2023-08-19 · cs.CL

FinEval: A Chinese Financial Domain Knowledge Evaluation Benchmark for Large Language Models

Xin Guo , Haotian Xia , Zhaowei Liu , Hanyang Cao , Zhi Yang , Zhiqiang Liu , Sizhe Wang , Jinyi Niu
show 9 more authors
Chuqi Wang Yanhui Wang Xiaolong Liang Xiaoming Huang Bing Zhu Zhongyu Wei Yun Chen Weining Shen Liwen Zhang
This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:STOYC5PUrecord.jsonopen to challenge →

classification cs.CL
keywords financialknowledgedomainquestionssecurityagentbenchmarkfineval
0
0 comments X
read the original abstract

Large language models have demonstrated outstanding performance in various natural language processing tasks, but their security capabilities in the financial domain have not been explored, and their performance on complex tasks like financial agent remains unknown. This paper presents FinEval, a benchmark designed to evaluate LLMs' financial domain knowledge and practical abilities. The dataset contains 8,351 questions categorized into four different key areas: Financial Academic Knowledge, Financial Industry Knowledge, Financial Security Knowledge, and Financial Agent. Financial Academic Knowledge comprises 4,661 multiple-choice questions spanning 34 subjects such as finance and economics. Financial Industry Knowledge contains 1,434 questions covering practical scenarios like investment research. Financial Security Knowledge assesses models through 1,640 questions on topics like application security and cryptography. Financial Agent evaluates tool usage and complex reasoning with 616 questions. FinEval has multiple evaluation settings, including zero-shot, five-shot with chain-of-thought, and assesses model performance using objective and subjective criteria. Our results show that Claude 3.5-Sonnet achieves the highest weighted average score of 72.9 across all financial domain categories under zero-shot setting. Our work provides a comprehensive benchmark closely aligned with Chinese financial domain.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. EconWebArena: Benchmarking Autonomous Agents on Economic Tasks in Realistic Web Environments

    cs.CL 2025-06 unverdicted novelty 7.0

    EconWebArena is a new benchmark with 360 curated economic tasks across 82 authoritative websites for evaluating multimodal web agents on navigation, grounding, and data extraction.

  2. FinDocMRE: A Benchmark for Document-Level Financial Multimodal Reasoning Evaluation

    cs.CE 2026-05 unverdicted novelty 6.0

    FinDocMRE is a new multi-image document-level benchmark spanning 12 financial domains and 5 task types, showing that 11 tested LMMs all score below 65 overall with particular weaknesses in numerical estimation and cro...

  3. MinT: Managed Infrastructure for Training and Serving Millions of LLMs

    cs.LG 2026-05 unverdicted novelty 6.0

    MinT enables efficient management of million-scale LoRA-adapted LLM policies over shared 1T-parameter base models by moving only small adapters through training and serving pipelines.

  4. MinT: Managed Infrastructure for Training and Serving Millions of LLMs

    cs.LG 2026-05 unverdicted novelty 5.0

    MinT is a system for managing million-scale LoRA adapter catalogs on shared 1T-parameter base models, with reported efficiency gains in adapter movement, multi-policy training, and catalog addressability.

  5. JT-SAFE-V2: Safety-by-Design Foundation Model with World-Context Data

    cs.AI 2026-05 unverdicted novelty 4.0

    JT-Safe-V2 is a safety-by-design LLM that reports SOTA scores on both capability and safety benchmarks while Safe-MoMA cuts inference cost over 30 percent.