Pith. sign in

REVIEW 5 major objections 8 minor 27 references

FinMaster: A Holistic Benchmark for Mastering Full-Pipeline Financial Workflows with LLMs

T0 review · 5 major / 8 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read FinMaster claims to be the first benchmark spanning full-pipeline financial workflows, and its experiments show LLM accuracy falling from over 90% on basic financial literacy to roughly 40% on complex multi-step tasks.

desk verdict FinMaster is a genuinely broad finance-LLM benchmark — simulator, 183 tasks, five company types — but its headline accuracy collapse is partly an artifact of an undefined exact-match scorer, and the abstract's specific numbers are missing from the results. read the letter →

arxiv 2505.13533 v1 pith:37DZ4I2P submitted 2025-05-18 cs.AI cs.LGq-fin.GN

Junzhe Jiang , Chang Yang , Aixin Cui , Sihan Jin , Ruiyu Wang , Bo Li , Xiao Huang , Dongning Sun
show 1 more author
Xinrun Wang
This is my paper · ORCID
classification cs.AIcs.LGq-fin.GN
keywords financialbenchmarklargelanguagemodelsworkflowsaccountingauditingconsultingsyntheticdatareasoning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FinMaster aims to be the first benchmark that measures large language models across an entire financial pipeline—basic literacy, accounting, auditing, and consulting—rather than isolated question-answering tasks. It combines a financial data simulator called FinSim, a suite of 183 tasks called FinSuite, and a unified evaluation interface called FinEval. On these tasks, the best current LLMs score above 90% on simple financial literacy questions but fall to about 40% on multi-step reasoning, and almost no model can generate a complete financial statement from transaction records. The paper argues that this accuracy collapse is driven by propagation of computation errors across multi-source data and that FinMaster provides the missing testbed for improving LLMs in realistic financial work.

What carries the argument

The central mechanism is the three-dimensional task metric $\langle \alpha, \beta, \gamma \rangle$, where $\alpha$ counts the fundamental data items needed, $\beta$ counts the distinct input sources that must be integrated, and $\gamma$ counts the required output values. This triple drives task difficulty and enables error attribution: the same accounting calculation can be posed with increasing $\alpha$ to expose error propagation, with cross-source $\beta$ to expose data-integration failure, and with wider $\gamma$ to expose output-structure collapse. Supporting it is FinSim's simulator, which generates synthetic transaction records and financial statements for five company archetypes, and FinEval's unified prompt-and-parse interface. The metric, not any single task, is what makes the benchmark's complexity claims comparable across accounting, auditing, and consulting.

What would settle it

Take FinMaster's task suite and run it on an equivalent set of real, privacy-stripped transaction ledgers from companies resembling the five simulated archetypes; if model accuracy on the real ledgers differs from the synthetic versions by more than the paper's observed run-to-run variance, the transfer claim fails. A cheaper check is to compute basic statement-arithmetic identities, such as total assets equals liabilities plus equity, on FinSim's outputs and on real audited statements and compare the error rates.

Watch

Extended reading notes

Core claim

The central claim is that LLMs are competent at recognizing financial concepts but not at executing financial workflows. Using FinSim, which simulates five company archetypes and generates transaction-level data, statements, and deliberately injected audit errors, the paper builds 183 tasks spanning 64 financial-literacy items, 49 accounting items, 35 auditing items, and 35 consulting items. Across GPT-4o-mini, GPT-4.1 variants, Claude-3.7-Sonnet, DeepSeek-V3, and o3-mini, accuracy drops from over 90% on basic tasks to around 40% on complex multi-step scenarios; single-metric calculations fall from 58% to 37% when multiple metrics must be computed together, and complete financial statement generation is at or below 3% for essentially all models tested. The paper attributes the failures to domain-knowledge gaps, omitted data during long reasoning chains, floating-point rounding, and internal consistency errors, and it presents FinMaster as the first benchmark that evaluates the whole accounting-auditing-consulting chain rather than isolated tasks.

Load-bearing premise

The load-bearing premise is that FinSim's synthetic transactions and statements faithfully reproduce real-world financial workflows, so that accuracy measured on them transfers to real companies with minimal distribution shift; the paper offers no comparison between its synthetic data and real financial data.

Editorial extensions

If this is right

  • If FinMaster is right, LLM accuracy on financial work should be reported by pipeline stage rather than by aggregate accuracy, because literacy, statement generation, auditing, and consulting show different failure modes.
  • A model's token usage does not predict accuracy: o3-mini uses far more tokens than DeepSeek-V3 yet outperforms it in auditing, so efficiency and reasoning quality matter more than compute spent.
  • Expanding the transaction cycle from 200 to 400 records degrades accounting and auditing accuracy while leaving consulting largely stable, implying that input length weakens procedural tasks before it affects analysis of already-prepared statements.
  • The near-zero full-statement generation accuracy sets a concrete floor for progress: any LLM claiming financial competence should first improve on FinMaster's statement-generation tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable next step would be to use FinSim's generator to create paired synthetic and real-world financial datasets from the same company archetypes and measure whether accuracy differences on real data are explained by the paper's complexity metric; the paper does not run this validation.
  • The $\langle \alpha, \beta, \gamma \rangle$ metric could be reused outside finance as a task-complexity tagging scheme for any multi-source quantitative workflow, such as tax preparation or supply-chain analysis.
  • Because the benchmark finds that isolated audit errors are harder for models than several correlated errors, a practical extension is to train or prompt models to check entries against expected-value ranges rather than only against neighboring rows.
  • FinSim's generated statements could double as training data for finance-specialized fine-tuning; the paper mentions this as future work but does not demonstrate it.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 8 minor

Summary. FinMaster introduces a three-module benchmark for evaluating LLMs on full-pipeline financial workflows. FinSim is a configurable generator of synthetic transactions and financial statements for five stylized company types; FinSuite contains 183 tasks across financial literacy (64), accounting (49), auditing (35), and consulting (35), each labeled with a three-dimensional complexity descriptor ⟨α,β,γ⟩ (input cardinality, cross-source integration, output dimensionality); FinEval provides a prompt template, API execution via LiteLLM, and JSON-based response parsing. Seven LLMs (GPT-4o-mini, GPT-4.1, GPT-4.1-mini, GPT-4.1-nano, o3-mini, DeepSeek-V3, Claude-3.7-Sonnet) are evaluated, with ablations across company types and operation-cycle lengths, token-usage analysis, and qualitative failure-case analysis. The headline claims are that accuracy falls from over 90% on basic literacy tasks to about 40% on complex multi-step tasks, that full statement generation is near 0% for most models, and that single-metric accuracy of 58% drops to 37% in multimetric scenarios, which the authors attribute to computational error propagation.

Significance. If the quantitative claims survive re-scoring, FinMaster would be a substantial community resource: it ships code and a leaderboard, provides a privacy-compliant synthetic data generator with controllable ground truth, covers the accounting-auditing-consulting pipeline more extensively than prior benchmarks (FinQA, FinBen, SECQUE), and reports a broad model sweep with company-type and operation-cycle ablations. The benchmark is not circular: the ⟨α,β,γ⟩ labels are descriptive configuration attributes rather than parameters fitted to model performance, and the reported accuracies are external measurements of LLM behavior. The qualitative trend — strong performance on definition-to-value extraction, degraded performance on multi-source multi-step computations, and greater variability for weaker models across simulated company types — is visible in the tables and figures, and the failure-case analysis (Table 2) is informative.

major comments (5)
  1. [§3.3–§4, Tables 2, 17–22] The manuscript never defines the accuracy metric used by FinEval, and the evidence indicates all-or-nothing exact matching. In Failure Case 3 of Table 2, o3-mini's ROA of −9.55% is counted as a failure against a ground truth of −9.56%, i.e., a 0.01 percentage-point rounding difference, implying string/numeric equality with zero tolerance. Statement-generation tasks require 31, 37, or 38 line items (Tables 17–19; the balance sheet requires initial and final values for each of its 37 items), and auditing tasks require up to γ=11 simultaneous outputs (Table 22). Under all-or-nothing scoring, per-instance accuracy is the product of per-item accuracies, so a model that is 95% correct per line item would score roughly 0.95^31 ≈ 20% on an income statement, 0.95^37 ≈ 15% on a balance sheet, and 0.95^38 ≈ 14% on a cash-flow statement. The universal 0% scores for [31,1,1], [37,1,2], and [38,1,1] in Tables 27–33, and the abstract's '58% to 37%' single-to-multimetric decline, are therefore at least partly combinatorial artifacts of the scoring rule rather than direct measurements of financial reasoning. The authors should state the scoring rule explicitly, report per-line-item or partial-credit accuracy alongside instance-level accuracy, and adopt tolerance-based numeric comparison (or justify exact match) before the accuracy-collapse interpretation can be accepted.
  2. [Abstract and §4.2] The headline quantitative claims are not traceable to the reported results. The claim that 'single-metric calculations initially demonstrating 58% accuracy decreased to 37% in multimetric scenarios' cannot be reconstructed from the tables: the closest entries are GPT-4o-mini's 58.89% on the [2,1,1] Current Ratio task (Table 32) and that model's 37.43% aggregate consulting accuracy (Table 25), but no aggregation rule is given, and the abstract presents these as a general finding about error propagation rather than a single model's numbers. Similarly, §4.2 states that accuracy 'drop[s] to 3% on statement generation tasks,' yet Tables 27–33 report 0% for every model on the [31,1,1], [37,1,2], and [38,1,1] statement-generation tasks; the 3% figure does not correspond to any statement-generation entry in the tables. The abstract and §4.2 should specify the model set, task subset, and aggregation rule used to obtain each headline number.
  3. [Tables 3–5 (FinSim output)] The sample ground-truth statements are internally inconsistent. In Table 3, Total Assets is 13,216,409.49 while Total Liabilities and Equity is 13,216,409, a difference of 0.49; in Table 4, Net Profit is −1,645,141.43, whereas Table 3 reports Retained Earnings of −1,645,141.46. Both discrepancies are at the level of precision the paper itself treats as consequential (Table 2 counts a 0.01 rounding difference as a failure). This conflicts with FAQ A.1's claim of 'precise, verifiable ground truth' and with the paper's low-error-tolerance motivation. The authors should document the simulator's rounding conventions, add statement-balancing checks to the generation pipeline, and clarify how the exact-match evaluation treats simulator-internal rounding differences.
  4. [§3.2, ⟨α,β,γ⟩ framework] The paper claims that ⟨α,β,γ⟩ 'precisely characterize[s] task features and complexity' and 'enables fine-grained error attribution,' but no analysis links accuracy to any of the three components. The auditing tasks all have α=13 yet scores range from 0% to 93% across β and γ (Tables 27–33), and Claude-3.7-Sonnet scores 93.33% on [14,3,5] but 0% on [15,3,7]. If the framework is to be a contribution, the paper should present accuracy as a function of α, β, and γ (e.g., a small regression or ablation); otherwise the claims should be softened to describe the labels as configurational descriptors rather than validated complexity measures.
  5. [FAQ A.1] The load-bearing premise of the benchmark's practical relevance is the claim that simulated data 'ensures LLM performance on simulated data transfers reliably to real-world scenarios, with minimal OOD divergence.' No comparison between FinSim's synthetic transactions/statements and real financial data is provided anywhere in the manuscript, and no external validation (e.g., review by practitioners, comparison with real statements of similar company profiles, or a synthetic-to-real transfer experiment) is offered. The paper should either provide such evidence or explicitly reframe cross-domain transferability as an open question rather than an established property.
minor comments (8)
  1. [§3.2] 'Fugure 4b' should read 'Figure 4b'.
  2. [Figures 15 and 19 captions] Figure 15's caption refers to 'Type IIIV' (presumably Type IV), and Figure 19's caption reads 'Accounting-Accounting-Companies of Type III'; both should be corrected.
  3. [Table 5] The line 'Inventory (5090000))' contains an extra closing parenthesis.
  4. [Table 1] The comparison table's glyphs render as '%' where check/cross marks were clearly intended, making the benchmark comparison difficult to read; please fix the symbol encoding.
  5. [§5 and §4.4] The conclusion contains a duplicated clause ('indicating that computational error propagation significantly impairs accuracy, indicating computational error propagation worsens performance'), and §4.4 repeats the sentence 'These findings indicate that company-specific operations increase model variability...' twice.
  6. [§3.3 and Appendix G] The experimental protocol does not state the number of evaluation runs, sampling temperature, or seed handling, and Tables 26–33 report single percentages without any variance or sample-size information, which makes the claimed 'consistent, reproducible generation' difficult to verify.
  7. [Table 12] The formula for the Cash to Current Debt Ratio, '(Cash and Cash Equivalents – Ending Balance)/Current Liabilities,' appears malformed; presumably the parenthetical is meant to label a line item rather than define a subtraction.
  8. [Appendix G.2] The prompt template contains placeholder formatting artifacts (e.g., unexpanded angle-bracket placeholders with stray underscores) and should match the released code in the repository.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found; FinMaster's quantitative claims are external measurements on a generated benchmark, not consequences of its own definitions.

full rationale

Walking FinMaster's derivation chain: FinSim generates synthetic transactions and financial statements; FinSuite defines 183 tasks with descriptive complexity labels (alpha, beta, gamma); FinEval prompts LLMs, parses JSON responses, and records accuracy; the paper then reports accuracy declines. None of these steps fits a parameter to the target result or defines an input in terms of an output. The reported drop from over 90% to 40%, and from 58% to 37%, comes from external LLM responses, so it cannot be circular by construction. The alpha, beta, gamma labels are descriptive dimensions, not fitted quantities; no equation in the paper forces accuracy to equal a product of per-item probabilities. There are no load-bearing self-citations: the cited works (FinQA, PIXIU, FinBen, FinEval, SECQUE, etc.) are external benchmarks, and no uniqueness theorem or rescaling ansatz is imported from the authors' prior work. The weakest premises, such as FAQ A.1's claim that simulated data transfers to real-world scenarios with minimal out-of-distribution divergence, are external-validity concerns rather than circular reductions, because a benchmark can be non-circular yet have questionable validity. Similarly, the concern that all-or-nothing exact matching on multi-item outputs may manufacture low accuracy on high-gamma tasks is a measurement-validity issue, not a circularity issue: the reported accuracy is still an externally observed quantity. Under the stated rules, no circular step can be exhibited, so the appropriate score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central contribution is an evaluation framework, not a physical theory. The main discretionary inputs are the hand-chosen simulator parameters, the operation cycle lengths, and the undisclosed error-injection rates; these set the difficulty of the benchmark and therefore shape all reported accuracy numbers. No new scientific entities are introduced.

free parameters (3)
  • Company type parameters
    Hand-chosen initial capital, purchase unit price, profit margin ranges, purchase/sales frequencies, and credit ratios in Table 6 define the five simulated business models; these choices shape all generated data and task difficulty but are not fitted to real company data.
  • Operation cycle length = 200 vs 400 transactions
    The ablation uses transaction volume as a proxy for operational time; the choice of 200 and 400 transaction counts is arbitrary and affects accounting and auditing accuracy results.
  • Audit error injection rates = not specified
    Twelve audit error types are embedded via randomized generation algorithms, but the paper does not report injection probabilities, so auditing task difficulty is not fully reproducible from the manuscript.
assumptions (3)
  • domain assumption FinSim-generated financial statements are internally correct and follow IFRS/GAAP standards.
    Accounting and consulting ground truth assumes the simulator aggregates transactions into statements without error; no independent verification of a generated statement is provided beyond one sample appendix table.
  • domain assumption Performance on synthetic data transfers to real-world financial workflows with minimal out-of-distribution divergence.
    FAQ A.1 asserts this transferability; the paper contains no synthetic-versus-real comparison to support it.
  • domain assumption The regex-based answer parser in FinEval correctly handles all valid answer formats.
    Structured answer parsing uses regular expressions to extract JSON; models that produce equivalent but differently formatted answers may be scored as incorrect.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FinMaster: A Holistic Benchmark for Mastering Full-Pipeline Financial Workflows with LLMs." pith.science (2026). https://pith.science/paper/37DZ4I2P

@misc{pith2026250513533,
  author       = {Pith},
  title        = {Pith review of: FinMaster: A Holistic Benchmark for Mastering Full-Pipeline Financial Workflows with LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/37DZ4I2P}},
  note         = {Machine review of arXiv:2505.13533}
}
read the original abstract

Financial tasks are pivotal to global economic stability; however, their execution faces challenges including labor intensive processes, low error tolerance, data fragmentation, and tool limitations. Although large language models (LLMs) have succeeded in various natural language processing tasks and have shown potential in automating workflows through reasoning and contextual understanding, current benchmarks for evaluating LLMs in finance lack sufficient domain-specific data, have simplistic task design, and incomplete evaluation frameworks. To address these gaps, this article presents FinMaster, a comprehensive financial benchmark designed to systematically assess the capabilities of LLM in financial literacy, accounting, auditing, and consulting. Specifically, FinMaster comprises three main modules: i) FinSim, which builds simulators that generate synthetic, privacy-compliant financial data for companies to replicate market dynamics; ii) FinSuite, which provides tasks in core financial domains, spanning 183 tasks of various types and difficulty levels; and iii) FinEval, which develops a unified interface for evaluation. Extensive experiments over state-of-the-art LLMs reveal critical capability gaps in financial reasoning, with accuracy dropping from over 90% on basic tasks to merely 40% on complex scenarios requiring multi-step reasoning. This degradation exhibits the propagation of computational errors, where single-metric calculations initially demonstrating 58% accuracy decreased to 37% in multimetric scenarios. To the best of our knowledge, FinMaster is the first benchmark that covers full-pipeline financial workflows with challenging tasks. We hope that FinMaster can bridge the gap between research and industry practitioners, driving the adoption of LLMs in real-world financial practices to enhance efficiency and accuracy.

Figures

Figures reproduced from arXiv: 2505.13533 by the authors.

Figure 1
Figure 1. The three main modules of FinMaster. financial statements, e.g., decimal errors or misclassifications, can trigger compliance risks or market volatility; iii) data fragmentation: financial data always originates from diverse sources, each with unique data structures and update frequencies. However, the latency in real-time scenarios and the poor compatibility between systems in integrating heterogeneous data may lea… view at source ↗
Figure 2
Figure 2. Companies Comparison We develop a financial data simulator, which simulates the daily financial activities of different types of companies and generates transaction records and financial statements. Types of Companies. To mirror real-world market diver￾sity, FinSim incorporates simulators of five distinct com￾pany types: Type I considers capital goods manufacturers with capital-intensive operations, low sales freque… view at source ↗
Figure 3
Figure 3. The workflow of FinSim. Generation Process [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (28 more)
Figure 4
Figure 4. Figure 4: Task taxonomy and architecture Financial Literacy Tasks. The possession of fundamental financial literacy and an understanding of financial statements forms the foundation for addressing complex tasks in accounting, auditing, and consulting. Before tackling domain-spec…
Figure 5
Figure 5. Figure 5: Financial literacy result FinMaster first assesses the basic financial knowledge and understanding of the financial statements of the selected models. As shown in [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Performance comparison across task types (color bars: model accuracy) [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Token usage and performance analysis across task types (color bars: model accuracy) [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Model performance comparison for different companies (error bars: standard deviation) [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Model accuracy for different operation time (error bars: standard deviation) Companies Operation Duration Comparison. In the simulation process, FinSim uses transac￾tion volume as a proxy for operational time pe￾riods, comparing short-cycle scenarios with 200 transacti…
Figure 10
Figure 10. Figure 10: Financial Literacy prompt and completion Token Result [PITH_FULL_IMAGE:figures/full_fig_p036_10.png]
Figure 11
Figure 11. Figure 11: Main task prompt and completion result for model Comparison [PITH_FULL_IMAGE:figures/full_fig_p036_11.png]
Figure 12
Figure 12. Figure 12: Accuracy of type I companies in financial literacy [PITH_FULL_IMAGE:figures/full_fig_p047_12.png]
Figure 13
Figure 13. Figure 13: Accuracy of Type II Companies in Financial Literacy [PITH_FULL_IMAGE:figures/full_fig_p047_13.png]
Figure 14
Figure 14. Figure 14: Accuracy of Type III Companies in Financial Literacy [PITH_FULL_IMAGE:figures/full_fig_p047_14.png]
Figure 15
Figure 15. Figure 15: Accuracy of Type IIIV Companies in Financial Literacy [PITH_FULL_IMAGE:figures/full_fig_p047_15.png]
Figure 16
Figure 16. Figure 16: Accuracy of Type V Companies in Financial Literacy [PITH_FULL_IMAGE:figures/full_fig_p047_16.png]
Figure 17
Figure 17. Figure 17: Accuracy of Type I Companies in Accounting [PITH_FULL_IMAGE:figures/full_fig_p048_17.png]
Figure 18
Figure 18. Figure 18: Accuracy of Type II Companies in Accounting [PITH_FULL_IMAGE:figures/full_fig_p048_18.png]
Figure 19
Figure 19. Figure 19: Accuracy of Type III Companies in Accounting [PITH_FULL_IMAGE:figures/full_fig_p048_19.png]
Figure 20
Figure 20. Figure 20: Accuracy of Type IIIV Companies in Accounting [PITH_FULL_IMAGE:figures/full_fig_p048_20.png]
Figure 21
Figure 21. Figure 21: Accuracy of Type V Companies in Accounting [PITH_FULL_IMAGE:figures/full_fig_p048_21.png]
Figure 22
Figure 22. Figure 22: Accuracy of Type I Companies in Audit 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 Tasks Claude-3.7-Sonnet DeepSeek-V3 GPT-4.1-mini GPT-4.1-nano GPT-4.1 GPT-4.1-mini.1 o3-mini Models 1.00 0.83 1.00 0.67 0.00 1.00 0.67…
Figure 23
Figure 23. Figure 23: Accuracy of Type II Companies in Audit 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 Tasks Claude-3.7-Sonnet DeepSeek-V3 GPT-4.1-mini GPT-4.1-nano GPT-4.1 GPT-4.1-mini.1 o3-mini Models 1.00 0.83 1.00 0.83 0.00 1.00 0.8…
Figure 24
Figure 24. Figure 24: Accuracy of Type III Companies in Audit 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 Tasks Claude-3.7-Sonnet DeepSeek-V3 GPT-4.1-mini GPT-4.1-nano GPT-4.1 GPT-4.1-mini.1 o3-mini Models 1.00 0.67 0.67 0.83 0.00 1.00 0.…
Figure 25
Figure 25. Figure 25: Accuracy of Type IIIV Companies in Audit [PITH_FULL_IMAGE:figures/full_fig_p049_25.png]
Figure 26
Figure 26. Figure 26: Accuracy of Type V Companies in Audit 49 [PITH_FULL_IMAGE:figures/full_fig_p049_26.png]
Figure 27
Figure 27. Figure 27: Accuracy of Type I Companies in Consulting [PITH_FULL_IMAGE:figures/full_fig_p050_27.png]
Figure 28
Figure 28. Figure 28: Accuracy of Type II Companies in Consulting [PITH_FULL_IMAGE:figures/full_fig_p050_28.png]
Figure 29
Figure 29. Figure 29: Accuracy of Type III Companies in Consulting [PITH_FULL_IMAGE:figures/full_fig_p050_29.png]
Figure 30
Figure 30. Figure 30: Accuracy of Type IIIV Companies in Consulting [PITH_FULL_IMAGE:figures/full_fig_p050_30.png]
Figure 31
Figure 31. Figure 31: Accuracy of Type V Companies in Consulting [PITH_FULL_IMAGE:figures/full_fig_p050_31.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 13 canonical work pages

  1. [2]

    Finan- cial distress prediction in an international context: A review and empirical analysis of altman’s z-score model

    Edward I Altman, Małgorzata Iwanicz-Drozdowska, Erkki K Laitinen, and Arto Suvas. Finan- cial distress prediction in an international context: A review and empirical analysis of altman’s z-score model. Journal of international financial management & accounting, 28(2):131–171, 2017

    work page 2017

  2. [3]

    BerriAI. Litellm. https://github.com/BerriAI/litellm, 2023

    work page 2023

  3. [4]

    The new business of consulting: the basics and beyond

    Elaine Biech. The new business of consulting: the basics and beyond . John Wiley & Sons, 2019

    work page 2019

  4. [5]

    The impact of consulting services on small and medium enterprises: Evidence from a randomized trial in mexico

    Miriam Bruhn, Dean Karlan, and Antoinette Schoar. The impact of consulting services on small and medium enterprises: Evidence from a randomized trial in mexico. Journal of Political Economy, 126(2):635–687, 2018

    work page 2018

  5. [6]

    Finqa: A dataset of numerical reasoning over financial data

    Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, et al. Finqa: A dataset of numerical reasoning over financial data. arXiv preprint arXiv:2109.00122, 2021

    arXiv 2021

  6. [7]

    Wiley GAAP 2010: Interpretation and application of generally accepted accounting principles

    Barry J Epstein, Ralph Nach, and Steven M Bragg. Wiley GAAP 2010: Interpretation and application of generally accepted accounting principles. John Wiley & Sons, 2009

    work page 2010

  7. [8]

    Accounting

    Jayne Godfrey, Allan Hodgson, Ann Tarca, Jane Hamilton, and Scott Holmen. Accounting. John Wiley & Sons, Inc, 2010

    work page 2010

  8. [9]

    Fintsb: A comprehensive and practical benchmark for financial time series forecasting

    Yifan Hu, Yuante Li, Peiyuan Liu, Yuxia Zhu, Naiqi Li, Tao Dai, Shu-tao Xia, Dawei Cheng, and Changjun Jiang. Fintsb: A comprehensive and practical benchmark for financial time series forecasting. arXiv preprint arXiv:2502.18834, 2025

    arXiv 2025

Show all 27 references
  1. [10]

    Financebench: A new benchmark for financial question answering

    Pranab Islam, Anand Kannappan, Douwe Kiela, Rebecca Qian, Nino Scherrer, and Bertie Vidgen. Financebench: A new benchmark for financial question answering. arXiv preprint arXiv:2311.11944, 2023

    2023 arXiv

  2. [11]

    Bizbench: A quantitative reasoning benchmark for business and finance

    Rik Koncel-Kedziorski, Michael Krumdick, Viet Lai, Varshini Reddy, Charles Lovering, and Chris Tanner. Bizbench: A quantitative reasoning benchmark for business and finance. arXiv preprint arXiv:2311.06602, 2023

    2023 arXiv

  3. [12]

    Deepseek-v3 technical report

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024

    2024 arXiv

  4. [13]

    Fingpt: Democratizing internet-scale data for financial large language models

    Xiao-Yang Liu, Guoxuan Wang, Hongyang Yang, and Daochen Zha. Fingpt: Democratizing internet-scale data for financial large language models. arXiv preprint arXiv:2307.10485, 2023

    2023 arXiv

  5. [14]

    GPT-4 technical report

    OpenAI. GPT-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  6. [15]

    Data analytics and its implication on auditing

    Debbi Chyntia Ovami and Iskandar Muda. Data analytics and its implication on auditing. In 12th International Conference on Green Technology (ICGT 2022), pages 93–101. Atlantis Press, 2023

    2022

  7. [16]

    An experimental analysis of the effect of recordkeeping over direct reciprocity

    Luis Paulo Guimarães dos Santos, Anderson José Freitas de Cerqueira, and César Valentim de Oliveira Carvalho. An experimental analysis of the effect of recordkeeping over direct reciprocity. Revista Contabilidade & Finanças, 32(86):359–375, 2020. 10

    2020

  8. [17]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  9. [18]

    Ifrs adoption and accounting quality: a review

    Naomi S Soderstrom and Kevin Jialin Sun. Ifrs adoption and accounting quality: a review. European accounting review, 16(4):675–702, 2007

    2007

  10. [19]

    The use of dupont analysis by market participants

    Mark T Soliman. The use of dupont analysis by market participants. The accounting review, 83(3):823–853, 2008

    2008

  11. [20]

    LLaMA: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. LLaMA: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  12. [21]

    Finben: A holistic financial benchmark for large language models

    Qianqian Xie, Weiguang Han, Zhengyu Chen, Ruoyu Xiang, Xiao Zhang, Yueru He, Mengxi Xiao, Dong Li, Yongfu Dai, Duanyu Feng, et al. Finben: A holistic financial benchmark for large language models. Advances in Neural Information Processing Systems, 37:95716–95743, 2024

    2024

  13. [22]

    Pixiu: A large language model, instruction data and evaluation benchmark for finance

    Qianqian Xie, Weiguang Han, Xiao Zhang, Yanzhao Lai, Min Peng, Alejandro Lopez-Lira, and Jimin Huang. Pixiu: A large language model, instruction data and evaluation benchmark for finance. arXiv preprint arXiv:2306.05443, 2023

    2023 arXiv

  14. [23]

    SECQUE: A benchmark for evaluating real-world financial analysis capabilities

    Noga Ben Yoash, Meni Brief, Oded Ovadia, Gil Shenderovitz, Moshik Mishaeli, Rachel Lemberg, and Eitam Sheetrit. SECQUE: A benchmark for evaluating real-world financial analysis capabilities. arXiv preprint arXiv:2504.04596, 2025

    2025 arXiv

  15. [24]

    Fincon: A synthesized llm multi-agent system with conceptual verbal reinforcement for enhanced financial decision making

    Yangyang Yu, Zhiyuan Yao, Haohang Li, Zhiyang Deng, Yuechen Jiang, Yupeng Cao, Zhi Chen, Jordan Suchow, Zhenyu Cui, Rong Liu, et al. Fincon: A synthesized llm multi-agent system with conceptual verbal reinforcement for enhanced financial decision making. Advances in Neural Inf...

    2024

  16. [25]

    Baichuan4-finance technical report

    Hanyu Zhang, Boyu Qiu, Yuhao Feng, Shuqi Li, Qian Ma, Xiyuan Zhang, Qiang Ju, Dong Yan, and Jian Xie. Baichuan4-finance technical report. arXiv preprint arXiv:2412.15270, 2024

    2024 arXiv

  17. [26]

    Fineval: A chinese financial domain knowledge evaluation benchmark for large language models

    Liwen Zhang, Weige Cai, Zhaowei Liu, Zhi Yang, Wei Dai, Yujie Liao, Qianru Qin, Yifei Li, Xingyu Liu, Zhiqiang Liu, et al. Fineval: A chinese financial domain knowledge evaluation benchmark for large language models. arXiv preprint arXiv:2308.09975, 2023

    2023 arXiv

  18. [27]

    Finagent: A multimodal foundation agent for financial trading: Tool-augmented, diversified, and generalist

    Wentao Zhang, Lingxuan Zhao, Haochong Xia, Shuo Sun, Jiaze Sun, Molei Qin, Xinyi Li, Yuqing Zhao, Yilei Zhao, Xinyu Cai, et al. Finagent: A multimodal foundation agent for financial trading: Tool-augmented, diversified, and generalist. arXiv e-prints, pages arXiv–2402, 2024

    2024

  19. [28]

    "" # < task_name > Task D e s c r i p t i o n : < task_description > # Examples : < i n _ c o n t e x t_ e x a m p l e s > # Problem to Solve : {

    Jie Zhu, Qian Chen, Huaixia Dou, Junhui Li, Lifan Guo, Feng Chen, and Chi Zhang. DianJin- R1: Evaluating and enhancing financial reasoning in large language models. arXiv preprint arXiv:2504.15716, 2025. 11 A Frequently Asked Questions (FAQs) A.1 Why Using Simulated Data is En...

    2025 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.