Pith. sign in

REVIEW 4 major objections 6 minor 7 cited by

Fino1: On the Transferability of Reasoning-Enhanced LLMs and Reinforcement Learning to Finance

T0 review · 4 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 14B model trained on 9,186 finance chain-of-thought traces outranks GPT-o1, GPT-4.5, and DeepSeek-R1 on a new financial-reasoning benchmark.

desk verdict The FinCoT corpus and FinReason benchmark are genuinely useful, but the 'consistently outperform' claim is not supported by the paper's own tables, and a possible FinQA train/eval overlap needs to be resolved before taking the headline results at face value. read the letter →

arxiv 2502.08127 v3 pith:RXGFDHZJ submitted 2025-02-12 cs.CL

Lingfei Qian , Weipeng Zhou , Yan Wang , Xueqing Peng , Han Yi , Yilun Zhao , Jimin Huang , Qianqian Xie
show 1 more author
Jian-yun Nie
This is my paper · ORCID
classification cs.CL
keywords financialreasoningchain-of-thoughtreinforcementlearningGRPOdomainadaptationlong-contextFinCoTReason
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that financial reasoning is not served by generic reasoning enhancement or model scale alone, and that a purpose-built chain-of-thought corpus plus the right reinforcement-learning update can make a small open model competitive with much larger closed models. The authors build FinCoT, 9,186 high-fidelity financial reasoning traces distilled from seven QA datasets through domain-guided generation, iterative verification and refinement, and difficulty filtering; then train Fin-o1-8B and Fin-o1-14B on it via supervised fine-tuning followed by GRPO. They evaluate 29 models on FinReason, a new benchmark spanning single-table, multi-table, long-context, and equation-based financial tasks, and report that Fin-o1-14B averages 61.07, ahead of GPT-o1-preview, GPT-4.5, and DeepSeek-R1 despite far fewer parameters. The paper's central message is that targeted data and optimisation, not scale, transfer reasoning into finance.

What carries the argument

The central object is FinCoT, a corpus of 9,186 chain-of-thought reasoning traces assembled by a three-stage pipeline: (1) domain guidance that fuses expert-curated sub-questions from sources like ConvFinQA and BizBench with programmatic steps into holistic questions answered by GPT-4o; (2) iterative refinement in which a GPT-4o verifier scores each trace and the generator revisits earlier steps to fix errors; and (3) difficulty-aware filtering that removes questions an 8B model can already answer, keeping only challenging cases. The training machinery is two-phase: supervised fine-tuning on FinCoT_SFT (7,686 traces) teaches the reasoning format, then reinforcement learning on 1,500 traces optimises reward. The GRPO reward is $r = \alpha_1 r_{\text{acc}} + \alpha_2 r_{\text{logic}} + \alpha_3 r_{\text{format}} + \alpha_4 r_{\text{length}} \cdot r_{\text{acc}}$, where the last term pays a bonus only when the answer is correct and the context exceeds 8,192 tokens. Evaluation runs on FinReason, a benchmark of FinQA, DM-Simplong, DM-Complong, and XBRL-Math with LLM-as-judge scoring after TheoremQA-style answer extraction.

What would settle it

Re-run DM-Complong with all models restricted to the same token budget or given retrieval, and check whether short-context baselines such as Llama-3-8B-Instruct and Llama-3-70B-Instruct recover their scores; if they do, the claimed long-context advantage of Fin-o1 over larger models is an artifact of asymmetric context lengths. A second check is to remove the length reward term from GRPO and see whether the DM-Complong gain from 23.33 (SFT) to 26.33 (SFT+GRPO) disappears, which would confirm that the reward term, not the data alone, drives long-context improvement.

Watch

Extended reading notes

Core claim

The paper's central claim is that domain-specific chain-of-thought supervision is the missing ingredient for financial reasoning in large language models. Concretely, it claims that a three-stage data pipeline—expert and programmatic sub-question guidance, iterative GPT-4o generation with verifier feedback, and difficulty-aware filtering that removes questions a small model can already solve—produces a corpus whose reasoning traces teach financial logic that generic CoT and scale cannot. Starting from Qwen3 backbones, supervised fine-tuning on 7,686 of these traces followed by GRPO with a reward combining accuracy, logical consistency, format, and a long-context-conditioned length bonus yields Fin-o1-8B and Fin-o1-14B that consistently outperform their own backbones and larger GPT-o1 and DeepSeek-R1 models, ranking second overall behind DeepSeek-V3 on FinReason. The paper also claims the first systematic comparison of PPO, DPO, and GRPO for financial reasoning, finding GRPO to be the only method with stable gains, and reports that reasoning-enhanced general models degrade in financial contexts, sometimes below their non-reasoning backbones.

Load-bearing premise

The long-document comparisons assume every evaluated model was given the same usable input, yet several baselines have 8k context windows while DM-Complong documents average roughly 40,000 tokens, so their low scores may come from truncated inputs rather than weaker reasoning.

Editorial extensions

If this is right

  • A 14B open model can reach 61.07 on FinReason—ahead of GPT-o1-preview (54.05), GPT-4.5 (60.43), and DeepSeek-R1 (60.87)—so a small domain-adapted model can outperform much larger general reasoners on financial QA.
  • GRPO with the multi-part reward lifts Fin-o1-8B from 52.28 (backbone) to 59.95, while PPO and DPO give smaller or negative gains, indicating that the choice of RL objective, not RL in general, is what transfers to finance.
  • Difficulty filtering carries real weight: SFT on filtered CoT averages 58.58 versus 56.28 on unfiltered CoT, so including easy examples dilutes the reasoning gain.
  • Finance-tuned baselines FinR1-7B and Dianjin-R1-7B fall below their Qwen2.5-7B backbone on DM-Simplong and DM-Complong, implying that CoT data without long-context reasoning practice can actively hurt robustness on long documents.
  • The FinReason protocol—four datasets covering short table QA, long single-table, long multi-table, and equation-based XBRL tasks—gives a repeatable way to separate domain understanding from general mathematical ability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The DM-Complong comparison may overstate long-context ability: with an average of 39,983 tokens per document and several baselines limited to 8k context windows, the zero-shot setup likely truncates those baselines, so part of their low scores may be input loss rather than reasoning failure; rerunning with retrieval or a fixed token budget would separate the two.
  • The error analysis suggests the remaining gap between Fin-o1 and DeepSeek-V3 is domain semantics rather than arithmetic: RL-tuned models over-reason and miss accounting conventions, so adding human-annotated financial reasoning paths or expert verifiers could be a testable extension of FinCoT's method.
  • The three-stage corpus recipe—sub-question guidance, iterative refinement, and difficulty filtering—is transferable to other expert-regulated domains such as medicine and law, where decomposable expert questions exist; the paper's contribution would then be the pipeline plus reward design, not the specific financial data.
  • The paper ablates data quality but not corpus size, so a FinCoT scale curve would clarify whether 9,186 traces are near the ceiling or whether more traces would close the remaining gap to DeepSeek-V3.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces three artifacts: FinCoT, a 9,186-example financial chain-of-thought corpus distilled from seven QA sources; Fin-o1-8B/14B, models obtained by SFT on FinCoT followed by PPO, DPO, or GRPO; and FinReason, a four-dataset benchmark (FinQA, DM-Simplong, DM-Complong, XBRL-Math) used to evaluate 29 models. The main claimed findings are that finance-tuned reasoning models outperform general and larger reasoning models such as GPT-o1 and DeepSeek-R1 on financial reasoning, that GRPO gives reliable gains while PPO and DPO do not, and that general reasoning models degrade on financial tasks. The manuscript includes ablation studies, an RL-method comparison, error analyses, and open-sourced models/data/code.

Significance. If the claims held, the paper would be a useful contribution: it provides a public financial CoT corpus, a systematic side-by-side comparison of PPO/DPO/GRPO for a domain-specific reasoning task, and a new benchmark with long-context and multi-table components. The strongest defensible result is the within-backbone improvement in Table 4, where Qwen3-8B improves from 52.28 to 59.95 under SFT+GRPO on the authors' evaluation, suggesting the FinCoT data and GRPO reward are not inert. The release of models, data, and code is a concrete strength. However, the headline comparative claims against GPT-o1 and DeepSeek-R1 are not supported by the paper's own numbers, and two confounds — possible train/evaluation overlap on FinQA and context-window truncation on DM-Complong — bear directly on the central transferability conclusion.

major comments (4)
  1. [Section 2.1 vs. Section 4.1; Limitations] FinCoT is constructed from FinQA [5] (Section 2.1), while FinReason evaluates on FinQA [4] with 1,100 examples (Section 4.1), and no sentence states that FinQA evaluation instances were excluded from FinCoT. The appended Limitations section even says 'our fine-tuning relies on a single dataset, FinQA, for reasoning path construction.' As written, the FinQA column of Table 3 and the averages that include it may reflect training-set memorization rather than transferable financial reasoning. The authors must either document an explicit split excluding all FinQA evaluation instances from FinCoT or re-run the affected comparisons after removing overlap.
  2. [Abstract and Table 3] The abstract's claim that 'our Fin-o1 models consistently outperform their backbones and larger GPT-o1 and DeepSeek-R1' is internally inconsistent with Table 3: Fin-o1-8B averages 59.95, below DeepSeek-R1's 60.87, and Fin-o1-8B scores 26.33 on DM-Complong versus DeepSeek-R1's 38.67; Fin-o1-14B also scores lower than DeepSeek-R1 on DM-Complong (27.33 vs. 38.67). The claim should be restricted to specific datasets and to Fin-o1-14B where the average comparison against DeepSeek-R1 holds, or the abstract should be revised.
  3. [Section D.2 and Table 5] DM-Complong has an average context length of 39,983 tokens (Table 2), yet the evaluation is zero-shot with no truncation or retrieval strategy (Section D.2), and Table 5 lists several baselines with 8k context windows, including Llama-3-8B-Instruct and Llama-3-70B-Instruct. Their very low DM-Complong scores — e.g., Llama-4-Scout at 0.67 and Qwen2.5-72B at 14.67 — are therefore likely artifacts of input truncation, not evidence about financial reasoning ability. The paper's conclusion that finance-tuned and general reasoning models 'degrade on lengthy documents' is confounded unless all models receive the same effective context or a truncation/retrieval strategy is explicitly controlled.
  4. [Section 3.2 Eq. (3) and Section 4.3] The same GPT-4o-based verifier is used to compute RL rewards during training (Eq. (3) and the reward description in Section 3.2) and to score final answers in evaluation (Section 4.3). Because Fin-o1 is explicitly optimized to satisfy GPT-4o's judgment, an evaluation using GPT-4o as judge can favor Fin-o1 over models that were not trained against that judge, independent of genuine financial reasoning quality. The paper should either use a judge independent of the reward model, report agreement with human evaluation, or present a rule-based exact-match analysis for the numeric answers.
minor comments (6)
  1. [Table 4] The row 'Fin-o1-8B (SFT with filtered CoT)' reads '71.7857.00', which appears to be a formatting error combining two numbers; please separate the FinQA and DM-Simplong entries.
  2. [Section 5, paragraph 2] The main text says Fin-o1-8B attains an average score of 58.58, but Table 3 reports Fin-o1-8B's average as 59.95; the text should refer to the correct table value or clarify that 58.58 refers to the SFT-only variant in Table 4.
  3. [Section 3.2, Eq. (3)] The weights α1, α2, α3, and α4 in the GRPO reward are never given numerical values; specifying them is necessary for reproducibility of the RL training.
  4. [Appendix D.1] The sentence describing gradient accumulation is duplicated ('The gradient accumulation is set to 2 steps, and a gradient accumulation of 2 steps'), and the effective batch size is not clearly specified.
  5. [References and naming] The paper cites [30] as 'Generalized preference optimization' when GRPO in the DeepSeek-R1 context is usually 'Group Relative Policy Optimization' (Shao et al., 2024); if the authors intentionally use Tang et al.'s method, they should clarify this choice. Also, 'Fin-o1-14' and 'Fin-o1-14B' are used inconsistently in the abstract and introduction.
  6. [Limitations] The Limitations section says evaluation covers 'only three financial reasoning tasks (FinQA, DM-Simplong, and XBRL-Math)' but the benchmark has four datasets including DM-Complong; this should be corrected for consistency.

Circularity Check

2 steps flagged · score 5.0 of 10

FinCoT is built from FinQA and FinReason evaluates FinQA with no stated split, while GPT-4o is used both as the RL reward verifier and as the evaluation judge; the claimed generalization is therefore partially circular.

  1. fitted input called prediction [Section 2.1 (FinCoT data collection), Section 4.1 (FinReason tasks), appended Limitations]
    "We construct FinCoT from diverse sources including FinQA [5], ConvFinQA [6], DocFinQA [27], TATQA [47], Econ_Logic [25], DocMath-Eval [45], and BizBench-QA [14]. ... (1) FinQA [4] serves as the foundational benchmark for quantitative reasoning over structured tables and basic financial texts. ... our fine-tuning relies on a single dataset, FinQA, for reasoning path construction."

    FinCoT is the supervised training signal for Fin-o1, and FinQA is also one of the four FinReason evaluation sets. The paper never states that the 1,100 FinQA evaluation instances were excluded from the FinCoT construction, and the Limitations passage says fine-tuning relies on FinQA for reasoning-path construction. Under that setup, the FinQA column of Table 3 is not an out-of-distribution transfer result: the SFT and GRPO gains on FinQA could be memorization of the training task. The abstract's conclusion that the results 'confirm the effectiveness of our data building and model training strategy' is therefore, for FinQA, a restatement of fitting to the same source rather than an independent prediction.

  2. self definitional [Section 2.2 (difficulty-aware filtering), Section 3.2 (Eq. 3 and GRPO reward), Section 4.3 (evaluation settings)]
    "We use Llama3.1-8B-Instruct to attempt each question, then apply a verifier (see Section 4.3) to check the answers. ... To effectively evaluate the generated response, we also leverage LLMs (GPT-4o) as judge. ... we adopt an LLM-as-judge evaluation paradigm."

    The same GPT-4o verifier is referenced for filtering FinCoT, for computing the GRPO accuracy reward in Eq. (3), and for the FinReason evaluation pipeline. Training therefore optimizes against GPT-4o's judgment, and the benchmark then measures the same GPT-4o-based extraction/judgment. This makes at least part of the reported improvement a measure of how well Fin-o1 satisfies the judge, not an independent measure of financial reasoning. The circularity is partial because the final extracted answers are compared with ground truth, but the shared verifier/judge makes the validation loop non-independent.

full rationale

The paper is largely an empirical study with external baselines, and most benchmark numbers are independent facts; there is no load-bearing self-citation or imported uniqueness theorem. The main circularity is data overlap: FinCoT is constructed from FinQA and FinReason evaluates FinQA, with no stated train/eval split, and the Limitations section explicitly says fine-tuning relies on FinQA for reasoning-path construction. A secondary circularity is the shared GPT-4o verifier used for CoT filtering, RL rewards, and evaluation. Separately, the abstract's 'consistently outperform ... DeepSeek-R1' claim is not supported by Table 3 for Fin-o1-8B (59.95 vs 60.87), but that is a correctness inconsistency rather than a circularity. Overall, the central generalization claim is at least partially circular, warranting a score near the middle of the scale.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper's central claims rest on trust in GPT-4o as a judge, on the completeness of the seven selected datasets, and on fair zero-shot evaluation across models with different context windows. The GRPO conclusion additionally depends on unreported reward weights. No new physical or conceptual entities are postulated.

free parameters (3)
  • GRPO reward weights alpha1, alpha2, alpha3, alpha4 = not reported
    Eq. (3) defines the GRPO reward as a weighted sum of accuracy, logic, format, and length components, but the paper never states the values of the four weights. The GRPO-versus-PPO/DPO conclusion depends on these hand-chosen weights.
  • Length reward context threshold = 8192 tokens
    Section 3.2 grants a length reward only when the context exceeds 8,192 tokens. This threshold is chosen by hand and favors long-document tasks in the GRPO objective.
  • Reward component point values = accuracy=1, logic=1, format=0.1, length=1
    Section 3.2 assigns fixed reward points to each GRPO component. These are arbitrary constants that shape the learned policy.
assumptions (4)
  • ad hoc to paper GPT-4o is a reliable judge of answer correctness and reasoning logic for financial tasks.
    GPT-4o is used for reward computation in Eq. (1) and Eq. (3), for CoT refinement in Section 2.2, for difficulty filtering, and for final benchmark scoring in Section 4.3. No agreement with human labels is reported.
  • domain assumption The seven source QA datasets define the full space of financial reasoning.
    Section 2.1 selects FinQA, ConvFinQA, TATQA, DocMath-Eval, EconLogic, BizBench, and DocFinQA as the components of FinCoT. No coverage analysis is provided.
  • ad hoc to paper Zero-shot prompting of a document longer than a model's context window is a fair evaluation.
    Section 4.3 applies the same zero-shot prompt to DM-Complong documents of roughly 40,000 tokens while several baselines have 8k context windows. Without truncation or retrieval handling, the comparison is unfair for those models.
  • ad hoc to paper A question solvable by Llama3.1-8B-Instruct is easy and can be safely removed from training.
    Section 2.2 uses this as the difficulty-aware filter. The validity of this proxy is not assessed in the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fino1: On the Transferability of Reasoning-Enhanced LLMs and Reinforcement Learning to Finance." pith.science (2026). https://pith.science/paper/RXGFDHZJ

@misc{pith2026250208127,
  author       = {Pith},
  title        = {Pith review of: Fino1: On the Transferability of Reasoning-Enhanced LLMs and Reinforcement Learning to Finance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RXGFDHZJ}},
  note         = {Machine review of arXiv:2502.08127}
}
read the original abstract

As the fundamental capability behind decision-making in finance, financial reasoning poses distinct challenges for LLMs. Although reinforcement learning (RL) have boosted generic reasoning, the progress in finance is hindered by the absence of empirical study of building effective financial chain-of-thought (CoT) corpus, a systematic comparison of different RL methods, and comprehensive benchmarks. To address these gaps, we introduce FinCoT, the first open high-fidelity CoT corpus for finance, distilled from seven QA datasets by a novel three-stage pipeline that incorporates domain supervision, iterative LLM refinement, and difficulty-aware filtering. Based on FinCoT, we develop Fin-o1, the first open financial reasoning models trained via supervised fine-tuning and GRPO-based RL. Our models outperform existing financial reasoning models and SOTA general models such as GPT-o1, DeepSeek-R1, and GPT-4.5. We also investigate the effectiveness of three different RL methods in improving domain-specific reasoning, offering the first such empirical study. We finally propose FinReason, the first financial reasoning benchmark covering multi-table analysis, long-context reasoning, and equation-based tasks, and evaluate 29 LLMs. Our extensive experiments reveal general reasoning models excel on standard benchmarks yet exhibit obvious performance degradation in financial contexts; even finance-tuned models like Dianjin-R1 and FinR1 degrade on lengthy documents. In contrast, our Fin-o1 models consistently outperform their backbones and larger GPT-o1 and DeepSeek-R1, confirming the effectiveness of our data building and model training strategy. Our study further shows that GRPO yields reliable gains whereas PPO and DPO do not, highlighting the need for targeted data and optimisation rather than scale alone.

Figures

Figures reproduced from arXiv: 2502.08127 by the authors.

Figure 1
Figure 1. Overall framework of development of FinCoT, Fin-o1 and FinReason. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Workflow of curating the combined question and examples of program progress. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Example of iterative refinement of the CoT process with a guidance. [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Error case 1. an error analysis using the FinQA dataset since these reasoning-enhanced models exhibit a significant performance drop on FinQA. We focus on cases where DeepSeek-R1 produces incorrect answers while DeepSeek-V3 generates correct ones. Two common error patt…
Figure 5
Figure 5. Figure 5: Error case 2. allowing it to provide the accurate answer. This highlights a fundamental gap in DeepSeek-R1’s financial reasoning capabilities, particularly in recognizing the accounting conventions used to report share-based compensation. G Prompt Prompt Template for D…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Fin-RATE: A Real-world Financial Analytics and Tracking Evaluation Benchmark for LLMs on SEC Filings

    cs.CE 2026-02 conditional novelty 6.0 of 10

    A 7,500-question SEC-filing benchmark with a 13-type error taxonomy shows LLM accuracy falls by 14–19 points when reasoning moves from single documents to cross-company or multi-year tasks.

  2. VerIF: Verification Engineering for Reinforcement Learning in Instruction Following

    cs.CL 2025-06 conditional novelty 6.0 of 10

    A hybrid verifier that combines rule-based code checks and a reasoning-LLM judge enables reinforcement learning to improve LLM instruction following on several benchmarks.

  3. Reasoning Like an Economist: Post-Training on Economic Problems Induces Strategic Generalization in LLMs

    cs.AI 2025-05 conditional novelty 6.0 of 10

    Post-training an LLM on economic reasoning problems improves accuracy on economic benchmarks and, without game-specific training, raises its Nash equilibrium frequency and win rates in strategic games.

  4. DataPrep-Bench: Benchmarking LLMs as Training Data Preparators

    cs.LG 2026-05 conditional novelty 5.0 of 10

    DataPrep-Bench jointly benchmarks data construction and data-quality evaluation for LLMs across six domains with downstream fine-tuning performance as ground truth.

  5. Agentar-Fin-R1: Enhancing Financial Intelligence through Domain Expertise, Training Efficiency, and Advanced Reasoning

    cs.CL 2025-07 conditional novelty 5.0 of 10

    Agentar-Fin-R1, an 8B and 32B financial LLM family, reports top scores on FinEval, FinanceIQ, and a new Finova benchmark while keeping general reasoning near its Qwen3 base.

  6. RoT: Enhancing Table Reasoning with Iterative Row-Wise Traversals

    cs.CL 2025-05 conditional novelty 5.0 of 10

    A training-free row-wise traversal prompt improves table question answering accuracy over long-CoT reasoning models on WikiTableQuestions and TableBench.

  7. Assessing the Capabilities and Limitations of FinGPT Model in Financial NLP Applications

    cs.CL 2025-07 reject novelty 3.0 of 10

    FinGPT matches GPT-4 on financial sentiment and headline classification, lags on QA and NER, and shows a bullish bias in stock movement prediction.

Reference graph

Works this paper leans on

49 extracted references · 14 canonical work pages · cited by 7 Pith papers

  1. [5]

    Finqa: A dataset of numerical reasoning over financial data, 2022

    Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, and William Yang Wang. Finqa: A dataset of numerical reasoning over financial data, 2022

    2022

  2. [4]

    Finqa: A dataset of numerical reasoning over financial data

    Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, et al. Finqa: A dataset of numerical reasoning over financial data. arXiv preprint arXiv:2109.00122, 2021

    arXiv 2021

  3. [1]

    Huatuogpt-o1, towards medical complex reasoning with llms

    Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang, Wanlong Liu, Rongsheng Wang, Jianye Hou, and Benyou Wang. Huatuogpt-o1, towards medical complex reasoning with llms. arXiv preprint arXiv:2412.18925, 2024

    arXiv 2024

  4. [2]

    What do neural networks learn in image classification? A frequency shortcut perspective

    Mark Chen et al. Automatic formal reasoning about large language models. arXiv preprint arXiv:2307.09829, 2023

    work page Pith review arXiv 2023

  5. [3]

    Theoremqa: A theorem-driven question answering dataset

    Wenhu Chen, Ming Yin, Max Ku, Pan Lu, Yixin Wan, Xueguang Ma, Jianyu Xu, Xinyi Wang, and Tony Xia. Theoremqa: A theorem-driven question answering dataset. arXiv preprint arXiv:2305.12524, 2023

    arXiv 2023

  6. [6]

    Convfinqa: Exploring the chain of numerical reasoning in conversational finance question answering

    Zhiyu Chen, Shiyang Li, Charese Smiley, Zhiqiang Ma, Sameena Shah, and William Yang Wang. Convfinqa: Exploring the chain of numerical reasoning in conversational finance question answering. arXiv preprint arXiv:2210.03849, 2022

    arXiv

  7. [7]

    Properties of moments of density for nonlocal mean field game equations with a quadratic cost function

    Iddo Drori et al. Neural-symbolic mathematical reasoning. arXiv preprint arXiv:2301.07076, 2023

    work page Pith review arXiv 2023

  8. [8]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    arXiv 2024

Show all 49 references
  1. [9]

    Joseph G. Flowers. Finance instruct 500k. https://huggingface.co/datasets/ Josephgflowers/Finance-Instruct-500k , 2025. Accessed: 2025-03-18

    2025

  2. [10]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  3. [11]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  4. [12]

    Openai o1 system card

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720, 2024

    2024 arXiv

  5. [13]

    Automate strategy finding with llm in quant investment

    Zhizhuo Kou, Holam Yu, Jingshu Peng, and Lei Chen. Automate strategy finding with llm in quant investment. arXiv preprint arXiv:2409.06289, 2024

    2024

  6. [14]

    Bizbench: A quantitative reasoning benchmark for business and finance

    Michael Krumdick, Rik Koncel-Kedziorski, Viet Dac Lai, Varshini Reddy, Charles Lovering, and Chris Tanner. Bizbench: A quantitative reasoning benchmark for business and finance. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume ...

    2024

  7. [15]

    Efficient memory management for large language model serving with pagedattention

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Principles, p...

    2023

  8. [16]

    Solving quantitative reasoning problems with language models

    Aitor Lewkowycz et al. Solving quantitative reasoning problems with language models. arXiv preprint arXiv:2206.14858, 2022. 11

    2022 arXiv

  9. [17]

    Deepseek-v3 technical report

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024

    2024 arXiv

  10. [18]

    Findabench: Benchmarking financial data analysis ability of large language models

    Shu Liu, Shangqing Zhao, Chenghao Jia, Xinlin Zhuang, Zhaoguang Long, Jie Zhou, Aimin Zhou, Man Lan, and Yang Chong. Findabench: Benchmarking financial data analysis ability of large language models. In Proceedings of the 31st International Conference on Computational Linguist...

    2025

  11. [19]

    Fin-r1: A large language model for financial reasoning through reinforcement learning

    Zhaowei Liu, Xin Guo, Fangqi Lou, Lingfeng Zeng, Jinyi Niu, Zixuan Wang, Jiajie Xu, Weige Cai, Ziwei Yang, Xueqian Zhao, et al. Fin-r1: A large language model for financial reasoning through reinforcement learning. arXiv preprint arXiv:2503.16252, 2025

    2025

  12. [20]

    Reft: Reasoning with reinforced fine-tuning

    Trung Quoc Luong, Xinbo Zhang, Zhanming Jie, Peng Sun, Xiaoran Jin, and Hang Li. Reft: Reasoning with reinforced fine-tuning. arXiv preprint arXiv:2401.08967, 3, 2024

    2024 arXiv

  13. [21]

    Xbrl-math: A dataset for mathematical reasoning in financial reports

    Wei Ma et al. Xbrl-math: A dataset for mathematical reasoning in financial reports. Conference on Financial Technology and Natural Language Processing, 2023

    2023

  14. [22]

    s1: Simple test-time scaling

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393, 2025

    2025 arXiv

  15. [23]

    Training language models to follow instructions with human feedback

    Long Ouyang et al. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155, 2022

    2022 arXiv

  16. [24]

    Plu- tus: Benchmarking large language models in low-resource greek finance

    Xueqing Peng, Triantafillos Papadopoulos, Efstathia Soufleri, Polydoros Giannouris, Ruoyu Xiang, Yan Wang, Lingfei Qian, Jimin Huang, Qianqian Xie, and Sophia Ananiadou. Plu- tus: Benchmarking large language models in low-resource greek finance. arXiv preprint arXiv:2502.18772, 2025

    2025 arXiv

  17. [25]

    Econlogicqa: A question-answering benchmark for evaluating large language models in economic sequential reasoning

    Yinzhu Quan and Zefang Liu. Econlogicqa: A question-answering benchmark for evaluating large language models in economic sequential reasoning. arXiv preprint arXiv:2405.07938, 2024

    2024 arXiv

  18. [26]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728–53741, 2023

    2023

  19. [27]

    Docfinqa: A long-context financial reasoning dataset

    Varshini Reddy, Rik Koncel-Kedziorski, Viet Dac Lai, Michael Krumdick, Charles Lovering, and Chris Tanner. Docfinqa: A long-context financial reasoning dataset. arXiv preprint arXiv:2401.06915, 2024

    2024

  20. [28]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  21. [29]

    Evaluating llms’ mathematical reasoning in financial document question answering

    Pragya Srivastava, Manuj Malik, Vivek Gupta, Tanuja Ganu, and Dan Roth. Evaluating llms’ mathematical reasoning in financial document question answering. In Findings of the Association for Computational Linguistics ACL 2024, pages 3853–3878, 2024

    2024

  22. [30]

    Generalized preference optimization: A unified approach to offline alignment

    Yunhao Tang, Zhaohan Daniel Guo, Zeyu Zheng, Daniele Calandriello, Rémi Munos, Mark Rowland, Pierre Harvey Richemond, Michal Valko, Bernardo Ávila Pires, and Bilal Piot. Generalized preference optimization: A unified approach to offline alignment. arXiv preprint arXiv:2402.05749, 2024

    2024 arXiv

  23. [31]

    Fincorpus

    Duxiaoman DI Team. Fincorpus. https://huggingface.co/datasets/Duxiaoman-DI/ FinCorpus, 2023. Accessed: 2024-03-18

    2023

  24. [32]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530, 2024

    2024 arXiv

  25. [33]

    Openai o1-preview vs

    Mohamad-Hani Temsah, Amr Jamal, Khalid Alhasan, Abdulkarim A Temsah, and Khalid H Malki. Openai o1-preview vs. chatgpt in healthcare: a new frontier in medical ai reasoning. Cureus, 16(10), 2024. 12

    2024

  26. [34]

    Finnlp-fnp-llmfinlegal-2025 shared task: Regulations challenge

    Keyi Wang, Jaisal Patel, Charlie Shen, Daniel Kim, Andy Zhu, Alex Lin, Luca Borella, Cailean Osborne, Matt White, Steve Yang, et al. Finnlp-fnp-llmfinlegal-2025 shared task: Regulations challenge. In Proceedings of the Joint Workshop of the 9th Financial Technology and Natural...

    2025

  27. [35]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  28. [36]

    Bloomberggpt: A large language model for finance

    Shijie Wu et al. Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303.17564, 2023

    2023 arXiv

  29. [37]

    The finben: An holistic financial benchmark for large language models

    Qianqian Xie, Weiguang Han, Zhengyu Chen, Ruoyu Xiang, Xiao Zhang, Yueru He, Mengxi Xiao, Dong Li, Yongfu Dai, Duanyu Feng, et al. The finben: An holistic financial benchmark for large language models. arXiv preprint arXiv:2402.12659, 2024

    2024 arXiv

  30. [38]

    Pixiu: a large language model, instruction data and evaluation benchmark for finance

    Qianqian Xie, Weiguang Han, Xiao Zhang, Yanzhao Lai, Min Peng, Alejandro Lopez-Lira, and Jimin Huang. Pixiu: a large language model, instruction data and evaluation benchmark for finance. In Proceedings of the 37th International Conference on Neural Information Processing Syst...

    2023

  31. [39]

    Open-finllms: Open multimodal large language models for financial applications

    Qianqian Xie, Dong Li, Mengxi Xiao, Zihao Jiang, Ruoyu Xiang, Xiao Zhang, Zhengyu Chen, Yueru He, Weiguang Han, Yuzhe Yang, et al. Open-finllms: Open multimodal large language models for financial applications. arXiv preprint arXiv:2408.11878, 2024

    2024 arXiv

  32. [40]

    An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, et al. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.12122, 2024

    2024 arXiv

  33. [41]

    Fingpt: Open-source financial large language models

    Hongyang Yang, Xiao-Yang Liu, and Christina Dan Wang. Fingpt: Open-source financial large language models. arXiv preprint arXiv:2306.06031, 2023

    2023

  34. [42]

    Finbert: A pre-trained financial language representation model for financial text mining

    Yi Yang et al. Finbert: A pre-trained financial language representation model for financial text mining. International Joint Conference on Artificial Intelligence, 2020

    2020

  35. [43]

    Limo: Less is more for reasoning

    Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. Limo: Less is more for reasoning. arXiv preprint arXiv:2502.03387, 2025

    2025 arXiv

  36. [44]

    Automatic chain of thought prompting in large language models

    Zhuosheng Zhang et al. Automatic chain of thought prompting in large language models. arXiv preprint arXiv:2302.12822, 2023

    2023 arXiv

  37. [45]

    Docmath-eval: Evaluating math reasoning capabilities of llms in understanding long and specialized documents

    Yilun Zhao, Yitao Long, Hongjun Liu, Ryo Kamoi, Linyong Nan, Lyuhao Chen, Yixin Liu, Xiangru Tang, Rui Zhang, and Arman Cohan. Docmath-eval: Evaluating math reasoning capabilities of llms in understanding long and specialized documents. In Proceedings of the 62nd Annual Meetin...

    2024

  38. [46]

    Evaluation of openai o1: Opportunities and challenges of agi

    Tianyang Zhong, Zhengliang Liu, Yi Pan, Yutong Zhang, Yifan Zhou, Shizhe Liang, Zihao Wu, Yanjun Lyu, Peng Shu, Xiaowei Yu, et al. Evaluation of openai o1: Opportunities and challenges of agi. arXiv preprint arXiv:2409.18486, 2024

    2024

  39. [47]

    Tat-qa: A question answering benchmark on a hybrid of tabular and textual content in finance, 2021

    Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, and Tat-Seng Chua. Tat-qa: A question answering benchmark on a hybrid of tabular and textual content in finance, 2021

    2021

  40. [48]

    Dianjin- r1: Evaluating and enhancing financial reasoning in large language models

    Jie Zhu, Qian Chen, Huaixia Dou, Junhui Li, Lifan Guo, Feng Chen, and Chi Zhang. Dianjin- r1: Evaluating and enhancing financial reasoning in large language models. arXiv preprint arXiv:2504.15716, 2025. 13 A Related Work Our work builds upon three main research directions: th...

    2025 arXiv

  41. [49]

    average share price

    P r o p e r t i e s and Equipment , N e t P r o p e r t i e s and equipment , net are c o m p r i s e d of the f o l l o w i n g : | | | | | | | | | | | | | | | | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | | | | | | | | | | | ...

    2022

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.