Pith. sign in

REVIEW 5 major objections 5 minor 25 references

Evaluating AI for Finance: Is AI Credible at Assessing Investment Risk?

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read No AI system consistently produces unbiased, accurate investment-risk scores across all demographic groups and countries.

desk verdict A timely question undermined by a rubric mismatch and unmatched profiles; the benchmark doesn't support the claim that no LLM is credible. read the letter →

arxiv 2505.18953 v2 pith:G4RM7W4N submitted 2025-05-25 cs.CL

Divij Chawla , Ashita Bhutada , Do Duc Anh , Abhinav Raghunathan , Vinod SP , Cathy Guo , Dar Win Liew , Prannaya Gupta
show 3 more authors
Rishabh Bhardwaj Rajat Bhardwaj Soujanya Poria
This is my paper · ORCID
classification cs.CL
keywords LLMevaluationinvestmentriskappetitedemographicbiasfairnessfinancialregulationtolerancescoringconsistency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether current large language models can credibly score how much investment risk a person should take on, and its answer is no. On a 1,720-profile benchmark whose ground-truth scores come from an additive five-part rubric, all eight tested models shift their scores when a user's country or gender changes, even though those attributes are supposed to be irrelevant. The paper defines credibility as correctness (closeness to the rubric's ideal score) plus consistency (stability across demographic groups), and it reports that no model achieves both. GPT-4o tracks low- and mid-risk targets closely but scores Nigerian and Indonesian profiles higher; open-weight models show irregular gender gaps. If the results hold, current LLMs are not ready for automated investment risk profiling without demographic fairness controls.

What carries the argument

The load-bearing object is FINRISK EVAL, a benchmark of 1,720 user profiles whose ground-truth risk score is computed by an additive rubric: RT = PFS + ISO + LAA + MCR + DOI, one term for each of five dimensions: personal and financial stability, investment strategy and objectives, liquidity and asset allocation, market and currency risks, and dependency on investments. Each profile pairs fixed financial features with a country and a binary gender, so any difference in model output between otherwise matching profiles is attributable to a demographic attribute the rubric itself ignores. This design turns demographic bias into a measurable quantity: the cross-country standard deviation of the model's mean score and the gender gap in each scenario.

What would settle it

Recompute the ground-truth scores for all 1,720 profiles using the rubric actually printed in Appendix A.6 rather than the one in Section 4; if the per-model deviation and consistency tables change materially, the reported correctness and bias findings depend on the scoring mismatch. A second decisive check is a token-swap experiment: take one profile and change only the country and name tokens while leaving the other sixteen features untouched; the paper's claim predicts systematic score shifts across every model, and their absence would refute it.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that none of the evaluated AI systems—GPT-4o, GPT-4o mini, Gemini 1.5 Pro, Claude 3.7 Sonnet, LLaMA 3.1 (405B), LLaMA 3.3 (70B), DeepSeek-V3, and Mistral small—consistently produces unbiased, accurate risk-tolerance scores across all demographic groups and countries. Each model's score is compared against the rubric-based ideal for conservative (low), moderate (mid), and aggressive (high) profiles; the deviations and cross-country standard deviations are the evidence. The authors find certain models align well in specific ranges, but no model keeps its output stable when country or gender changes, and the authors read this instability as a failure of financial fairness and regulatory compliance. The concrete examples include GPT-4o assigning higher risk scores to Nigerian and Indonesian profiles and open-weight models exhibiting inconsistent gender-based scoring.

Load-bearing premise

The evaluation assumes that the scoring rubric used to compute ground-truth risk scores is the same rubric the models are given in the prompt, but the paper contradicts this for the monthly-investment feature: the scoring section awards +2 for investing under 10% of income and −2 for over 30%, while the prompt awards +1 for over 20% and −1 for under 20%.

Editorial extensions

If this is right

  • Regulators should treat demographic consistency as a precondition for deploying LLMs in suitability and risk-profiling contexts.
  • Model choice should be risk-band specific, since alignment on low and mid profiles, as with GPT-4o, does not carry over to aggressive profiles.
  • Fine-tuning and calibration targets should include demographic invariance rather than only average correctness.
  • Deploying these models without such controls creates legal exposure under frameworks like the EU AI Act, GDPR, the Equal Credit Opportunity Act, and MAS Fairness, Ethics, Accountability and Transparency principles.
  • A standardized public benchmark built on this profile structure could let firms audit future models the same way.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the rubric in Section 4 and the prompt in Appendix A.6 disagree on at least the monthly-investment rule, so the reported deviation numbers may grade models against a standard they were not asked to follow; rerunning with a single consistent rubric would separate true miscalibration from prompt artifacts.
  • Without a human-advisor baseline, the same benchmark could also be read as evidence that risk-appetite scoring is inherently noisy; comparing human advisors on identical profiles would clarify whether the observed spread is an AI-specific failure.
  • The profile design swaps names and countries embedded in the interview text, so the test measures sensitivity to the whole demographic token set; a fully controlled experiment would vary just one token at a time to estimate causal effects of each attribute.
  • The benchmark could be extended to non-binary gender identities, more countries, and other protected attributes, and could be used to test whether fairness fine-tuning actually shrinks the reported inconsistencies.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces FINRISK EVAL, a benchmark of 1,720 synthetic user profiles spanning 10 countries and two genders, each described by 16 financial features plus demographic attributes. A hand-built scoring rubric (Section 3 and 4) assigns each feature a numeric value and sums them into a Risk Tolerance (RT) score, with profiles categorized as Conservative, Moderate, or Aggressive. Eight LLMs (GPT-4o, GPT-4o mini, Gemini 1.5 Pro, Claude 3.7 Sonnet, DeepSeek-V3, LLaMA 3.1 405B, LLaMA 3.3 70B, and Mistral Small) are prompted to output an RT score for each profile. The paper then analyzes correctness (deviation from scenario-level ideal scores) and consistency (variation in predicted scores across countries and genders). It reports that while some models align with ideal scores in low- and mid-risk scenarios, none is both accurate and consistent across all demographic groups, and it concludes that current LLMs are not ready for automated risk profiling and that their inconsistencies violate AI and finance regulations.

Significance. If the evaluation were valid, the paper would provide a valuable public benchmark for auditing LLMs in a high-stakes financial task. The dataset construction (1,720 profiles, 10 countries, balanced gender) and the explicit separation of demographic from financial features are useful ideas, and the authors test a diverse set of proprietary and open-weight models. The paper also makes a concrete, falsifiable prediction: no current LLM produces risk scores that are simultaneously accurate and stable across demographics. However, the evaluation has several load-bearing methodological flaws that invalidate the reported numbers: the prompt rubric differs from the ground-truth rubric on a key feature, the correctness metric compares against arbitrary scenario-level ideals rather than per-profile ground truths, and the consistency analysis does not control for potentially different financial features across countries and genders. The circularity between the ground-truth rubric and the prompt further weakens the claim that the study assesses models' independent risk-assessment ability. As a result, the paper's central empirical claims are not supported.

major comments (5)
  1. [Section 4 vs. Appendix A.6] The ground-truth scoring rubric and the prompt rubric disagree on the 'Investment Amount Monthly' feature. Section 4 states that 'contributing less than 10% of income monthly suggests lower risk tolerance (+2 points), whereas investing more than 30% implies higher exposure and results in -2.' Appendix A.6 instructs models to award '+1 (>20%), 0, -1 (<20%)' for the same feature. A model that follows the prompt will therefore produce scores that cannot match the ground-truth values, and the deviation scores in Tables 1 and 2 do not measure the model's accuracy. The authors must either use identical rubrics or analyze the discrepancy explicitly.
  2. [Section 6.1, Tables 1 and 2] The correctness analysis compares each country's mean predicted score to fixed 'ideal' values (-5, 10, 21.5) for the Low, Mid, and High scenarios, rather than to the profile-specific ground-truth RT scores defined in Section 3. Because each profile has its own computed RT score, the deviation should be computed as predicted score minus ground-truth score per profile and then aggregated. The chosen ideals are not derived from the dataset (they are not the midpoints of the stated ranges), and they ignore the distribution of ground-truth scores within each scenario. Consequently, the reported 'mean difference' values do not indicate how well a model predicts individual risk tolerance.
  3. [Sections 4.1 and 6.2-6.4] The consistency analysis treats country and gender as isolated variables, but the paper never states that profiles are matched on the 16 financial features across these groups. Section 4.1 distinguishes 'highly populous' from 'less populous' countries and notes that individuals in the former 'lack access to formal banking systems' and have 'limited financial history,' implying the interview_text may contain different financial information. If so, higher scores for Nigeria and Indonesia (Table 3) could reflect legitimate differences in income, assets, or expenses rather than demographic bias. No matching, covariate adjustment, or ablation is reported, so the country- and gender-bias claims are not supported.
  4. [Introduction, Section 3, and Appendix A.6] The evaluation is circular with respect to the paper's central research question. The ground-truth RT score is defined by the same hand-built rubric that is placed verbatim into the model prompt, so a model that adheres to the prompt will reproduce the authors' scoring formula. The resulting 'correctness' scores therefore measure instruction-following, not the model's independent ability to assess investment risk appetite. The paper's framing ('Can AI Credible at Assessing Investment Risk?' and the claim that models fail to 'accurately predict an individual's risk appetite') is not supported by this design.
  5. [Abstract and Section 7] The claim that the observed inconsistencies 'violate AI and finance regulations' is not substantiated. The paper does not identify a specific regulatory provision or standard that the models' score differences violate, nor does it report any statistical test (e.g., significance of country/gender differences) that would support a finding of discrimination. Differences of 0.3-1.0 points, as reported in Sections 6.3-6.4, may be within the noise of the scoring procedure; without inferential statistics or a regulatory benchmark, the violation claim is overreaching.
minor comments (5)
  1. [Section 4 vs. Appendix A.6] The rubric and prompt also differ on several features beyond the monthly investment amount (e.g., Age is given as '+2, +1, 0' in the prompt without the thresholds specified in Section 4), and the prompt's allowed range of -15 to 28 is inconsistent with the -14 to 28 range defined in Section 4.
  2. [Section 5] The list of evaluated models is confusing: Section 5 says 'LLaMA 3.1 (70B and 405B)' were evaluated, but later in the same section it refers to 'LLaMA 3.1 (8B/70B)' among skipped models; please clarify which LLaMA variants were actually used in the reported results.
  3. [General] The paper would benefit from including at least one example of the interview_text for a profile, so readers can verify that all 16 features are represented and that country and gender are the only varying attributes across the constructed profiles.
  4. [Table 4] Table 4 is very dense and hard to parse; a separate column showing the female-male difference per country, or smaller tables per scenario, would help the reader assess the reported gender effects.
  5. [Section 6.1] The text describes mean differences such as 7.27 as 'deviation' without specifying whether these are signed or absolute deviations; since all Low-scenario entries are positive in Table 1, the sign indicates overestimation relative to the ideal, but the caption and text should state this explicitly.

Circularity Check

1 steps flagged · score 6.0 of 10

Correctness axis is self-referential: the ground-truth RT score is the paper's own five-category point system, and the model prompt asks the model to compute its score with that same five-category scoring system.

  1. self definitional [Sections 3, 4 and Appendix A.6; applied in Section 6.1 Correctness Analysis]
    "The profile-specific ground truth tolerance scores are mathematically computed using a total risk score expression (Section 3)... RT = PFS + ISO + LAA + MCR + DOI. ... Your task is to compute risk score using the following scoring system: 1. Personal & Financial Stability factors ... 5. Dependency on Investments factor ..."

    The correctness axis defines the 'ideal' RT as the output of the paper's own five-category additive point system, and the prompt given to every model instructs it to compute the score using that same five-category scoring system. The model is therefore being graded on how closely it reproduces the authors' rubric, not on any independent ability to assess risk appetite. A model that applies the instructed categories faithfully would reproduce the ground-truth structure by construction; deviations measure instruction-following, and the paper's 'accurate risk scores' conclusion is self-referential.

full rationale

The benchmark's consistency analysis is not circular: comparing model scores across countries and genders is an empirical test. However, the correctness analysis is partly self-referential because the ground truth is the paper's own scoring rubric and the model prompt contains the same five-category point system. A perfect 'rubric-follower' would trivially match the authors' ideal, so 'accuracy' reduces, in substantial part, to prompt compliance rather than independent risk-assessment credibility. This finding is mitigated but not eliminated by the fact that Appendix A.6 and Section 4 diverge on at least one feature (monthly investment amount: Section 4 gives +2 for under 10% and -2 for over 30%, while the prompt gives +1 for over 20% and -1 for under 20%), meaning the model is also graded against a standard different from the one it was asked to follow. The country-level bias conclusion is additionally confounded because the paper never states that the 16 financial features are matched across countries or genders; Section 4.1 itself describes country groups as differing in banking infrastructure and financial history, so higher Nigerian/Indonesian scores could reflect legitimate financial-capacity differences rather than bias. No load-bearing self-citations or imported uniqueness theorems appear in the paper. The overall score reflects partial circularity concentrated in the correctness axis, while the consistency claim retains independent, though incompletely controlled, empirical content.

Assumptions & free parameters 8 free parameters · 4 assumptions · 1 invented entities

The central claim depends on a hand-authored scoring rubric that is also given to the models in the prompt. Every threshold and weight in that rubric is a free parameter. The rubric contains an internal contradiction, and the RT score itself is an invented construct with no external validation.

free parameters (8)
  • Age point thresholds = <30: +2, 30-50: +1, >50: 0
    Hand-chosen cutoffs for risk capacity; no calibration against real investors.
  • Income thresholds (USD) = >$100K: +2, $50-100K: +1, <$50K: -1
    Arbitrary income bands used in ground-truth RT.
  • Debt-to-asset ratio thresholds = <20%: +2, 20-40%: +1, >40%: -2
    Arbitrary cutoffs.
  • Expense-to-income thresholds = <30%: +2, 30-50%: 0, >50%: -2
    Arbitrary cutoffs.
  • Investment tenure thresholds = >15y: +2, 5-15y: +1, <5y: 0
    Arbitrary cutoffs.
  • Target net worth ratio thresholds = <5x income: +2, 5-10x: +1, >10x: -2
    Arbitrary cutoffs.
  • Monthly investment amount mapping = Section 4: <10% +2, >30% -2; Appendix A.6: >20% +1, <20% -1
    Internal inconsistency between ground-truth rubric and model prompt.
  • Dimension weights = All five dimensions weighted equally
    No justification for equal weighting of PFS, ISO, LAA, MCR, DOI.
assumptions (4)
  • domain assumption The chosen 16 features are a complete and correct set of determinants of investment risk tolerance.
    Feature selection is based on regulations and literature, but completeness is not demonstrated; the paper does not use an established validated instrument.
  • domain assumption Higher summed scores correspond to higher actual risk appetite.
    The sign and magnitude of each feature's contribution are assumed without empirical validation.
  • ad hoc to paper The ideal scenario targets (-5, 10, 21.5) are appropriate reference values for low, mid, and high risk profiles.
    The paper does not explain how these ideals relate to the profile-specific ground truths or to real risk-tolerance distributions.
  • ad hoc to paper The ground-truth rubric (Section 4) is identical to the rubric in the prompt (Appendix A.6).
    This assumption is false for the monthly investment feature, invalidating the correctness comparison.
invented entities (1)
  • Risk Tolerance (RT) score
    purpose: Ground truth for evaluating LLM predictions in FINRISK EVAL
    A hand-built additive score with arbitrary weights and thresholds; not validated against real investor behavior or external risk-tolerance instruments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Evaluating AI for Finance: Is AI Credible at Assessing Investment Risk?." pith.science (2026). https://pith.science/paper/G4RM7W4N

@misc{pith2026250518953,
  author       = {Pith},
  title        = {Pith review of: Evaluating AI for Finance: Is AI Credible at Assessing Investment Risk?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G4RM7W4N}},
  note         = {Machine review of arXiv:2505.18953}
}
read the original abstract

We assess whether AI systems can credibly evaluate investment risk appetite-a task that must be thoroughly validated before automation. Our analysis was conducted on proprietary systems (GPT, Claude, Gemini) and open-weight models (LLaMA, DeepSeek, Mistral), using carefully curated user profiles that reflect real users with varying attributes such as country and gender. As a result, the models exhibit significant variance in score distributions when user attributes-such as country or gender-that should not influence risk computation are changed. For example, GPT-4o assigns higher risk scores to Nigerian and Indonesian profiles. While some models align closely with expected scores in the Low- and Mid-risk ranges, none maintain consistent scores across regions and demographics, thereby violating AI and finance regulations.

Figures

Figures reproduced from arXiv: 2505.18953 by the authors.

Figure 1
Figure 1. Factors determining investment risk profile. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 22 canonical work pages

  1. [1]

    Filippo Chiarello, Vito Giordano, Irene Spada, Simone Barandoni, and Gualtiero Fantoni. 2024. Future applications of generative large language models: A data-driven case study on chatgpt. Technovation, 133:103002

    work page 2024

  2. [2]

    Hanjun Dai, Hongyu Zhang, and Zhi Wang. 2023. Fairness-aware fine-tuning for ai investment advice systems. Proceedings of the 2023 IEEE International Conference on Big Data

    work page 2023

  3. [3]

    ESMA . 2023. https://www.esma.europa.eu/sites/default/files/2023-04/ESMA35-43-3172_Guidelines_on_certain_aspects_of_the_MiFID_II_suitability_requirements.pdf Guidelines on certain aspects of the MiFID II suitability requirements . Accessed: 2025-03-17

    work page 2023

  4. [4]

    Heather M. Farrell. 2006. The role of emergency funds in a comprehensive financial plan. Journal of Personal Finance

    work page 2006

  5. [5]

    FCA . 2018. https://www.handbook.fca.org.uk/handbook/COBS/9A/2.html Assessing Suitability and Risk Appetite Guidelines . Accessed: 2025-03-17

    work page 2018

  6. [6]

    FINRA . 2012. https://www.finra.org/rules-guidance/key-topics/suitability Suitability and Risk Tolerance Guidelines . Accessed: 2025-03-17

    work page 2012

  7. [7]

    FSA . 2022. https://www.fsa.go.jp/en/laws_regulations/eng_kinsa.pdf Guidelines on Investment Suitability and Risk Tolerance . Accessed: 2025-03-17

    work page 2022

  8. [8]

    Abhishek Garg and Anirban Ghosh. 2022. Counterfactual evaluation of bias in financial ai systems. Proceedings of the 2022 International Conference on Financial Technology

    work page 2022

Show all 25 references
  1. [9]

    Gartner. 2023. https://www.technologyrecord.com/article/more-than-80-per-cent-of-enterprises-to-adopt-some-form-of-generative-ai-by-2026-says-gartner More than 80 per cent of enterprises to adopt some form of generative ai by 2026, says gartner . Technology Record

    2023

  2. [10]

    Grable and Ruth H

    John E. Grable and Ruth H. Lytton. 1999. Financial risk tolerance revisited: The development of a risk assessment instrument. Financial Services Review

    1999

  3. [11]

    Xiang Guo, Jian Li, and Wei Zhang. 2024. Investment advice and risk assessment using large language models: A study on biases. arXiv preprint arXiv:2405.11231

    2024 arXiv

  4. [12]

    IDC . 2024. https://blogs.microsoft.com/blog/2024/11/12/idcs-2024-ai-opportunity-study-top-five-ai-trends-to-watch/ 2024 Business Opportunity of AI: Generative AI Delivering New Business Value and Increasing ROI . Technical report, International Data Corporation (IDC) . InfoBr...

    2024

  5. [13]

    Taylor Larimore, Michael Lindauer, and Mel LeBoeuf. 2009. The Bogleheads' Guide to Retirement Planning. Wiley

    2009

  6. [14]

    Yifan Liu, Jiaqi Wang, and Zhi Zhang. 2024. Bias in ai investment advice: An analysis of home bias in large language models. arXiv preprint arXiv:2209.04538

    2024 arXiv

  7. [15]

    Scott M Lundberg and Su-In Lee. 2017. Shap: Shapley additive explanations for ai models. Proceedings of the 2017 Conference on Advances in Neural Information Processing Systems

    2017

  8. [16]

    David Madras, Eric Creager, Toniann Pitassi, and Richard Zemel. 2018. Adversarial training for fairness: Mitigating bias in ai financial advice. Proceedings of the 2018 Conference on Neural Information Processing Systems (NeurIPS)

    2018

  9. [17]

    Harry Markowitz. 1952. Portfolio selection. The Journal of Finance

    1952

  10. [18]

    MAS . 2023. https://www.mas.gov.sg/regulation/notices/notice-faa-n16 Notice FAA-N16: Recommendations on Investment Products . Accessed: 2025-03-17

    2023

  11. [19]

    Inioluwa Deborah Raji and Joy Buolamwini. 2020. Actionable auditing of ai systems in financial services: A fairness perspective. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency

    2020

  12. [20]

    Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. Lime: Local interpretable model-agnostic explanations. Proceedings of the 2016 Conference on Knowledge Discovery and Data Mining (KDD)

    2016

  13. [21]

    Stanley and William D

    Thomas J. Stanley and William D. Danko. 1996. The Millionaire Next Door: The Surprising Secrets of America's Wealthy. Longstreet Press

    1996

  14. [22]

    Jin Zhao, Shuang Wang, Jun Yao, Li Ding, and Haoyan Li. 2019. Learning disentangled representations for fairness in classification. In Proceedings of the 2019 Conference on Fairness, Accountability, and Transparency (FAT* 2019), pages 243--252. ACM

    2019

  15. [23]

    Yuhan Zhi, Xiaoyu Zhang, Longtian Wang, Shumin Jiang, Shiqing Ma, Xiaohong Guan, and Chao Shen. 2025. Exposing product bias in llm investment recommendation. arXiv preprint arXiv:2503.08750

    2025 arXiv

  16. [24]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  17. [25]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.