pith. sign in

arxiv: 2604.27022 · v1 · submitted 2026-04-29 · 💻 cs.HC

Breaking Bad Financial Habits: How LLM Conversations Correct Financial Misconceptions

Jillian Ross , Eric So , Andrew W. Lo This is my paper

Pith reviewed 2026-05-07 13:03 UTC · model grok-4.3

classification 💻 cs.HC
keywords financial misconceptionsLLM conversationsfinancial literacybehavioral financecorrective interventionsrecipient receptivityinvestor behavior
0
0 comments X

The pith

Purposefully designed LLM conversations durably correct financial misconceptions when they include corrective intent and match user sophistication.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Financial misconceptions lead to real costs such as panic selling during downturns and avoiding equity markets, yet traditional literacy programs struggle with reach and lasting behavioral change. The paper demonstrates that LLMs can produce durable corrections to these misconceptions across three pre-registered studies. This outcome holds only when the LLM is prompted with corrective intent rather than neutral discussion, and when its responses align with the recipient's level of financial knowledge so they are seen as credible. Without those two elements, conversations fail to outperform simple self-reflection and can even strengthen misconceptions. The finding points to a potential low-cost, scalable route for addressing resistant financial errors that conventional approaches have not solved.

Core claim

Across three pre-registered studies, purposefully designed LLMs durably correct financial misconceptions. This requires two factors: corrective intent, where the LLM is prompted to discuss and fix the misconception, and recipient receptivity, where responses match the participant's financial sophistication. Without corrective intent, conversations produce corrections no better than unassisted self-reflection and can entrench misconceptions. Responses pitched below a participant's sophistication level are judged less credible and yield weaker corrections.

What carries the argument

The paired requirements of corrective intent in the LLM prompt and recipient receptivity through matched sophistication level, which together enable durable correction of misconceptions that undirected or mismatched conversations cannot achieve.

Load-bearing premise

That the corrections observed in controlled study settings will translate into lasting real-world changes in financial behavior such as reduced panic selling or increased market participation.

What would settle it

A longitudinal study measuring actual trading records or market participation rates before and after LLM conversations that finds no reduction in panic selling or increase in equity holdings among treated participants compared to controls.

read the original abstract

Financial misconceptions carry direct economic costs, from panic selling to equity market avoidance, yet they are notoriously resistant to correction. Traditional financial literacy interventions are constrained by cost, reach, and a persistent gap between knowledge and behavioral change. Across three pre-registered studies, we find that purposefully designed LLMs can durably correct financial misconceptions. Critically, two factors are necessary for this effect. First, corrective intent: LLMs prompted only to discuss a misconception produce corrections no better than unassisted self-reflection, and undirected LLM conversations can actively entrench misconceptions. Second, recipient receptivity: financial concepts are often foreign to the investors who misapply them, and LLM responses pitched below a participant's financial sophistication are judged as less credible and produce substantially weaker corrections. LLMs thus offer a scalable alternative to traditional financial literacy intervention, but only when designed with both factors in mind.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript reports results from three pre-registered studies claiming that LLMs can durably correct financial misconceptions when designed with corrective intent and matched to recipient receptivity; undirected or mismatched conversations produce no benefit or entrench misconceptions, positioning LLMs as a scalable alternative to traditional financial literacy interventions.

Significance. If the core empirical findings hold, the work offers a practical framework for using LLMs in financial education to address misconceptions linked to costly behaviors. The emphasis on intent and receptivity provides concrete design guidance, and pre-registration is a methodological strength that reduces certain biases.

major comments (2)
  1. [Abstract] Abstract: The assertion that LLMs 'durably correct financial misconceptions' and constitute a 'scalable alternative' rests on immediate post-conversation knowledge or attitude shifts in hypothetical scenarios; without evidence of sustained changes in real financial behaviors (such as actual trading activity, panic selling, or market participation over time), the durability and real-world impact claims are not supported by the reported studies.
  2. [Abstract and Studies 1-3] Abstract and Studies 1-3: The necessity of 'corrective intent' and 'recipient receptivity' is presented as critical, yet the manuscript does not provide sample sizes, statistical power calculations, effect sizes, or explicit controls for demand effects, preventing full evaluation of whether the two-factor requirement is robustly demonstrated across the experiments.
minor comments (1)
  1. [Abstract] Abstract: The phrase 'purposefully designed LLMs' is used without a concise definition of the prompting strategies; adding one sentence clarifying the operationalization of corrective intent would improve readability.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments, which help clarify the scope of our claims and improve statistical transparency. We address each major point below and commit to revisions that align the manuscript more precisely with the reported evidence.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The assertion that LLMs 'durably correct financial misconceptions' and constitute a 'scalable alternative' rests on immediate post-conversation knowledge or attitude shifts in hypothetical scenarios; without evidence of sustained changes in real financial behaviors (such as actual trading activity, panic selling, or market participation over time), the durability and real-world impact claims are not supported by the reported studies.

    Authors: We agree that the three pre-registered studies measure immediate and follow-up changes in knowledge and attitudes within controlled, hypothetical scenarios rather than long-term real-world financial behaviors. The reported durability applies to persistence of corrected misconceptions in these experimental measures. We will revise the abstract and add a limitations paragraph to specify that claims of durability and scalability refer to knowledge/attitude corrections in the study contexts, while noting the absence of direct behavioral data and the value of future field studies on actual trading or market participation. revision: yes

  2. Referee: [Abstract and Studies 1-3] Abstract and Studies 1-3: The necessity of 'corrective intent' and 'recipient receptivity' is presented as critical, yet the manuscript does not provide sample sizes, statistical power calculations, effect sizes, or explicit controls for demand effects, preventing full evaluation of whether the two-factor requirement is robustly demonstrated across the experiments.

    Authors: The studies are pre-registered and the full text reports sample sizes, power analyses, and effect sizes for the key interactions supporting the two-factor model. Demand effects were addressed via control conditions and attention checks. To improve accessibility, we will add an explicit statistical reporting subsection with sample sizes, achieved power, effect sizes, and demand-effect controls, plus updated tables summarizing these metrics for each study. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical pre-registered studies with no derivations or self-referential claims

full rationale

The paper reports results from three pre-registered behavioral studies measuring immediate post-conversation knowledge and attitude shifts in hypothetical scenarios. No mathematical derivations, equations, fitted parameters, or predictive models are present. Claims rest directly on experimental outcomes rather than any reduction to inputs by construction, self-citation chains, or ansatz smuggling. Self-citations (if any) are not load-bearing for a derivation chain because none exists. The work is self-contained against standard benchmarks for empirical HCI and behavioral finance experiments.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on empirical findings from controlled studies rather than new theoretical constructs or derivations; it relies on standard assumptions of experimental design in behavioral science.

axioms (1)
  • domain assumption Standard assumptions of randomized controlled experiments hold, including random assignment, no spillover effects, and that self-reported receptivity and correction measures reflect true belief change.
    The studies are described as pre-registered experiments testing causal effects of LLM design factors.

pith-pipeline@v0.9.0 · 5445 in / 1291 out tokens · 76679 ms · 2026-05-07T13:03:09.842531+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

31 extracted references · 31 canonical work pages · 1 internal anchor

  1. [1]

    science 185(4157), 1124–1131 (1974)

    Tversky, A., Kahneman, D.: Judgment under uncertainty: Heuristics and biases: Biases in judgments reveal some heuristics of thinking under uncertainty. science 185(4157), 1124–1131 (1974)

    work page 1974

  2. [2]

    Handbook of the Economics of Finance1, 1053–1128 (2003)

    Barberis, N., Thaler, R.: A survey of behavioral finance. Handbook of the Economics of Finance1, 1053–1128 (2003)

    work page 2003

  3. [3]

    Journal of Experimental Psychology: General125(4), 387 (1996)

    Wilson, T.D., Houston, C.E., Etling, K.M., Brekke, N.: A new look at anchoring effects: basic anchoring and its antecedents. Journal of Experimental Psychology: General125(4), 387 (1996)

    work page 1996

  4. [4]

    American economic review96(4), 1069–1090 (2006)

    Barberis, N., Huang, M., Thaler, R.H.: Individual preferences, monetary gambles, and stock market participation: A case for narrow framing. American economic review96(4), 1069–1090 (2006)

    work page 2006

  5. [5]

    Journal of Financial Economics102(1), 1–27 (2011)

    Bailey, W., Kumar, A., Ng, D.: Behavioral biases of mutual fund investors. Journal of Financial Economics102(1), 1–27 (2011)

    work page 2011

  6. [6]

    Swiss journal of economics and statistics155(1), 1–8 (2019)

    Lusardi, A.: Financial literacy and the need for financial education: evidence and implications. Swiss journal of economics and statistics155(1), 1–8 (2019)

    work page 2019

  7. [7]

    Journal of Financial economics101(2), 449–472 (2011)

    Van Rooij, M., Lusardi, A., Alessie, R.: Financial literacy and stock market participation. Journal of Financial economics101(2), 449–472 (2011)

    work page 2011

  8. [8]

    Social Indicators Research143(1), 325–353 (2019)

    Karakurum-Ozdemir, K., Kokkizil, M., Uysal, G.: Financial literacy in developing countries. Social Indicators Research143(1), 325–353 (2019)

    work page 2019

  9. [9]

    The Journal of Finance76(2), 587–621 (2021)

    Linnainmaa, J.T., Melzer, B.T., Previtero, A.: The misguided beliefs of financial advisors. The Journal of Finance76(2), 587–621 (2021)

    work page 2021

  10. [10]

    The Review of Financial Studies32(5), 1983–2020 (2019)

    D’Acunto, F., Prabhala, N., Rossi, A.G.: The promises and pitfalls of robo- advising. The Review of Financial Studies32(5), 1983–2020 (2019)

    work page 1983

  11. [11]

    Hackethal, A., Haliassos, M., Jappelli, T.: Financial advisors: A case of babysit- ters? Journal of Banking & Finance36(2), 509–524 (2012)

    work page 2012

  12. [12]

    World Bank

    Klapper, L., Lusardi, A., Van Oudheusden, P.: Financial literacy around the world. World Bank. Washington DC: World Bank2, 218–237 (2015)

    work page 2015

  13. [13]

    Journal of Experimental Psychology: Applied29(1), 52 (2023)

    Altay, S., Hacquin, A.-S., Chevallier, C., Mercier, H.: Information delivered by a chatbot has a positive impact on covid-19 vaccines attitudes and intentions. Journal of Experimental Psychology: Applied29(1), 52 (2023)

    work page 2023

  14. [14]

    Nature Human Behaviour6(4), 579–592 (2022) 14 Preprint

    Altay, S., Schwartz, M., Hacquin, A.-S., Allard, A., Blancke, S., Mercier, H.: Scal- ing up interactive argumentation by providing counterarguments with a chatbot. Nature Human Behaviour6(4), 579–592 (2022) 14 Preprint

    work page 2022

  15. [15]

    Journal of medical Internet research25, 40789 (2023)

    Aggarwal, A., Tam, C.C., Wu, D., Li, X., Qiao, S.: Artificial intelligence–based chatbots for promoting health behavioral changes: systematic review. Journal of medical Internet research25, 40789 (2023)

    work page 2023

  16. [16]

    On the Conversational Persuasive- ness of Large Language Models: A Randomized Controlled Trial, March 2024

    Salvi, F., Ribeiro, M.H., Gallotti, R., West, R.: On the conversational persua- siveness of large language models: A randomized controlled trial. arXiv preprint arXiv:2403.14380 (2024)

    work page arXiv 2024

  17. [17]

    Science385(6714), 1814 (2024)

    Costello, T.H., Pennycook, G., Rand, D.G.: Durably reducing conspiracy beliefs through dialogues with ai. Science385(6714), 1814 (2024)

    work page 2024

  18. [18]

    In: Com- munication and Persuasion: Central and Peripheral Routes to Attitude Change, pp

    Petty, R.E., Cacioppo, J.T.: Message elaboration versus peripheral cues. In: Com- munication and Persuasion: Central and Peripheral Routes to Attitude Change, pp. 141–172. Springer, New York, NY (1986)

    work page 1986

  19. [19]

    GPT-4o System Card

    Hurst, A., Lerer, A., Goucher, A.P., Perelman, A., Ramesh, A., Clark, A., Ostrow, A., Welihinda, A., Hayes, A., Radford, A., et al.: Gpt-4o system card. arXiv preprint arXiv:2410.21276 (2024)

    work page internal anchor Pith review arXiv 2024

  20. [20]

    Journal of financial economics88(2), 299–322 (2008)

    Frazzini, A., Lamont, O.A.: Dumb money: Mutual fund flows and the cross-section of stock returns. Journal of financial economics88(2), 299–322 (2008)

    work page 2008

  21. [21]

    The Journal of Finance52(1), 57–82 (1997)

    Carhart, M.M.: On persistence in mutual fund performance. The Journal of Finance52(1), 57–82 (1997)

    work page 1997

  22. [22]

    The Journal of Finance65(5), 1915–1947 (2010)

    Fama, E.F., French, K.R.: Luck versus skill in the cross-section of mutual fund returns. The Journal of Finance65(5), 1915–1947 (2010)

    work page 1915

  23. [23]

    Annual Review of Financial Economics8(1), 197–219 (2016)

    Kothari, S.P., So, E., Verdi, R.: Analysts’ forecasts and asset pricing: A survey. Annual Review of Financial Economics8(1), 197–219 (2016)

    work page 2016

  24. [24]

    American Economic Review91(1), 79–98 (2001)

    Benartzi, S., Thaler, R.H.: Naive diversification strategies in defined contribution saving plans. American Economic Review91(1), 79–98 (2001)

    work page 2001

  25. [25]

    Review of Finance 12(3), 433–463 (2008)

    Goetzmann, W.N., Kumar, A.: Equity portfolio diversification. Review of Finance 12(3), 433–463 (2008)

    work page 2008

  26. [26]

    Journal of Financial and Quantitative Analysis44(6), 1375–1401 (2009)

    Kumar, A.: Hard-to-value stocks, behavioral biases, and informed trading. Journal of Financial and Quantitative Analysis44(6), 1375–1401 (2009)

    work page 2009

  27. [27]

    The Journal of Finance74(5), 2153–2199 (2019)

    Hartzmark, S.M., Solomon, D.H.: The dividend disconnect. The Journal of Finance74(5), 2153–2199 (2019)

    work page 2019

  28. [28]

    The Review of Financial Studies23(4), 1405– 1432 (2010)

    Choi, J.J., Laibson, D., Madrian, B.C.: Why does the law of one price fail? an experiment on index mutual funds. The Review of Financial Studies23(4), 1405– 1432 (2010)

    work page 2010

  29. [29]

    The Journal of Business78(6), 2095–2120 (2005)

    Barber, B.M., Odean, T., Zheng, L.: Out of sight, out of mind: The effects of 15 Preprint expenses on mutual fund flows. The Journal of Business78(6), 2095–2120 (2005)

    work page 2095

  30. [30]

    The Journal of Finance59(1), 261–288 (2004)

    Elton, E.J., Gruber, M.J., Busse, J.A.: Are investors rational? choices among index funds. The Journal of Finance59(1), 261–288 (2004)

    work page 2004

  31. [31]

    {{Sophistication}}

    Gruber, M.J.: Another puzzle: The growth in actively managed mutual funds. The Journal of Finance51(3), 783–810 (1996) 16 Preprint A Appendix A.1 Study 1 Results and Analysis BeliefShifti =β 0 +β 11[Shift]i +β 21[Evaluate]i +β 31[Self-Articulate]i +ε i (3) BeliefShifti =β 0 +β 11[Shift]i +β 21[Evaluate]i +β 31[Self-Articulate]i +β 4Initiali +ε i (4) Belie...

    work page 1996