Pith. sign in

REVIEW 3 major objections 8 minor 3 cited by

Pitfalls in Evaluating Language Model Forecasters

T0 review · 3 major / 8 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Claims that LLMs match human forecasters are not yet established by current benchmarks.

desk verdict A sound, well-hedged methodological critique of LLM forecasting benchmarks that doesn't overclaim and deserves refereeing, even though it doesn't prove scores are inflated. read the letter →

arxiv 2506.00723 v1 pith:MYPJ2XRD submitted 2025-05-31 cs.LG cs.AIcs.IR

Daniel Paleka , Shashwat Goel , Jonas Geiping , Florian Tramèr This is my paper
classification cs.LGcs.AIcs.IR
keywords LLMforecastingtemporalleakagebacktestingbenchmarkevaluationBrierscorecalibrationpredictionmarketdata
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that existing evaluations showing LLMs rival or exceed human forecasters are untrustworthy. It identifies two broad categories of problems: temporal leakage, which lets a model use post-forecast information through logical deductions, retrieval, and model cutoffs; and benchmark-design issues, which let a system score well without genuine forecasting skill. The paper gives concrete examples from prior work, including questions that are trivially answerable once a forecaster knows it is being backtested. If the analysis is right, current performance claims for LLM forecasters are likely inflated, and more rigorous evaluation methodologies are needed before the community can confidently assess their forecasting abilities.

What carries the argument

The paper's analysis is carried by a catalogue of specific leakage and gaming channels, each illustrated with concrete examples, plus two toy models that formalize the mechanisms. The leakage channels are: logical leakage from selecting resolved questions, retrieval leakage from incorrect date metadata and future-trained ranking, and cutoff leakage from unreliable training-cutoff claims. The gaming channels are piggybacking on human forecasts, betting on shared latent variables, distributional skew, and metric artifacts. The latent-variable toy model shows that a calibrated forecaster can never maximize its chance of winning a contest against forecasters who commit to one world; the label-noise toy model shows that clamping probability estimates improves logarithmic score.

What would settle it

A decisive test would be to run a top-performing LLM forecaster on the same benchmark questions with every leakage channel closed—using a verified past-corpus for retrieval, a release-date cutoff, and no access to human forecast aggregates or post-resolution data—then compare the resulting Brier and logarithmic scores against the originally reported scores. If the scores drop by less than a pre-registered margin, the leakage concerns would be shown not to explain the reported performance.

Watch

Extended reading notes

Core claim

The central claim is that backtesting LLM forecasters is far more difficult than it appears, and that current benchmarks admit shortcuts that can inflate scores. Logical leakage arises because question selection implicitly encodes the future: a forecaster who knows it is being evaluated in 2025 can deduce that a question about an event that has not yet resolved by that date must resolve negatively. Unreliable date-restricted retrieval means search engines return future-dated or future-ranked content even when filters are applied. Model knowledge cutoffs are not guarantees and can be extended by system prompts and later preference data. Even with a clean evaluation, the paper argues that piggybacking on human forecasts makes 'matching humans' claims circular, betting on shared latent variables can win benchmarks without calibrated judgment, benchmark question distributions are heavily skewed, and common metrics such as accuracy, calibration, and Brier score can reward the wrong sort of behavior. The paper also shows that using backtests as a training objective introduces its own temporal leakage.

Load-bearing premise

The argument's load-bearing premise is that the leakage channels occur often enough in real benchmarks to materially inflate scores, rather than merely being possible in theory; the paper gives concrete instances but does not measure their aggregate impact on final performance claims.

Editorial extensions

If this is right

  • Reported LLM forecasting scores on current benchmarks should not be read as evidence of human-level or superhuman predictive skill.
  • Evaluation protocols should use release dates instead of knowledge cutoffs, restrict retrieval to a dated corpus, and measure the forecaster's edge over the human crowd rather than absolute agreement.
  • Training on backtests will introduce temporal leakage unless the pipeline separates forecasting from memorization of earlier events.
  • Live evaluation on prediction markets with profit and risk-adjusted return reporting is a more trustworthy end-to-end test than backtests.
  • Current benchmark leaderboards may mis-rank models because different systems exploit shortcuts to different degrees.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same leakage channels likely affect other temporal prediction tasks, such as medical outcome prediction or economic event forecasting, whenever models are evaluated on past data with retrieval or fine-tuning.
  • The latent-variable betting incentive suggests that leaderboards for forecasters may systematically select overconfident models, so a top-ranked system could be less calibrated than an unranked one.
  • A testable extension would be to run a top forecaster with all leakage channels blocked and compare scores against published results; a small gap would indicate the leaks are not the main driver of performance claims.
  • The piggybacking critique implies that 'matching human forecasters' claims should be compared against the best available baseline of incremental value over public crowd forecasts, not against absolute agreement with them.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. The paper argues that evaluating LLM forecasters through backtesting is fraught with pitfalls that can undermine performance claims. It identifies temporal leakage (logical leakage from resolved-question selection, unreliable date-restricted retrieval, and model cutoff uncertainty) and extrapolation issues (piggybacking on human forecasts, benchmark gaming via correlated bets, skewed data distributions, and imperfect metrics). It provides concrete examples, including percentages of trivially resolvable questions in prior benchmarks and a DeBERTa shortcut classifier, plus two analytically solved toy models showing that label noise can make clamping optimal for logarithmic score and that betting on a shared latent can maximize win probability. The authors carefully limit their conclusion: they do not claim that existing scores are inflated, but argue that the community should be cautious about current performance claims.

Significance. This is a timely and valuable critique that systematizes scattered concerns into a clear checklist for evaluating forecasting LLMs and gives concrete demonstrations that leakage channels are not merely hypothetical. Its main strength is the honest framing: the central claim is about warranted caution, not about proving inflation, so the absence of end-to-end score deltas is a limitation rather than a fatal flaw. The two toy models in Appendices D and E are clean, analytically worked out, and appear correct; the paper also explicitly states what would be needed to convert its concerns into quantified conclusions. The result, if accepted, is an agenda-setting paper that should influence how future forecasting benchmarks are designed and reported.

major comments (3)
  1. [Section 2.1 and Appendix C] The quantitative evidence for practical leakage is not reproducible as reported. The paper reports 'at least 3.8%' of Halawi et al. and 'at least 10%' of Tao et al. as trivially resolvable, but it does not define the criterion used to label a question as trivial, nor does it release the annotations. Likewise, Appendix C reports DeBERTa shortcut-classifier accuracies up to 80% without code, confidence intervals, or a full experimental protocol (e.g., number of questions, hyperparameters, number of seeds). Since these numbers are used to argue that leakage is a practical concern and not merely a theoretical one, the authors should either release the protocol and annotations and report error bars, or explicitly downgrade these numbers to illustrative estimates.
  2. [Section 2.1] The logical-leakage argument applies only if the forecaster knows that it is being backtested, knows the resolution date or evaluation window, and knows that questions were selected because they resolved. The paper states this condition internally, but the empirical paragraphs then treat the 3.8% and 10% figures as evidence of leakage without testing or even discussing whether the systems in Halawi et al. and Tao et al. actually receive the relevant information or would exploit it. The authors should add a sensitivity test (e.g., prompting with and without the backtest date) or explicitly state in the empirical subsection that these percentages measure a conditional possibility, not a demonstrated exploit.
  3. [Section 3.2 and Appendix E] The gaming argument shows that, in a contest with a shared latent variable, a forecaster who bets on one latent outcome can maximize the probability of winning. However, the paper does not connect this to any observed behavior of current LLM forecasters or to the actual scoring and selection protocols of the cited benchmarks. The authors acknowledge this ('It remains unclear if current LLM forecasters resort to such strategies'), but the section would be more proportionately framed if it explicitly stated that this is a theoretical possibility and that no current benchmark has been shown to be gameable in this way. A small Monte Carlo demonstration of win probabilities under the Appendix E model would also make the practical relevance easier to assess.
minor comments (8)
  1. [Abstract] Consider adding one sentence that explicitly disclaims any claim that existing Brier or logarithmic scores are inflated; the Discussion already contains this caveat, and placing it in the abstract would help prevent misreading of the paper's scope.
  2. [Section 1] The time-traveler analogy is slightly confusing: if someone from 2035 asks whether alien life will be found before 2040, the deduction 'yes' applies only if the question is known to be part of a backtest in which it has already resolved. Please clarify that the leakage comes from the resolved-question selection, not from the mere fact of being asked by a time traveler.
  3. [Appendix C] Please report the exact number of questions used for each benchmark, the DeBERTa variant, and whether results are averaged over multiple seeds or a single run. Without these details, the striking 55-80% accuracy figures are difficult to assess.
  4. [Section 2.1] The estimate that 'over 90%' of Dai et al. questions are overly specific is presented without a protocol and is described as inherently subjective; please move this to a clearly labeled informal estimate or provide an annotation methodology so that readers can distinguish it from the reproducible 3.8% and 10% numbers.
  5. [Figure 4] The caption contains a grammar error: 'Figure 4d and Figure 4c shows' should be 'show'. Also, please specify the exact date-restriction parameters used for each search so the examples can be reproduced.
  6. [Appendix D] The notation uses p both as the random true probability and as the integration variable; renaming the integration variable (e.g., u) in the integrals for E[ell] and E[ell_t] would improve readability.
  7. [References] The reference 'Review of Financial Sttidies' should be 'Review of Financial Studies'.
  8. [Section 3.1] The piggybacking claim would be strengthened by checking whether the cited benchmarks actually grant the LLM access to retrieval of prediction-market pages; as written, the claim is conditional on an assumption that may not hold for all compared systems.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation; only a minor non-load-bearing self-citation to the authors' own prior dataset.

full rationale

This paper is a critique of LLM forecasting evaluation practices. It does not fit parameters to data and then predict a closely related quantity, nor does it derive a conclusion from an assumption that already contains the conclusion. The central leakage arguments rest on externally documented evidence: search-engine screenshots, elicited GPT-4o responses, question-level counts from Halawi et al. and Tao et al., and two self-contained toy proofs in Appendices D and E. The only author-overlapping citation is Paleka et al. (2024), which is used to identify back-generated news datasets, to note an incomplete augmentation fix, and to cite a 1-5% label-noise rate. These uses are supporting details, not the load-bearing premise of the paper's main claim. The Appendix C DeBERTa shortcut classifier is a new analysis performed on an existing dataset, so it is not a fitted input renamed as a prediction. The paper explicitly disclaims quantitative impact on the prior performance claims ('We also do not have proof that the benchmark issues we uncover would lower the performance claims of LLM forecasters'), which weakens the conclusion but does not make the reasoning circular. No equation reduces to its own input, and no benchmark outcome is forced by a self-citation chain. The score of 2 reflects only the presence of a minor, non-load-bearing self-citation.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

The paper makes no fitted predictions and introduces no new entities. It rests on domain assumptions that leakage channels can occur and are not eliminated by current safeguards, as stated in Sections 2 through 4. The toy models in Appendices D and E use standard probability assumptions plus explicitly modeled priors and noise, and they are clearly labeled as examples.

assumptions (6)
  • domain assumption A backtested forecaster that knows the evaluation date can deduce that all questions in the benchmark must have resolved by that date.
    Central to Issue 1 (logical leakage) in Section 2.1; requires the model to know the backtesting setup and the benchmark construction protocol.
  • domain assumption Search engines with date restrictions return results whose ranking is partly determined by post-date knowledge.
    Invoked for the retrieval leakage examples in Section 2.2 and Appendix A; screenshots support the possibility but not a measured rate.
  • domain assumption Stated knowledge cutoff dates are not reliable boundaries for a model's information.
    Used in Section 2.3; supported by the gpt-4o handling example and system prompt leaks, but not by a broad audit.
  • domain assumption Human forecasts from prediction markets are likely present in LLM training data or retrievable at evaluation time.
    Basis of the piggybacking critique in Section 3.1; the paper gives Manifold examples but no direct evidence that a specific benchmark's model retrieved them.
  • domain assumption Optimizing a model on a chronological sequence of backtested events corrupts the test signal through parameter updates.
    Section 4's futurity concern; this is a forward-looking argument, not empirically demonstrated.
  • ad hoc to paper For the label-noise toy model, true probabilities are uniformly distributed on [0,1] and label noise is symmetric with constant rate eta.
    Appendix D explicitly assumes this model; it is a self-contained worked example, not a claim about real datasets.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Pitfalls in Evaluating Language Model Forecasters." pith.science (2026). https://pith.science/paper/MYPJ2XRD

@misc{pith2026250600723,
  author       = {Pith},
  title        = {Pith review of: Pitfalls in Evaluating Language Model Forecasters},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MYPJ2XRD}},
  note         = {Machine review of arXiv:2506.00723}
}
read the original abstract

Large language models (LLMs) have recently been applied to forecasting tasks, with some works claiming these systems match or exceed human performance. In this paper, we argue that, as a community, we should be careful about such conclusions as evaluating LLM forecasters presents unique challenges. We identify two broad categories of issues: (1) difficulty in trusting evaluation results due to many forms of temporal leakage, and (2) difficulty in extrapolating from evaluation performance to real-world forecasting. Through systematic analysis and concrete examples from prior work, we demonstrate how evaluation flaws can raise concerns about current and future performance claims. We argue that more rigorous evaluation methodologies are needed to confidently assess the forecasting abilities of LLMs.

Figures

Figures reproduced from arXiv: 2506.00723 by the authors.

Figure 1
Figure 1. Examples of hard-to-filter temporal leakage in search engines, bypassing date restrictions. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Knowledge cutoff inconsistency in gpt-4o-2024-08-06: when jailbroken, the model appears [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Search results with date restriction showing an article with January 2024 publication date [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Additional examples of retrieval bias in search engines when using date restrictions. [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: Manifold emphasizes Personal use-cases over News and current events, whereas the latter [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Resolved questions from Manifold Markets by month, with colors representing the number [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: The gap between the knowledge cutoff and when the model is relevant is getting smaller. [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: ForecastBench obtains questions from multiple sources, but from each source, questions [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Global Merger-Arbitrage Forecasting with Language Models

    cs.CL 2026-07 conditional novelty 6.5 of 10

    Expert-context research agents plus hindsight-guided finetuning cut class-balanced Brier score on merger outcomes to 0.151, beating calibrated market prices, XGBoost, and frontier LLMs.

  2. Agentic Forecasting using Sequential Bayesian Updating of Linguistic Beliefs

    cs.AI 2026-04 unverdicted novelty 6.5 of 10

    BLF achieves state-of-the-art binary forecasting on ForecastBench by using linguistic belief states updated in tool-use loops, hierarchical multi-trial logit averaging, and hierarchical Platt scaling calibration.

  3. Advancing Event Forecasting through Massive Training of Large Language Models: Challenges, Solutions, and Broader Impacts

    cs.LG 2025-07 conditional novelty 5.0 of 10

    A position paper advocating large-scale training of event forecasting LLMs, with proposals for label selection, counterfactual training data, auxiliary rewards, and multi-source datasets.

Reference graph

Works this paper leans on

35 extracted references · 25 canonical work pages · cited by 3 Pith papers

  1. [1]

    Who predicted 2022?, 2023

    Scott Alexander. Who predicted 2022?, 2023. URL https://www.astralcodexten.com/p/who-predicted-2022. Accessed on 12-May-2025

    work page 2022

  2. [2]

    A backtesting protocol in the era of machine learning

    Robert D Arnott, Campbell R Harvey, and Harry Markowitz. A backtesting protocol in the era of machine learning. Available at SSRN 3275654, 2018

    work page 2018

  3. [3]

    The probability of backtest overfitting

    David H Bailey, Jonathan Borwein, Marcos Lopez de Prado, and Qiji Jim Zhu. The probability of backtest overfitting. Journal of Computational Finance (Risk Journals), 2015

    work page 2015

  4. [4]

    Contra papers claiming superhuman AI forecasting, 2024

    Nikos Bosse, Peter M \"u hlbacher, Lawrence Phillips, and Dan Schwarz. Contra papers claiming superhuman AI forecasting, 2024. URL https://www.lesswrong.com/posts/uGkRcHqatmPkvpGLq/contra-papers-claiming-superhuman-ai-forecasting

    work page 2024

  5. [5]

    Long-horizon predictability: a cautionary tale

    Jacob Boudoukh, Ronen Israel, and Matthew Richardson. Long-horizon predictability: a cautionary tale. Financial Analysts Journal, 2019

    work page 2019

  6. [6]

    AI forecasting bots incoming: comment section, 2024

    Gwern Branwen. AI forecasting bots incoming: comment section, 2024. URL https://www.lesswrong.com/posts/4kuXNhPf9FBwok7tK/ai-forecasting-bots-incoming?commentId=MirX9bPg232BuzMBq. Accessed 12 May 2025

    work page 2024

  7. [7]

    Point-in-time vs

    Ernest Breitschwerdt. Point-in-time vs. lagged fundamentals: This time i(t)'s different? Research report, S&P Capital IQ, Quantamental Research, 2015. URL https://www.spglobal.com/marketintelligence/en/documents/sp-capitaliq-quantamental-point-in-time-vs-lagged-fundamentals.pdf. Accessed: 2025-05-27

    work page 2015

  8. [8]

    Are LLMs prescient? A continuous evaluation using daily news as the oracle

    Hui Dai, Ryan Teehan, and Mengye Ren. Are LLMs prescient? A continuous evaluation using daily news as the oracle. In ICML, 2025. URL https://arxiv.org/abs/2411.08324

    arXiv 2025

Show all 35 references
  1. [9]

    Polymarket settles a market incorrectly -- again, 2024

    Chris Gerlacher. Polymarket settles a market incorrectly -- again, 2024. URL https://predictionnews.com/news/polymarket-settles-a-market-incorrectly-again. Accessed on 12-May-2025

    2024

  2. [10]

    Survivorship bias and mutual fund performance

    Elton M Gruber and C Blake. Survivorship bias and mutual fund performance. Review of Financial Sttidies, 9: 0 1097--1120, 1996

    1996

  3. [11]

    Knowledge cutoff issues of GPT -4o regarding Phan et al

    Danny Halawi. Knowledge cutoff issues of GPT -4o regarding Phan et al. (2024), 2024. URL https://x.com/dannyhalawi15/status/1833295067764953397

    2024

  4. [12]

    Approaching human-level forecasting with language models, 2024

    Danny Halawi, Fred Zhang, Chen Yueh-Han, and Jacob Steinhardt. Approaching human-level forecasting with language models, 2024

    2024

  5. [13]

    Introducing the SalemCSPi forecasting tournament, 2022

    Richard Hanania. Introducing the SalemCSPi forecasting tournament, 2022. URL https://www.cspicenter.com/p/introducing-the-salemcspi-forecasting. Accessed on 12-May-2025

    2022

  6. [14]

    The emerging science of machine learning benchmarks

    Moritz Hardt. The emerging science of machine learning benchmarks. Online at https://mlbenchmarks.org, 2025. Manuscript

    2025

  7. [15]

    Reasoning and tools for human-level forecasting

    Elvis Hsieh, Preston Fu, and Jonathan Chen. Reasoning and tools for human-level forecasting. arXiv preprint arXiv:2408.12036, 2024

    2024 arXiv

  8. [16]

    asgeirtj/system\_prompts\_leaks/claude.txt, 2025

    Asgeir Thor Johnson. asgeirtj/system\_prompts\_leaks/claude.txt, 2025. URL https://github.com/asgeirtj/system\_prompts\_leaks/blob/f7d92dec4a9a9f5e4d11e4384f8239fb7ca3be05/claude.txt. Accessed 12 May 2025

    2025

  9. [17]

    Ezra Karger, Houtan Bastani, Chen Yueh-Han, Zachary Jacobs, Danny Halawi, Fred Zhang, and Philip E. Tetlock. ForecastBench : A dynamic benchmark of AI forecasting capabilities, 2024 a . URL https://arxiv.org/abs/2409.19839

    2024 arXiv

  10. [18]

    Ezra Karger, Houtan Bastani, Chen Yueh-Han, Zachary Jacobs, Danny Halawi, Fred Zhang, and Philip E. Tetlock. forecastingresearch/forecastbench/, 2024 b . URL https://github.com/forecastingresearch/forecastbench/tree/8e236823e3683584330fc31bf41196a81ce28626/src/helpers

    2024

  11. [19]

    Questionable practices in machine learning

    Gavin Leech, Juan J Vazquez, Niclas Kupper, Misha Yagudin, and Laurence Aitchison. Questionable practices in machine learning. arXiv preprint arXiv:2407.12220, 2024

    2024 arXiv

  12. [20]

    Acx2025 tournament, 2025

    Metaculus. Acx2025 tournament, 2025. URL https://www.metaculus.com/tournament/ACX2025/. Accessed on 12-May-2025

    2025

  13. [21]

    Consistency checks for language model forecasters

    Daniel Paleka, Abhimanyu Pallavi Sudhir, Alejandro Alvarez, Vineeth Bhat, Adam Shen, Evan Wang, and Florian Tram \`e r. Consistency checks for language model forecasters. arXiv preprint arXiv:2412.18544, 2024

    2024 arXiv

  14. [22]

    LLMs are superhuman forecasters, 2024

    Long Phan, Adam Khoja, Mantas Mazeika, and Dan Hendrycks. LLMs are superhuman forecasters, 2024. URL https://drive.google.com/file/d/1Tc_xY1NM-US4mZ4OpzxrpTudyo1W4KsE. Center for AI Safety, UC Berkeley

    2024

  15. [23]

    White House planning face-to-face meeting with Biden , Xi , 2023

    Reuters. White House planning face-to-face meeting with Biden , Xi , 2023. URL https://www.reuters.com/world/white-house-planning-face-to-face-meeting-with-biden-xi-washington-post-2023-10-05/

    2023

  16. [24]

    Against calibration, 2023

    Eigil Fjeldgren Rischel. Against calibration, 2023. URL https://erischel.com/againstcalibration/. Accessed on 12-May-2025

    2023

  17. [25]

    Elicitation of personal probabilities and expectations

    Leonard J Savage. Elicitation of personal probabilities and expectations. Journal of the American Statistical Association, 66 0 (336): 0 783--801, 1971

    1971

  18. [26]

    Wisdom of the silicon crowd: LLM ensemble prediction capabilities rival human crowd accuracy

    Philipp Schoenegger, Indre Tuminauskaite, Peter S Park, Rafael Valdece Sousa Bastos, and Philip E Tetlock. Wisdom of the silicon crowd: LLM ensemble prediction capabilities rival human crowd accuracy. Science Advances, 10 0 (45): 0 eadp1528, 2024

    2024

  19. [27]

    Alignment problems with current forecasting platforms

    Nu \ n o Sempere and Alex Lawsen. Alignment problems with current forecasting platforms. arXiv preprint arXiv:2106.11248, 2021

    2021 arXiv

  20. [28]

    Capital asset prices: A theory of market equilibrium under conditions of risk

    William F Sharpe. Capital asset prices: A theory of market equilibrium under conditions of risk. The journal of finance, 19 0 (3): 0 425--442, 1964

    1964

  21. [29]

    Risk-adjusted performance of mutual funds

    Katerina Simons. Risk-adjusted performance of mutual funds. New England Economic Review, 9: 0 33--48, 1998

    1998

  22. [30]

    PROPHET : An inferable future forecasting benchmark with causal intervened likelihood estimation

    Zhengwei Tao, Zhi Jin, Bincheng Li, Xiaoying Bai, Haiyan Zhao, Chengfeng Dou, Xiancai Chen, Jia Li, Linyu Li, and Chongyang Tao. PROPHET : An inferable future forecasting benchmark with causal intervened likelihood estimation. arXiv preprint arXiv:2504.01509, 2025

    2025

  23. [31]

    Leonard J. Tashman. Out‐of‐sample tests of forecasting accuracy: An analysis and review. International Journal of Forecasting, 16 0 (4): 0 437--450, 2000. doi:10.1016/S0169-2070(00)00065-0

    doi

  24. [32]

    Biden , Xi talks in san francisco, 2023

    Japan Times. Biden , Xi talks in san francisco, 2023. URL https://www.japantimes.co.jp/news/2023/11/08/world/politics/biden-xi-talks-san-francisco/

    2023

  25. [33]

    Haooowang/llm-knowledge-cutoff-dates, 2025

    Hao Wang. Haooowang/llm-knowledge-cutoff-dates, 2025. URL https://github.com/HaoooWang/llm-knowledge-cutoff-dates/blob/f2ea76a47437c2787cc651838b5c7af4720d1c0a/README.md. Accessed 12 May 2025

    2025

  26. [34]

    Continual learning for large language models: A survey, 2024

    Tongtong Wu, Linhao Luo, Yuan-Fang Li, Shirui Pan, Thuy-Trang Vu, and Gholamreza Haffari. Continual learning for large language models: A survey, 2024. URL https://arxiv.org/abs/2402.01364

    2024 arXiv

  27. [35]

    Forecasting future world events with neural networks

    Andy Zou, Tristan Xiao, Ryan Jia, Joe Kwon, Mantas Mazeika, Richard Li, Dawn Song, Jacob Steinhardt, Owain Evans, and Dan Hendrycks. Forecasting future world events with neural networks. arXiv preprint arXiv:2206.15474, 2022

    2022 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.