Pith. sign in

REVIEW 3 major objections 5 minor 23 references

M6 Investment Challenge: The Role of Luck and Strategic Considerations

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The extreme Sharpe ratios atop the M6 investment challenge are statistically compatible with pure luck, and a rank-chasing portfolio can improve winning odds without any abnormal return.

desk verdict Statistical half is worth taking seriously; the strategic half is a useful cautionary toy, but its 'can improve your chances' claim is tested only against passive opponents. read the letter →

arxiv 2412.04490 v1 pith:FX7R4E4F submitted 2024-11-21 q-fin.PM

Filip Staněk This is my paper · ORCID
classification q-fin.PM MSC 91G1062F4062P05
keywords M6forecastingcompetitioninvestmentchallengeSharperatiorankmaximizationdynamicprogrammingmarketefficiencytournamentincentivesportfolioweights
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The M6 investment challenge asked 163 teams to submit monthly portfolios over 100 assets for a year. This paper claims that the extreme risk-adjusted returns (Sharpe ratios) at the top of its leaderboard are not statistical evidence of skill: after correcting a multivariate equality-of-Sharpe-ratios test for the large number of teams and the short one-year window, the observed spread carries a p-value of 0.926, meaning it is compatible with all teams having the same expected Sharpe ratio. The paper then claims that the contest's reward structure makes the goal of finishing in the top ranks different from the goal of maximizing expected return. In a calibrated model, a team that chooses portfolio weights to maximize the probability of a target rank, shifting between short and long positions based on its current standing, can achieve the same winning probability as a team with nearly double the market's expected return, while accepting a negative expected return. Submitted portfolio weights from M6 line up with this prediction: teams with below-median shares of long positions were roughly ten times more likely to reach a top-10 rank.

What carries the argument

The argument runs on a calibrated null model and a dynamic optimization. The null model assumes asset returns are jointly normal, temporally independent, and homogeneous, and that ordinary teams pick portfolios at random from a menu of long, zero, and short positions whose proportions are fitted to the M6 leaderboard via simulated moments; this model reproduces the observed mean, dispersion, and tails of team scores. The equality-of-Sharpe-ratios test is a multivariate test of the hypothesis that all teams have the same expected risk-adjusted return, and it is made usable with 163 teams over one year by replacing its asymptotic critical values with critical values from a null-imposed wild bootstrap. The rank-chasing strategy is the solution of a dynamic program whose value function is the probability of finishing at or above the target rank. Its state is the gap $\Delta_m$ between the player's cumulative score and the score of the competitor currently at the target rank, and its control is the share of long positions $\beta^+$; backward induction yields a policy that starts short-heavy to create dispersion from the long-heavy field and switches to mimicking the field when a good rank is within reach.

What would settle it

Run the same bootstrap-corrected equality-of-Sharpe-ratios test on a fresh M6-style contest with 163 teams over twelve months; a p-value well below 0.05 would reject the pure-luck explanation for the original leaderboard. Alternatively, simulate the contest with several teams using the rank-optimization policy at once: if no optimizer beats the passive benchmark, the single-optimizer assumption is the load-bearing part.

Watch

Extended reading notes

Core claim

The paper's central claim is that the M6 investment challenge does not demonstrate that anyone can consistently beat the market, and that contest rank, not expected return, is the objective a rational competitor should optimize. A bootstrap-corrected test of the hypothesis that all teams have equal expected Sharpe ratios returns a p-value of 0.926, so the leaderboard's tail performance is what chance alone would produce when 163 teams are observed for one year. The paper then derives a dynamic policy that varies the share of short positions according to the gap between the player's cumulative score and the score of the competitor currently at the target rank. In both a calibrated simulation and a bootstrap environment built from the actual M6 returns and submitted weights, this rank-optimizing portfolio wins the top rank more often than a passive long-heavy portfolio even though its expected return is negative. The paper concludes that the task of winning the investment challenge is not identical to the task of earning the best investment returns, and that contest design can reward correlation-breaking risk-taking even in the absence of forecasting ability.

Load-bearing premise

The rank-chasing result assumes that 162 teams passively keep the same long-heavy random portfolio while only one team optimizes for rank; if many teams chased rank at the same time, the edge could shrink or disappear, and that simultaneous-adoption case is not analyzed.

Editorial extensions

If this is right

  • The top Sharpe ratios in M6 are compatible with the null model that every team has the same expected risk-adjusted return, so the leaderboard by itself is not evidence against market efficiency.
  • A rank-optimizing team can attain the same probability of first place as a team that consistently earns almost double the market return, while holding a portfolio with negative expected returns.
  • The strategic advantage comes from making one's returns less correlated with the field's long-heavy portfolios, which raises the probability of both a top rank and a bottom rank.
  • Empirically, M6 teams with below-median shares of long positions were about ten times as likely to reach a top-10 rank, and only one of the 25 top-5 finishers used an above-median share of long positions.
  • Because the strategy relies only on public leaderboard information and an estimate of competitors' long bias, it was feasible for teams to implement during the actual contest.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same logic plausibly extends to other winner-take-all contests where competitors can choose risk that is uncorrelated with the field, such as forecasting competitions scored by relative rank; organizers should expect variance-seeking submissions near the top.
  • A testable prediction of the paper's mechanism is that the rank-chasing premium will shrink once the strategy becomes widely known, as teams shift toward lower long shares and the field becomes less uniformly long-heavy.
  • Contest design could be changed to dampen the incentive: for instance, by scoring multiple intervals jointly, penalizing portfolio variance, or rewarding consistency rather than final rank, organizers could reduce the payoff to extreme correlation-breaking bets.
  • The paper focuses on the possibility of gaming the contest; it does not establish whether any individual M6 winner actually used such a strategy, so the empirical weight patterns should be read as indirect rather than direct evidence of intent.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies the M6 investment challenge. It first tests whether the cross-sectional dispersion of Sharpe ratios (IRs) is compatible with equal expected Sharpe ratios using the Wright-Yam-Yung (WYY) test with bootstrap critical values; the reported p-value is 0.926, so the null is not rejected. A stylized Gaussian model with ternary random portfolios is calibrated to M6 leaderboard moments and used to assess the test's size and to compare tail behavior. Second, the paper formulates the problem of maximizing the probability of a top rank as a dynamic program over the proportion of long positions, solves it under the stylized model, and evaluates the resulting 'rank optimization' strategy in simulations and in a bootstrap environment built from actual M6 submissions. The strategy raises the probability of a top rank despite a lower expected IR. Empirical analysis of submitted weights shows an inverted-U relation between average long proportion and attained rank, and teams with below-median long proportions are more likely to reach top ranks.

Significance. If the results hold, the paper makes a useful contribution to the interpretation of forecasting competitions: in a large one-year field, extreme Sharpe ratios can arise by chance, and the unadjusted WYY test can severely over-reject unless bootstrap critical values are used. The second part provides a clean demonstration that rank-based objectives can induce portfolio choices very different from mean-variance optimization, with implications for competition design. The paper's strengths include a replication repository, explicit size simulations for the bootstrap test, and evaluation of the strategic policy in both a parametric model and a bootstrap environment based on actual submissions. These make the central claims checkable. The main limitations are that the strategic result is single-agent, the dynamic program is solved under approximations whose accuracy is not established, and the finite-sample validity of the bootstrap is checked under one calibrated model; these issues affect the generality of the headline claims but are addressable.

major comments (3)
  1. [Section 4.2, Tables 5 and 6; abstract] The claim that a team can improve its chances of winning by rank-optimizing portfolio weights is established only against a fixed distribution of passive opponents. The value function in Eq. (27) takes the opponents' IR distribution as exogenous, and the mechanism in Figure 1 works by making the focal team's returns negatively correlated with the opponents' average long bias. If a substantial fraction of teams adopted similar rank-aware policies, the correlation structure and hence the value function would change, and the reported advantage could shrink or disappear. No equilibrium or multi-agent analysis is provided. I ask for a robustness experiment in which the fraction of rank-optimizing opponents is varied, or, failing that, a clear statement in the abstract and conclusions that this is a unilateral-deviation result rather than a general tournament-equilibrium result. This is load-bearing for the paper's second main claim.
  2. [Section 4.1, Eqs. (27)-(28) and footnote 10] The dynamic program is solved under two approximations: the state is reduced to the scalar gap Delta_m to the q-th ranked competitor, and returns are made additive by per-submission standardization. The state reduction is not justified in the text. The distribution of the future q-th order statistic of the opponents' cumulative IRs, and hence the probability of finishing in the top q, depends on the entire vector of opponents' cumulative IRs, not only on the current q-th largest value. Consequently, the policy obtained is not necessarily the optimal strategy for problem (26), and the paper's repeated use of 'optimal' overstates what is derived. Please either demonstrate that Delta_m is a sufficient statistic (or bound the approximation error for the quantities reported in Tables 5 and 6), or relabel the policy as a heuristic and adjust the claims accordingly.
  3. [Section 3.2, Eq. (20) and Table 2b] The bootstrap critical values are generated by multiplying each team's submitted weights by independent Rademacher variables, which imposes zero cross-sectional correlation across teams in the bootstrap samples. The size simulations in Table 2b are conducted under the same stylized model A1-A2 that was calibrated to the M6 leaderboard, so they provide only one check of finite-sample validity. Because the p-value of 0.926 in Table 3 is the paper's main statistical evidence for the luck conclusion, I would like to see a robustness check under a return-generating process with stronger cross-sectional dependence (e.g., a one-factor model with heterogeneous loadings) or a bootstrap that preserves cross-sectional dependence by multiplying common time-series factors rather than individual team weights. The current result may be correct, but the evidence for finite-sample validity is narrower than the conclusion requires.
minor comments (5)
  1. [Section 3.1, Table 1] The simulated global mean IR of 0.50 differs from the observed -3.06, and the statement that the simulated statistics 'align remarkably well' should be qualified; the global level is not matched, only the monthly pattern and dispersion.
  2. [Section 3.2] Please report the effective number of teams K used in the WYY test after aggregating the 14 identical dummy submissions; the text mentions the aggregation but not the resulting K.
  3. [Section 4.2, Table 6] In the bootstrap evaluation, state explicitly that the rank-optimization policy is the one derived under A1-A2 and is not re-optimized on the bootstrap distribution, and discuss whether re-optimization would change the results.
  4. [Figure 6 and Table 7] The empirical relation between below-median beta+ and top ranks is observational; the text should more explicitly say that it is consistent with the strategic mechanism but does not establish that teams deliberately used the derived policy.
  5. [Eqs. (29)-(31)] The construction of the strategic portfolio would be clearer if the counts of long and short positions were written as functions such as round(beta+ I) and round(beta- I), rather than the phrase 'beta+(Delta_m,m) * I times'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's central claims rest on bootstrap inference and on empirical checks that are not built into the model inputs.

full rationale

The derivation chain is self-contained in the relevant sense. The headline p-value of 0.926 for the WYY test is computed from the actual submitted portfolio weights with a null-imposed wild bootstrap, not from parameters fitted inside the paper's stylized model; the stylized model is used only to check the finite-sample size of the bootstrap procedure under the null. The A2 baseline portfolio is explicitly calibrated to leaderboard moments via method of simulated moments, and Table 1 is presented as calibration/verification rather than as an out-of-sample prediction, so the later statement that extreme tail outcomes are possible under the null is not a fitted input disguised as a finding. The strategic rank-optimization result is derived in a single-agent dynamic program against fixed baseline opponents, but its empirical confirmation in Table 7 and Figure 6 uses actual submitted short/long positions and actual ranks, which were not inputs to the optimization; the bootstrap environment in Table 6 further validates the strategy using resampled real submissions. The paper also explicitly disclaims causal claims about individual teams, which is a limitation but not a circularity. No load-bearing step reduces, by construction or by self-citation, to its own inputs.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central statistical claim depends mostly on standard asymptotic theory and a data-based bootstrap. The stylized model used for simulations and for the strategic optimization depends on parameters fitted to M6 data, which raises the circularity burden. No new physical or economic entities are introduced.

free parameters (4)
  • baseline portfolio counts (n_+, n_0, n_-) = 38, 29, 33
    Estimated by method of simulated moments to match the mean and kurtosis of observed M6 IR distributions (Appendix A.2). Used to generate baseline random portfolios in A2.
  • daily return mean mu_r = 0.00037
    Long-term average yearly S&P 500 nominal return converted to daily; taken from Webster (2023), not fitted to M6 target results, but used in A1 simulations.
  • return covariance parameters sigma_rr, sigma_rr' = 0.00038, 0.00013
    MLE estimates from M6-period returns under compound symmetry (Appendix A.1). Used in A1 and in the tangency portfolio construction.
  • predictability degree lambda = 0, 0.0001, 0.0003, 0.001
    Chosen scenario values for the tangency portfolio in Section 3.3; not fitted, used to illustrate how predictability translates into rank probabilities.
assumptions (5)
  • domain assumption A1: asset returns are jointly normal, independent over time, homogeneous, with compound symmetry covariance.
    Invoked throughout Sections 3 and 4; ignores fat tails, volatility clustering, and cross-asset heterogeneity.
  • domain assumption A2: baseline teams choose portfolios by random sampling of +1/0/-1 positions with counts n_+, n_0, n_-.
    Used to simulate the competition and to derive the optimal rank strategy; calibrated to M6 via simulated moments.
  • domain assumption A1': returns decompose into predictable and unpredictable IID normal components with parameter lambda.
    Extension of A1 in Section 3.3 to model a team with genuine predictability.
  • standard math WYY test statistic T^2 converges to chi-square with K-1 degrees of freedom under H0.
    Taken from Wright et al. (2014); the paper corrects its finite-sample level via bootstrap critical values.
  • standard math The null-imposed wild bootstrap (Rademacher sign flips on portfolio weights) yields a valid null distribution for the test statistic.
    Relies on theory in Wright et al. (2014) and Ledoit and Wolf (2008); validated by simulation only under A1-A2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of M6 Investment Challenge: The Role of Luck and Strategic Considerations." pith.science (2026). https://pith.science/paper/FX7R4E4F

@misc{pith2026241204490,
  author       = {Pith},
  title        = {Pith review of: M6 Investment Challenge: The Role of Luck and Strategic Considerations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FX7R4E4F}},
  note         = {Machine review of arXiv:2412.04490}
}
read the original abstract

This article investigates the influence of luck and strategic considerations on performance of teams participating in the M6 investment challenge. We find that there is insufficient evidence to suggest that the extreme Sharpe ratios observed are beyond what one would expect by chance, given the number of teams, and thus not necessarily indicative of the possibility of consistently attaining abnormal returns. Furthermore, we introduce a stylized model of the competition to derive and analyze a portfolio strategy optimized for attaining the top rank. The results demonstrate that the task of achieving the top rank is not necessarily identical to that of attaining the best investment returns in expectation. It is possible to improve one's chances of winning, even without the ability to attain abnormal returns, by choosing portfolio weights adversarially based on the current competition ranking. Empirical analysis of submitted portfolio weights aligns with this finding.

Figures

Figures reproduced from arXiv: 2412.04490 by the authors.

Figure 1
Figure 1. displays the optimal portfolio for q = 1 as a function of m and ∆m computed under A1-A2.10 The 9We consider only positions −0.01 and 0.01 as the presence of zero positions seems to alter the distribution of IRTm,k, and even more im￾portantly, the distribution of IRTm,k relative to IRTm,k ′ of the baseline portfolio, only very little, due to the normalization in Eq. 4 (see Fig. B.7 in Appendix B). 10For the purpose o… view at source ↗
Figure 2
Figure 2. Histogram of Ranks (Simulated) [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Histogram of Ranks (Bootstrapped) In both [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Average Proportion of Long Positions β + m,k [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Absolute Change of Rank as Function of β + m,k The banded lines represent loess estimates of mean with their corresponding standard deviations. Considering Figures 4 and 5 and the simulation re￾sults in the theoretical part of this section, one would expect both the be…
Figure 6
Figure 6. Figure 6: β¯+ ·,k as Function of Attained Rank The banded lines represent loess estimates of mean with their corresponding standard deviations. year, these finite sample distortions can be mitigated by replacing the critical values with bootstrap counterparts. Applying the WYY t…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 20 canonical work pages

  1. [1]

    , author Harlow, W.V

    author Brown, K.C. , author Harlow, W.V. , author Starks, L.T. , year 1996 . title Of Tournaments and Temptations : An Analysis of Managerial Incentives in the Mutual Fund Industry . journal The Journal of Finance volume 51 , pages 85--110 . :10.2307/2329303, http://arxiv.org/abs/2329303 arXiv:2329303

    work page arXiv 1996

  2. [2]

    , author Lopez, J.A

    author Diebold, F.X. , author Lopez, J.A. , year 1995 . title Modeling Volatility Dynamics , in: editor Hoover, K.D. (Ed.), booktitle Macroeconometrics: Developments , Tensions , and Prospects . publisher Springer Netherlands , address Dordrecht . Recent Economic Thought Series , pp. pages 427--472 . :10.1007/978-94-011-0669-6_11

    work page doi:10.1007/978-94-011-0669-6_11 1995

  3. [3]

    , author Holmen, M

    author Dijk, O. , author Holmen, M. , author Kirchler, M. , year 2014 . title Rank matters-- The impact of social competition on portfolio choice . journal European Economic Review volume 66 , pages 97--110 . :10.1016/j.euroecorev.2013.11.010

    work page doi:10.1016/j.euroecorev.2013.11.010 2014

  4. [4]

    , author Gruber, M.J

    author Elton, E.J. , author Gruber, M.J. , author Blake, C.R. , year 2003 . title Incentive Fees and Mutual Funds . journal The Journal of Finance volume 58 , pages 779--804 . :10.1111/1540-6261.00545

    work page arXiv 2003

  5. [5]

    , author Korkie, B.M

    author Jobson, J.D. , author Korkie, B.M. , year 1981 . title Performance Hypothesis Testing with the Sharpe and Treynor Measures . journal The Journal of Finance volume 36 , pages 889--908 . :10.2307/2327554, http://arxiv.org/abs/2327554 arXiv:2327554

    work page arXiv 1981

  6. [6]

    , year 2016

    author Kourtis, A. , year 2016 . title The Sharpe ratio of estimated efficient portfolios . journal Finance Research Letters volume 17 , pages 72--78 . :10.1016/j.frl.2016.01.009

    work page doi:10.1016/j.frl.2016.01.009 2016

  7. [7]

    , year 2011

    author Krasny, Y. , year 2011 . title Asset Pricing with Status Risk . journal The Quarterly Journal of Finance volume 01 , pages 495--549 . :10.1142/S2010139211000134

    work page doi:10.1142/s2010139211000134 2011

  8. [8]

    , author Wolf, M

    author Ledoit, O. , author Wolf, M. , year 2008 . title Robust performance hypothesis testing with the Sharpe ratio . journal Journal of Empirical Finance volume 15 , pages 850--859 . :10.1016/j.jempfin.2008.03.002

    doi:10.1016/j.jempfin.2008.03.002 2008

Show all 23 references
  1. [9]

    , author Wong, W.K

    author Leung, P.l. , author Wong, W.K. , year 2008 . title On Testing the Equality of the Multiple Sharpe Ratios , with Application on the Evaluation of Ishares . journal Journal of Risk volume 10 , pages 15--31 . :10.2139/ssrn.907270

    2008 doi

  2. [10]

    , year 2011

    author Lin, J. , year 2011 . title Fund convexity and tail risk-taking . journal Unpublished working paper

    2011

  3. [11]

    , author Gaba, A

    author Makridakis, S. , author Gaba, A. , author Hollyman, R. , author Petropoulos, F. , author Spiliotis, E. , author Swanson, N. , year 2022 . title The M6 Financial Duathlon Competition Guidelines

    2022

  4. [12]

    , author Spiliotis, E

    author Makridakis, S. , author Spiliotis, E. , author Hollyman, R. , author Petropoulos, F. , author Swanson, N. , author Gaba, A. , year 2023 . title The M6 forecasting competition: Bridging the gap between forecasting and investment decisions . :10.48550/arXiv.2310.13357, ht...

    2023 doi arXiv

  5. [13]

    , year 2005

    author Malkiel, B.G. , year 2005 . title Reflections on the Efficient Market Hypothesis : 30 Years Later . journal Financial Review volume 40 , pages 1--9 . :10.1111/j.0732-8516.2005.00090.x

    2005

  6. [14]

    , year 1989

    author McFadden, D. , year 1989 . title A Method of Simulated Moments for Estimation of Discrete Response Models Without Numerical Integration . journal Econometrica volume 57 , pages 995--1026 . :10.2307/1913621, http://arxiv.org/abs/1913621 arXiv:1913621

    1989

  7. [15]

    , year 2003

    author Memmel, C. , year 2003 . title Performance Hypothesis Testing with the Sharpe Ratio

    2003

  8. [16]

    , year 2023

    author Michaus, M.P. , year 2023 . title FinQBoost : Machine Learning for Portfolio Forecasting . howpublished https://miguelpmich.medium.com/finqboost-machine-learning-for-portfolio-forecasting-55e62b00ebca

    2023

  9. [17]

    , author Sliwka, D

    author Nieken, P. , author Sliwka, D. , year 2010 . title Risk-taking tournaments -- Theory and experimental evidence . journal Journal of Economic Psychology volume 31 , pages 254--268 . :10.1016/j.joep.2009.03.009

    2010 doi

  10. [18]

    , author Romano, J.P

    author Politis, D.N. , author Romano, J.P. , author Wolf, M. , year 1999 . title Subsampling . edition 1999th edition ed., publisher Springer , address New York Heidelberg

    1999

  11. [19]

    , year 1984

    author Seber, G.A.F. , year 1984 . title Multivariate Observations . edition 1st edition ed., publisher Wiley , address New York

    1984

  12. [20]

    title Winning 6000\ with a few lines of code

    author Shrish , year 2023 . title Winning 6000\ with a few lines of code

    2023

  13. [21]

    , year 2023

    author Webster, I. , year 2023 . title S& P 500 Returns since 1930 . howpublished https://www.officialdata.org/us/stocks/s-p-500/1930

    2023

  14. [22]

    , author Yam, S.C.P

    author Wright, J. , author Yam, S.C.P. , author Pang Yung, S. , year 2014 . title A test for the equality of multiple Sharpe ratios . journal Journal of Risk volume 16

    2014

  15. [23]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.