Pith. sign in

REVIEW 5 major objections 6 minor 2 cited by

LLM-Based Routing in Mixture of Experts: A Novel Framework for Trading

T0 review · 5 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LLMoE replaces the learned router with an LLM, claiming gains of 25%+ in risk-adjusted trading returns.

desk verdict Promising idea, unsupported central claim: no ablation isolates the LLM router's contribution, and closest baselines are missing. read the letter →

arxiv 2501.09636 v2 pith:JAGM3N3P submitted 2025-01-16 cs.LG q-fin.TR

Kuan-Ming Liu , Ming-Chih Lo This is my paper · ORCID
classification cs.LGq-fin.TR
keywords mixtureofexpertsLLMroutingstockmovementpredictionmultimodaltradingall-inall-outrisk-adjustedreturns
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes LLMoE, a mixture-of-experts architecture in which a large language model, rather than a trained neural router, decides each day which of two specialist predictors—one for optimistic periods and one for pessimistic periods—should forecast the next day's price direction. It argues that the LLM's ability to read both engineered price features and news headlines gives it a global, interpretable view that static routers lack. On ten-year MSFT and AAPL backtests using an all-in/all-out trading rule, LLMoE reports the highest total return, Sharpe ratio, and Calmar ratio among all compared models, including conventional two- and ten-expert MoE baselines, with gains over 25% in key risk-adjusted metrics. A sympathetic reader would take the paper's intended contribution as evidence that LLM-driven routing is a viable and flexible replacement for learned routers in financial MoE systems.

What carries the argument

The load-bearing mechanism is the LLM router: a pretrained large language model turns each five-day window into a descriptive string of numeric features and news text, then classifies it as Optimistic or Pessimistic. The label routes the window to one of two feedforward networks, and the choice is made by the argmax of the LLM's predicted probabilities, so no gating network is trained. The LLM's natural-language reasoning is produced as a byproduct for interpretability, while the all-in/all-out trading rule converts the chosen expert's binary prediction into a position.

What would settle it

Train the same two experts but route with a constant 'optimistic' label, or with a simple news-sentiment rule, and compare returns to LLMoE; if either matches the reported Sharpe and total return, the LLM router is not the cause. Also recompute the all-in/all-out backtest with realistic transaction costs and slippage: if the 31–65% total returns collapse to baseline levels, the trading rule, not the routing, explains the result.

Watch

Extended reading notes

Core claim

LLMoE's central claim is that expert selection in a trading MoE should be performed by a large language model that reads a rolling five-day window of engineered price features plus the corresponding news headline, and emits an Optimistic or Pessimistic label with a natural-language explanation. Two feedforward experts are trained separately on the windows labeled by the router, and the selected expert's next-day direction prediction drives a binary all-in/all-out position. On MSFT and AAPL datasets spanning 2006–2016, the paper reports that LLMoE achieves the best total return (65.44% and 31.43%), Sharpe ratio (2.14 and 1.17), and Calmar ratio among every baseline, exceeding the best static MoE by more than 25% on the headline risk-adjusted metrics. The intended discovery is that a pretrained LLM router with world knowledge yields better and more interpretable expert allocation than a neural router trained only on numerical data.

Load-bearing premise

The performance gain is credited to the LLM router, but the paper never validates the router's Optimistic/Pessimistic labels or compares it against a trivial router, so the improvement could come from the expert architecture or the data split rather than from the routing mechanism.

Editorial extensions

If this is right

  • Routing decisions can be made without training a gating network, potentially avoiding router collapse in data-scarce financial settings.
  • An MoE can exploit textual news directly through the router's context window, removing the need for a separate sentiment-analysis module.
  • The reported reduction in maximum drawdown on MSFT, from 17.82% for the two-expert MoE to 11.32% for LLMoE, implies improved downside control in the tested period.
  • The router's natural-language reasoning gives a human-readable audit trail for each trade, which static neural routers do not provide.
  • The same router-plus-experts structure can be repurposed for other prediction tasks by changing the prompt and retraining the expert heads.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial: The paper does not ablate the router, so the cleanest test of its central claim is to freeze the two experts and swap the LLM router for an always-optimistic label or a simple lexicon sentiment rule; if either matches LLMoE's returns, the LLM's reasoning is not the source of the gain.
  • Editorial: Because the backtest assumes zero transaction costs and full liquidation, the reported 31–65% total returns are upper bounds; a cost-aware replay of the same all-in/all-out signals would show how much of the edge survives real trading frictions.
  • Editorial: If the routing labels prove reliable, they could be distilled into a small classifier to make the system cheaper at inference time, or reused as a regime indicator for risk management.
  • Editorial: The 80/20 chronological split may place the test mostly in a single market regime; extending the evaluation across multiple bull and bear periods would separate routing skill from a favorable upswing.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes LLMoE, a mixture-of-experts framework in which a pretrained LLM (Llama3.2) serves as a router. The router classifies a five-day rolling window of 55 numerical features plus news headlines into Optimistic or Pessimistic, and the corresponding label selects one of two feedforward expert networks trained on those labels. The selected expert's binary prediction drives an All-in All-out trading strategy. The authors evaluate on MSFT and AAPL over 2006-2016, with a test period of roughly 2014-2016, comparing against LightGBM, MLP, LSTM, DNNE, and MoE with 2 and 10 experts. They report that LLMoE achieves the highest Total Return, Sharpe Ratio, and Calmar Ratio on both datasets, with over 25% improvements claimed in the conclusion.

Significance. The idea of replacing a learned neural router with an LLM router to combine numerical and textual data is timely and, if properly validated, would be a useful contribution to the growing literature on LLM-based trading and interpretable MoE. The paper gives a concrete architecture and reports positive results on two real-world datasets, and the explicit reasoning output of the router is a promising step toward interpretability. However, the evaluation does not isolate the router's contribution, omits the closest published baselines, uses a cost-free All-in All-out backtest, and contains clear signs of incompleteness. These issues prevent the current manuscript from substantiating its state-of-the-art claim.

major comments (5)
  1. [Experiment Results, Table 1] The central claim that LLMoE outperforms baselines because of LLM-based routing is not supported by any ablation isolating the router. The router's Optimistic/Pessimistic labels are never validated: there is no accuracy of the labels against realized next-day direction, no comparison with an always-optimistic router, random labels, or a simple sentiment-lexicon rule, and no variant that keeps the two-expert architecture but replaces the LLM router with a static or random router. Without such an ablation, the large gains in Table 1 (e.g., MSFT TR 65.44 vs. 33.92 and SR 2.14 vs. 1.21) cannot be attributed to the LLM router; they could equally arise from the two-expert split, the particular test window, or the all-in-all-out execution.
  2. [Baseline Models and Appendix] The paper explicitly cites AlphaMix, TradExpert, and MIGA as related state-of-the-art MoE works, yet none of these is included in Table 1. Since the conclusion claims superiority over 'state-of-the-art MoE models,' the comparison set (LightGBM, MLP, LSTM, DNNE, MoE-2, MoE-10) is insufficient to support that claim. The absence of AlphaMix is especially consequential because the Related Work describes it as the closest competitor and the manuscript's own MoE baselines are simple static routers without the three-stage design discussed for AlphaMix.
  3. [Trading Algorithm Generation and Experiment Results] The All-in All-out backtest assumes zero transaction costs, slippage, and market impact, and the paper does not report turnover or a cost-sensitivity analysis. For a strategy that moves between fully invested and fully liquidated on daily predictions (TR 65.44% on MSFT and 31.43% on AAPL over roughly two years), realistic costs can materially reduce or erase the reported return advantage. Because the headline result is a return comparison, this is a load-bearing limitation rather than a minor execution detail.
  4. [Other Experimental Details] The paper states that experiments were repeated with ten random seeds and confidence intervals were computed, but no statistical significance test is reported. The phrase 'significantly outperformed' in the results section is not backed by a pairwise test, a bootstrap on daily returns, or any multiple-comparison correction. Given that the paper's main claim is outperformance, a significance test is necessary to interpret the reported mean and standard-deviation intervals.
  5. [Expert Prediction and Appendix] The manuscript appears incomplete. The text 'Temporal Graph Networks for Graph Anomaly Detection in Financial Networks' is inserted in the middle of the Expert Prediction subsection, and several details promised in the appendix ('Appendix - Experiment Setup Details', 'Appendix - Expert Model Architecture') are either missing or under-specified, including baseline hyperparameters, the exact LLM prompt and decoding settings, and the handling of days with missing news. These omissions prevent reproduction and further undermine the completeness needed for a state-of-the-art claim.
minor comments (6)
  1. [Experiment Setup] Table 1 and the surrounding text contain a typo in the metric name: 'V olatility' should be 'Volatility'.
  2. [References] The reference list contains duplicate entries for Yoo et al. (2021a and 2021b), which appear to describe the same paper.
  3. [Related Work] The related-work section refers to AlphaMix by name, but the citation is given only as Sun et al. 2023; the text should make the AlphaMix attribution explicit.
  4. [Table 3] The feature table defines zd5 explicitly, but the formulas for zd10, zd15, zd20, zd25, and zd30 are only implied; a generic formula for zd_n would be clearer.
  5. [Implementation and Evaluation, Router] The router prompt and decoding settings are not described; without these, the router component is not reproducible.
  6. [Appendix, Dataset Splits] The training period is given as 2006-12-07 to 2014-12-02 in the text, while Table 2 lists the start date as 06/12/01; these dates should be reconciled.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found: LLMoE is an empirical architecture comparison, not a derivation that reduces to its inputs.

full rationale

The paper makes no first-principles derivation or theoretical claim whose conclusion is equivalent to its assumptions. The LLM router assigns an Optimistic or Pessimistic label to each five-day window, and two feedforward experts are trained on the instances separated by those labels; at test time the router selects one expert and the expert's sigmoid output drives an all-in-all-out trading rule. This is a standard pseudo-labeling or self-training style pipeline. The router labels are intermediate variables, not fitted parameters, and the final predictions come from the selected expert rather than from the router label itself. The central claim is an empirical comparison against external baselines (MLP, LSTM, LightGBM, DNNE, and static MoE) on held-out test windows, so the reported improvements are not forced by construction. No self-citation is load-bearing: the references are prior external works, and no uniqueness theorem or prior result by the same authors is invoked to rule out alternatives. The absence of an ablation isolating the LLM router's contribution, and the lack of router-label validation, are experimental validity concerns, but they do not make the reported comparison circular. The unrealistic zero-transaction-cost assumption likewise affects the backtest realism, not the logical derivation. Therefore no circularity step meets the evidentiary bar, and the score is 0.

Assumptions & free parameters 1 free parameters · 2 assumptions · 0 invented entities

The framework relies on two main untested premises: the reliability of the LLM as a zero-shot financial sentiment router, and the costless execution assumption. The prompt is an unreported hand-chosen configuration, so it counts as a free parameter.

free parameters (1)
  • LLM router prompt and decoding settings = unspecified
    The prompt template, model size, temperature, and token sampling for Llama3.2 are not reported; these choices determine the router labels and are not tuned or ablated.
assumptions (2)
  • domain assumption The LLM router's zero-shot classifications are reliable enough to split training data and route test instances.
    The paper uses Llama3.2 without fine-tuning or calibration and does not measure routing accuracy against ground truth sentiment.
  • domain assumption Trading costs are zero: the All-in All-out strategy can be executed without transaction costs, slippage, or market impact.
    The backtest assumes perfect execution, which is unrealistic for daily trading of MSFT and AAPL.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LLM-Based Routing in Mixture of Experts: A Novel Framework for Trading." pith.science (2026). https://pith.science/paper/JAGM3N3P

@misc{pith2026250109636,
  author       = {Pith},
  title        = {Pith review of: LLM-Based Routing in Mixture of Experts: A Novel Framework for Trading},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JAGM3N3P}},
  note         = {Machine review of arXiv:2501.09636}
}
read the original abstract

Recent advances in deep learning and large language models (LLMs) have facilitated the deployment of the mixture-of-experts (MoE) mechanism in the stock investment domain. While these models have demonstrated promising trading performance, they are often unimodal, neglecting the wealth of information available in other modalities, such as textual data. Moreover, the traditional neural network-based router selection mechanism fails to consider contextual and real-world nuances, resulting in suboptimal expert selection. To address these limitations, we propose LLMoE, a novel framework that employs LLMs as the router within the MoE architecture. Specifically, we replace the conventional neural network-based router with LLMs, leveraging their extensive world knowledge and reasoning capabilities to select experts based on historical price data and stock news. This approach provides a more effective and interpretable selection mechanism. Our experiments on multimodal real-world stock datasets demonstrate that LLMoE outperforms state-of-the-art MoE models and other deep neural network approaches. Additionally, the flexible architecture of LLMoE allows for easy adaptation to various downstream tasks.

Figures

Figures reproduced from arXiv: 2501.09636 by the authors.

Figure 1
Figure 1. An illustration comparing traditional single-model [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the LLMoE framework, illustrating [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. iLENS: Interpretable LLM-Guided Mixture-of-Experts for Neuroimaging Survival Analysis

    cs.LG 2026-06 conditional novelty 5.0 of 10

    LLM-guided mixture-of-experts yields competitive C-index and stronger LogRank subtype separation for MCI-to-AD conversion on ADNI neuroimaging plus clinical notes.

  2. Integrating Large Language Models in Financial Investments and Market Analysis: A Survey

    q-fin.GN 2025-06 conditional novelty 1.0 of 10

    A survey that organizes recent LLM-in-finance research into four framework categories and summarizes the reported methods, datasets, and performance of about 30 systems.

Reference graph

Works this paper leans on

16 extracted references · 11 canonical work pages · cited by 2 Pith papers

  1. [1]

    Botunac, I.; Bosna, J.; and Matetic, M. 2024. Optimization of Traditional Stock Market Strategies Using the LSTM Hybrid Approach. Inf., 15(3): 136

    work page 2024

  2. [2]

    Ding, Q.; Shi, H.; and Liu, B. 2024. TradExpert: Revolutionizing Trading with Mixture of Expert LLMs. arXiv preprint arXiv:2411.00782

    arXiv 2024

  3. [3]

    Hu, Z.; Liu, W.; Bian, J.; Liu, X.; and Liu, T.-Y. 2018. A Deep Learning Framework for News-oriented Stock Trend Prediction. Proceedings of the 11th ACM International Conference on Web Search and Data Mining, 297--305

    work page 2018

  4. [4]

    Kato, T. 2015. VWAP execution as an optimal strategy. JSIAM Lett. , 7: 33--36

    work page 2015

  5. [5]

    Kou, Z.; Yu, H.; Peng, J.; and Chen, L. 2024. Automate Strategy Finding with LLM in Quant investment. CoRR, abs/2409.06289

    arXiv 2024

  6. [6]

    Li, K.; and Xu, J. 2023. An Attention-Based Multi-Gate Mixture-of-Experts Model for Quantitative Stock Selection. International Journal of Trade, Economics and Finance, 14(3): 165--173

    work page 2023

  7. [7]

    Sawhney, R.; Agarwal, S.; Wadhwa, A.; and Shah, R. R. 2020. Deep Attentive Learning for Stock Movement Prediction From Social Media Text and Company Correlations. 8415--8426

    work page 2020

  8. [8]

    Sun, S.; Wang, X.; Xue, W.; Lou, X.; and An, B. 2023. Mastering Stock Markets with Efficient Mixture of Diversified Trading Experts. Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2109--2119

    work page 2023

Show all 16 references
  1. [9]

    Toner, W.; and Darlow, L. N. 2024. An Analysis of Linear Time Series Forecasting Models. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024 . OpenReview.net

    2024

  2. [10]

    Xu, W.; Liu, W.; Xu, C.; Bian, J.; Yin, J.; and Liu, T. 2021. REST: Relational Event-driven Stock Trend Forecasting. In Leskovec, J.; Grobelnik, M.; Najork, M.; Tang, J.; and Zia, L., eds., WWW '21: The Web Conference 2021, Virtual Event / Ljubljana, Slovenia, April 19-23, 202...

    2021

  3. [11]

    Yoo, J.; Soun, Y.; chan Park, Y.; and Kang, U. 2021 a . Accurate Multivariate Stock Movement Prediction via Data-Axis Transformer with Multi-Level Contexts. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 313--323

    2021

  4. [12]

    Yoo, J.; Soun, Y.; Park, Y.; and Kang, U. 2021 b . Accurate Multivariate Stock Movement Prediction via Data-Axis Transformer with Multi-Level Contexts. In Zhu, F.; Ooi, B. C.; and Miao, C., eds., KDD '21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, V...

    2021

  5. [13]

    Yu, Z.; Wu, Y.; Wang, G.; and Weng, H. 2024. MIGA: Mixture-of-Experts with Group Aggregation for Stock Market Prediction. arXiv preprint arXiv:2410.02241

    2024 arXiv

  6. [14]

    Zeng, A.; Chen, M.; Zhang, L.; and Xu, Q. 2022. Are Transformers Effective for Time Series Forecasting? CoRR, abs/2205.13504

    2022 arXiv

  7. [15]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...

  8. [16]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.