{"id":"61198b65-99a3-4c37-8c9b-dddaab1b8e19","arxiv_id":"2608.11785","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A query-key router plus a sparse expert-selection-update mechanism lets a frozen LLM adapt its mixture-of-experts routing to trading context, beating 22 baselines in backtests.","lead":"TradingMoE adds lightweight expert networks to a frozen language model and trains a router to pick the right experts for each trading decision, adapting as markets shift. In backtests it reports returns roughly 30% higher than the best of 22 baseline systems, with caveats about the pretraining data window.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Stock headline margin is not leakage-controlled: the backbone postdates the 2023 test window, and the only forward-only stock evidence (a 3-month 2026 paper trade) has no reported baseline margin, so the 30.89% gain over LightGBM is not cleanly established.","rationale":"The paper is a competent method paper: TradingMoE's components are clearly described, the theory is correct as far as it goes, ablations show each component matters, and the Qwen2.5-7B Crypto control shows some advantage survives a pre-dating backbone. I do not see fraud or fatal inconsistency. The reader's weakest assumption—temporal leakage of the backbone—is the right one, but the sharper version is that the Stock-specific headline has no leakage-controlled replication at all: the Qwen2.5-7B control applies only to Crypto, and the 2026 paper trade is a different window without reported numbers. Thus the 30.89% Stock figure is the least secure part of the central claim. The concrete test above would settle it. Verdict stays CONDITIONAL: accept only after the Stock leakage control is run and reported.","tokens_in":26005,"tokens_out":15410,"duration_ms":161888,"concrete_test":"Rerun the exact Stock test (2023-06-13 to 2023-12-31) with a backbone whose pretraining cutoff predates the test window, e.g., Llama-2-7B or Qwen1.5-7B, using the same TradingMoE training on Stock train/validation and the same hyperparameters and simulator as Section 5. Report cumulative return and Sharpe vs LightGBM. If the 30.89pp margin collapses or shrinks materially, the headline Stock claim should be re-scaled; if it persists, the leakage concern is settled.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's own Section 5.4 concedes that LLM backtests may suffer temporal leakage when the pretraining corpus overlaps the evaluation period. For Crypto, the Qwen2.5-7B control (released 2024-09, before 2025) is a valid check; it lowers the margin over TRA from 30.78pp to 25.37pp (Table 5), so the Crypto claim is partly robust. For Stock, no comparable control exists: Qwen2.5-7B was released after the 2023-06-to-2023-12 test window and cannot rule out memorization; the paper-trading run is a different 2026 window and is only shown in Figure 4 without numerical metrics. The abstract's 30.89% Stock gain therefore rests entirely on Qwen3.5-9B, whose training data includes the test period. If the backbone's latent knowledge of 2023 H2 outcomes contributes to routing, the Stock margin is overstated. This is the most load-bearing soft spot because it directly affects the headline quantitative claim; the controlled Crypto evidence suggests a genuine but possibly smaller advantage.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TradingMoE, a sparse mixture-of-experts layer for frozen LLMs used in daily trading-decision generation. It introduces a query-key router that computes low-dimensional token queries and matches them with learnable expert keys, and a sparse expert-selection update that samples inactive experts, compares them with the lowest-scored active expert, and injects a detached routing-margin signal in the same backward pass. Two theoretical propositions show that inactive-expert sampling yields an unbiased gradient estimate and that the router update is consistent with first-order replacement benefit. Experiments on a Stock benchmark (test window 2023-06-13 to 2023-12-31) and a Crypto benchmark (2025) report cumulative returns of 49.08% and 73.79%, outperforming the best baselines by 30.89 and 30.78 percentage points. Section 5.4 additionally reports a forward-only Stock paper-trading run and a Qwen2.5-7B Crypto control.","tokens_in":26217,"tokens_out":6251,"duration_ms":61800,"significance":"If the results are trusted, the paper makes a useful contribution to LLM-based trading: it documents a concrete failure of native MoE routers (Pearson correlation -0.015 with replacement gains), identifies a low-rank structure in token-expert credit, and proposes a training mechanism that is computationally sparse. Strengths include the authors' explicit acknowledgment of temporal-leakage risk, a five-seed statistical robustness protocol (Appendix B.5), transaction-cost sensitivity analysis (Appendix E.3), and the fact that Propositions 1 and 2 are consistency/identifiability statements about the proposed gradient update rather than circular derivations of the empirical results. The controlled Crypto experiment in Table 5 suggests a genuine but possibly smaller advantage. The main unresolved point is that the headline Stock margin currently lacks a leakage-free quantitative estimate.","major_comments":[{"comment":"The headline Stock result is not leakage-controlled: the backbone Qwen3.5-9B was released in 2026-03, after the Stock test window (2023-06-13 to 2023-12-31), so its pretraining corpus may contain information about the evaluation period. The only forward-only Stock evidence is a three-month paper-trading run shown in Figure 4, which reports no numerical metrics and no baseline margin. Consequently the abstract's 30.89pp improvement over LightGBM rests entirely on a post-dating backbone, and the claim is not cleanly established. The Crypto control in Table 5 is valid for Crypto and reduces the margin from 30.78pp to 25.37pp, but it cannot control for the Stock window because Qwen2.5-7B also postdates 2023. Please provide a Stock-specific leakage control (e.g., a backbone whose pretraining corpus provably excludes the Stock test window, or a comparable differential test) or clearly state that the headline Stock margin is measured only in the uncontrolled setting.","section":"§5.4, Table 5, Figure 4"},{"comment":"The sentence 'the potential temporal overlap has only a limited impact' is supported only by a Crypto comparison between TradingMoE(Qwen2.5-7B) and TradingMoE(Qwen3.5-9B). That comparison says nothing about the Stock benchmark, where the backbone postdates the test window. The subsequent conclusion that 'the overall advantage primarily comes from the proposed method rather than temporal leakage' is therefore stronger than the evidence supports for Stock.","section":"§5.4, final paragraph"},{"comment":"The forward-only paper-trading experiment is reported only as a figure without cumulative-return values, Sharpe ratios, or a table of baseline margins; moreover, the DeepSeek curve switches from DeepSeek-V3.2 to DeepSeek-V4-Pro mid-period (Appendix D), which may confound that particular comparison. Without numeric results, the statement that the advantage persists under forward-only deployment is not quantitatively verifiable.","section":"§5.4, Figure 4"}],"minor_comments":[{"comment":"In the first observation about Table 2, 'it it obtains' should read 'it obtains'.","section":"§5.2"},{"comment":"In the FinAgent row, '1.5624.57%' appears to be two values run together; please separate the Sharpe ratio and maximum drawdown for readability.","section":"Table 2"},{"comment":"The date '2026-6-30' should be written as '2026-06-30' for consistency with the other dates in the paper.","section":"§5.4"},{"comment":"The phrase 'spare expert selection update' should be 'sparse expert selection update'.","section":"§5.3"},{"comment":"The authors correctly state that 'SVD fitting and comparison use the same matrix, so this diagnostic evaluates representational capacity rather than held-out prediction.' Because Table 1 is used as motivation for the low-rank router, this caveat should also appear near Table 1 in the main text so readers do not overinterpret the 98% better-than-native route rate as a predictive result.","section":"Appendix A.4"},{"comment":"Add axis labels and units to Figure 4, and report the terminal cumulative returns of all displayed curves in the text or caption so the forward-only result can be assessed numerically.","section":"Figure 4"}],"recommendation":"major_revision","confidential_remarks":"The paper's internal logic is sound, and the reader's circularity concern is not supported: Propositions 1 and 2 are consistency statements about the proposed update, not derivations of the empirical result from itself. The substantive gap is the Stock leakage control, which is load-bearing for the headline claim and should be addressed before acceptance. Also ask the authors to replace the 'Conference'17' template placeholder before final submission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: TradingMoE has a genuinely new mechanism—a query-key router plus a sparse expert selection update that samples inactive experts and injects a detached margin-credit signal into the router. That is not present in the Switch/GShard/Mixtral line or in the external-expert trading routers, and it is motivated by a sensible diagnostic: native router scores have near-zero correlation with counterfactual replacement gains, and the token–expert credit matrix is roughly low-rank. The paper includes leakage controls and multi-seed tests, which puts it ahead of most LLM-trading papers.\n\nThe soft spot is the headline stock number. Qwen3.5-9B was released in March 2026, after the 2023 stock test window. The paper's own Section 5.4 acknowledges this. The Crypto control with Qwen2.5-7B (predates 2025) shaves the margin from 30.78pp to 25.37pp, so the Crypto claim is partly robust. But the forward-only stock evidence is a three-month 2026 paper trade shown only as a curve, with no baseline margin reported. So the abstract's 30.89% stock gain is not cleanly established. The authors should make the leakage-controlled runs the primary evidence, report the stock paper-trading margin numerically, and add error bars.\n\nThe theory is minor but sound: Propositions 1 and 2 are unbiasedness and chain-rule consistency, not circular derivations. The low-rank diagnostic is in-sample—SVD fit and evaluation use the same matrix—so it shows representational capacity, not prediction. That limitation is stated in Appendix A.4, which is honest.\n\nNo red flags on data handling or citation behavior. Code is linked but anonymous, and no data is shipped, so reproducibility is not yet verifiable.\n\nThis paper deserves a serious referee. The contribution is real, the evaluation is broad, and the main flaw is fixable by reframing what the headline evidence is. I would send it out with a request for the leakage-controlled results to lead, numeric paper-trading baselines, and a clearer label on the in-sample diagnostic.","headline":"Genuinely new MoE routing mechanism, but the headline stock gain is not leakage-controlled; the Crypto control and the method itself justify a serious look.","tokens_in":26846,"tokens_out":2696,"would_cite":true,"duration_ms":26592,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TradingMoE claims that a query–key router with sparse expert replacement lifts cumulative trading returns by roughly 30 percentage points over the best baselines on stocks and crypto.","keywords":["large language models","ai4trading","mixture-of-experts","expert routing","query-key router","counterfactual expert credit","low-rank structure","paper trading"],"falsifier":"The decisive check is a leakage-controlled rerun of the Stock backtest: train TradingMoE with its protocol fixed but on a backbone whose pretraining corpus provably ends before the 2023-06-13 test start, and compare cumulative return against LightGBM over the same 138 trading days; if the margin falls far below the reported 30.89 percentage points, memorization, not routing, carried the headline result. A routing-specific check is to apply the paper's own diagnostic to TradingMoE's trained router: if the query–key margins show near-zero or negative correlation with measured replacement gains on held-out decision tokens, the premise that counterfactual credit improves expert selection would be contradicted.","tokens_in":25728,"feed_emoji":"📈","tokens_out":22930,"duration_ms":189652,"temperature":0.7,"pith_summary":"TradingMoE tries to establish that the bottleneck in LLM-based trading is not the language model itself but the router that decides which specialized experts process each token of the trading decision. It reports that conventional internal MoE routers assign scores that barely track how much an expert actually improves the decision (correlation of −0.015), and that expert usefulness across tokens lives in a compact, approximately low-rank structure. On that basis, the paper freezes a pretrained dense LLM, adds lightweight residual experts, and routes them with a query–key matcher that compares a per-token demand query with learnable expert keys, plus a mechanism that samples inactive experts and lets them replace the weakest active expert when market context shifts. Against 22 baselines, TradingMoE reports cumulative returns of 49.08% on the stock test set and 73.79% on a held-out crypto year — 30.89 and 30.7 percentage points above the strongest baselines, and 37.81 and 80.17 percentage points above buy-and-hold. If these margins survive forward-only deployment, the method offers a recipe for specializing frozen LLMs to evolving markets without retraining the backbone.","feed_headline":"Smarter expert routing adds 30 points of return over best baselines","feed_subtitle":"Paper claims ~30-percentage-point higher cumulative returns on stock and crypto backtests.","key_machinery":"The load-bearing object is the counterfactual expert credit, a first-order estimate of how replacing an active expert with a candidate would change the trading-decision loss. For token $t$, layer $\\ell$, and expert $j$ it is $C^{\\ell}_{t,j} = -\\langle \\partial \\mathcal{L}_{\\mathrm{task}}/\\partial \\mathbf{h}^{\\ell}_t,\\, e^{\\ell}_j(\\mathbf{h}^{\\ell}_t) \\rangle$; relative to the weakest active expert it becomes $\\Delta C^{\\ell}_{t,i} = \\alpha^{\\ell}_{t,\\mathrm{low}}(C^{\\ell}_{t,i} - C^{\\ell}_{t,\\mathrm{low}})$, which preserves within-token expert ordering. This credit object plays three roles: its measured low-rank structure motivates parameterizing the router's score matrix as $\\mathbf{S}^{\\ell} = \\mathbf{Q}^{\\ell}(\\mathbf{U}^{\\ell})^{\\top}$ with $\\mathrm{rank}(\\mathbf{S}^{\\ell}) \\le d_q$ (the paper uses $d_q = 16$ and 64 experts); the query–key router realizes that factorization; and the sparse selection update injects a signal $\\boldsymbol{\\xi}^{\\ell}_t$ that cancels in the forward pass but, during backpropagation, raises or lowers each sampled inactive expert's routing margin exactly when first-order credit predicts replacement would help or hurt. Two propositions guarantee the mechanism is well-behaved: the sampled margin gradient is an unbiased estimate of the average over all inactive experts, and the induced router update is consistent with the local loss reduction of expert replacement.","core_discovery":"The central claim is that expert routing for trading should be driven by counterfactual expert credit — an estimate of how much swapping one expert for another would reduce the trading-decision loss — rather than by naive router scores trained only through the task loss. Two empirical findings ground this claim. First, native router scores have essentially zero correlation with measured replacement gains (−0.015), and 66.76% of decision tokens have at least one unselected expert that would have improved the decision. Second, token–expert credit matrices are approximately low-rank, with rank-16 reconstructions retaining 74.2% and 77.9% of credit energy on the OLMoE and DeepSeek-V2-Lite backbones. TradingMoE implements the finding as a query–key router, in which a low-dimensional per-token query encodes the expertise demanded under the current market context and is matched against learnable expert keys, so the score matrix $\\mathbf{S}^{\\ell} = \\mathbf{Q}^{\\ell}(\\mathbf{U}^{\\ell})^{\\top}$ has rank at most the query dimension. A sparse expert selection update then samples a few inactive experts per decision token, compares each against the lowest-scored active expert, and adjusts routing margins in the same backward pass; the paper proves the sampled update is unbiased over inactive experts and directionally consistent with first-order loss reduction from replacement. On the reported benchmarks this yields 49.08% cumulative return on the stock test set and 73.79% on the crypto year, beating the strongest baseline in each market by 30.89 and 30.7 percentage points.","pith_inferences":["The low-rank credit structure is measured only on two pretrained MoE backbones; if it is a general property of expert usefulness in non-stationary decision tasks, the query–key router would transfer to other LLM decision problems where the right specialization shifts with context — my inference, not demonstrated in the paper.","The Stock test window falls inside the training corpus of any 2024-or-later backbone, so the cleanest confirmation of the 30.89-percentage-point stock margin would be a re-run with a backbone whose pretraining provably ends before mid-2023 — a control the leakage-controlled section does not include.","Because the selection update samples only $m = 2$ of the roughly 60 inactive experts per token, regime adaptation is limited by sampling noise; making the sampling credit-weighted or increasing $m$ could sharpen responsiveness to regime shifts at modest extra compute.","The reported cost sensitivity implies the method's edge survives roughly twice the one-way transaction-cost rate that breaks the best baselines (about 28 bps vs. 13–14 bps), so the practical deployment window is wider than for the alternatives even though daily rebalancing eventually becomes unprofitable at extreme costs."],"forward_implications":["Frozen dense LLMs can be specialized to trading cheaply: only small residual experts, expert keys, and query heads are trained, while the pretrained backbone stays untouched and the added computation stays sparse (top-$k$ of 4 among 64 experts per layer).","Router design becomes the primary lever for LLM trading performance: the diagnostics show naive router scores carry almost no information about expert contribution, and the ablation attributes most of the gain to the query–key credit-based routing rather than to the selection update alone.","The same model, trained only on U.S. stocks, transfers to a different asset class: it ranks first on the held-out crypto year and stays positive in forward-only paper trading, suggesting the routing mechanism rather than market-specific memorization drives the edge.","Market-regime shifts are handled by construction: inactive experts are periodically sampled as challengers and can replace the weakest active expert, so the route can change as the market context encoded in the token queries changes.","The stock advantage is not a single-run artifact: across five seeds the margin over LightGBM is significant under a robust Sharpe-difference test (two-sided $p = 0.018$) and a HAC-adjusted mean test ($p = 0.007$)."],"supporting_citations":[{"why":"Supplies the Switch-Transformer sparse-MoE formulation the paper treats as conventional internal routing, whose router scores are diagnosed as misaligned with expert contribution.","marker":"[14]"},{"why":"Foundational sparsely-gated MoE layer defining top-k expert selection and the task-loss-only training objective the paper critiques.","marker":"[42]"},{"why":"Provides the first-order gradient-based importance approximation that the counterfactual expert credit formula is modeled on.","marker":"[34]"},{"why":"Neural-pruning importance estimation is the second source for the first-order credit approximation used in replacement gains.","marker":"[35]"},{"why":"DeepSeek-V2 is one of the two pretrained MoE backbones whose token–expert credit matrix shows the low-rank structure (77.9% energy at rank 16).","marker":"[28]"},{"why":"OLMoE is the other diagnostic backbone, retaining 74.2% of credit energy in a rank-16 reconstruction of its credit matrix.","marker":"[36]"},{"why":"FNSPID supplies the stock price records, the time-aligned news corpus, and the chronological Stock benchmark splits.","marker":"[12]"},{"why":"LightGBM is the strongest baseline on Stock; the 30.89-percentage-point margin and the statistical tests are measured against its daily return series.","marker":"[23]"},{"why":"The Ledoit–Wolf Sharpe-difference test supports the claim that the seed-averaged advantage over LightGBM is unlikely by chance.","marker":"[24]"},{"why":"The stationary bootstrap produces the confidence intervals that bound the seed-averaged cumulative return and Sharpe ratio.","marker":"[40]"}],"fun_headline_variants":["Adaptive expert router earns 30-point trading edge","Swap-in experts improve stock and crypto returns by 30 points","Low-rank expert routing beats best baseline by 30 points","Router that tests expert swaps lifts trading gains 30 points","TradingMoE: expert selection that adapts to markets, +30 points"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline margins assume that the frozen language model's pretraining did not effectively give it knowledge of the stock and crypto evaluation periods, so the reported gains come from the routing method rather than from memorized market outcomes.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive expert router earns 30-point trading edge","Swap-in experts improve stock and crypto returns by 30 points","Low-rank expert routing beats best baseline by 30 points","Router that tests expert swaps lifts trading gains 30 points","TradingMoE: expert selection that adapts to markets, +30 points"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001333,"raw_usage":{"total_tokens":5522,"prompt_tokens":1144,"completion_tokens":4378,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":760,"completion_tokens_details":{"reasoning_tokens":4292}},"tokens_in":760,"tokens_out":4378,"duration_ms":35191,"temperature":1.0,"reasoning_tokens":4292,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:27:34.364471+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"The decisive check is a leakage-controlled rerun of the Stock backtest: train TradingMoE with its protocol fixed but on a backbone whose pretraining corpus provably ends before the 2023-06-13 test start, and compare cumulative return against LightGBM over the same 138 trading days; if the margin falls far below the reported 30.89 percentage points, memorization, not routing, carried the headline result. A routing-specific check is to apply the paper's own diagnostic to TradingMoE's trained router: if the query–key margins show near-zero or negative correlation with measured replacement gains on held-out decision tokens, the premise that counterfactual credit improves expert selection would be contradicted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Foundational sparsely-gated MoE layer defining top-k expert selection and the task-loss-only training objective the paper critiques."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the first-order gradient-based importance approximation that the counterfactual expert credit formula is modeled on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Neural-pruning importance estimation is the second source for the first-order credit approximation used in replacement gains."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"OLMoE is the other diagnostic backbone, retaining 74.2% of credit energy in a rank-16 reconstruction of its credit matrix."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"FNSPID supplies the stock price records, the time-aligned news corpus, and the chronological Stock benchmark splits."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"LightGBM is the strongest baseline on Stock; the 30.89-percentage-point margin and the statistical tests are measured against its daily return series."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The Ledoit–Wolf Sharpe-difference test supports the claim that the seed-averaged advantage over LightGBM is unlikely by chance."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The stationary bootstrap produces the confidence intervals that bound the seed-averaged cumulative return and Sharpe ratio."}],"review_version":1}