{"id":"36265d31-dd7b-44aa-968d-3764d78ef078","arxiv_id":"2502.00828","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A decision-focused neural network using GPT-2 embeddings for stocks and macro context reports higher Sharpe ratios and terminal wealth than deep learning baselines, but flawed proofs and survivorship-biased data undermine the claim.","lead":"A new portfolio-selection model combines frozen language-model embeddings with a decision-aware training loss to beat several deep learning baselines on US equity backtests. The paper's theoretical proofs and data construction contain serious errors, so the reported outperformance is not yet credible.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported outperformance rests on a survivorship-biased test universe: Section 4.1.1 keeps only firms that stayed in the index for 2010-2023, conditioning the backtest on future survival. A point-in-time constituent backtest is needed.","rationale":"The paper's central claim is empirical: DINN 'consistently outperforms' state-of-the-art models on S&P100 and DOW30. For that claim to hold, the backtest must be a valid out-of-sample evaluation on the stated indices. Section 4.1.1 violates this by keeping only firms that remained in the index for the entire 2010-2023 sample. That is exactly the survivorship bias the authors claim to address; it is a look-ahead filter on the test set. Removing it is the first check any reviewer should run. The reader identified the same weakest assumption; I agree. I considered whether the false proof of Proposition 1 is a more fundamental flaw. It is a real internal inconsistency: the sequence tilde-mu(k) converges to (mu1-delta, mu2) with delta > 0, not to mu, so the conclusion that optimal weights do not converge is vacuous. However, the paper's headline contribution is the empirical method and its claimed decision-quality gains; the proposition is illustrative motivation, not the basis of the backtest comparison. Similarly, ignoring transaction costs and not releasing code are serious reproducibility problems, but the survivor filter is an explicit, checkable design choice that alone invalidates the stated test universe. A point-in-time re-run with delisting returns and costs would settle the matter directly. Given the central claim is unsupported as presented, the REJECT verdict stands; no change from the reader's recommendation.","tokens_in":31160,"tokens_out":4469,"duration_ms":46230,"concrete_test":"Re-run the full experiment with point-in-time index constituents from CRSP/Compustat: at each rebalance date t, include every firm then in the S&P100 or DOW30; if a firm is later dropped or delisted, sell at the reported delisting return and continue; do not filter by 2023 membership. Use the same training/validation split, and add realistic per-trade costs (e.g., 10-20 bps per side) with the stated rebalance frequency. Recompute Table 1's Sharpe, terminal wealth, and ranking versus baselines and an equal-weight portfolio. If DINN still dominates all baselines on the full, point-in-time universe, the survivorship concern is resolved; if the margin shrinks or reverses, the headline claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.1.1 states: 'To address potential survivorship bias, we include only companies that maintained consistent index membership throughout the study period.' This is not a correction; it is textbook survivorship bias. The 'S&P100' and 'DOW30' datasets are survivor-only subsets (50 of 100, 30 of 30), so every backtested path is conditioned on ex-post index membership over 2010-2023. Any firm that was dropped or delisted--disproportionately poor performers or bankruptcies--is excluded, which inflates returns and Sharpe ratios for all models and makes the absolute levels in Table 1 (43.5%/63.3% annualized, terminal wealth 3.02x/4.47x) uninterpretable as realizable strategy performance. Because the baselines are evaluated on the same survivor set, relative ranking is also contaminated: model selection and hyperparameters were likely chosen on this set, and DINN can exploit survivor-specific patterns that would not exist under point-in-time membership. This single design choice undermines the central claim 'consistently outperforms state-of-the-art deep learning models' regardless of the architecture's merits. The theoretical and other empirical issues (e.g., Proposition 1's limiting argument is invalid because the constructed sequence converges to (mu1-delta, mu2), not mu; no transaction costs; no code) reinforce, but do not displace, this primary concern.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes DINN, a decision-informed neural network that combines pretrained LLM embeddings with prob-sparse cross-attention, a differentiable mean-variance optimization layer, and a hybrid training objective mixing mean squared error with a decision-regret term. The authors claim that minimizing prediction error alone is suboptimal and that DINN consistently outperforms state-of-the-art deep learning baselines on the S&P 100 and DOW 30, with reported annualized returns of 43.5% and 63.3%, Sharpe ratios of 1.04 and 1.29, and terminal wealth of 3.02x and 4.47x. They also provide sensitivity theorems for the optimization layer, a theoretical proposition on the prediction-decision gap, and a gradient-based interpretability analysis of asset importance.","tokens_in":31490,"tokens_out":7300,"duration_ms":71629,"significance":"The combination of large language model embeddings with decision-focused learning for portfolio construction is timely and potentially impactful. The paper has several strengths: the end-to-end differentiable optimization with a regret-based loss is well motivated, the attention mechanism provides an interpretability route, and the authors explicitly consider data leakage by choosing GPT-2. However, the reported empirical superiority is not credible because the test universe is conditioned on future index membership, and the theoretical results contain load-bearing errors. As presented, the evidence does not support the paper's central claims.","major_comments":[{"comment":"The sentence 'To address potential survivorship bias, we include only companies that maintained consistent index membership throughout the study period' describes the opposite of a bias correction. Restricting the S&P 100 to 50 firms and the DOW 30 to 30 firms that survived from 2010 to 2023 conditions the backtest on future index membership, which is textbook survivorship bias: firms that were dropped, merged, or delisted, disproportionately poor performers, are excluded. Consequently the absolute levels in Table 1 (annualized returns 43.53% and 63.25%, terminal wealth 3.0213 and 4.4715) are not realizable strategy performance, and the relative ranking against baselines is contaminated because all models are evaluated on the same survivor set. A point-in-time constituent backtest with delisting returns is required before any outperformance claim can be assessed.","section":"Section 4.1.1"},{"comment":"The proof of Theorem 1 derives Eq. (43) after explicitly setting 2λ=1, so the sensitivity formula (30) is not valid for the general problem stated in Eq. (29); the correct derivative contains a factor 1/(2λ). More importantly, the implemented optimization problem in Eq. (21) includes the box constraints 0≤w_i≤1 and an inequality constraint ‖L w‖_2≤s, whereas Theorem 1 is proved only for an unconstrained-in-sign problem with an equality risk constraint. The KKT conditions for Eq. (21) are different, so Eq. (30) is not established for the actual optimization layer. Since Section 4.5 and Table 3 interpret gradient magnitudes |∂ŵ/∂μ| as evidence of decision-aware learning, this gap is load-bearing.","section":"Section 3.4 / Appendix A.1, Theorem 1"},{"comment":"The constructed sequence is ˜μ^(k) = (μ_1 − δ + 1/k, μ_2) with δ = (μ_1 − μ_2)/2 > 0. As k→∞ this sequence converges to (μ_1 − δ, μ_2), not to the true mean μ = (μ_1, μ_2); the example therefore does not show that MSE-convergent predictions can induce non-convergent optimal portfolios. The subsequent computation of w^(k)_1 confirms that the limit is 1/2 + (μ_1 − μ_2)/(8λ), which differs from w*_1 only because the wrong limit was used. Thus Proposition 1, as stated and proved, is invalid.","section":"Proposition 1, Section 4.3"},{"comment":"The reported returns, Sharpe ratios, and terminal wealth are computed on gross returns without transaction costs or turnover constraints. DINN rebalances at each forecast horizon and its optimization layer can produce high-turnover allocations; with realistic transaction costs and market impact, the 43.5% and 63.3% annualized returns could be materially reduced, and the ranking against baselines could change. The paper should report net-of-cost performance or at minimum turnover and a break-even cost analysis.","section":"Section 4.2 / Table 1"}],"minor_comments":[{"comment":"The notation for the return window switches between ˆr_{t:t+H} and ˆr_{t+1:t+H} in Eqs. (23)-(25); please make the indexing consistent throughout.","section":"Section 3.3.4, Eq. (23)"},{"comment":"The reference 'Waswani et al. 2017' is a typo for 'Vaswani et al. 2017'.","section":"Section 2.2"},{"comment":"The caption describes a comparison of portfolio standard deviation across trials, but the panels display Sharpe ratio, terminal wealth, maximum drawdown, and VaR; the caption and the figure content are inconsistent.","section":"Figure 3"},{"comment":"The caption repeats the formula |∂ŵ_{t+h}/∂μ_{t+h}| although the table reports gradients with respect to the Cholesky factor L_{t+h}; update the caption to match the table.","section":"Table A.3"},{"comment":"The statement that code is available at 'Anonymous Github' does not provide a URL or repository identifier; without an accessible repository, the reproducibility claim cannot be verified.","section":"Section 4.1"},{"comment":"The opening sentence refers to 'Appendix K', which appears to be a leftover placeholder and should be corrected.","section":"Appendix A.4"}],"recommendation":"reject","confidential_remarks":"The manuscript's main empirical claim is invalidated by the survivorship-biased test universe, and the theoretical results contain errors that would require substantial revision. A resubmission would need a point-in-time backtest, corrected proofs of Theorems 1 and 2 and Proposition 1, and explicit treatment of transaction costs."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe short version: the model architecture is a genuinely new combination, but the paper's central claim that DINN 'consistently outperforms' state-of-the-art models is not supported as presented. The main problem is the test universe: Section 4.1.1 keeps only firms that stayed in the index for 2010-2023, which is survivorship bias, not a fix for it. Every backtested path is conditioned on future survival, so the 43.5% and 63.3% annualized returns are not realizable strategy performance. Because hyperparameters and model selection were done on this same set, the relative ranking is also suspect. No transaction costs are mentioned, and the promised code appears to be an anonymous GitHub link that is not a reproducible artifact.\n\nI want to give credit where it is due. The paper is, to my knowledge, the first to combine frozen LLM embeddings with decision-focused learning for mean-variance portfolio optimization. The dual prob-sparse cross-attention (one for macro, one for inter-asset) is a reasonable way to drag textual and numerical signals into one latent space, and the hybrid MSE-plus-regret loss is a sensible objective. The gradient-based analysis in Section 4.5, showing that the model gives lower prediction error for assets with high weight sensitivity, is an intriguing observation, even if it is not a formal claim.\n\nThe theory is shaky. Theorem 1's proof sets 2λ = 1 without justification, and the resulting sensitivity formula is off by the λ scaling, so the claimed derivative is not the true one for the implemented problem. Proposition 1 is worse: the constructed sequence tilde μ(k) converges to ( (μ1+μ2)/2, μ2 ), not to μ, so the non-convergence of weights is a tautology. That proof does not demonstrate what it claims. These are not minor typos; they are the arguments that motivate the decision-focused loss.\n\nBottom line: the idea is worth pursuing, but this version does not support the claims. A carefully redone version with a point-in-time constituent list, transaction costs, and corrected proofs would be a solid paper. I would send it to peer review with the expectation of major revision, because the architectural novelty merits referee time. But if the editor chooses to desk reject, I would not object. For your own work, I would not cite it yet.","headline":"A genuinely new LLM+DFL architecture, but the headline results rest on a survivor-only backtest and the proofs have load-bearing errors; worth a serious redo, not a citation yet.","tokens_in":31994,"tokens_out":3501,"would_cite":false,"duration_ms":33922,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Predicting returns accurately is not enough: DINN trains portfolio weights end-to-end with a decision-regret loss and beats prediction-only deep learning baselines.","keywords":["portfolio optimization","decision-focused learning","large language models","cross-attention","differentiable optimization","mean-variance optimization","estimation error","survivorship bias"],"falsifier":"Re-run DINN and the baselines on the full constituent lists of both indices as of each rebalance date, carrying delisted companies' final returns through the backtest rather than deleting them; if the Sharpe ratio and terminal wealth gaps over the best baseline shrink or reverse, the central empirical claim fails.","tokens_in":30975,"feed_emoji":"📈","tokens_out":9536,"duration_ms":83795,"temperature":0.7,"pith_summary":"This paper tries to establish that portfolio selection should be trained on decision quality, not forecast accuracy alone. It proposes DINN, a decision-informed neural network in which a cross-attention mechanism fuses LLM-derived embeddings of inter-asset and macroeconomic information with decomposed return series, and a differentiable optimization layer converts the resulting return forecasts directly into long-only portfolio weights. Training minimizes a hybrid loss that combines mean squared forecast error with decision regret, defined as the gap between the true mean-variance objective evaluated at predicted weights and at hindsight-optimal weights. On S&P 100 and Dow 30 data from 2010 to 2023, the paper reports that DINN beats Transformer- and LLM-based baselines on annualized return, Sharpe and Sortino ratios, return-over-VaR, and terminal wealth, with much lower trial-to-trial variability. A sympathetic reader would care because this is a direct test of the claim that prediction-error minimization is the wrong objective for investment decisions.","feed_headline":"Decision-aware portfolio model beats prediction-only rivals","feed_subtitle":"Trained to minimize decision regret, it reports 43.5% and 63.3% annual returns on S&P 100 and Dow 30.","key_machinery":"The load-bearing mechanism is the differentiable portfolio-optimization layer coupled with a hybrid loss. Returns are first decomposed into long-term trend and short-term residual components using multi-scale averages; these are fed through prob-sparse cross-attention against two LLM-produced embedding sets, one encoding pairwise stock and sector outperformance counts, the other encoding macroeconomic statistics from irregularly sampled indicators. A frozen pretrained LLM, GPT-2, chosen to limit data leakage, refines the fused representations before a linear projection produces normalized return forecasts, which are denormalized and passed to a second-order cone optimization layer that outputs portfolio weights. The hybrid objective $\\mathcal{L} = \\beta \\mathcal{L}_{\\mathrm{MSE}} + (1-\\beta)\\mathcal{L}_{\\mathrm{Decision}}$ with $\\beta=0.4$ balances forecast accuracy against decision regret, and Theorems 1 and 2 provide the weight sensitivities needed to backpropagate through the optimization layer.","core_discovery":"The central claim is that the two-stage predict-then-optimize workflow is why improved forecasts do not reliably improve portfolios, and that the fix is to differentiate through the portfolio optimization itself. DINN's optimization layer solves a second-order cone formulation of mean-variance optimization with full-investment and long-only constraints; because the layer is differentiable, gradients of the decision regret $\\Delta J_{t+h}=\\hat J_{t+h}-J^\\star_{t+h}$ flow back through the predicted return and covariance estimates. Theorems 1 and 2 give closed-form sensitivities $\\partial \\hat w_{t+h}/\\partial \\hat\\mu_{t+h}$ and $\\partial \\hat w_{t+h}/\\partial \\hat L_{t+h}$ for this propagation. Proposition 1 supplies a two-asset example in which predicted returns converge to the truth in squared error while the induced optimal portfolio weights do not converge to the true optimum, which is the paper's theoretical reason for rejecting MSE-only training. Empirically, the paper attributes DINN's advantage to its attention mechanism concentrating on assets whose misprediction would be most costly: across four macroeconomic regimes, assets with the largest gradient sensitivities show lower MSE and MAE than assets with small sensitivities, and portfolios built from attention-selected assets beat random and uniform portfolios on Sharpe ratio and drawdown.","pith_inferences":["A testable extension the paper leaves implicit: the same decision-regret construction could be applied to non-convex or non-differentiable objectives such as expected shortfall or drawdown-constrained portfolios, which the paper lists as future work rather than a demonstrated result.","Swapping the frozen GPT-2 backbone for a more capable LLM would separate two explanations: if DINN's edge comes mainly from decision-focused training, performance should change little, whereas if the embeddings matter, the gap should widen.","Because the backtest keeps only companies that stayed in each index for the whole 2010-2023 period, the reported terminal wealth figures are best interpreted as upper bounds; including delisted companies would be the natural stress test of the practical claim."],"forward_implications":["If the results hold, any forecasting model used for portfolio selection can be upgraded by appending a differentiable optimization layer and a decision-regret term without changing the underlying return predictor.","The gradient sensitivities $\\partial \\hat w/\\partial \\hat\\mu$ and $\\partial \\hat w/\\partial \\hat L$ give a practical asset-ranking rule: concentrate forecasting capacity on assets where estimation error would move portfolio weights most.","LLM embeddings of macroeconomic variables give a route to include irregularly sampled economic indicators directly in the differentiable pipeline rather than as separately estimated factors.","The reported reduction in trial-to-trial variability suggests decision-focused training stabilizes portfolio construction even when return forecasts remain noisy."],"supporting_citations":[{"why":"Defines the mean-variance objective that the differentiable optimization layer instantiates.","marker":"Markowitz 1952"},{"why":"Documents that errors in mean estimates dominate variance errors, motivating the decision-focused loss.","marker":"Chopra and Ziemba 1993"},{"why":"Shows the predict-then-optimize gap that the paper's regret-based training directly addresses.","marker":"Elmachtoub and Grigas 2022"},{"why":"Supplies the differentiable convex optimization layers that make end-to-end training through the portfolio problem possible.","marker":"Agrawal et al. 2019"},{"why":"Provides the prob-sparse attention mechanism used to fuse LLM embeddings with temporal data.","marker":"Zhou et al. 2021"},{"why":"Introduces the LLM-reprogramming approach the paper adapts for macroeconomic prompt embeddings.","marker":"Jin et al. 2024"}],"fun_headline_variants":["Portfolio model trains on decision regret, not forecast error","LLM-integrated optimizer closes prediction-decision gap","Attention-driven model prioritizes assets that matter most","Direct decision optimization beats predict-then-optimize"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The empirical claim assumes in Section 4.1.1 that a backtest containing only companies which stayed in each index for the full 2010-2023 period is representative; because firms that were dropped or delisted are removed, the reported Sharpe ratios and terminal wealth could be inflated relative to a live trading experience.","fun_headline_variants_meta":{"raw":{"variants":["Portfolio model trains on decision regret, not forecast error","LLM-integrated optimizer closes prediction-decision gap","Attention-driven model prioritizes assets that matter most","Direct decision optimization beats predict-then-optimize"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000335,"raw_usage":{"total_tokens":1870,"prompt_tokens":972,"completion_tokens":898,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":588,"completion_tokens_details":{"reasoning_tokens":835}},"tokens_in":588,"tokens_out":898,"duration_ms":10063,"temperature":1.0,"reasoning_tokens":835,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T17:33:19.123704+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run DINN and the baselines on the full constituent lists of both indices as of each rebalance date, carrying delisted companies' final returns through the backtest rather than deleting them; if the Sharpe ratio and terminal wealth gaps over the best baseline shrink or reverse, the central empirical claim fails.","supporting_citations":[{"cited_title":"The Journal of Finance , 1952, 7, 77–91","cited_arxiv_id":null,"evidence_quote":"Defines the mean-variance objective that the differentiable optimization layer instantiates."},{"cited_title":"predict, then optimize","cited_arxiv_id":null,"evidence_quote":"Shows the predict-then-optimize gap that the paper's regret-based training directly addresses."},{"cited_title":"and Zhang, W., Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting","cited_arxiv_id":null,"evidence_quote":"Provides the prob-sparse attention mechanism used to fuse LLM embeddings with temporal data."}],"review_version":1}