{"id":"9e67674e-18ba-421e-a0d7-700b52ccb26d","arxiv_id":"2509.10531","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"FinXplore combines PPO for stock allocation and DQN for commodity exploration, reporting higher backtest returns on NIFTY and DJIA, but the evaluation uses future information in the acceptance rule.","lead":"This paper introduces FinXplore, a two-agent deep reinforcement learning system where one agent picks stocks and another recommends commodities, aiming to beat benchmarks on Indian and US markets. The reported results rely on a selection rule that uses the current period's outcome to decide that period's portfolio, which is a look-ahead bias.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1's trailing-60d Sharpe acceptance rule creates an in-sample selection/reward loop; without a stated point-in-time protocol, the reported 127.91%/49.56% returns are not shown to be tradeable and may reflect look-ahead or overfitting.","rationale":"Read in good faith, the paper proposes a dual-agent DRL framework and claims empirical superiority. The load-bearing condition is that the backtest measures a strategy that could have been traded. Algorithm 1's acceptance/reward loop uses f(.) over the last 60 portfolio returns, and no point-in-time convention is stated, so the exact information set at decision time is undefined. The reader's interpretation—that the realized return of the upcoming/current period is used to decide the allocation—would be look-ahead and invalidate the results. A more charitable interpretation—that the 60-day window ends at the previous close—makes the rule a trailing-Sharpe selector, so the reported performance may be in-sample selection rather than evidence of DRL skill. Either way, the reported gap between FinXplore and 'Without Exploration' is not a valid demonstration of the framework's superiority. This is a concrete, testable flaw rather than a style issue, so I maintain the reject direction while framing it as an unresolved temporal/selection protocol rather than a proven look-ahead.","tokens_in":9244,"tokens_out":9214,"duration_ms":110589,"concrete_test":"Run a strict point-in-time backtest for both NIFTY and DJIA: at decision time t, compute SR_current and SR_new using only returns up to t-1, accept a_E iff SR_new > SR_current, and hold the chosen portfolio for the next period. Use the same hyperparameters, seeds, and transaction costs. Compare cumulative return and Sharpe with Table II/III. If the FinXplore edge collapses or reverses, the original protocol used contemporaneous/future information; if the edge persists, test whether accepted assets' next-period returns are actually better than rejected ones to distinguish look-ahead from in-sample selection.","verdict_should_be":"REJECT","load_bearing_attack":"The central outperformance claim rests on Algorithm 1 (lines 6-10). At each rebalance, SR_current and SR_new are computed with f(.), the Sharpe ratio over the last 60 portfolio returns, and a_E is accepted only if SR_new > SR_current; R_PPO and R_DQN are set to these same SR values. Thus the final portfolio is chosen to maximize the in-sample trailing Sharpe, and both agents are rewarded for exactly that selection. The paper never specifies whether s_t includes close-of-day t returns and whether the decision is executed for the next period. If the 60-day window extends through the period about to be traded, this is textbook look-ahead; if it ends at the previous close, the rule is a momentum/selection rule rather than evidence that the dual-agent DRL policy has learned predictive skill. In either reading, the 127.91% / 49.56% results are not established as a tradeable out-of-sample strategy, and the gap over the 'Without Exploration' baseline is partly a construction artifact of the acceptance rule. The fixed κ=10% further weakens the 'adaptive balancing' claim, but evaluation validity is the load-bearing issue.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FinXplore, a dual-agent deep reinforcement learning framework for portfolio optimization. A PPO agent allocates wealth within a pre-defined investment universe, while a DQN agent recommends adding assets from an extended universe (commodities). The two agents are trained on NIFTY and DJIA data from 2011-2021 and backtested on 2022-2024, with reported cumulative returns of 127.91% (NIFTY) and 49.56% (DJIA), surpassing all benchmarks. The paper claims this demonstrates the benefit of dynamically balancing exploitation of existing assets with exploration of new opportunities.","tokens_in":9570,"tokens_out":3012,"duration_ms":28040,"significance":"The dual-agent architecture and the idea of expanding the investment universe via a learned exploration policy are potentially interesting and go beyond fixed-universe DRL portfolio methods. The use of two established RL algorithms (PPO and DQN), evaluation against several baselines, and reporting of multiple risk-adjusted metrics are positive features. However, the main empirical claim is not credible as presented: the acceptance rule in Algorithm 1 appears to use realized returns from the period being traded, making the reported backtest results unattainable in real time. Because the central result rests on this evaluation, the contribution is not currently established.","major_comments":[{"comment":"Algorithm 1 (lines 2-10) computes SR_current = f(a_U) and SR_new = f(a_E) at each rebalancing step and selects a_E only if SR_new > SR_current. Section II-A defines the reward as SR_60, the Sharpe ratio over the last 60 portfolio returns, but does not specify whether the current period's return is included in that window. If the 60-day window includes the period about to be traded, the acceptance decision uses the same realized outcome that the policy is trying to predict, which is a look-ahead bias. If the window ends at the previous close, the rule is a mechanical ex post selection rule rather than evidence of learned predictive skill. In neither reading are the reported cumulative returns (Table II: 127.91%; Table III: 49.56%) established as tradeable out-of-sample results.","section":"Algorithm 1 and Section II-A"},{"comment":"The comparison against the 'Without Exploration' DRL baseline is biased by construction. Algorithm 1 always chooses the allocation with the higher realized 60-day Sharpe ratio, so FinXplore's realized Sharpe is at least that of the no-exploration portfolio in every period. This mechanically guarantees a nonnegative ΔSR for the DQN reward and systematically favors FinXplore. The roughly 34pp (NIFTY) and 12pp (DJIA) cumulative-return gaps therefore largely reflect the selection rule, not an improved learned policy.","section":"Tables II and III, 'Without Exploration' baseline"},{"comment":"The paper repeatedly claims the framework 'dynamically balances' exploitation and exploration, but the exploration weight is fixed at κ=10% (Table I) for all periods. Algorithm 1 only toggles between the full existing-universe allocation and the allocation that includes the suggested asset; there is no mechanism that adjusts the fraction of wealth devoted to exploration in response to market conditions. Thus the central 'adaptive balancing' claim is unsupported by the implemented method.","section":"Table I and Sections III-V"}],"minor_comments":[{"comment":"There are typographical errors: 'effciency' in the Abstract, 'to access the risk-adjusted portfolio return' should be 'to assess', and 'closed' should be 'close' in Section IV-A. A careful proofread is needed.","section":"Throughout"},{"comment":"Hyperparameter tuning is described only by search ranges (Table I); the final selected hyperparameters are not reported. This hampers reproducibility, especially since no code repository is mentioned.","section":"Section IV-B"},{"comment":"Markowitz and the market index are reported without standard deviations, while DRL methods have means ± std from five runs. It would be helpful to know whether the reported point estimates are stable or based on a single run.","section":"Tables II and III"},{"comment":"The quarterly-return figures are plotted only for FinXplore, 'Without Explore', and 'Markowitz Baseline', which is a subset of the models in Tables II and III. The caption for Figure 4 labels the third series 'Baseline' but does not identify which baseline it is.","section":"Figures 4 and 5"}],"recommendation":"reject","confidential_remarks":"The look-ahead issue in Algorithm 1 is fundamental to the evaluation and cannot be resolved by a local fix; any corrected point-in-time protocol would require rerunning all experiments and could produce different conclusions. The paper also does not release code or a precise decision-timing specification, making independent verification difficult. The novelty relative to Aquino et al. is incremental, and the evaluation as it stands is not publishable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this paper extends Aquino et al.'s extended-universe idea by using PPO for exploitation and DQN for exploration. That's a legitimate, if incremental, combination. But the backtest design in Algorithm 1 makes the reported returns unreliable, and the claim that exploration helps is not established.\n\nWhat's good: The two-agent setup is clearly described, the state includes technical indicators, and the authors use standard DRL components. The results are reported with standard deviations from five runs, which is better than many finance papers. The writing is straightforward.\n\nThe problem: Algorithm 1 decides whether to include the DQN-suggested asset by comparing the trailing 60-day Sharpe ratios of the PPO allocation and the reoptimized allocation. As written, it is ambiguous whether the 60-day window includes the current period's return. If it does, that's textbook look-ahead. If it ends at the previous close, the rule is a momentum/selection rule that picks, ex post, the better of two candidate portfolios for each period. Under either reading, the reported cumulative returns (127.91% on NIFTY, 49.56% on DJIA) are not the returns of a single tradeable strategy; they are the results of a selection mechanism that uses realized performance to choose allocations. The gap over the 'Without Exploration' baseline is therefore partly a construction artifact, not a clean test of the exploration concept.\n\nThe paper also claims the framework is 'adaptive' but fixes kappa at 10%, and it never specifies the exact point-in-time protocol. That is a fixable omission, but until it's addressed, the empirical results cannot be taken as evidence.\n\nOne more subtle issue: the reward R_PPO = SR_current is the Sharpe ratio of the current proposed portfolio over the past 60 days, not the realized return of the portfolio the agent actually held. This makes the training signal a backtest of the current suggestion rather than a reflection of realized performance. That is a bit unusual and should be explained.\n\nBottom line: the architecture is reasonable and the paper is readable, but the evaluation has a load-bearing flaw. The authors should rerun the backtest with a strict point-in-time protocol, and compare against a baseline that also gets the same 'pick the better of two' selection rule without DRL, to isolate the effect of exploration. Without that, the superiority claim is unsupported.\n\nFor a reading group, this is a useful case study in backtest bias. I wouldn't cite it in my own work yet. If the protocol gets fixed, it could become a solid incremental contribution.","headline":"The dual-agent exploration idea is a legitimate extension of Aquino et al., but Algorithm 1's acceptance rule makes the backtest results unreadable as tradeable performance.","tokens_in":10012,"tokens_out":7204,"would_cite":false,"duration_ms":69440,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FinXplore claims that a second reinforcement-learning agent, which scouts new assets and is accepted only when it improves a 60-day risk-adjusted return, beats fixed-universe benchmarks on NIFTY and DJIA backtests.","keywords":["portfolio optimization","deep reinforcement learning","PPO","DQN","exploration-exploitation trade-off","Sharpe ratio","investment universe","backtesting"],"falsifier":"Re-run the NIFTY and DJIA backtests with Algorithm 1's acceptance test computed on Sharpe ratios that end before the rebalance (for example, through t−1 only, or via a walk-forward procedure that retrains the agents without touching the test period). If the corrected FinXplore does not beat the without-exploration agent and the market index by the reported margins, the superior performance is an artifact of look-ahead rather than of exploration.","tokens_in":9171,"feed_emoji":"📈","tokens_out":9427,"duration_ms":89772,"temperature":0.7,"pith_summary":"The paper sets out to show that portfolio optimization should not be stuck with a fixed set of assets. FinXplore pairs one reinforcement-learning agent (a policy-gradient method called PPO) that allocates wealth across the familiar universe with a second agent (deep Q-learning) that proposes new assets from an extended universe of commodities. The second agent is rewarded by how much its suggestion raises the portfolio's 60-day Sharpe ratio, and the first agent accepts the suggestion only when that ratio improves. On backtests from January 2022 to November 2024, the paper reports cumulative returns of 127.91% on NIFTY stocks versus 37.61% for the index, and 49.56% versus 23.23% on DJIA, with the best Sharpe and Calmar ratios among all compared strategies. If those results hold, a guided exploration channel is a genuine addition to DRL portfolio management rather than a speculative extra.","feed_headline":"FinXplore: a second AI agent scouting new assets claims 128% returns","feed_subtitle":"On NIFTY and DJIA backtests, FinXplore beats one-agent and fixed-universe portfolios on return and risk-adjusted gain.","key_machinery":"The central mechanism is Algorithm 1's accept/reject loop with Sharpe-ratio gating. At each rebalancing period, Agent 1 produces portfolio weights; Agent 2 proposes an asset; the framework re-optimizes weights with κ=10% of wealth reserved for the proposed asset, computes the 60-day Sharpe ratio before and after, and keeps the new asset only if the ratio rises. The same ΔSR is the reward that trains the DQN agent, while the PPO agent is trained on the Sharpe ratio of the chosen portfolio. The Sharpe-ratio gate is what converts open-ended exploration into targeted diversification, and the rolling 60-day window plus a 0.05% transaction cost defines how the environment evaluates each action.","core_discovery":"The discovery claim is that exploitation and exploration can be cleanly separated into two cooperating deep-RL agents, and that the collaboration improves the portfolio on both raw and risk-adjusted terms. Agent 1 (PPO) learns how to allocate wealth among the existing n assets; Agent 2 (DQN) learns which asset from the extended universe to propose. The link between them is the reward difference ΔSR = SR_new − SR_current, so Agent 2 is trained to find assets that improve the risk-adjusted portfolio, and Agent 1's acceptance rule (keep the new asset only when SR_new > SR_current) makes the allocation adaptive to changing markets. The reported experiments are intended to demonstrate that this d","pith_inferences":["A corrected experiment that computes both Sharpe ratios only from data available before the rebalance would settle whether the reported edge survives without look-ahead; this is the first test to run before adopting the strategy.","The exploration action space could be widened from five commodities to a large pool of ETFs or sectors, with the DQN learning a ranking over candidates rather than a fixed choice; the ΔSR reward supports that generalization directly.","The fixed κ=10% exploration budget is an obvious place to make the framework adaptive: a meta-controller could raise the budget in regimes where accepted proposals consistently improve Sharpe and lower it during turbulent quarters.","A practical deployment would need rebalancing-frequency and transaction-cost sensitivity analysis; daily rebalancing at 0.05% cost may not be realistic once the exploration channel is added."],"forward_implications":["On the paper's reported results, FinXplore beats every benchmark on cumulative return, annualized return, Sharpe ratio, and Calmar ratio on both the NIFTY and DJIA datasets.","The gap between FinXplore and the DRL agent without exploration (127.91% vs 93.86% on NIFTY; 49.56% vs 37.15% on DJIA) indicates that the exploration channel, not the choice of PPO alone, drives most of the improvement.","The Sharpe-ratio acceptance rule keeps the framework from being forced into bad suggestions: it only allocates the 10% exploration budget when the proposed asset improves the portfolio's risk-adjusted return.","The quarterly-return plots show that exploration can produce worse quarters as well as better ones, so the framework's edge is statistical over the full period rather than uniform across regimes."],"supporting_citations":[{"why":"Defines the mean-variance optimization approach that serves as a classic baseline in the experiments.","marker":"[1]"},{"why":"Supplies one of the deep-RL portfolio methods used as a baseline in the backtests.","marker":"[7]"},{"why":"Supplies the second deep-RL portfolio baseline with augmented asset movement prediction states.","marker":"[11]"},{"why":"Establishes the idea of exploring a new risky asset in an extended investment universe, which FinXplore extends from mean-variance to deep RL.","marker":"[13]"},{"why":"Supplies the market environment design—state space, action space, and reward setup—that the FinXplore environment is built on.","marker":"[14]"},{"why":"Defines the Sharpe ratio, the risk-adjusted performance metric used both as the reward and as the acceptance criterion.","marker":"[15]"},{"why":"Supplies the 0.05% transaction-cost assumption and the hyperparameter ranges used in the experiments.","marker":"[16]"},{"why":"Defines the PPO algorithm used by Agent 1 for asset allocation in the existing universe.","marker":"[17]"},{"why":"Defines the deep Q-learning algorithm used by Agent 2 for proposing assets from the extended universe.","marker":"[18]"}],"fun_headline_variants":["Two-agent DRL balances investments and discovers new ones","FinXplore beats one-agent portfolios with dual AI roles","AI framework explores new assets while optimizing existing","Adaptive DRL: two agents for risk-adjusted gains","FinXplore: split AI to exploit and explore markets"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The load-bearing premise is that the agent can know the realized returns of the period it is about to rebalance: Algorithm 1 accepts a newly proposed asset only when SR_new, computed from the latest 60 portfolio returns, exceeds SR_current, which means the decision uses the same period's outcome to choose that period's trade. If a backtest is instead forced to act only on information available before the trade, this acceptance rule may no longer support the reported returns.","fun_headline_variants_meta":{"raw":{"variants":["Two-agent DRL balances investments and discovers new ones","FinXplore beats one-agent portfolios with dual AI roles","AI framework explores new assets while optimizing existing","Adaptive DRL: two agents for risk-adjusted gains","FinXplore: split AI to exploit and explore markets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000545,"raw_usage":{"total_tokens":2416,"prompt_tokens":690,"completion_tokens":1726,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":434,"completion_tokens_details":{"reasoning_tokens":1661}},"tokens_in":434,"tokens_out":1726,"duration_ms":14549,"temperature":1.0,"reasoning_tokens":1661,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T05:42:13.338757+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the NIFTY and DJIA backtests with Algorithm 1's acceptance test computed on Sharpe ratios that end before the rebalance (for example, through t−1 only, or via a walk-forward procedure that retrains the agents without touching the test period). If the corrected FinXplore does not beat the without-exploration agent and the market index by the reported margins, the superior performance is an artifact of look-ahead rather than of exploration.","supporting_citations":[{"cited_title":"Portfolio selection,","cited_arxiv_id":null,"evidence_quote":"Defines the mean-variance optimization approach that serves as a classic baseline in the experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the second deep-RL portfolio baseline with augmented asset movement prediction states."},{"cited_title":"Portfolio selection with exploration of new investment opportunities","cited_arxiv_id":null,"evidence_quote":"Establishes the idea of exploring a new risky asset in an extended investment universe, which FinXplore extends from mean-variance to deep RL."},{"cited_title":"FinRL-Meta: Market environments and benchmarks for data-driven financial reinforcement learning","cited_arxiv_id":null,"evidence_quote":"Supplies the market environment design—state space, action space, and reward setup—that the FinXplore environment is built on."},{"cited_title":"The sharpe ratio. Streetwise–the Best of the Journal of Portfolio Management","cited_arxiv_id":null,"evidence_quote":"Defines the Sharpe ratio, the risk-adjusted performance metric used both as the reward and as the acceptance criterion."},{"cited_title":"Dynamic Rein- forced Ensemble using Bayesian Optimization for Stock Trading","cited_arxiv_id":null,"evidence_quote":"Supplies the 0.05% transaction-cost assumption and the hyperparameter ranges used in the experiments."}],"review_version":1}