{"id":"77690e54-3b30-4fde-956b-1e5229f169e6","arxiv_id":"1908.08036","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"PPO and DQN produced mixed one-month backtest results on three forex pairs, but the evidence is too weak to claim reinforcement learning is a validated trading approach.","lead":"The authors tested two reinforcement learning algorithms, DQN and PPO, as foreign exchange trading agents, encoding price history as images. They report one month of backtest results on three currency pairs, but the study is too limited and under-specified to support its feasibility claim.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported PPO-over-DQN advantage may be an artifact of evaluating stop-loss/stop-gain triggers on 4-hour closing prices instead of actual intra-bar extremes, so the central feasibility claim is not established.","rationale":"The reader identified the same weakest assumption: the simulator only tests stop orders on closing prices. I agree this is the most load-bearing concern because the entire learning loop—action outcomes, reward computation, and comparative performance—depends on whether stop-loss and stop-gain points are triggered correctly. A Martingale-style strategy with small stop distances on 4-hour bars is exactly the regime where intra-bar highs and lows matter most. If the trigger logic is wrong, the PPO-versus-DQN result could be an artifact of the approximation rather than a property of the algorithms. Additional issues such as missing transaction costs, the identical constant-agent profit of 753 for EUR/USD and GBP/USD, and the apparent EUR/USD versus AUD/USD label inconsistency in Table I would still need attention, but the intra-bar trigger question is the single decisive gap. The proposed test would settle whether the concern lands; however, even if the results are robust under intra-bar triggers, the absence of costs, repeated trials, and code/data would still prevent acceptance. Hence the reader's REJECT verdict remains appropriate.","tokens_in":4221,"tokens_out":2364,"duration_ms":28132,"concrete_test":"Replay the same 1–31 December 2018 backtest for all six models (PEU, DEU, PGU, DGU, PAU, DAU) using the same trained policies, but trigger stop-loss and stop-gain orders whenever the 4-hour bar's high or low crosses the threshold, rather than only when the closing price does. Compare the resulting net profit, profit factor, and max drawdown with Table II. If any reported value changes materially, or if the PPO-over-DQN ordering reverses on any currency pair, the closing-price approximation is load-bearing and the central claim fails as stated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that PPO reliably optimizes a Sure-Fire/Martingale-style forex strategy and outperforms DQN. That claim depends entirely on the backtest in Table II correctly identifying when stop-loss and stop-gain orders are triggered. Section III.2 explicitly states: 'the closing price is used as the data point rather than considering whether the transactions are triggered at a high point or low point in the data.' For 4-hour bars and stop distances of only 20–30 pips, intra-bar high/low movements routinely cross these thresholds even when the bar closes inside them. If the simulator only checks closing prices, it will both miss real stop-outs and falsely keep positions open, which changes the number of additional buy-ins, the realized profit, and the reward signal used by PPO/DQN. Since the reward is defined as Profit times a discount depending on the number of buy-ins, a biased trigger simulator can systematically change which policy looks optimal. Therefore the Table II comparison of PPO versus DQN, and the feasibility conclusion, rest on an untested and likely false approximation of trade execution.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a deep reinforcement learning system for foreign exchange trading. Price series over 4-hour bars are encoded as Gramian Angular Field (GAF) heat maps and fed to a CNN, which selects among three discrete actions that parameterize a Sure-Fire/Martingale-style strategy: the maximum number of additional buy-ins (1, 2, or 3), the initial trade direction (buy or sell), and the stop-gain distance (20, 25, or 30 pips). The reward is net profit multiplied by a discount factor that penalizes additional buy-ins. The system is trained on EUR/USD, GBP/USD, and AUD/USD from 1 August 2018 to 30 November 2018 and evaluated on December 2018. The paper compares DQN, PPO, and a constant baseline on net profit, profit factor, and max drawdown, concluding that PPO-based trading is feasible and outperforms DQN.","tokens_in":4421,"tokens_out":3420,"duration_ms":70102,"significance":"If the results were reliable, the paper would demonstrate a practical way to use deep RL with GAF-encoded states to tune a Martingale variant, with PPO outperforming DQN. The paper has some strengths: it includes a constant baseline, it is transparent about the execution approximation in the backtest, and it performs a simple out-of-sample split. However, the central claims rest on a single month of test data and on a closing-price-only trigger simulation that is not appropriate for stop-loss/stop-gain orders, so the significance of the findings is currently not established.","major_comments":[{"comment":"The backtest triggers stop-loss and stop-gain by checking only the 4-hour closing price, explicitly ignoring intra-bar high and low points. With stop distances of only 20–30 pips, intra-bar price extremes routinely cross these thresholds even when the bar closes inside them. The assertion that 'time scales smaller than one day rarely contained stop-loss and stop-gain points' is unsupported and likely false for 4-hour bars. This approximation can both miss real stop-outs and falsely keep positions open, which changes the number of additional buy-ins, the realized profit, the drawdown, and the reward signal. Since the paper's central claim is that PPO reliably optimizes the strategy, Table II and the feasibility conclusion are not established by the reported experiments.","section":"III.2 (Trade Environment Settings)"},{"comment":"The evaluation period is a single month (1–31 December 2018) with no error bars, no multiple random seeds, and no statistical significance tests. Observed differences such as CGU (753) versus PGU (763) are well within the noise expected for a single month of trading data, so the conclusion that PPO is better than DQN is not supported. Moreover, the paper claims 'quite feasible' performance while its own data show that on EUR/USD the constant baseline (CEU, net profit 753) outperforms both DQN (DEU, 486) and PPO (PEU, 615). A feasibility claim should address this direct counterexample rather than only emphasizing the profit factor and drawdown.","section":"IV, Table II"},{"comment":"The reward is defined as Profit × Discount, where Discount = 1.0 − 0.1 × (number of additional buy-ins). The number of additional buy-ins is determined by the same closing-price-only trigger simulator that is used in the backtest. Therefore, the reward signal that trains DQN and PPO is biased by the flawed execution model: the agent is optimized to maximize profit under a simulator that both misses real stop-outs and postpones buy-in triggers. The reported PPO-over-DQN advantage could be an artifact of this simulator rather than a genuine property of the reinforcement learning algorithms.","section":"III.3 (Reward Design) and III.2"},{"comment":"The experimental setup is not reproducible: the paper does not specify the CNN architecture, the number of layers, learning rates, discount factors, exploration schedules, minibatch sizes, or any hyperparameters for DQN or PPO. The only architecture description is 'a CNN is used as the policy network or Q-network.' Without these details, another researcher cannot re-run the comparison, and the claimed performance difference between PPO and DQN cannot be independently verified.","section":"III (Experiment Model Design)"}],"minor_comments":[{"comment":"In the rightmost column, DAU and PAU are labeled as 'EUR/USD'; based on the surrounding text and the CAU entry, these should be 'AUD/USD'.","section":"Table I"},{"comment":"The citations appear as '[ ?]' in the DQN and GAF subsections; these placeholder references must be completed before publication.","section":"II.A and II.C"},{"comment":"There is a typo 'Oue experiment environment designs' that should read 'Our experiment environment designs.'","section":"III (opening line)"},{"comment":"The phrase 'favorable investment performance is achieved as long as the model is able to handle complex and random processes and the state is able to describe the environment' is too vague to be falsifiable; it should be replaced by a concrete operational criterion tied to the reported metrics.","section":"Abstract and Conclusion"}],"recommendation":"reject","confidential_remarks":"The central flaw is the execution model in Section III.2, which the authors themselves state: only closing prices are used to decide whether stop-loss and stop-gain orders are triggered. For a strategy with 20–30 pip stops and 4-hour bars, this is not a minor approximation; it invalidates the reward signal and the performance comparison. Combined with the one-month test period and missing hyperparameters, the findings cannot support the paper's conclusions. The placeholder citations and table labeling errors also suggest the manuscript is not ready for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You asked about Tsai and Wang's Deep RL for Foreign Exchange Trading. The short version: this is a routine application of DQN and PPO to a Martingale-style FX strategy, and the empirical claim is not supported by the evidence they present. The one genuinely novel piece is the hand-designed action space for the Sure-Fire strategy (max buy-ins, first direction, stop-gain level) plus the reward discount that penalizes additional buy-ins. That is a reasonable engineering formulation, and I'll give them credit for including a constant baseline and reporting profit factor and max drawdown. They are also explicitly honest about their main simplification, which is more than many backtest papers do.\n\nThe soft spots are serious, and the one flagged in the stress test is the load-bearing flaw. Section III.2 says they use the closing price for each 4-hour bar and do not consider whether stop-loss or stop-gain triggers actually occurred at intra-bar highs or lows. With 4-hour bars and stop distances of 20–30 pips, intra-bar moves will frequently cross those thresholds even when the bar closes inside them. That means the simulator both misses real stop-outs and keeps positions open that should have been closed. Because the reward is Profit times a discount based on the number of buy-ins, a biased trigger simulator can systematically change which policy looks optimal. So the Table II comparison between PPO and DQN, and the conclusion that PPO is feasible and better, rest on an untested and likely false execution model. The paper's own admission of this makes the problem worse, not better: it is a stated limitation that directly undercuts the headline result.\n\nOther issues reinforce the rejection. The test window is a single month, there are no error bars or statistical tests, and the constant baseline is not described clearly. They dismiss transaction costs and slippage by saying the minimum 20-pip earn is greater than the spread and fee, but that is hand-waving without any cost data. There is also a typo in Table I: DAU and PAU are labeled EUR/USD instead of AUD/USD. And the citation pattern is thin: no citation for DQN or GAF, with placeholders left in the text. None of these alone would kill a good paper, but together they paint a picture of a preliminary engineering report, not a scientific study.\n\nThe stress-test note is correct and useful. The central feasibility claim is not established by the current evidence. This is not a case where external skepticism is overreaching; the paper itself contains the key limitation.\n\nWho is this for? Someone exploring RL for trading might get value as a cautionary example of backtest pitfalls, particularly the danger of using close-only trigger checks. But it does not deserve a serious referee. I would desk-reject it: the load-bearing flaw is acknowledged internally, the results are statistically void, and the contribution is a narrow application of known methods. If the authors ever fix the execution model, add proper out-of-sample testing with error bars, and release code and data, it could be worth a second look.\n\nRecommendation: reject, do not send to peer review.","headline":"A routine RL-for-trading application whose central PPO-vs-DQN claim rests on a backtest that checks stop triggers only at 4-hour closes, which the authors themselves admit; the feasibility result is therefore not established.","tokens_in":4931,"tokens_out":2044,"would_cite":false,"duration_ms":21303,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a PPO reinforcement-learning agent outperforms a DQN agent at optimizing a Martingale-style foreign-exchange trading strategy on three currency pairs.","keywords":["reinforcement learning","foreign exchange trading","Proximal Policy Optimization","Deep Q-Network","Gramian Angular Field","Martingale strategy","time series encoding","trading strategy optimization"],"falsifier":"Use the same trained agents and the same December 2018 window, but trigger stop-loss and stop-gain orders with each four-hour bar's high and low instead of its close; if PPO's net profit falls below the constant baseline or its drawdown exceeds the reported values, the feasibility claim fails.","tokens_in":4012,"feed_emoji":"📈","tokens_out":7383,"duration_ms":70897,"temperature":0.7,"pith_summary":"This paper tries to establish that deep reinforcement learning is a workable way to run a foreign-exchange trading strategy, and specifically that the Proximal Policy Optimization (PPO) algorithm performs better than Deep Q-Network (DQN) on the same trading task. The agent's goal is to optimize a Martingale-style 'Sure-Fire' arbitrage policy by choosing how many extra buy-ins to allow and where to place stop-gain orders. Prices are fed to a CNN as heat-map images produced by the Gramian Angular Field encoding of four-hour OHLC bars. On a one-month test across EUR/USD, GBP/USD, and AUD/USD, the authors report that PPO models produced positive returns with lower or comparable drawdowns, and they conclude that RL-based optimization of forex strategies is feasible. The practical appeal is that the agent learns from rewards and interactions rather than from predicting unstable price trends.","feed_headline":"PPO beats DQN in a one-month forex trading backtest","feed_subtitle":"In the December 2018 backtest, PPO earned higher net profit than DQN on all three currency pairs.","key_machinery":"The load-bearing machinery is the pairing of a Gramian Angular Field state encoder with a reward that discounts profit by the number of additional buy-ins. GAF turns a 12-bar sliding window of four-hour opening, high, low, and closing prices into a $12 \\times 12 \\times 4$ heat-map tensor, so a CNN can treat price history as an image. The reward is $\\text{Profit} \\times (1.0 - 0.1 \\times \\text{number of additional buy-ins})$, which tells the agent that every extra Martingale layer costs expected return and should be avoided unless necessary. The action set, covering the cap on additional buy-ins, the initial buy/sell direction, and the stop-gain level, defines the strategy space the agent is optimizing. This design lets the PPO/DQN comparison isolate the effect of the learning algorithm itself.","core_discovery":"The paper's central claim is that a PPO-based agent can reliably optimize the Sure-Fire Martingale trading policy, and that PPO outperforms DQN when both are given the same Gramian Angular Field encoded state. In the December 2018 evaluation, the PPO models earned net profits of 615 on EUR/USD, 763 on GBP/USD, and 597 on AUD/USD, compared with DQN's 486, 717, and 402; on GBP/USD and AUD/USD, PPO also beat the constant baseline, while on EUR/USD it earned less but with a much lower drawdown. The authors attribute the gap not to higher risk but to better timing: PPO made profitable decisions earlier and with lower drawdown than DQN. The intended conclusion is that reinforcement learning is feasible for forex strategy development as long as the state representation captures the environment and the reward is shaped to discourage excessive Martingale buy-ins.","pith_inferences":["The reported results depend on a trigger-detection shortcut: using each bar's closing price to decide whether stops were hit. A natural next experiment is to re-run the backtest with intra-bar high/low trigger detection; the one-month profits would likely shrink, though the PPO-versus-DQN ranking might survive.","Because the test period is a single month, the claimed superiority of PPO may be regime-specific; extending the same state and reward design across years and market conditions would test whether the finding generalizes.","The same GAF-plus-discounted-reward recipe could be applied to other assets or to Martingale variants beyond the three-step ladder described here.","If this feasibility result holds, it suggests that for strategies with known structural rules, RL can tune parameters such as ladder depth and stop levels that are otherwise set by hand."],"forward_implications":["A trader can treat the optimization of a Martingale-style strategy as a reinforcement-learning problem and expect the learned policy to be at least as profitable as a fixed baseline while carrying lower drawdown.","Proximal Policy Optimization should be preferred over Deep Q-Network for discrete-action trading tasks with GAF-encoded price states, because it captured profit more consistently on the currency pairs tested.","The GAF image representation is sufficient for a CNN-based agent to make profitable trading decisions from four-hour OHLC data.","Penalizing each additional buy-in in the reward is a workable way to encode risk aversion into an RL trading agent."],"supporting_citations":[{"why":"Supplies the reinforcement-learning framework the whole system is built on, including the DP/Monte Carlo/TD categorization used to introduce the algorithms.","marker":"[1]"},{"why":"Provides the policy-gradient method from which PPO is derived; the paper cites it to justify updating the policy directly from rewards.","marker":"[2]"}],"fun_headline_variants":["PPO beats DQN on all three forex pairs in backtest","Reinforcement learning forex: PPO outperforms DQN","RL forex strategy: PPO wins with lower drawdown","PPO vs DQN in forex: higher profit, lower risk","Forex RL backtest: PPO tops DQN"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that checking stop-loss and stop-gain triggers against each four-hour bar's closing price tells the same story as checking them against the bar's high and low; if intra-bar moves would have fired the stops, the reported profits and drawdowns could be materially wrong.","fun_headline_variants_meta":{"raw":{"variants":["PPO beats DQN on all three forex pairs in backtest","Reinforcement learning forex: PPO outperforms DQN","RL forex strategy: PPO wins with lower drawdown","PPO vs DQN in forex: higher profit, lower risk","Forex RL backtest: PPO tops DQN"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000334,"raw_usage":{"total_tokens":1852,"prompt_tokens":940,"completion_tokens":912,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":556,"completion_tokens_details":{"reasoning_tokens":822}},"tokens_in":556,"tokens_out":912,"duration_ms":523843,"temperature":1.0,"reasoning_tokens":822,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:59:30.925578+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Use the same trained agents and the same December 2018 window, but trigger stop-loss and stop-gain orders with each four-hour bar's high and low instead of its close; if PPO's net profit falls below the constant baseline or its drawdown exceeds the reported values, the feasibility claim fails.","supporting_citations":[],"review_version":1}