{"id":"ddc7e587-bbc8-44d0-b0ae-4cf0219d657f","arxiv_id":"2501.04068","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A DRQN reinforcement learning agent selects F1 tire and pit strategies and achieves P5.33 average finishing position on a simulated 2023 Bahrain GP, beating a fixed-strategy baseline and a Mercedes SOTA model.","lead":"This paper trains a deep reinforcement learning model, RSRL, to choose tire compounds and pit stop timing in Formula One race simulations, and it reports a slightly better average finishing position than two baseline strategies on a simulated 2023 Bahrain Grand Prix. It adds explainability tools to show why the model makes each decision, aiming to make the approach usable by real race strategists.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"P5.33 headline is selected on the test race with no confidence intervals; the reported advantage may be a model-selection artifact rather than a robust property.","rationale":"The reader's weakest assumption centered on proprietary simulator fidelity, which is a legitimate external-validity concern. My concern is more specific and more directly fatal to the paper's central quantitative claim: the P5.33 figure is produced by selecting the best-performing model on the test race itself, with no held-out evaluation and no confidence intervals. This selection bias inflates the apparent margin over fixed baselines and is compounded by the absence of variance reporting. The internal claim — that RSRL beats Fixed Strategy and Mercedes SOTA in the simulator — is falsifiable within the simulator, so the load-bearing issue is test-set selection and missing statistics, not simulator realism. I partially agree with the reader because they did mention 'selecting the best model on the test race' and 'no error bars,' but they gave more weight to simulator fidelity. The paper has genuine positives: a modular architecture, comparison against an external SOTA baseline, and a generalisability study spanning 14 tracks. However, the generalisability table itself undermines the Bahrain headline, since all three RSRL variants in Table 3 perform worse than baselines on Bahrain. A pre-registered, seed-averaged evaluation with confidence intervals would settle whether the headline effect is real; the reader's CONDITIONAL verdict already requires essentially this, so no verdict change is needed.","tokens_in":13163,"tokens_out":3278,"duration_ms":31875,"concrete_test":"Pre-register the Section 4.1 evaluation: fix the architecture and hyperparameters, train K=10 RSRL instances on Bahrain with different random seeds, and evaluate every instance on 1920 fresh simulations per seed. Report the mean and 95% bootstrap confidence interval of finishing position for each instance, for the ensemble mean, and for the best instance selected on a separate validation set of simulations. The headline stands only if the validation-selected model (or the average instance) beats Fixed Strategy's P5.63 with a confidence interval excluding zero; if the best-of-K on the test set is no longer best on a holdout set, the current P5.33 is a selection artifact.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing concern is not simulator fidelity per se but that the headline comparison in Section 4.1 is statistically uncontrolled. The authors state that 'through testing, we found that the best-performing version of RSRL for the 2023 Bahrain Grand Prix achieved an average finishing position of P5.33.' This means the test race was used to select the model or hyperparameters, so P5.33 is the maximum of a set, not an unbiased estimate. By contrast, Fixed Strategy (P5.63) and Mercedes SOTA (P5.86) are single fixed baselines. Selection on the test set inflates the apparent advantage; with unknown simulation variance and no confidence intervals, a 0.30-position gap over 1920 simulations cannot be distinguished from noise or from multiple-comparison luck. Supporting evidence: Table 3 reports three separately trained RSRL models, all of which finish worse than both baselines on Bahrain (9.62, 6.47, and 6.45 versus 5.63 and 5.86), making P5.33 an outlier rather than a robust property of the method. The internal claim that RSRL outperforms the baselines therefore rests on a single, possibly optimistically selected number.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces RSRL, a DRQN-based reinforcement learning agent for Formula One race strategy (tyre compound selection and pit-stop timing), trained inside a proprietary Monte Carlo race simulator provided by Mercedes-AMG PETRONAS F1 Team. The central empirical claim is that RSRL achieves an average finishing position of P5.33 on the 2023 Bahrain Grand Prix over 1920 simulations, outperforming a Fixed Strategy baseline (P5.63) and Mercedes' SOTA model (P5.86). The paper also presents a generalisability study across 14 tracks for models trained on 1, 2 and 9 tracks, and supplements the policy with three XAI techniques: TimeSHAP feature importance, VIPER decision-tree surrogates, and decision-tree counterfactuals.","tokens_in":13437,"tokens_out":4080,"duration_ms":41188,"significance":"If the reported results are statistically sound, the paper would be a useful applied contribution: it demonstrates a portable RL architecture for real-time race strategy, evaluates against an industry black-box baseline, and combines policy learning with several XAI tools in a domain where such explanations are directly relevant to human strategists. The system-architecture abstraction and the generalisability experiment across multiple tracks are constructive strengths. However, the headline advantage is currently supported only by a point estimate obtained after testing several model versions on the same race, with no uncertainty quantification; the result is not yet established as a robust property of the method.","major_comments":[{"comment":"The main comparison is not statistically controlled because the test race is used for model selection. The sentence 'Through testing, we found that the best-performing version of RSRL for the 2023 Bahrain Grand Prix achieved an average finishing position of P5.33' indicates that P5.33 is the best of several evaluated variants, whereas the Fixed Strategy and Mercedes SOTA baselines are single fixed models. Table 3 shows other RSRL instances (RSRL-1, RSRL-2, RSRL-9) finishing Bahrain at 9.62, 6.47 and 6.45, all worse than both baselines, so the reported advantage is an outlier rather than a robust property. The manuscript should select the model on a validation split, or report all evaluated variants with a multiple-comparison correction, and then evaluate the selected model on a held-out race.","section":"§4.1"},{"comment":"All headline results are point estimates without any measure of dispersion or uncertainty. The 1920-simulation comparison in §4.1 and the approximately 900-simulation-per-track comparisons in Table 3 report only average finishing positions; no standard deviations, confidence intervals, or significance tests are given. Given the stochastic simulator, a 0.30-position gap (P5.33 vs P5.63) or a 0.53-position gap (P5.33 vs P5.86) cannot be distinguished from simulation noise. The same issue affects the generalisability claims, e.g. 'RSRL-1 and RSRL-2 finish on average 0.93 and 1.77 positions ahead of the SOTA model respectively when tested on their training tracks.' The authors should provide bootstrap confidence intervals or paired permutation tests, and report per-model distributions rather than only means.","section":"§4.1 and §4.2"},{"comment":"The XAI fidelity evaluation does not support the stated conclusions. For TimeSHAP, the MAE of 124.39 is normalized by the maximum terminal reward of 2500 to claim '5% normalised MAE', but the intermediate rewards are mostly 1 or -10, so this normalization is not meaningful unless the error is measured in reward units or against a baseline such as constant/random attribution. For VIPER, the 0.926 accuracy and 0.910 F1-score are computed on 100 simulations with a strongly imbalanced label distribution (4699 no-pit true labels in Table 4), so per-class precision and recall are needed to assess fidelity. The counterfactual 'fidelity' measure, which counts the number of feature changes, is a proximity measure and does not directly measure how closely the counterfactual follows the model's decision boundary; this should be clarified or replaced with a decision-boundary-based check.","section":"§4.3"},{"comment":"The reward function in the displayed equation does not match the prose description. The text says a penalty of -10 is given for 'extraneous pitstops beyond the first valid pitstop,' but the equation applies -10 whenever a_t is a pit action and s_vf is true, without tracking whether a valid pitstop has already occurred. Since this reward affects the learned policy and hence the performance claim, the definition should be corrected or clarified, and the implementation should be checked against the intended semantics.","section":"§3.1"}],"minor_comments":[{"comment":"Table 1 lists the ranges of Tyre Degradation, Gap Ahead, Gap Behind, and Gap to Leader as '?', leaving the scaling bounds unspecified; the authors should provide the actual ranges used in the custom scaling functions.","section":"§3.1 / Table 1"},{"comment":"The text says 'The Lap Number plot is the most influential,' but the state space in Table 1 does not contain a Lap Number feature; the relevant feature appears to be Race Progress. The caption and the text should be made consistent.","section":"§4.3 / Figure 4"},{"comment":"There is a typo in 'we would like to to improve strategy predictions'; the duplicated 'to' should be removed.","section":"§5"},{"comment":"Reference [7] is cited as 'Molnar and Dandl's [7]' but the reference list entry is authored by Dandl, Molnar, Binder, and Bischl; the in-text citation and reference entry should be aligned.","section":"§2 / Reference [7]"},{"comment":"The statement 'there are considerably more no pit decisions being taken through the 100 simulations' is imprecise; it should say that the simulated trajectories contain many more no-pit timesteps than pit timesteps, which explains the class imbalance in the confusion matrix.","section":"§4.3"}],"recommendation":"major_revision","confidential_remarks":"The paper's empirical claims depend on a proprietary simulator, which limits external reproducibility, but this alone is not a reason for rejection. The decisive issue is the lack of statistical control: the headline P5.33 appears to be selected on the test race, and no uncertainty measures are reported anywhere. If the authors can provide a proper train/validation/test split, confidence intervals, and corrected XAI fidelity metrics, the paper could become acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the good. The paper does something that hasn't been done before in F1: a DRQN controller for pit strategy, with a clean abstraction layer that could port to live data, and it's evaluated against two independent baselines, including Mercedes' own SOTA model. The generalisability study across 14 tracks is a useful empirical map of the trade-off between specialization and generalization. The XAI section is honest descriptive work, not overclaimed.\n\nThe soft spot is the load-bearing one: the P5.33 headline is the result of 'the best-performing version of RSRL for the 2023 Bahrain Grand Prix.' In other words, the test race was used to select the model. That makes P5.33 the max of a set, not an unbiased estimate. Table 3 makes the problem concrete: three other trained RSRL models all finish worse than both baselines on Bahrain (9.62, 6.47, 6.45 vs 5.63 and 5.86). So the claim that RSRL beats the baselines rests on a single, possibly lucky checkpoint. There are no error bars, no significance tests. The 0.30-position improvement over Fixed Strategy could easily be noise or selection luck. This is the key issue; the simulator fidelity concern is secondary.\n\nWhat holds up: the external baselines are real, the problem formalization is careful, and the authors are upfront about the black-box simulator. The XAI fidelity metrics are descriptive but not misleading.\n\nThis paper deserves a serious referee. It's a solid applied-RL case study with a real industry connection. But the evaluation needs a proper hold-out protocol, multiple seeds, and confidence intervals before the quantitative claim can be taken at face value. I'd ask for a revision, not a rejection.","headline":"A genuine applied-RL contribution with real external baselines, but the headline number is selected on the test set and the statistical reporting is too thin to back the central claim.","tokens_in":14002,"tokens_out":2525,"would_cite":false,"duration_ms":25157,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims a lap-by-lap reinforcement learning agent can outperform Mercedes' own strategy model in the team's race simulator.","keywords":["reinforcement learning","explainable AI","Formula One","race strategy","deep recurrent Q-network","Monte Carlo simulation","tyre degradation","counterfactual explanations"],"falsifier":"Record the finishing position of each of the 1920 simulations and run a permutation or bootstrap test comparing RSRL's mean P5.33 against Mercedes SOTA's P5.86; if the difference is not statistically significant, the central performance claim reduces to noise. A complementary check is to replay RSRL's learned strategies in a different simulator or on real historical race data and compare their finishing positions with the baselines.","tokens_in":12970,"feed_emoji":"🏎️","tokens_out":8996,"duration_ms":80602,"temperature":0.7,"pith_summary":"The paper claims that a reinforcement learning agent, RSRL, can decide lap by lap whether to pit and which tyre compound to fit, and that this agent performs better than two established alternatives in a Formula One race simulator. On the 2023 Bahrain Grand Prix, with a car whose pace corresponds to an expected finish of P5.5, RSRL averages P5.33 over 1920 simulations, compared with P5.63 for the Fixed Strategy model and P5.86 for the state-of-the-art model supplied by the Mercedes team. The authors argue that the improvement is meaningful because the simulator's stochasticity and the car's pace make consistently finishing near the front impossible. They also report that training on one or two tracks produces excellent results on those tracks but transfers poorly, while training on nine tracks transfers better to unseen tracks at a cost on the training tracks. The paper adds explanation tools to make the agent's strategies inspectable.","feed_headline":"RL agent averages P5.33, beating F1 baselines","feed_subtitle":"The RL agent scored better average results over 1,920 Bahrain simulations than Mercedes' own model.","key_machinery":"The central object is the RSRL agent, a deep recurrent Q-network (DRQN) that uses a recurrent neural network to incorporate information from previous laps when predicting Q-values, which is what lets it handle partial observability such as whether it is catching or losing ground to the car ahead. The agent is connected to the simulator through an abstraction layer: custom UnifiedRaceState and UnifiedRaceStrategy classes with a translator convert the simulator's proprietary data into a fixed state representation, so the same agent can be trained on the Monte Carlo simulator and deployed on other data sources. The reward function is terminal-position based, giving 100 times the F1 points for the eventual finish, with a penalty for extra pit stops and a large penalty for illegal actions, which lets the agent learn purely from race outcomes without intermediate reward shaping.","core_discovery":"RSRL is a deep recurrent Q-network that treats each lap of a race as a decision step. At each step the available actions are no pit stop, pit for soft tyres, pit for medium tyres, or pit for hard tyres; the reward is mostly terminal, equal to 100 times the Formula One points for the final finishing position, with penalties for extra pit stops and heavy penalties for invalid actions. Trained inside the team's black-box Monte Carlo simulator, the agent learns to select strategies that lift a car with P5.5 expected pace to an average P5.33 finish on the Bahrain 2023 test race, ahead of both baselines. The paper's generalisability study shows that the number of training tracks controls a trade-off between performance on known circuits and robustness on unseen circuits.","pith_inferences":["A natural next step the paper leaves implicit is to report the variance across the 1920 simulation seeds; without a confidence interval or significance test, the 5.33 vs 5.63 vs 5.86 ordering could be within sampling noise.","If the result holds, the same DRQN-plus-abstraction design should transfer to other tyre-limited racing series such as Formula E or endurance racing, because their strategy problems share the same action structure of when to pit and which compound to fit.","The fact that RSRL's counterfactual for an early soft-tyre stop lands on lap 14, the lap where the actual 2023 Bahrain winner pitted, suggests the learned policy may track real-world optimal behaviour; this could be checked by comparing RSRL's strategies with the actual winning strategies of other grands prix."],"forward_implications":["A team using RSRL could choose strategy online, one lap at a time, instead of committing to a small set of pre-computed candidate strategies and evaluating them with Monte Carlo simulation.","The P5.33 average over 1920 simulations is better than the P5.63 of the Fixed Strategy baseline and the P5.86 of the Mercedes SOTA model, so the agent extracts more from a car with P5.5 expected pace.","The generalisability study implies a practical rule: train on a small number of tracks for peak performance on a known calendar, or train on many tracks for robustness to unseen circuits, accepting a performance loss on the training tracks.","The XAI components mean a strategist can inspect why a particular lap's decision was made, test what would change the decision, and retain a decision-tree-level view of the whole policy.","The abstraction layer between the agent and the simulator means the same trained model can be deployed with live race data or a driver-in-the-loop simulator without retraining the core decision logic."],"supporting_citations":[{"why":"Supplies the deep recurrent Q-network architecture that carries the agent's temporal reasoning over laps.","marker":"[9]"},{"why":"Provides the DQN-for-race-strategy motivation and the -1000 penalty for invalid actions that RSRL adopts.","marker":"[5]"},{"why":"Defines VIPER, the imitation-learning method used to extract a decision-tree surrogate of RSRL's policy.","marker":"[2]"},{"why":"Defines TimeSHAP, the sequential feature-attribution method used for lap-level feature importance.","marker":"[3]"},{"why":"Defines the decision-tree counterfactual generation used to show minimal changes that would flip a strategy decision.","marker":"[6]"},{"why":"Prior neural-network 'virtual strategy engineer' for F1 pit decisions; the paper extends this line with RL and XAI.","marker":"[11]"},{"why":"Dynamic-programming analysis of pit-stop strategies that serves as the optimization-related baseline in related work.","marker":"[12]"}],"fun_headline_variants":["RSRL: P5.33 avg, beats P5.63 F1 baseline","Explainable RL drives F1 to P5.33 avg finish","RL pit strategies: P5.33 finish, tops P5.63 baseline","F1 RL: P5.33 avg, with decision-tree explainability","RSRL beats F1 baselines, explains its own calls"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes the Mercedes Monte Carlo simulator faithfully represents real Formula One racing, including tyre degradation, traffic, and safety-car behaviour, so a strategy that wins in simulation will also win on track.","fun_headline_variants_meta":{"raw":{"variants":["RSRL: P5.33 avg, beats P5.63 F1 baseline","Explainable RL drives F1 to P5.33 avg finish","RL pit strategies: P5.33 finish, tops P5.63 baseline","F1 RL: P5.33 avg, with decision-tree explainability","RSRL beats F1 baselines, explains its own calls"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000743,"raw_usage":{"total_tokens":3307,"prompt_tokens":931,"completion_tokens":2376,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":547,"completion_tokens_details":{"reasoning_tokens":2277}},"tokens_in":547,"tokens_out":2376,"duration_ms":15876,"temperature":1.0,"reasoning_tokens":2277,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:47:00.588395+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Record the finishing position of each of the 1920 simulations and run a permutation or bootstrap test comparing RSRL's mean P5.33 against Mercedes SOTA's P5.86; if the difference is not statistically significant, the central performance claim reduces to noise. A complementary check is to replay RSRL's learned strategies in a different simulator or on real historical race data and compare their finishing positions with the baselines.","supporting_citations":[{"cited_title":"Hausknecht and Peter Stone","cited_arxiv_id":null,"evidence_quote":"Supplies the deep recurrent Q-network architecture that carries the agent's temporal reasoning over laps."},{"cited_title":"Mastering Nordschleife -- A comprehensive race simulation for AI strategy decision-making in motorsports","cited_arxiv_id":"2306.16088","evidence_quote":"Provides the DQN-for-race-strategy motivation and the -1000 penalty for invalid actions that RSRL adopts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines VIPER, the imitation-learning method used to extract a decision-tree surrogate of RSRL's policy."},{"cited_title":"Carreira-Perpiñán and Suryabhan Singh Hada","cited_arxiv_id":null,"evidence_quote":"Defines the decision-tree counterfactual generation used to show minimal changes that would flip a strategy decision."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Prior neural-network 'virtual strategy engineer' for F1 pit decisions; the paper extends this line with RL and XAI."},{"cited_title":"Carrasco Heine and Charles Thraves","cited_arxiv_id":null,"evidence_quote":"Dynamic-programming analysis of pit-stop strategies that serves as the optimization-related baseline in related work."}],"review_version":1}