{"id":"ffc65408-c317-41ea-a0b5-f965b5679864","arxiv_id":"2412.00725","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"Across 12 Atari games, Decision Transformer tends to outperform Decision Mamba in games with larger action spaces and more complex visuals, while Decision Mamba wins in simpler games, though the statistical support is weak.","lead":"This paper compares two sequence-modeling algorithms for Atari games and tries to explain which one wins based on game features like number of actions and visual complexity. The headline finding, that simpler games favor Decision Mamba and busier games favor Decision Transformer, rests on a small and statistically fragile analysis.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed primacy of action/visual complexity rests on two high-leverage games; leave-one-out analysis is needed before feature importances can support the conclusion.","rationale":"The reader's weakest assumption—that a random forest on 12 games with post-hoc outlier removal yields unstable estimates—is correct and is the main issue, but I have identified a more specific mechanism: the target variable's variance is concentrated in Hero and KungFuMaster, so the reported feature importances likely reflect two high-leverage games. This can be tested directly with leave-one-game-out or by dropping those games. The reader's REJECT verdict is unchanged because the central claim is not supported by the evidence as presented. I do not see a reason to move the verdict; the concern strengthens the rejection rather than altering it.","tokens_in":17098,"tokens_out":3436,"duration_ms":34525,"concrete_test":"Run a leave-one-game-out analysis: for each of the 12 games, fit the same random forest on the remaining 11 games, predict the held-out game, and record feature importances. Also refit after excluding Hero and KungFuMaster (and separately excluding Breakout). If Num_Actions importance drops below roughly 0.2 or the held-out RMSE is not materially better than predicting the mean, the central claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (abstract, §4.2.1, §6) is that action-space complexity and visual complexity, especially compression ratio, are the primary determinants of the DT–DM performance gap. The quantitative support is a random forest (RF) feature-importance/SHAP analysis on 12 games. The target variable y = DT−DM (Table 5) is dominated by exactly two games: Hero (+22.6) and KungFuMaster (+24.1), with Breakout as a large negative outlier (−58.0). Every other game has |y| ≤ 10.05, and most 18-action games (Seaquest, Alien, BankHeist, BattleZone, RoadRunner, FishingDerby) have y between −3.49 and +7.30. Thus Num_Actions importance of 0.444 and Compression_Ratio importance of 0.213 could arise from the RF memorizing these few extreme points rather than from a generalizable relationship. The reported 6-fold CV RMSE of 17.57 ± 12.35 is larger than most observed |y|, indicating poor held-out predictive ability; a model that cannot predict held-out games cannot support a causal 'primary determining factors' claim. The outlier-removal protocol in Table 3 (removing outliers to reduce standard deviation while retaining at least 3 seeds) also changes the y values for four games. With n=12, no significance testing, and six correlated features (Num_Actions–Image_Entropy r=0.68, Image_Entropy–Compression_Ratio r=−0.70), the feature importances are highly unstable. The conclusion therefore overreaches: the data support a hypothesis about Hero/KungFuMaster-like games, not a general claim about action and visual complexity as primary determinants.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper compares Decision Transformer (DT) and Decision Mamba (DM) on 12 Atari games, using normalized scores from offline DQN-replay data. It introduces three visual-complexity metrics (image entropy, compression ratio, feature count) and analyzes the performance gap with random forest regression, SHAP values, and Pearson correlations. The authors conclude that action-space complexity and visual complexity (especially compression ratio) are the primary determinants of the DT–DM gap, and they test two action-fusion strategies on Hero and KungFuMaster to probe the action-space hypothesis. The stated contributions are an empirical comparison, identification of influential game characteristics, and an action-space simplification method.","tokens_in":17471,"tokens_out":2969,"duration_ms":29742,"significance":"If the central claim were well supported, it would offer a practical rule for architecture selection in offline RL: measure the number of actions and frame compressibility, then choose DT or DM accordingly. The paper does provide a reproducible experimental setup, a novel set of visual-complexity metrics, and open code, which are useful building blocks. However, the quantitative evidence for the claim rests on a random forest fit to 12 games, with no significance tests, no reported R-squared, and post-hoc outlier removal that changes headline numbers. The result is therefore best read as a hypothesis-generation exercise, not a validated explanation of DT–DM differences.","major_comments":[{"comment":"The central claim that action-space complexity and visual complexity are the primary determinants of the DT–DM gap is not supported by the statistical evidence. The random forest is fitted to n=12 games with 6 features, and the reported 6-fold CV RMSE is 17.57 ± 12.35, which is larger than the magnitude of most observed performance differences in Table 5 (all but three |y| ≤ 10.05). The feature importances are thus likely driven by the three extreme games (Hero, KungFuMaster, Breakout) rather than a generalizable relationship. I recommend reporting leave-one-out predictions, permutation-based importance, and a confidence interval or significance test for the importances, or substantially tempering the causal wording in the abstract and conclusion.","section":"§4.2.1, Figure 3"},{"comment":"The outlier-removal protocol is not pre-registered and changes the headline results. For KungFuMaster at context length 10, the DT normalized score changes from 29.41 (Table 3, after outlier removal) to 19.65 (Appendix E, raw), and for Qbert at context length 30 it changes from 4.94 to 9.06. Because the paper's narrative about which model wins and by how much depends on these numbers, the analysis should present both raw and trimmed results and justify the trimming criteria. Without that, the performance gaps used in the regression and correlation analyses are not stable.","section":"Table 3 vs. Appendix E"},{"comment":"The feature set is highly collinear: Num_Actions correlates 0.68 with Image_Entropy and −0.53 with Compression_Ratio, while Image_Entropy correlates −0.70 with Compression_Ratio. With only 12 data points, random forest feature importances and SHAP values cannot reliably disentangle these correlated contributions, so the ranking in Figures 3 and 4 is not robust. I recommend reporting variance inflation factors, partial correlations, or a sensitivity analysis with orthogonalized features before drawing conclusions about which metric is 'primary.'","section":"§4.2.2, Figure 5"},{"comment":"The action-fusion experiments are limited to two games and do not isolate action-space complexity from other changes. When actions are fused, the training distribution changes and the model's effective policy class changes, so the observed performance drop in DT could be due to altered data statistics rather than to the reduced action count. The paper's conclusion that 'action space complexity alone cannot fully account' for the gaps is therefore not directly tested. A control condition, such as random action relabeling that preserves the action count, or additional games, would be needed to support that inference.","section":"§4.3.2, Tables 7–8"}],"minor_comments":[{"comment":"The text refers to 'the rest of the dissertation'; in a journal paper this should be 'paper' or 'manuscript.'","section":"§1, last paragraph"},{"comment":"The choice to use 5× the maximum return for all games is a free parameter that may differentially affect DT and DM. Please report a sensitivity analysis for at least one game, or justify why a single multiplier is fair across the 12 games.","section":"§3.3.2, Expected Return Setup"},{"comment":"The variable n_test is used in the MSE formula but not defined; specify that it is the size of the held-out fold.","section":"Algorithm 1, line 11"},{"comment":"There is a typo: 'Random Walk scores and Human Players soirees' should be 'scores.' Also, the sentence about bias is vague; specify what bias is introduced by action fusion.","section":"§4.3.2, first paragraph"},{"comment":"The caption says 'σ is SiLU function' but does not mention the ⊗ symbol; expand the caption for readability.","section":"Figure 1 caption"},{"comment":"For Pong, the DM score has a very large standard deviation (64.31 ± 53.24), which suggests instability; the paper does not discuss this outlier in the context of the correlation results.","section":"Table 5"}],"recommendation":"reject","confidential_remarks":"The manuscript appears to be an early-stage write-up (it self-identifies as a 'dissertation' in Section 1) and reads more like a project report than a journal paper. The core issue is sample size: with 12 games, 6 correlated features, and no inferential statistics, the central claim about 'primary determining factors' cannot be supported. While the data and metrics could be a useful resource, the analysis would need substantially more games, proper significance testing, and a pre-specified outlier protocol to be publishable as stated. I would encourage the authors to either expand the empirical study considerably or reframe the paper as a descriptive case study with explicitly tentative conclusions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Hi — quick take on arXiv:2412.00725. The useful core is the extended comparison: DT and DM on 12 Atari games with a table of game characteristics (actions, trajectory length, image entropy, compression ratio, feature count). That data is a real contribution; anyone working on sequence-model RL will want to know how these two architectures split on a broader game set than the original papers. The action-fusion experiments are also a nice idea — they try to isolate action-space complexity by fusing actions while preserving the full action set at evaluation, and the results are suggestive (DT loses much of its edge in Hero and KungFuMaster when the action space is simplified). I'd call that a modest but genuine addition.\n\nThe soft spot is exactly where the stress test lands. The central claim — that action-space complexity and compression ratio are the primary determinants of the DT–DM gap — comes from a random forest with 6 features fitted to 12 games. With n=12, the 6-fold CV RMSE of 17.57 ± 12.35 is larger than most observed gaps. Feature importances on this sample size are not stable; the two extreme games (Hero +22.6, KungFuMaster +24.1) dominate the target, and Breakout is a large negative outlier (−58.0). The paper reports no significance tests, no leave-one-out stability check, and the outlier-removal protocol in the initial four-game experiments changes headline numbers when compared to the raw scores in Appendix E. The features are also correlated (Num_Actions–Image_Entropy r=0.68), so the importance rankings are fragile. That doesn't make the comparison useless, but it makes the phrase \"primary determining factors\" an overreach. The data support a hypothesis about high-action, high-visual-complexity games; they don't establish a general rule.\n\nAlso, the code link says \"Github\" with no URL, so the experiments aren't directly reproducible from the paper as written. Minor: 3 seeds for extended runs is thin, though the standard deviations are reported.\n\nWho is this for? Someone comparing DT to linear-time sequence models on Atari, or looking for heuristics about when attention helps. The comparison table is worth having. The analysis section needs major revision, not rejection. A serious referee should see it: the paper is honest, the data are real, and the question — what game properties drive architecture choice — is important. My recommendation: send to review with the expectation of heavy revision, and require a stability analysis (leave-one-out, bootstrapped significance) or a softened conclusion.","headline":"Useful 12-game DT/DM comparison, but the claim that action/visual complexity are primary determinants is not supported by a 12-game random forest without significance tests.","tokens_in":17947,"tokens_out":3599,"would_cite":true,"duration_ms":30694,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Across 12 Atari games, the gap between Decision Mamba and Decision Transformer is driven mainly by two cheap game statistics: the number of actions and the compressibility of game frames.","keywords":["Decision Transformer","Decision Mamba","offline reinforcement learning","Atari games","visual complexity","action space complexity","compression ratio","random forest regression"],"falsifier":"A decisive check would be to decouple the two factors by modifying games: take a visually simple game such as Breakout and add visual texture without changing its 4-action set, or take Hero and keep its 18 actions while simplifying the graphics. If the DT–DM gap tracks compression ratio when action count is held fixed, the visual-complexity claim is supported; if it tracks only action count, the paper's hierarchy is wrong. A cheaper statistical falsifier is to resample the 12 games thousands of times and check whether action count remains the top random-forest feature; if the top feature changes frequently, the reported importance ranking is unstable.","tokens_in":16901,"feed_emoji":"🕹️","tokens_out":5021,"duration_ms":44391,"temperature":0.7,"pith_summary":"This paper tries to establish that the performance gap between the Decision Transformer (DT) and Decision Mamba (DM) in Atari games is driven mainly by two game properties: the size of the action space and the visual complexity of the game, measured most usefully by compression ratio. It argues that DM tends to win in visually simple, low-action environments, while DT tends to win in visually complex, high-action games. If this is right, a practitioner could choose between the two architectures based on two cheap numbers computed from the environment, rather than training both models. The paper supports the claim with random forest feature importance, SHAP values, Pearson correlations, and action-fusion experiments that manipulate the action space directly.","feed_headline":"Frame compressibility and action count predict DT vs Mamba wins","feed_subtitle":"Across 12 Atari games, Decision Mamba wins simple looks, Decision Transformer wins complex ones.","key_machinery":"The argument is carried by a specific target variable and a small feature set. The target is the performance difference $S_{\\text{DT}} - S_{\\text{DM}}$ computed from normalized scores, where 0 is a random agent and 100 is human-level play. The explanatory features are six game characteristics: number of actions, average trajectory length, average steps to the first non-zero reward, image entropy, compression ratio (uncompressed over compressed frame size, computed losslessly with zlib), and SIFT feature count. The inference machinery is a random forest regressor with 1000 trees and six-fold cross-validation, whose feature importances are reported alongside SHAP values, plus Pearson correlations and two action-space simplification interventions (simple action fusion and frequency-based action fusion) that reduce the action space while preserving all original actions during evaluation.","core_discovery":"The central discovery claimed is that, across 12 Atari games trained offline on the DQN-replay dataset, the normalized-score gap between Decision Transformer and Decision Mamba is not random: it tracks the number of actions and the compression ratio of game frames. DM generally wins on visually simple, low-action games (Breakout, Pong, Seaquest, RoadRunner, SpaceInvaders), while DT generally wins on visually complex, high-action games (Hero, KungFuMaster). The paper supports this with feature-importance and SHAP analyses that rank action count first and compression ratio second, and with action-fusion experiments in which reducing the action space of Hero and KungFuMaster shrinks DT's advantage over DM. The conclusion is that both action-space complexity and visual complexity, particularly as captured by compression ratio, are the primary determinants of the DT–DM performance gap.","pith_inferences":["A testable screening rule that the paper does not itself propose is: before training on a new Atari-like environment, compute the action count and zlib compression ratio of its frames; if both are low, expect DM to win, otherwise expect DT to win.","Because action count and image entropy correlate strongly (r = 0.68) in the paper's 12-game table, the regression may not fully separate the two factors; an experiment that manipulates visual complexity while holding action count fixed would sharpen the causal claim.","The action-fusion results suggest that DT's advantage is not purely architectural but comes from exploiting a larger action vocabulary; extending the comparison to continuous-control tasks with discrete action abstractions would test whether the pattern generalizes beyond Atari.","A direct architectural probe, such as examining DT's attention patterns and DM's selection vectors on visually simple versus visually complex frames, could reveal the mechanism behind the claimed complexity-dependent gap."],"forward_implications":["In Atari-like offline reinforcement learning, Decision Mamba is the safer choice in visually simple games with small action sets, while Decision Transformer is preferable in games with many actions and complex visuals.","Action space complexity plays a causal role: simplifying the action space of Hero and KungFuMaster via action fusion drops DT's normalized score from about 30 to roughly 16–19, erasing most of DT's advantage over DM.","Compression ratio can serve as a low-cost proxy for the visual complexity that matters to the DT–DM gap, since it ranks second in importance in both random forest and SHAP analyses.","Architecture evaluation in offline RL should report game-characteristic metrics alongside normalized scores, because the win/loss pattern is not intrinsic to the architecture alone but depends on the environment.","If the two-metric rule holds, model selection between DT and DM can be guided by a quick pre-training scan of action count and frame compressibility for each new Atari environment."],"supporting_citations":[{"why":"Supplies the Decision Transformer baseline and the offline training/evaluation recipe that the paper reproduces.","marker":"[1]"},{"why":"Supplies the Decision Mamba architecture whose selective state-space blocks replace DT's causal self-attention.","marker":"[2]"},{"why":"Supplies the Mamba block itself and the motivation for comparing attention-based and state-space sequence models.","marker":"[3]"},{"why":"Supplies the DQN-replay dataset, the source of all offline trajectories and game-characteristic metrics.","marker":"[12]"},{"why":"Supplies the random-walk and human benchmark scores used to normalize raw Atari scores.","marker":"[14]"},{"why":"Supplies the random-walk and human benchmark scores used to normalize raw Atari scores.","marker":"[15]"},{"why":"Supplies the random forest regression method used to compute feature importances for the game metrics.","marker":"[21]"},{"why":"Supplies the SHAP framework used as a complementary feature-importance analysis.","marker":"[22]"}],"fun_headline_variants":["Action count and frame compression pick DT or Mamba winner","Simple games go Mamba, complex go DT: Atari study","Two complexity metrics predict DT vs Mamba in Atari","Visual and action complexity decide DT-Mamba showdown"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The ranking of factors is read out of a random forest fitted to only 12 games, with six-fold cross-validation and post-hoc outlier removal, so the feature-importance ordering may be a property of a few games rather than of Atari games in general.","fun_headline_variants_meta":{"raw":{"variants":["Action count and frame compression pick DT or Mamba winner","Simple games go Mamba, complex go DT: Atari study","Two complexity metrics predict DT vs Mamba in Atari","Visual and action complexity decide DT-Mamba showdown"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000253,"raw_usage":{"total_tokens":1569,"prompt_tokens":957,"completion_tokens":612,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":573,"completion_tokens_details":{"reasoning_tokens":545}},"tokens_in":573,"tokens_out":612,"duration_ms":5605,"temperature":1.0,"reasoning_tokens":545,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:04:16.265918+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive check would be to decouple the two factors by modifying games: take a visually simple game such as Breakout and add visual texture without changing its 4-action set, or take Hero and keep its 18 actions while simplifying the graphics. If the DT–DM gap tracks compression ratio when action count is held fixed, the visual-complexity claim is supported; if it tracks only action count, the paper's hierarchy is wrong. A cheaper statistical falsifier is to resample the 12 games thousands of times and check whether action count remains the top random-forest feature; if the top feature changes frequently, the reported importance ranking is unstable.","supporting_citations":[{"cited_title":"Decision transformer: Reinforcement learning via sequence modeling,","cited_arxiv_id":null,"evidence_quote":"Supplies the Decision Transformer baseline and the offline training/evaluation recipe that the paper reproduces."},{"cited_title":"An optimistic perspective on offline reinforce- ment learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the DQN-replay dataset, the source of all offline trajectories and game-characteristic metrics."},{"cited_title":"Mastering atari with discrete world models,","cited_arxiv_id":null,"evidence_quote":"Supplies the random-walk and human benchmark scores used to normalize raw Atari scores."},{"cited_title":"Mastering atari games with limited data,","cited_arxiv_id":null,"evidence_quote":"Supplies the random-walk and human benchmark scores used to normalize raw Atari scores."},{"cited_title":"A unified approach to interpreting model predictions,","cited_arxiv_id":null,"evidence_quote":"Supplies the SHAP framework used as a complementary feature-importance analysis."}],"review_version":1}