{"id":"5cd99041-fce5-46ff-9294-03d51ac988a3","arxiv_id":"2608.05989","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A model-free visual RL agent that jointly trains latent self-prediction and next-observation prediction, mediated by two adapters, improves aggregate DMControl scores over prior methods.","lead":"This paper introduces OG-SPR, a model-free reinforcement learning method that learns visual representations by predicting both future latent states and next observations. On 28 DeepMind Control Suite tasks it reports higher average scores than recent self-predictive and observation-predictive baselines, with large gains on dog and humanoid tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The self-prediction weight λ_Self-predictive=5.0 is selected on dog-run and quadruped-run, so the paper's headline dog-domain gains are partly a tuning artifact; a holdout re-test of λ is needed.","rationale":"The reader's weakest-assumption analysis already identified both the anonymous baseline and the λ_Self-predictive selection on quadruped-run and dog-run. I focused on the λ selection because it is internally checkable with the paper's own experimental setup and directly attacks the 'particularly pronounced gains in dog and humanoid' clause of the central claim. If λ=5.0 is a selection artifact, the dog-domain evidence is substantially weakened, and the aggregate DrQv2-normalized mean of 5.05 becomes less trustworthy even though the raw-score aggregate improvement would likely persist. The anonymous-baseline issue is also serious and independently supports the conditional verdict, but it is primarily a verifiability problem; the λ issue is a correctness risk that can be settled by re-running with a small grid. The paper has real strengths: five seeds per task, bootstrap CIs, component ablations, representation probing, and a controlled backbone matched to MR.Q and AnonMethod. Those strengths mean the claim should not be rejected outright; it should remain conditional pending a λ-sensitivity/holdout check and public release of the baseline and code. Hence I recommend no change to the reader's CONDITIONAL verdict.","tokens_in":24047,"tokens_out":7491,"duration_ms":77179,"concrete_test":"Re-run the complete 28-task DMControl protocol with λ_Self-predictive ∈ {2.0, 3.0, 4.0, 5.0, 6.0} using the released code, and report per-task and aggregate raw and DrQv2-normalized scores with 95% bootstrap CIs. The decisive comparison is whether λ=5.0 is best on the two tuning tasks (dog-run, quadruped-run) but not on held-out tasks, especially dog-stand, dog-trot, dog-walk, and the three humanoid tasks. If held-out dog and humanoid gains vanish at λ=2.0 and λ=4.0, the 'particularly pronounced gains in dog' claim is a tuning artifact; if λ=5.0 also wins on held-out tasks and aggregate results beat AnonMethod at every tested λ, the central transfer claim survives. Also report the random scores used in the DrQv2-normalized formula, since they are not given in the paper.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of 'particularly pronounced gains in challenging domains such as dog and humanoid' is weakened by the hyperparameter selection procedure described in the Implementation Details section. The paper states that λ_Self-predictive was selected by searching candidate values spaced by 1.0 on a subset containing only quadruped-run and dog-run, and the chosen value 5.0 was then fixed for all 28 tasks. Consequently, the dog-run result (90 vs. 51 for MR.Q and 55 for AnonMethod), which is among the largest relative gains in the dog domain, is not independent evidence: it is produced by a method whose one tuned scalar was selected on that very task. The same applies to quadruped-run (543 vs. 503). The paper provides no λ-sensitivity analysis; Table 2 ablates the self-prediction objective entirely but never varies its weight, so the reader cannot tell whether 5.0 is broadly beneficial or narrowly good on the two tuning tasks. The humanoid tasks were not in the tuning subset and therefore provide more independent support, but their 500k-step scores are extremely small (e.g., humanoid-run 3 vs. 1), and the 2M-step humanoid results also do not include a λ check. If λ=5.0 is a local optimum for dog-run and quadruped-run but not for held-out tasks, the aggregate DrQv2-normalized mean of 5.05 and the 'pronounced dog gains' clause would be inflated by selection rather than by genuine transfer. This is a standard overfitting risk, not an allegation of misconduct, but it is load-bearing for the paper's strongest empirical claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes OG-SPR, a model-free visual continuous-control RL algorithm that trains an actor-critic agent with three auxiliary objectives: multi-step latent self-prediction, next-observation prediction, and short-term value prediction. To prevent the latent self-prediction objective from over-constraining the shared representation, the authors introduce two lightweight adapters that form an adapter-mediated self-prediction branch. The paper evaluates OG-SPR on 28 DeepMind Control Suite tasks at 500k environment steps (plus 2M-step humanoid runs), reporting improvements over DrQ-v2, TD-MPC2, DreamerV3, MR.Q, and an anonymous observation-predictive baseline called AnonMethod. The authors also report ablations removing each auxiliary objective and each adapter component, a linear-probing representation analysis, and an Atari100k adaptation. The central claim is that OG-SPR achieves a new state of the art for aggregate performance on DMControl, with particularly large gains on dog and humanoid tasks.","tokens_in":24420,"tokens_out":4821,"duration_ms":45562,"significance":"If the empirical claims hold, OG-SPR would be a meaningful contribution: it demonstrates a design for combining latent self-prediction and observation prediction in model-free visual RL, and its aggregate improvements are supported by five-seed runs with bootstrap confidence intervals, controlled comparisons against strong baselines, and consistent ablations. The paper also ships implementation details in the supplementary material, reports wall-clock time, and honestly states limitations. However, the headline result is contingent on two load-bearing issues: the self-prediction weight is tuned on two of the very tasks used to advertise the dog-domain gains, and the observation-predictive baseline is an anonymous, suppressed-citation method whose correctness cannot be checked from the main text. The DrQv2-normalized mean is also sensitive to tasks with very small raw scores.","major_comments":[{"comment":"The choice of lambda_Self-predictive = 5.0 is made by a grid search over integer values on quadruped-run and dog-run only, and this same value is then used for all 28 tasks. Since dog-run is one of the two tasks on which the headline 'particularly pronounced gains in the dog domain' is based (dog-run 90 vs 55 for AnonMethod and 51 for MR.Q in Table 1), the dog-domain improvement is not independent evidence for the method. The manuscript provides no sensitivity analysis varying this weight, and the ablation in Table 2 only removes the self-prediction objective entirely rather than varying lambda. I request a lambda-sensitivity analysis (e.g., lambda in {2,3,4,5,6} on the tuning tasks plus several held-out tasks such as humanoid-walk, cheetah-run, quadruped-walk, and acrobot-swingup) and, if possible, a re-run of the full 28-task evaluation with the SPR default lambda=2.0 so the reader can see how much of the aggregate DrQv2-normalized mean of 5.05 is attributable to the selected value. This is load-bearing because the paper's own conclusion acknowledges that fixed auxiliary loss weights 'may not be optimal for all tasks.'","section":"Implementation Details"},{"comment":"The headline aggregate metric 'DrQv2-normalized mean = 5.05' is driven in large part by tasks where raw scores are extremely small. For example, humanoid-run raw score is 3 for OG-SPR versus 1 for MR.Q and AnonMethod (Table 1), so the normalized score is roughly 3 on a baseline denominator of approximately 1; the dog and 2M humanoid tasks behave similarly. The paper reports the IQM (1.61 for OG-SPR vs 1.29 for MR.Q) as a robustness check, but the main text continues to emphasize the 1.8x mean improvement without reporting the per-task normalized scores or a median. Please report the full per-task DrQv2-normalized table (or at least the distribution and the contribution of dog/humanoid tasks to the mean), and discuss how much of the 5.05 mean is due to the small denominators in these tasks. If the aggregate claim is 'improves aggregate performance,' it should be stated at the level of both mean and IQM with the small-score caveat made explicit.","section":"Table 1 and Evaluation Protocol"},{"comment":"The comparison against observation-predictive methods rests entirely on 'AnonMethod (Anonymous 2026)', whose citation is suppressed and whose implementation is said to be provided only in the supplementary material. As written, a reader cannot verify that this is a strong, correctly re-implemented baseline: the baseline's exact architecture, hyperparameter values, and training protocol are not stated in the main text, and the suppressed reference prevents attribution or comparison with any published work. For a journal submission, this is a verifiability gap in a load-bearing comparison. The revision should either cite the actual published source (after de-anonymization) or, if the method is unpublished, provide a complete standalone description and a persistent repository for both AnonMethod and OG-SPR so that the baselines can be reproduced without relying on an anonymous supplement.","section":"References and Supplementary Material"}],"minor_comments":[{"comment":"The caption should state precisely that the baseline score is the stronger result between MR.Q and AnonMethod per task, and the axes should be labeled with task names and percentage improvement.","section":"Figure 1"},{"comment":"The two-hot encoding and the symexp return support are introduced without a formal definition; please specify the bin placement, the number of bins, and how symexp is inverted.","section":"Equation (9)"},{"comment":"On Atari100k, OG-SPR is not consistently better than AnonMethod on the human-normalized mean, yet the text says it 'remains competitive' and ranks first or second on three of four metrics; the narrative should more carefully acknowledge that the gains do not transfer to discrete-action domains.","section":"Table 3"},{"comment":"The limitations paragraph mentions fixed auxiliary loss weights but not the sensitivity of the self-prediction horizon K or the choice of n-step returns; a sentence noting these additional fixed hyperparameters would be appropriate.","section":"Conclusion"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is an empirical benchmark paper whose central contribution depends on a comparison against an anonymous baseline. The editor should decide whether an anonymous, suppressed-citation baseline is acceptable for the venue, and should ask the authors to de-anonymize or fully document the baseline during revision. The lambda-selection issue is the main technical risk; a sensitivity analysis would likely resolve it."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Key point: this is a real empirical contribution. The new piece is the adapter-mediated self-prediction branch, which keeps the shared representation from being directly constrained by the latent self-prediction loss. Prior work did latent self-prediction or observation prediction, but not both in one model-free agent, and the two-adapter design is a sensible fix for the over-constraint problem. The paper does that part well: ablations are systematic (remove each objective hurts, share or remove adapters hurts), and the representation probing gives a plausible mechanistic story. The Atari100k extension is a good sanity check even though gains are weaker there.\n\nThe main soft spot is exactly what the stress-test flags: lambda_Self-predictive=5.0 was selected on quadruped-run and dog-run, and dog-run is where the largest relative gain appears. So the claim of \"particularly pronounced gains in the dog domain\" is partly a tuning artifact. Humanoid tasks were held out, so those gains are more independent, but absolute scores like 3 vs 1 are tiny, and the DrQv2-normalized mean of 5.05 is inflated by dividing by near-zero denominators. The IQM normalized score, 1.61 vs 1.29, is a more measured but still real improvement.\n\nThe second concern is verifiability. AnonMethod is anonymous; the authors say its implementation is in the supplementary material, but a state-of-the-art comparison against an unreviewable baseline is weaker than it should be. OG-SPR code is also only in the supplementary material, not a public repo. Both are fixable.\n\nOverall, the central claim holds up in aggregate, with the lambda caveat. The paper deserves a serious referee, but the authors should be asked for a holdout lambda check, a sensitivity analysis around lambda=5.0, public code, and a less anonymous comparison baseline.","headline":"A genuine adapter-based combination of latent self-prediction and observation prediction with consistent aggregate gains, but the dog-domain headline is partly a tuning artifact and the anonymous baseline needs disclosure.","tokens_in":24909,"tokens_out":1500,"would_cite":false,"duration_ms":15651,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A model-free visual RL agent learns faster when its representations are trained by both latent self-prediction and next-observation prediction, routed through two adapters so neither objective over-constrains the shared encoder.","keywords":["visual reinforcement learning","continuous control","sample efficiency","self-predictive representation learning","observation prediction","model-free reinforcement learning","DMControl benchmark","auxiliary losses"],"falsifier":"Re-run the 28-task comparison with $\\lambda_{\\text{Self-predictive}}$ tuned per task (or on a held-out validation set) and recompute the aggregate DrQ-v2-normalized mean; if the advantage over the two predictive baselines disappears or reverses, the claim that combining the two objectives drives the gain fails. A second check is to run the released implementation of the anonymous observation-predictive baseline and confirm its reported task scores.","tokens_in":23863,"feed_emoji":"🤖","tokens_out":11374,"duration_ms":93803,"temperature":0.7,"pith_summary":"This paper tries to establish that a model-free reinforcement learning agent for pixel-based continuous control becomes more sample-efficient when its representation learner combines two complementary dynamics-prediction objectives: multi-step latent self-prediction and next-observation prediction. The authors argue that latent self-prediction alone leaves latent transitions unaligned with observation-level dynamics, while observation prediction alone does not enforce long-horizon latent predictability. The proposed method, OG-SPR, routes the self-prediction objective through two lightweight adapters so it does not directly constrain the shared encoder, which the paper's ablation study identifies as the key to making the combination work. On 28 tasks of the DMControl benchmark at 500k environment steps, OG-SPR reports higher aggregate raw and DrQ-v2-normalized scores than the strongest self-predictive and observation-predictive baselines, with the largest relative gains on dog and humanoid tasks.","feed_headline":"Pairing latent and pixel prediction tops 28 visual control tasks","feed_subtitle":"Pairing the two prediction objectives lifts the mean DrQ-v2-normalized score from 2.8 to 5.1.","key_machinery":"The load-bearing mechanism is the adapter-mediated self-prediction branch: two lightweight linear adapters, Adapter 1 ($u_1$) after the observation encoder and Adapter 2 ($u_2$) after the state-action encoder, define the latent space in which multi-step self-prediction targets are computed, while the shared state-action representation is trained only by value learning, next-observation reconstruction, and short-term value prediction. This decoupling prevents the self-prediction objective from over-constraining the shared representation; the paper's ablations show that removing or sharing the adapters degrades performance, so the decoupling itself is a necessary part of the method.","core_discovery":"The paper's central claim is that latent self-prediction and observation prediction are complementary representation-learning signals for model-free visual control, and that they can be combined profitably once self-prediction is decoupled from the shared representation by an adapter-mediated branch. OG-SPR minimizes, alongside a TD3-style value loss, a $K=5$-step latent self-prediction MSE loss computed in the adapter-defined space, a next-observation reconstruction loss in a normalized observation space, and a short-term value-prediction loss. The reported aggregate result on 28 DMControl tasks at 500k environment steps is a mean raw score of 626 (versus 607 for the observation-predictive baseline and 599 for the self-predictive baseline) and a mean DrQ-v2-normalized score of 5.05, about 1.8 times the self-predictive baseline's 2.84 and 1.9 times the observation-predictive baseline's 2.65. Ablations removing either predictive objective, sharing the adapters, or removing them entirely all lower aggregate performance, which the authors read as evidence that both objectives and the adapter decoupling carry the reported gain.","pith_inferences":["Editorial inference: the two-adapter design suggests a general recipe for combining predictive auxiliary losses in RL: route each predictive objective through its own projection head that reads from, but does not write to, the shared representation, so conflicting geometric constraints do not cancel.","Editorial inference: the complementarity hypothesis implies that other observation-level grounding signals (optical flow, depth, object segmentation) could replace or augment image reconstruction as the grounding objective and may yield further data-efficiency gains on tasks where pixel reconstruction is cheap but not informative enough.","Editorial inference: because the paper fixes $\\lambda_{\\text{Self-predictive}} = 5.0$ across all 28 tasks after tuning on two dog tasks, a natural testable extension is per-task or adaptive weighting of the three auxiliary losses; the authors themselves list fixed weights as a limitation.","Editorial inference: the DrQ-v2-normalized aggregate amplifies low-scoring tasks (small denominator), so the large normalized gap (2.84/2.65 to 5.05) is driven substantially by dog and humanoid tasks; future comparisons should report per-task raw scores alongside normalized aggregates."],"forward_implications":["If the central claim is right, OG-SPR sets a new aggregate benchmark for model-free pixel-based continuous control at 500k environment steps, ahead of both the self-predictive and the observation-predictive baselines it is compared against.","The largest gains appear on the hardest tasks: improvements of 20.9-63.8% on dog tasks and 61.7-333.7% on humanoid tasks over the stronger baseline, with the humanoid gap persisting at a relaxed 2M-step budget.","Both auxiliary objectives are necessary: ablations that drop either self-prediction or observation prediction lose aggregate raw and normalized score, and the loss is largest when self-prediction is removed.","The adapter design matters: sharing the two adapters or removing them (so self-prediction is imposed directly on the shared representation) also lowers aggregate performance, supporting the over-constraining hypothesis.","The method remains competitive on discrete-action Atari100k without task-specific engineering, ranking first or second on three of the four aggregate metrics reported."],"supporting_citations":[{"why":"Supplies the strong self-predictive baseline (MR.Q) and the actor-critic backbone (TD7-style) that OG-SPR builds on; its aggregate results are the main comparison to beat.","marker":"(Fujimoto et al. 2025)"},{"why":"Supplies the observation-predictive baseline, the normalized observation space, the next-observation reconstruction objective, and the short-term value-prediction loss that OG-SPR adopts; the suppressed citation is the strongest comparison method.","marker":"(Anonymous 2026)"},{"why":"Supplies the multi-step latent self-prediction objective and target-network mechanism (SPR), plus the default loss weight 2.0 that the authors start from for tuning.","marker":"(Schwarzer et al. 2021)"},{"why":"Supplies DrQ-v2, the model-free baseline and random-shift augmentation strategy, and defines the normalization baseline for the DrQ-v2-normalized aggregate metric.","marker":"(Yarats et al. 2022)"},{"why":"Supplies the TD3-style value learning with clipped target-policy smoothing and the target-value construction used by OG-SPR.","marker":"(Fujimoto, van Hoof, and Meger 2018)"},{"why":"Supplies the LAP prioritized replay buffer and the Huber-loss equivalence that OG-SPR uses for value learning.","marker":"(Fujimoto, Meger, and Precup 2020)"},{"why":"Defines the DMControl benchmark, whose 28 tasks are the experimental substrate for the central claim.","marker":"(Tassa et al. 2018)"}],"fun_headline_variants":["OG-SPR pairs latent and pixel prediction to top 28 control tasks","Adapter-decoupled self-prediction gives visual RL a 1.8× boost","Combine latent and pixel learning: OG-SPR rules 28 control tasks","Two predictive objectives beat one on 28 DMControl tasks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The aggregate win rests on the self-prediction weight $\\lambda_{\\text{Self-predictive}} = 5.0$, tuned on only quadruped-run and dog-run, transferring to all 28 tasks, and on the anonymous observation-predictive baseline being a strong, correctly re-implemented method.","fun_headline_variants_meta":{"raw":{"variants":["OG-SPR pairs latent and pixel prediction to top 28 control tasks","Adapter-decoupled self-prediction gives visual RL a 1.8× boost","Combine latent and pixel learning: OG-SPR rules 28 control tasks","Two predictive objectives beat one on 28 DMControl tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001506,"raw_usage":{"total_tokens":6109,"prompt_tokens":1081,"completion_tokens":5028,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":697,"completion_tokens_details":{"reasoning_tokens":4949}},"tokens_in":697,"tokens_out":5028,"duration_ms":38711,"temperature":1.0,"reasoning_tokens":4949,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T19:44:42.227729+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the 28-task comparison with $\\lambda_{\\text{Self-predictive}}$ tuned per task (or on a held-out validation set) and recompute the aggregate DrQ-v2-normalized mean; if the advantage over the two predictive baselines disappears or reverses, the claim that combining the two objectives drives the gain fails. A second check is to run the released implementation of the anonymous observation-predictive baseline and confirm its reported task scores.","supporting_citations":[],"review_version":1}