{"id":"776b668c-9868-49fd-a9f9-bfb6f4a1ea87","arxiv_id":"2505.09561","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Past-Token Prediction, an auxiliary loss that makes diffusion policies predict past action tokens, improves long-context imitation learning success roughly 3x over baselines while a cached-embedding recipe cuts training cost.","lead":"This robotics paper trains diffusion policies to also predict their own past actions, which helps them use long observation histories. The authors report about 3x higher success on long-context manipulation benchmarks and a large training speedup from caching frozen image features.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The abstract's 'more than 10x training speedup' is contradicted by the paper's own Fig 8 and Takeaway 4, which report a 5x speedup; this is a load-bearing quantitative claim that needs correction or explicit evidence.","rationale":"The reader's weakest assumption is the frozen short-context encoder, which is a legitimate generalizability concern. I agree with that concern as a secondary issue. However, the most load-bearing defect in the central claim is the training-speedup number: the abstract advertises >10x, while the paper's own evidence section reports a 5x speedup (20% of training time to match) and Fig 8's caption says 'over 5x'. This is not a matter of external consensus or speculative extrapolation; it is an internal numerical contradiction in the headline result. If the true speedup is 5x, the efficiency contribution is still meaningful but the central claim is overstated by at least 2x. The performance claim (3x/81% vs 16% in simulation) remains plausible and is supported by Table 3 with seed variance; the real-world results without error bars are weaker but not independently disqualifying. Because the central claim can be repaired by a correction and the simulation results are reproducible in principle, the appropriate verdict remains conditional acceptance rather than rejection. My check would settle the speedup discrepancy directly from raw logs; until that is done, the abstract should not be taken at face value.","tokens_in":15147,"tokens_out":9519,"duration_ms":95412,"concrete_test":"Recover the raw training logs or rerun the cached versus uncached comparison from Fig 8 on at least one task (e.g., Tool Hang) with identical hardware and the reported hyperparameters. Compute wall-clock time (or epochs) to the final PTP performance for both regimes, and also record whether the uncached run ever succeeds within the two-day budget. If time-to-target for cached is ~20% of uncached, the speedup is ~5x and the '>10x' headline should be revised; if some configuration reaches >10x, the exact comparison (including treatment of runs that never succeed) should be stated explicitly in the paper.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is twofold: 3x performance and >10x training speedup. The speedup half is internally inconsistent. Section 5.1, Takeaway 4 states that with caching the method 'matches performance in just 20% of the training time and surpasses it within 40% of the compute budget.' Twenty percent of the time is a 5x speedup, not >10x; forty percent is 2.5x. The Fig 8 caption likewise says 'Caching speeds up training by over 5x'. No other number in the paper supports the abstract's 'more than 10x'. If the uncached Tool Hang run never reaches the target success within two days, the ratio to a fixed-performance threshold is not a finite 10x; it is an undefined/infinite ratio, and using that to advertise '>10x' is misleading. The speedup is load-bearing because the multistage cached-embedding recipe is presented as a central contribution: if the honestly measured speedup is 5x, the abstract and Figure 1 overstate the efficiency gain by at least a factor of two. The Limitations section discusses inference overhead but does not acknowledge this training-speed discrepancy.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes Past-Token Prediction (PTP), an auxiliary objective for diffusion-based visuomotor policies that jointly predicts past and future action tokens, together with a multistage training recipe that pre-trains a visual encoder on short contexts, caches its embeddings, and then trains a long-context policy head conditioned on the cached embeddings. At inference, PTP is used as a self-verification criterion to select among sampled action chunks by comparing reconstructed past actions with the executed ones. The paper reports large gains over no-PTP and no-history baselines on six simulation tasks and four real-world tasks, and it claims in the Abstract and Figure 1 that the method improves long-context diffusion policy performance by 3x while accelerating training by more than 10x.","tokens_in":15348,"tokens_out":6564,"duration_ms":65189,"significance":"If the empirical results hold, PTP is a simple and broadly applicable modification to diffusion policies that addresses a real limitation: history-conditioned diffusion policies underperform on history-critical tasks. The evaluation is broad for a method paper, covering six simulation tasks and four real-world tasks across two platforms, with external success-rate metrics and several useful ablations (encoder-only vs decoder-only PTP, chunk-size dependence, and past-action conditioning). The authors also include an explicit Limitations section. The main risks are an internally inconsistent training-speedup claim, a diagnostic that may conflate sampling stochasticity with weak temporal dependency, and a load-bearing encoder-freezing assumption that is supported by a single-task ablation. These issues are correctable, but they affect the strength of the central claims as currently written.","major_comments":[{"comment":"The claimed 'more than 10x' training speedup in the Abstract and Fig. 1 is inconsistent with the paper's own efficiency results. Takeaway 4 states that the caching-based recipe matches performance in just 20% of the training time and surpasses it within 40% of the compute budget, which corresponds to at most a 5x speedup, and the Fig. 8 caption says 'Caching speeds up training by over 5x.' If the uncached Tool Hang run never reaches the target success within the two-day budget, then the speedup ratio to a fixed-performance threshold is not well defined. Please correct the Abstract/Fig. 1 claim or report the measured compute-time curves that support a greater-than-10x speedup.","section":"Abstract, Fig. 1, §5.1 Takeaway 4, Fig. 8"},{"comment":"The action-predictability diagnostic does not cleanly isolate temporal action dependency from sampling stochasticity. Diffusion policies generate actions by sampling from a learned conditional distribution, so even a policy that strongly depends on past actions will produce rollout actions with irreducible sampling variance; this variance will inflate the MSE of an MLP predictor fitted to previous actions and push the predictability ratio below 1. The claim that modern diffusion policies 'underuse past action information' therefore needs a control, for example comparing prediction error given the full history against prediction error given past actions only, or evaluating rollouts under matched noise. In addition, §3 defines the probe as predicting the current action from the previous action alone, while Appendix B.3.4 defines it as predicting from a window of K=15 past actions; this discrepancy must be reconciled.","section":"§3, Appendix B.3.4, Fig. 2"},{"comment":"The multistage recipe freezes a visual encoder pretrained on two-frame contexts and relies on cached embeddings for long-context policy training. The only direct evidence that this preserves performance is the Push-T ablation in Appendix A.1 (Fig. 12). Since the main experiments all use the frozen-encoder recipe, a failure of the frozen features on other tasks would affect both the performance and the efficiency claims. I ask for a per-task, or at least multi-task, comparison between end-to-end training and the cached-embedding recipe, or a quantitative report of how downstream success varies with encoder quality across the evaluated tasks.","section":"§A.1, Fig. 12, §4.2"}],"minor_comments":[{"comment":"The Mean column in Table 4 is inconsistent with the per-task entries: for the 16-observation row, the per-task values 0.64, 0.85, 0.82, 0.51, and 0.81 average to 0.73, not the reported 0.77, and similar discrepancies appear in the other rows. Please recompute the means or explain the discrepancy.","section":"Table 4"},{"comment":"The sentence reporting 'an average improvement of over 30%' and 'over 60%' should be clarified as percentage-point improvements; the relative improvement over no-PTP based on Table 3 averages is roughly 390%, not 60%.","section":"§5.1, text before Fig. 9"},{"comment":"The action-predictability figures lack axis labels and error bars, which makes it difficult to assess the magnitude and variability of the reported ratios; please add these details.","section":"Fig. 2 and Fig. 6"},{"comment":"The task names 'ALOHA' and 'Long Square' should be made consistent with 'Long-Horizon ALOHA' and 'Long-Horizon Square' used elsewhere in the text.","section":"Table 3"},{"comment":"The '3x performance' claim should specify the baseline it is compared against (no-PTP history-conditioned policy vs no-history policy), since these give very different improvement ratios.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The most urgent fix is the training-speedup claim, which is internally inconsistent: the Abstract and Figure 1 say 'more than 10x,' while Section 5.1 and Figure 8 report 'over 5x.' This is a load-bearing quantitative claim that needs correction or explicit evidence. The Section 3 diagnostic also needs sharper validation, and the encoder-freezing assumption should be supported by more than the Push-T ablation. I see no evidence of fabrication; the zero-variance entries in Table 3 are plausible if those baselines collapsed during training, but the authors should report training details for those runs. There is also a self-citation concern: the test-time verification mechanism closely parallels the same group's prior work [19], and the novelty overlap is not discussed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know before reading: the main result is probably true and useful — adding a past-action prediction loss to a diffusion policy head makes long-context imitation work much better than history-conditioning alone. But the paper's headline '>10x training speedup' is contradicted by its own Figure 8 and Takeaway 4, which say 'over 5x' and '20% of training time'. That mismatch is not cosmetic; the speedup is a central claimed contribution. Also, the diagnostic in Section 3 that motivates the method is weaker than the results: the action-predictability ratio almost certainly conflates diffusion sampling stochasticity with weak temporal dependency, because a stochastic policy will be hard to predict from previous actions even if it conditions on history perfectly. The empirical payoff is still there, but the story needs rewording.\n\nWhat is genuinely new: the specific package — training with PTP, freezing a short-context encoder and caching its embeddings, and using the PTP head to select samples at test time — plus evaluation across six simulation and four real-world tasks. The simulation numbers are strong: Table 3 shows PTP at 75-98% on Tool Hang, Transport, ALOHA, and Long Square where no-PTP collapses to 0-20%. The decoder-only ablation is clean and explains why the caching recipe works. The real-world results have no error bars, which is a minor but real omission for a 4x claim. The overlap with the authors' own Bidirectional Decoding [19] is real; the idea of predicting past actions and using consistency as a selection criterion is present there. Here it is used as an auxiliary training loss, which is a distinct contribution, but the paper should discuss that boundary explicitly.\n\nThe weakest load-bearing assumption is that a visual encoder pretrained on two frames suffices for long-context tasks. The paper supports this only with the Push-T ablation in Appendix A.1. If that assumption breaks on other tasks, the training speedup and the performance gains both weaken. That is a testable claim, and a reviewer should ask for at least one additional task where the encoder is adapted to long contexts.\n\nBottom line: this is a solid, useful empirical paper for anyone working on diffusion policies or long-horizon imitation. It deserves peer review. I would send it with a request for major revision: fix the speedup number, report trials for the real-world results, and pull the diagnostic language back to what the metric can actually show.","headline":"A simple auxiliary objective that appears to deliver real gains for long-context diffusion policies, but the advertised training speedup is inconsistent with the paper's own numbers.","tokens_in":15922,"tokens_out":2880,"would_cite":true,"duration_ms":30072,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that adding past-token prediction to long-context diffusion policies improves success by 3x and cuts training cost by more than 10x.","keywords":["imitation learning","diffusion policies","long-context policies","past-token prediction","temporal action dependency","behavior cloning","test-time verification","robot manipulation"],"falsifier":"Train a PTP long-context policy on a task where the current frame is visually ambiguous (for example, several identical objects, with the identity of the target visible only in an early frame), then compare the frozen short-context encoder against the same encoder fine-tuned on long contexts. If the frozen-encoder policy fails while the fine-tuned one succeeds, the claim that short-context features suffice for long-context control is falsified; if both succeed, the assumption holds.","tokens_in":14868,"feed_emoji":"🤖","tokens_out":9111,"duration_ms":88019,"temperature":0.7,"pith_summary":"This paper claims that the main obstacle to long-context imitation learning is not just memory cost but a subtle failure mode: modern diffusion policies underuse the information in past actions, producing rollouts whose temporal action dependencies are weaker than those in the demonstrations. The proposed remedy is past-token prediction (PTP), an auxiliary task in which the policy predicts the action tokens it already executed alongside the future action chunk it is about to execute. Adding this objective raises average success from roughly 16 percent to 81 percent across six simulated tasks, and from 15 percent to 70 percent across four real-world history-critical tasks, while a two-stage recipe (freeze a short-context visual encoder, cache its embeddings, train only the long-context policy head) reduces training cost by more than 10x. If these results hold, PTP is a cheap way to make policies that genuinely use history without the memory blow-up that has pushed most systems to truncate context.","feed_headline":"Past-token prediction triples history-aware robot policy success","feed_subtitle":"A simple auxiliary loss makes diffusion policies actually use history: 70% real-world success at 10x lower training cost.","key_machinery":"The central object is Past-Token Prediction (PTP), an auxiliary training objective that asks the policy, given a window of past observations, to output both the already-taken actions and the upcoming future actions. The diagnostic that motivates it is the action predictability ratio $\\epsilon_{\\pi^*}/\\epsilon_\\pi$, comparing how well the current action can be predicted from the previous action in expert demonstrations versus policy rollouts; a ratio below one marks underuse of past actions. Three mechanisms carry the argument: PTP as a regularizer on the policy head, a multistage recipe that freezes and caches a short-context visual encoder so long-context decoder training costs about the same as short-context training, and test-time candidate selection by past-action consistency. The ablation showing that decoder-only PTP matches full PTP while encoder-only PTP does not is what licenses the frozen-encoder recipe.","core_discovery":"The central discovery is the flipped copycat problem: regression-based policies over-predict the current action from the previous action, while diffusion-based policies do the opposite and underpredict it, even when conditioned on long observation histories. The paper argues that adding past-token prediction to the training objective bridges this gap, and that the benefit lives almost entirely in the action decoder, not the visual encoder. That localization justifies freezing a visual encoder pretrained on two-frame contexts, caching its embeddings, and training a long-context policy head on top; the paper reports that this recipe raises long-context diffusion policy success by 3x on average and accelerates training by over 10x. At inference, the same past-token objective becomes a self-verification signal: sample several action chunks and pick the one whose reconstructed past actions best match the actions actually executed.","pith_inferences":["A natural extension, not tested in the paper, is to apply PTP-style supervision to other generative policy families, such as flow-matching or autoregressive action-token policies, whose heads are also trained with next-token-only objectives; if the mechanism is generic, the same auxiliary loss should tighten their temporal dependencies.","The action predictability ratio could serve as a cheap diagnostic before full evaluation: a rollout ratio well below one would flag a diffusion policy as a candidate for PTP-style regularization, potentially saving compute in policy selection.","The frozen-encoder assumption is the most likely boundary of the method; tasks where the disambiguating visual information appears only in an early frame and is absent from the current frame would test whether a two-frame-pretrained encoder already encodes what long-horizon decisions need."],"forward_implications":["Long observation histories become practical for imitation learning: a 16-frame context can be trained with roughly the memory and compute of a 2-frame context, so history-critical tasks no longer need to truncate the past.","Closed-loop execution improves: PTP-trained policies with chunk size one and long history reach 0.64 to 0.85 success on several simulated tasks, meaning reactivity does not have to be sacrificed for temporal memory.","The failure-mode diagnosis points in opposite directions for the two policy families: regression policies should be regularized against copying past actions, while diffusion policies should be pushed toward stronger coupling between past and future actions.","Test-time compute becomes a usable lever: sampling five action chunks and selecting by past-token consistency adds about five percentage points of success, so cheaper checkpoints can partly compensate for less training with more sampling."],"supporting_citations":[{"why":"Supplies the base visuomotor diffusion policy and Push-T task whose long-context degradation motivates the method.","marker":"[8]"},{"why":"Supplies the simulation benchmark and human-demonstration datasets used for the square, tool-hang, and transport evaluations.","marker":"[21]"},{"why":"Defines copycat behavior and the action-predictability diagnostic the paper reuses to measure temporal dependency.","marker":"[41]"},{"why":"Represents the context-truncation and summarization approach that the paper positions PTP against.","marker":"[48]"},{"why":"Prior resampling-by-consistency method that PTP test-time verification extends.","marker":"[19]"},{"why":"Provides the bimanual manipulation setup used for long-horizon and tape-replacement tasks.","marker":"[47]"},{"why":"Supplies the past-action leakage regularization framing and the predictability-ratio metric.","marker":"[31]"}],"fun_headline_variants":["Past-token prediction fixes long-context diffusion policies","Auxiliary task helps diffusion policies use history: 3x success","Predict past actions to train long-context policies 10x faster","Flipped copycat: past-token loss for history-aware policies"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that a visual encoder trained on only two frames extracts features rich enough to support decisions that need sixteen frames of history; if long-horizon tasks require visual information that appears only across a longer time span, the frozen-encoder recipe and its speedups may not transfer.","fun_headline_variants_meta":{"raw":{"variants":["Past-token prediction fixes long-context diffusion policies","Auxiliary task helps diffusion policies use history: 3x success","Predict past actions to train long-context policies 10x faster","Flipped copycat: past-token loss for history-aware policies"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000338,"raw_usage":{"total_tokens":1895,"prompt_tokens":1001,"completion_tokens":894,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":617,"completion_tokens_details":{"reasoning_tokens":820}},"tokens_in":617,"tokens_out":894,"duration_ms":7772,"temperature":1.0,"reasoning_tokens":820,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:29:36.421534+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a PTP long-context policy on a task where the current frame is visually ambiguous (for example, several identical objects, with the identity of the target visible only in an early frame), then compare the frozen short-context encoder against the same encoder fine-tuned on long contexts. If the frozen-encoder policy fails while the fine-tuned one succeeds, the claim that short-context features suffice for long-context control is falsified; if both succeed, the assumption holds.","supporting_citations":[{"cited_title":"Diffusion Policy: Visuomotor Policy Learning via Action Diffusion","cited_arxiv_id":null,"evidence_quote":"Supplies the base visuomotor diffusion policy and Push-T task whose long-context degradation motivates the method."},{"cited_title":"What Matters in Learn- ing from Offline Human Demonstrations for Robot Manipulation","cited_arxiv_id":null,"evidence_quote":"Supplies the simulation benchmark and human-demonstration datasets used for the square, tool-hang, and transport evaluations."},{"cited_title":"Fighting Copycat Agents in Behavioral Cloning from Observation Histories","cited_arxiv_id":null,"evidence_quote":"Defines copycat behavior and the action-predictability diagnostic the paper reuses to measure temporal dependency."},{"cited_title":"Regularized Behavior Cloning for Blocking the Leakage of Past Action Information.Advances in Neural Information Processing Systems, 36:2128–2153, December 2023","cited_arxiv_id":null,"evidence_quote":"Supplies the past-action leakage regularization framing and the predictability-ratio metric."}],"review_version":1}