{"id":"2c24c239-c64c-4403-9678-e46551f32df0","arxiv_id":"2505.14709","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"FastCar reuses cached MLP outputs from the previous frame for tokens with high temporal attention scores, reducing video generation decoding work by up to 87% at moderate quality loss.","lead":"FastCar speeds up auto-regressive video generation by reusing cached neural network outputs from the previous video frame when the model's attention scores suggest the frames are similar. It reports about a 2x decoding speedup on an FPGA at some cost in generation quality, and it can be combined with sparse attention methods.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Main risk: replay staleness may compound across frames; Eq. (8) reuses previous MLP outputs that feed future TAS/hidden states, and the one-step Theorem 4.7 plus 8-frame experiments don't establish bounded long-horizon drift.","rationale":"Good-faith reading: FastCar is a training-free inference shortcut that reuses the previous frame's MLP output whenever the mean TAS exceeds tau. The paper's own empirical tables show a real gain at 8 frames: 1.76x at 80% replay with VBench 71.5 vs 74.1, and roughly 2.1x when combined with sparse attention. Those numbers are internally consistent with the TFLOPs reductions and with the claim that MLP dominates AR decoding latency. The method is therefore plausible for short clips, and I would not reject it on the basis of the 8-frame results alone. Why the drift concern is load-bearing: the paper's stated advantage includes 'high-resolution and long-duration video generation' (Abstract, Section 7) and 'alleviated drifting' (Table 2, Figure 6). All of that is argued from 8-frame experiments. Eq. (8) replaces a computed value with a cached one, and this replacement is not a local perturbation: the stale value is fed into the next layer, and in an autoregressive video model it also influences the hidden states used to form queries/keys for later frames. The TAS that controls replay is itself computed on these possibly stale hidden states, so the system can enter a regime where replay begets replay. The theorems in Appendix 10 bound one-step Lipschitz deviations, not the trajectory of deviations over T frames; a one-step bound with constant larger than 1 can still grow exponentially over time, and the paper does not estimate that constant or measure trajectory divergence. Secondary concerns I am not making the headline: the proof of Theorem 4.4 has a scaling inconsistency (Definition 4.3 divides by sqrt(d), while the proof treats s_{t,i} as a cosine of unit vectors, so the claimed bound does not literally follow), and the headline '2.1x' comes from FastCar+StreamingLLM rather than FastCar alone (Table 1 tops out at 1.76x). These are real and should be fixed, but they do not by themselves falsify the empirical speedup; the replay-drift issue is what threatens the central long-duration quality claim. The concrete test above would settle it.","tokens_in":18268,"tokens_out":7873,"duration_ms":80024,"concrete_test":"Generate the same VBench prompts with dense VILA-U, FastCar at roughly 50/80/87% replay, and FastCar+SA (local 16/64) for 32 and 64 frames at 256x256; report VBench total and per-frame LPIPS against the dense rollout. Also instrument per-layer hidden-state divergence d(t,l) = mean_i ||Y^l_FastCar(t,i) - Y^l_dense(t,i)||_2 and check whether d grows with frame index t. If VBench drop stays within about 3 points and d does not increase monotonically with t, the drift concern is resolved; if d grows with t or VBench degrades sharply, Eq. (8)'s long-horizon validity and the paper's long-duration claim fail.","verdict_should_be":"UNCHANGED","load_bearing_attack":"FastCar's central claim is that thresholding the mean TAS (Eq. 7) and replaying the cached MLP output of the aligned previous-frame token (Eq. 8) keeps generation quality close to dense while cutting MLP compute. The condition that must hold is that the replayed Y_{t-1,i} is a good proxy for the true Y_{t,i}, not merely for one layer-step but along the entire autoregressive chain. Replay alters the input to the next transformer block, so the hidden state at (t,i) becomes stale, and that stale state is later used to compute queries/keys for subsequent frames, including the TAS that decides further replays. The approximation error can therefore feed back into itself: a replayed output at layer l can degrade the representation at layer l+1, which can raise future TAS values and trigger more replays, and so on. Theorem 4.7 only upper-bounds a one-step difference between adjacent-frame MLP outputs under an assumed small input difference ||X_{j,:} - X_{j-,:}||; it does not analyze how that input difference evolves after replays at previous layers or frames. All quality experiments are 8-frame, 256x256 (Section 6.1), and the abstract/conclusion claims high-resolution and long-duration advantage without measuring it. If the one-step error accumulates, the VBench-quality drop at 8 frames (74.1% to ~71.5%) is not evidence about 32- or 64-frame videos.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"FastCar proposes to accelerate auto-regressive video generation by exploiting temporal redundancy in MLP modules. The core idea is to compute a Temporal Attention Score (TAS) for each token and, when the score exceeds a threshold, skip the MLP computation and reuse the cached MLP output from the aligned token in the previous frame. The paper provides a theoretical analysis intended to show that high TAS implies small MLP output differences, describes an FPGA accelerator with dynamic resource scheduling, and reports experiments on VILA-U showing up to 1.77x speedup with FastCar alone and up to 2.13x when combined with sparse attention, with modest quality degradation.","tokens_in":18535,"tokens_out":11752,"duration_ms":104988,"significance":"The empirical direction is potentially valuable: MLP latency is indeed the bottleneck in AR video decoding, and exploiting temporal redundancy is orthogonal to sparse attention. The paper ships code and a reproducible experimental setup on a recent open model. However, the theoretical justification has a scaling error that breaks the formal link between TAS and output similarity, the FPGA measurement is not fully described, and the long-duration claims outrun the 8-frame evaluation. If the authors correct the theory and clarify the hardware methodology, the method could be a useful contribution to efficient AR video generation on edge devices.","major_comments":[{"comment":"The proof of Theorem 4.4 contains a scaling error that invalidates the stated bound. Eq. (5) defines TAS as s_{t,i} = <q_j, k_{j-}> / sqrt(d). The proof Step 1 claims that under the normalization assumption ||q_j||=||k_{j-}||=1, s_{t,i} equals the cosine similarity and then applies the Law of Cosines as ||q_j - k_{j-}||^2 = 2(1 - s_{t,i}). However, with the scaling in Eq. (5), the cosine similarity is <q_j, k_{j-}> = sqrt(d) * s_{t,i}, so the correct identity is ||q_j - k_{j-}||^2 = 2(1 - sqrt(d) * s_{t,i}). The bound in Eq. (9) and Theorem 4.7 therefore do not follow. Moreover, for the LLaMA-2-7B model used in the experiments (d=4096), sqrt(d)=64, so the normalized scores range in [-0.0156, 0.0156]. The threshold values reported in Section 6.3 (tau from 0 to -16) are all far outside this range, which would force every token to replay under Eq. (5), contradicting the reported replay ratios (3.96% at tau=0, 87.49% at tau=-8). This indicates an inconsistency between the formal definition of TAS and the score used in the implementation. The theoretical justification of the replay decision is thus not valid as written. Please correct the scaling (either redefine TAS as the unscaled cosine similarity or revise the proof) and state explicitly which score is used in the experiments.","section":"Section 4.4 / Appendix 10.1, Eq. (5)"},{"comment":"The abstract and conclusion claim 'more than 2.1x decoding speedup' and advantages for 'high-resolution and long-duration video generation,' but these claims are not supported by the reported evidence. In Table 1, FastCar alone reaches at most 1.77x speedup (80% replay); the 2.1x+ figures appear only in the combined Ours+Sparse Attn rows of Table 2 (e.g., 2.13x at 87% replay with local size 16). In addition, all quality experiments use 8 frames at 256x256 resolution (Section 6.1), with no experiments on longer videos. Theorem 4.7 is a one-step Lipschitz bound: it relates the MLP output difference at adjacent frames to the input difference and TAS, but it does not analyze how that input difference evolves when earlier layers or earlier frames have already been replayed. Because a replayed output feeds into the next layer and affects the TAS used to decide further replays, the bound does not rule out error accumulation over many frames. The claimed scalability to long-duration video generation is therefore an extrapolation rather than a demonstrated result. Either add experiments on longer videos (e.g., 16 or 32 frames) and ideally a sequence-level error bound, or temper the claims.","section":"Section 6.1, Section 6.2, Section 7 (Table 2)"},{"comment":"The FPGA evaluation is underspecified relative to the reported whole-video latencies. The text says 'Latency and power are tested using a prefill sequence length of 256' (Section 6.1) and 'we implement multiple accelerator cores on the FPGA' (Section 5), but it does not state the hardware configuration (number of cores, clock frequency, on-chip SRAM capacity, off-chip bandwidth), how the dense baseline is mapped onto the accelerator, whether the reported latency corresponds to the full generation of an 8-frame video or only a portion, or how power is measured. The power efficiency values in Tables 1 and 2 are given as GFLOPs/W, but the measured power numbers are not reported. Without this information, the claimed 2.1x speedup and energy efficiency advantages cannot be replicated or verified. Please provide a complete hardware methodology, including the mapping of the target model (or a clear statement of which components run on the FPGA) and the measurement setup.","section":"Section 5, Section 6.1, Section 6.2 (Tables 1-2)"}],"minor_comments":[{"comment":"The units of latency appear inconsistent: the header says '(s)' but dense latency of 689.7 is more plausibly milliseconds for 8-frame, 256x256 generation on an A100. Please verify and correct the units.","section":"Table 1"},{"comment":"The left panel of Figure 4 is difficult to read: the axes are not clearly labeled (the meaning of 'Ratio' and its relation to the LPIPS and VBench score curves is unclear). Please redraw with explicit axis labels.","section":"Figure 4 left"},{"comment":"References [4] and [12] are the same paper (Sun et al., 'Autoregressive model beats diffusion...'); please remove the duplicate.","section":"References"},{"comment":"The phrase 'up to sqrt(d) scaling' in Step 1 is self-contradictory: if s is scaled by sqrt(d), it is not the cosine similarity. This is related to the major scaling issue and should be clarified.","section":"Appendix 10.1, Step 1"},{"comment":"Remark 4.5 states that TAS 'does not accumulate information across layers,' but because a replay decision in a shallow layer changes the hidden state that becomes the input to deeper layers, the TAS in deeper layers is affected by earlier replay decisions. The statement should be qualified to avoid overclaiming the stability of the signal.","section":"Section 4.4, Remark 4.5"}],"recommendation":"major_revision","confidential_remarks":"The paper's empirical claims are plausible and the code availability is a plus, but the theoretical scaling error is central to the paper's claimed justification and should be fixed before acceptance. I would also ask the authors to reconcile the threshold values with the formal definition of TAS, as this may indicate that the actual implementation uses a different score than Eq. (5)."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"FastCar is worth your time if you care about efficient AR video generation. The core claim is simple and credible: in autoregressive video decoding, MLP modules dominate latency, and the MLP output for a token closely resembles the same spatial token in the previous frame. So skip the MLP when a temporal attention score says it is safe, and reuse the cached output. The experiments back this up: at 80% replay on VILA-U (8 frames, 256x256), VBench drops from 74.1% to 71.5% while FLOPs nearly halve. That is a real result.\n\nThe best parts are the profiling insight—attention is not the bottleneck in AR video, unlike diffusion—and the fact that TAS is computed for free in the existing attention module. The FPGA accelerator with dynamic resource scheduling is a meaningful extra, and the combination with sparse attention is clever and honestly reported.\n\nNow the soft spots. First, Theorem 4.4 has a scaling mismatch: Definition 4.3 divides the dot product by sqrt(d), but the proof treats st,i directly as cosine similarity. The constant C could absorb the factor, but as written the theorem does not follow. This is fixable and not load-bearing; the empirical correlation in Figure 2 is independent evidence.\n\nSecond, the replay staleness concern is real. Equation (8) reuses previous-frame MLP outputs into the next block, which changes the hidden states and future TAS values. Theorem 4.7 only bounds a single step; it says nothing about error accumulation over many frames. All quality experiments are 8 frames, so the abstract's claims about high-resolution, long-duration video are extrapolation, not evidence. This is the paper's main limitation, and it should be stated as such.\n\nThird, the 2.1x speedup headline comes from FastCar plus sparse attention, not FastCar alone (which reaches 1.76x at 80% replay). The paper does present the combined numbers fairly in Table 2, but the abstract blurs it. Fourth, the FPGA measurement is thin—no clock frequency, no per-module latency breakdown—so treat the hardware speedups as indicative rather than definitive.\n\nOverall: this is a solid, honest paper with a new idea and enough empirical support to justify serious review. The theory needs a fix and the long-horizon question needs either more frames or a real drift analysis. I would send it to reviewers and ask them to focus on the accumulation issue. For a reading group on efficient generation, it would generate good discussion; I would not cite it as the final word on long-duration video, but the replay technique itself is citable.","headline":"A genuinely new, practically useful trick—replaying MLP outputs guided by an attention score—with a solid empirical core, a fixable theory slip, and a real open question about long-horizon drift.","tokens_in":19129,"tokens_out":2274,"would_cite":true,"duration_ms":25864,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FastCar speeds auto-regressive video decoding up to 2.1x by replaying cached outputs for temporally similar tokens, with quality close to the unmodified model.","keywords":["auto-regressive video generation","temporal redundancy","MLP cache replay","temporal attention score","edge inference","FPGA acceleration","dynamic resource scheduling","video generation efficiency"],"falsifier":"Run the same model on 32- or 64-frame videos at a fixed replay threshold and check whether per-frame video quality relative to the unmodified model declines with frame index, or whether the replay ratio climbs toward 100 percent within a few frames; either outcome would show that the one-step stability bound does not cover the regime where replays chain together.","tokens_in":18040,"feed_emoji":"🎬","tokens_out":18141,"duration_ms":158921,"temperature":0.7,"pith_summary":"FastCar claims that the decode phase of auto-regressive video generation can be made substantially faster, without retraining, by exploiting temporal redundancy in the transformer's feed-forward (MLP) layers. Profiling shows that MLP modules, not attention, dominate decode latency, and that MLP outputs for the same spatial position in neighboring frames are highly similar. The framework introduces a Temporal Attention Score (TAS) read for free from the already-computed attention, and replays the previous frame's cached MLP output whenever the mean score clears a threshold. This gives edge deployments a training-free way to attack the true latency bottleneck rather than the attention modules targeted by sparse-attention methods. In the FPGA hardware implementation, the paper reports up to 1.77x speedup with replay alone at an 80% replay ratio and more than 2.1x when combined with sparse attention, with the VBench video-quality score dropping from 74.1% to 71.5%.","feed_headline":"Reusing last frame's cached output makes video generation 2.1x faster","feed_subtitle":"A temporal attention score decides which tokens can skip recomputation, keeping quality close to the unmodified model.","key_machinery":"The load-bearing object is the Temporal Attention Score (TAS), defined for a token $(t,i)$ as the mean over heads of the scaled dot product between the current token's query and the aligned previous-frame token's key, $\\bar{s}_{t,i} = \\frac{1}{h}\\sum_{m=1}^{h} \\langle q^{(m)}_{(t,i)}, k^{(m)}_{(t-1,i)}\\rangle / \\sqrt{d}$. It is the gate in the replay rule $Y_{(t,i)} = Y_{(t-1,i)}$ if $\\bar{s}_{t,i} \\ge \\tau$, and it doubles as the theoretical quantity: a chain of Lipschitz bounds (Theorems 4.4, 4.6, and 4.7) shows that high TAS, combined with similar inputs, bounds the MLP output difference. The same per-token replay pattern is then fed to a Dynamic Resource Scheduling unit on the FPGA, which redistributes skipped batches across cores to keep utilization balanced.","core_discovery":"The central discovery is that temporal redundancy in auto-regressive video transformers is directly readable from attention, and reading it enables safe cache replay. For each token $(t,i)$, the paper defines the Temporal Attention Score as the mean over attention heads of the scaled dot product between the token's query and the key of its aligned predecessor $(t-1,i)$. Because that score comes out of the attention module that already runs before the MLP, the gate costs nothing extra. The paper proves, under bounded-hidden-state and Lipschitz assumptions, that a high score bounds the difference between the current and previous MLP outputs (Theorem 4.7), and on that basis sets $Y_{(t,i)} = Y_{(t-1,i)}$ when the mean score is at least a threshold $\\tau$, otherwise computing normally. The result is a conditional computation that skips the dominant cost of decoding for tokens whose representation is changing little.","pith_inferences":["A direct extension would apply the same gate to other heavy per-token modules, such as selected attention projections or normalization, whenever a temporal attention score indicates that a token's representation is barely moving; nothing in the mechanism restricts it to MLPs.","Because replay changes the hidden states that feed the next layer, the gate creates a feedback path: a replayed output influences the next frame's temporal attention score, which decides whether the next frame replays too. The paper's experiments stop at eight frames, so measuring replay ratio and per-frame quality on 32- or 64-frame videos would show whether that feedback stays benign.","The TAS signal could be reused outside generation, for example as a cheap per-token staleness measure for scheduling cache refreshes in other autoregressive modalities such as audio or multi-view sequences, where consecutive tokens are also highly correlated."],"forward_implications":["At an 80% replay ratio the framework removes 45% of decoder computations and cuts latency by 1.77x while keeping the VBench total score at 71.5%, within 2.6 points of the unmodified model's 74.1%.","The replay decision adds no extra computation, because the temporal attention score is read from the attention module that must run anyway, so the gain scales with the share of tokens whose MLP computation can be skipped.","FastCar is complementary to sparse attention: combining the two reaches more than 2.1x speedup and keeps quality stable where sparse attention alone drops sharply, so the method can be stacked with other efficiency techniques.","Replay is distributed unevenly across layers, with shallow and deep layers replaying most and intermediate layers replaying least, which locates temporal dynamics mainly in the middle of the network.","The FPGA accelerator with Dynamic Resource Scheduling converts the workload imbalance caused by selective replay into balanced core utilization, which is what turns the computation savings into the reported latency and energy-efficiency gains."],"supporting_citations":[{"why":"The open-source auto-regressive video model whose decoder is profiled, modified, and accelerated; all quality and latency numbers are measured on it.","marker":"[37]"},{"why":"The sparse-attention method used as the primary comparison baseline and as the technique FastCar is combined with in the composite results.","marker":"[43]"},{"why":"The benchmark supplying the VBench total, quality, and semantic scores that support the claim that replay preserves generation quality.","marker":"[57]"},{"why":"The prior Lipschitz-continuity result for attention that the proof of Theorem 4.4 invokes to bound attention output differences.","marker":"[36]"},{"why":"The prior work cited as justification for treating query and key vectors as normalized in the theoretical analysis.","marker":"[41]"}],"fun_headline_variants":["Cache replay: 2.1x faster video gen on edge","Skip redundant MLP calls for 2.1x faster video","Attention-based cache replay speeds video gen 2.1x","Replay cache when attention says so: 2.1x speedup","Temporal attention gates cache reuse in video AR models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that reusing the previous frame's stored output for a token stays accurate frame after frame, even when replays chain together and feed stale values into later layers; the paper's theory bounds only one step and its experiments cover only eight-frame videos.","fun_headline_variants_meta":{"raw":{"variants":["Cache replay: 2.1x faster video gen on edge","Skip redundant MLP calls for 2.1x faster video","Attention-based cache replay speeds video gen 2.1x","Replay cache when attention says so: 2.1x speedup","Temporal attention gates cache reuse in video AR models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000921,"raw_usage":{"total_tokens":3976,"prompt_tokens":995,"completion_tokens":2981,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":611,"completion_tokens_details":{"reasoning_tokens":2892}},"tokens_in":611,"tokens_out":2981,"duration_ms":19008,"temperature":1.0,"reasoning_tokens":2892,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:47:26.012667+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same model on 32- or 64-frame videos at a fixed replay threshold and check whether per-frame video quality relative to the unmodified model declines with frame index, or whether the replay ratio climbs toward 100 percent within a few frames; either outcome would show that the one-step stability bound does not cover the regime where replays chain together.","supporting_citations":[{"cited_title":"Lazydit: Lazy learning for the acceleration of diffusion transformers","cited_arxiv_id":null,"evidence_quote":"The prior Lipschitz-continuity result for attention that the proof of Theorem 4.4 invokes to bound attention output differences."},{"cited_title":"Rossi, Hao Tan, Tong Yu, Xiang Chen, Yufan Zhou, Tong Sun, Pu Zhao, Yanzhi Wang, and Jiuxiang Gu","cited_arxiv_id":null,"evidence_quote":"The prior work cited as justification for treating query and key vectors as normalized in the theoretical analysis."}],"review_version":1}