{"id":"817c4e84-3ab6-49a1-bb58-222432642930","arxiv_id":"2505.08561","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A trajectory-attention token sampler trained jointly with a masked video autoencoder via PPO improves downstream action recognition accuracy over random- and activity-based masking baselines.","lead":"This paper adds a trainable sampler to masked video autoencoders that learns to hide the most motion-informative tokens instead of using random or fixed masks, and trains the sampler with the PPO reinforcement learning algorithm. Across four action-recognition benchmarks, the authors report consistent gains over the VideoMAE and AdaMAE baselines at mask ratios from 0.85 to 0.95, with comparable or lower pre-training memory.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"PPO objective in Eq. (7)-(8) uses an importance ratio over a different masked-set distribution than the stored episodes, so the policy-gradient estimate is not a valid PPO update as written.","rationale":"The reader's weakest assumption was the reconstruction-error-as-informativeness reward, and the reader noted the PPO importance-ratio inconsistency only as one of several concerns. I agree the reward assumption is unvalidated, but the more load-bearing issue is the PPO objective itself: Eq. (8)'s ratio r(theta) = pi_theta(Im'|X)/pi_theta_old(Im|X) compares probabilities of different masked sets, so it does not estimate any policy gradient for the stored reward. This is not merely a notation slip; it affects the central methodological contribution of 'unified training using PPO from scratch'. The empirical results could still be valid if the implementation uses the correct ratio, but the paper as written does not establish its key algorithmic claim. The concrete test—computing the correct ratio and rerunning the small-scale UCF101 experiment—would settle whether the issue is pedagogical or substantive. I therefore keep the reader's CONDITIONAL verdict: the empirical claim is plausible and reproducible code is promised, but the algorithmic inconsistency must be resolved before acceptance. I agree with the reader's identification of the PPO inconsistency, though I weight it more heavily than the reward-design assumption.","tokens_in":18829,"tokens_out":1841,"duration_ms":16118,"concrete_test":"Re-derive the policy-gradient objective with the actual sampling distribution. Concretely, compute the correct off-policy ratio for sampling a set without replacement: for a stored episode (X, Im, LR), the ratio should be pi_theta(Im|X) / pi_theta_old(Im|X), where both numerator and denominator are evaluated on the same stored masked set Im. Implement this corrected ratio (computing full-set log-probabilities by summing log-probabilities of sequentially sampled tokens) and re-run the UCF101 rho=0.95 experiment. If the corrected objective trains stably and achieves similar fine-tuning accuracy, the concern is a notation/implementation fix; if training diverges or accuracy drops below AdaMAE, the paper's PPO formulation was load-bearing and needs revision.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that TATS and the MAE can be jointly optimized from scratch with PPO, and that this PPO training stably selects motion-centric tokens. The paper's own equations do not instantiate a valid PPO update. In Algorithm 1 Phase 1, episodes store pi_theta_old(Im|X), the probability of the full masked set Im, together with reward LR(phi) computed from that Im. In Phase 2, the current policy samples a new set Iv' ~ Categorical(N, pi_theta(X)), giving Im' = complement(Iv'), and Eq. (8) defines r(theta) = pi_theta(Im'|X) / pi_theta_old(Im|X). This ratio compares probabilities of two different masked sets sampled from different distributions; it is not the standard importance ratio pi_theta(a|s)/pi_old(a|s) for a single action a. The stored pi_theta_old(Im|X) is not the probability of the new Im' under the old policy, and pi_theta(Im'|X) is not evaluated on the action that produced the stored reward. Moreover, Section 3.2 defines a categorical distribution over individual tokens and samples without replacement, so the probability of a full set is not a simple product and is never computed. The advantage A = LR - psi_theta_old(X_mu) is also stored from an old value estimate but re-used with current-policy ratios. As written, J_CLIP is not a lower-bound surrogate for any well-defined policy gradient objective, so the claim that PPO stabilizes joint training from scratch is not supported by the equations, even though the empirical recipe may still work. The reader flagged this as an internal inconsistency in the importance ratio; I agree and consider it the most load-bearing weakness because it undermines the paper's main methodological contribution (the unified PPO training recipe), independent of the reconstruction-error reward assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TATS (Trajectory-Aware Adaptive Token Sampler), a token-selection module for masked video autoencoders, and a training recipe that alternates between optimizing the MAE and TATS with a PPO-style objective. TATS uses Trajectory Attention to define a per-token sampling distribution, masks space-time tokens accordingly, and is trained with a reward based on the reconstruction error of the masked tokens. The authors report fine-tuned action recognition results on UCF101, HMDB51, a K400 subset, a SSv2 subset, and full SSv2 at mask ratios 0.85/0.90/0.95, claiming consistent gains over VideoMAE and AdaMAE with pretraining memory below AdaMAE. They also provide ablations on attention type, decoder depth, reconstruction loss, TA block count, and memory usage, plus qualitative mask visualizations.","tokens_in":19150,"tokens_out":8019,"duration_ms":72549,"significance":"If the methodological issues are resolved, the work is a useful step toward learned, input-adaptive masking in video masked autoencoders. The paper provides internally consistent comparisons with baselines trained under the same schedule, releases source code, evaluates on four benchmarks with a large-scale SSv2 run, and includes qualitative evidence that the learned masks concentrate on moving regions. The memory comparison against AdaMAE is a concrete practical contribution. However, the central theoretical novelty, the PPO-based joint training, is not correctly specified in the current equations, and the UCF101/HMDB51 results are confounded by hyperparameter selection on the same datasets. The significance of the empirical claims therefore cannot be fully assessed in the present form.","major_comments":[{"comment":"The PPO surrogate in Eq. (8) is not a valid policy-gradient objective as written. In Phase 1 the stored episode contains pi_theta_old(Im|X) and the reward LR(phi) for the masked set Im actually used. In Phase 2 the current policy samples a new set Im' and Eq. (8) sets r(theta)=pi_theta(Im'|X)/pi_theta_old(Im|X). A valid importance ratio for PPO must be pi_theta(a|s)/pi_old(a|s) for the same action a whose reward is used; here the numerator is evaluated at Im' while the denominator and the reward correspond to Im, so the ratio mixes probabilities of two different actions and the stored baseline psi_old is not the baseline of Im'. Consequently J_CLIP is not a surrogate for any well-defined policy-gradient objective, and the claim that PPO stabilizes joint training is unsupported by the equations. Please correct the update (e.g., use r(theta)=pi_theta(Im|X)/pi_old(Im|X) with the stored episodes, or sample a fresh set and recompute the reward), and define the probability of a full masked set under the without-replacement categorical sampling of Section 3.2, since it is not the product of per-token probabilities.","section":"Section 3.3, Eq. (7)-(8); Algorithm 1"},{"comment":"The fine-tuning results on UCF101 and HMDB51 in Table 1 are not independent of hyperparameter selection. Tables 5 and 6 select (mo,k) and (c1,c2,c3) by minimizing reconstruction error on UCF101 and HMDB51, and the same datasets' downstream accuracies are then reported in Table 1. Although the selection uses an unsupervised reconstruction criterion rather than action labels, it is still a form of model selection on the datasets whose fine-tuning numbers are the headline results, and the baselines are run with default configurations. Please either tune hyperparameters on a held-out validation split (or on K400/SSv2 only) before reporting UCF101/HMDB51, or provide an explicit argument that reconstruction-error-based selection cannot bias the reported downstream comparison.","section":"Supplementary Section 8.2, Tables 5 and 6; Table 1"},{"comment":"The reward design assumes that reconstruction error is a valid proxy for token informativeness: tokens with low reconstruction error are called redundant, while motion-centric tokens are assumed to have high reconstruction error. This premise is asserted rather than demonstrated, and the whole policy objective rests on it. If high reconstruction error instead reflects noise, occlusion, or rare static texture, the sampler would not select motion. The paper should supply quantitative evidence for the proxy, for example by correlating TATS-selected tokens with optical flow or annotated motion regions, or by including an ablation with a reversed/inverted reward to show that the direction of the reward, not just the added capacity, drives the gains.","section":"Section 3.3"}],"minor_comments":[{"comment":"The text and pseudocode write Im = Iv, but Im should be the complement of Iv; as written, the masked and visible sets are identical.","section":"Section 3.2 and Algorithm 1, line 10"},{"comment":"The text says the best decoder depth is '# Blocks = 1' with 81.75%, but Table 3b reports 81.75% for '# Blocks = 4' and 81.46% for '# Blocks = 1'. Please reconcile the text and table.","section":"Section 4.2, item 2 vs Table 3b"},{"comment":"The entropy term H(X,pi_theta)(.) is not defined; write H(pi_theta(.|X)) and state whether it is averaged over the sampled set or over all tokens.","section":"Eq. (7)"},{"comment":"The claim that pretraining is memory efficient should be qualified as relative to AdaMAE only: Table 3c shows a saving of less than 1 GB over AdaMAE and an increase of about 4.4 GB over VideoMAE.","section":"Abstract and Section 5; Table 3c"}],"recommendation":"major_revision","confidential_remarks":"The main gating issue is the PPO update in Eq. (8)/Algorithm 1, which is not a valid policy-gradient estimator as written; the hyperparameter-selection issue for UCF101/HMDB51 is also important. Both are fixable within the manuscript's scope, so I recommend major revision rather than rejection. If the authors can correct the objective to a well-defined surrogate, or clarify that the implementation differs from the written equations, the paper could become publishable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is real: use trajectory attention to score spatiotemporal tokens and train the scorer jointly with a masked autoencoder through a clipped policy-gradient objective. That combination is new relative to AdaMAE's REINFORCE-based sampler, and it avoids external optical flow or semantic models. The empirical work is also honest and fairly complete for a small-scale study: consistent top-1 gains over VideoMAE and AdaMAE at mask ratios 0.85, 0.90, and 0.95 across UCF101, HMDB51, a K400 subset, and an SSv2 subset, plus transfer results, ablations, memory numbers, and a large-scale SSv2 run with a clear margin. The code is released. That is real evidence, and the paper deserves a serious referee.\n\nBut the central methodological claim—a unified PPO recipe for stable joint training—does not hold up as written. In Eq. (7)–(8) and Algorithm 1, the importance ratio compares pi_theta(Im'|X) under the current policy with pi_theta_old(Im|X) stored for a different masked set. The stored reward and advantage also come from Im, not Im'. That is not the standard PPO ratio pi_theta(a|s)/pi_old(a|s) for a single action. On top of that, the policy is a categorical distribution over tokens sampled without replacement, so the probability of a full masked set is not a simple product and is never computed. As written, J_CLIP is not a lower-bound surrogate for any well-defined policy gradient objective. The empirical recipe may still work—the results suggest it does—but the equations do not specify the method, and the paper's main contribution is the training recipe itself. This is a load-bearing flaw, not a cosmetic one.\n\nOther concerns are milder. There are no error bars or multi-seed runs anywhere, which matters for a 2–4 point gain over AdaMAE. The hyperparameters (mo, k, c1–c3) are tuned by minimizing reconstruction error on UCF101 and HMDB51, the same datasets used for final evaluation (supp Tables 5–6), so there is some selection risk, though the consistent gains on K400 and SSv2 help. The baseline set is only VideoMAE and AdaMAE, so the \"state-of-the-art\" language in the abstract and conclusion overreaches. The reward assumption—that high reconstruction error marks informative motion tokens—is plausible but never independently validated; it could in principle select noise or rare patches.\n\nWho is this for? Researchers working on adaptive masking for video MAE, particularly those interested in RL-based token selection. The empirical findings are worth paying attention to, but the PPO formulation needs to be corrected or clearly reframed as an approximate heuristic with proper justification. I would send it to review, with a strong request for major revision and validation of the RL objective.","headline":"A genuinely new adaptive-masking recipe with solid empirical gains over two baselines, but the PPO objective as written is not a valid PPO update, and that flaw sits at the center of the paper's main methodological claim.","tokens_in":19792,"tokens_out":2050,"would_cite":false,"duration_ms":21469,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A learned token sampler watches how video tokens move and keeps the motion-rich ones visible, so masked video autoencoders can pre-train with up to 95% of tokens hidden and still fine-tune to strong action-recognition accuracy.","keywords":["masked video modeling","adaptive token sampling","trajectory attention","proximal policy optimization","action recognition","self-supervised pre-training","video transformer","token masking"],"falsifier":"Compare the tokens TATS keeps visible against tokens marked by dense optical flow or frame differencing on a held-out set of videos: if the intersection-over-union between selected tokens and true motion regions is no better than random, and downstream accuracy is unchanged when the reward is shuffled, the claim that trajectory-guided selection drives the gains would be falsified.","tokens_in":18586,"feed_emoji":"🎬","tokens_out":5269,"duration_ms":51256,"temperature":0.7,"pith_summary":"The paper tries to show that masked video pre-training does not need to guess where to mask: a small learned module can watch how tokens move across frames and choose which tokens to keep visible. That module, trained jointly with the video autoencoder by reinforcement learning, is meant to keep motion-carrying tokens available while hiding redundant ones, allowing pre-training at 85 to 95 percent mask ratios without losing downstream action-recognition accuracy. If this works, aggressive masking can be adaptive rather than random, and pre-training compute is spent on tokens that actually matter. The authors report consistent fine-tuning gains over a random-masking baseline and a REINFORCE-trained adaptive-masking baseline across four action-recognition benchmarks.","feed_headline":"Reinforcement learning picks the video tokens worth keeping","feed_subtitle":"A learned sampler keeps motion-rich tokens visible under heavy masking, lifting action-recognition accuracy without optical flow.","key_machinery":"The central object is TATS, a two-branch module whose policy branch runs Trajectory Attention (a linear-complexity attention that pools features along the probabilistic path of each token between frames) through a linear layer and softmax to produce a categorical distribution over space-time tokens, while a value branch predicts expected reward from the mean token representation. Sampling visible tokens from that distribution defines the mask. PPO with a clipped objective, a value loss, and an entropy term updates the sampler using reconstruction error as reward, and an alternating freeze-and-unfreeze schedule lets the autoencoder and sampler co-train from scratch. The machinery's job is to turn 'mask the right tokens' into a learnable policy rather than a hand-designed heuristic.","core_discovery":"The central claim is that a trajectory-aware token sampler (TATS) can be inserted into a masked video autoencoder and optimized end-to-end with PPO so that it learns to select motion-centric space-time tokens. The reward for the sampler is the autoencoder's reconstruction error: tokens that are hard to reconstruct are treated as informative and worth keeping visible. With this reward and a two-phase alternating update scheme that keeps the two networks stable when trained from scratch, the paper reports that pre-training under mask ratios 0.85, 0.90, and 0.95 yields fine-tuned action-recognition accuracy above both a random tube-masking baseline and a REINFORCE-trained adaptive sampler, across UCF101, HMDB51, Kinetics-400, and Something-Something v2, with pre-training memory lower than the adaptive baseline.","pith_inferences":["Beyond the paper's comparisons, the reconstruction-error reward is doing double duty: it selects tokens and also acts as a curriculum signal, and it would be worth testing whether a cheap motion estimate such as frame differencing produces the same masks.","Because TATS only needs token trajectories, the same recipe could apply to masked image modeling with pseudo-motion from augmentations or multi-view images, or to point-cloud and 3D token streams where trajectories are explicit.","The alternating PPO schedule is a general template for any masking policy whose reward comes from a co-trained reconstruction network; a testable consequence is that replacing the categorical sampler with a differentiable Gumbel-softmax relaxation should match its performance if the reconstruction-error reward premise is correct.","If the reward is replaced by shuffled or noise-corrupted reconstruction errors, the selected masks and downstream accuracy should degrade; if they do not, reconstruction error is not the informative signal the paper claims."],"forward_implications":["Adaptive trajectory-based masking makes very aggressive masking practical: at mask ratios up to 0.95, fine-tuned action-recognition accuracy stays above that of fixed or randomly masked baselines trained at the same ratio.","The learned masking policy transfers across datasets, so pre-training on one benchmark and fine-tuning on another gives consistent accuracy gains over the baselines in most settings.","No optical flow or external semantic model is required; the trajectory signal is learned from raw video, keeping the pre-training pipeline self-contained and its memory use comparable to a standard masked video autoencoder.","The two-phase PPO training recipe provides a stable way to jointly optimize a masking policy and the reconstruction network that supplies its reward, which is the main obstacle to adaptive masking in this setting.","The TATS module can be dropped into an existing masked autoencoder architecture without changing the reconstruction target or the encoder-decoder design."],"supporting_citations":[{"why":"provides the adaptive-masking baseline with a REINFORCE-trained token sampler that TATS extends and is compared against.","marker":"[4]"},{"why":"supplies the standard random tube-masking baseline and the VideoMAE encoder-decoder backbone that TATS is integrated into.","marker":"[52]"},{"why":"provides the PPO clipped surrogate objective used to stabilize joint optimization of the sampler and the autoencoder.","marker":"[46]"},{"why":"defines Trajectory Attention, the mechanism TATS uses to model token motion paths between frames.","marker":"[42]"},{"why":"introduces the masked autoencoder framework whose video variant and reconstruction target the paper adopts.","marker":"[23]"},{"why":"represents the motion-guided masking approach that relies on external optical flow, the dependency TATS aims to remove.","marker":"[25]"}],"fun_headline_variants":["RL-guided token sampling makes video masking adaptive","Trajectory-aware sampler picks motion tokens for video MAE","PPO-trained sampler keeps action-relevant tokens visible","Learn to mask: RL selects the video tokens that matter","Heavy video masking? RL finds the tokens worth keeping"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that reconstruction error measures informativeness: the sampler is rewarded for keeping tokens that are hard to reconstruct, assuming those are exactly the motion-bearing tokens; if hard-to-reconstruct tokens are instead noisy or rare patches, the selection target is wrong.","fun_headline_variants_meta":{"raw":{"variants":["RL-guided token sampling makes video masking adaptive","Trajectory-aware sampler picks motion tokens for video MAE","PPO-trained sampler keeps action-relevant tokens visible","Learn to mask: RL selects the video tokens that matter","Heavy video masking? RL finds the tokens worth keeping"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000286,"raw_usage":{"total_tokens":1682,"prompt_tokens":945,"completion_tokens":737,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":561,"completion_tokens_details":{"reasoning_tokens":659}},"tokens_in":561,"tokens_out":737,"duration_ms":7804,"temperature":1.0,"reasoning_tokens":659,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:51:35.881377+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare the tokens TATS keeps visible against tokens marked by dense optical flow or frame differencing on a held-out set of videos: if the intersection-over-union between selected tokens and true motion regions is no better than random, and downstream accuracy is unchanged when the reward is shuffled, the claim that trajectory-guided selection drives the gains would be falsified.","supporting_citations":[{"cited_title":"Adamae: Adaptive masking for efficient spatiotempo- ral learning with masked autoencoders","cited_arxiv_id":null,"evidence_quote":"provides the adaptive-masking baseline with a REINFORCE-trained token sampler that TATS extends and is compared against."},{"cited_title":"Video- MAE: Masked autoencoders are data-efficient learners for self-supervised video pre-training","cited_arxiv_id":null,"evidence_quote":"supplies the standard random tube-masking baseline and the VideoMAE encoder-decoder backbone that TATS is integrated into."},{"cited_title":"Keeping your eye on the ball: Trajectory at- tention in video transformers","cited_arxiv_id":null,"evidence_quote":"defines Trajectory Attention, the mechanism TATS uses to model token motion paths between frames."},{"cited_title":"Masked autoencoders are scalable 9 vision learners","cited_arxiv_id":null,"evidence_quote":"introduces the masked autoencoder framework whose video variant and reconstruction target the paper adopts."},{"cited_title":"Mgmae: Motion guided masking for video masked autoencoding","cited_arxiv_id":null,"evidence_quote":"represents the motion-guided masking approach that relies on external optical flow, the dependency TATS aims to remove."}],"review_version":1}