{"id":"86829929-41a3-4a1c-b224-0dc067bc0e28","arxiv_id":"2505.15529","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Clapper achieves 13x visual token compression in video VLMs with maintained or improved QA accuracy using a slow-fast representation and a TimePerceiver module.","lead":"Clapper is a video-language model that compresses visual tokens to 61 per frame, about 13x fewer than typical encoders, while keeping QA accuracy on standard benchmarks. It combines a slow-fast frame strategy with a new TimePerceiver module to preserve both spatial detail and temporal dynamics.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Controlled 'no-compromise' evidence comes from 1/10-scale ablations; full-data comparisons are not controlled, so the 13x claim is conditional.","rationale":"The paper has real strengths: a concrete architecture, a useful fixed-token-budget evaluation idea, and competitive full-data benchmark numbers. The most load-bearing weakness is not the architectural details of TimePerceiver but the scale at which the controlled evidence is produced. The only comparisons that hold architecture and training data fixed are the ablations, and those explicitly use one-tenth of the training data. The full-data results compare against external models with different training sets, frame counts, and evaluation settings, so they cannot by themselves prove that the 13x compression causes no accuracy loss. This is a checkable, non-adversarial concern: if full-scale ablations reproduce the reduced-scale advantage, the claim is strongly supported; if not, the paper's central contribution needs qualification. The reader's conditional verdict remains appropriate, and I do not see grounds to reject or accept outright without the requested evidence.","tokens_in":13889,"tokens_out":16208,"duration_ms":143958,"concrete_test":"Train the 4x baseline, a standard Perceiver variant, and TimePerceiver using the exact full Stage 2 recipe: the full 1.6M video-instruction samples, one epoch, identical optimizer, and the same evaluation harness. Evaluate all three on VideoMME, MVBench, TempCompass, and MLVU at both 64 and 96 frames. If TimePerceiver's advantage over the 4x baseline from Table 3 (MVBench +1.8, TempCompass +2.4, VideoMME +0.2) is not reproduced at full scale, or if the standard Perceiver matches or beats TimePerceiver, then the 13x-no-compromise claim is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Clapper's central claim is that 13x per-frame compression does not compromise QA accuracy. The only controlled tests of this claim are Tables 3 and 4, but Section 4.3 states that all analysis models were trained on one-tenth of the Stage 2 data, roughly 160k samples. The full-data results in Tables 1 and 2 are not accompanied by a full-data 4x baseline or a full-data standard Perceiver trained on the same data and evaluated with the same frames and harness. External comparison models such as LLaVA-Video and MiniCPM-V differ in architecture, training data, and evaluation protocol, and Table 1 also varies frame counts between systems (Clapper uses 96 frames where some baselines use 64), so those comparisons cannot isolate the effect of token compression. Thus the 'without compromising QA accuracy' claim currently rests on reduced-scale ablations. At full data scale, the 4x baseline might improve more than TimePerceiver's measured gains, or a standard Perceiver might match TimePerceiver, which would weaken the headline contribution.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Clapper, a video VLM that reuses a SigLIP image encoder and adds a TimePerceiver module to compress each 4-frame segment into 49 temporal tokens, while retaining a spatially pooled 196-token keyframe. The resulting representation uses 245 tokens per segment, or about 61 tokens per frame, a 13x reduction from the encoder's 784 tokens per frame. Training is done in two stages on video-caption and instruction data, starting from LLaVA-OneVision-SI with a Qwen2 backbone. The paper reports results on TempCompass, MVBench, PerceptionTest, LongVideoBench, MLVU, and VideoMME, including comparisons under fixed visual-token budgets of 2k and 6k tokens. Ablations compare temporal pooling, spatial pooling, a standard Perceiver, and TimePerceiver, as well as the two-stage training design. The paper also includes a Limitations section noting the lack of length extrapolation beyond 96 frames and the absence of RLHF alignment.","tokens_in":14103,"tokens_out":6742,"duration_ms":58481,"significance":"If the full-scale results are representative, Clapper would demonstrate that strong video QA performance is achievable with roughly one thirteenth of the encoder's per-frame tokens, which is practically valuable for long-video VLM deployment. The paper's compression ratio is an architectural consequence rather than a fitted hyperparameter, and the explicit reporting of token counts and fixed-token-budget evaluations is a useful contribution to fairer benchmarking. The ablation framework is sensible and includes a comparison against a standard Perceiver. However, the headline 'without compromising QA accuracy' claim is currently supported only by 1/10-scale controlled experiments; at full scale, the comparisons are confounded by model, data, and frame-count differences. The claimed motivating degradation analysis is also absent. These issues make the contribution promising but not yet fully established.","major_comments":[{"comment":"The only controlled tests of the central '13x compression without compromising QA accuracy' claim are performed at one-tenth of the Stage 2 training data, as stated in the first sentence of §4.3. The full-data results in Tables 1 and 2 compare Clapper with externally reported models that differ in architecture, training data, and evaluation frames, so they cannot isolate the effect of token compression on accuracy. To make the headline claim stand, please add a full-data controlled comparison with at least a 4x spatial-pooling baseline and a standard Perceiver module trained on the same data and evaluated with the same frames and harness, or explicitly re-scope the claim to the reduced-scale ablations.","section":"§4.3 (Tables 3–4)"},{"comment":"The paper asserts that 'most existing VLMs suffer severe performance degradation in long video understanding tasks when compressing visual tokens below a quarter of their original visual tokens,' but no empirical analysis supporting this statement appears anywhere in the manuscript. This claim motivates the entire approach and is presented as a finding, yet there is no figure, table, or experiment showing the degradation. Please add the supporting analysis, or if it is from prior work, cite it explicitly and describe it.","section":"Abstract and §1"},{"comment":"Comparisons in Table 1 mix numbers cited from original papers with numbers obtained by integrating models into LMMs-Eval, and the frame counts differ across models (e.g., Clapper is evaluated with 96 frames while several baselines use 32 or 64). Because Video QA accuracy depends on both frame count and evaluation harness, the reported differences cannot be attributed to token compression alone. Please mark each entry as cited or re-evaluated, and provide at least one matched comparison at the same frame count and token budget for the systems that are claimed to be outperformed.","section":"§4.2, Table 1"},{"comment":"TimePerceiver constructs its queries by spatial average pooling with stride 4 followed by temporal averaging over the four frames. This design presumes that a single pooled summary of the segment is a sufficient prior for cross-attending to the full features. The paper does not ablate this query-construction choice (e.g., comparing it with learned queries or temporal-stride variants), so it is unclear whether the reported gains come from the cross-attention mechanism or from this specific pooling prior. Adding such an ablation would strengthen the causal interpretation of the module's contribution.","section":"§3.1, Figure 2"}],"minor_comments":[{"comment":"The phrase 'we achieves' should be corrected to 'we achieve'.","section":"Abstract"},{"comment":"The word 'TimePerveiver' is a typo for 'TimePerceiver' and should be corrected.","section":"§4.1"},{"comment":"The text contains inconsistent spacing artifacts such as 'LLaV A' and 'LLaV A-Video'; these should be unified as 'LLaVA' and 'LLaVA-Video' throughout.","section":"Global"},{"comment":"The axis labels and legend in Figure 3 are very small and difficult to read at print size; consider enlarging the text and adding value labels to the plotted curves.","section":"Figure 3"},{"comment":"The abbreviation 'wo' in the Table 2 caption is not defined; please spell out 'without subtitles' or define the abbreviation explicitly.","section":"§4.2, Table 2"},{"comment":"The sentence about the last video segment says that a 2–3 frame segment 'still occupies 245 tokens,' which means the per-frame token average is not exactly 61 for all videos; please clarify how the reported 61 tokens/frame figure is computed.","section":"§3.1"},{"comment":"The Limitations section states that performance 'may degrade as the video length increases' for videos longer than 5 minutes; this should be reconciled with the abstract's claim about long-video understanding.","section":"§6"}],"recommendation":"major_revision","confidential_remarks":"My overall assessment: the method is interesting and the compression ratio is an architectural consequence rather than a fitted quantity, so the work is not circular. However, the main empirical claim is currently conditional on reduced-scale ablations, and the motivating degradation analysis is not shown. This is fixable either by adding full-data controlled experiments or by carefully rewording the claim. I therefore recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take. The paper does something real: it adds a TimePerceiver module to a standard VLM and shows, in controlled small-scale ablations, that it beats both simple pooling and a standard Perceiver at aggressive compression. The fixed-token-budget evaluation protocol is a genuinely useful contribution—too many video VLM papers report accuracy without saying how many tokens they used. The architecture is simple, the compression ratio is defined clearly, and the authors include a limitations section that concedes length extrapolation and hallucination risks. Credit where due.\n\nThe soft spot is exactly what the stress-test note says. The central claim—13x compression without compromising QA accuracy—rests on Tables 3 and 4, and those models are trained on one-tenth of the Stage 2 data. The full-data numbers in Table 1 are assembled from original papers and reimplementations with differing frame counts and token counts, so they cannot isolate the effect of compression. A full-data 4x baseline might close the gap. Also, the paper's motivating empirical analysis—the claim that most VLMs degrade below a quarter of their visual tokens—is asserted but never shown. That analysis should be a figure or a table; its absence weakens the narrative.\n\nThe design assumption behind TimePerceiver—that pooled-frame summaries make good queries for cross-attending to full features—is plausible but under-examined. The authors don't probe what information the pooling discards. That's a minor concern given the benchmark range, but it is the crux of the method.\n\nNo code or weights are released, so reproducibility is currently limited. The fixed-token protocol is valuable enough that I'd encourage engagement.\n\nFor a referee: yes, this deserves a serious review. The method is incremental but practical, and the evaluation protocol is worth standardizing. The referee should ask for a full-data baseline and the missing empirical analysis, but the core architecture is sound.","headline":"Useful, honest efficiency work on video VLMs, but the headline no-compromise 13x claim is only backed by 1/10-scale ablations; full-data results are competitive but not controlled.","tokens_in":14629,"tokens_out":2040,"would_cite":false,"duration_ms":19490,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Clapper compresses video tokens 13x and keeps QA accuracy across VideoMME, MLVU, and TempCompass.","keywords":["video understanding","vision-language model","token compression","temporal modeling","TimePerceiver","slow-fast representation","video question answering","efficient inference"],"falsifier":"Take a benchmark or a synthesized set of short clips where the correct answer depends on a small, brief, spatially localized change—a card flip, a counter changing, an object appearing in one frame—and compare Clapper at 61 tokens per frame against its 4x baseline at 196 tokens per frame on the same frames. If accuracy on those items falls well below the baseline while static-spatial items stay flat, the pooled prior has discarded the needed detail and the 13x claim does not hold for fine-grained temporal reasoning.","tokens_in":13695,"feed_emoji":"🎬","tokens_out":5790,"duration_ms":47098,"temperature":0.7,"pith_summary":"Clapper claims that a video language model can compress the visual tokens produced per frame by about 13x—from 784 tokens per frame out of the image encoder to 61—without paying a quality penalty on multiple-choice video QA, and that this removes the usual trade-off where aggressive compression degrades long-video understanding. The paper argues that the right division of labor is a slow-fast representation: keep one high-resolution keyframe per four-frame segment for spatial detail, and let a small trainable cross-attention module call TimePerceiver summarise the temporal changes across those four frames into 49 tokens. With that design Clapper reports 62.0% on VideoMME, 69.8% on MLVU, and 67.4% on TempCompass, all under 6,000 visual tokens per video, and it documents that current VLMs degrade when forced below a quarter of their original token count. If true, this makes long-video QA practical at much lower compute and gives the field a way to compare models under a fixed visual token budget.","feed_headline":"Clapper shrinks video tokens 13x without losing QA accuracy","feed_subtitle":"With ~61 tokens per frame, it beats far larger-token models on three video QA benchmarks under 6k-token budgets.","key_machinery":"The load-bearing mechanism is TimePerceiver, a trainable module that takes 2–4 frame features from the vision encoder and emits a fixed 49 tokens per segment. Its input queries come from aggressive spatial (stride 4) then temporal average pooling, so the queries encode an averaged summary of what the segment looks like; those queries then cross-attend to the full-resolution flattened frame features, with keys and values computed from the concatenation of the pooled summary and the full features. This lets the module spend its few output tokens on regions that actually change across frames, while the separately retained high-resolution keyframe carries static scene and object details. The combination moves compression from 4x (spatial pooling applied to every frame) to 13x while improving or holding accuracy on MVBench, TempCompass, and VideoMME.","core_discovery":"The paper's central claim is that a Perceiver-style module whose cross-attention queries are derived from a heavily pooled summary of four frame features preserves the information needed for video QA while cutting tokens by 13x. Concretely, each four-frame segment is represented by a 196-token keyframe (spatial pooling of the first frame with stride 2) plus a 49-token TimePerceiver output, so a segment costs 245 tokens rather than the 3136 tokens the encoder would emit for those four frames. The TimePerceiver builds its queries by spatial average pooling with stride 4 and then temporal average pooling over the four frames, then cross-attends to the full flattened features, with keys and values formed from the concatenation of that pooled prior and the full features. On six video QA benchmarks, the method is competitive with or better than models using many more tokens per frame, and it outperforms simpler 16x compressions (temporal pooling, spatial pooling, plain Perceiver) on the frame-sensitive VideoMME benchmark.","pith_inferences":["One consequence the paper leaves implicit: the 49-token temporal summary is only as good as the pooled prior that seeds its queries, so questions whose answer lives in a single brief high-frequency event at small spatial scale may be the first to break as compression is pushed further.","A natural extension is to make the keyframe-to-temporal-token ratio adaptive—allocating more keyframes in high-motion segments and more temporal tokens in static segments—which TimePerceiver's fixed 49-token budget currently cannot do.","The paper's fixed-token-budget evaluation could be adopted as a reporting protocol for other video VLMs; this editorial suggestion would separate genuine quality gains from gains that merely come from feeding more tokens to the model.","A testable extension is to push Clapper beyond 96 frames by training on longer sampled sequences or by adding length-extrapolation methods, which the paper identifies as a limitation and as future work."],"forward_implications":["Under a fixed 2k or 6k visual-token budget on VideoMME, Clapper beats compared open models at the same budget, including on short and medium videos, so deployment can trade tokens for accuracy predictably.","Because average tokens per frame is independent of the number of frames, the same representation can extend to more frames and longer videos inside an LLM context; the paper's frame-sweep results show MLVU improving steadily as frames increase from 16 to 128.","The ablation results imply that compressing to 13x with TimePerceiver generalizes better than 16x pooling or a plain Perceiver, and that the two-stage training recipe (video-caption warm-up then instruction tuning) is needed for those gains, since direct fine-tuning hurts VideoMME.","Reporting both the number of frames and the average tokens per frame becomes a meaningful evaluation norm, because models with more tokens per frame are not directly comparable unless the total visual token budget is fixed."],"supporting_citations":[{"why":"Supplies the SigLIP image encoder that outputs the 784-token frame features Clapper compresses.","marker":"(Zhai et al., 2023)"},{"why":"The LLaVA-OneVision 7B model is the base VLM that Clapper finetunes, providing the vision encoder, MLP, and LLM backbone.","marker":"(Li et al., 2024a)"},{"why":"Provides the Perceiver-style cross-attention design that TimePerceiver adapts, including the concatenation of the pooled summary with keys and values.","marker":"(Alayrac et al., 2022)"},{"why":"Supplies the LLaVA-Video-178K video-caption pairs used in stage-1 warm-up and additional video instruction samples used in stage-2 fine-tuning.","marker":"(Zhang et al., 2024f)"},{"why":"VideoMME is the primary frame-sensitive benchmark used for overall results and the fixed-token-budget analysis.","marker":"(Fu et al., 2024)"},{"why":"MLVU is the long-video benchmark where Clapper reports 69.8% and shows steady gains with more frames.","marker":"(Zhou et al., 2024)"},{"why":"TempCompass is the short-video temporal-comprehension benchmark where Clapper reports 67.4%.","marker":"(Liu et al., 2024b)"},{"why":"LMMs-Eval is the evaluation harness used to standardize benchmark results across models.","marker":"(Zhang et al., 2024a)"}],"fun_headline_variants":["13x token cut: Clapper's TimePerceiver keeps video QA accuracy","Clapper: 61 tokens per frame, beats big-token models on video QA","Slow-fast video encoding: Clapper compresses 13x without QA loss","TimePerceiver module: Clapper slashes video tokens 13x","Clapper: Under 6k tokens per video, still tops video QA benchmarks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that a summary of each four-frame segment produced by average-pooling in space and time is a good enough prior that cross-attention can recover the dynamic details the 49 tokens should carry; if that prior drops a detail a question needs, the 13x compression loses accuracy no matter how the cross-attention is trained.","fun_headline_variants_meta":{"raw":{"variants":["13x token cut: Clapper's TimePerceiver keeps video QA accuracy","Clapper: 61 tokens per frame, beats big-token models on video QA","Slow-fast video encoding: Clapper compresses 13x without QA loss","TimePerceiver module: Clapper slashes video tokens 13x","Clapper: Under 6k tokens per video, still tops video QA benchmarks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000229,"raw_usage":{"total_tokens":1499,"prompt_tokens":984,"completion_tokens":515,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":600,"completion_tokens_details":{"reasoning_tokens":412}},"tokens_in":600,"tokens_out":515,"duration_ms":4220,"temperature":1.0,"reasoning_tokens":412,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:14:51.627327+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a benchmark or a synthesized set of short clips where the correct answer depends on a small, brief, spatially localized change—a card flip, a counter changing, an object appearing in one frame—and compare Clapper at 61 tokens per frame against its 4x baseline at 196 tokens per frame on the same frames. If accuracy on those items falls well below the baseline while static-spatial items stay flat, the pooled prior has discarded the needed detail and the 13x claim does not hold for fine-grained temporal reasoning.","supporting_citations":[{"cited_title":"Menick, Sebastian Borgeaud, Andy Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikolaj Binkowski, Ricardo Barreira, Oriol Vinyals, Andrew Zisserman, and Kar \\' e n Simonyan","cited_arxiv_id":null,"evidence_quote":"Provides the Perceiver-style cross-attention design that TimePerceiver adapts, including the concatenation of the pooled summary with keys and values."}],"review_version":1}