{"id":"99ac590d-83ad-41db-86e5-c2adf00301cd","arxiv_id":"2412.07704","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"GEXIA expands a single-grained video-text dataset into multiple granularities and aligns them with granularity-dependent iterative attention, achieving strong long-form video understanding results.","lead":"This paper describes a way to train video-language models on multi-grained data by expanding a short-clip dataset via concatenation and LLM summarization, and aligning the resulting pairs with an attention module that runs a variable number of iterations. The method reports strong results on long-form video benchmarks, even though the expanded pretraining data originates from short clips.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Long-form gains may be an input-length artefact: Table 1 compares against baselines using fewer frames and shorter text, so the GEX/IAM contribution is not isolated.","rationale":"The reader's stated weakest_assumption is that GEX-concatenated clips lack the narrative structure of natural long videos, so pretraining on them may not transfer. That assumption is partly mitigated by the paper itself: Appendix A, Table 6, shows random concatenation still gives a 3.2-point R@1 improvement over SVST-only data, so temporal narrative coherence is not necessary for the transfer effect. A more load-bearing issue is that the strongest evidence for long-form skill compares against baselines with shorter inputs: GEXIA uses 32 frames and up to 77 text tokens on ActivityNet, while the reported ViCLIP and CLIP4Clip comparisons typically use fewer frames and shorter text. The paper's internal GEX ablation, which controls #iter but not input length, shows only a small fraction of the headline gap, so the large zero-shot ActivityNet margin over ViCLIP cannot be cleanly attributed to GEX or IAM. This does not invalidate the method, and I credit the paper for including ablations of data composition, #iter sensitivity, encoder initialization, and random concatenation. However, the central claim as stated needs a matched-input baseline before unconditional acceptance. My recommendation therefore remains conditional, matching the reader's verdict; the concern strengthens the rationale rather than moving the verdict.","tokens_in":23792,"tokens_out":13941,"duration_ms":134988,"concrete_test":"Run the pretrained ViCLIP checkpoint on ActivityNet Captions zero-shot T2V/V2T using GEXIA's exact inference configuration: d=32 frames with the same sampling scheme, text kept to 77 tokens, and no GEX/IAM modification. If this baseline moves substantially from 15.1 toward the reported 35.3, the long-form advantage is largely an input-length artefact; if the gap remains, the multi-grained pretraining story is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central long-form claim rests on Table 1, especially the ActivityNet zero-shot result: GEXIA reaches 35.3/35.0 T2V/V2T R@1 while ViCLIP, trained on the same underlying InternVid-10M-FLT corpus, reaches 15.1/24.0 with a larger ViT-L/14 backbone. The comparison is not controlled for input length. Section 4.1 sets d=32 frames for long videos, and Section 3.2 truncates/pads text to the full CLIP context (77 tokens); ActivityNet paragraphs average 49.2 words (Appendix E). ViCLIP and CLIP4Clip, as commonly configured, sample 8-12 frames and truncate text to 32 tokens. Thus the 20-point zero-shot gap may be largely from seeing more of the video and query, not from GEX multi-granularity or IAM. The paper's own GEX ablation (Table 3) fixes #iter but evaluates with the same increased input length and shows only a 1.5-3.7 R@1 gain from adding LVLT/LVST. Table 4b also shows the chosen #iter is not uniquely determined: on LSMDC, the 3-3 setup gives higher T2V R@1 (14.1) than the chosen 1-1 setup (12.9). Without a plain CLIP baseline using the same frame count and text truncation, the headline 'excels in long-form' is not causally attributed to the proposed method.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GEXIA, a video-language pretraining framework with two components: GEX, a data pipeline that expands a single-grained dataset into multiple granularities by temporal concatenation (Integration) and LLM summarization (Compression), and IAM, an iterative cross-attention module that maps variable-length dense features to fixed-size embeddings with an input-length-dependent iteration count. The model is pretrained on InternVid-10M-FLT and evaluated on retrieval (ActivityNet, MSR-VTT, LSMDC), classification (LVU, COIN, Charades-Ego), and VQA (How2QA). The paper reports state-of-the-art or comparable results and highlights long-form video understanding gains.","tokens_in":24093,"tokens_out":8774,"duration_ms":71369,"significance":"If the results are validated, the GEX pipeline is a low-cost way to create multi-grained pretraining data without data collection, and the IAM offers a straightforward architecture for cross-modal alignment across varying input lengths. The strong results on COIN and LVU suggest the approach has practical merit. However, the key claim of long-form superiority is currently confounded by uncontrolled input-length differences between GEXIA and the baselines, and the #iter hyperparameter appears to be selected on test metrics. These issues must be resolved before the headline claims can be accepted.","major_comments":[{"comment":"The central long-form claim is confounded by input length. GEXIA uses d=32 frames for long videos and the full CLIP text context (77 tokens), whereas the compared baselines (e.g., ViCLIP, CLIP4Clip) typically sample 8–12 frames and truncate text to 32 tokens. Under this confound, the 20-point zero-shot T2V gain on ActivityNet over ViCLIP (35.3 vs 15.1) cannot be attributed to the proposed GEX/IAM method. The paper's own ablation in Table 3 shows that, with the same input length (and #iter=1), adding the multi-grained LVLT/LVST data improves T2V R@1 by only 1.5 points (41.6 to 43.1). To support the 'excels in long-form' claim, the authors should compare against a plain CLIP/mean-pooling baseline that uses the same d=32 frames and 77-token text, and should include a no-IAM (or #iter=0) ablation on the same dense features.","section":"Section 4.1, Table 1, Table 3"},{"comment":"The per-dataset selection of #iter is made on the reported test metrics, and the stated rule is not consistently supported by the data. On LSMDC zero-shot T2V R@1, the 3-3 setup (14.1) outperforms the chosen 1-1 setup (12.9), which contradicts the claim that #iter should be higher for longer inputs and lower for short ones. Since the authors do not report results on a validation split, the reported numbers—especially the zero-shot and finetuned exceptions—are likely optimistic. The authors should either pre-register a fixed rule for #iter (e.g., based on input length thresholds) and report a validation-based selection, or provide error bars over multiple seeds.","section":"Section 4.4, Table 4b"},{"comment":"The GEX assumption that temporally concatenating same-source clips yields realistic long videos is not validated. The ablation in Table 6 shows random concatenation also gives a large gain over SVST only (44.8 vs 41.6 T2V R@1), while the additional gain from source-ID ordering is small (45.3 vs 44.8). This suggests that simply presenting the model with longer visual input drives most of the improvement, and the proposed integration strategy adds little. The paper should analyze the synthetic long videos (e.g., narrative continuity, shot distribution) and, more importantly, run a controlled experiment that uses the SVST-only pretrained model with d=32 at inference on long-video benchmarks to determine whether GEX is actually necessary for the long-form gains.","section":"Section 3.1, Table 6"}],"minor_comments":[{"comment":"The phrase 'A#iter V angA#iter T' contains a typo; it should be 'A#iter V and A#iter T'.","section":"Section 3.2"},{"comment":"The number of finetuning epochs is inconsistent: 50 epochs for retrieval in Section 4.2, but 100 epochs for other downstream tasks in Section 4.3; please clarify.","section":"Section 4.2"},{"comment":"The y-axis ranges are narrow (34.0–35.5 for R@1), which visually exaggerates the effect; include error bars or start the axis at zero.","section":"Figure 3"},{"comment":"The grouping of rows into 'Zero-shot', 'Finetuned', 'Model with Similar FLOPs', and 'Retrieval-specific Model' is confusing because the categories overlap; consider a clearer structure or a separate table.","section":"Table 1"},{"comment":"The paper does not mention code or data release; a reproducibility statement would be helpful.","section":"Reproducibility"}],"recommendation":"major_revision","confidential_remarks":"The paper is from an industry lab and uses a self-hosted LLM for summarization; the authors should ensure the LLM setup is described sufficiently. The main concern for the editor is the experimental confound, which is substantial but fixable with controlled comparisons. The method's novelty over Perceiver-VL is incremental, but the GEX data pipeline is a practical contribution if the confound is resolved."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, what's actually new: GEX is a simple, scalable way to turn a single-grained dataset into a multi-grained one by concatenating clips from the same source and LLM-summarizing the concatenated text. That is a legitimate contribution, and the paper tests it fairly well: random concatenation still helps, source-ID ordering helps more, and the short summaries are semantically close to the long texts (cosine 0.79). IAM is essentially Perceiver-style cross-attention with a granularity-dependent iteration count and one unrolled iteration; it's a sensible engineering choice, not a conceptual leap. The evaluation spans seven datasets and includes a nice extension to image-text with #iter=0.\n\nNow the soft spots, in order.\n\n1) The long-form headline is not isolated. Table 1's ActivityNet zero-shot comparison pits GEXIA (32 frames, full 77-token text) against ViCLIP, which as commonly configured samples far fewer frames and truncates text. The 20-point R@1 gap could be mostly input length. The stress-test note lands. Table 3, which holds input length roughly fixed, shows only +1.5 T2V R@1 from adding the long granularities. So the causal role of GEX/IAM in long-form gains is plausible, not demonstrated.\n\n2) The #iter choices are made after seeing test results. Table 4b is the clearest sign: the paper says LSMDC is best with 1-1, yet 3-3 gives higher T2V R@1 (14.1 vs 12.9). That's a small internal contradiction and a real post-hoc selection issue. A fixed policy set on validation splits, or a sensitivity analysis, would fix it.\n\n3) No error bars, no released code or data. For a method that depends on LLM-generated training data and a few hyperparameters, that limits independent verification.\n\nNone of this kills the paper. GEX is worth knowing about, and IAM is a reasonable module. But the abstract's 'excels in long-form' claim is stronger than the controlled evidence. I'd send it to review, with the expectation that the authors add a same-cost/input-length baseline, move #iter selection to validation, and release artifacts. Right now it's a conditional accept.","headline":"GEX is a genuinely useful data-expansion idea and IAM is a reasonable module, but the long-form gains are confounded by input length and #iter is chosen after seeing test numbers.","tokens_in":24648,"tokens_out":4593,"would_cite":true,"duration_ms":40847,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GEXIA claims multi-grained video-language alignment can be learned from a single-grained dataset by concatenation, summarization, and iterative approximation, yielding long-form video understanding from short-clip pretraining.","keywords":["video-language pretraining","multi-grained video-text alignment","granularity expansion","iterative approximation","long-form video understanding","contrastive learning","video-text retrieval","text summarization"],"falsifier":"Train the same GEXIA model twice on the same amount of GEX-expanded data, once with concatenated clips kept in their original time order and once with the order shuffled within each source video, then compare zero-shot retrieval on ActivityNet Captions. If shuffled-order pretraining performs as well as time-ordered pretraining, the temporal-coherence assumption that carries the long-form claim is not doing the work.","tokens_in":23575,"feed_emoji":"🎬","tokens_out":7903,"duration_ms":61705,"temperature":0.7,"pith_summary":"Video-text data in the wild is multi-grained: videos run from seconds to hours, and the text paired with a video can be a one-line caption or a full paragraph. Existing pretraining datasets are effectively single-grained, and most models align video and text at one fixed level of detail. This paper proposes GEXIA, a two-part recipe: Granularity Expansion (GEX) turns a single-grained dataset into a multi-grained one by concatenating same-source short clips and texts into long clips and by summarizing the long texts into short ones, and an Iterative Approximation Module (IAM) compresses variable-length video and text features into fixed-size embeddings in a shared low-dimensional space. Trained only on short video clips, the resulting model reports state-of-the-art or comparable results on seven benchmarks and does especially well on long-form video tasks, which the paper attributes to the multi-grained pretraining data and granularity-aware iteration.","feed_headline":"Stitching short clips teaches long-video understanding","feed_subtitle":"A short-clip dataset expanded into three granularities matches or beats specialized long-video models on seven benchmarks.","key_machinery":"The load-bearing machinery is the pair of GEX and IAM. GEX (Granularity Expansion) takes an existing single-grained dataset and produces new granularities without new data collection: Video Integration and Text Integration concatenate videos and texts that share a source ID, in time order, forming long-video-long-text pairs, and Text Compression uses a large language model to summarize the long text into short text, forming long-video-short-text pairs. IAM (Iterative Approximation Module) is the model component that absorbs variable-length dense features: a learnable base embedding serves as the query in a cross-attention block whose key and value come from the dense feature, followed by a self-attention block, and the whole block is repeated a number of times (#iter) that is set according to the input granularity, with longer videos and texts receiving more iterations. The output is a fixed-size low-dimensional embedding per modality, so conventional contrastive learning can align any number of granularities in one space.","core_discovery":"The paper's central claim is that multi-grained video-language alignment can be learned from a single-grained dataset and that this is what unlocks long-form video understanding. It argues that GEX-generated long-video-long-text and long-video-short-text pairs make the model aware of granularity, while the IAM allows one model to embed any granularity into the same low-dimensional space by running more iteration steps on longer inputs. The authors report state-of-the-art or comparable performance on cross-modal retrieval (ActivityNet, MSR-VTT, LSMDC), video classification (LVU, COIN), egocentric action classification (Charades-Ego), and video question answering (How2QA), with the largest gains on long-video benchmarks, and they show that a model pretrained on only short clips can outperform methods pretrained on long-video datasets.","pith_inferences":["Editorial inference: The ablations show random concatenation already improves over single-grained pretraining, which suggests part of the gain comes from simply feeding the model longer context, and only a smaller part from temporal coherence within the original video; a follow-up study that separates these two factors would clarify the mechanism.","Editorial inference: A natural stress test is to shuffle the order of concatenated clips during pretraining; if order does not matter, the model is learning a bag-of-clips representation rather than temporal sequence, which would narrow the claim about long-form understanding.","Editorial inference: The same expansion-and-approximate recipe could be applied to other paired modalities with variable granularity, such as audio-transcript pairs or image sets with captions, since neither GEX nor IAM depends on video-specific structure.","Editorial inference: The paper evaluates long-form gains on benchmark datasets whose videos were originally collected as continuous recordings; the synthetic concatenation assumption would be tested more sharply on genuinely continuous, causally structured long videos, where a stitched clip sequence lacks narrative dependencies."],"forward_implications":["If the central claim holds, long-form video understanding can be improved without constructing a large naturally long-video pretraining dataset; stitching short clips and summarizing texts suffices.","A single pretrained model can be adapted to a new granularity at inference time by changing #iter, as the paper demonstrates on image-text data with #iter = 0 for the video branch.","The GEX operations are recursive and have no limit on the number of granularities, so the pipeline can in principle produce arbitrarily layered video-text alignments from one dataset.","Because the video and text encoders stay separate until the final contrastive loss, the approach can reuse existing image and text encoders, keeping retrieval complexity at O(Nv Nt).","The gains on ActivityNet, LVU, and COIN suggest that granularity-aware data expansion can compensate for a much smaller pretraining corpus (10M pairs versus 136M in some baselines)."],"supporting_citations":[{"why":"supplies the base single-grained short-clip dataset that GEX expands into multi-grained pairs.","marker":"[59]"},{"why":"provides the pretrained image and text encoders and the contrastive objective the alignment builds on.","marker":"[46]"},{"why":"supplies the iterative attention idea that IAM adapts for granularity-dependent approximation.","marker":"[25]"},{"why":"the large language model used for Text Compression to create long-video-short-text pairs.","marker":"[11]"},{"why":"the main retrieval baseline with similar backbone and inference budget that GEXIA is compared against.","marker":"[39]"},{"why":"a hierarchical multi-grained baseline that requires a multi-grained pretraining dataset, which GEX aims to make unnecessary.","marker":"[3]"},{"why":"a long-form video pretraining baseline that GEXIA matches or exceeds while using fewer pretraining pairs.","marker":"[51]"}],"fun_headline_variants":["From short clips to long videos: GEXIA scales multi-grained alignment","Short clips, long videos: one model learns all granularities","GEXIA expands granularity to teach long-video understanding","Teach long-video understanding with short clips alone","Multi-grained video-language: GEXIA's iterative approximation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's long-form gains rest on the assumption that concatenating short clips from the same source in time order produces synthetic long videos similar enough to naturally occurring long videos that pretraining on them transfers to real long-video tasks.","fun_headline_variants_meta":{"raw":{"variants":["From short clips to long videos: GEXIA scales multi-grained alignment","Short clips, long videos: one model learns all granularities","GEXIA expands granularity to teach long-video understanding","Teach long-video understanding with short clips alone","Multi-grained video-language: GEXIA's iterative approximation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000513,"raw_usage":{"total_tokens":2463,"prompt_tokens":883,"completion_tokens":1580,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":499,"completion_tokens_details":{"reasoning_tokens":1496}},"tokens_in":499,"tokens_out":1580,"duration_ms":9844,"temperature":1.0,"reasoning_tokens":1496,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:34:34.748605+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same GEXIA model twice on the same amount of GEX-expanded data, once with concatenated clips kept in their original time order and once with the order shuffled within each source video, then compare zero-shot retrieval on ActivityNet Captions. If shuffled-order pretraining performs as well as time-ordered pretraining, the temporal-coherence assumption that carries the long-form claim is not doing the work.","supporting_citations":[{"cited_title":"Learning transferable visual models from natural language supervi- sion","cited_arxiv_id":null,"evidence_quote":"provides the pretrained image and text encoders and the contrastive objective the alignment builds on."},{"cited_title":"Gonzalez, Ion Stoica, and Eric P","cited_arxiv_id":null,"evidence_quote":"the large language model used for Text Compression to create long-video-short-text pairs."},{"cited_title":"Clip4clip: An empirical study of clip for end to end video clip retrieval and captioning","cited_arxiv_id":null,"evidence_quote":"the main retrieval baseline with similar backbone and inference budget that GEXIA is compared against."},{"cited_title":"Hiervl: Learning hierarchical video- language embeddings","cited_arxiv_id":null,"evidence_quote":"a hierarchical multi-grained baseline that requires a multi-grained pretraining dataset, which GEX aims to make unnecessary."},{"cited_title":"Long-form video-language pre- training with multimodal temporal contrastive learning","cited_arxiv_id":null,"evidence_quote":"a long-form video pretraining baseline that GEXIA matches or exceeds while using fewer pretraining pairs."}],"review_version":1}