{"id":"1ed26c8c-3dc2-4801-8c74-dcab3d91e30a","arxiv_id":"2412.12735","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Giraffe extends Qwen-VL and Qwen2-VL to 128K context using a new multimodal RoPE extension, a curated data recipe, and hybrid-resolution inference, reaching state-of-the-art results among open-source long-context VLMs.","lead":"The authors describe a set of design choices, including a new position embedding method called M-RoPE++ and a curated dataset, that extend Qwen-based visual language models to 128K context while keeping short-context accuracy. The resulting model, Giraffe, beats similarly-sized open-source long-context VLMs on video and multi-image benchmarks and is competitive with GPT-4V.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 6's M-RoPE++ ramp mixes a dimension index with wavelength ratios, yielding a discontinuous schedule that does not match the interpolate/extrapolate method described; with no released code, the central superiority claim is not tied to a verifiable method.","rationale":"The reader's weakest assumption concerned whether temporal frequency bands were fully covered during pretraining. That is an explanatory assumption rather than the most load-bearing condition: even if the coverage story were wrong, Table 2's held-data comparison could still show M-RoPE++ winning. The load-bearing condition is that the paper actually specifies the method that produced the reported numbers. Eq. 6, the only formal definition of M-RoPE++, is dimensionally inconsistent because it mixes dimension index d with wavelength ratios r. The resulting schedule is discontinuous and can behave like frequency magnification instead of interpolation, contradicting the stated rationale. Since code and models are promised but not yet verifiable in v1, the central contribution cannot be independently checked as written. This is a specific, correctable issue; it does not invalidate the empirical claims, but it makes the current text insufficient for verification. The reader's CONDITIONAL verdict remains appropriate; the condition should include fixing Eq. 6 and releasing code so that the evaluated formula can be confirmed.","tokens_in":16486,"tokens_out":14821,"duration_ms":135246,"concrete_test":"Reproduce the M-RoPE++ frequency multiplier θ'_d/θ_d from Eq. 6 using the paper's parameters (e.g., D = 128 rotary dims, split 2x/5x, L' = 128K, s = 8, base 10^6) and plot it over d. Verify continuity at d = 2x and d = 5x, and whether the 'height' band is actually interpolated (multiplier between 1/s and 1). Then implement the printed formula and the r_d-corrected variant in a minimal RoPE attention probe and check which one reproduces the Table 2 ranking on a small VideoMME/Visual Haystack slice. If only the corrected variant reproduces the results, Eq. 6 is a material error and the evaluated method is underspecified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"§3.2 defines M-RoPE++ with s = L'/L_V, λ_d = 2πb^{2d/|D|}, r = L'/λ, then gives the piecewise schedule in Eq. 6. In the middle term the ramp is written as (d − r_{5x})/(r_{2x} − r_{5x}): the numerator is a raw dimension index d, while the denominator uses wavelength-ratio values r at d = 2x and 5x. These are different quantities, so the expression is not a function of d in the intended YaRN-style way. With Qwen2-VL's rotary dimensions split 2:3:3, the printed multiplier at d = 2x is not 1 and at d = 5x is not 1/s; it is discontinuous at both boundaries and, depending on hyperparameters, can exceed 1 inside the 'interpolated' height band. The corrected ramp should use r_d − r_{5x} in the numerator, which makes the schedule continuous and monotone. Because no code or model is released in v1, a reader cannot tell whether Table 2 was produced by the printed formula or a corrected one. This matters because the central claim that M-RoPE++ beats PI and NTK at the same training length depends on exactly this schedule; if the printed schedule is not the evaluated schedule, the paper as written does not specify its own main contribution.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a systematic study of how to extend the context length of visual language models to 128K tokens, using the Qwen-VL and Qwen2-VL families as backbones. It introduces three design components: the ETVLM data recipe (mixing long text, short visual instruction, interleaved image, and video instruction data), the M-RoPE++ position-embedding scheme (extrapolating the temporal rotary dimensions and interpolating the spatial ones), and hybrid-resolution video encoding (combining high-resolution key frames with compressed frames). The resulting GIRAFFE models are evaluated on single-image, multi-image, and long-video benchmarks, and the paper claims state-of-the-art performance among similarly sized open-source long VLMs and competitiveness with GPT-4V.","tokens_in":16811,"tokens_out":10406,"duration_ms":88802,"significance":"If the claims hold, the paper would provide a practical and unusually thorough recipe for converting an existing VLM into a long-context model without multi-stage alignment, and the public release of ETVLM would be a useful community asset. The experimental scope is a clear strength: data-ratio ablations, length ablations, position-embedding comparisons, training-strategy comparisons, and resolution/frame-count trade-offs are all reported on a common evaluation suite. The central claims are, however, tied to a position-embedding formula that is not correctly specified as printed, and the headline state-of-the-art comparison does not control for the number of input frames; both issues need to be resolved before the contribution is verifiable and fairly benchmarked.","major_comments":[{"comment":"Equation (6) is not a valid piecewise schedule as printed. On the middle branch, the factor (d − r_{5x})/(r_{2x} − r_{5x}) subtracts the wavelength-ratio value r_{5x} from the raw block index d. These are different quantities, so the ramp does not in general equal 1 at d = 2x and does not equal 1/s at d = 5x; the schedule is discontinuous at the boundaries of the interval 2x < d ≤ 5x. If the intended expression is (r_d − r_{5x})/(r_{2x} − r_{5x}), where r_d = L′/λ_d, the equation should be corrected. Because the claimed advantage of M-RoPE++ over PI, NTK, and direct extrapolation in Table 2 depends on this exact schedule, and because no code is available in v1, the current manuscript does not fully specify its main technical contribution and cannot be independently reproduced.","section":"§3.2, Eq. (6)"},{"comment":"The headline comparison is confounded by frame count. In Table 6, GIRAFFE is evaluated with 768 or 1024 frames, while Qwen2-VL-7B is evaluated with 256 frames, LongVILA-8B with 256 frames, and LongVA-7B with 128 frames. Tables 4 and 5 show that increasing the frame count for GIRAFFE from 512 to 1024 improves the VideoMME-Long score by several points (for example, from 57.6 to 59.1 in Table 5 under the hybrid-resolution setting, and to 60.5 in the final model row). A matched-frame-count comparison at 256 frames is therefore needed to support the claim that GIRAFFE is state-of-the-art because of the extension method rather than because it is shown more frames. At minimum, the manuscript should report the same model at 256 frames and disclose the total visual token budget used per method.","section":"Table 6, §4.3"},{"comment":"The method section is ambiguous about what is actually trained. The heading and Finding 5 say \"hybrid-resolution training,\" but the evidence in Table 5 consists of inference-time configurations (L,m,s), and the final row of Table 6 is labeled \"w/ Hybrid-res train&inf\" without describing the training-time counterpart. The manuscript does not state how high- and low-resolution frames are sampled during optimization, whether the full model or only the LLM backbone is updated under this schedule, or whether the same (L,m,s)=(4,240,3) setting is used for both training and inference. Without this information, hybrid-resolution training is not a reproducible contribution.","section":"§3.5, Table 5"}],"minor_comments":[{"comment":"The parameter x in the \"16x-dimensional\" M-RoPE matrix and in the segment boundaries 2x, 5x, and 8x is never explicitly defined in the main text; please state that x is the number of rotary blocks allocated per segment.","section":"§3.2, Eq. (1) and Eq. (6)"},{"comment":"The 60% accuracy threshold used to define effective length is introduced without justification; since claims such as \"effective length is less than one-third of the training length\" depend on this threshold, please report how the conclusions change when the threshold is varied.","section":"§3.1, Figure 4"},{"comment":"The text says Su's blog recommends an optimal RoPE base of 4.9×10^6 and then describes this as closely approximating the selected base of 5×10^5, but these values differ by an order of magnitude; this appears to be a typo and should be corrected.","section":"Appendix B"},{"comment":"The claim that temporal frequencies are already fully covered during pre-training, while spatial frequencies are not, is asserted without direct evidence; an analysis of frequency coverage or an ablation that interpolates the temporal band instead of extrapolating it would make the design rationale testable.","section":"§3.2"},{"comment":"The table header mixes frame-count columns for VideoMME and the image-count column for Visual Haystacks, so the entry \"100\" could be read as either; please relabel the columns to make clear that the last column is the number of Visual Haystack images.","section":"Table 2"},{"comment":"The comparison between single-stage and progressive extending says both use the same number of training steps, but it does not state whether they also use the same total number of tokens or the same data mix; this should be clarified for the ablation to be interpretable.","section":"Appendix C"}],"recommendation":"major_revision","confidential_remarks":"The central issue is that Eq. (6) as printed cannot be the schedule that produced Table 2; if it is a typo, it is easy to fix, and the rest of the empirical study is useful. The matched-frame-count issue in Table 6 is also fixable by adding comparisons or by explicitly justifying unequal budgets. I would not reject on the current evidence, but the revision should prioritize the corrected formula, a clear description of hybrid-resolution training, and a fairer comparison protocol."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Zhi, here's my read on GIRAFFE. The useful core is the empirical study: what happens to Qwen-VL/Qwen2-VL when you extend context to 128K with different data mixes, position methods, and frame/resolution trade-offs. The ETVLM data recipe and the finding that short multimodal instruction data is needed to retain short-context performance are genuinely useful. The hybrid-resolution training idea (SlowFast-style, high-res anchor frames plus low-res fillers) is clean and the token-savings numbers look plausible. The paper deserves credit for asking the right three questions and running the ablations.\n\nThe soft spot is the M-RoPE++ formula in Eq. 6. The middle term writes (d − r_{5x})/(r_{2x} − r_{5x}), where d is a raw dimension index and r is a wavelength ratio. Those are different quantities, so the ramp does not go from 0 to 1 at the boundaries; the schedule is discontinuous and can even exceed 1 inside the height band. The intended YaRN-style form would be (r_d − r_{5x})/(r_{2x} − r_{5x}). Since there is no released code, I cannot tell whether Table 2 was produced by the printed formula or a corrected version. That matters because the main technical claim—M-RoPE++ beats PI and NTK at the same training length—rests on exactly this schedule. This is a fixable presentation error, but it is in the paper's central contribution.\n\nOther issues are more standard. All results are single runs, no error bars. The final benchmark comparison gives Giraffe 768–1024 frames while most open baselines get 256 or fewer; that is not apples-to-apples, though the authors do show a frame-count sweep in Table 4. Hyperparameters like RoPE base and data ratios were tuned on the same benchmarks used for final claims, so some optimism bias is baked in. These are addressable in revision.\n\nThe citation pattern looks fine and the related work is honest. The paper is not circular—the scores come from external benchmarks. It is a competent empirical design study, not a field redefinition.\n\nFor whom: anyone working on long-context VLMs or video understanding will get value from the data-recipe findings and the resolution/frame trade-off analysis. The M-RoPE++ claim needs a corrected formula and released code before I'd trust it.\n\nRecommendation: send it to peer review. A serious referee can sort out the formula and push for error bars and code release. The topic is important and the empirical work is substantial. I'd want the revision before accepting, but desk-rejecting this would be wrong.","headline":"A useful long-context VLM recipe with one unverifiable core formula: the M-RoPE++ schedule in Eq. 6 mixes a dimension index with wavelength ratios, so the method as printed is not well-defined.","tokens_in":17338,"tokens_out":2649,"would_cite":true,"duration_ms":21414,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Giraffe extends Qwen2-VL to a 128K-token context with a curated data recipe, a dimension-aware position embedding, and hybrid-resolution inference, reaching open-source state of the art on long-video benchmarks and competitive parity with…","keywords":["context length extension","visual language models","M-RoPE++","rotary position embedding","video understanding","long-context benchmarks","instruction tuning","hybrid-resolution training"],"falsifier":"Take the same Qwen2-VL extension recipe and swap M-RoPE++'s assignment, interpolating the temporal band while extrapolating height and width. If this swapped variant does not lose to M-RoPE++ on a long-video temporal-order benchmark, the paper's claim that temporal frequencies were already well covered is wrong.","tokens_in":16271,"feed_emoji":"🎬","tokens_out":5863,"duration_ms":45749,"temperature":0.7,"pith_summary":"The paper works out a full pipeline for taking an existing visual language model and stretching its context window from 16K to 128K tokens. It claims that three design decisions together do this: a curated mix of short and long multimodal instruction data (ETVLM), a new rotary position embedding variant (M-RoPE++) that extrapolates the temporal axis while interpolating height and width, and a hybrid-resolution video encoding during inference. The resulting model, Giraffe, is reported to be state-of-the-art among similarly sized open-source long VLMs and competitive with GPT-4V on long-video benchmarks while keeping short-image performance. The paper also reports that the effective context length of VLMs, like that of LLMs, falls short of training length, and that M-RoPE++ extends that effective length further than position interpolation or NTK scaling.","feed_headline":"Open VLM hits 128K context, matches GPT-4V on long video","feed_subtitle":"Balanced data, a split position embedding, and hybrid resolution lift VideoMME scores without hurting single-image accuracy.","key_machinery":"The central object is M-RoPE++, a piecewise re-scaling of the rotary frequencies in M-RoPE, the three-dimensional rotary position embedding used by Qwen2-VL. M-RoPE allocates 16x rotary blocks in a 2:3:3 ratio to temporal, height, and width indices; M-RoPE++ applies no scaling to the low-dimensional (temporal) blocks, a linear interpolation in the middle (height) block range, and full position interpolation to the high-dimensional (width) blocks. This asymmetry is what lets the model extrapolate to longer videos without blurring the order of nearby frames, and it is the mechanism that the paper credits for the effective-length gains.","core_discovery":"On its own terms, the paper's discovery is that the right way to extend the context of a VLM that uses M-RoPE is to treat the temporal, height, and width frequency bands differently: keep the temporal rotary base untouched (extrapolate), linearly interpolate the height band, and fully interpolate the width band. This piecewise scheme, M-RoPE++, gives longer effective length and better downstream video performance than direct extrapolation, PI, or NTK-aware scaling under the same training budget. Combined with a balanced data recipe and one-stage instruction tuning of the LLM backbone, it yields Giraffe, a 128K-context model that leads open-source long VLMs and approaches GPT-4V on VideoMME and LongVideoBench.","pith_inferences":["If the temporal-coverage premise is transferable, M-RoPE++ should extend other rotary-split multimodal backbones, not just Qwen2-VL; a quick test would run the same piecewise scaling on any model with a t:h:w rotary split.","The paper's frame-resolution trade-off suggests a token-budget view of long-video understanding: at a fixed token count, high-resolution anchors plus low-resolution context beats uniform resolution, which implies adaptive frame sampling could push effective length further.","The ETVLM balance principle (short data as a floor, video data as the main lever) is plausible for other long-context multimodal tasks and can be tested by sweeping the ratio on a held-out task set.","The GPT-4V comparisons are on specific benchmark segments and frame counts; a head-to-head with controlled frame sampling and temporal-order probes would separate positioning gains from resolution gains."],"forward_implications":["A VLM extension can be done in one stage: instruction-tune the LLM backbone on mixed data, with no separate text-context extension or visual alignment phase.","Data recipes for long-context VLMs should hold short instruction data to a substantial share; the paper's recommended 60% long-data ratio keeps long gains without sacrificing short benchmarks.","Hybrid-resolution inference (a high-resolution frame followed by compressed frames) delivers long-video gains at roughly half the visual tokens.","The effective-length fall-short phenomenon transfers from LLMs to VLMs: even after training to 128K, PI- and NTK-extended models stay effective to about 40K, while M-RoPE++ raises that ceiling.","With 1024 frames, Giraffe exceeds GPT-4V on some VideoMME and LongVideoBench segments."],"supporting_citations":[{"why":"Base model; supplies the M-RoPE embedding and the 16K pre-trained context Giraffe extends.","marker":"(Wang et al., 2024a)"},{"why":"Baseline method; position interpolation that M-RoPE++ is compared against.","marker":"(Chen et al., 2023b)"},{"why":"Baseline method; NTK-aware RoPE scaling that M-RoPE++ is compared against.","marker":"(LocalLLaMA, 2023)"},{"why":"Provides the wavelength and scaling ratio definitions M-RoPE++ builds on.","marker":"(Peng et al., 2023)"},{"why":"Benchmark; the single-needle test used to measure effective context length.","marker":"(Wu et al., 2024c)"},{"why":"Benchmark; the main long-video evaluation where Giraffe is competitive with GPT-4V.","marker":"(Fu et al., 2024a)"},{"why":"Benchmark; second long-video evaluation with temporal segments.","marker":"(Wu et al., 2024a)"}],"fun_headline_variants":["GIRAFFE extends VLM context to 128K with M-RoPE++","Open VLM reaches 128K context, rivals GPT-4V","M-RoPE++ and data recipe give VLMs 128K context","GIRAFFE: 128K context VLM, competitive with GPT-4V"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole M-RoPE++ advantage rests on the assumption that Qwen2-VL's pre-training already covered the temporal frequency bands of the rotary embedding, so those bands can be extrapolated; if those frequencies were undertrained, the method's gains would come from something other than the coverage story.","fun_headline_variants_meta":{"raw":{"variants":["GIRAFFE extends VLM context to 128K with M-RoPE++","Open VLM reaches 128K context, rivals GPT-4V","M-RoPE++ and data recipe give VLMs 128K context","GIRAFFE: 128K context VLM, competitive with GPT-4V"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000167,"raw_usage":{"total_tokens":1267,"prompt_tokens":965,"completion_tokens":302,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":581,"completion_tokens_details":{"reasoning_tokens":214}},"tokens_in":581,"tokens_out":302,"duration_ms":3307,"temperature":1.0,"reasoning_tokens":214,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:47:37.387101+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the same Qwen2-VL extension recipe and swap M-RoPE++'s assignment, interpolating the temporal band while extrapolating height and width. If this swapped variant does not lose to M-RoPE++ on a long-video temporal-order benchmark, the paper's claim that temporal frequencies were already well covered is wrong.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Baseline method; NTK-aware RoPE scaling that M-RoPE++ is compared against."}],"review_version":1}