Pith. sign in

REVIEW 5 major objections 4 minor 8 cited by

CoS: Chain-of-Shot Prompting for Long Video Understanding

T0 review · 5 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Chain-of-Shot prompting improves long-video understanding by using yes/no mosaic reviews to select task-relevant shots and re-weighting the model's predictions against irrelevant ones, with no training required.

desk verdict A solid, training-free plug-in with consistent small gains, but the pseudo-grounding labels are unvalidated and need a random-label control before the shot-selection claim holds. read the letter →

arxiv 2502.06428 v2 pith:RUOVSYBE submitted 2025-02-10 cs.CV

Jian Hu , Zixu Cheng , Chenyang Si , Wei Li , Shaogang Gong This is my paper · ORCID
classification cs.CV
keywords longvideounderstandingmulti-modallargelanguagemodelsshotselectionvisualpromptoptimizationtemporalgroundingbinarysummarychain-of-shotpromptingtest-time
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to solve the shot-selection problem in long-video question answering: when a video is too long to feed entirely to a multimodal model, which frames should be kept and which can be dropped? The proposed Chain-of-Shot (CoS) prompting treats shot choice as a test-time visual prompt optimization: a general-purpose vision-language model answers yes/no on mosaics of four frames, those answers act as pseudo temporal grounding that splits shots into task-relevant positives and irrelevant negatives, and the model's next-token logits are re-weighted so that positives are amplified and negatives suppressed. The authors claim this training-free plug-in improves accuracy across three baselines (LongVA, Video-XL, LLaVA-Video) and five datasets (VideoMME, MLVU, LongVideoBench, NEXT-QA, MVBench), with gains of roughly 0.7 to 5 accuracy points. If correct, long-video understanding could be improved without fine-tuning or token compression, purely by making better shot selections at inference time.

What carries the argument

The load-bearing mechanism is the co-reasoning equation (Eq. 8): $\operatorname{logit}(X) + \alpha\operatorname{logit}(S_p) - \alpha\operatorname{logit}(S_n)$, applied in softmax space, with $\alpha = 1 - |S_p|/|X|$. The positive and negative sequences come from a binary video summary in which mosaics of four consecutive shots are classified yes/no by LLaVA1.5-13B using a query-derived prompt, producing pseudo temporal grounding without any temporal annotations. The mosaic grouping is what makes the binary coding affordable: it cuts the number of yes/no calls by a factor of four while still letting the model see temporally adjacent shots together.

What would settle it

Run CoS on a fixed subset of VideoMME with three label sources for $S_p$/$S_n$: LLaVA's yes/no outputs, ground-truth temporal annotations, and random coin flips. If random labels match LLaVA's labels in final accuracy, the pseudo grounding carries no signal and the gains must come from somewhere else; if ground-truth labels barely beat LLaVA's labels, the grounding step is already near the ceiling.

Watch

Extended reading notes

Core claim

The central claim is that long-video understanding can be improved by optimizing the visual input itself at test time, rather than only the textual prompt. CoS first builds a binary video summary: every four consecutive sampled shots are tiled into one mosaic image, and LLaVA1.5-13B is asked a yes/no question derived from the video query; 'yes' groups mark task-relevant shots and 'no' groups mark irrelevant ones. From this binary code it constructs a positive sub-shot sequence $S_p$ (task-relevant shots) and a negative sub-shot sequence $S_n$ (irrelevant shots), each matched in length to the original sampled sequence $X$. The final prediction combines the three inputs through $\operatorname{logit}(X) + \alpha\operatorname{logit}(S_p) - \alpha\operatorname{logit}(S_n)$, with $\alpha = 1 - |S_p|/|X|$, so the sparser the relevant content, the more weight the contrast receives. The paper argues this co-reasoning lets the model focus on sparsely distributed task-relevant shots while the original video provides a safety net for anything the pseudo grounding mislabels. On VideoMME, MLVU, LongVideoBench, NEXT-QA, and MVBench, CoS reports consistent accuracy gains over all three baselines.

Load-bearing premise

The method assumes the yes/no answers that LLaVA1.5-13B gives on mosaics of four shots reliably identify which shots contain task-relevant content, even though these pseudo labels are never checked against ground-truth temporal annotations.

Editorial extensions

If this is right

  • CoS can be added to any existing video MLLM at inference time with no training and no new parameters, and the reported gains span three different architectures and video lengths from short clips to hour-scale videos.
  • Performance stays stable as the shot sampling rate rises from 96 to 192 frames, suggesting the reweighting absorbs the extra noise that would otherwise dilute attention.
  • The dynamic weight $\alpha$ makes the method self-calibrating: when few shots are flagged relevant, $S_p$ and $S_n$ have larger influence; when most shots are relevant, the original video dominates and the selection module fades out.
  • The gains are smaller on Video-XL, which already has a built-in long-context attention mechanism, implying CoS is most useful for models without such internal support.
  • The same plug-in also improves accuracy on short-video benchmarks NEXT-QA and MVBench, so the visual-prompt effect is not limited to long-video settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension would swap the LLaVA1.5-13B grounding model for a stronger or weaker vision-language model; if the gains scale with the spatial reasoning quality of the grounding model, it would confirm that pseudo grounding quality, not the reweighting formula, sets the ceiling.
  • Because the binary labels are never validated against human temporal annotations, an oracle experiment using ground-truth relevance labels would separate the method's ceiling from the grounding error; the paper's ablation keeps the original video feed as a patch for mistakes in those labels, which hints the grounding step may be the main source of error.
  • The same positive-minus-negative logit contrast could be applied to other query-conditioned inputs, such as audio tracks or text segments, wherever relevance can be decided by a cheap binary classifier.
  • If CoS is reliable, it could be stacked with token-compression methods: compress or drop tokens only for shots marked negative, reducing context use further than either approach alone.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes Chain-of-Shot (CoS), a training-free test-time plug-in for long-video understanding. CoS uses a separate MLLM (LLaVA1.5-13B) to produce a binary summary over mosaics of k=4 consecutive shots, pseudo-grounding task-relevant shots; it then constructs positive and negative shot sets and combines their logits with the original video's logits via Eq. 8, with a dynamic weight alpha = 1 - |Sp|/|X| (Eq. 9). The method is evaluated on three baselines (LongVA, VideoXL, LLaVA-Video) across five benchmarks (VideoMME, MLVU, LongVideoBench, NEXT-QA, MVBench), reporting consistent accuracy improvements over the corresponding baselines.

Significance. If the reported gains are reproducible, CoS is a simple, training-free, and potentially widely applicable method: it requires no parameter updates, works with existing long-video MLLMs, and the authors provide code. The framing of shot selection as test-time visual-prompt optimization is a useful perspective, and the paper's scope across multiple datasets and baselines is a strength. However, the current evidence does not yet fully support the claim that task-relevant shot selection is the driver of the gains: the pseudo-grounding step is never validated against any ground truth, no random-label control is given, hyperparameters are selected on the same benchmark that is also a headline result, and no statistical significance or error bars are reported. These gaps are load-bearing for the central narrative, so the manuscript needs substantial additional experimental support.

major comments (5)
  1. [Section 3.2, Eq. 4, Table 4, Eq. 9] The pseudo temporal grounding is never validated, and no random-label control is reported. The paper never measures precision, recall, or IoU of the binary labels ('yes'/'no') against any ground-truth task relevance. Table 4, row 2 shows that Sp and Sn alone collapse below the baseline (VideoXL 48.4 vs 54.7; LLaVA-Video 54.3 vs 63.3), confirming that the contrastive terms only act as a logit-level correction on top of X. To support the 'shot selection' narrative, a control experiment with random or permuted binary labels (matching the same |Sp|/|X| ratio) is necessary: if random labels yield similar gains, the improvement cannot be attributed to task-relevant selection. Moreover, alpha = 1 - |Sp|/|X| is largest when the positive set is small, but a noisy binary summary can produce a small |Sp| precisely because of false negatives; in that case the method amplifies the least reliable labels. The paper should report grounding accuracy and/or a sensitivity analysis under label corruption.
  2. [Section 3.2, Table 5] The hyperparameters k=4 (aggregation shot count) and the binary-summary MLLM (LLaVA1.5-13B) are selected via ablations on VideoMME (Table 5(c) and Table 5(a)), and VideoMME is then reported as a main result in Table 1. This is a form of test-set selection that can inflate the VideoMME numbers. The authors should either choose hyperparameters on a validation split or on a separate development dataset and then freeze them across all evaluations, or demonstrate that the reported results are stable under a range of k and binary-summary MLLM choices.
  3. [Section 4.1, Tables 1-3] No error bars, number of runs, or significance tests are reported. Several improvements are small (e.g., VideoMME w/o sub. avg improvements of 0.8-1.1 points for VideoXL and LongVA), and the paper uses the word 'significant' without any statistical support. The authors should report results over multiple seeds (or bootstrap over the benchmark instances) and perform paired significance tests across the evaluated subsets, since the improvements on some cells may be within run-to-run variance.
  4. [Section 3.2, Eq. 2] The construction of the 'keyword list' used in the binary prompt Ps is never described. The text says the prompt is 'Is anything in the keyword list present in the image? Just answer yes or no.' plus the video-specific question Qi, but the paper does not explain how the keyword list is generated from the question. This is central to the reproducibility of the binary video summary, because the entire pseudo-grounding step depends on this prompt.
  5. [Section 4.1, Table 5(c)] The claim that 'time complexity and space complexity remain the same as the baseline, both being O(n)' is misleading. Equation 8 requires running the baseline MLLM three times (on X, Sp, and Sn), and the binary summary adds a separate forward pass of LLaVA over the mosaic images. The 'time' column in Table 5(c) covers only the aggregation phase, not the full end-to-end CoS pipeline. The paper should report actual end-to-end inference time (or FLOPs) relative to the baseline, since the practical value of a plug-in depends on its real computational overhead.
minor comments (4)
  1. [Section 3.3.1, Eqs. 6-7] The variable k is reused for the aggregation shot count in Eq. 3 and as a candidate index in Eqs. 6-7, which is confusing; a different index such as l or j' should be used in the construction equations.
  2. [Table 2] The entry 'Shargpt4Video' is a typo; it should be 'Sharegpt4Video'.
  3. [Section 3.2] The phrase 'inspired by the idea of using image gird' contains a typo: 'gird' should be 'grid'.
  4. [Section 4.2, Shot Selection Model Analysis] In the analysis of Table 5(a), the model is referred to as 'MinichatGPT' in the text but 'miniChatGPT' in the table; the naming should be consistent.

Circularity Check

1 steps flagged · score 4.0 of 10

VideoMME is used both to tune CoS hyperparameters and as a headline reported result, making that benchmark partially self-referential; the other four datasets keep the central claim substantially independent.

  1. fitted input called prediction [Section 3.2 (k=4 selection after Eq. 4), Section 4.1 Implementation Details, Table 5, and Section 4.2 VideoMME results in Table 1]
    "Here, we set k as 4, and LLaV A (Liu et al., 2024a) as the MLLM, More analysis on the hyper-parameter selection is in Tab.5. ... Parameter ablation study on VideoMME with LongV A as the baseline. ... As shown in Tab.1, we integrated CoS into three baselines ... Results show that CoS achieves significant improvements across all baselines and temporal scales (short, medium, and long videos)."

    Table 5 is explicitly an ablation on VideoMME and is used to choose k, the binary-summary MLLM, and the shot-sampling rate. Those same choices are then used to produce the VideoMME rows in Table 1, which is presented as principal evidence of CoS's gains across temporal scales. The VideoMME accuracy is therefore the quantity used for model selection rather than an independent prediction of CoS's benefit on that benchmark. The other four datasets (MLVU, LongVideoBench, NEXT-QA, MVBench) were not used in this selection and retain independent supporting value, so the circularity is partial and confined largely to VideoMME.

full rationale

The CoS derivation itself is not circular by construction: Eq. 8 combines logits on the original sample X with logits on positive and negative sub-shots Sp and Sn, and Sp/Sn are built from the binary summary rather than being definitionally equal to the output quantity. The central empirical claim rests primarily on external benchmark accuracies, not on a self-citation chain. The only concrete self-referential step is the hyperparameter tuning: k=4, LLaVA1.5-13B as the binary-summary MLLM, and the sampling counts are selected via the Table 5 ablation on VideoMME, and the same VideoMME numbers are then presented in Table 1 as evidence. This matches the fitted-input-called-prediction pattern for that benchmark. There is one self-citation (Kangaroo, whose author list includes Jian Hu), but it is only used as a comparison row in Table 1 and is not load-bearing. A random-label control or a separate validation split for VideoMME would strengthen the VideoMME claim; that is an experimental-validity concern rather than circularity of the derivation, so it does not further raise the score.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The method relies on several unstated or loosely stated assumptions: the accuracy of pseudo temporal grounding, the validity of logit-level contrast, and the reliability of the chosen benchmarks. These are domain assumptions rather than mathematical axioms. The free parameters are the mosaic size k, the choice of binary summary MLLM, and the heuristic weighting alpha, all chosen by hand or by ablation on the main benchmark.

free parameters (3)
  • Aggregation shot count k = 4
    Chosen via ablation on VideoMME (Tab. 5c) to balance grounding granularity and inference speed; not derived from first principles.
  • Binary summary MLLM = LLaVA1.5-13B
    Selected via ablation on VideoMME (Tab. 5a) as the best performer among tested MLLMs; this choice is tuned on the same benchmark used for main results.
  • Dynamic weighting alpha = 1 - |Sp|/|X|
    Heuristic weighting of positive/negative logits introduced in Eq. 9; not learned or justified quantitatively, and its form is chosen by hand.
assumptions (3)
  • domain assumption Binary yes/no judgments by LLaVA1.5-13B on 4-shot mosaics accurately separate task-relevant from task-irrelevant shots.
    Invoked in Section 3.2 (Eq. 4) to build pseudo temporal grounding; never validated against annotated temporal labels.
  • domain assumption Combining logits of the original, positive, and negative shot sets as in Eq. 8 approximates the benefit of conditioning the model on filtered visual evidence.
    The product/softmax form in Eq. 8 is an unproved modeling choice; the paper provides no derivation or empirical validation of the combination rule.
  • domain assumption Benchmark accuracy on VideoMME, MLVU, LongVideoBench, NEXT-QA, and MVBench is a valid measure of long-video understanding.
    The paper treats these benchmarks as ground truth for effectiveness without discussing benchmark contamination or metric reliability.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CoS: Chain-of-Shot Prompting for Long Video Understanding." pith.science (2026). https://pith.science/paper/RUOVSYBE

@misc{pith2026250206428,
  author       = {Pith},
  title        = {Pith review of: CoS: Chain-of-Shot Prompting for Long Video Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RUOVSYBE}},
  note         = {Machine review of arXiv:2502.06428}
}
read the original abstract

Multi-modal Large Language Models (MLLMs) struggle with long videos due to the need for excessive visual tokens. These tokens exceed massively the context length of MLLMs, resulting in filled by redundant task-irrelevant shots. How to select shots is an unsolved critical problem: sparse sampling risks missing key details, while exhaustive sampling overwhelms the model with irrelevant content, leading to video misunderstanding. To solve this problem, we propose Chain-of-Shot prompting (CoS). The key idea is to frame shot selection as test-time visual prompt optimisation, choosing shots adaptive to video understanding semantic task by optimising shots-task alignment. CoS has two key parts: (1) a binary video summary mechanism that performs pseudo temporal grounding, discovering a binary coding to identify task-relevant shots, and (2) a video co-reasoning module that deploys the binary coding to pair (learning to align) task-relevant positive shots with irrelevant negative shots. It embeds the optimised shot selections into the original video, facilitating a focus on relevant context to optimize long video understanding. Experiments across three baselines and five datasets demonstrate the effectiveness and adaptability of CoS. Code given in https://lwpyh.github.io/CoS.

Figures

Figures reproduced from arXiv: 2502.06428 by the authors.

Figure 1
Figure 1. The effects of changing shot-sampling rates on video understanding task performance on videos of different lengths in the VideoMME (Fu et al., 2024a) dataset. Two models are evaluated including LongVA (Zhang et al., 2024a) and Video￾XL (Shu et al., 2024). As the number of sampled shots increased, performance did not consistently improve across various video lengths. That is because while sparse sampling may miss cru… view at source ↗
Figure 2
Figure 2. The critical problem of how to select shots in video understanding. In a video that depicts how a boy gradually gains a dragon’s trust, different sampling methods create two distinct narratives: split video A shows the boy being attacked by the dragon, while split video B shows him happily sharing food with the dragon. This shows that minor differences in video sampling leads to significant variations in semantic un… view at source ↗
Figure 3
Figure 3. The overall framework of CoS. It first utilises LLaVA to perform a mosaicing binary coding to bootstrap video summarisation for temporal grounding on a long video. Specifically, every four shots are aggregated into a mosaicing composition image. LLaVA determines whether task-related elements exist within each composition image by encoding a binary value of 1 or 0 (‘yes’ or ‘no’), thereby identifying sparsely distrib… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: An qualitative evaluation example from MLVU (Zhou et al., 2024) dataset. visual reasoning and summaries to achieve pseudo tempo￾ral grounding, where Qwen2’s superior temporal reasoning capabilities have limited scope for impact. Impact of Shot-sampling Rate. As depicte…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Evidence-Driven Dynamic Visual Selector for Efficient Long Video Understanding

    cs.CV 2026-08 conditional novelty 6.0 of 10

    EviSelect uses the target multimodal model's internal attention as a prior to dynamically select frames, sampling rates, and resolutions, achieving about 50% token reduction and a 3.9x speedup with better benchmark accuracy.

  2. Efficient Frame Selection for Long Videos at Test Time with Attention-Based MLLM Selectors

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Attention maps from a small MLLM can serve as a training-free, query-conditioned frame selector, improving long-video QA accuracy under fixed frame budgets.

  3. AdsQA: Towards Advertisement Video Understanding

    cs.CV 2025-09 conditional novelty 6.0 of 10

    AdsQA adds an ad-video question-answering benchmark and ReAd-R, a GRPO-trained model that beats 7B baselines but not larger closed models.

  4. ViaRL: Adaptive Temporal Grounding via Visual Iterated Amplification Reinforcement Learning

    cs.CV 2025-05 conditional novelty 6.0 of 10

    ViaRL uses rule-based reinforcement learning to train a frame selector for video QA, improving Qwen2.5-VL on VideoMME, LVBench, and MLVU by several points.

  5. Uncertainty-quantified Rollout Policy Adaptation for Unlabelled Cross-domain Temporal Grounding

    cs.CV 2025-08 conditional novelty 5.0 of 10

    Rollout-averaged pseudo labels with variance-based confidence weighting let a GRPO-trained temporal grounding model adapt to an unlabelled target domain from only 100-200 videos.

  6. ViTCoT: Video-Text Interleaved Chain-of-Thought for Boosting Video Understanding in Large Language Models

    cs.CV 2025-07 conditional novelty 5.0 of 10

    Interleaving key video frames into step-by-step reasoning improves video question answering by 1.7 to 5.5 points over text-only chain-of-thought on a new self-built benchmark.

  7. CyberV: Cybernetics for Test-time Scaling in Video Understanding

    cs.CV 2025-06 conditional novelty 5.0 of 10

    A training-free test-time feedback loop, using attention drift to select key frames, improves video MLLM accuracy, with the largest gains on knowledge-heavy VideoMMMU.

  8. Empowering Multimodal LLMs with External Tools: A Comprehensive Survey

    cs.CV 2025-08 unverdicted novelty 2.0 of 10

    A survey paper maps how external tools are used to augment multimodal large language models across data, tasks, evaluation, and future directions.

Reference graph

Works this paper leans on

20 extracted references · 20 linked inside Pith · cited by 8 Pith papers

  1. [2]

    Video-of-thought: Step-by-step video reasoning from perception to cognition

    Fei, H., Wu, S., Ji, W., Zhang, H., Zhang, M., Lee, M.- L., and Hsu, W. Video-of-thought: Step-by-step video reasoning from perception to cognition. In Forty-first International Conference on Machine Learning, 2024a. Fei, J., Li, D., Deng, Z., Wang, Z., Liu, G., and Wang, H. Video-ccam: Enhancing video-language understand- ing with causal cross-attention ...

    arXiv

  2. [4]

    Himakunthala, V ., Ouyang, A., Rose, D., He, R., Mei, A., Lu, Y ., Sonar, C., Saxon, M., and Wang, W. Y . Let’s think frame by frame with vip: A video infilling and prediction dataset for evaluating video chain-of-thought. arXiv preprint arXiv:2305.13903,

    arXiv

  3. [6]

    An image grid can be worth a video: Zero-shot video question answering using a vlm

    Kim, W., Choi, C., Lee, W., and Rhee, W. An image grid can be worth a video: Zero-shot video question answering using a vlm. arXiv preprint arXiv:2403.18406,

    arXiv

  4. [7]

    Video-llava: Learning united visual represen- tation by alignment before projection

    Lin, B., Ye, Y ., Zhu, B., Cui, J., Ning, M., Jin, P., and Yuan, L. Video-llava: Learning united visual represen- tation by alignment before projection. arXiv preprint arXiv:2311.10122,

    arXiv

  5. [8]

    Liu, H., Li, C., Wu, Q., and Lee, Y . J. Visual instruction tun- ing. Advances in neural information processing systems, 36, 2024a. Liu, J., Wang, Y ., Ma, H., Wu, X., Ma, X., Wei, X., Jiao, J., Wu, E., and Hu, J. Kangaroo: A powerful video- language model supporting long-context video input. arXiv preprint arXiv:2408.15542, 2024b. Liu, R., Li, C., Tang, ...

    arXiv

  6. [9]

    Maaz, M., Rasheed, H., Khan, S., and Khan, F. S. Video-chatgpt: Towards detailed video understanding via large vision and language models. arXiv preprint arXiv:2306.05424,

    arXiv

  7. [10]

    Unlocking video- llm via agent-of-thoughts distillation

    Shi, Y ., Di, S., Chen, Q., and Xie, W. Unlocking video- llm via agent-of-thoughts distillation. arXiv preprint arXiv:2412.01694,

    arXiv

  8. [11]

    Video-xl: Extra-long vision language model for hour-scale video understanding

    Shu, Y ., Zhang, P., Liu, Z., Qin, M., Zhou, J., Huang, T., and Zhao, B. Video-xl: Extra-long vision language model for hour-scale video understanding. arXiv preprint arXiv:2409.14485,

    arXiv

Show all 20 references
  1. [12]

    I., Burnell, R., Bai, L., Gulati, A., Tanzer, G., Vincent, D., Pan, Z., Wang, S., et al

    Team, G., Georgiev, P., Lei, V . I., Burnell, R., Bai, L., Gulati, A., Tanzer, G., Vincent, D., Pan, Z., Wang, S., et al. Gemini 1.5: Unlocking multimodal understand- ing across millions of tokens of context. arXiv preprint arXiv:2403.05530,

    arXiv

  2. [13]

    Chatvideo: A tracklet-centric multimodal and versatile video understanding system

    Wang, J., Chen, D., Luo, C., Dai, X., Yuan, L., Wu, Z., and Jiang, Y .-G. Chatvideo: A tracklet-centric multimodal and versatile video understanding system. arXiv preprint arXiv:2304.14407,

    arXiv

  3. [14]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

    Wang, P., Bai, S., Tan, S., Wang, S., Fan, Z., Bai, J., Chen, K., Liu, X., Wang, J., Ge, W., et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191, 2024a. Wang, X., Song, D., Chen, S., Zhang, C., and Wang,...

    arXiv

  4. [16]

    Longvila: Scaling long- context visual language models for long videos

    Xue, F., Chen, Y ., Li, D., Hu, Q., Zhu, L., Li, X., Fang, Y ., Tang, H., Yang, S., Liu, Z., et al. Longvila: Scaling long- context visual language models for long videos. arXiv preprint arXiv:2408.10188,

    arXiv

  5. [17]

    mplug-owl: Modulariza- tion empowers large language models with multimodality

    Ye, Q., Xu, H., Xu, G., Ye, J., Yan, M., Zhou, Y ., Wang, J., Hu, A., Shi, P., Shi, Y ., et al. mplug-owl: Modulariza- tion empowers large language models with multimodality. arXiv preprint arXiv:2304.14178,

    arXiv

  6. [18]

    Long con- text transfer from language to vision

    Zhang, P., Zhang, K., Li, B., Zeng, G., Yang, J., Zhang, Y ., Wang, Z., Tan, H., Li, C., and Liu, Z. Long con- text transfer from language to vision. arXiv preprint arXiv:2406.16852, 2024a. Zhang, Y ., Li, B., Liu, h., Lee, Y . j., Gui, L., Fu, D., Feng, J., Liu, Z., and Li, C...

    2024 arXiv

  7. [19]

    Mlvu: A comprehensive benchmark for multi-task long video understanding

    Zhou, J., Shu, Y ., Zhao, B., Wu, B., Xiao, S., Yang, X., Xiong, Y ., Zhang, B., Huang, T., and Liu, Z. Mlvu: A comprehensive benchmark for multi-task long video understanding. arXiv preprint arXiv:2406.04264,

    arXiv

  8. [20]

    Minigpt-4: Enhancing vision-language understanding with advanced large language models

    Zhu, D., Chen, J., Shen, X., Li, X., and Elhoseiny, M. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592,

    arXiv

  9. [2021]

    K., and Feng, J

    Xu, L., Zhao, Y ., Zhou, D., Lin, Z., Ng, S. K., and Feng, J. Pllava: Parameter-free llava extension from images to videos for video dense captioning. arXiv preprint arXiv:2404.16994,

    arXiv

  10. [2023]

    Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D

    Jiang, A. Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D. S., Casas, D. d. l., Hanna, E. B., Bressand, F., et al. Mixtral of experts. arXiv preprint arXiv:2401.04088,

    arXiv

  11. [2024]

    Sharegpt4video: Improving video understand- ing and generation with better captions

    Chen, L., Wei, X., Li, J., Dong, X., Zhang, P., Zang, Y ., Chen, Z., Duan, H., Lin, B., Tang, Z., et al. Sharegpt4video: Improving video understand- ing and generation with better captions. arXiv preprint arXiv:2406.04325, 2024a. Chen, Z., Wu, J., Wang, W., Su, W., Chen, G., X...

    arXiv

  12. [2025]

    Videoespresso: A large-scale chain-of-thought dataset for fine-grained video reasoning via core frame selection

    Han, S., Huang, W., Shi, H., Zhuo, L., Su, X., Zhang, S., Zhou, X., Qi, X., Liao, Y ., and Liu, S. Videoespresso: A large-scale chain-of-thought dataset for fine-grained video reasoning via core frame selection. arXiv preprint arXiv:2411.14794,

    arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.