Pith. sign in

REVIEW 2 major objections 6 minor 2 cited by

QuickVideo: Real-Time Long Video Understanding with System Algorithm Co-Design

T0 review · 2 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read QuickVideo claims that parallel keyframe-aligned video decoding, grouped KV-cache pruning, and CPU-GPU overlap cut end-to-end inference for a 30-minute video from 69.7 seconds to 20.0 seconds, making near-real-time long-video…

desk verdict Measured speedups look real, but the headline accuracy claim is tested at a different operating point than the latency demo. read the letter →

arxiv 2505.16175 v2 pith:LLQRKOVX submitted 2025-05-22 cs.CV cs.AI

Benjamin Schneider , Dongfu Jiang , Chao Du , Tianyu Pang , Wenhu Chen This is my paper
classification cs.CVcs.AI
keywords longvideounderstandingLLMinferenceparalleldecodingkeyframe-alignedintervalsKVcachepruninggroupedprefillCPU-GPUoverlaplatencyreduction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

QuickVideo is a systems-and-algorithm co-design aimed at removing the two delays that make long-video understanding with VideoLLMs impractical: sequential decoding of a compressed bitstream into frames, and prefilling millions of video tokens into the language model. The paper's central claim is that both stages can be attacked at once, by decoding intervals of the video in parallel on CPU cores, prefilling independent groups of frames on the GPU while pruning away low-information key-value entries, and overlapping the two stages so the GPU is almost never idle. The result reported is a more-than-threefold end-to-end speedup on a 30-minute video, from 69.7 seconds to 20.0 seconds, on a single A100 GPU. If accurate, this lets hour-long videos be processed at realistic 1-2 FPS sampling rates on commonly available hardware, instead of dropping to one frame per minute to fit in memory.

What carries the argument

The load-bearing mechanism is keyframe-aligned interval decoding with a shared-memory frame tensor: scan packet metadata to find keyframes, choose $c$ intervals whose boundaries are the closest keyframes to evenly spaced timestamps, seek once per interval, decode packets in parallel, and write selected frames into one preallocated tensor via a timestamp-to-index mapping. The companion mechanism is group-based prefill with KV-cache pruning, selecting indices by smallest L2 norm of the key vectors, plus overlapped scheduling that forms a producer-consumer pipeline between CPU decoding and GPU prefill. Together they convert two sequential, latency-heavy stages into a single pipelined stage whose cost is governed by the slower of decoding and prefill rather than their sum.

What would settle it

Run VideoMME, LongVideoBench, LVBench, and MLVU with QuickPrefill configured exactly as in Section 4.3 ($\rho=0.2$, 32-frame groups, 1 FPS) and compare the average accuracy against the no-pruning baseline. If the accuracy gap is materially larger than the 3% degradation seen at $\rho=0.5$, then the 20.0-second result measures speed but not the 'high-quality understanding' the paper claims at that operating point.

Watch

Extended reading notes

Core claim

On the paper's own terms, QuickVideo establishes that long-video inference does not have to wait for the whole video to be decoded, nor does the model have to attend to every video token during prefill. QuickCodec splits the video bitstream into keyframe-aligned intervals and decodes them concurrently across CPU cores, avoiding the repeated seek operations that sequential decoders perform for dense frame sampling. QuickPrefill feeds video tokens to the model in independent groups, reducing peak activation memory by the number of groups, and prunes each group's KV cache to a retention ratio $\rho$ using small key norms as the importance score; at $\rho=0.5$ the reported accuracy stays above 95% of the no-pruning baseline on VideoMME, LongVideoBench, LVBench, and MLVU. A producer-consumer scheduling scheme overlaps decoding with prefill, so total time approaches $\max(t_{\text{dec}}, t_{\text{prefill}})$ plus a small startup cost. The headline measurement is 20.0 seconds end-to-end for a 30-minute video at 1 FPS versus a 69.7-second baseline.

Load-bearing premise

The headline 20.0-second latency is measured at a retention ratio of 0.2 with 32-frame groups, but accuracy is only reported for that ratio with 16-frame groups (about 90.6% of baseline) and for 32-frame groups only at ratio 0.5; the claim that the speedup preserves answer quality therefore rests on an operating point that was never benchmarked end-to-end.

Editorial extensions

If this is right

  • Hour-long videos can be processed at native 1-2 FPS sampling rather than sparse one-frame-per-minute sampling, preserving fine-grained temporal and motion information.
  • Video decoding time stops being a fixed delay before model inference; on 16-32 core machines it drops 2-3x and scales to 64 cores.
  • KV cache and activation memory shrink by roughly the number of groups and by $1/\rho$, so a single 40GB GPU can prefill sequences whose full KV cache would need hundreds of gigabytes.
  • End-to-end latency approaches the maximum of decode time and prefill time, so further work on either stage independently will continue to reduce total latency.
  • The scheme is model-agnostic and can be dropped into existing VideoLLM pipelines to replace their decoding and prefill stages.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The headline latency is measured at $\rho=0.2$ with 32-frame groups, but accuracy is only reported for that ratio with 16-frame groups (about 90.6% of baseline) and for 32-frame groups only at $\rho=0.5$; the exact operating point of the headline result should be benchmarked before trusting the 'high-quality understanding' phrasing.
  • The decoder's advantage depends on keyframe density and sampling interval; for sampling gaps larger than roughly 4 seconds, seek-based decoders can win, so a hybrid that switches strategies when sampling is sparse would generalize QuickCodec's gains.
  • The key-norm pruning result suggests that importance heuristics from text KV compression transfer to densely sampled video tokens; a similar grouped-prefill-plus-pruning recipe could apply to other long multimodal contexts such as audio or multi-page documents.
  • The producer-consumer overlap generalizes: any system with an ordered CPU pre-processing stage and a GPU inference stage should see a max rather than sum latency, provided outputs can be consumed group-by-group.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes QuickVideo, a system-algorithm co-design for accelerating long-video understanding with VideoLLMs. It introduces three components: QuickCodec, a parallel CPU video decoder that splits a video into keyframe-aligned intervals decoded concurrently; QuickPrefill, a grouped prefill scheme with KV-cache pruning controlled by a retention ratio; and an overlapped execution scheme that pipelines CPU decoding and GPU prefill. On a 30-minute video with Qwen2.5-VL-7B, the authors report reducing end-to-end latency from 69.7 s to 20.0 s (roughly 3.5x). Decoder benchmarks show 2-3x speedups over Decord and TorchCodec on an hour-long H.264 video, and prefill accuracy is reported on four benchmarks at retention ratio 0.5, with ablations over group size and retention ratio.

Significance. If the measured results hold, the co-design is practically valuable: it makes 1-2 FPS sampling of long videos feasible on a single GPU and addresses a latency bottleneck (video decoding) that is often ignored. The paper provides direct timing measurements with 95% confidence intervals, an open-source code repository, and it does not fit parameters to benchmarks; the pruning heuristic is adopted from prior work and evaluated on held-out benchmarks. The main weakness is that the headline end-to-end configuration is not evaluated for accuracy, and the stated '<3% accuracy degradation' is not supported by the reported numbers at that configuration.

major comments (2)
  1. [Section 4.3 and Table 2] The headline latency result uses QuickPrefill with retention ratio rho=0.2 and group size 32, but no accuracy is reported for this configuration. The closest reported measurement (Table 2, group size 16, rho=0.2) shows an average accuracy of 54.34 versus a 59.97 no-pruning baseline, i.e., 90.6% of baseline, a 9.4% relative drop. The group-size ablation in Table 2 is only at rho=0.5 and does not establish that group size 32 recovers accuracy at rho=0.2. Because the claim of 'scalable, high-quality video understanding' depends on the actual operating point, please measure accuracy at rho=0.2 with group size 32 (and ideally at the 1,800-frame length used in Section 4.3 rather than the 256-frame setting) or revise the claim accordingly.
  2. [Section 1 and Table 2] The introduction states that QuickPrefill incurs 'less than 3% accuracy degradation in most benchmarks.' This is not supported by the reported data even at the settings where accuracy is measured. At rho=0.5 with group size 16 (Table 2), LVBench drops from 43.90 to 41.90 (a 4.6% relative degradation); at group size 32 with rho=0.5, LVBench drops to 41.51 (a 5.4% degradation). At rho=0.2 with group size 16, the average degradation is 9.4%. The claim should be qualified with the specific configuration and per-benchmark numbers, or removed.
minor comments (6)
  1. [Abstract vs. body] The abstract refers to the decoder as 'QuickDecoder' while the body and contributions use 'QuickCodec'; please unify the terminology.
  2. [Equation (3)] Equation (3) is mis-rendered with stray punctuation (', ;' inside the formula) and is hard to read; the max expression should be typeset clearly with both terms labeled.
  3. [Abstract and Section 4.3] The abstract says the system 'reduces ... time by a minute,' but the measured end-to-end reduction in Section 4.3 is 49.7 seconds; adjust the wording to match the reported number.
  4. [Figure 5] The legend and stacked bars in Figure 5 are difficult to parse: the category 'Prefill+Video Loading' overlaps conceptually with the separate 'Prefill' and 'Video loading' categories; clarify which components are included in each bar.
  5. [Table 1] The column header 'VideoMME w/o subtitle' is ambiguous; specify the benchmark split and any subtitle filtering in the table caption or text.
  6. [Section 4.1 and Appendix G] The 2-3x decoder speedup is measured on a single one-hour H.264 video; Appendix G acknowledges this limitation, but the main text presents the result without caveat. Please add a sentence in Section 4.1 noting that the timing study uses one video and one codec.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: each claimed contribution is a measured system effect or an externally sourced heuristic evaluated on held-out benchmarks, with no fitted parameter renamed as a prediction.

full rationale

QuickVideo makes no derived predictions and fits no parameters to the data it later reports. QuickDecoder's speedup is measured end-to-end against Decord and TorchCodec on a benchmark video; the parallel keyframe-interval decomposition is an implementation technique, and the correctness argument (interval spans and pts ordering) does not assume the speedup it claims. QuickPrefill's pruning importance function (Key Norms, small) is taken from prior work (Devoto et al., EMNLP 2024) and evaluated on VideoMME, LongVideoBench, LVBench, and MLVU against a no-pruning baseline; the reported accuracy trade-off is an empirical result, not an identity. Equation (2) selects by TopK of an importance score and is an algorithmic definition, while the latency model in Equation (3) is a pipeline timing formula, not a derivation that presupposes the outcome. The headline 20.0-second result in Section 4.3 is a measured latency on a 30-minute video at rho=0.2/group 32; accuracy at that exact operating point is not reported, and the nearest rho=0.2 point in Table 2 shows a roughly 9.4% average drop. That is a missing-evidence/correctness risk, not circularity: the speedup measurement stands independently, and the accuracy claim would need a new measurement rather than a mathematical reduction. The paper also declares in Appendix G that VideoLLMs are not yet trained for dense long-video sampling, a limitation that does not feed back into any premise of the efficiency experiments. I find no load-bearing self-citation, no fitted-input-called-prediction step, and no equation that reduces any claimed result to its own inputs.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper is an empirical systems contribution. It introduces no fitted physical parameters and no invented entities. The main hand-set choices are the retention ratio, group size, and interval count; the operating point used for the latency demo (rho=0.2) is not validated for accuracy. The system relies on standard codec properties and standard attention memory math.

free parameters (4)
  • retention ratio rho = 0.2 in end-to-end demo; 0.5 in accuracy tables
    Chosen manually for the latency demo; accuracy at rho=0.2 is not reported, so the operating point is not validated against the accuracy claim.
  • group size = 32 frames in end-to-end demo; 16 frames in accuracy tables
    Selected for the demo; ablation shows group size has minimal impact, but the specific demo value is a hand choice.
  • number of intervals s = 64 in Section 4.3
    Number of keyframe-aligned intervals for the overlapped pipeline; affects the overlap granularity and was not swept in the reported demo.
  • KV pruning heuristic = Key Norms (small)
    Selected as default based on superior performance in Table 1; this is model selection on the evaluation benchmarks, not an independent fit.
assumptions (4)
  • domain assumption Decoding from a keyframe produces all frames with presentation timestamp greater than or equal to the keyframe's pts, in display order.
    Algorithm 2's correctness proof relies on this codec property (Section 2.3, Algorithm 2 lines 7-19).
  • domain assumption The frame index is recoverable from pts via Equation (1), assuming ptsmin and ptsmax bound the stream and frames are uniformly spaced.
    Used to map decoded frames to the packed output tensor; edge cases with reordered packets or null pts are only partly discussed in Appendix A.
  • domain assumption Group-by-group prefill with stored KV caches produces the same attention output as full-sequence prefill for the target VideoLLM.
    This holds for causal attention with no cross-group dependencies beyond KVs; the paper relies on it without an explicit proof.
  • domain assumption The four benchmarks (VideoMME, LongVideoBench, LVBench, MLVU) are representative of long video understanding tasks.
    Accuracy retention claims are generalized from these benchmarks; the demo video itself is not a public benchmark.

how reviews work

0 comments
Cite this review

Pith. "Pith review of QuickVideo: Real-Time Long Video Understanding with System Algorithm Co-Design." pith.science (2026). https://pith.science/paper/LLQRKOVX

@misc{pith2026250516175,
  author       = {Pith},
  title        = {Pith review of: QuickVideo: Real-Time Long Video Understanding with System Algorithm Co-Design},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LLQRKOVX}},
  note         = {Machine review of arXiv:2505.16175}
}
read the original abstract

Long-video understanding has emerged as a crucial capability in real-world applications such as video surveillance, meeting summarization, educational lecture analysis, and sports broadcasting. However, it remains computationally prohibitive for VideoLLMs, primarily due to two bottlenecks: 1) sequential video decoding, the process of converting the raw bit stream to RGB frames can take up to a minute for hour-long video inputs, and 2) costly prefilling of up to several million tokens for LLM inference, resulting in high latency and memory use. To address these challenges, we propose QuickVideo, a system-algorithm co-design that substantially accelerates long-video understanding to support real-time downstream applications. It comprises three key innovations: QuickDecoder, a parallelized CPU-based video decoder that achieves 2-3 times speedup by splitting videos into keyframe-aligned intervals processed concurrently; QuickPrefill, a memory-efficient prefilling method using KV-cache pruning to support more frames with less GPU memory; and an overlapping scheme that overlaps CPU video decoding with GPU inference. Together, these components infernece time reduce by a minute on long video inputs, enabling scalable, high-quality video understanding even on limited hardware. Experiments show that QuickVideo generalizes across durations and sampling rates, making long video processing feasible in practice.

Figures

Figures reproduced from arXiv: 2505.16175 by the authors.

Figure 1
Figure 1. An overview of how QUICKVIDEO overlaps video decoding on CPU (QUICKCODEC) and prefill on GPU (QUICKPREFILL). QUICKCODEC concurrently processes intervals of the compressed video bit stream. QUICKPREFILL uses independent groups of frames, therefore it can begin prefill once the first frames are decoded, outputting carefully selected KV vectors. As QUICKCODEC loads frames synchronously, QUICKPREFILL can process the nex… view at source ↗
Figure 2
Figure 2. Speed comparison of Decord, TorchCodec (with Resize), and [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Video decoding performance across different video durations (1 FPS sampling). Speed Across Video Durations. Our framework relies on pre-computing intervals and sufficient keyframes for parallelization. Therefore, we expect reduced benefits for shorter videos. We benchmark QUICKCODEC on videos of varying lengths, from 1 minute to 1 hour, using the same source video (an hour-long movie) cut to different durations. All… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Ablation study on group size and retention ratio. Data from [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Latency breakdown for video loading, prefill, and LLM decoding in end-to-end inference. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Video decoding performance for dif￾ferent video durations with 1 FPS sampling. As QUICKCODEC does not seek between loading frames, all video frames are decoded during video loading. Conversely, seek-based frameworks skip decoding segments of video if there are large ga…

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. CoVR-R:Reason-Aware Composed Video Retrieval

    cs.CV 2026-03 conditional novelty 6.0 of 10

    Zero-shot LMM reasoning over edit after-effects (states, phases, camera, tempo) plus a new CoVR-R benchmark yields large recall gains on implicit-effect composed video retrieval without task-specific training.

  2. cuRAMSES: Scalable AMR Optimizations for Large-Scale Cosmological Simulations

    astro-ph.GA 2026-04 conditional novelty 5.0 of 10

    Recursive k-section domain decomposition, Morton-key hashing, and GPU dispatch cut communication and memory bottlenecks in RAMSES while preserving conservation to ~0.5%.

Reference graph

Works this paper leans on

49 extracted references · 16 canonical work pages · cited by 2 Pith papers

  1. [1]

    Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebr’on, and Sumit K. Sanghai. Gqa: Training generalized multi-query transformer models from multi-head checkpoints. ArXiv, abs/2305.13245, 2023. URL https://api.semanticscholar.org/ CorpusID:258833177

    arXiv 2023

  2. [2]

    Deep Architectures for Content Moderation and Movie Content Rating

    Fatih Cagatay Akyon and Alptekin Temizel. Deep architectures for content moderation and movie content rating, 2022. URL https://arxiv.org/abs/2212.04533

    work page Pith review arXiv 2022

  3. [3]

    Amazon ec2 p5 instances

    Amazon. Amazon ec2 p5 instances. https://aws.amazon.com/ec2/instance-types/ p5/, 2025. Accessed: 2025-05-10

    work page 2025

  4. [4]

    Qwen2.5-vl technical report,

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report,

  5. [5]

    YouTube: hours of video uploaded every minute 2022| Statista — statista.com

    Laura Ceci. YouTube: hours of video uploaded every minute 2022| Statista — statista.com. https://www.statista.com/statistics/259477/ hours-of-video-uploaded-to-youtube-every-minute/ , 2024. [Accessed 16- 05-2025]

    work page 2022

  6. [6]

    An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models

    Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang. An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models. In European Conference on Computer Vision , 2024. URL https://api.semanticscholar.org/CorpusID:268358224

    work page 2024

  7. [8]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling, 2025

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, Lixin Gu, Xuehui Wang, Qingyun Li, Yimin Ren, Zixuan Chen, Jiapeng Luo, Jiahao Wang, Tan Jiang, Bo Wang, Conghui He, Botian Shi, Xingcheng Zhang, Han Lv, Yi Wang, Wenqi Shao, Pei Chu, Zhongying Tu, Tong He, Zhiyong Wu, Huipeng Deng, Jia...

    arXiv 2025

  8. [10]

    Flashattention-2: Faster attention with better parallelism and work partition- ing

    Tri Dao. Flashattention-2: Faster attention with better parallelism and work partition- ing. ArXiv, abs/2307.08691, 2023. URL https://api.semanticscholar.org/CorpusID: 259936734

    arXiv 2023

Show all 49 references
  1. [11]

    A simple and effective l_2 norm-based strategy for KV cache compression

    Alessio Devoto, Yu Zhao, Simone Scardapane, and Pasquale Minervini. A simple and effective l_2 norm-based strategy for KV cache compression. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Langua...

    2024 doi

  2. [12]

    Distributed (Deep) Machine Learning Community. Decord. https://github.com/dmlc/ decord, 2019. Accessed: 2025-05-10

    2019

  3. [13]

    Global Expansion of AI Surveillance

    Steven Feldstein. Global Expansion of AI Surveillance. Carnegie Endowment for International Peace, 2022

    2022

  4. [14]

    Video- mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analy- sis

    Chaoyou Fu, Yuhan Dai, Yondong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, Peixian Chen, Yanwei Li, Shaohui Lin, Sirui Zhao, Ke Li, Tong Xu, Xiawu Zheng, Enhong Chen, Rongrong Ji, and Xing Sun. Video- mme: The first-ever compre...

    2024 arXiv

  5. [15]

    Gpu machine types

    Google. Gpu machine types. https://cloud.google.com/compute/docs/gpus, 2025. Accessed: 2025-05-10

    2025

  6. [16]

    Attention score is not all you need for token importance indicator in kv cache reduction: Value also matters

    Zhiyu Guo, Hidetaka Kamigaito, and Taro Watanabe. Attention score is not all you need for token importance indicator in kv cache reduction: Value also matters. In Conference on Empiri- cal Methods in Natural Language Processing, 2024. URL https://api.semanticscholar. org/Corpu...

    2024

  7. [17]

    The video codec landscape in 2020

    Michel Kerdranvat, Ya Chen, Rémi Jullian, Franck Galpin, and Edouard François. The video codec landscape in 2020. ITU Journal: ICT Discoveries, 3(1):73–83, 2020

    2020

  8. [18]

    Mpeg-4 overview

    Rob Koenen. Mpeg-4 overview. Technical report, International Organization for Standardiza- tion, 1999. URL https://sound.media.mit.edu/resources/mpeg4/audio/general/ w3156.pdf

    1999

  9. [19]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large lan- guage model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Syste...

    2023

  10. [20]

    Llava-onevision: Easy visual task transfer

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer. ArXiv, abs/2408.03326, 2024. URL https://api.semanticscholar.org/CorpusID: 271719914

    2024 arXiv

  11. [21]

    Video-llava: Learning united visual representation by alignment before projection

    Bin Lin, Bin Zhu, Yang Ye, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection. In Conference on Empirical Methods in Natural Language Processing , 2023. URL https://api.semanticscholar. org/CorpusID:265281544

    2023

  12. [22]

    Xingyu Liu, Patrick Carrington, Xiang ’Anthony’ Chen, and Amy Pavel. What makes videos accessible to blind and visually impaired people? In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, CHI ’21, New York, NY , USA, 2021. Association for Computin...

    2021

  13. [23]

    Torchvision: Pytorch’s computer vision library

    TorchVision maintainers and contributors. Torchvision: Pytorch’s computer vision library. https://github.com/pytorch/vision, 2016

    2016

  14. [24]

    Nd-h100-v5 sizes series

    Microsoft. Nd-h100-v5 sizes series. https://learn.microsoft.com/en-us/azure/ virtual-machines/sizes/gpu-accelerated/ndh100v5-series , 2025. Accessed: 2025-05-10

    2025

  15. [25]

    Slowfocus: Enhancing fine-grained temporal understanding in video llm

    Ming Nie, Dan Ding, Chunwei Wang, Yuanfan Guo, Jianhua Han, Hang Xu, and Li Zhang. Slowfocus: Enhancing fine-grained temporal understanding in video llm. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Informati...

    2024

  16. [26]

    Cosmos world foundation model platform for physical ai, 2025

    NVIDIA, :, Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, Yifan Ding, Daniel Dworakowski, Jiaojiao Fan, Michele Fenzi, Francesco Ferroni, Sanja Fidler, Dieter Fox, Songwei Ge, Yunhao Ge, Jinwei ...

    2025 arXiv

  17. [27]

    torchcodec

    PyTorch Team. torchcodec. https://github.com/pytorch/torchcodec, 2025. Accessed: 2025-05-10

    2025

  18. [28]

    Llava-prumerge: Adaptive token reduction for efficient large multimodal models

    Yuzhang Shang, Mu Cai, Bingxin Xu, Yong Jae Lee, and Yan Yan. Llava-prumerge: Adaptive token reduction for efficient large multimodal models. ArXiv, abs/2403.15388, 2024. URL https://api.semanticscholar.org/CorpusID:268667281

    2024

  19. [29]

    Noam M. Shazeer. Glu variants improve transformer. ArXiv, abs/2002.05202, 2020. URL https://api.semanticscholar.org/CorpusID:211096588

    2002 arXiv

  20. [30]

    Reducing traffic wastage in video streaming via bandwidth-efficient bitrate adaptation

    Hairong Su, Shibo Wang, Shusen Yang, Tianchi Huang, and Xuebin Ren. Reducing traffic wastage in video streaming via bandwidth-efficient bitrate adaptation. IEEE Transactions on Mobile Computing, 23(11):10361–10377, November 2024. ISSN 2161-9875. doi: 10.1109/tmc. 2024.3373498....

    2024

  21. [31]

    Sullivan, Jens-Rainer Ohm, Woo-Jin Han, and Thomas Wiegand

    Gary J. Sullivan, Jens-Rainer Ohm, Woo-Jin Han, and Thomas Wiegand. Overview of the high efficiency video coding (hevc) standard. IEEE Transactions on Circuits and Systems for Video Technology, 22(12):1649–1668, 2012. doi: 10.1109/TCSVT.2012.2221191

    2012

  22. [32]

    Converting video formats with ffmpeg

    Suramya Tomar. Converting video formats with ffmpeg. Linux Journal, 2006(146):10, 2006

    2006

  23. [33]

    Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J

    Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, E...

    2020

  24. [34]

    Lvbench: An extreme long video understanding benchmark

    Weihan Wang, Zehai He, Wenyi Hong, Yean Cheng, Xiaohan Zhang, Ji Qi, Shiyu Huang, Bin Xu, Yuxiao Dong, Ming Ding, and Jie Tang. Lvbench: An extreme long video understanding benchmark. ArXiv, abs/2406.08035, 2024. URL https://api.semanticscholar.org/ CorpusID:270391637

    2024 arXiv

  25. [35]

    Internvideo2.5: Empowering video mllms with long and rich context modeling

    Yi Wang, Xinhao Li, Ziang Yan, Yinan He, Jiashuo Yu, Xiangyun Zeng, Chenting Wang, Changlian Ma, Haian Huang, Jianfei Gao, Min Dou, Kaiming Chen, Wenhai Wang, Yu Qiao, Yali Wang, and Limin Wang. Internvideo2.5: Empowering video mllms with long and rich context modeling. ArXiv,...

    2025 arXiv

  26. [36]

    Stop looking for important tokens in multimodal language models: Du- plication matters more

    Zichen Wen, Yifeng Gao, Shaobo Wang, Junyuan Zhang, Qintong Zhang, Weijia Li, Conghui He, and Linfeng Zhang. Stop looking for important tokens in multimodal language models: Du- plication matters more. ArXiv, abs/2502.11494, 2025. URL https://api.semanticscholar. org/CorpusID:...

    2025 arXiv

  27. [37]

    Longvlm: Efficient long video understanding via large language models, 2024

    Yuetian Weng, Mingfei Han, Haoyu He, Xiaojun Chang, and Bohan Zhuang. Longvlm: Efficient long video understanding via large language models, 2024. URL https://arxiv.org/abs/ 2404.03384

    2024 arXiv

  28. [38]

    Wiegand, G.J

    T. Wiegand, G.J. Sullivan, G. Bjontegaard, and A. Luthra. Overview of the h.264/avc video coding standard. IEEE Transactions on Circuits and Systems for Video Technology , 13(7): 560–576, 2003. doi: 10.1109/TCSVT.2003.815165

    2003

  29. [39]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020

  30. [40]

    Longvideobench: A benchmark for long-context interleaved video-language understanding

    Haoning Wu, Dongxu Li, Bei Chen, and Junnan Li. Longvideobench: A benchmark for long-context interleaved video-language understanding. ArXiv, abs/2407.15754, 2024. URL https://api.semanticscholar.org/CorpusID:271329356

    2024 arXiv

  31. [41]

    Efficient streaming language models with attention sinks

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. ArXiv, abs/2309.17453, 2023. URL https://api. semanticscholar.org/CorpusID:263310483

    2023 arXiv

  32. [42]

    Pyramiddrop: Accelerating your large vision-language models via pyramid visual redundancy reduction

    Long Xing, Qidong Huang, Xiao wen Dong, Jiajie Lu, Pan Zhang, Yuhang Zang, Yuhang Cao, Conghui He, Jiaqi Wang, Feng Wu, and Dahua Lin. Pyramiddrop: Accelerating your large vision-language models via pyramid visual redundancy reduction. ArXiv, abs/2410.17247, 2024. URL https://...

    arXiv

  33. [43]

    Towards surveillance video-and-language understanding: New dataset, baselines, and challenges,

    Tongtong Yuan, Xuange Zhang, Kun Liu, Bo Liu, Chen Chen, Jian Jin, and Zhenzhen Jiao. Towards surveillance video-and-language understanding: New dataset, baselines, and challenges,

  34. [44]

    Lmms-eval: Reality check on the evaluation of large multimodal models, 2024

    Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, and Ziwei Liu. Lmms-eval: Reality check on the evaluation of large multimodal models, 2024. URL https://arxiv.org/abs/2407.12772

    2024 arXiv

  35. [45]

    Barrett, Zhangyang Wang, and Beidi Chen

    Zhenyu (Allen) Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark W. Barrett, Zhangyang Wang, and Beidi Chen. H2o: Heavy-hitter oracle for efficient generative inference of large language mod- els. ArXiv, ab...

    2023 arXiv

  36. [46]

    Mlvu: A comprehensive benchmark for multi-task long video understanding

    Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Shitao Xiao, Xi Yang, Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu. Mlvu: A comprehensive benchmark for multi-task long video understanding. arXiv preprint arXiv:2406.04264, 2024

    2024 arXiv

  37. [47]

    A survey on generative ai and llm for video generation, understanding, and streaming, 2024

    Pengyuan Zhou, Lin Wang, Zhi Liu, Yanbin Hao, Pan Hui, Sasu Tarkoma, and Jussi Kangasharju. A survey on generative ai and llm for video generation, understanding, and streaming, 2024. URL https://arxiv.org/abs/2404.16038

    2024 arXiv

  38. [49]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models, 2025

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, Zhangwei Gao, Erfei Cui, Xuehui Wang, Yue Cao, Yangzhou Liu, Xingguang Wei, Hongjie Zhang, Haomin Wang, Weiye Xu, Hao Li, Jiahao Wang, Nianchen 14 Deng, Songze ...

    2025 arXiv

  39. [2023]

    URL https://arxiv.org/abs/2309.13925

    arXiv

  40. [2024]

    URL https://arxiv.org/abs/2406.07476

    arXiv

  41. [2025]

    URL https://arxiv.org/abs/2502.13923

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.