Pith. sign in

REVIEW 5 major objections 6 minor 58 references

SceneRAG: Scene-level Retrieval-Augmented Generation for Video Understanding

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that segmenting long videos into narrative scenes, rather than fixed chunks, lifts retrieval-augmented video question answering to pairwise win rates of up to 72.5 percent over prior RAG systems on a 134-hour benchmark.

desk verdict Sensible scene-level RAG integration, but the unvalidated LLM-judge evaluation makes the headline win rates unreliable. read the letter →

arxiv 2506.07600 v1 pith:WKR6LH6J submitted 2025-06-09 cs.CV cs.AI

Nianbo Zeng , Haowen Hou , Fei Richard Yu , Si Shi , Ying Tiffany He This is my paper
classification cs.CVcs.AI
keywords videounderstandingretrieval-augmentedgenerationscenesegmentationmultimodalknowledgegraphlong-formmulti-hopretrievalLLMasjudgeentityrelationextraction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the standard recipe for retrieval-augmented generation over video — cutting the footage into fixed-length chunks and retrieving from those chunks — breaks the narrative unit that question answering actually needs, and that scene-level structure should replace it. SceneRAG therefore asks a large language model to split a video into coherent scenes using ASR transcripts and timestamps, refines those boundaries with silence heuristics and iterative correction, and builds a per-scene multimodal knowledge graph that fuses visual captions with transcripts. On the LongerVideos benchmark, more than 134 hours of lectures, documentaries, and entertainment, the method reports win rates up to 72.5 percent over prior RAG baselines in LLM-judged pairwise comparisons, with the highest average overall win rate of 70.8 percent against graph-based RAG. The payoff, if the result holds, is that retrieval granularity — not just bigger models or better encoders — is a first-order lever for long-video understanding.

What carries the argument

The load-bearing mechanism is LLM-driven scene segmentation coupled with a scene-level multimodal knowledge graph. Segmentation works chunk-wise: each five-minute chunk with a ten-second overlap is transcribed, an LLM proposes scene intervals of roughly 15 to 60 seconds, and heuristic refinements — silence-aware assignment, short-scene merging, boundary alignment to punctuation — enforce continuity and coverage, with failed outputs triggering re-prompting under adaptive constraints. Each resulting scene contributes two extraction channels, a vision-language caption of sampled keyframes and the ASR transcript, whose entities and relations are fused into a knowledge graph whose nodes are scenes, entities, and events and whose edges encode co-occurrence and temporal adjacency. This graph carries the argument's long-range component: it lets a query hop from a keyword to entities that recur in temporally distant scenes, and the token-budgeted scene retrieval then assembles exactly the segments that answer the question.

What would settle it

Re-run the LongerVideos comparisons with human raters scoring SceneRAG against GraphRAG-global and VideoRAG on the same five dimensions; if human win rates do not exceed 50 percent or diverge sharply from the LLM-judged 70.8 and 56.9 percent, the central empirical claim is refuted. A cheaper check: re-compute the aggregated win rates under each of the three judges separately and require every judge to keep SceneRAG above 65 percent on the overall winner dimension, which the appendix tables already show to fail for GPT-4.1-nano.

Watch

Extended reading notes

Core claim

The paper's central claim is that fixed-length chunks are the wrong unit for video retrieval because they straddle semantic boundaries and flatten the long-range references — recurring characters, thematic motifs, quantitative callbacks — that real questions depend on. SceneRAG operationalizes the fix as a three-stage pipeline: an LLM proposes scene boundaries from the timestamped transcript of each five-minute chunk with a ten-second overlap, and the raw cuts are sharpened by promoting silences longer than ten seconds to scene breaks, splitting shorter silences between neighbours, merging scenes under ten seconds, and snapping boundaries to sentence-level punctuation, with up to four retries and escalation to a stronger model on failure. Each final scene is then represented by two text channels, a vision-language description of sampled keyframes and the raw transcript, from which entities and relations are extracted separately and fused by LLM-assisted disambiguation into a scene-level knowledge graph built incrementally as scenes are produced. Given a query, SceneRAG retrieves a token-budgeted set of scenes by embedding similarity over both channels and by multi-hop traversal of the graph, then feeds transcripts, query-focused captions, and graph substructure to an LLM for the answer. The reported outcome is that this beats NaiveRAG, both GraphRAG variants, LightRAG, and VideoRAG across every evaluation dimension and domain, with the highest average overall win rate of 70.8 percent against GraphRAG-global and 56.9 percent against the multimodal VideoRAG baseline.

Load-bearing premise

The load-bearing premise is that the GPT-based judges' pairwise win rates are a reliable proxy for what human readers would actually prefer, since every headline number in the evaluation rests on those judgments and none of them are checked against human ratings.

Editorial extensions

If this is right

  • Any downstream task on long video that needs assembled context — summarization, entity tracking, cross-video question answering — should inherit the win because retrieval units now match narrative units rather than fixed durations.
  • Fusing visual captions with transcripts at scene level captures entity relations that either modality alone misses, since the visual channel describes actions and layouts while the transcript carries spoken reasoning.
  • Multi-hop graph retrieval lets answers span non-contiguous moments: the case study answers a cost-and-efficiency question from two distant scenes totalling 122 seconds, where VideoRAG retrieved nine segments totalling 270 seconds.
  • The pipeline is built from frozen foundation models — an LLM for segmentation and generation, a VLM for captions, a multimodal encoder for retrieval — so it transfers to new video domains without task-specific training.
  • The gains concentrate on reasoning dimensions (Comprehensiveness, Empowerment, Depth reach 70 to 74 percent win rates) while Density stays near parity, implying scene structure improves synthesis of dispersed evidence rather than raw information coverage.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The headline win rates are judge-dependent: for the same SceneRAG-versus-GraphRAG-global comparison, the aggregated overall win rate sits at 70.8 percent, but the appendix breaks it into 62.7 percent under GPT-4.1-nano and 81.5 percent under GPT-4.1-mini, a spread of nearly 19 points that the averaging hides; a single-judge replication would reveal how much of the reported advantage is real.
  • Because scene boundaries come from transcripts alone, scenes defined purely by visuals — a cut to a new location, a montage, an emotional beat without dialogue — are mis-segmented; injecting shot-boundary or music-change detectors as hard constraints is a cheap, testable upgrade the paper itself names as future work.
  • If scene granularity is the true driver of the gains, a cheaper substitute — punctuation, silence, and speaker-turn boundaries without any LLM — might capture a large share of the improvement; a direct ablation of LLM segmentation versus pure heuristics on retrieval precision would settle how much the expensive component buys.
  • The same recipe transfers to other long structured media, such as meeting recordings, podcasts, and surveillance footage, where recurring entities and dispersed references create the same multi-hop retrieval problem that scene-level graphs thread together here.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. SceneRAG proposes a scene-level retrieval-augmented generation pipeline for long-form video understanding. The system segments videos into narrative-consistent scenes by having an LLM process ASR transcripts and timestamps, refines boundaries with silence-aware heuristics and iterative correction, builds a multimodal knowledge graph per scene, and performs token-budgeted retrieval over scene embeddings and graph relations before answer generation. The empirical claim is that this scene-aware graph-based organization materially improves video question answering, supported by LLM-judged win-rate comparisons against NaiveRAG, GraphRAG variants, LightRAG, and VideoRAG on the LongerVideos benchmark, together with ablations and a case study.

Significance. If the empirical claims were fully validated, SceneRAG would be a useful, modular contribution to video RAG: the pipeline is clearly specified, the ablation and case study illustrate the intended mechanism, and the approach directly targets the acknowledged weakness of fixed-chunk segmentation. The paper's strengths include a detailed algorithm description, a public benchmark evaluation, and a transparent case study with timestamped evidence. However, the central evidence rests on LLM-as-judge win rates that are not calibrated against human judgment, and the per-judge disagreement is large enough to call the headline numbers into question. The paper also provides no retrieval-quality metrics and no validation of the scene boundaries themselves, so the mechanism behind the claimed improvement is only indirectly supported. With human-validated evaluation, retrieval-level metrics, and a more careful presentation of ablations, the work could be significant for the video understanding and RAG communities. No code or data release is mentioned, so reproducibility is not yet demonstrated.

major comments (5)
  1. [§4.2, Tables 6–8] The headline 70.8% Overall Winner win rate against GraphRAG2 is an unweighted average of three per-judge rates that are far apart: 62.7% (GPT-4o-mini, Table 6), 81.5% (GPT-4.1-mini, Table 7), and 68.3% (GPT-4.1-nano, Table 8). More concerning, the Density dimension against NaiveRAG flips direction across judges: GPT-4o-mini gives SceneRAG 60.9%, GPT-4.1-mini gives 41.0%, and GPT-4.1-nano gives 48.2%. Because no human preference data are reported, there is no way to determine which judge, if any, reflects genuine answer-quality differences. This is load-bearing for the central claim. Please add a human-validated subset of pairwise judgments (e.g., 50–100 comparisons), report per-judge agreement statistics, and either present results separately by judge or provide a principled aggregation method.
  2. [§4.4, Table 4] The text claims that 'LLM-based scene-aware segmentation enhances performance across all metrics compared to raw input,' but Table 4 shows non-monotonic behavior: on the Lecture domain the +LLM condition slightly lowers Overall Score (from 4.37 to 4.35) and Empowerment (from 4.49 to 4.45) relative to fixed segmentation. The larger gain appears when rules are added on top (Overall Score 4.37 to 4.50), so the marginal contribution of the LLM segmentation step is not cleanly isolated from the rule-based post-processing. Please report per-condition variance or significance tests and discuss this non-monotonicity honestly.
  3. [§4.1, §4.3] The evaluation does not include any retrieval-quality metrics (e.g., recall@k, precision, or nDCG against human-annotated relevant segments). SceneRAG's contribution is primarily in how scenes are segmented and retrieved, but the only retrieval evidence is the single-query case study in §4.3 with Table 3. That single example cannot support the general claim of 'precise retrieval.' Please add retrieval-level evaluation on LongerVideos or a labeled subset, in addition to end-to-end win rates.
  4. [Abstract, §4.2] The abstract's 'win rate of up to 72.5 percent' is a single cell in Table 2 (Trustworthiness vs GraphRAG2, All), not the representative result. The more meaningful aggregate is the 70.8% Overall Winner average, but even that is an average over widely varying per-judge scores. Relatedly, the text claims SceneRAG shows 'consistent improvements' across all dimensions, yet Table 2 contains Density near 50% or below in several cells (e.g., 50.0% vs NaiveRAG All, 49.6% vs NaiveRAG Documentary, and 49.1% vs VideoRAG Entertainment). Please report the aggregate result rather than a cherry-picked dimension, and qualify the consistency claim to acknowledge ties/losses.
  5. [§3.2, Appendix B] The paper acknowledges in Appendix B that segmentation is primarily transcript-driven and may miss visually grounded transitions, yet the evaluation never measures the quality of the scene boundaries themselves. Since scene-boundary quality is the central premise of the method, the paper should include a direct segmentation-evaluation component, such as boundary mis-detection rate or segment coherence against human annotations on a subset, or at least report statistics on scene durations, boundary agreement with ASR sentence boundaries, and failure cases. Without this, the mechanism behind the claimed end-to-end gains remains unverified.
minor comments (6)
  1. [§4.2, Figure 2] The caption for Figure 2 says 'Quantitative Comparison with NaiveRAG Using a 5-Point Likert Scale,' but the text introducing it discusses comparison with large vision-language models. Please clarify what is being plotted and provide the LVM comparison numbers in a table or in the figure itself.
  2. [§4.1] The evaluation dimensions are called 'human-centric,' but no human judges are involved; consider using a less misleading term such as 'generation-quality dimensions.'
  3. [§3.4, Eq. (14)] The token-budgeted subset selection in Eq. (14) is a knapsack-like optimization, but no algorithm for solving it is given; please state whether it is solved exactly or approximated, and with what complexity.
  4. [Appendix A, Algorithm 1] In Algorithm 1, the function LLMFunc(P, history) is called with a 'history' argument that is never defined; please specify what information is passed in history.
  5. [References] References [9] and [10] appear to be incomplete or placeholder entries; please update them with full bibliographic details.
  6. [Appendix E, Table 11] Several transcript excerpts in Table 11 contain 'Cloud' where 'Claude' is presumably intended (e.g., the first scene). If these are direct ASR outputs, consider noting that transcription errors are preserved in the raw transcripts.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the SceneRAG pipeline is an empirical system whose claimed win-rates are measured by external pairwise judgments, not derived from or defined by its own inputs; the LLM-as-judge variability is an evaluation-validity concern, not a circular derivation.

full rationale

This paper is an empirical systems paper rather than a derivation. The claimed contribution is a pipeline: LLM-based scene segmentation over ASR transcripts and timestamps (Eqs. 1-7), heuristic refinement, multimodal scene-level knowledge-graph construction (Eqs. 8-11), token-budgeted cosine retrieval (Eq. 14), and LLM generation over retrieved context (Eqs. 15-16). None of these equations defines the headline win-rate in terms of a fitted parameter, and none of the reported numbers is a fitted input renamed as a prediction. The headline result, 70.8% average overall win-rate against GraphRAG-global, is a measured aggregate of pairwise LLM-judged comparisons in Table 2, not a quantity recovered from the model equations. The ablation and graph-expansion tables compare variants of the same pipeline, which is a controlled internal comparison, not circular. The LLM-as-judge protocol does raise a serious empirical-validity concern: the three judges disagree substantially in Appendix C, and the Density dimension flips direction across judges (e.g., 60.9% for GPT-4o-mini versus 41.0% for GPT-4.1-mini against NaiveRAG). That is a threat to the strength of the empirical claim, but it is not circularity: the judge outputs are not constructed from the claim, and no parameter was fitted to make the claim true. The paper also discloses in Appendix B that segmentation is transcript-driven and can miss visually grounded transitions; again, this is an acknowledged limitation, not a circular step. There are no load-bearing self-citations: references to GraphRAG [13], VideoRAG [36], and psychology [47] are external prior work, and the paper invokes no uniqueness theorem or prior result from its own authors. Under the rule that circularity requires a specific reduction by definition, fitting, or self-citation chain, no such step is present.

Assumptions & free parameters 8 free parameters · 6 assumptions · 0 invented entities

The central claim rests on hand-chosen hyperparameters and untested assumptions about transcript fidelity, LLM segmentation ability, embedding quality, and the validity of LLM-as-judge evaluation. No ground-truth scene labels, retrieval quality metrics, or human evaluation are provided.

free parameters (8)
  • Chunk length L = 5 minutes
    Fixed chunk duration for initial segmentation; affects boundary localization and transcript context.
  • Chunk overlap = 10 seconds
    Overlap between adjacent chunks to preserve context continuity; hand-chosen.
  • Silence threshold epsilon = 10 seconds
    Defines whether a silent interval is split between adjacent scenes or promoted to an independent [SILENT] scene.
  • Minimum scene duration = 10 seconds
    Scenes shorter than this are merged with neighbors during post-hoc correction.
  • Target scene duration range = 15-60 seconds
    Prompt constraint for LLM segmentation; used to trigger re-prompting on violation.
  • Token budget tau = 2400 tokens
    Maximum total token length of retrieved scene content in Eq. (14).
  • Keyframe sampling interval = every 6 seconds, k <= 10
    Used to sample frames for VLM description generation.
  • LLM retry count / escalation = up to 4 retries, escalate to GPT-4o
    Adaptive re-prompting if boundary constraints violated.
assumptions (6)
  • domain assumption ASR transcripts faithfully reflect narrative content and temporal structure
    Segmentation is driven by ASR transcripts plus timestamps; if transcripts are noisy (as the authors note in Appendix B), scene boundaries degrade.
  • domain assumption LLM can produce narrative-consistent scene boundaries from transcripts
    Core of Section 3.2; no ground-truth scene labels or human evaluation of boundary quality are provided.
  • domain assumption VLM descriptions of sampled keyframes capture visual information relevant to entity/relation extraction
    Used to build visual modality channel; visual cues are not used in segmentation itself (admitted in Appendix B).
  • standard math ImageBind/text-embedding-3-small embeddings give reliable semantic similarity for retrieval
    Retrieval in Eq. (13)-(14) relies on cosine similarity in embedding space; no retrieval precision/recall measured.
  • domain assumption LLM-as-judge win-rates reflect human-perceived quality
    Evaluation protocol in Section 4.1 uses LLM judges without human validation; win-rates vary widely across judges.
  • domain assumption Knowledge graph edges from co-occurrence and temporal adjacency support useful multi-hop retrieval
    Section 3.3 constructs graph but no graph-quality or multi-hop retrieval ablation is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SceneRAG: Scene-level Retrieval-Augmented Generation for Video Understanding." pith.science (2026). https://pith.science/paper/WKR6LH6J

@misc{pith2026250607600,
  author       = {Pith},
  title        = {Pith review of: SceneRAG: Scene-level Retrieval-Augmented Generation for Video Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WKR6LH6J}},
  note         = {Machine review of arXiv:2506.07600}
}
read the original abstract

Despite recent advances in retrieval-augmented generation (RAG) for video understanding, effectively understanding long-form video content remains underexplored due to the vast scale and high complexity of video data. Current RAG approaches typically segment videos into fixed-length chunks, which often disrupts the continuity of contextual information and fails to capture authentic scene boundaries. Inspired by the human ability to naturally organize continuous experiences into coherent scenes, we present SceneRAG, a unified framework that leverages large language models to segment videos into narrative-consistent scenes by processing ASR transcripts alongside temporal metadata. SceneRAG further sharpens these initial boundaries through lightweight heuristics and iterative correction. For each scene, the framework fuses information from both visual and textual modalities to extract entity relations and dynamically builds a knowledge graph, enabling robust multi-hop retrieval and generation that account for long-range dependencies. Experiments on the LongerVideos benchmark, featuring over 134 hours of diverse content, confirm that SceneRAG substantially outperforms prior baselines, achieving a win rate of up to 72.5 percent on generation tasks.

Figures

Figures reproduced from arXiv: 2506.07600 by the authors.

Figure 1
Figure 1. Overview of SceneRAG. Given a long video, SceneRAG segments it into scenes highlighted [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Quantitative Comparison with NaiveRAG Using a 5-Point Likert Scale. The “All” column [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Error correction instructions for scene segmentation. [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Instructions for scene segmentation and formatting. [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]
Figure 5
Figure 5. Figure 5: Quantitative Comparison by gpt-4o-mini. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: Quantitative Comparison by gpt-4.1-mini. [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]
Figure 7
Figure 7. Figure 7: Quantitative Comparison by gpt-4.1-nano. [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

58 extracted references · 29 canonical work pages

  1. [1]

    Ask in any modality: A comprehensive survey on multimodal retrieval-augmented generation

    Mohammad Mahdi Abootorabi, Amirhosein Zobeiri, Mahdi Dehghani, Mohammadali Moham- madkhani, Bardia Mohammadi, Omid Ghahroodi, Mahdieh Soleymani Baghshah, and Ehsaneddin Asgari. Ask in any modality: A comprehensive survey on multimodal retrieval-augmented generation. arXiv preprint arXiv:2502.08826, 2025

    arXiv 2025

  2. [2]

    Flamingo: a visual language model for few-shot learning.Advances in neural information processing systems, 35: 23716–23736, 2022

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning.Advances in neural information processing systems, 35: 23716–23736, 2022

    2022

  3. [3]

    Vivit: A video vision transformer

    Anurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Lu ˇci´c, and Cordelia Schmid. Vivit: A video vision transformer. InProceedings of the IEEE/CVF international conference on computer vision, pages 6836–6846, 2021

    2021

  4. [4]

    Unified graph structured models for video understanding

    Anurag Arnab, Chen Sun, and Cordelia Schmid. Unified graph structured models for video understanding. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 8117–8126, 2021

    work page 2021

  5. [5]

    Self-rag: Learning to retrieve, generate, and critique through self-reflection

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. Self-rag: Learning to retrieve, generate, and critique through self-reflection. InThe Twelfth International Conference on Learning Representations, 2023

    2023

  6. [6]

    Sharegpt4video: Improving video understanding and generation with better captions.Advances in Neural Information Processing Systems, 37:19472–19495, 2024

    Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Zhenyu Tang, Li Yuan, et al. Sharegpt4video: Improving video understanding and generation with better captions.Advances in Neural Information Processing Systems, 37:19472–19495, 2024

    work page 2024

  7. [7]

    Wavrag: Audio-integrated retrieval augmented generation for spoken dialogue models

    Yifu Chen, Shengpeng Ji, Haoxiao Wang, Ziqing Wang, Siyu Chen, Jinzheng He, Jin Xu, and Zhou Zhao. Wavrag: Audio-integrated retrieval augmented generation for spoken dialogue models. arXiv preprint arXiv:2502.14727, 2025

    arXiv 2025

  8. [8]

    Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms.arXiv preprint arXiv:2406.07476, 2024

    Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, et al. Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms.arXiv preprint arXiv:2406.07476, 2024

    arXiv 2024

Show all 58 references
  1. [9]

    W. Chu, J. Lee, P. Xu, et al. Graphrag: A graph-enhanced retrieval-augmented generation framework. InNeurIPS, 2024

    2024

  2. [10]

    Sscformer: Push the limit of chunk-wise conformer for streaming asr using sequentially sampled chunks and chunked causal convolution

    Sampled Chunks. Sscformer: Push the limit of chunk-wise conformer for streaming asr using sequentially sampled chunks and chunked causal convolution

  3. [11]

    Event segmentation and seven types of narrative discontinuity in popular movies.Acta psychologica, 149:69–77, 2014

    James E Cutting. Event segmentation and seven types of narrative discontinuity in popular movies.Acta psychologica, 149:69–77, 2014

    2014

  4. [12]

    Large-scale narrative events in popular cinema

    James E Cutting and Kacie L Armstrong. Large-scale narrative events in popular cinema. Cognitive Research: Principles and Implications, 4:1–18, 2019

    2019

  5. [13]

    From local to global: A graph rag approach to query-focused summarization.arXiv preprint arXiv:2404.16130, 2024

    Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson. From local to global: A graph rag approach to query-focused summarization.arXiv preprint arXiv:2404.16130, 2024

    2024 arXiv

  6. [14]

    Videoagent: A memory-augmented multimodal agent for video understanding

    Yue Fan, Xiaojian Ma, Rujie Wu, Yuntao Du, Jiaqi Li, Zhi Gao, and Qing Li. Videoagent: A memory-augmented multimodal agent for video understanding. InEuropean Conference on Computer Vision, pages 75–92. Springer, 2024

    2024

  7. [15]

    Distil-whisper: Robust knowledge distillation via large-scale pseudo labelling.arXiv preprint arXiv:2311.00430, 2023

    Sanchit Gandhi, Patrick von Platen, and Alexander M Rush. Distil-whisper: Robust knowledge distillation via large-scale pseudo labelling.arXiv preprint arXiv:2311.00430, 2023

    2023 arXiv

  8. [16]

    Retrieval-augmented generation for large language models: A survey.arXiv preprint arXiv:2312.10997, 2:1, 2023

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yixin Dai, Jiawei Sun, Haofen Wang, and Haofen Wang. Retrieval-augmented generation for large language models: A survey.arXiv preprint arXiv:2312.10997, 2:1, 2023. 10

    2023 arXiv

  9. [17]

    Imagebind: One embedding space to bind them all

    Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Imagebind: One embedding space to bind them all. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15180–15190, 2023

    2023

  10. [18]

    Lightrag: Simple and fast retrieval-augmented generation

    Zirui Guo, Lianghao Xia, Yanhua Yu, Tu Ao, and Chao Huang. Lightrag: Simple and fast retrieval-augmented generation. 2024

    2024

  11. [19]

    Videorag: Retrieval- augmented generation over video corpus.arXiv preprint arXiv:2501.05874, 2025

    Soyeong Jeong, Kangsan Kim, Jinheon Baek, and Sung Ju Hwang. Videorag: Retrieval- augmented generation over video corpus.arXiv preprint arXiv:2501.05874, 2025

    2025 arXiv

  12. [20]

    Learning temporal video procedure segmentation from an automatically collected large dataset

    Lei Ji, Chenfei Wu, Daisy Zhou, Kun Yan, Edward Cui, Xilin Chen, and Nan Duan. Learning temporal video procedure segmentation from an automatically collected large dataset. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1506–1515, 2022

    2022

  13. [21]

    Diffusionret: Generative text-video retrieval with diffusion model

    Peng Jin, Hao Li, Zesen Cheng, Kehan Li, Xiangyang Ji, Chang Liu, Li Yuan, and Jie Chen. Diffusionret: Generative text-video retrieval with diffusion model. InProceedings of the IEEE/CVF international conference on computer vision, pages 2470–2481, 2023

    2023

  14. [22]

    Scene graph generation strategy with co-occurrence knowledge and learnable term frequency.arXiv preprint arXiv:2405.12648, 2024

    Hyeongjin Kim, Sangwon Kim, Dasom Ahn, Jong Taek Lee, and Byoung Chul Ko. Scene graph generation strategy with co-occurrence knowledge and learnable term frequency.arXiv preprint arXiv:2405.12648, 2024

    2024 arXiv

  15. [23]

    Multimodal reasoning with multimodal knowledge graph.arXiv preprint arXiv:2406.02030, 2024

    Junlin Lee, Yequan Wang, Jing Li, and Min Zhang. Multimodal reasoning with multimodal knowledge graph.arXiv preprint arXiv:2406.02030, 2024

    2024 arXiv

  16. [24]

    Retrieval-augmented generation for knowledge-intensive nlp tasks.Advances in neural information processing systems, 33:9459–9474, 2020

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks.Advances in neural information processing systems,...

    2020

  17. [25]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. InInternational conference on machine learning, pages 12888–12900. PMLR, 2022

    2022

  18. [26]

    Videochat: Chat-centric video understanding.arXiv preprint arXiv:2305.06355, 2023

    KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. Videochat: Chat-centric video understanding.arXiv preprint arXiv:2305.06355, 2023

    2023 arXiv

  19. [27]

    Llama-vid: An image is worth 2 tokens in large language models

    Yanwei Li, Chengyao Wang, and Jiaya Jia. Llama-vid: An image is worth 2 tokens in large language models. InEuropean Conference on Computer Vision, pages 323–340. Springer, 2024

    2024

  20. [28]

    Univl: A unified video and language pre-training model for multimodal understanding and generation.arXiv preprint arXiv:2002.06353, 2020

    Huaishao Luo, Lei Ji, Botian Shi, Haoyang Huang, Nan Duan, Tianrui Li, Jason Li, Taroon Bharti, and Ming Zhou. Univl: A unified video and language pre-training model for multimodal understanding and generation.arXiv preprint arXiv:2002.06353, 2020

    2002 arXiv

  21. [29]

    The impact of continuity editing in narrative film on event segmentation.Cognitive science, 35(8):1489–1517, 2011

    Joseph P Magliano and Jeffrey M Zacks. The impact of continuity editing in narrative film on event segmentation.Cognitive science, 35(8):1489–1517, 2011

    2011

  22. [30]

    Learning joint embedding with multimodal cues for cross-modal video-text retrieval

    Niluthpol Chowdhury Mithun, Juncheng Li, Florian Metze, and Amit K Roy-Chowdhury. Learning joint embedding with multimodal cues for cross-modal video-text retrieval. InProceedings of the 2018 ACM on international conference on multimedia retrieval, pages 19–27, 2018

    2018

  23. [31]

    Boundary-aware self-supervised learning for video scene segmentation.arXiv preprint arXiv:2201.05277, 2022

    Jonghwan Mun, Minchul Shin, Gunsoo Han, Sangho Lee, Seongsu Ha, Joonseok Lee, and Eun-Sol Kim. Boundary-aware self-supervised learning for video scene segmentation.arXiv preprint arXiv:2201.05277, 2022

    2022 arXiv

  24. [32]

    Video shot boundary detection: a review

    Gautam Pal, Dwijen Rudrapaul, Suvojit Acharjee, Ruben Ray, Sayan Chakraborty, and Nilanjan Dey. Video shot boundary detection: a review. InEmerging ICT for Bridging the Future-Proceedings of the 49th Annual Convention of the Computer Society of India CSI V olume 2, pages 119–1...

    2015

  25. [33]

    What is a multi-modal knowledge graph: A survey.Big Data Research, 32:100380, 2023

    Jinghui Peng, Xinyu Hu, Wenbo Huang, and Jian Yang. What is a multi-modal knowledge graph: A survey.Big Data Research, 32:100380, 2023. 11

    2023

  26. [34]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pag...

    2021

  27. [35]

    Robust speech recognition via large-scale weak supervision

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. InInternational conference on machine learning, pages 28492–28518. PMLR, 2023

    2023

  28. [36]

    Videorag: Retrieval-augmented generation with extreme long-context videos.arXiv preprint arXiv:2502.01549, 2025

    Xubin Ren, Lingrui Xu, Long Xia, Shuaiqiang Wang, Dawei Yin, and Chao Huang. Videorag: Retrieval-augmented generation with extreme long-context videos.arXiv preprint arXiv:2502.01549, 2025

    2025 arXiv

  29. [37]

    Temporal video segmentation to scenes using high-level audiovisual features.IEEE Transactions on Circuits and Systems for Video Technology, 21(8):1163–1177, 2011

    Panagiotis Sidiropoulos, Vasileios Mezaris, Ioannis Kompatsiaris, Hugo Meinedo, Miguel Bugalho, and Isabel Trancoso. Temporal video segmentation to scenes using high-level audiovisual features.IEEE Transactions on Circuits and Systems for Video Technology, 21(8):1163–1177, 2011

    2011

  30. [38]

    Videobert: A joint model for video and language representation learning

    Chen Sun, Austin Myers, Carl V ondrick, Kevin Murphy, and Cordelia Schmid. Videobert: A joint model for video and language representation learning. InProceedings of the IEEE/CVF international conference on computer vision, pages 7464–7473, 2019

    2019

  31. [39]

    Temporal scene montage for self- supervised video scene boundary detection.ACM Transactions on Multimedia Computing, Communi- cations and Applications, 20(7):1–19, 2024

    Jiawei Tan, Pingan Yang, Lu Chen, and Hongxing Wang. Temporal scene montage for self- supervised video scene boundary detection.ACM Transactions on Multimedia Computing, Communi- cations and Applications, 20(7):1–19, 2024

    2024

  32. [40]

    Videoagent: Long-form video understanding with large language model as agent

    Xiaohan Wang, Yuhui Zhang, Orr Zohar, and Serena Yeung-Levy. Videoagent: Long-form video understanding with large language model as agent. InEuropean Conference on Computer Vision, pages 58–76. Springer, 2024

    2024

  33. [41]

    Scene consistency representation learning for video scene segmentation

    Haoqian Wu, Keyu Chen, Yanan Luo, Ruizhi Qiao, Bo Ren, Haozhe Liu, Weicheng Xie, and Linlin Shen. Scene consistency representation learning for video scene segmentation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14021–14030, 2022

    2022

  34. [42]

    Videoclip: Contrastive pre-training for zero-shot video-text understanding.arXiv preprint arXiv:2109.14084, 2021

    Hu Xu, Gargi Ghosh, Po-Yao Huang, Dmytro Okhonko, Armen Aghajanyan, Florian Metze, Luke Zettlemoyer, and Christoph Feichtenhofer. Videoclip: Contrastive pre-training for zero-shot video-text understanding.arXiv preprint arXiv:2109.14084, 2021

    2021 arXiv

  35. [43]

    Retrieval- augmented egocentric video captioning

    Jilan Xu, Yifei Huang, Junlin Hou, Guo Chen, Yuejie Zhang, Rui Feng, and Weidi Xie. Retrieval- augmented egocentric video captioning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13525–13536, 2024

    2024

  36. [44]

    Minicpm-v: A gpt-4v level mllm on your phone.arXiv preprint arXiv:2408.01800, 2024

    Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level mllm on your phone.arXiv preprint arXiv:2408.01800, 2024

    2024 arXiv

  37. [45]

    Momentseeker: A comprehensive benchmark and a strong baseline for moment retrieval within long videos.arXiv preprint arXiv:2502.12558, 2025

    Huaying Yuan, Jian Ni, Yueze Wang, Junjie Zhou, Zhengyang Liang, Zheng Liu, Zhao Cao, Zhicheng Dou, and Ji-Rong Wen. Momentseeker: A comprehensive benchmark and a strong baseline for moment retrieval within long videos.arXiv preprint arXiv:2502.12558, 2025

    2025

  38. [46]

    A formal study of shot boundary detection.IEEE transactions on circuits and systems for video technology, 17(2):168–186, 2007

    Jinhui Yuan, Huiyi Wang, Lan Xiao, Wujie Zheng, Jianmin Li, Fuzong Lin, and Bo Zhang. A formal study of shot boundary detection.IEEE transactions on circuits and systems for video technology, 17(2):168–186, 2007

    2007

  39. [47]

    The brain’s cutting- room floor: Segmentation of narrative cinema.Frontiers in human neuroscience, 4:168, 2010

    Jeffrey M Zacks, Nicole K Speer, Khena M Swallow, and Corey J Maley. The brain’s cutting- room floor: Segmentation of narrative cinema.Frontiers in human neuroscience, 4:168, 2010

    2010

  40. [48]

    Merlot reserve: Neural script knowledge through vision and language and sound

    Rowan Zellers, Jiasen Lu, Ximing Lu, Youngjae Yu, Yanpeng Zhao, Mohammadreza Salehi, Aditya Kusupati, Jack Hessel, Ali Farhadi, and Yejin Choi. Merlot reserve: Neural script knowledge through vision and language and sound. InProceedings of the IEEE/CVF Conference on Computer V...

    2022

  41. [49]

    Rakg: Document-level retrieval augmented knowledge graph construction.arXiv preprint arXiv:2504.09823, 2025

    Hairong Zhang, Jiaheng Si, Guohang Yan, Boyuan Qi, Pinlong Cai, Song Mao, Ding Wang, and Botian Shi. Rakg: Document-level retrieval augmented knowledge graph construction.arXiv preprint arXiv:2504.09823, 2025. 12

    2025 arXiv

  42. [50]

    Video-llama: An instruction-tuned audio-visual language model for video understanding.arXiv preprint arXiv:2306.02858, 2023

    Hang Zhang, Xin Li, and Lidong Bing. Video-llama: An instruction-tuned audio-visual language model for video understanding.arXiv preprint arXiv:2306.02858, 2023. 13 A Implementation Details SceneRAG adopts a modular architecture inspired by the experimental setup of VideoRAG. ...

    2023 arXiv

  43. [54]

    Please maintain the required format in your response

    Ensure each scene contains detailed descriptions, dialogues, or events to form a coherent narrative unit. Please maintain the required format in your response. scene_segmentation_too_short Output Error Correction Request: Some scenes have been split with time ranges that are t...

  44. [55]

    Ensure each scene has a duration between 15 and 60 seconds

  45. [59]

    Please maintain the required format in your response

    Each scene should follow the previous one in a logical time sequence without gaps or overlaps. Please maintain the required format in your response. scene_segmentation_too_long Output Error Correction Request: Some scenes have been split with time ranges that are too long. Spl...

  46. [60]

    The duration of each scene should ideally not exceed 60 seconds

  47. [61]

    Verify that the scenes are divided correctly based on the content

  48. [62]

    Ensure each scene starts with a time mark

  49. [63]

    Ensure each scene contains detailed descriptions, dialogues, or events to form a coherent narrative unit

  50. [64]

    Is This the End of RAG? Anthropic’s NEW Prompt Caching

    Each scene should follow the previous one in a logical time sequence without gaps or overlaps. Please maintain the required format in your response. Figure 3: Error correction instructions for scene segmentation. B Limitations and Future Work SceneRAG has two main limitations....

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.