Pith. sign in

REVIEW 4 major objections 6 minor 59 references

LAVA: Language Driven Scalable and Versatile Traffic Video Analytics

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Lava claims a language-driven query system can retrieve vehicles, count them, and rank frames across hour-long traffic videos using natural-language predicates, with gains of 14% in F1, 0.39 lower aggregation error, 86% top-k precision…

desk verdict A solid, honest systems paper: real benchmark, sensible pipeline, but the key pseudo-labeling module needs a noise analysis before the headline numbers can be trusted. read the letter →

arxiv 2507.19821 v2 pith:UJHC3SJG submitted 2025-07-26 cs.CV cs.MM

Yanrui Yu , Tianfei Zhou , Jiaxin Sun , Lianpeng Qiao , Lizhong Ding , Ye Yuan , Guoren Wang This is my paper
classification cs.CVcs.MM
keywords videoanalyticsnaturallanguagequeryopen-vocabularydetectionprompttuningmulti-armedbandittrajectoryextractiontrafficThompsonsampling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that video analytics need not be locked into predefined object classes or rigid SQL-style query patterns. It introduces Lava, a system that accepts natural-language queries—such as 'trams with red and yellow coloration' or 'black pickup truck'—and answers selection, top-k, and aggregation queries on hour-long traffic footage. The authors argue that by combining multi-armed bandit segmentation, per-video prompt-tuned CLIP filtering, and motion-pattern trajectory association, Lava makes open-vocabulary retrieval accurate and fast enough for real camera networks. If correct, this would let operators interrogate surveillance archives in everyday language rather than writing code or manually scanning frames.

What carries the argument

The load-bearing mechanism is a per-video adaptation loop. For each video, Lava uniformly samples frames, runs YOLO-World to produce confidence-scored detections, and treats detections above a confidence threshold (0.85) as positive and below it as negative pseudo-labels; these labels prompt-tune a CLIP model with learnable visual and textual prompt vectors, turning generic open-vocabulary matching into video-specific semantic filtering. A multi-armed bandit (Thompson sampling with Gamma-distributed reward estimates) decides which video segments to scrutinize, and fuzzy C-means clustering of early-frame trajectories provides motion patterns that tie detections across time into long trajectories.

What would settle it

Manually label a random subset of the pseudo-training boxes used for prompt tuning and compute the false-positive rate among the high-confidence positives; a substantial rate (e.g., above 10%) would indicate that the reported gains come from detector bias rather than video-specific semantic adaptation.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that a three-stage pipeline can make language-driven video analytics scalable: a Thompson-sampling bandit localizes relevant segments with a fixed sampling budget; a video-specific CLIP model, prompt-tuned on pseudo-labels mined from YOLO-World detections, filters false detections and matches fine-grained attributes; and motion-pattern assignment, initialized from the first five minutes of each scene, links detections into complete trajectories. The authors report that this combination outperforms adapted baselines on their benchmark—average selection F1 of 0.64 versus 0.50 for the best open-vocabulary detector baseline, lower aggregation error, top-k precision of 0.86, and average query time of 149 seconds per one-hour video.

Load-bearing premise

The per-video CLIP tuning takes YOLO-World detections with confidence above 0.85 as true positives and lower-confidence detections as true negatives on a sparse uniform frame sample, without any measured noise rate.

Editorial extensions

If this is right

  • Operators could query surveillance archives in natural language without predefined taxonomies, since the system accepts open-vocabulary predicates.
  • Hour-long videos become processable in about 149 seconds on average, making interactive analytics feasible on camera-network scale.
  • One pipeline covers selection, top-k, and aggregation queries, replacing multiple specialized systems.
  • The new benchmark of 18 predicates over six traffic datasets provides a reusable evaluation standard for language-driven video analytics.
  • Per-video prompt tuning of a small vision-language model offers a recipe for adapting open-world detectors to fixed-viewpoint scenes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The gains attributed to video-specific CLIP filtering probably depend on YOLO-World's confidence calibration; re-running the ablation with a different open-vocabulary detector would show how much of the effect is detector-specific.
  • The first-five-minutes motion-pattern assumption may break on scenes where traffic behavior shifts over the hour (rush hour versus night); an extension would reinitialize patterns periodically.
  • The same three-stage architecture could transfer to other fixed-camera domains like retail or industrial monitoring, but the trajectory stage would need scene-specific re-initialization.
  • A significance analysis over the 18 queries would be needed to tell whether the 0.14 F1 gain and 0.39 MAPE drop are stable or driven by a few favorable predicates.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents LAVA, a language-driven video analytics system for traffic footage that supports selection, top-k, and aggregation queries expressed in natural language. The system combines Thompson-sampling-based segment localization, video-specific prompt tuning of CLIP over YOLO-World detections, and motion-pattern-based trajectory extraction. The authors also build a new benchmark of 18 natural-language predicates over six existing traffic video datasets with human-verified labels. On this benchmark they report an average selection F1 of 0.64, a 0.14 improvement over the strongest baseline, a reduction in aggregation MAPE of 0.39, top-k precision of 0.86, and an average query time of 149 seconds, about 9.6x faster than YOLO-World. The paper includes ablations showing that each of the three components contributes to the reported results.

Significance. If the reported results are reliable, LAVA is a useful step toward practical open-vocabulary video analytics: it addresses a real deployment gap by combining open-vocabulary detection with sampling and trajectory association, and it scales to hour-long videos at a fraction of the cost of dense VLM processing. The authors also make their code and dataset available, and the human labeling protocol with majority agreement is a strength. The main significance is conditional on the validity of the self-created benchmark and on the robustness of the video-specific prompt-tuning module, both of which currently rest on assumptions that are not quantified.

major comments (4)
  1. [3.4.1, 5.1, Table 4] The video-specific training sample mining in Section 3.4.1 uses YOLO-World confidence scores as pseudo-labels: detections above threshold 0.85 become positives and low-confidence detections become negatives, but the paper gives no estimate of pseudo-label precision or recall. This is load-bearing because the ablation in Table 4 shows that removing video-specific detection drops Caldot1 selection F1 from 0.63 to 0.38 and increases Caldot1 aggregation MAPE from 0.58 to 1.39. A high-confidence YOLO-World detection can be correct for the coarse category but wrong for the fine-grained predicate (e.g., color or roof rack), and a genuinely matching object that is blurred or occluded may be converted into a false negative. The authors should quantify pseudo-label noise against the human ground truth, report sensitivity to the 0.85 threshold, and show that the tuned CLIP is not merely reinforcing YOLO-World's own confusions.
  2. [4.2, Step 2] The ground-truth construction protocol selects only three candidate frames per tracked vehicle for human labeling. The paper does not state whether all other frames are treated as negative for selection and top-k queries. If they are, then the reported F1 and precision figures are computed against an incomplete ground truth: frames containing the query object but not among the three selected candidates would be counted as false positives, systematically penalizing recall. Conversely, if only the candidate frames are scored, the task is much easier than full-video retrieval. The authors should clarify the labeling of non-candidate frames and either provide complete frame-level annotations for at least a subset of videos or adjust the evaluation protocol and report both settings.
  3. [4.2, Step 1; 5.1] The benchmark predicates are constructed with the help of CLIP: GPT-4o generates captions and then CLIP is used as a classifier to assign captions to bounding boxes. Since LAVA prompt-tunes a CLIP backbone and CLIP is also used as a baseline, the benchmark may be inadvertently favorable to CLIP-based methods. The paper should analyze how much the final predicate set depends on the CLIP selection step, for example by comparing with predicates selected without CLIP or by evaluating on a held-out set of predicates not generated through CLIP.
  4. [5.1, Tables 2-4] All reported accuracy numbers are single-run point estimates with no variance, confidence intervals, or significance tests. Thompson sampling and prompt tuning are stochastic, and the claimed 14% F1 improvement and 0.39 MAPE reduction are averages over only six videos and eighteen predicates per condition. The authors should run each configuration multiple times (or at least report seed sensitivity) and provide standard deviations and, where feasible, paired significance tests, so that the reported gains can be distinguished from random variation.
minor comments (6)
  1. [Abstract vs. Introduction] The abstract reports a 14% F1 improvement while the introduction reports a 15% improvement; please reconcile these numbers.
  2. [Figure 4] Figure 4 shows only box plots for aggregation MAPE; the reader cannot see per-predicate values. A supplementary table with the per-predicate MAPE for each method would make the 0.39 improvement reproducible and checkable.
  3. [Theorem 1, Eq. (1)-(2)] The unbiasedness claim in Theorem 1 is stated without the sampling assumptions needed to make it meaningful. If frames are sampled uniformly at random without replacement from each segment, the equality holds in expectation; the paper should state this assumption explicitly.
  4. [3.4.1 vs. 5.1] Section 3.4.1 says uniformly sampled frames are drawn 'across the entire video duration,' while Section 5.1 says 'training frames (one every 100).' Given the train/test split described in Section 5.2, please clarify whether prompt tuning is performed only on the training subset to avoid any appearance of test leakage.
  5. [5.2] There is a typo in the paragraph following Figure 5: 'igure 5' should be 'Figure 5.'
  6. [Eq. (3)] The stabilization constants alpha_0 and beta_0 are introduced but never assigned values; please state them in the implementation details.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central Lava claims rest on human-annotated ground truth, and no prediction reduces to a fitted input or to a self-citation chain.

full rationale

Lava's core evaluation is self-contained: selection, aggregation, and top-k results are measured against human-annotated labels (Section 4.2, Step 3: each frame is evaluated by three annotators and accepted only when at least two agree), so the ground truth is not generated by any Lava component or by CLIP. The benchmark predicates are initially constructed with GPT-4o and CLIP, but this only defines the query language; it does not define the correctness labels. The video-specific prompt tuning in Section 3.4.1 uses YOLO-World confidence thresholds as pseudo-labels for CLIP; this is a self-training step with potential label-noise bias, but the tuned CLIP is evaluated on held-out test frames, so its outputs are not forced to equal its training labels by any equation in the paper. No fitted parameter is renamed as a prediction: the Thompson-Sampling estimates in Section 3.3 are used only to select segments, not to report query answers. The paper's self-citations (e.g., [16] for prompt-tuning data efficiency) are supporting and not load-bearing for the main claims. The absence of a pseudo-label noise estimate is a robustness/correctness limitation, not a circular derivation. Therefore the circularity score is 0.

Assumptions & free parameters 10 free parameters · 5 assumptions · 0 invented entities

The central claim rests on empirical engineering components. The free parameters are mostly hand-chosen thresholds, sampling budgets, and clustering settings. The axioms are domain assumptions about pseudo-label reliability, temporal stationarity, and benchmark validity. No new physical or world-level entities are introduced; the video-specific prompts and motion-pattern clusters are internal software representations, not independently evidenced entities.

free parameters (10)
  • pseudo-label confidence threshold = 0.85
    Used in Section 3.4.1 and Section 5.1 to turn YOLO-World detections into positive and negative prompt-tuning samples; chosen by hand, not derived.
  • Thompson sampling rounds Imax = 2000
    Section 5.1; total sampled frames per query, balancing speed and accuracy; chosen by hand.
  • segment count = 500
    Section 5.1; the video is divided into 500 equal-length segments treated as bandit arms; chosen by hand.
  • trajectory refinement frames n = 5
    Section 3.5.3 and Section 5.1; number of frames sampled per detection to confirm a trajectory; chosen by hand.
  • ReID similarity threshold tau_r
    Section 3.5.3; threshold for accepting a detection as the same object; mentioned but not reported.
  • candidate motion pattern count k
    Section 3.5.2; number of nearest trajectories considered for assignment; value not reported.
  • FCM cluster count and fuzzifier
    Section 3.5.1; fuzzy c-means parameters for motion pattern clustering; values not reported.
  • motion pattern initialization window = first 5 minutes
    Section 3.5.1; initial trajectories from the first five minutes are used to establish motion patterns; chosen by hand.
  • prompt length for visual and textual prompts
    Section 3.4.2; number of learnable prompt vectors per branch is not specified.
  • sampling stabilization constants alpha_0 and beta_0
    Section 3.3.2; small constants added to Gamma shape and rate to avoid zero values; values not reported.
assumptions (5)
  • ad hoc to paper Segment rewards are modeled as Gamma-distributed with mean equal to x_obs divided by n_obs, as in Equation 3.
    No derivation is given that Gamma is the correct distribution for segment relevance; it is used to enable Thompson sampling.
  • domain assumption YOLO-World detections with confidence above 0.85 on uniformly sampled training frames are reliable positives, and low-confidence detections are reliable negatives, for prompt tuning.
    Section 3.4.1 and Section 5.1; if pseudo-labels are noisy, the prompt-tuned CLIP inherits the noise, and no pseudo-label accuracy estimate is provided.
  • domain assumption Motion patterns initialized from the first five minutes generalize to the entire one-hour video.
    Section 3.5.1; the fixed-viewpoint assumption is used to make early trajectories representative, but temporal stationarity is not validated per dataset.
  • domain assumption CLIP-based assignment of GPT-4o captions to existing bounding boxes produces semantically meaningful query predicates for the benchmark.
    Section 4.2 Step 1; predicates are selected using CLIP scores and frequency, which may bias the benchmark toward attributes that CLIP can detect.
  • domain assumption BoT-SORT tracks used for candidate frame selection cover all query-relevant objects.
    Section 4.2 Step 2; if tracking misses objects, the human-annotated candidate frames omit them, which would bias selection and top-k ground truth.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LAVA: Language Driven Scalable and Versatile Traffic Video Analytics." pith.science (2026). https://pith.science/paper/UJHC3SJG

@misc{pith2026250719821,
  author       = {Pith},
  title        = {Pith review of: LAVA: Language Driven Scalable and Versatile Traffic Video Analytics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UJHC3SJG}},
  note         = {Machine review of arXiv:2507.19821}
}
abstract

In modern urban environments, camera networks generate massive amounts of operational footage -- reaching petabytes each day -- making scalable video analytics essential for efficient processing. Many existing approaches adopt an SQL-based paradigm for querying such large-scale video databases; however, this constrains queries to rigid patterns with predefined semantic categories, significantly limiting analytical flexibility. In this work, we explore a language-driven video analytics paradigm aimed at enabling flexible and efficient querying of high-volume video data driven by natural language. Particularly, we build \textsc{Lava}, a system that accepts natural language queries and retrieves traffic targets across multiple levels of granularity and arbitrary categories. \textsc{Lava} comprises three main components: 1) a multi-armed bandit-based efficient sampling method for video segment-level localization; 2) a video-specific open-world detection module for object-level retrieval; and 3) a long-term object trajectory extraction scheme for temporal object association, yielding complete trajectories for object-of-interests. To support comprehensive evaluation, we further develop a novel benchmark by providing diverse, semantically rich natural language predicates and fine-grained annotations for multiple videos. Experiments on this benchmark demonstrate that \textsc{Lava} improves $F_1$-scores for selection queries by $\mathbf{14\%}$, reduces MPAE for aggregation queries by $\mathbf{0.39}$, and achieves top-$k$ precision of $\mathbf{86\%}$, while processing videos $ \mathbf{9.6\times} $ faster than the most accurate baseline. Our code and dataset are available at https://github.com/yuyanrui/LAVA.

Figures

Figures reproduced from arXiv: 2507.19821 by the authors.

Figure 1
Figure 1. (a) Traditional scalable video analytics methods (e.g., No￾Scope [20], LEAP [47], BlazeIt [19], OTIF [3]) are SQL-based, closed￾world systems, limited to querying predefined object classes (e.g., car, bus). (b) Zelda [38] is a recent effort towards open-world video ana￾lytics using language queries but is constrained to image-level query. (c) Our system, Lava, tackles all these limitations, and supports ver￾satile v… view at source ↗
Figure 2
Figure 2. Illustration of Lava framework. Lava extracts object trajectories from high-volume videos specified in a language query in a three-stage process: (i) Relevant Video Segment Localization (§3.3) first equally partitions the video and initializes a prior for Thompson Sampling, which iteratively refines segment probabilities to identify those most likely to contain relevant objects; (ii) Video-Specific Open-World Object… view at source ↗
Figure 3
Figure 3. Illustration of (a) video-specific training sample mining and (b) prompt tuning. See §3.4 for details. detections exhibiting low confidence are treated as negative sam￾ples, thus marking them as irrelevant or false detections relative to the specified predicate. This deliberate and selective filtering substantially enhances the reliability and quality of our training data by minimizing ambiguities and label noise, e… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Aggregation Query Performance over 18 Predicates. Box plots show the MAPE distribution: boxes mark the IQR with the median as the horizontal line, and circles are outliers. Lower and tighter distributions indicate better performance. Lava achieves the lowest median MAP…
Figure 5
Figure 5. Figure 5: Stage-wise runtime for three queries (White semi-truck, Suv with roof rack, Suv without roof rack) in the Jackson Town dataset with selectivities of 0.03, 0.37, and 0.89, representing low, medium, and high selectivity scenarios, respectively. The Segment Localization s…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

59 extracted references · 44 canonical work pages

  1. [1]

    Nir Aharon, Roy Orfaig, and Ben-Zion Bobrovsky. 2022. BoT-SORT: Robust associations multi-pedestrian tracking. arXiv preprint arXiv:2206.14651 (2022)

    arXiv 2022

  2. [2]

    Favyen Bastani, Songtao He, Arjun Balasingam, Karthik Gopalakrishnan, Mo- hammad Alizadeh, Hari Balakrishnan, Michael Cafarella, Tim Kraska, and Sam Madden. 2020. Miris: Fast object track queries in video. In SIGMOD

    work page 2020

  3. [3]

    Favyen Bastani and Samuel Madden. 2022. OTIF: Efficient tracker pre-processing over large video datasets. In SIGMOD

    work page 2022

  4. [4]

    James C Bezdek, Robert Ehrlich, and William Full. 1984. FCM: The fuzzy c-means clustering algorithm. Computers & geosciences (1984)

    work page 1984

  5. [5]

    Jiashen Cao, Karan Sarkar, Ramyad Hadidi, Joy Arulraj, and Hyesoon Kim. 2022. Figo: Fine-grained query optimization in video analytics. In SIGMOD

    work page 2022

  6. [6]

    Qianwen Cao and Heyan Huang. 2023. Video Visual Relation Detection With Contextual Knowledge Embedding. TKDE 35, 12 (2023), 13083–13095

    work page 2023

  7. [7]

    Ziliang Chen, Xin Huang, Quanlong Guan, Liang Lin, and Weiqi Luo. 2023. A retrospect to multi-prompt learning across vision and language. In ICCV

    work page 2023

  8. [8]

    Tianheng Cheng, Lin Song, Yixiao Ge, Wenyu Liu, Xinggang Wang, and Ying Shan. 2024. Yolo-world: Real-time open-vocabulary object detection. In CVPR

    work page 2024

Show all 59 references
  1. [9]

    Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, and Lidong Bing. 2024. VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs. arXiv preprint arXiv:2406.07476 (2024)

    2024 arXiv

  2. [10]

    Eulrang Cho, Jooyeon Kim, and Hyunwoo J Kim. 2023. Distribution-aware prompt tuning for vision-language models. In ICCV

    2023

  3. [11]

    Jianfeng Dong, Xun Wang, Leimin Zhang, Chaoxi Xu, Gang Yang, and Xirong Li

  4. [12]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv prepri...

    2020 arXiv

  5. [13]

    Sainyam Galhotra, Yue Gong, and Raul Castro Fernandez. 2023. Metam: Goal- oriented data discovery. In ICDE

    2023

  6. [14]

    Lingxiao He, Xingyu Liao, Wu Liu, Xinchen Liu, Peng Cheng, and Tao Mei

  7. [15]

    Josefa Z Hernández, Sascha Ossowski, and Ana Garcıa-Serrano. 2002. Multiagent architectures for intelligent traffic management systems. Transportation Research Part C: Emerging Technologies (2002)

    2002

  8. [16]

    Shiyu Hou, Tianfei Zhou, Shuai Zhang, Ye Yuan, and Guoren Wang. 2025. Prompt Tuning In a Compact Attribute Space. In AAAI

    2025

  9. [17]

    Kevin Hsieh, Ganesh Ananthanarayanan, Peter Bodik, Shivaram Venkataraman, Paramvir Bahl, Matthai Philipose, Phillip B Gibbons, and Onur Mutlu. 2018. Focus: Querying large video datasets with low latency and low cost. In OSDI

    2018

  10. [18]

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276 (2024)

    2024 arXiv

  11. [19]

    Daniel Kang, Peter Bailis, and Matei Zaharia. 2018. BlazeIt: Optimizing Declara- tive Aggregation and Limit Queries for Neural Network-Based Video Analytics. VLDB 13, 4 (2018)

    2018

  12. [20]

    Daniel Kang, John Emmons, Firas Abuzaid, Peter Bailis, and Matei Zaharia. [n. d.]. NoScope: Optimizing Neural Network Queries over Video at Scale.VLDB ([n. d.])

  13. [21]

    Daniel Kang, John Guibas, Peter D Bailis, Tatsunori Hashimoto, and Matei Za- haria. 2022. TASTI: semantic indexes for machine learning-based queries over unstructured data. In SIGMOD

    2022

  14. [22]

    Daniel Kang, Francisco Romero, Peter D Bailis, Christos Kozyrakis, and Matei Zaharia. 2022. VIVA: An End-to-End System for Interactive Video Analytics.. In CIDR

    2022

  15. [23]

    Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. 2023. Maple: Multi-modal prompt learning. In CVPR

    2023

  16. [24]

    Muhammad Uzair Khattak, Syed Talal Wasim, Muzammal Naseer, Salman Khan, Ming-Hsuan Yang, and Fahad Shahbaz Khan. 2023. Self-regulating prompts: Foundational model adaptation without forgetting. In CVPR

    2023

  17. [25]

    Nick Koudas, Raymond Li, and Ioannis Xarchakos. 2020. Video monitoring queries. TKDE 34, 10 (2020), 5023–5036

    2020

  18. [26]

    Ziliang Lai, Chenxia Han, Chris Liu, Pengfei Zhang, Eric Lo, and Ben Kao. 2021. Top-K Deep Video Analytics: A Probabilistic Approach. In SIGMOD

    2021

  19. [27]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In ICML

    2023

  20. [28]

    KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. 2023. Videochat: Chat-centric video understanding. arXiv preprint arXiv:2305.06355 (2023)

    2023 arXiv

  21. [29]

    Meiyu Liang, Junping Du, Linghui Li, Zhe Xue, Xiaoxiao Wang, Feifei Kou, and Xu Wang. 2020. Video super-resolution reconstruction based on deep learning and spatio-temporal feature self-similarity. TKDE 34, 9 (2020), 4538–4553

    2020

  22. [30]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruc- tion tuning. In NeurIPS

    2023

  23. [31]

    Jiajun Liu, Yibing Wang, Hanghang Ma, Xiaoping Wu, Xiaoqi Ma, Xiaoming Wei, Jianbin Jiao, Enhua Wu, and Jie Hu. 2024. Kangaroo: A powerful video-language model supporting long-context video input. arXiv preprint arXiv:2408.15542 (2024)

    2024 arXiv

  24. [32]

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. [n. d.]. Grounding dino: Marry- ing dino with grounded pre-training for open-set object detection. In ECCV

  25. [33]

    Yongdong Luo, Xiawu Zheng, Xiao Yang, Guilin Li, Haojia Lin, Jinfa Huang, Jiayi Ji, Fei Chao, Jiebo Luo, and Rongrong Ji. 2024. Video-RAG: Visually- aligned Retrieval-Augmented Long Video Comprehension. arXiv preprint arXiv:2411.13093 (2024)

    2024

  26. [34]

    Oscar Moll, Favyen Bastani, Sam Madden, Mike Stonebraker, Vijay Gadepally, and Tim Kraska. 2022. Exsample: Efficient searches on video repositories through adaptive sampling. In ICDE

    2022

  27. [35]

    Oscar Moll, Manuel Favela, Samuel Madden, Vijay Gadepally, and Michael Ca- farella. 2023. SeeSaw: interactive ad-hoc search over image databases. InSIGMOD

    2023

  28. [36]

    Ian Osband, Zheng Wen, Seyed Mohammad Asghari, Vikranth Dwaracherla, Morteza Ibrahimi, Xiuyuan Lu, and Benjamin Van Roy. 2023. Approximate thompson sampling via epistemic neural networks. In Uncertainty in Artificial Intelligence

    2023

  29. [37]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In ICML

    2021

  30. [38]

    Francisco Romero, Caleb Winston, Johann Hauswald, Matei Zaharia, and Christos Kozyrakis. 2023. Zelda: Video analytics using vision-language models. arXiv preprint arXiv:2305.03785 (2023)

    2023 arXiv

  31. [39]

    Daniel J Russo, Benjamin Van Roy, Abbas Kazerouni, Ian Osband, Zheng Wen, et al. 2018. A tutorial on thompson sampling.Foundations and Trends® in Machine Learning (2018)

    2018

  32. [40]

    Wenbo Shao, Yanchao Xu, Jun Li, Chen Lv, Weida Wang, and Hong Wang. 2023. How does traffic environment quantitatively affect the autonomous driving prediction? IEEE Trans. Intell. Transp. Syst. 24, 10 (2023), 11238–11253

    2023

  33. [41]

    Enxin Song, Wenhao Chai, Guanhong Wang, Yucheng Zhang, Haoyang Zhou, Feiyang Wu, Haozhe Chi, Xun Guo, Tian Ye, Yanting Zhang, et al. 2024. Moviechat: From dense token to sparse memory for long video understanding. In CVPR

    2024

  34. [42]

    Xi Tang, Jihao Qiu, Lingxi Xie, Yunjie Tian, Jianbin Jiao, and Qixiang Ye. 2025. Adaptive Keyframe Sampling for Long Video Understanding. In CVPR

    2025

  35. [43]

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530 (2024)

    2024 arXiv

  36. [44]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. 2024. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191 (2024)

    2024 arXiv

  37. [45]

    Qi Wang, Yanrui Yu, Ye Yuan, Rui Mao, and Tianfei Zhou. 2025. VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuning. arXiv preprint arXiv:2505.12434 (2025)

    2025

  38. [46]

    Ziyang Wang, Shoubin Yu, Elias Stengel-Eskin, Jaehong Yoon, Feng Cheng, Gedas Bertasius, and Mohit Bansal. 2025. VideoTree: Adaptive Tree-based Video Repre- sentation for LLM Reasoning on Long Videos. In CVPR

    2025

  39. [47]

    Yanchao Xu, Dongxiang Zhang, Shuhao Zhang, Sai Wu, Zexu Feng, and Gang Chen. 2024. Predictive and Near-Optimal Sampling for View Materialization in Video Databases. In SIGMOD

    2024

  40. [48]

    Zhuangdi Xu, Gaurav Tarlok Kakkar, Joy Arulraj, and Umakishore Ramachandran

  41. [49]

    Ye Yuan, Bo Tang, Tianfei Zhou, Zhiwei Zhang, and Jianbin Qin. 2024. nsdb: Architecting the next generation database by integrating neural and symbolic systems. VLDB (2024)

    2024

  42. [50]

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. Sig- moid loss for language image pre-training. In ICCV. 11975–11986

    2023

  43. [51]

    Dongxiang Zhang, Teng Ma, Junnan Hu, Yijun Bei, Kian-Lee Tan, and Gang Chen

  44. [52]

    Peiyuan Zhang, Kaichen Zhang, Bo Li, Guangtao Zeng, Jingkang Yang, Yuanhan Zhang, Ziyue Wang, Haoran Tan, Chunyuan Li, and Ziwei Liu. 2024. Long context transfer from language to vision. arXiv preprint arXiv:2406.16852 (2024)

    2024 arXiv

  45. [53]

    Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chun- yuan Li. 2024. Video instruction tuning with synthetic data. arXiv preprint arXiv:2410.02713 (2024)

    2024 arXiv

  46. [54]

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. 2022. Learning to Prompt for Vision-Language Models. IJCV 130, 9 (2022), 2337–2348

    2022

  47. [55]

    Zheqing Zhu and Benjamin Van Roy. 2023. Scalable neural contextual bandit for recommender systems. In Proc. ACM Int. Conf. Inf. Knowl. Manag

    2023

  48. [2019]

    TKDE 33, 5 (2019), 1946–1959

    Feature re-learning with data augmentation for video relevance prediction. TKDE 33, 5 (2019), 1946–1959

    2019

  49. [2020]

    arXiv preprint arXiv:2006.02631 (2020)

    FastReID: A Pytorch Toolbox for General Instance Re-identification. arXiv preprint arXiv:2006.02631 (2020)

    2020 arXiv

  50. [2022]

    In SIGMOD

    EVA: A symbolic approach to accelerating exploratory video analytics with materialized views. In SIGMOD

  51. [2023]

    VLDB 17, 3 (2023), 604–616

    Co-Movement Pattern Mining from Videos. VLDB 17, 3 (2023), 604–616

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.