Pith. sign in

REVIEW 5 major objections 8 minor 57 references

LOVO: Efficient Complex Object Query in Large-Scale Video Datasets

T0 review · 5 major / 8 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Video queries that name arbitrary objects and attributes are answered by one-time embedding indexing plus a two-stage search, with reported search latency up to 85x lower than a query-dependent baseline.

desk verdict A promising two-stage video-object-query prototype whose headline numbers outrun its evaluation; the fast-search stage is never validated, and complex-query accuracy lands below 'near-optimal'. read the letter →

arxiv 2507.14301 v1 pith:UQDO4PIH submitted 2025-07-18 cs.IR cs.CVcs.DB

Yuxin Liu , Yuezhang Peng , Hefeng Zhou , Hongze Liu , Xinyu Lu , Jiong Lou , Chentao Wu , Wei Zhao
show 1 more author
Jie Li
This is my paper
classification cs.IRcs.CVcs.DB
keywords complexobjectquerylarge-scalevideodatasetsvectordatabaseapproximatenearestneighborsearchcross-modalrerankopen-vocabularydetectionproductquantizationinvertedmulti-index
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LOVO is a system for querying large video collections by describing the object one wants in natural language—'a red car side by side with another car, positioned in the center of the road'—rather than asking for a predefined class. Its central move is to convert every key frame into patch-level visual embeddings once, offline, and store those embeddings in an inverted vector index, so that later queries are answered by embedding similarity instead of by reprocessing the footage. The paper claims this design supports arbitrary, open-vocabulary object queries with accuracy comparable to the best available methods while cutting search latency by up to a factor of 85 relative to query-dependent systems, and keeping index-construction cost far lower than class-indexed alternatives. The two-stage query path—fast approximate nearest-neighbor search to recall candidate frames, then a cross-modality rerank that fuses text with visual features—is what lets the system have both low latency and fine-grained accuracy.

What carries the argument

The load-bearing structure is the two-stage query pipeline over pre-computed patch embeddings. Stage one uses an inverted multi-index with product quantization, in which the embedding space is split into subspaces, each subspace is quantized to cluster centroids, and a query is scored against centroids plus precomputed residual vectors to retrieve top-k candidate patches with cosine similarity as the score after L2 normalization. Stage two feeds the top-k frames and the query text through a cross-modality transformer whose image-to-text and text-to-image cross-attention layers refine the alignment and produce scores and bounding boxes. The embedding-augmented index replaces class labels, and the decoupled visual and text encoders make indexing independent of any specific query.

What would settle it

Run LOVO on a dataset where the correct answer requires a phrase that the fast-search step discards—for example, 'the red car to the left of the blue truck'—and check whether that correct object ever appears among the top-k candidates before reranking. If the discarded relational words are essential, the object is absent from the candidate pool and no amount of rerank can recover it, so final accuracy collapses.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that complex object queries in large-scale video reduce to a vector-similarity problem. A single offline pass with a pre-trained Vision Transformer embeds patches of sampled key frames and predicts a bounding box for each patch; those patch embeddings, together with box coordinates and frame identifiers, are stored in a product-quantized inverted multi-index inside a vector database. At query time a text encoder converts the user's sentence into an embedding, an approximate nearest-neighbor search over the index recalls the top-k candidate patches, and a cross-modality transformer reranks those candidates by fusing textual with visual features before returning frames and boxes. The claim is that this replaces the two dominant paradigms—class-limited pre-built indexes and per-query model-based scanning—with one open-vocabulary, one-time-indexing pipeline that is both accurate and fast.

Load-bearing premise

The system rests on the premise that off-the-shelf vision and text encoders are already aligned well enough that cosine similarity between a natural-language query embedding and patch-level visual embeddings ranks the right objects near the top of traffic and surveillance footage with no fine-tuning on that footage, and on a phrase-extraction step that silently discards query words.

Editorial extensions

If this is right

  • A single offline feature-extraction pass serves all future queries, so issuing many distinct queries against the same footage no longer multiplies video processing cost.
  • Queries can reference unseen object classes and detailed attributes because candidate selection works on embedding similarity rather than a closed label set.
  • Rerank cost stays roughly independent of dataset size because only the top-k candidates from fast search receive cross-modality scoring.
  • Index construction is cheaper than class-indexed alternatives because it avoids training or running per-query detection models, though it still pays the one-time cost of visual encoding.
  • Reported results show the highest average precision among compared systems on all evaluated queries, with search time up to 85x lower than FiGO and 9x lower than MIRIS.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the architecture transfers to other open-vocabulary object-query domains such as remote sensing, medical video, or retail footage without retraining, since only the key-frame extractor and the pre-trained encoders are domain-specific.
  • Editorial inference: the ablation's weak result on the relational query Q2.2 suggests that the phrase-extraction step, not the reranker, is the current bottleneck for spatial-language queries, and preserving relational words would be a testable improvement.
  • Editorial inference: comparing LOVO against a vision-language detector fine-tuned on the same traffic footage would test whether the reported accuracy gap comes from the indexing design or from the encoders' domain alignment.
  • Editorial inference: the same pipeline with a stronger open-vocabulary detector producing the patch embeddings should raise recall at the fast-search stage and further improve final accuracy, since rerank can only reorder what the index retrieves.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 8 minor

Summary. The paper proposes LOVO, a video object query system for large-scale datasets. Offline, it extracts key frames, encodes them with a pre-trained ViT into patch-level visual embeddings, predicts bounding boxes, and stores the embeddings in a Milvus vector database using product quantization and an inverted multi-index. At query time, a text query is encoded, a fast approximate nearest-neighbor search retrieves top-k candidates, and a cross-modality transformer reranks those candidates and predicts boxes. The authors claim near-optimal query accuracy, up to 85x lower search latency than baselines, and reduced index construction cost, evaluating on Cityscapes, Bellevue, QVHighlights, Beach, and ActivityNet-QA extensions.

Significance. If the claims were fully validated, LOVO would be a useful advance: it addresses a real gap in video analytics by supporting open-vocabulary, natural-language object queries with an index-based architecture, and its decoupled design, two-stage query strategy, ablation study, and scalability experiments are sensible. The paper is strongest where it documents the system architecture and the modularity of the pipeline. However, the evaluation as written does not yet establish the load-bearing claims of near-optimal accuracy and large latency gains; the lack of error bars, unspecified hyperparameters, no released code, and an unexplained query-phrase heuristic are serious reproducibility and evidence gaps.

major comments (5)
  1. [§VI-A and §VII-E, Table IV] The fast-search stage is load-bearing but is never validated on its own. Section VI-A describes a heuristic phrase-extraction step that discards relational and positional words such as "side by side", "in the center", and "walking on the road", yet these are precisely the components that make queries like Q2.2 complex. Because the cross-modal rerank in §VI-B only reorders candidates, any object missed by fast search is unrecoverable. Table IV shows that without rerank, Q2.2 has AveP 0.09, and even the full system reaches only 0.29; the paper reports no fast-stage recall@k for any query. Please specify the phrase-extraction algorithm, report recall@k and candidate-set quality for each query, and discuss why discarding relational words does not break complex queries.
  2. [§V-A and §VII-A] The similarity metric assumes that the text query vector q and the patch-level visual embedding c_jk live in a jointly aligned embedding space, but the paper does not identify the text encoder or state that q and c_jk share a common space. Section IV-B uses a pre-trained ViT-B/32 encoder for patches, and CLIP-style alignment is not established for patch tokens, nor is fine-tuning on the traffic/surveillance domains described. This unstated domain-alignment assumption is central to the whole pipeline. Please provide evidence for text-patch alignment, for example zero-shot retrieval accuracy of the fast stage, or give fine-tuning details.
  3. [§V-C, Algorithm 1 line 16] The candidate patch ID is determined by a majority vote over the P product-quantization subspace components. Because PQ decomposes a vector into components that may come from different database vectors, the majority-vote result could be a patch ID that is not an exact nearest neighbor of the query. The paper does not analyze this behavior or compare it against exact nearest-neighbor patch identification. The key hyperparameters—k, number of clusters M, subspace count P, number of top clusters A, rerank output n, and key-frame interval—are also not reported, so the accuracy/runtime trade-off is not reproducible.
  4. [§VII-B, §VII-C, Fig. 6, Fig. 8] The headline claims of "near-optimal query accuracy" and "up to 85x lower search latency" are not supported by the experiments as reported. No error bars or repeated trials are shown; Table IV gives single AveP values and Fig. 8 gives single acceleration factors. "Near-optimal" is never given an operational definition, and no oracle or upper-bound accuracy is provided. Given that Q2.2 achieves AveP 0.29, the accuracy claim is particularly difficult to assess. Please add variance estimates, define the reference for optimality, and publish the exact configurations used for all methods.
  5. [§VII-A, Datasets] The ground-truth bounding boxes were initialized with Bytetrack and then "manually checked and labeled" by the authors, and the queries were also authored by the authors. No inter-annotator agreement or verification protocol is described. Since evaluation uses author-created queries and labels, selection bias and annotation noise could inflate the reported numbers. Please release the labels, queries, and code, or provide an independent annotation protocol, to allow verification.
minor comments (8)
  1. [Abstract] The abstract contains formatting glitches in "compL ex O bject" and "V ideO"; these should be corrected.
  2. [§III] The text says "Cross-Modelity Transformer"; this should read "Cross-Modality Transformer".
  3. [§VII-B] The caption of Fig. 6 contains the typo "datatset"; please fix it.
  4. [References, [38]] The paper says the system is integrated with Milvus, but reference [38] is "Manu: a cloud native vector database management system", which is not the Milvus paper. Please cite the correct Milvus reference.
  5. [§V-B] In the product-quantization notation, the cluster index is written as m while M is the number of centroids, and the symbol cm,p is used both for the codebook element and for the quantized component; please disambiguate.
  6. [§VII-C] The sentence "LOVO achieves speeds at most 9 times faster than MIRIS and 85 times faster than FiGO" should read "up to 9 times faster... and up to 85 times faster", since the figure shows maximum factors rather than uniformly attained speedups.
  7. [Fig. 11] The x-axis labels in Fig. 11(a) and 11(b) ("Frame Numbers" and "Insert Entities") should have units and explicit dataset sizes for each plotted point.
  8. [§VII-F, Table VII] The ActivityNet-QA extension reports only LOVO's results, with no baseline comparison; this should be stated more prominently so readers do not interpret it as a comparative evaluation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: LOVO is an empirical systems composition of external, pre-trained components; no derivation reduces to its own inputs.

full rationale

LOVO is an empirical systems paper rather than a formal derivation, and its central claim is a two-stage pipeline (fast approximate nearest-neighbor search followed by cross-modality rerank) built from independently published components: ViT patch embeddings, product quantization and inverted multi-indexing [31,33], and GLIP/Grounding-DINO-style cross-attention fusion [36,37]. These components are pre-trained or externally specified, not fitted to the paper's evaluation labels, so the reported accuracy/latency trade-offs are not forced by construction. The paper's own ablation (Table IV) shows that removing rerank degrades complex-query performance substantially (Q2.2 AveP drops from 0.29 to 0.09), which is a substantive empirical finding about the pipeline, not a circular redefinition of success. The evaluation does use author-designed queries and manually checked ground-truth labels, which raises selection-bias and reproducibility concerns rather than circularity, because the system parameters are not optimized against those labels. The acknowledged limitation in Section IV-C that small patches may fragment multi-patch objects is a genuine stated failure mode and further indicates that the paper is not defining its conclusions into existence. No load-bearing self-citation or imported uniqueness theorem appears; the only co-author overlap in the reference list is background citation [1], which does not support any core claim. Overall, no circular step can be exhibited from the paper's own equations or citations.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities; its load-bearing assumptions are all about transferability of pretrained models, completeness of key-frame selection, correctness of manual labels, and the unspecified query-phrase heuristic. The free parameters listed above are system hyperparameters that are never published, so an independent replication cannot set them.

free parameters (5)
  • Fast-search candidate count k = not stated
    Algorithm 2 takes k as input; all latency and accuracy results depend on it, but no value or sensitivity analysis is given (Section VI-A, VII-E).
  • Rerank output frame number n = not stated
    The number of frames returned by Algorithm 2 affects AveP; the value used in experiments is not reported (Section VI-B, VII).
  • PQ codebook size M and subspace count P = not stated
    Product quantization parameters determine index quality and search speed; their values are omitted in Section V-B, and no search-quality tradeoff curve is shown.
  • Query phrase-extraction rule = not stated
    Section VI-A states that only relevant phrases are extracted and other words discarded, with no algorithm or examples beyond one sentence; this hand-crafted behavior directly influences fast-search results.
  • Key-frame selection thresholds and interval = not stated
    MVmed-based selection uses motion-vector change thresholds and scene-change criteria; the specific settings are not given, and the ablation shows a large storage effect (7976 MB to 2453 MB, Section VII-E).
assumptions (5)
  • domain assumption Pre-trained ViT-B/32 and text transformer embeddings are cross-modally aligned for the target domain
    Fast search relies on dot product between query text embedding and patch visual embedding ranking relevant objects (Section V-A, VI-A); no fine-tuning or domain adaptation is described.
  • domain assumption MVmed key-frame selection retains all objects relevant to arbitrary future queries
    If a target object appears only between selected key frames, it is never indexed; no recall analysis of key-frame selection against dense-frame processing is provided (Section IV-A).
  • domain assumption Ground-truth labels produced by Bytetrack plus manual checking are correct
    All AveP computations depend on these labels; no inter-annotator agreement, label-error analysis, or release of the labeled data is provided (Section VII-A).
  • domain assumption The cross-modality rerank model generalizes from pretraining to traffic and surveillance scenes
    The rerank transformer (feature enhancer and decoder) is described as a pretrained model applied as-is; no fine-tuning or domain-transfer evaluation is described (Section VI-B).
  • domain assumption ANN recall loss does not materially change ranking for the reported queries
    The ablation shows ANNS sometimes improves AveP, but no recall-versus-precision tradeoff curve or exact-search comparison per dataset is given (Section VII-E).

how reviews work

0 comments
Cite this review

Pith. "Pith review of LOVO: Efficient Complex Object Query in Large-Scale Video Datasets." pith.science (2026). https://pith.science/paper/UQDO4PIH

@misc{pith2026250714301,
  author       = {Pith},
  title        = {Pith review of: LOVO: Efficient Complex Object Query in Large-Scale Video Datasets},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UQDO4PIH}},
  note         = {Machine review of arXiv:2507.14301}
}
abstract

The widespread deployment of cameras has led to an exponential increase in video data, creating vast opportunities for applications such as traffic management and crime surveillance. However, querying specific objects from large-scale video datasets presents challenges, including (1) processing massive and continuously growing data volumes, (2) supporting complex query requirements, and (3) ensuring low-latency execution. Existing video analysis methods struggle with either limited adaptability to unseen object classes or suffer from high query latency. In this paper, we present LOVO, a novel system designed to efficiently handle comp$\underline{L}$ex $\underline{O}$bject queries in large-scale $\underline{V}$ide$\underline{O}$ datasets. Agnostic to user queries, LOVO performs one-time feature extraction using pre-trained visual encoders, generating compact visual embeddings for key frames to build an efficient index. These visual embeddings, along with associated bounding boxes, are organized in an inverted multi-index structure within a vector database, which supports queries for any objects. During the query phase, LOVO transforms object queries to query embeddings and conducts fast approximate nearest-neighbor searches on the visual embeddings. Finally, a cross-modal rerank is performed to refine the results by fusing visual features with detailed textual features. Evaluation on real-world video datasets demonstrates that LOVO outperforms existing methods in handling complex queries, with near-optimal query accuracy and up to 85x lower search latency, while significantly reducing index construction costs. This system redefines the state-of-the-art object query approaches in video analysis, setting a new benchmark for complex object queries with a novel, scalable, and efficient approach that excels in dynamic environments.

Figures

Figures reproduced from arXiv: 2507.14301 by the authors.

Figure 1
Figure 1. Existing methods vs. LOVO. and track specific targets within dynamic environments [12]– [19]. Object query forms the foundation of these video analysis applications, so it is urgent to design an efficient system for complex object queries within large-scale video datasets. Querying specific objects from massive video datasets is nontrivial, presenting several significant challenges. (1) Mas￾sive and continuously gro… view at source ↗
Figure 2
Figure 2. Efficiency and performance comparison of existing methods. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. LOVO workflow for complex object query in large-scale video database. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Workflow of video summary process. patch and serves as an initial estimate of the bounding box location. MLP refines this estimate by predicting an offset that adjusts the anchor box to fit the object within the patch. Additionally, the patch embeddings zjk are passed …
Figure 5
Figure 5. Figure 5: Workflow of the cross-modality rerank. This leverages cross-modality [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Quantitative evaluation by average precision of LOVO against baselines. [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Qualitative analysis of LOVO against baselines. We here present [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Runtime comparison of MIRIS, FiGO, and LOVO across different [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 10
Figure 10. Figure 10: Runtime comparison of total execution time and query search time. [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]
Figure 9
Figure 9. Figure 9: Time distribution for query execution in different datasets. [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 11
Figure 11. Figure 11: Comparison of execution times across different scales. [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

57 extracted references · 47 canonical work pages

  1. [1]

    Video Analytics with Zero-streaming Cameras

    M. Xu, T. Xu, Y . Liu, X. Liu, G. Huang, and F. X. Lin, “Supporting video queries on zero-streaming cameras,” CoRR, vol. abs/1904.12342, 2019

    work page Pith review arXiv 1904

  2. [2]

    Focus: Querying large video datasets with low latency and low cost,

    K. Hsieh, G. Ananthanarayanan, P. Bodik, S. Venkataraman, P. Bahl, M. Philipose, P. B. Gibbons, and O. Mutlu, “Focus: Querying large video datasets with low latency and low cost,” in 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18) . Carlsbad, CA: USENIX Association, Oct. 2018, pp. 269–286

    work page 2018

  3. [3]

    Dovedb: A declarative and low-latency video database,

    Z. Xiao, D. Zhang, Z. Li, S. Wu, K. Tan, and G. Chen, “Dovedb: A declarative and low-latency video database,” Proc. VLDB Endow. , vol. 16, no. 12, pp. 3906–3909, 2023

    work page 2023

  4. [4]

    Exsample: Efficient searches on video repositories through adaptive sampling,

    O. Moll, F. Bastani, S. Madden, M. Stonebraker, V . Gadepally, and T. Kraska, “Exsample: Efficient searches on video repositories through adaptive sampling,” in 2022 IEEE 38th International Conference on Data Engineering (ICDE) . Kuala Lumpur, Malaysia: IEEE, 2022, pp. 2956–2968

    work page 2022

  5. [5]

    Noscope: optimizing neural network queries over video at scale,

    D. Kang, J. Emmons, F. Abuzaid, P. Bailis, and M. Zaharia, “Noscope: optimizing neural network queries over video at scale,” Proc. VLDB Endow., vol. 10, no. 11, p. 1586–1597, Aug. 2017

    work page 2017

  6. [6]

    Towards large-scale, automated, accurate detection of CCTV camera objects using computer vision. Applications and implications for privacy, safety, and cybersecurity. (Preprint)

    H. Turtiainen, A. Costin, T. Lahtinen, L. Sintonen, and T. Hamalainen, “Towards large-scale, automated, accurate detection of cctv camera objects using computer vision. applications and implications for privacy, safety, and cybersecurity.(preprint),” arXiv preprint arXiv:2006.03870 , 2020

    work page Pith review arXiv 2006

  7. [7]

    Video-zilla: An indexing layer for large-scale video analytics,

    B. Hu, P. Guo, and W. Hu, “Video-zilla: An indexing layer for large-scale video analytics,” in Proceedings of the 2022 International Conference on Management of Data , ser. SIGMOD ’22. New York, NY , USA: Association for Computing Machinery, 2022, p. 1905–1919

    work page 2022

  8. [8]

    Moving vehicles detection based on adaptive motion histogram,

    W. Zhang, Q. J. Wu, and H. bing Yin, “Moving vehicles detection based on adaptive motion histogram,” Digital Signal Processing, vol. 20, no. 3, pp. 793–805, 2010

    work page 2010

Show all 57 references
  1. [9]

    Vision-based turning movement monitoring: count, speed & waiting time estimation,

    M. S. Shirazi and B. T. Morris, “Vision-based turning movement monitoring: count, speed & waiting time estimation,” IEEE Intelligent Transportation Systems Magazine, vol. 8, no. 1, pp. 23–34, 2016

    2016

  2. [10]

    Effectiveness and social costs of public area surveillance for crime prevention,

    B. C. Welsh, D. P. Farrington, and S. A. Taheri, “Effectiveness and social costs of public area surveillance for crime prevention,” Annual Review of Law and Social Science , vol. 11, no. V olume 11, 2015, pp. 111–130, 2015. [Online]. Available: https://www.annualreviews.org/ c...

    2015 doi

  3. [11]

    Emergency situation monitoring service using context motion tracking of chronic disease patients,

    S.-H. Kim and K. Chung, “Emergency situation monitoring service using context motion tracking of chronic disease patients,” Cluster Computing, vol. 18, pp. 747–759, 2015

    2015

  4. [12]

    OTIF: efficient tracker pre-processing over large video datasets,

    F. Bastani and S. Madden, “OTIF: efficient tracker pre-processing over large video datasets,” in SIGMOD ’22: International Conference on Management of Data, Philadelphia, PA, USA, June 12 - 17, 2022 , Z. G. Ives, A. Bonifati, and A. E. Abbadi, Eds. Philadelphia, PA, USA: ACM, ...

    2022

  5. [13]

    Blazeit: Optimizing declarative aggregation and limit queries for neural network-based video analytics,

    D. Kang, P. Bailis, and M. Zaharia, “Blazeit: Optimizing declarative aggregation and limit queries for neural network-based video analytics,” Proc. VLDB Endow., vol. 13, no. 4, pp. 533–546, 2019

    2019

  6. [14]

    Seiden: Revisiting query processing in video database systems,

    J. Bang, G. T. Kakkar, P. Chunduri, S. Mitra, and J. Arulraj, “Seiden: Revisiting query processing in video database systems,” Proc. VLDB Endow., vol. 16, no. 9, p. 2289–2301, May 2023

    2023

  7. [15]

    Eva: A symbolic approach to accelerating exploratory video analytics with materialized views,

    Z. Xu, G. T. Kakkar, J. Arulraj, and U. Ramachandran, “Eva: A symbolic approach to accelerating exploratory video analytics with materialized views,” in Proceedings of the 2022 International Conference on Man- agement of Data, ser. SIGMOD ’22. New York, NY , USA: Association f...

    2022

  8. [16]

    Top-k deep video analytics: A probabilistic approach,

    Z. Lai, C. Han, C. Liu, P. Zhang, E. Lo, and B. Kao, “Top-k deep video analytics: A probabilistic approach,” in Proceedings of the 2021 International Conference on Management of Data , ser. SIGMOD ’21. New York, NY , USA: Association for Computing Machinery, 2021, p. 1037–1050

    2021

  9. [17]

    Figo: Fine-grained query optimization in video analytics,

    J. Cao, K. Sarkar, R. Hadidi, J. Arulraj, and H. Kim, “Figo: Fine-grained query optimization in video analytics,” in SIGMOD ’22: International Conference on Management of Data, Philadelphia, PA, USA, June 12 - 17, 2022, Z. G. Ives, A. Bonifati, and A. E. Abbadi, Eds. Philadelp...

    2022

  10. [18]

    Physi- cal representation-based predicate optimization for a visual analytics database,

    M. R. Anderson, M. Cafarella, G. Ros, and T. F. Wenisch, “Physi- cal representation-based predicate optimization for a visual analytics database,” in 2019 IEEE 35th International Conference on Data Engi- neering (ICDE). Macau, China: IEEE, 2019, pp. 1466–1477

    2019

  11. [19]

    Predictive and near-optimal sampling for view materialization in video databases,

    Y . Xu, D. Zhang, S. Zhang, S. Wu, Z. Feng, and G. Chen, “Predictive and near-optimal sampling for view materialization in video databases,” Proc. ACM Manag. Data , vol. 2, no. 1, Mar. 2024

    2024

  12. [20]

    Microsoft coco: Common objects in context,

    T.-Y . Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, C. L. Zitnick, and P. Doll ´ar, “Microsoft coco: Common objects in context,” 2015. [Online]. Available: https://arxiv.org/abs/1405.0312

    2015 arXiv

  13. [21]

    Equi-vocal: Synthesizing queries for compositional video events from limited user interactions,

    E. Zhang, M. Daum, D. He, B. Haynes, R. Krishna, and M. Balazinska, “Equi-vocal: Synthesizing queries for compositional video events from limited user interactions,” Proc. VLDB Endow. , vol. 16, no. 11, p. 2714–2727, Jul. 2023

    2023

  14. [22]

    Ranked window query retrieval over video repositories,

    Y . Chen, X. Yu, and N. Koudas, “Ranked window query retrieval over video repositories,” in 2022 IEEE 38th International Conference on Data Engineering (ICDE). IEEE, 2022, pp. 2776–2791

    2022

  15. [23]

    Accelerating machine learning inference with probabilistic predicates,

    Y . Lu, A. Chowdhery, S. Kandula, and S. Chaudhuri, “Accelerating machine learning inference with probabilistic predicates,” inProceedings of the 2018 International Conference on Management of Data , ser. SIGMOD ’18. New York, NY , USA: Association for Computing Machinery, 201...

    2018

  16. [24]

    Miris: Fast object track queries in video,

    F. Bastani, S. He, A. Balasingam, K. Gopalakrishnan, M. Alizadeh, H. Balakrishnan, M. Cafarella, T. Kraska, and S. Madden, “Miris: Fast object track queries in video,” inProceedings of the 2020 ACM SIGMOD International Conference on Management of Data , ser. SIGMOD ’20. New Yo...

    2020

  17. [25]

    Physi- cal representation-based predicate optimization for a visual analytics database,

    M. R. Anderson, M. Cafarella, G. Ros, and T. F. Wenisch, “Physi- cal representation-based predicate optimization for a visual analytics database,” in 2019 IEEE 35th International Conference on Data Engi- neering (ICDE). IEEE, 2019, pp. 1466–1477

    2019

  18. [26]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection,

    S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, Q. Jiang, C. Li, J. Yang, H. Su, J. Zhu, and L. Zhang, “Grounding dino: Marrying dino with grounded pre-training for open-set object detection,” 2024

    2024

  19. [27]

    Traffic video dataset,

    City of Bellevue, “Traffic video dataset,” https://github.com/ City-of-Bellevue/TrafficVideoDataset, 2017, accessed: 2024-08-14

    2017

  20. [28]

    Mvmed: Fast multi-object tracking in the compressed domain,

    L. Bommes, X. Lin, and J. Zhou, “Mvmed: Fast multi-object tracking in the compressed domain,” in 2020 15th IEEE Conference on Industrial Electronics and Applications (ICIEA) , 2020, pp. 1419–1424

    2020

  21. [29]

    Simple open-vocabulary object detection with vision transformers,

    M. Minderer, A. Gritsenko, A. Stone, M. Neumann, D. Weissenborn, A. Dosovitskiy, A. Mahendran, A. Arnab, M. Dehghani, Z. Shen, X. Wang, X. Zhai, T. Kipf, and N. Houlsby, “Simple open-vocabulary object detection with vision transformers,” ECCV, 2022

    2022

  22. [30]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929 , 2020

    2010 arXiv

  23. [31]

    Product quantization for nearest neighbor search,

    H. Jegou, M. Douze, and C. Schmid, “Product quantization for nearest neighbor search,” IEEE transactions on pattern analysis and machine intelligence, vol. 33, no. 1, pp. 117–128, 2010

    2010

  24. [32]

    Least squares quantization in pcm,

    S. Lloyd, “Least squares quantization in pcm,” IEEE Transactions on Information Theory, vol. 28, no. 2, pp. 129–137, 1982

    1982

  25. [33]

    The inverted multi-index,

    A. Babenko and V . Lempitsky, “The inverted multi-index,” in 2012 IEEE Conference on Computer Vision and Pattern Recognition , 2012, pp. 3069–3076

    2012

  26. [34]

    Efficient large-scale approximate nearest neighbor search on the gpu,

    P. Wieschollek, O. Wang, A. Sorkine-Hornung, and H. P. A. Lensch, “Efficient large-scale approximate nearest neighbor search on the gpu,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016

    2016

  27. [35]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805 , 2018

    2018 arXiv

  28. [36]

    Grounded language-image pre- training,

    L. H. Li, P. Zhang, H. Zhang, J. Yang, C. Li, Y . Zhong, L. Wang, L. Yuan, L. Zhang, J.-N. Hwang et al., “Grounded language-image pre- training,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 10 965–10 975

    2022

  29. [37]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection,

    S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, Q. Jiang, C. Li, J. Yang, H. Su, J. Zhu, and L. Zhang, “Grounding dino: Marrying dino with grounded pre-training for open-set object detection,” 2024. [Online]. Available: https://arxiv.org/abs/2303.05499

    2024 arXiv

  30. [38]

    Manu: a cloud native vector database management system,

    R. Guo, X. Luan, L. Xiang, X. Yan, X. Yi, J. Luo, Q. Cheng, W. Xu, J. Luo, F. Liu, Z. Cao, Y . Qiao, T. Wang, B. Tang, and C. Xie, “Manu: a cloud native vector database management system,” Proceedings of the VLDB Endowment, vol. 15, no. 12, pp. 3548–3561, 2022

    2022

  31. [39]

    Umt: Unified multi-modal transformers for joint video moment retrieval and highlight detection,

    Y . Liu, S. Li, Y . Wu, C.-W. Chen, Y . Shan, and X. Qie, “Umt: Unified multi-modal transformers for joint video moment retrieval and highlight detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . New Orleans, LA, USA: IEEE/CVF, 20...

    2022

  32. [40]

    The cityscapes dataset for semantic urban scene understanding,

    M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benen- son, U. Franke, S. Roth, and B. Schiele, “The cityscapes dataset for semantic urban scene understanding,” in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2016

    2016

  33. [41]

    Qvhighlights: Detecting moments and highlights in videos via natural language queries,

    J. Lei, T. L. Berg, and M. Bansal, “Qvhighlights: Detecting moments and highlights in videos via natural language queries,” 2021

    2021

  34. [42]

    Bytetrack: Multi-object tracking by associating every detection box,

    Y . Zhang, P. Sun, Y . Jiang, D. Yu, F. Weng, Z. Yuan, P. Luo, W. Liu, and X. Wang, “Bytetrack: Multi-object tracking by associating every detection box,” in European conference on computer vision . Springer, 2022, pp. 1–21

    2022

  35. [43]

    Sketchql: Video moment querying with a visual query interface,

    R. Wu, P. Chunduri, A. Payani, X. Chu, J. Arulraj, and K. Rong, “Sketchql: Video moment querying with a visual query interface,” Proceedings of the ACM on Management of Data , vol. 2, no. 4, pp. 1–27, 2024

    2024

  36. [44]

    Zelda: Video analytics using vision-language models,

    F. Romero, C. Winston, J. Hauswald, M. Zaharia, and C. Kozyrakis, “Zelda: Video analytics using vision-language models,” arXiv preprint arXiv:2305.03785, 2023

    2023 arXiv

  37. [45]

    V ocalexplore: Pay-as-you-go video data exploration and model building,

    M. Daum, E. Zhang, D. He, S. Mussmann, B. Haynes, R. Krishna, and M. Balazinska, “V ocalexplore: Pay-as-you-go video data exploration and model building,” Proc. VLDB Endow. , vol. 16, no. 13, p. 4188–4201, Sep. 2023

    2023

  38. [46]

    Panoptic video scene graph genera- tion,

    J. Yang, W. Peng, X. Li, Z. Guo, L. Chen, B. Li, Z. Ma, K. Zhou, W. Zhang, C. C. Loy, and Z. Liu, “Panoptic video scene graph genera- tion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver, BC, Canada: IEEE/CVF, June 2023,...

    2023

  39. [47]

    Learning transferable visual models from natural language supervi- sion,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning transferable visual models from natural language supervi- sion,” in Proceedings of the 38th International Conference on Machine...

    2021

  40. [48]

    One token to seg them all: Language instructed reasoning segmentation in videos,

    Z. Bai, T. He, H. Mei, P. Wang, Z. Gao, J. Chen, Z. Zhang, and M. Z. Shou, “One token to seg them all: Language instructed reasoning segmentation in videos,” Advances in Neural Information Processing Systems, vol. 37, pp. 6833–6859, 2025

    2025

  41. [49]

    Seesaw: Interactive ad-hoc search over image databases,

    O. Moll, M. Favela, S. Madden, V . Gadepally, and M. Cafarella, “Seesaw: Interactive ad-hoc search over image databases,” Proc. ACM Manag. Data, vol. 1, no. 4, Dec. 2023

    2023

  42. [50]

    Object detection in 20 years: A survey,

    Z. Zou, K. Chen, Z. Shi, Y . Guo, and J. Ye, “Object detection in 20 years: A survey,” Proceedings of the IEEE, vol. 111, no. 3, pp. 257–276, 2023

    2023

  43. [51]

    Activitynet- qa: A dataset for understanding complex web videos via question answering,

    Z. Yu, D. Xu, J. Yu, T. Yu, Z. Zhao, Y . Zhuang, and D. Tao, “Activitynet- qa: A dataset for understanding complex web videos via question answering,” in AAAI, 2019, pp. 9127–9134

    2019

  44. [52]

    Viva: An end-to-end system for interactive video analytics,

    D. Kang, F. Romero, P. Bailis, C. Kozyrakis, and M. Zaharia, “Viva: An end-to-end system for interactive video analytics,” in Proceedings of the 12th Conference on Innovative Data Systems Research (CIDR) . Chaminade, CA, USA: www.cidrdb.org, January 2022

    2022

  45. [53]

    Optimizing video queries with declarative clues,

    D. Chao, Y . Chen, N. Koudas, and X. Yu, “Optimizing video queries with declarative clues,” Proc. VLDB Endow., vol. 17, no. 11, p. 3256–3268, Aug. 2024

    2024

  46. [54]

    Tvm: A tile-based video management framework,

    T. Zhong, Z. Zhang, G. Lu, Y . Yuan, Y .-P. Wang, and G. Wang, “Tvm: A tile-based video management framework,” Proceedings of the VLDB Endowment, vol. 17, no. 4, pp. 671–684, 2023

    2023

  47. [55]

    Surveil- lance video querying with a human-in-the-loop,

    M. Stonebraker, B. Bhargava, M. Cafarella, Z. Collins, J. McClellan, A. Sipser, T. Sun, A. Nesen, K. Solaiman, G. Mani et al. , “Surveil- lance video querying with a human-in-the-loop,” in Proceedings of the Workshop on Human-In-the-Loop Data Analytics with SIGMOD , 2020

    2020

  48. [56]

    Feature centric multi-modal information retrieval in open world environment (femmir),

    K. Solaiman and B. Bhargava, “Feature centric multi-modal information retrieval in open world environment (femmir),” Authorea Preprints , 2023

    2023

  49. [57]

    Clip4clip: An empirical study of clip for end to end video clip retrieval and captioning,

    H. Luo, L. Ji, M. Zhong, Y . Chen, W. Lei, N. Duan, and T. Li, “Clip4clip: An empirical study of clip for end to end video clip retrieval and captioning,” Neurocomputing, vol. 508, pp. 293–304, 2022

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.