REVIEW 3 major objections 5 minor 24 references
diveXplore at the Video Browser Showdown 2024
T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read diveXplore 2024's redesign makes it a competitive video search system for the Video Browser Showdown 2024, its authors argue.
desk verdict A clear, incremental VBS system description whose 'very competitive' claim needs evidence; fine for peer review if the claim is softened. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is a FAISS index of OpenCLIP ViT-H/14 embeddings computed for every keyframe, which lets the system answer both free-text and visual similarity queries from one representation. Around that sits a Node.js query server, the named central component, that can forward parts of a query to different index instances and to the metadata database, then merge results by filtering, ranking, or temporal constraints, so heterogeneous searches compose in a single interactive loop. Completing the machinery are the interface optimizations — hover-to-inspect neighboring keyframes, keyboard-only navigation, video summaries and shot lists, direct similarity search, and the Xplore view over clusters of similar videos — that speed up the human's inspection and exploration of candidate results.
What would settle it
The published VBS 2024 evaluation results would settle the 'very competitive' claim: if diveXplore finishes at or below the median on known-item search for the Marine or Surgery datasets, the claim is contradicted. Independently, take a sample of text queries from each dataset, retrieve the top-10 keyframes from the FAISS index by embedding similarity, and measure how often a relevant keyframe appears; low recall on Marine or Surgery would show the load-bearing embedding index does not transfer to those domains.
Extended reading notes
Core claim
On the authors' own terms, diveXplore 2024 is a competitive interactive video retrieval system because every keyframe of every video is embedded once with OpenCLIP ViT-H/14 trained on LAION-2B and stored in a FAISS index; the query server can split a user's request into parallel embedding searches and metadata filters and merge the results, including temporal merges that require ordered matches in nearby shots; and the interface lets users inspect shot lists, play videos, view summaries, jump across keyframes from the same video, and explore clusters of similar videos. The same pipeline is applied to all three VBS datasets (V3C, Marine, Surgery), with shot detection for V3C and Marine, ORB-based keyframe extraction for surgery, and additional analysis of objects, scenes, events, OCR text, and surgical actions. The authors assert that this combination of an improved OpenCLIP embedding index, a flexible distributed query server, and an optimized interface makes the system 'very competitive' at VBS 2024, and they single out the surgical content analysis as essential because laparoscopic footage is highly redundant.
Load-bearing premise
The entire search quality rests on the untested assumption that OpenCLIP embeddings trained on general web images rank the relevant keyframes near the top for the V3C, Marine, and Surgery datasets alike; if that transfer fails, the parallel query server and interface cannot rescue retrieval.
Editorial extensions
If this is right
- One embedding index serves both free-text and visual similarity search across all three datasets, so the same interaction works for visual known-item search and textual known-item search.
- Parallel query execution lets users combine embedding search with metadata filters, such as object detections or scene concepts, in a single step instead of running separate searches.
- Temporal merging over multiple free-text queries supports searches of the form 'first X, then Y nearby', which single-shot ranking cannot express.
- The surgical-specific analysis is presented as crucial for handling the highly redundant laparoscopic content, suggesting that generic embeddings alone are not enough for that dataset.
- Fast context inspection and cluster exploration are promoted as necessary to keep both expert and novice users quick enough for VBS's timed sessions.
Reading between the lines
- The paper's competitiveness claim is not tested inside the paper; the actual VBS 2024 standings, which appear in the competition proceedings, would be the direct evidence for whether the redesign paid off.
- OpenCLIP was trained on general web image-text pairs, so its transfer to diving and laparoscopic video is far from guaranteed; measuring per-domain retrieval precision, or fine-tuning on domain keyframes, would tell whether the embedding index is the weak link.
- The query-server pattern — splitting a query between an embedding index and a metadata store and merging the results — is a generic multimodal retrieval middleware design that could be reused for photo archives, medical image search, or other media collections without the video-specific UI.
- The temporal merging capability suggests a useful experiment: compare success rates on event-level known-item queries ('find the clip where X happens, then Y') against single-query text search on V3C, which would quantify the value of the parallel query server beyond speed.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper describes diveXplore 2024, an interactive video retrieval system for the Video Browser Showdown (VBS) 2024. The system integrates OpenCLIP ViT-H/14 embeddings for free-text and similarity search, a redesigned query server that can distribute and merge parallel queries, and an optimized user interface with new exploration views. The authors state that these changes make the system 'very competitive.' The paper presents the system architecture, the analysis components summarized in Table 2, and several UI features, but it contains no experimental evaluation and no VBS 2024 results.
Significance. If the system performs as claimed, this paper provides a useful system description for the VBS community, and the combination of OpenCLIP embeddings with a parallel query server is a coherent architectural choice. The manuscript is clearly structured and honest about the components used. However, its significance is limited by the absence of any evaluation: the central claim of competitiveness is not backed by retrieval experiments, VBS scores, or ablations. As a system description it is informative, but as a demonstration of effectiveness it currently falls short.
major comments (3)
- [3. Conclusion] The claim that the improved index, distributed query server, and UI optimizations 'make our system very competitive' is not supported by any experimental evidence in the manuscript. No VBS 2024 results, offline retrieval evaluations, or ablations are reported. Since competitiveness is the paper's central assertion, this is a load-bearing omission. The authors should either add evaluation data (e.g., VBS 2024 scores or an offline known-item search experiment on the three datasets) or substantially temper the claim to a purely descriptive system paper.
- [2.1, Table 2] For the Marine dataset, OpenCLIP embeddings are the only content-based retrieval signal; Table 2 shows no other analysis applied to M. The paper provides no validation that a model trained on LAION-2B transfers to underwater diving videos. For the Surgery dataset, the authors themselves state in Section 3 that 'specific content analysis' is 'crucial' because of highly redundant content, implicitly acknowledging that generic embeddings are insufficient, yet no experiment demonstrates that OpenCLIP ranks relevant surgical or marine keyframes above irrelevant ones. A targeted retrieval evaluation on these two domain-specific collections is needed to substantiate the competitiveness claim.
- [2.2] The query server is described as enabling 'flexible and efficient' parallel queries, but no performance measurements, scaling tests, or comparisons with the previous diveXplore version are provided. Without timing data or a relative comparison, the efficiency claim remains unsupported. Adding at least a simple latency benchmark or a qualitative comparison of query merging strategies would help.
minor comments (5)
- [2.1, Table 2] The table cites 'OpenCLIP ViT-H/14 [12]', but reference [12] is Radford et al.'s CLIP paper, not the OpenCLIP implementation; the correct citation for OpenCLIP is reference [4].
- [1] The phrase 'Ad-hoch search' should be 'ad-hoc search', and 'datset' should be 'dataset'.
- [2.1] The sentence 'Figure 1 shows the that architecture' contains an extra word; 'the that' should be 'the'.
- [Abstract] The word 'redesigend' should be 'redesigned'.
- [1] In the example query description, 'baloon' should be 'balloon'.
Circularity Check
No circularity: the paper is a system description with no derivation that reduces to its inputs.
full rationale
The paper is a short interactive-video-retrieval system description. It contains no equations, no fitted parameters, and no formal derivation chain whose conclusion is assumed in its premises. The components (OpenCLIP ViT-H/14, FAISS, TransNetv2, YOLOv7, etc.) are cited external models and datasets, not results derived within the paper. The paper's central claim that diveXplore is "very competitive" (Section 3) is an assertion based on integrating those components, not a quantity computed from them, so it cannot be circular. The self-citations ([11], [16], [17], [18]) point to prior toolchain components such as endoscopic keyframe extraction and action recognition; these are supporting references for specific analysis modules, not used to justify the paper's own competitiveness claim. The skeptical concern that OpenCLIP's transfer to Marine and Surgery domains is unvalidated is a legitimate correctness/evidence gap, but it is not a circularity issue. Therefore the appropriate finding is no significant circularity.
Assumptions & free parameters
free parameters (1)
- Marine keyframe sampling interval =
1 second
assumptions (3)
- domain assumption OpenCLIP ViT-H/14 (LAION-2B) embeddings capture semantic similarity relevant to VBS retrieval queries.
- domain assumption Shot boundary detection via TransNetv2 and keyframe extraction produce representative frames for retrieval.
- domain assumption Off-the-shelf detectors for objects, scenes, text, and actions provide correct metadata for combined queries.
Cite this review
Pith. "Pith review of diveXplore at the Video Browser Showdown 2024." pith.science (2026). https://pith.science/paper/4YFXKJPM
@misc{pith2026250820560,
author = {Pith},
title = {Pith review of: diveXplore at the Video Browser Showdown 2024},
year = {2026},
howpublished = {\url{https://pith.science/paper/4YFXKJPM}},
note = {Machine review of arXiv:2508.20560}
}
read the original abstract
According to our experience from VBS2023 and the feedback from the IVR4B special session at CBMI2023, we have largely revised the diveXplore system for VBS2024. It now integrates OpenCLIP trained on the LAION-2B dataset for image/text embeddings that are used for free-text and visual similarity search, a query server that is able to distribute different queries and merge the results, a user interface optimized for fast browsing, as well as an exploration view for large clusters of similar videos (e.g., weddings, paraglider events, snow and ice scenery, etc.).
Figures
Reference graph
Works this paper leans on
-
[1]
In: Proceedings of the IEEE/CVF international conference on computer vision
Baek, J., Kim, G., Lee, J., Park, S., Han, D., Yun, S., Oh, S.J., Lee, H.: What is wrong with scene text recognition model comparisons? dataset and model analysis. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 4715–4723 (2019)
work page 2019
-
[2]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Baek, Y., Lee, B., Han, D., Yun, S., Lee, H.: Character region awareness for text detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9365–9374 (2019)
work page 2019
-
[3]
In: Proc
Berns, F., Rossetto, L., Schoeffmann, K., Beecks, C., Awad, G.: V3c1 dataset: An evaluation of content characteristics. In: Proc. of the 2019 on Intl. Conf. on Multimedia Retrieval. pp. 334–338. ACM (2019)
2019
-
[4]
https://doi.org/10.48550/ARXIV.2212.07143, https://arxiv.org/abs/2212.07143
Cherti, M., Beaumont, R., Wightman, R., Wortsman, M., Ilharco, G., Gordon, C., Schuhmann, C., Schmidt, L., Jitsev, J.: Reproducible scaling laws for contrastive language-image learning (2022). https://doi.org/10.48550/ARXIV.2212.07143, https://arxiv.org/abs/2212.07143
-
[5]
IEEE Transactions on Big Data7(3), 535–547 (2019)
Johnson, J., Douze, M., Jégou, H.: Billion-scale similarity search with GPUs. IEEE Transactions on Big Data7(3), 535–547 (2019)
2019
-
[6]
Kletz, S., Schoeffmann, K., Leibetseder, A., Benois-Pineau, J., Husslein, H.: In- strument recognition in laparoscopy for technical skill assessment. In: MultiMedia Modeling: 26th International Conference, MMM 2020, Daejeon, South Korea, Jan- uary 5–8, 2020, Proceedings, Part II 26. pp. 589–600. Springer (2020)
work page 2020
-
[7]
In: International Conference on Multimedia Modeling
Leibetseder, A., Schoeffmann, K.: divexplore 6.0: Itec’s interactive video explo- ration system at vbs 2022. In: International Conference on Multimedia Modeling. pp. 569–574. Springer (2022)
work page 2022
-
[8]
Lokoč, J., Andreadis, S., Bailer, W., Duane, A., Gurrin, C., Ma, Z., Messina, N., Nguyen, T.N., Peška, L., Rossetto, L., et al.: Interactive video retrieval in the age of effective joint embedding deep models: lessons from the 11th vbs. Multimedia Systems pp. 1–24 (2023)
work page 2023
Show all 24 references
-
[9]
ACM Trans
Lokoč, J., Kovalčík, G., Münzer, B., Schöffmann, K., Bailer, W., Gasser, R., Vrochidis, S., Nguyen, P.A., Rujikietgumjorn, S., Barthel, K.U.: Interactive search or sequential browsing? a detailed analysis of the video browser showdown 2018. ACM Trans. Multimedia Comput. Commun...
2018 doi
-
[10]
IEEE Trans- actions on Pattern Analysis and Machine Intelligence 42(2), 502–508 (2020), https://doi.org/10.1109/TPAMI.2019.2901464
Monfort, M., Vondrick, C., Oliva, A., Andonian, A., Zhou, B., Ramakrish- nan, K., Bargal, S.A., Yan, T., Brown, L.M., Fan, Q., Gutfreund, D.: Mo- ments in time dataset: One million videos for event understanding. IEEE Trans- actions on Pattern Analysis and Machine Intelligence...
2020
-
[11]
In: 2023 IEEE 36th International Symposium on Computer-Based Medical Systems (CBMS)
Nasirihaghighi, S., Ghamsarian, N., Stefanics, D., Schoeffmann, K., Husslein, H.: Action recognition in video recordings from gynecologic laparoscopy. In: 2023 IEEE 36th International Symposium on Computer-Based Medical Systems (CBMS). pp. 29–34. IEEE (2023)
2023
-
[12]
In: International Conference on Machine Learning
Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International Conference on Machine Learning. pp. 8748–8763. PMLR (2021)
2021
-
[13]
Schoeffmann et al
Rossetto, L., Gasser, R., Lokoč, J., Bailer, W., Schoeffmann, K., Muen- zer, B., Souček, T., Nguyen, P.A., Bolettieri, P., Leibetseder, A., Vrochidis, 8 K. Schoeffmann et al. S.: Interactive video retrieval in the age of deep learning – detailed eval- uation of vbs 2019. IEEE ...
2021
-
[14]
In: MultiMedia Modeling: 27th International Conference, MMM 2021, Prague, Czech Republic, June 22–24, 2021, Proceedings, Part II 27
Rossetto,L.,Gasser,R.,Sauter,L.,Bernstein,A.,Schuldt,H.:Asystemforinterac- tive multimedia retrieval evaluations. In: MultiMedia Modeling: 27th International Conference, MMM 2021, Prague, Czech Republic, June 22–24, 2021, Proceedings, Part II 27. pp. 385–390. Springer (2021)
2021
-
[15]
arXiv preprint arXiv:2105.01475 (2021)
Rossetto, L., Schoeffmann, K., Bernstein, A.: Insights on the v3c2 dataset. arXiv preprint arXiv:2105.01475 (2021)
2021 arXiv
-
[16]
In: Proceedings of the 20th International Conference on Content-based Multimedia Indexing (CBMI 2023)
Schoeffmann, K.: divexb: An interactive video retrieval system for beginners. In: Proceedings of the 20th International Conference on Content-based Multimedia Indexing (CBMI 2023). pp. 1–6. IEEE (2023)
2023
-
[17]
Multimedia Tools and Applications74, 11187–11206 (2015)
Schoeffmann, K., Del Fabro, M., Szkaliczki, T., Böszörmenyi, L., Keckstein, J.: Keyframe extraction in endoscopic video. Multimedia Tools and Applications74, 11187–11206 (2015)
2015
-
[18]
In: International Conference on Multimedia Modeling
Schoeffmann, K., Stefanics, D., Leibetseder, A.: divexplore at the video browser showdown 2023. In: International Conference on Multimedia Modeling. pp. 684–
2023
-
[19]
arXiv preprint arXiv:2111.02114 (2021)
Schuhmann, C., Vencu, R., Beaumont, R., Kaczmarczyk, R., Mullis, C., Katta, A., Coombes,T.,Jitsev,J.,Komatsuzaki,A.:Laion-400m:Opendatasetofclip-filtered 400 million image-text pairs. arXiv preprint arXiv:2111.02114 (2021)
2021 arXiv
-
[20]
arXiv preprint arXiv:2008.04838 (2020)
Souček, T., Lokoč, J.: Transnet v2: an effective deep network architecture for fast shot transition detection. arXiv preprint arXiv:2008.04838 (2020)
2020 arXiv
-
[21]
In: International conference on machine learning
Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International conference on machine learning. pp. 6105–6114. PMLR (2019)
2019
-
[22]
In: International Conference on Multimedia Modeling
Truong, Q.T., Vu, T.A., Ha, T.S., Lokoč, J., Wong, Y.H., Joneja, A., Yeung, S.K.: Marine video kit: a new marine video dataset for content-based analysis and retrieval. In: International Conference on Multimedia Modeling. pp. 539–550. Springer (2023)
2023
-
[23]
arXiv preprint arXiv:2207.02696 (2022)
Wang, C.Y., Bochkovskiy, A., Liao, H.Y.M.: Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv preprint arXiv:2207.02696 (2022)
2022 arXiv
-
[24]
IEEE Transactions on Pattern Analysis and Machine Intelligence 40(6), 1452–1464 (2018), https://doi.org/10.1109/TPAMI.2017.2723009
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., Torralba, A.: Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 40(6), 1452–1464 (2018), https://doi.org/10.1109/TPAMI.2017.2723009
2018
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.