Pith. sign in

REVIEW 4 major objections 5 minor 52 references

HERO-VQL: Hierarchical, Egocentric and Robust Visual Query Localization

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read HERO-VQL claims egocentric object search is improved by top-down hierarchical attention plus ego-motion-aware consistency training, reporting 0.38 tAP25 and 0.28 stAP25 on VQ2D, 7 and 6 points over the next best method.

desk verdict HERO-VQL is a solid incremental contribution with a plausible SOTA claim, but the exact margins over baselines are unverified due to unspecified inference protocols and no error bars. read the letter →

arxiv 2509.00385 v1 pith:OXFJBXBM submitted 2025-08-30 cs.CV

Joohyun Chang , Soyeon Hong , Hyogun Lee , Seong Jong Ha , Dongho Lee , Seong Tae Kim , Jinwoo Choi This is my paper
classification cs.CV
keywords visualquerylocalizationegocentricvideotop-downattentionconsistencytrainingdataaugmentationVQ2Dtransformerdecoderprincipalcomponentanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Visual query localization (VQL) is the task of taking one image of an object and finding that object's last appearance in a long first-person video. The paper claims that the main obstacle—abrupt viewpoint changes and partial occlusion—can be overcome by two additions to a transformer-based matcher. First, Top-down Attention Guidance makes the decoder attend globally before locally: the query's class token is used to bias attention toward object-relevant regions, then principal-component score maps of the query features steer each attention head toward a different object part. Second, Egocentric Augmentation based Consistency Training replaces the query image with another annotated instance of the same object, reorders video frames to exaggerate object motion, and penalizes disagreements between predictions on the original and reordered clips. On the VQ2D benchmark the full method reports a temporal average precision (tAP25) of 0.38 and spatio-temporal AP (stAP25) of 0.28, which the paper states is 7 and 6 points above the second-best method. If those numbers hold, the recipe offers a practical route to making augmented-reality glasses and home robots locate 'the object I am looking at' reliably in noisy first-person footage.

What carries the argument

Top-down Attention Guidance (TAG) is the central mechanism: a two-level attention bias injected into a transformer spatial decoder. The high-level guide αhigh is formed from the query class token's attention against penultimate video patch tokens, and is added to self-attention so the decoder first finds the broad object region; the mid-level guide αmid is a set of R principal-component score maps of the centered query features, with one map added to each cross-attention head, so each head specializes in a distinct object part. The companion training mechanism is Egocentric Augmentation based Consistency Training (EgoACT), whose QueryAug and MotionAug distort query and video in task-specific

What would settle it

Run all baseline methods through the same inference protocol—median filter kernel size 5 and a 0.7× peak-score threshold—on VQ2D with the same pretrained features and backbone; if the 7-point tAP25 gap over VQLoC shrinks below the ablation differences, the central claim that TAG and EgoACT drive the gain is not supported.

Watch

Extended reading notes

Core claim

The central claim is that robustness in egocentric visual query localization comes from matching in two stages and from training deliberately with egocentric distortions. TAG implements the two stages by adding a high-level attention guide—computed from the query class token's similarity to video patch tokens in the encoder's penultimate layer—to the decoder's self-attention, and a mid-level guide—one principal-component score map of the centered query features per attention head—to cross-attention. The paper argues this mirrors the human top-down perceptual process of recognizing an object's category before inspecting its parts. EgoACT trains the model to tolerate appearance and motion vari

Load-bearing premise

The performance comparison assumes the published baseline numbers were obtained under the same inference procedure (median filter size and score threshold) as HERO-VQL; if the baselines were scored with different inference settings, the reported margins could change.

Editorial extensions

If this is right

  • On VQ2D, the method reports the highest tAP25 (0.38) and stAP25 (0.28), so if the comparison is fair, egocentric VQL no longer requires detector-plus-tracker pipelines to reach top accuracy.
  • Removing TAG or EgoACT drops tAP25 by roughly 4 and 3 points respectively, which the paper reads as evidence that both global-to-local attention and motion-augmented consistency are necessary, not optional.
  • The QueryAug ablation shows random replacement of the query with a ground-truth instance beats both most-similar and least-similar choices, implying diversity of training queries, not difficulty or similarity, drives the gain.
  • The MotionAug ablation shows displacement-maximizing frame reordering beats random reordering and no reordering, so deliberately exaggerating ego-motion during training is beneficial.
  • The backbone ablation with CLIP suggests the recipe transfers beyond DINOv2 features, although absolute accuracy is lower.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's claims: MotionAug's max-displacement reordering could be turned into a curriculum—starting from mild reordering and increasing displacement during training—which the current fixed greedy order does not explore.
  • Beyond the paper's claims: The PCA-based part decomposition is an explicit linear probe of DINOv2 features; one could test whether learned part prototypes or clustering replace PCA without losing the mid-level guide's benefit.
  • Beyond the paper's claims: CT loss compares whole-clip predictions; a frame-wise or attention-level consistency loss might be stronger and would isolate whether stability comes from output smoothing or representation invariance.
  • Beyond the paper's claims: If random QueryAug works better than least/most similar instances, then instance diversity is the driver; a direct test would be to sample by feature distance in an online, annotation-free way.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes HERO-VQL, a method for egocentric visual query localization (VQL) on the VQ2D benchmark. Two main components are introduced: Top-down Attention Guidance (TAG), which uses the encoder's class token to produce a high-level attention guide and PCA-based score maps as a mid-level attention guide for the spatial decoder, and Egocentric Augmentation based Consistency Training (EgoACT), composed of QueryAug (replacing the query with a random ground-truth instance of the same object) and MotionAug (reordering frames by bounding-box displacement) plus a consistency loss. Experiments on VQ2D report state-of-the-art tAP25/stAP25 of 0.38/0.28 on validation and 0.37/0.28 on test, with ablations attributing gains to TAG, EgoACT, CT loss, and an attention-score repair heuristic described in the supplementary material.

Significance. If the reported comparison is properly controlled, the gains over the previous best method VQLoC (7 points tAP25, 6 points stAP25) would be a meaningful advance for VQ2D. The method is practical: it uses a frozen DINOv2 encoder, adds no extra modality, and the two proposed mechanisms are clearly motivated by the egocentric challenges. The paper is commendable for running ablations over component choices, backbone transfer, and design alternatives, and for including qualitative attention visualizations. The training-time use of ground-truth instances in QueryAug is a legitimate augmentation strategy and does not constitute a circular evaluation. However, the central SOTA claim is currently not fully supported because the baseline evaluation protocol is unspecified, and a key component of the method appears only in the supplementary material.

major comments (4)
  1. [Table 1, §B.2] The SOTA comparison does not state whether the baseline rows (SiamRCNN, NFM, CocoFormer, VQLoC) are taken from original publications or re-implemented under identical conditions. Inference details in §B.2 — median filter with kernel size 5 and threshold 0.7×peak score — are specified only for HERO-VQL. Since tAP25/stAP25 are computed over score-ranked temporal segments, this post-processing can change the selected segment and materially affect scores. If baselines were not run with the same protocol, the claimed 7/6-point margins could be inflated. Please state the source of each baseline number or re-run all baselines under the same inference protocol.
  2. [§3.2.1, Eq. (1); §C.3; Table 4] The high-level attention guide in the main paper, Eq. (1), is defined on the raw penultimate video features. The supplementary (§C.3) adds an 'attention score repair' step that replaces high-norm tokens with the mean of neighboring tokens before computing the guide. Table 4 shows this repair contributes 3.2 points tAP25 and 2.4 points stAP25. The main method description is therefore incomplete: a load-bearing heuristic is hidden in the supplementary and is absent from the equation and Figure 2. Please integrate the repair into the main derivation or state it explicitly in Eq. (1) and the main text.
  3. [§3.4, Eq. (3) and LCT definition] The task loss is defined as Ltask(C, Ĉ), where C is ground truth and Ĉ is a prediction. The consistency loss is then written as LCT = Ltask(Ĉ, Ĉ′), which reuses a ground-truth-vs-prediction loss for two prediction sets. This is not well-defined as written: are boxes/confidences from the original clip treated as pseudo-labels for the augmented clip, or is a different symmetric loss intended? Please specify the exact functional form of LCT. This matters because CT loss is one of the paper's two main contributions.
  4. [§4.4, Tables 2(b) and 2(e)] No variance estimates or multiple-seed results are reported. Several ablation differences are small — CT loss improves tAP25 by 1.5 points; removing mid-level guidance costs 3.5 points; removing high-level guidance costs 1.9 points. Without error bars or repeated runs, these differences are not statistically assessable, especially given the known instability of end-to-end training on egocentric video. Please report mean±std across at least three seeds for the central ablations, or clearly state that the results are single-run and interpret the small differences accordingly.
minor comments (5)
  1. [Table 1] The text claims 'significantly outperforming baselines,' but HERO-VQL's validation recovery is lower than VQLoC (44.9 vs. 47.1), and test recovery is essentially tied (45.3 vs. 45.1). The SOTA claim should be restricted to tAP25/stAP25, not recovery.
  2. [Eq. (2), Table 3] The PCA scaling parameter τ in Eq. (2) is not given a value anywhere, and the number of principal components R (which must match the number of attention heads) is not stated. Please add these values to Table 3 or the text.
  3. [§C.3, Table 3] The criterion for identifying 'high-norm' tokens in the attention score repair is not defined. Please specify the selection rule (e.g., norm threshold or top-k fraction) and the neighborhood size used for the repair.
  4. [Eq. (8)–(11), Table 3] The TAG loss weights λtoken and λmap are not listed in Table 3. Without these values, the total loss in Eq. (3) is not reproducible.
  5. [Table 2] Values are reported as percentages in Table 2 (e.g., 37.5) but as decimals in Table 1 (0.38). Please use a consistent notation to avoid confusion. Also, Table 2(b) lacks a 'neither high-level nor mid-level' condition; please clarify whether EgoACT is fixed in that ablation.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: SOTA claim rests on external VQ2D benchmark; no self-citation or definitional reduction.

full rationale

The derivation chain is self-contained. HERO-VQL's components (TAG, EgoACT, CT loss) are defined from encoder features and training objectives and evaluated on the public VQ2D benchmark against published methods. No equation reduces a prediction to a fitted parameter; QueryAug uses ground-truth instances only as a training-time augmentation, and the inference post-processing described in Sec. B.2 (median filter with kernel 5, threshold 0.7 times peak score) is applied to model outputs, not fitted to the reported metrics. The paper contains no load-bearing self-citations; references [16,24,40,41] are external prior work. The only concern is whether the baseline rows in Table 1 used identical inference post-processing, which could affect comparability but is not circularity.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The central claim of state-of-the-art performance rests on many hand-chosen hyperparameters and on assumptions about DINOv2 feature semantics and the validity of egocentric augmentations. No new physical or conceptual entities are introduced beyond the method components themselves.

free parameters (5)
  • Loss weights β, γ, λ, μ = 0.16, 0.16, 0.6, 0.1
    Chosen by validation; the loss composition directly affects the reported performance.
  • QueryAug probability p = 0.5
    Tuned on validation; ablation shows performance peaks at 0.5.
  • PCA activation scaling τ = not stated
    Controls sharpness of the activation in Eq. (2); no value given in the paper.
  • Number of principal components R = not stated
    Sets the number of mid-level attention guides and matches the cross-attention head count.
  • TAG loss weights λtoken, λmap = not stated
    Balance the token-wise and map-wise entropy losses in Eq. (11).
assumptions (6)
  • domain assumption DINOv2 pre-trained features transfer to egocentric object matching
    The encoder is used as-is with no fine-tuning described; the central claim depends on these features being discriminative.
  • domain assumption Ground-truth instances used for QueryAug do not create a train/test distribution mismatch
    Training swaps the query with a GT frame from the same video, while test-time queries are external images; this gap is not analyzed.
  • domain assumption Frame reordering in MotionAug preserves supervision validity
    The reordered video is paired with reordered ground truth, and CT loss assumes the reordered clip is a valid training signal.
  • domain assumption The encoder class token encodes global object context
    Basis of the high-level attention guide in Eq. (1).
  • domain assumption PCA of query features isolates discriminative object parts
    Basis of the mid-level attention guide in Eq. (2).
  • standard math Standard transformer decoder, TSM, and PyTorch behave correctly
    Adopted from prior work without formal verification or additional proof.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HERO-VQL: Hierarchical, Egocentric and Robust Visual Query Localization." pith.science (2026). https://pith.science/paper/OXFJBXBM

@misc{pith2026250900385,
  author       = {Pith},
  title        = {Pith review of: HERO-VQL: Hierarchical, Egocentric and Robust Visual Query Localization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OXFJBXBM}},
  note         = {Machine review of arXiv:2509.00385}
}
read the original abstract

In this work, we tackle the egocentric visual query localization (VQL), where a model should localize the query object in a long-form egocentric video. Frequent and abrupt viewpoint changes in egocentric videos cause significant object appearance variations and partial occlusions, making it difficult for existing methods to achieve accurate localization. To tackle these challenges, we introduce Hierarchical, Egocentric and RObust Visual Query Localization (HERO-VQL), a novel method inspired by human cognitive process in object recognition. We propose i) Top-down Attention Guidance (TAG) and ii) Egocentric Augmentation based Consistency Training (EgoACT). Top-down Attention Guidance refines the attention mechanism by leveraging the class token for high-level context and principal component score maps for fine-grained localization. To enhance learning in diverse and challenging matching scenarios, EgoAug enhances query diversity by replacing the query with a randomly selected corresponding object from groundtruth annotations and simulates extreme viewpoint changes by reordering video frames. Additionally, CT loss enforces stable object localization across different augmentation scenarios. Extensive experiments on VQ2D dataset validate that HERO-VQL effectively handles egocentric challenges, significantly outperforming baselines.

Figures

Figures reproduced from arXiv: 2509.00385 by the authors.

Figure 1
Figure 1. Egocentric visual query localization (VQL). (a) Given an egocentric video and a query image of an object, the goal is to localize the last occurrence of the query object in the video. (b) Unlike third-person videos, egocentric videos undergo abrupt viewpoint changes due to the camera wearer’s movements. (c) These viewpoint changes introduce significant challenges in VQL, including variations in object appearance acr… view at source ↗
Figure 2
Figure 2. Overview of HERO-VQL. (a) Given a video and a query image, we extract fea￾ture vectors using a pre-trained visual encoder. We feed the feature vectors through a spatial decoder, followed by a temporal module and a prediction head outputs per-frame bound￾ing boxes and scores. (b) TAG guides the spatial decoder’s attention using a high-level cue to capture the overall query context and a mid-level cue to enhance the u… view at source ↗
Figure 3
Figure 3. Visualization of TAG. We show the query image and four frames from each video with TAG’s high-level and mid-level attention guides. The high-level attention guide emphasize the region of the target object and the mid-level attention guide attends to spe￾cific object parts in the query feature. They improve the model’s ability to localize objects accurately, under varying perspectives or partial visibility. Effect of… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Architecture visualization of baseline. Algorithm 1: Ego-motion Augmentation (MotionAug) Input: RGB video Ivideo = {I1,I2,...,IT }, GT frames FGT ⊆ Ivideo Output: Reordered video I ′ video 1 Initialize I ′ video = Ivideo // Copy video 2 Initialize F ′ GT = [ ] // Reord…
Figure 5
Figure 5. Figure 5: Visualization of additional details in high-level attention guidance. Query t Input Frame t High-level Attention Guide r Mid-level Attention Guide (a) Query t Input Frame t High-level Attention Guide r Mid-level Attention Guide (b) [PITH_FULL_IMAGE:figures/full_fig_p0…
Figure 6
Figure 6. Figure 6: Visualization of TAG. Z L−1 video represents the M video patch tokens, and Z L−1 cls corresponds to the class token of the query image. To compute high-level attention guide αhigh, we use the query projection matrix WQ ∈ R D×D and the key projection matrix WK ∈ R D×D f…
Figure 7
Figure 7. Figure 7: Qualitative examples in videos with partial object visibility. In each row, we show the query image and five frames from each video with predicted bounding boxes of HERO-VQL, VQLoC [24], ground-truth boxes, and a confidence score curve. Query Predicted(HERO-VQL, VQLoC)…
Figure 8
Figure 8. Figure 8: Qualitative examples in videos with fast-moving object. Query Predicted(HERO-VQL, VQLoC) bounding boxes t (a) (b) (c) [PITH_FULL_IMAGE:figures/full_fig_p023_8.png]
Figure 9
Figure 9. Figure 9: Qualitative examples in an unseen and real-world video from YouTube [PITH_FULL_IMAGE:figures/full_fig_p023_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

52 extracted references · 43 canonical work pages

  1. [1]

    The reverse hierarchy theory of visual perceptual learning

    Merav Ahissar and Shaul Hochstein. The reverse hierarchy theory of visual perceptual learning. Trends Cogn. Sci., 8(10):457–464, 2004

    work page 2004

  2. [2]

    Unsupervised learning of visual features by contrasting cluster assign- ments

    Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Ar- mand Joulin. Unsupervised learning of visual features by contrasting cluster assign- ments. In NeurIPS, 2020

    work page 2020

  3. [3]

    Generative pretraining from pixels

    Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, and Ilya Sutskever. Generative pretraining from pixels. In ICML, 2020

    2020

  4. [4]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In ICML, 2020

    2020

  5. [5]

    Detect what you can: Detecting and representing objects using holistic models and body parts

    Xianjie Chen, Roozbeh Mottaghi, Xiaobai Liu, Sanja Fidler, Raquel Urtasun, and Alan Yuille. Detect what you can: Detecting and representing objects using holistic models and body parts. In CVPR, 2014

    2014

  6. [6]

    Seq- track: Unified sequence-to-sequence learning for single- and multi-modal visual object tracking

    Xin Chen, Ben Kang, Jiawen Zhu, Dong Wang, Houwen Peng, and Huchuan Lu. Seq- track: Unified sequence-to-sequence learning for single- and multi-modal visual object tracking. In CVPR, 2023

    work page 2023

  7. [7]

    Category-aware allocation transformer for weakly supervised object localization

    Zhiwei Chen, Jinren Ding, Liujuan Cao, Yunhang Shen, Shengchuan Zhang, Guannan Jiang, and Rongrong Ji. Category-aware allocation transformer for weakly supervised object localization. In ICCV, 2023

    work page 2023

  8. [8]

    Unsupervised object dis- covery and localization in the wild: Part-based matching with bottom-up region pro- posals

    Minsu Cho, Suha Kwak, Cordelia Schmid, and Jean Ponce. Unsupervised object dis- covery and localization in the wild: Part-based matching with bottom-up region pro- posals. In CVPR, 2015

    work page 2015

Show all 52 references
  1. [9]

    Mixformer: End-to-end tracking with iterative mixed attention

    Yutao Cui, Cheng Jiang, Limin Wang, and Gangshan Wu. Mixformer: End-to-end tracking with iterative mixed attention. In CVPR, 2022

    2022

  2. [10]

    Scaling egocentric vision: The epic-kitchens dataset

    Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Sanja Fidler, Antonino Furnari, Evangelos Kazakos, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, and Michael Wray. Scaling egocentric vision: The epic-kitchens dataset. In ECCV, 2018. 12 CHANG ET AL.: HERO-VQL...

    2018

  3. [11]

    Augmented reality smart glasses in industrial assembly: Current status and future challenges

    Oscar Danielsson, Magnus Holm, and Anna Syberfeldt. Augmented reality smart glasses in industrial assembly: Current status and future challenges. J. Ind. Inf. Integr ., 20:100175, 2020

    2020

  4. [12]

    Is first person vision challenging for object tracking? In ICCVW, 2021

    Matteo Dunnhofer, Antonino Furnari, Giovanni Maria Farinella, and Christian Miche- loni. Is first person vision challenging for object tracking? In ICCVW, 2021

    2021

  5. [13]

    Pairwise body-part attention for recognizing human-object interactions

    Hao-Shu Fang, Jinkun Cao, Yu-Wing Tai, and Cewu Lu. Pairwise body-part attention for recognizing human-object interactions. In ECCV, 2018

    2018

  6. [14]

    Amego: Ac- tive memory from long egocentric videos

    Gabriele Goletto, Tushar Nagarajan, Giuseppe Averta, and Dima Damen. Amego: Ac- tive memory from long egocentric videos. In ECCV, 2024

    2024

  7. [15]

    Robotic home assistant care-o-bot® 3 - product vision and innovation platform

    Birgit Graf, Ulrich Reiser, Martin Hägele, Kathrin Mauz, and Peter Klein. Robotic home assistant care-o-bot® 3 - product vision and innovation platform. In ARSOW, 2009

    2009

  8. [16]

    Ego4d: Around the world in 3,000 hours of egocentric video

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. In CVPR, 2022

    2022

  9. [17]

    Kristen Grauman, Andrew Westbury, Lorenzo Torresani, Kris Kitani, Jitendra Malik, Triantafyllos Afouras, Kumar Ashutosh, Vijay Baiyya, Siddhant Bansal, Bikram Boote, Eugene Byrne, Zach Chavis, Joya Chen, Feng Cheng, Fu-Jen Chu, Sean Crane, Avijit Dasgupta, Jing Dong, Maria Esc...

    2024

  10. [18]

    Bootstrap your own latent-a new approach to self- supervised learning

    Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mo- hammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self- supervised learning. In NeurIPS, 2020

    2020

  11. [19]

    Momentum con- trast for unsupervised visual representation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum con- trast for unsupervised visual representation learning. In CVPR, 2020. CHANG ET AL.: HERO-VQL: HIERARCHICAL, EGOCENTRIC AND ROBUST VQL 13

    2020

  12. [20]

    View from the top: Hierarchies and reverse hier- archies in the visual system

    Shaul Hochstein and Merav Ahissar. View from the top: Hierarchies and reverse hier- archies in the visual system. Neuron, 36(5):791–804, 2002

    2002

  13. [21]

    Egocentric audio-visual object localization

    Chao Huang, Yapeng Tian, Anurag Kumar, and Chenliang Xu. Egocentric audio-visual object localization. In CVPR, 2023

    2023

  14. [22]

    Video recap: Recursive captioning of hour-long videos

    Md Mohaiminul Islam, Ngan Ho, Xitong Yang, Tushar Nagarajan, Lorenzo Torresani, and Gedas Bertasius. Video recap: Recursive captioning of hour-long videos. InCVPR, 2024

    2024

  15. [23]

    Detrs with hybrid matching

    Ding Jia, Yuhui Yuan, Haodi He, Xiaopei Wu, Haojun Yu, Weihong Lin, Lei Sun, Chao Zhang, and Han Hu. Detrs with hybrid matching. In CVPR, 2023

    2023

  16. [24]

    Single-stage visual query localization in egocentric videos

    Hanwen Jiang, Santhosh Ramakrishnan, and Kristen Grauman. Single-stage visual query localization in egocentric videos. In NeurIPS, 2023

    2023

  17. [25]

    Sam2mot: A novel paradigm of multi-object tracking by segmentation

    Junjie Jiang, Zelin Wang, Manqi Zhao, Yin Li, and DongSheng Jiang. Sam2mot: A novel paradigm of multi-object tracking by segmentation. arXiv preprint arXiv:2504.04519, 2025

    2025

  18. [26]

    Refego: Referring expression compre- hension dataset from first-person perception of ego4d

    Shuhei Kurita, Naoki Katsura, and Eri Onami. Refego: Referring expression compre- hension dataset from first-person perception of ego4d. In ICCV, 2023

    2023

  19. [27]

    Locate: Localize and transfer object parts for weakly supervised affordance grounding

    Gen Li, Varun Jampani, Deqing Sun, and Laura Sevilla-Lara. Locate: Localize and transfer object parts for weakly supervised affordance grounding. In CVPR, 2023

    2023

  20. [28]

    Tsm: Temporal shift module for efficient video understanding

    Ji Lin, Chuang Gan, and Song Han. Tsm: Temporal shift module for efficient video understanding. In ICCV, 2019

    2019

  21. [29]

    Focal loss for dense object detection

    T Lin. Focal loss for dense object detection. arXiv preprint arXiv:1708.02002, 2017

    2017 arXiv

  22. [30]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2019

    2019

  23. [31]

    Egoloc: Revisiting 3d object localization from egocentric videos with visual queries

    Jinjie Mai, Abdullah Hamdi, Silvio Giancola, Chen Zhao, and Bernard Ghanem. Egoloc: Revisiting 3d object localization from egocentric videos with visual queries. In ICCV, 2023

    2023

  24. [32]

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El- Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Ra...

    2024

  25. [33]

    Automatic differentiation in pytorch

    Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. In NeurIPSW, 2017

    2017

  26. [34]

    Learning transferable visual models from natural lan- guage supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sand- hini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural lan- guage supervision. In ICML, 2021. 1...

    2021

  27. [35]

    Generalized intersection over union: A metric and a loss for bounding box regression

    Hamid Rezatofighi, Nathan Tsoi, JunYoung Gwak, Amir Sadeghian, Ian Reid, and Sil- vio Savarese. Generalized intersection over union: A metric and a loss for bounding box regression. In CVPR, 2019

    2019

  28. [36]

    Ego4d goal-step: Toward hierarchical understanding of procedural activities

    Yale Song, Gene Byrne, Tushar Nagarajan, Huiyu Wang, Miguel Martin, and Lorenzo Torresani. Ego4d goal-step: Toward hierarchical understanding of procedural activities. In NeurIPS, 2023

    2023

  29. [37]

    Transformer tracking with cyclic shifting window attention

    Zikai Song, Junqing Yu, Yi-Ping Phoebe Chen, and Wei Yang. Transformer tracking with cyclic shifting window attention. In CVPR, 2022

    2022

  30. [38]

    Egotracks: A long-term ego- centric visual object tracking dataset

    Hao Tang, Kevin Liang, Matt Feiszli, and Weiyao Wang. Egotracks: A long-term ego- centric visual object tracking dataset. In NeurIPS, 2023

    2023

  31. [39]

    Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N

    Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, 2017

    2017

  32. [40]

    Negative frames matter in egocentric visual query 2d localization

    Mengmeng Xu, Cheng-Yang Fu, Yanghao Li, Bernard Ghanem, Juan-Manuel Perez- Rua, and Tao Xiang. Negative frames matter in egocentric visual query 2d localization. arXiv preprint arXiv:2208.01949, 2022

    2022 arXiv

  33. [41]

    Where is my wallet? modeling object proposal sets for egocentric visual query localization

    Mengmeng Xu, Yanghao Li, Cheng-Yang Fu, Bernard Ghanem, Tao Xiang, and Juan- Manuel Pérez-Rúa. Where is my wallet? modeling object proposal sets for egocentric visual query localization. In CVPR, 2023

    2023

  34. [42]

    Learning spatio- temporal transformer for visual tracking

    Bin Yan, Houwen Peng, Jianlong Fu, Dong Wang, and Huchuan Lu. Learning spatio- temporal transformer for visual tracking. In ICCV, 2021

    2021

  35. [43]

    Samurai: Adapting segment anything model for zero-shot visual tracking with motion-aware memory

    Cheng-Yen Yang, Hsiang-Wei Huang, Wenhao Chai, Zhongyu Jiang, and Jenq-Neng Hwang. Samurai: Adapting segment anything model for zero-shot visual tracking with motion-aware memory. arXiv preprint arXiv:2411.11922, 2024

    2024 arXiv

  36. [44]

    Self- produced guidance for weakly-supervised object localization

    Xiaolin Zhang, Yunchao Wei, Guoliang Kang, Yi Yang, and Thomas Huang. Self- produced guidance for weakly-supervised object localization. In ECCV, 2018

    2018

  37. [45]

    Visual prompt multi-modal tracking

    Jiawen Zhu, Simiao Lai, Xin Chen, Dong Wang, and Huchuan Lu. Visual prompt multi-modal tracking. In CVPR, 2023. CHANG ET AL.: HERO-VQL: HIERARCHICAL, EGOCENTRIC AND ROBUST VQL 15 Supplementary Material In this supplementary material, we provide comprehensive dataset/baseline a...

    2023

  38. [46]

    Implementation details

  39. [47]

    Comprehensive quantitative results

  40. [48]

    Details VQ2D dataset is a large-scale egocentric video dataset for localization that con- tains about 6K clips

    Comprehensive qualitative evaluations A Dataset Details In this section, we provide a detailed description of Visual Queries 2D localization (VQ2D) [16] dataset. Details VQ2D dataset is a large-scale egocentric video dataset for localization that con- tains about 6K clips. The...

  41. [49]

    Specifically, we evaluate tAP25 at a tIoU threshold of 0.25, indicating how well the predicted temporal boundaries align with the ground truth

    tAP 25: Temporal Average Precision (AP) calculated at a threshold of temporal Inter- section over Union (tIoU). Specifically, we evaluate tAP25 at a tIoU threshold of 0.25, indicating how well the predicted temporal boundaries align with the ground truth. Higher values suggest...

  42. [50]

    This metric measures the overlap between the predicted and ground-truth spatio-temporal volumes, capturing both spatial accu- racy and temporal alignment

    stAP 25: Spatio-temporal Average Precision (AP) calculated at a threshold of spatio- temporal Intersection over Union (stIoU). This metric measures the overlap between the predicted and ground-truth spatio-temporal volumes, capturing both spatial accu- racy and temporal alignment

  43. [51]

    For a frame to be considered correctly recovered, the spatial IoU between the prediction and ground truth must exceed 0.5

    Recovery % (Rec %): Frame-level recall, which measures the proportion of frames correctly identified within a video. For a frame to be considered correctly recovered, the spatial IoU between the prediction and ground truth must exceed 0.5

  44. [52]

    This metric empha- sizes identifying whether the prediction captures even a minimal level of overlap with the ground truth

    Success rate (Succ.): Spatio-temporal precision, representing the proportion of spatio- temporal predictions that meet a minimum IoU threshold of 0.05. This metric empha- sizes identifying whether the prediction captures even a minimal level of overlap with the ground truth. B...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.