Pith. sign in

REVIEW 2 major objections 4 minor 64 references

Is Tracking really more challenging in First Person Egocentric Vision?

T0 review · 2 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that the performance drop once attributed to first-person egocentric tracking is substantially a property of the human-object activity domain, and uses synchronized FPV–TPV videos to isolate the true viewpoint-specific…

desk verdict Useful synchronized FPV/TPV benchmark, but the 'domain not viewpoint' claim rests on a frame-rate-confounded external comparison. read the letter →

arxiv 2507.16015 v1 pith:BUTPVIHD submitted 2025-07-21 cs.CV

Matteo Dunnhofer , Zaira Manigrasso , Christian Micheloni This is my paper
classification cs.CV
keywords egocentricvisionfirst-personvisualobjecttrackingvideosegmentationbenchmarkviewpointbiashuman-objectinteractionsynchronizedevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether object tracking is really harder in first-person egocentric video (FPV) or whether earlier benchmarks merely compared FPV footage against third-person (TPV) datasets with different objects, scenes, and activities. To separate the two, the authors build VISTA, a benchmark of 544 synchronized FPV–TPV video pairs of the same human-object activities, and run state-of-the-art trackers on both views under an identical one-pass protocol. They find that FPV does cost generic trackers a real performance penalty, but a large share of the drop previously blamed on the viewpoint is actually the difficulty of the human-object activity domain: the same trackers also score far below their usual benchmarks when watching the same activities from a third-person camera. The paper further shows that hand-object interaction is not an FPV-specific challenge, that viewpoint-specific training data biases models toward whichever view they were trained on, and that object disappearance and re-detection is the dominant FPV difficulty.

What carries the argument

The load-bearing instrument is the Synchronized One-Pass Evaluation (SOPE) protocol, an extension of the standard one-pass and semi-supervised evaluation to pairs of synchronized videos: each tracker is initialized separately on the first frame of the FPV and the TPV recording of the same activity, runs online on both, and is scored against aligned annotations. The difference between views is condensed into a mean weighted signed difference $\Delta\sigma$ for AUC, normalized precision, and generalized success robustness, and displayed on a Performance Difference Plot whose diagonal marks view-independent behavior. The substrate is VISTA, a dataset of 544 test and 1,831 training FPV–TPV pairs built from synchronized ego and exo streams, resized, frame-aligned, and annotated with masks at one frame per second, plus per-frame attributes — motion blur, scale variation, distractor presence, hand-object interaction, field-of-view position, and others — that let the authors attribute the gap to specific factors.

What would settle it

Run the same trackers under SOPE on VISTA paired with a different TPV camera per activity — the average across all available exocentric cameras, or a randomly selected one — instead of only the clearest annotated view; if the FPV-minus-TPV differences shrink to zero or reverse under that choice, the viewpoint-specific penalty is an artifact of the yardstick, while persistence across camera choices would support the paper's claim. A second check: if a tracker trained from the same data but only on the TPV half of VISTA's training set matches or beats the FPV-trained version on long-term FPV tracking, the paper's claim that TPV optimization is the weaker training signal would be contradicted.

Watch

Extended reading notes

Core claim

The paper's central claim, stated in its results section, is that a significant portion of the performance drop is due to challenges of the human-object interaction domain rather than only the FPV perspective itself. Concretely, generic trackers show negative mean signed differences (FPV minus TPV) on all three metrics — AUC, normalized precision, and generalized success robustness — confirming that FPV retains a genuine extra difficulty even when the scene, activity, and object are identical. But the same trackers' absolute scores in both views sit far below their published results on standard benchmarks such as LaSOT and YouTube-VOS, evidence that human-object activity videos are themselves hard from any viewpoint. The paper also argues the opposite of a common assumption: objects involved in hand interactions are actually easier to track in FPV because they appear larger and at higher resolution, and in long-term tracking a model optimized on TPV underperforms one optimized on FPV in its own view, contradicting earlier claims that long-term FPV tracking is uniquely hard. Foundation-model trackers (SAM2-based) show almost no viewpoint bias during periods of target visibility, indicating that the FPV penalty is concentrated in object disappearances and re-detection.

Load-bearing premise

Each FPV video is compared against the single third-person camera that the source dataset selected because it gives the clearest view of the activity; if that camera is an atypically easy or well-framed viewpoint, then the measured FPV-versus-TPV gap misstates how much of the difficulty is truly about the first-person perspective.

Editorial extensions

If this is right

  • Improving object re-detection after disappearance is the highest-leverage direction for FPV tracking, since occlusion and out-of-view events drive most of the viewpoint penalty.
  • Hand-object interaction does not need to be treated as an egocentric-specific failure mode; it favors FPV if anything, so FPV trackers should be tested for generalization to the same interactions seen from third-person cameras.
  • Training on a single viewpoint teaches viewpoint bias: models tuned on TPV sacrifice FPV accuracy and vice versa, while fine-tuning on generic data plus both views narrows the gap.
  • Foundation-model trackers are essentially viewpoint-agnostic while the target is visible, so the FPV gap is not a fixed property of the data but a feature of how smaller models handle re-detection.
  • Future egocentric tracking benchmarks should include a matched TPV control stream, otherwise domain difficulty and viewpoint difficulty cannot be separated.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extension: the comparison pairs each FPV video with the single TPV camera that the source dataset selected as the clearest view of the scene, so the measured FPV penalty could understate the role of camera geometry; re-running SOPE against other or averaged TPV cameras would test how much of the gap is truly viewpoint-specific.
  • Extension: the same confound — comparing an embodied-camera domain against mismatched external datasets — likely inflates the apparent difficulty of other egocentric tasks such as action recognition or hand-object interaction understanding, and the matched-view protocol could be imported there.
  • Extension: a large part of what the field calls egocentric challenges may be a training-distribution mismatch, since models trained on largely static, natural-scene videos are not built for human-object activity at all; this points toward activity-centric pretraining rather than viewpoint-specific architectures.
  • Extension: if TPV-optimized trackers genuinely underperform FPV-optimized ones on long-term tracking, then the large static-camera benchmarks that dominate tracker pretraining may be teaching a bias that hurts models even in realistic third-person footage of human activity.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper introduces VISTA, a benchmark of 544 synchronized FPV-TPV video pairs from EgoExo4D, to disentangle whether the performance drop of visual object trackers in egocentric vision is caused by the first-person viewpoint or by the broader human-object activity domain. The authors propose a Synchronized One-Pass Evaluation (SOPE) protocol, evaluate 16 state-of-the-art VOT/VOS/VOTS trackers, and report that generic trackers consistently underperform in FPV relative to TPV on matched content, but that the absolute scores in both views are far below those on generic benchmarks such as LaSOT and YouTubeVOS. They interpret this as evidence that a significant portion of the drop is due to the human-object interaction domain rather than the FPV perspective alone. Additional analyses attribute FPV difficulty to object disappearances, egomotion, wide field-of-view distortion, and distractors, while finding that hand-object interactions are not harder in FPV. The paper also shows that viewpoint-specific training biases trackers toward the trained viewpoint.

Significance. If the main claim holds, the paper is a valuable contribution: it is the first controlled comparison of tracking in synchronized FPV and TPV videos of the same activities, and it provides a reusable protocol (SOPE) and a sizeable synchronized dataset. The internal FPV-vs-TPV comparison on matched content is a meaningful advance over prior cross-dataset benchmark comparisons, and the finding that hand-object interaction is not FPV-specific challenges a common assumption in egocentric vision. The paper ships public code, data, and results, which strengthens reproducibility. However, the external 'domain, not viewpoint' conclusion hinges on comparisons to published numbers from different temporal-sampling protocols, which needs to be addressed before the central claim is fully supported.

major comments (2)
  1. [Sec. 5 / Sec. 3.2] The load-bearing claim that a significant portion of the performance drop is due to the human-object activity domain is supported by comparing VISTA TPV scores to published LaSOT/YouTubeVOS numbers, e.g., 'OSTrack achieves an AUC-TPV of 49.2, down from a 71.1 AUC on the LaSOT benchmark' and 'SAM2-M ... AUC-TPV of 58.8, lower than its Gscore of 89.3 on YouTubeVOS.' This comparison is not controlled for temporal sampling: VISTA videos are stored at 5 FPS with annotations at 1 FPS (Sec. 3.2), while LaSOT and YouTubeVOS are evaluated on native 30 FPS frames with dense annotations. Tracking performance is known to be frame-rate sensitive; a 6x reduction in frame rate increases inter-frame displacement and makes re-acquisition substantially harder. A nontrivial part of the 20+ point gap could therefore be a subsampling artifact rather than a property of the human-object activity domain. Because the 'domain, not viewpoint' conclusion explicitly rests on this absolute scale comparison, this confound is load-bearing. Please control for frame rate, for example by subsampling LaSOT/YouTubeVOS to 5 FPS and 1 FPS annotations, or by comparing against another 5 FPS benchmark such as EgoTracks, or by reporting both native and matched frame-rate numbers. The internal FPV-TPV delta is matched in frame rate and is not affected by this issue.
  2. [Supp. A.4] The SOPE comparison assumes that the chosen TPV view is a fair and representative yardstick for third-person vision. However, Supp. A.4 states that 'we select the TPV view that was originally annotated with object tracks in EgoExo4D, as it provides the clearest observation of the scene.' Deliberately selecting the clearest TPV view makes TPV easier than a typical third-person observation, which can inflate the measured FPV-vs-TPV gap and distort the magnitude of the 'FPV is harder' conclusion. The limitation section acknowledges that TPV camera placement was not controlled, but the specific selection criterion should be justified quantitatively (e.g., by reporting results with multiple TPV views or by analyzing how the chosen view compares to alternative ones in terms of occlusion, target size, and distance). This does not undermine the direction of the internal comparison, but it does affect the external validity of the reported delta magnitudes and the nuanced claim that FPV is less hard than previously suggested.
minor comments (4)
  1. [Sec. 5] The only statistical test reported is 'A T-Test on pov-AUC scores yielded a p-value<0.004.' This is insufficiently specified: no test statistic, degrees of freedom, or number of sequences is given, and it is unclear whether the test is across trackers or across sequences. Given the consistent sign across many trackers, this is not fatal, but the paper should report per-sequence delta distributions (e.g., confidence intervals or box plots) to support the significance claim.
  2. [Supp. A.6] The HOI attribute is computed using only the FPV view and then assigned to the synchronized TPV frames. This could introduce label noise in TPV, where the hand-object overlap may differ. Please clarify whether the HOI labels were validated in TPV or discuss the potential impact on the Fig. 6 conclusion.
  3. [Fig. 3] The caption states 'Scores in brackets are ordered in descending order by Δσ,' but the bracketed triples in the plots appear to be [FPV, TPV, Δ]. Please clarify the ordering convention or rephrase to avoid confusion.
  4. [Throughout] The manuscript contains several typos and grammatical slips: 'contstraints' (Sec. 3.2), 'previuosly' (Sec. 3.2), 'wether' (Introduction), 'continusouly' (Fig. 4 caption), 'becuase' (Supp. A.6), 'readibility' (Supp. Table 4), and 'development an benchmarking' (Supp. A.2). These should be corrected.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: VISTA's FPV-vs-TPV delta is a direct synchronized measurement, and the domain-effect comparison uses independent external benchmark scores; self-citations are motivational or consistency checks, not load-bearing.

full rationale

The paper is an empirical benchmark study, not a derivation, so the circularity burden is low. The central FPV-vs-TPV comparison uses the SOPE protocol on synchronized, frame-aligned videos from EgoExo4D, with ground-truth annotations taken from an external dataset. The reported delta (Eq. 1) is a direct measurement of performance difference, not a fitted parameter later renamed as a prediction. The claim that 'a significant portion of the performance drop is due to challenges of the human-object interaction domain rather than only the FPV perspective' rests on comparing VISTA TPV scores to published scores on LaSOT and YouTubeVOS. Those external benchmark numbers are independent evidence; the comparison may be confounded by frame-rate and annotation-density differences (VISTA stores 5 FPS video with 1 FPS annotations versus native 30 FPS benchmarks), but a confound is a validity concern, not a circular reduction. No equation defines FPV difficulty in terms of TPV difficulty, and no self-citation is used to establish the benchmark's conclusions: TREK-150 and EgoTracks are cited to motivate the question and to note consistency with prior findings, while the new measurements are self-contained. The TPV camera selection in Supp. A.4 is a dataset-construction assumption, not a circular step. I find no instance where a result is equivalent to its inputs by construction, and no fitted input is presented as a prediction. Score 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new physical entities are postulated. The benchmark's conclusions rest on dataset curation choices (TPV selection, filtering, weighting, attribute thresholds) and on the assumption that the chosen TPV view is a fair baseline. The most impactful hand-chosen quantity is the sequence-length weighting, which alters the sign of the viewpoint gap for some trackers.

free parameters (3)
  • sequence-length weighting (omega_i = |A_i|) = annotation count per sequence
    Scores are weighted by the number of annotations before averaging (Eq. 1, Supp. A.7). Supp. Tab. 3 shows this choice flips the sign of delta AUC for some trackers (e.g., DAM4SAM from +0.9 to -4.1), so the headline results depend on it.
  • attribute thresholds = 0.15 (IV), 100 (MB Laplacian variance), 0.5 IoU (STA/MOV and HOI), 0.5 cosine similarity (distractors)
    Hand-chosen thresholds define the frame-level attributes used in Figs. 4-7; no sensitivity analysis is reported for any threshold.
  • TPV camera selection = 'clearest observation of the scene'
    The TPV view is chosen as the one with the clearest observation (Supp. A.4), potentially making the TPV baseline easier; the authors list this as a limitation (Supp. D).
assumptions (4)
  • domain assumption Synchronized FPV and TPV frames capture the same scene, so delta_sigma isolates viewpoint effects.
    Sec. 3.1 states that shared scene, synchronized frames, equal length, and aligned annotations 'ensure that delta_sigma captures only the differences in tracker performance due to the viewpoint change.'
  • domain assumption Ground-truth masks from EgoExo4D are correct and remain aligned after filtering and resizing.
    Sec. 3.2 and Supp. A.4 describe filtering, resizing to 720p, and saving at 5 FPS; the fidelity of the original annotations is taken for granted.
  • domain assumption A single TPV camera is a representative sample of third-person observation for the activity.
    Supp. A.4 selects one exocentric camera per scene; the authors acknowledge placement was not controlled (Supp. D).
  • domain assumption Published scores on LaSOT and YouTubeVOS are comparable despite different evaluation protocols.
    Sec. 5 uses published AUC and G scores to claim human-object activity videos are harder regardless of viewpoint; the trackers are not re-run under a matched protocol.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Is Tracking really more challenging in First Person Egocentric Vision?." pith.science (2026). https://pith.science/paper/BUTPVIHD

@misc{pith2026250716015,
  author       = {Pith},
  title        = {Pith review of: Is Tracking really more challenging in First Person Egocentric Vision?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BUTPVIHD}},
  note         = {Machine review of arXiv:2507.16015}
}
read the original abstract

Visual object tracking and segmentation are becoming fundamental tasks for understanding human activities in egocentric vision. Recent research has benchmarked state-of-the-art methods and concluded that first person egocentric vision presents challenges compared to previously studied domains. However, these claims are based on evaluations conducted across significantly different scenarios. Many of the challenging characteristics attributed to egocentric vision are also present in third person videos of human-object activities. This raises a critical question: how much of the observed performance drop stems from the unique first person viewpoint inherent to egocentric vision versus the domain of human-object activities? To address this question, we introduce a new benchmark study designed to disentangle such factors. Our evaluation strategy enables a more precise separation of challenges related to the first person perspective from those linked to the broader domain of human-object activity understanding. By doing so, we provide deeper insights into the true sources of difficulty in egocentric tracking and segmentation, facilitating more targeted advancements on this task.

Figures

Figures reproduced from arXiv: 2507.16015 by the authors.

Figure 1
Figure 1. We isolate the real impact of first person vision (FPV) in object tracking evaluation. (a) Previous benchmarks claimed the challenges of tracking in FPV by comparing algorithm performance on third person vision (TPV) datasets having mismatched object categories and contexts. This inconsistency makes it difficult to isolate the true impact of the FPV viewpoint. (b) This paper addresses this problem by directly evalua… view at source ↗
Figure 2
Figure 2. Example of a synchronized FPV and TPV video. The VISTA benchmark dataset is composed of pairs of annotated videos V FPV , V TPV where FPV frames F FPV = {F FPV t } T −1 t=0 are synchronized to TPV frames F TPV = {F TPV t } T −1 t=0 and both are annotated with the ground-truth state of a target object. In this example, annotations are segmentation masks A FPV = {MFPV t } T −1 t=0 , A TPV = {MTPV t } T −1 t=0 . 60,986… view at source ↗
Figure 3
Figure 3. Performance Difference Plots for FPV and TPV. (a) When tasked with tracking a target object throughout an entire human￾object activity, generic object trackers perform better in TPV, confirming the challenges posed by FPV. FPV-specific trackers improve FPV performance and lose accuracy in TPV, while TPV-trained trackers perform better in TPV and sacrifice FPV accuracy. (b) When tracking a target object for a short p… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: FPV is difficult due to the continusouly moving cam￾era, but TPV is not easier when objects move. We assessed the viewpoint impact on trackers in frames with objects labeled as be￾ing static or moving in the scene. Refer to [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: The field of view affects tracking in FPV. We analyzed the impact of object deformation due to distance from the image center on tracking accuracy. (a) shows the distance ranges used to cluster ground-truth mask barycenters. The plots depict AUC as a function of distan…
Figure 7
Figure 7. Figure 7: The effects of object and camera motion, along with distractors, pose significant challenges in FPV. We analyzed tracking performance in relation to frame attributes to identify visual variability factors specific of FPV and TPV. Refer to [PITH_FULL_IMAGE:figures/full…
Figure 8
Figure 8. Figure 8: Object categories represented in VISTA. This word￾cloud visualizes the categories and the frequency of the target ob￾jects available in our benchmarks’s training and test sets. equipment, kit parts, etc. The VISTA dataset consists of 6 human-object activities — bike re…
Figure 10
Figure 10. Figure 10: Qualitative example # 2. Here, we illustrate the behav￾ior of DAM4SAM [51] on a sequence from VISTA’s evaluation set DTEST. The frames F pov t are overlaid with the predicted seg￾mentation masks Mcpov t (shown in light green). Below the frames, we report the sequence-…
Figure 11
Figure 11. Figure 11: Qualitative example # 3. Here, we illustrate the be￾havior of DAM4SAM [51] on a sequence from VISTA’s evalua￾tion set DTEST. The frames F pov t are overlaid with the predicted segmentation masks Mcpov t (shown in light green). Below the frames, we report the sequence-…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

64 extracted references · 59 canonical work pages

  1. [1]

    Ar- trackv2: Prompting autoregressive tracker where to look and how to describe

    Yifan Bai, Zeyang Zhao, Yihong Gong, and Xing Wei. Ar- trackv2: Prompting autoregressive tracker where to look and how to describe. InCVPR, 2024. 5

    work page 2024

  2. [2]

    Seqtrack: Sequence to sequence learning for visual ob- ject tracking

    Xin Chen, Houwen Peng, Dong Wang, Huchuan Lu, and Han Hu. Seqtrack: Sequence to sequence learning for visual ob- ject tracking. InCVPR, 2023. 5

    work page 2023

  3. [3]

    Xmem: Long- term video object segmentation with an atkinson-shiffrin memory model

    Ho Kei Cheng and Alexander G Schwing. Xmem: Long- term video object segmentation with an atkinson-shiffrin memory model. InECCV, 2022. 5, 4

    work page 2022

  4. [4]

    Putting the object back into video object segmentation

    Ho Kei Cheng, Seoung Wug Oh, Brian Price, Joon-Young Lee, and Alexander Schwing. Putting the object back into video object segmentation. InCVPR, 2024. 5

    work page 2024

  5. [5]

    Scaling egocentric vision: The epic-kitchens dataset

    Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Sanja Fidler, Antonino Furnari, Evangelos Kazakos, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, et al. Scaling egocentric vision: The epic-kitchens dataset. In ECCV, 2018. 3, 7, 8

    work page 2018

  6. [6]

    Epic-kitchens visor benchmark: Video segmenta- tions and object relations.NeurIPS, 2022

    Ahmad Darkhalil, Dandan Shan, Bin Zhu, Jian Ma, Amlan Kar, Richard Higgins, Sanja Fidler, David Fouhey, and Dima Damen. Epic-kitchens visor benchmark: Video segmenta- tions and object relations.NeurIPS, 2022. 1, 2, 4, 5, 6, 7

    work page 2022

  7. [7]

    Egopoints: Advancing point tracking for ego- centric videos

    Ahmad Darkhalil, Rhodri Guerrier, Adam W Harley, and Dima Damen. Egopoints: Advancing point tracking for ego- centric videos. InWACV, 2025. 2, 4

    work page 2025

  8. [8]

    Mose: A new dataset for video object segmentation in complex scenes

    Henghui Ding, Chang Liu, Shuting He, Xudong Jiang, Philip HS Torr, and Song Bai. Mose: A new dataset for video object segmentation in complex scenes. InICCV, 2023. 3

    work page 2023

Show all 64 references
  1. [9]

    Is first person vision challenging for object tracking? InICCVW, 2021

    Matteo Dunnhofer, Antonino Furnari, Giovanni Maria Farinella, and Christian Micheloni. Is first person vision challenging for object tracking? InICCVW, 2021. 2

    2021

  2. [10]

    Visual object tracking in first person vision.IJCV, 2023

    Matteo Dunnhofer, Antonino Furnari, Giovanni Maria Farinella, and Christian Micheloni. Visual object tracking in first person vision.IJCV, 2023. 1, 2, 4, 5, 6, 8, 3

    2023

  3. [11]

    Tracking skiers from the top to the bottom

    Matteo Dunnhofer, Luca Sordi, Niki Martinel, and Christian Micheloni. Tracking skiers from the top to the bottom. In WACV, 2024. 2

    2024

  4. [12]

    Project aria: A new tool for egocentric multi-modal ai research.arXiv preprint arXiv:2308.13561, 2023

    Jakob Engel, Kiran Somasundaram, Michael Goesele, Albert Sun, Alexander Gamino, Andrew Turner, Arjang Talattof, Arnie Yuan, Bilal Souti, Brighid Meredith, et al. Project aria: A new tool for egocentric multi-modal ai research.arXiv preprint arXiv:2308.13561, 2023. 7, 2

    2023 arXiv

  5. [13]

    LaSOT: A High-quality Benchmark for Large-scale Single Object Tracking

    Heng Fan, Liting Lin, Fan Yang, Peng Chu, Ge Deng, Sijia Yu, Hexin Bai, Yong Xu, Chunyuan Liao, and Haibin Ling. LaSOT: A High-quality Benchmark for Large-scale Single Object Tracking. InCVPR, 2019. 1, 2, 4, 5, 6

    2019

  6. [14]

    Amego: Active memory from long egocentric videos

    Gabriele Goletto, Tushar Nagarajan, Giuseppe Averta, and Dima Damen. Amego: Active memory from long egocentric videos. InECCV, 2024. 1, 3

    2024

  7. [15]

    Ego4d: Around the world in 3,000 hours of egocentric video

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. In CVPR, 2022. 1, 3, 4, 7, 8

    2022

  8. [16]

    Ego-exo4d: Understanding skilled human activity from first-and third-person perspectives

    Kristen Grauman, Andrew Westbury, Lorenzo Torresani, Kris Kitani, Jitendra Malik, Triantafyllos Afouras, Kumar Ashutosh, Vijay Baiyya, Siddhant Bansal, Bikram Boote, et al. Ego-exo4d: Understanding skilled human activity from first-and third-person perspectives. InCVPR, 2024. ...

    2024

  9. [17]

    Ego3dt: Tracking every 3d object in ego-centric videos

    Shengyu Hao, Wenhao Chai, Zhonghan Zhao, Meiqi Sun, Wendi Hu, Jieyang Zhou, Yixian Zhao, Qi Li, Yizhou Wang, Xi Li, et al. Ego3dt: Tracking every 3d object in ego-centric videos. InACM MM, 2024. 1

    2024

  10. [18]

    Particle video revisited: Tracking through occlusions using point trajectories

    Adam W Harley, Zhaoyuan Fang, and Katerina Fragkiadaki. Particle video revisited: Tracking through occlusions using point trajectories. InECCV, 2022. 2

    2022

  11. [19]

    Lvos: A benchmark for long-term video object segmentation

    Lingyi Hong, Wenchao Chen, Zhongying Liu, Wei Zhang, Pinxue Guo, Zhaoyu Chen, and Wenqiang Zhang. Lvos: A benchmark for long-term video object segmentation. In ICCV, 2023. 3

    2023

  12. [20]

    GOT-10k: A Large High-Diversity Benchmark for Generic Object Track- ing in the Wild.IEEE TPAMI, 2019

    Lianghua Huang, Xin Zhao, and Kaiqi Huang. GOT-10k: A Large High-Diversity Benchmark for Generic Object Track- ing in the Wild.IEEE TPAMI, 2019. 1, 2, 5, 4

    2019

  13. [21]

    Co- tracker: It is better to track together

    Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Co- tracker: It is better to track together. InECCV, 2024. 2

    2024

  14. [22]

    Need for speed: A benchmark for higher frame rate object tracking

    Hamed Kiani Galoogahi, Ashton Fagg, Chen Huang, Deva Ramanan, and Simon Lucey. Need for speed: A benchmark for higher frame rate object tracking. InICCV, 2017. 2

    2017

  15. [23]

    The vi- sual object tracking vot2015 challenge results

    Matej Kristan, Jiri Matas, Ales Leonardis, Michael Fels- berg, Luka Cehovin, Gustavo Fernandez, Tomas V ojir, Gus- tav Hager, Georg Nebehay, and Roman Pflugfelder. The vi- sual object tracking vot2015 challenge results. InICCVW,

  16. [24]

    A novel performance evaluation methodology for single-target trackers.IEEE TPAMI, 2016

    Matej Kristan, Jiri Matas, Ale ˇs Leonardis, Tom´aˇs V oj´ıˇr, Ro- man Pflugfelder, Gustavo Fernandez, Georg Nebehay, Fatih Porikli, and Luka ˇCehovin. A novel performance evaluation methodology for single-target trackers.IEEE TPAMI, 2016. 4, 2, 3

    2016

  17. [25]

    The sixth visual object tracking vot2018 challenge results

    Matej Kristan, Ales Leonardis, Jiri Matas, Michael Fels- berg, Roman Pflugfelder, LukaˇCehovin Zajc, Tomas V ojir, Goutam Bhat, Alan Lukezic, Abdelrahman Eldesokey, et al. The sixth visual object tracking vot2018 challenge results. In ECCVW, 2018. 2

    2018

  18. [26]

    The seventh visual object tracking vot2019 chal- lenge results

    Matej Kristan, Jiri Matas, Ales Leonardis, Michael Fels- berg, Roman Pflugfelder, Joni-Kristian Kamarainen, Luka ˇCehovin Zajc, Ondrej Drbohlav, Alan Lukezic, Amanda Berg, et al. The seventh visual object tracking vot2019 chal- lenge results. InICCVW, 2019. 2, 1

    2019

  19. [27]

    The eighth visual object tracking vot2020 chal- lenge results

    Matej Kristan, Ale ˇs Leonardis, Ji ˇr´ı Matas, Michael Fels- berg, Roman Pflugfelder, Joni-Kristian K ¨am¨ar¨ainen, Martin Danelljan, Luka ˇCehovin Zajc, Alan Luke ˇziˇc, Ondrej Dr- 9 bohlav, et al. The eighth visual object tracking vot2020 chal- lenge results. InECCVW, 2020. 2, 3, 4

    2020

  20. [28]

    The first visual object tracking segmentation vots2023 chal- lenge results

    Matej Kristan, Ji ˇr´ı Matas, Martin Danelljan, Michael Fels- berg, Hyung Jin Chang, Luka ˇCehovin Zajc, Alan Luke ˇziˇc, Ondrej Drbohlav, Zhongqun Zhang, Khanh-Tung Tran, et al. The first visual object tracking segmentation vots2023 chal- lenge results. InICCVW, 2023. 1, 2, 4, 3

    2023

  21. [29]

    The second visual object tracking segmentation vots2024 challenge re- sults

    Matej Kristan, Ji ˇr´ı Matas, Pavel Tokmakov, Michael Fels- berg, Luka ˇCehovin Zajc, Alan Luke ˇziˇc, et al. The second visual object tracking segmentation vots2024 challenge re- sults. InECCVW, 2024. 2, 3

    2024

  22. [30]

    Delving into egocen- tric actions

    Yin Li, Zhefan Ye, and James M Rehg. Delving into egocen- tric actions. InCVPR, 2015. 3

    2015

  23. [31]

    Encoding color information for visual tracking: Algorithms and bench- mark.IEEE TIP, 2015

    Pengpeng Liang, Erik Blasch, and Haibin Ling. Encoding color information for visual tracking: Algorithms and bench- mark.IEEE TIP, 2015. 2

    2015

  24. [32]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, 2014. 2, 3, 4

    2014

  25. [33]

    Nt- vot211: A large-scale benchmark for night-time visual object tracking

    Yu Liu, Arif Mahmood, and Muhammad Haris Khan. Nt- vot211: A large-scale benchmark for night-time visual object tracking. InACCV, 2024. 2

    2024

  26. [34]

    Hota: A higher order metric for evaluating multi-object tracking.IJCV, 2021

    Jonathon Luiten, Aljosa Osep, Patrick Dendorfer, Philip Torr, Andreas Geiger, Laura Leal-Taix´e, and Bastian Leibe. Hota: A higher order metric for evaluating multi-object tracking.IJCV, 2021. 1

    2021

  27. [35]

    Cdtb: A color and depth visual object tracking dataset and benchmark

    Alan Lukezic, Ugur Kart, Jani Kapyla, Ahmed Durmush, Joni-Kristian Kamarainen, Jiri Matas, and Matej Kristan. Cdtb: A color and depth visual object tracking dataset and benchmark. InICCV, 2019. 2

    2019

  28. [36]

    Performance evaluation methodology for long-term single-object tracking.IEEE Transactions on Cy- bernetics, 2020

    Alan Luke ´ziˇc, Luka ˇCehovin Zajc, Tom´aˇs V oj´ıˇr, Jiˇr´ı Matas, and Matej Kristan. Performance evaluation methodology for long-term single-object tracking.IEEE Transactions on Cy- bernetics, 2020. 4

    2020

  29. [37]

    Online episodic memory visual query localization with egocentric streaming object memory.arXiv preprint arXiv:2411.16934,

    Zaira Manigrasso, Matteo Dunnhofer, Antonino Furnari, Moritz Nottebaum, Antonio Finocchiaro, Davide Marana, Giovanni Maria Farinella, and Christian Micheloni. Online episodic memory visual query localization with egocentric streaming object memory.arXiv preprint arXiv:2411.16934,

    arXiv

  30. [38]

    Learning target candidate association to keep track of what not to track

    Christoph Mayer, Martin Danelljan, Danda Pani Paudel, and Luc Van Gool. Learning target candidate association to keep track of what not to track. InICCV, 2021. 5

    2021

  31. [39]

    Beyond sot: Tracking multiple generic objects at once

    Christoph Mayer, Martin Danelljan, Ming-Hsuan Yang, Vit- torio Ferrari, Luc Van Gool, and Alina Kuznetsova. Beyond sot: Tracking multiple generic objects at once. InWACV,

  32. [40]

    TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild

    Matthias M ¨uller, Adel Bibi, Silvio Giancola, Salman Alsub- aihi, and Bernard Ghanem. TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild. In ECCV, 2018. 1, 2, 5, 4

    2018

  33. [41]

    Avist: A benchmark for visual object tracking in adverse visibility

    Mubashir Noman, Wafa Al Ghallabi, Daniya Najiha, Christoph Mayer, Akshay Dudhane, Martin Danelljan, Hisham Cholakkal, Salman Khan, Luc Van Gool, and Fa- had Shahbaz Khan. Avist: A benchmark for visual object tracking in adverse visibility. InBMVC, 2022. 2

    2022

  34. [42]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 2

    2023 arXiv

  35. [43]

    A benchmark dataset and evaluation methodology for video object segmentation

    Federico Perazzi, Jordi Pont-Tuset, Brian McWilliams, Luc Van Gool, Markus Gross, and Alexander Sorkine-Hornung. A benchmark dataset and evaluation methodology for video object segmentation. InCVPR, 2016. 1, 2, 3, 4, 5, 6

    2016

  36. [44]

    Detecting activities of daily living in first-person camera views

    Hamed Pirsiavash and Deva Ramanan. Detecting activities of daily living in first-person camera views. InCVPR, 2012. 3

    2012

  37. [45]

    Spatial cognition from egocentric video: Out of sight, not out of mind

    Chiara Plizzari, Shubham Goel, Toby Perrett, Jacob Chalk, Angjoo Kanazawa, and Dima Damen. Spatial cognition from egocentric video: Out of sight, not out of mind. In3DV,

  38. [46]

    The 2017 davis challenge on video object segmentation.arXiv preprint arXiv:1704.00675, 2017

    Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles, Pablo Ar- bel´aez, Alex Sorkine-Hornung, and Luc Van Gool. The 2017 davis challenge on video object segmentation.arXiv preprint arXiv:1704.00675, 2017. 3

    2017 arXiv

  39. [47]

    Sam 2: Seg- ment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R¨adle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Seg- ment anything in images and videos. InICLR, 2025. 5, 6, 2, 3

    2025

  40. [48]

    Understanding human hands in contact at internet scale

    Dandan Shan, Jiaqi Geng, Michelle Shu, and David F Fouhey. Understanding human hands in contact at internet scale. InCVPR, 2020. 3

    2020

  41. [49]

    Egotracks: A long-term egocentric visual object tracking dataset.NeurIPS, 2023

    Hao Tang, Kevin J Liang, Kristen Grauman, Matt Feiszli, and Weiyao Wang. Egotracks: A long-term egocentric visual object tracking dataset.NeurIPS, 2023. 1, 2, 4, 5, 6, 7

    2023

  42. [50]

    Breaking the” object” in video object segmentation

    Pavel Tokmakov, Jie Li, and Adrien Gaidon. Breaking the” object” in video object segmentation. InCVPR, 2023. 2, 4

    2023

  43. [51]

    A distractor-aware memory for visual object tracking with sam2

    Jovana Videnovic, Alan Lukezic, and Matej Kristan. A distractor-aware memory for visual object tracking with sam2. InCVPR, 2025. 5, 7, 8

    2025

  44. [52]

    Some comments on the evaluation of model performance.Bulletin of the American Meteorological Soci- ety, 1982

    Cort J Willmott. Some comments on the evaluation of model performance.Bulletin of the American Meteorological Soci- ety, 1982. 4

    1982

  45. [53]

    Online object tracking: A benchmark

    Yi Wu, Jongwoo Lim, and Ming-Hsuan Yang. Online object tracking: A benchmark. InCVPR, 2013. 1, 2, 3, 4

    2013

  46. [54]

    Object track- ing benchmark.IEEE TPAMI, 2015

    Yi Wu, Jongwoo Lim, and Ming-Hsuan Yang. Object track- ing benchmark.IEEE TPAMI, 2015. 2, 3, 1

    2015

  47. [55]

    Youtube-vos: Sequence-to-sequence video object segmentation

    Ning Xu, Linjie Yang, Yuchen Fan, Jianchao Yang, Dingcheng Yue, Yuchen Liang, Brian Price, Scott Cohen, and Thomas Huang. Youtube-vos: Sequence-to-sequence video object segmentation. InECCV, 2018. 1, 3, 6, 4

    2018

  48. [56]

    Learning spatio-temporal transformer for vi- sual tracking

    Bin Yan, Houwen Peng, Jianlong Fu, Dong Wang, and Huchuan Lu. Learning spatio-temporal transformer for vi- sual tracking. InICCV, 2021. 5, 3

    2021

  49. [57]

    Towards grand unification of object tracking

    Bin Yan, Yi Jiang, Peize Sun, Dong Wang, Zehuan Yuan, Ping Luo, and Huchuan Lu. Towards grand unification of object tracking. InECCV, 2022. 5

    2022

  50. [58]

    Samurai: Adapting segment anything model for zero-shot visual tracking with motion-aware memory.arXiv preprint arXiv:2411.11922,

    Cheng-Yen Yang, Hsiang-Wei Huang, Wenhao Chai, Zhongyu Jiang, and Jenq-Neng Hwang. Samurai: Adapting segment anything model for zero-shot visual tracking with motion-aware memory.arXiv preprint arXiv:2411.11922,

    arXiv

  51. [59]

    Associating ob- jects with transformers for video object segmentation

    Zongxin Yang, Yunchao Wei, and Yi Yang. Associating ob- jects with transformers for video object segmentation. In NeurIPS, 2021. 5

    2021

  52. [60]

    Joint feature learning and relation modeling for tracking: A one-stream framework

    Botao Ye, Hong Chang, Bingpeng Ma, Shiguang Shan, and Xilin Chen. Joint feature learning and relation modeling for tracking: A one-stream framework. InECCV, 2022. 5, 6

    2022

  53. [61]

    Instance tracking in 3d scenes from egocentric videos

    Yunhan Zhao, Haoyu Ma, Shu Kong, and Charless Fowlkes. Instance tracking in 3d scenes from egocentric videos. In CVPR, 2024. 1, 4

    2024

  54. [62]

    Pointodyssey: A large-scale synthetic dataset for long-term point tracking

    Yang Zheng, Adam W Harley, Bokui Shen, Gordon Wet- zstein, and Leonidas J Guibas. Pointodyssey: A large-scale synthetic dataset for long-term point tracking. InICCV,

  55. [1280]

    In each FPV-TPV pair, we select the TPV view that was originally annotated with object tracks in Ego-Exo4D, as it provides the clearest observation of the scene [16]

    The placement and number of these exocentric cam- eras vary per scenario to ensure optimal coverage without obstructing participants’ activities [16]. In each FPV-TPV pair, we select the TPV view that was originally annotated with object tracks in Ego-Exo4D, as it provides the...

  56. [2023]

    We emphasize that the goal of this paper is not to develop new tracking methods for FPV but to enhance understand- ing of the task of object tracking in FPV

    2 11 Is Tracking really more challenging in First Person Egocentric Vision? Supplementary Material In this appendix, we present additional motivations, de- tails, and results regarding our benchmark study. We emphasize that the goal of this paper is not to develop new tracking...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.