Pith. sign in

REVIEW 5 minor 40 references

A transformer that learns to track objects by jointly weighing appearance and 3D location keeps identities and locations alive even after long absences from the camera's view.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Whareformer learns feed-forward track assignment over relative appearance and 3D distances with a New Track token, reaching SOTA OSNOM performance across EPIC-KITCHENS, IT3DEgo and HD-EPIC from 56 training videos.

T0 review reviewed 2026-07-10 challenge →

load-bearing objection First learned OSNOM tracker that actually generalizes: solid absolute gains on three datasets from 56 training videos, with the relative-distance + NT-token design doing the real work.

arxiv 2607.08537 v1 pith:IF2RTWUU submitted 2026-07-09 cs.CV

Whareformer: Learning to Track What is Where in Long Egocentric Videos

Jacob Chalk , Saptarshi Sinha , Dima Damen , Yannis Kalantidis , Diane Larlus This is my paper
classification cs.CV
keywords egocentric video3D object trackingobject permanenceOSNOMtransformer track assignmentDenStreamrelative appearance-location distances
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Egocentric video is hard for trackers because the camera wearer constantly moves objects, occludes them, and leaves them behind for minutes at a time. The OSNOM task asks a model to keep a coherent 3D map of every active object throughout a long video, updating both identity and location even when the object is out of sight. Prior solutions relied on hand-tuned cost thresholds and the Hungarian algorithm. Whareformer is the first learning-based solution: it stores an evolving memory of each track's appearance (via online clustering) and recent 3D locations, then feeds relative appearance and location distances, plus a dedicated New Track token, into a transformer that decides the assignment in one forward pass. Because the model reasons over relative distances rather than absolute features, it trains on only 56 kitchen videos yet generalises to hundreds of longer videos across three different datasets and cameras, beating the previous state of the art by large absolute margins on both localisation and identity metrics.

Core claim

Whareformer shows that a transformer which jointly reasons over relative appearance distances and relative 3D location distances, together with an explicit New Track token, can learn online track assignment for the OSNOM task from a small set of labelled trajectories and still generalise to unseen kitchens, non-kitchen scenes, and different recording devices.

What carries the argument

The What-and-Where Transformer: an updatable memory of DenStream appearance clusters plus a short 3D location buffer, whose pairwise relative distances to a new observation are embedded and concatenated with a learnable New Track token, then scored by a single-layer transformer encoder that outputs a probability over existing tracks or new-track creation.

Load-bearing premise

The 3D locations obtained by lifting monocular depth into a reconstructed world frame are accurate and stable enough that Euclidean distances between observations and tracks remain a reliable signal for assignment.

What would settle it

Replace the aligned depth pipeline with noisy monocular depth or deliberately drifted reconstructions on the same EPIC-KITCHENS test set and measure whether mPCL and IDF1 collapse relative to the aligned baseline reported in Table 5.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

0 major / 5 minor

Summary. The paper introduces Whareformer, the first learning-based method for the OSNOM task of online long-term 3D object tracking in egocentric video. Observations are encoded by PCA-reduced DINOv2 appearance features and monocular-depth lifts into a world frame; tracks maintain an evolving DenStream appearance summary and a short location buffer. Relative Euclidean distances are projected into assignment-likelihood embeddings, prepended with a learnable New Track token, and scored by a single-layer transformer that jointly decides matching versus new-track creation. The model is trained with teacher forcing (optionally DAgger) on 56 EPIC-KITCHENS videos and evaluated on 260 long sequences spanning EPIC-KITCHENS (unseen kitchens), IT3DEgo and HD-EPIC, reporting large absolute gains over LMK and other baselines on both mPCL and IDF1.

Significance. If the reported gains hold, the work supplies a practical, data-efficient solution to a recently formalised and practically important problem: maintaining object permanence and identity under prolonged occlusions and viewpoint changes typical of egocentric video. The relative-distance formulation, explicit New Track token, and DenStream memory are clean, reusable design choices that generalise across three datasets and recording devices with only 56 training videos. Extensive controlled ablations (appearance/location/NT token, encoder depth, DenStream parameters, depth alignment, conflict resolution) and qualitative multi-dataset examples make the contribution reproducible and falsifiable. The result is therefore of clear interest to the egocentric-vision and 3D-tracking communities.

minor comments (5)
  1. No error bars or statistical significance tests accompany the mPCL/IDF1 numbers in Table 1 or the PCL curves in Figure 3; given the large absolute margins this is not load-bearing, but a short note on variance across videos would strengthen the claim.
  2. Section 3.7 and Table 12 describe three conflict-resolution strategies that yield essentially identical scores; a one-sentence statement of the chosen default (greedy) and its asymptotic cost would improve reproducibility.
  3. The DenStream parameters (ε, µ, λ) and the precise pruning threshold τ_p are given only in the appendix; moving the selected values into the main Implementation Details paragraph would help readers who do not consult the supplement.
  4. Figure 4 and Figure 5 captions refer to green/red colour coding that is clear in the PDF but would benefit from an explicit legend for colour-blind readers.
  5. A few minor typos appear (e.g., “Whareformerconsiders” missing space in the contributions paragraph; “T eacher F orcing” in Table 6).

Circularity Check

0 steps flagged

No circularity: supervised learning of track assignment from external GT IDs; relative distances and NT token are free parameters, not definitions of the target metrics.

full rationale

Whareformer is an empirical supervised model. Training uses teacher forcing on ground-truth track IDs (Section 3.7) with a standard cross-entropy loss; the model never defines mPCL or IDF1 in terms of its own parameters. The inputs to the assignment module are relative Euclidean distances (Eq. 1) between observation descriptors and track memory (DenStream appearance + short location buffer), projected by a learned linear map g (Eq. 2) and scored by a transformer that also contains a free New Track token. These quantities are free parameters optimized against external labels; they are not fitted constants that force the reported metrics by construction. Self-citations to the OSNOM task definition and the LMK baseline [26] supply the problem statement, the 3D lifting pipeline, and the evaluation protocol; they are not used as uniqueness theorems or as the sole justification of the central claim. Ablations (Tables 2–6) and cross-dataset transfer further demonstrate that the gains are not tautological. Consequently the derivation chain contains no self-definitional step, no fitted-input-called-prediction, and no load-bearing self-citation circularity.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 2 invented entities

The central empirical claim rests on standard CV assumptions (DINOv2 features, monocular depth lifts, ground-truth masks for training) plus a handful of hand-chosen hyperparameters for DenStream and the transformer. No new physical entities are postulated; the New Track token is an architectural device, not an ontological claim.

free parameters (6)
  • DenStream ε (micro-cluster radius) = 0.25
    Set to 0.25; controls when a new appearance cluster is spawned. Ablated but chosen by hand on training data.
  • DenStream µ (promotion threshold) = 10.0
    Set to 10.0; decides when a transient cluster becomes persistent.
  • DenStream λ (temporal decay) = 1e-3
    Set to 1e-3; controls forgetting of old appearance clusters.
  • PCA dimension D = 256
    Appearance features reduced to 256 dimensions; chosen after ablation.
  • Transformer embedding dim d / heads / layers = 64 / 32 / 1
    d=64, 32 heads, 1 layer; selected by validation performance.
  • Location buffer length W = ~1 s
    Approximately 1 second of recent 3D locations; fixed design choice.
axioms (4)
  • domain assumption DINOv2 CLS features (ViT-g) after mask isolation and PCA are sufficiently discriminative for long-term re-identification under viewpoint and interaction changes.
    Section 3.2; inherited from prior OSNOM and IT3DEgo pipelines.
  • domain assumption Monocular depth aligned to a static 3D reconstruction yields metric locations accurate enough that Euclidean distances remain useful for association even after minutes of absence.
    Section 3.2 and Table 5; same lifting as LMK [26].
  • domain assumption Ground-truth object masks and track IDs are available at training time for teacher forcing.
    Section 3.7; standard supervised tracking assumption.
  • ad hoc to paper A single linear projection of concatenated appearance and location distances followed by a 1-layer transformer is expressive enough to model assignment decisions.
    Sections 3.4–3.5; architectural choice validated by ablation but not derived.
invented entities (2)
  • New Track (NT) token no independent evidence
    purpose: Explicit learnable option for creating a new track identity instead of thresholding match costs.
    Section 3.5; architectural device with no independent physical existence; ablated in Table 2.
  • Whareformer assignment likelihood embeddings no independent evidence
    purpose: Map relative (appearance, location) distances into a space where a transformer can score assignments.
    Equation (2); learned projection g; no external evidence beyond the paper’s own metrics.

reviewed 2026-07-10 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Whareformer: Learning to Track What is Where in Long Egocentric Videos." pith.science (2026). https://pith.science/paper/IF2RTWUU

@misc{pith2026260708537,
  author       = {Pith},
  title        = {Pith review of: Whareformer: Learning to Track What is Where in Long Egocentric Videos},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IF2RTWUU}},
  note         = {Machine review of arXiv:2607.08537}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

The recently established 'Out of Sight, Not out of Mind' (OSNOM) task for egocentric videos focuses on tracking objects that are moved by the camera wearer, online, maintaining knowledge of instance locations throughout the video even when they leave the field of view or become heavily occluded. In this paper, we propose the first learning-based solution to the OSNOM task: Whareformer, a transformer-based model with two components: an updatable memory of established tracks and a track assignment module that associates observations with existing tracks in a feed-forward manner. Whareformer jointly reasons over evolving object appearance (what) and updated 3D location (where), and employs a dedicated New Track token to reason about novel objects. Thanks to its design choices of using relative distances and evolving track representations, Whareformer is trained on a small set of 56 videos but achieves SOTA performance on 260 long test videos from three datasets: EPIC-KITCHENS-100 (unseen videos), IT3DEgo, and HD-EPIC, with significant absolute improvements over prior work.

Figures

Figures reproduced from arXiv: 2607.08537 by Diane Larlus, Dima Damen, Jacob Chalk, Saptarshi Sinha, Yannis Kalantidis.

Figure 1
Figure 1. Figure 1: Overview of Whareformer. In the OSNOM task, the goal is to assign the current observation (highlighted in yellow in the current frame) of a 3D egocentric scene with one of the known objects (which respectively correspond to a kettle in red, a knife in green, or a tin of chopped tomatoes in blue). We propose Whareformer, a model that jointly reasons about the appearance and the location of objects to decide… view at source ↗
Figure 2
Figure 2. Figure 2: Whareformer architecture. The current observation is represented by ap￾pearance an and location ln descriptors (Section 3.2), which are fed into our first module (Section 3.5). Using a memory of tracks constructed so far (Section 3.3), it produces embeddings representing the likelihood that this observation corresponds to each track. These embeddings, combined with a new track (NT) token, are fed to a tran… view at source ↗
Figure 3
Figure 3. Figure 3: Comparisons across datasets. PCL over sequence length (time). We com￾pare to LMK [26], the SOTA on EPIC. Whareformer consistently outperforms LMK. Qualitative results. We present qualitative results in [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative results. For each example (left: mug, right: chopping board), we show the ground-truth (GT) track and the track containing the object’s first observa￾tion for LMK and Whareformer. For sampled frames along the trajectory we show the corresponding segmented object and 3D trajectory. Green underlined crops and 3D orbs denote correct associations with the GT while red indicate incorrect ones. Here,… view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative comparison for challenging scenarios. [PITH_FULL_IMAGE:figures/full_fig_p013_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 40 canonical work pages · 1 internal anchor

  1. [1]

    In: European Conference on Computer Vision (ECCV) (2020)

    Achlioptas, P., Abdelreheem, A., Xia, F., Elhoseiny, M., Guibas, L.: Referit3d: Neural listeners for fine-grained 3d object identification in real-world scenes. In: European Conference on Computer Vision (ECCV) (2020)

    work page 2020

  2. [2]

    Journal of Experimental Algorithmics (JEA)17(2012)

    Ackermann,M.R.,Märtens,M.,Raupach,C.,Swierkot,K.,Lammersen,C.,Sohler, C.: Streamkm++ a clustering algorithm for data streams. Journal of Experimental Algorithmics (JEA)17(2012)

    work page 2012

  3. [3]

    In: International Conference on Image Analysis and Processing (ICIAP) (2015)

    Alletto, S., Serra, G., Cucchiara, R.: Egocentric object tracking: An odometry- based solution. In: International Conference on Image Analysis and Processing (ICIAP) (2015)

    work page 2015

  4. [4]

    Ardeshir,S.,Borji,A.:Integratingegocentricvideosintop-viewsurveillancevideos: Jointidentificationandtemporalalignment.In:EuropeanConferenceonComputer Vision (ECCV) (2018)

    work page 2018

  5. [5]

    In: European Conference on Computer Vision (ECCV) (2024)

    Avetisyan, A., Xie, C., Howard-Jenkins, H., Yang, T.Y., Aroudj, S., Patra, S., Zhang, F., Frost, D., Holland, L., Orme, C., et al.: Scenescript: Reconstructing scenes with an autoregressive structured language model. In: European Conference on Computer Vision (ECCV) (2024)

    work page 2024

  6. [6]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2025)

    Banerjee, P., Shkodrani, S., Moulon, P., Hampali, S., Han, S., Zhang, F., Zhang, L., Fountain, J., Miller, E., Basol, S., et al.: Hot3d: Hand and object tracking in 3d from egocentric multi-view videos. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2025)

    work page 2025

  7. [7]

    In: Asian Conference on Computer Vision (ACCV) (2024)

    Bhalgat, Y., Tschernezki, V., Laina, I., Henriques, J.F., Vedaldi, A., Zisserman, A.: 3d-aware instance segmentation and tracking in egocentric videos. In: Asian Conference on Computer Vision (ACCV) (2024)

    work page 2024

  8. [8]

    Cao, F., Ester, M., Qian, W., Zhou, A.: Density-based clustering over an evolving datastreamwithnoise.In:ProceedingsoftheSixthSIAMInternationalConference on Data Mining (ICDM) (2006)

    work page 2006

  9. [9]

    In: European Conference on Computer Vision (ECCV) (2020)

    Chen, D.Z., Chang, A.X., Nießner, M.: Scanrefer: 3d object localization in rgb- d scans using natural language. In: European Conference on Computer Vision (ECCV) (2020)

    work page 2020

  10. [10]

    International Journal of Computer Vision (IJCV)130(2022)

    Damen, D., Doughty, H., Farinella, G.M., Furnari, A., Ma, J., Kazakos, E., Molti- santi, D., Munro, J., Perrett, T., Price, W., Wray, M.: Rescaling egocentric vision: Collection, pipeline and challenges for epic-kitchens-100. International Journal of Computer Vision (IJCV)130(2022)

    work page 2022

  11. [11]

    International Journal of Computer Vision (IJCV)131(2023)

    Dunnhofer, M., Furnari, A., Farinella, G.M., Micheloni, C.: Visual object tracking in first person vision. International Journal of Computer Vision (IJCV)131(2023)

    work page 2023

  12. [12]

    Dunnhofer, M., Manigrasso, Z., Micheloni, C.: Is tracking really more challenging in first person egocentric vision? In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2025)

    work page 2025

  13. [13]

    Project Aria: A New Tool for Egocentric Multi-Modal AI Research

    Engel, J., Somasundaram, K., Goesele, M., Sun, A., Gamino, A., Turner, A., Ta- lattof, A., Yuan, A., Souti, B., Meredith, B., et al.: Project aria: A new tool for egocentric multi-modal ai research. arXiv preprint arXiv:2308.13561 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  14. [14]

    In: European Conference on Computer Vision (ECCV) (2024)

    Goletto, G., Nagarajan, T., Averta, G., Damen, D.: Amego: Active memory from long egocentric videos. In: European Conference on Computer Vision (ECCV) (2024)

    work page 2024

  15. [15]

    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)44(2021) Whareformer: Learning to Track What is Where in Long Egocentric Videos 17

    Han, R., Feng, W., Zhang, Y., Zhao, J., Wang, S.: Multiple human association and tracking from egocentric and complementary top views. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)44(2021) Whareformer: Learning to Track What is Where in Long Egocentric Videos 17

    work page 2021

  16. [16]

    Pro- ceedings of the 32nd ACM International Conference on Multimedia (ACM) (2024)

    Hao, S., Chai, W., Zhao, Z., Sun, M., Hu, W., Zhou, J., Zhao, Y., Li, Q., Wang, Y., Li, X., Wang, G.: Ego3dt: Tracking every 3d object in ego-centric videos. Pro- ceedings of the 32nd ACM International Conference on Multimedia (ACM) (2024)

    work page 2024

  17. [17]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2023)

    Huang, M., Li, X., Hu, J., Peng, H., Lyu, S.: Tracking multiple deformable objects in egocentric videos. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2023)

    work page 2023

  18. [18]

    In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2025)

    Khosla, S., V, S.T., Schwing, A., Hoiem, D.: Relocate: A simple training-free base- line for visual query localization using region-based representations. In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2025)

    work page 2025

  19. [19]

    In: International Conference on Learning Representations (ICLR) (2017)

    Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: International Conference on Learning Representations (ICLR) (2017)

    work page 2017

  20. [20]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (2023)

    Mai, J., Hamdi, A., Giancola, S., Zhao, C., Ghanem, B.: Egoloc: Revisiting 3d object localization from egocentric videos with visual queries. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (2023)

    work page 2023

  21. [21]

    2026 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (2024)

    Manigrasso, Z., Dunnhofer, M., Furnari, A., Nottebaum, M., Finocchiaro, A., Marana, D., Farinella, G.M., Micheloni, C.: Online episodic memory visual query localization with egocentric streaming object memory. 2026 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (2024)

    work page 2026

  22. [22]

    In: Forty-second International Conference on Machine Learning (ICML) (2025)

    McVay, P., Arnaud, S., Martin, A., Majumdar, A., Jatavallabhula, K.M., Thomas, P., Partsey, R., Dugas, D., Gejji, A., Sax, A., et al.: Locate 3d: Real-world ob- ject localization via self-supervised learning in 3d. In: Forty-second International Conference on Machine Learning (ICML) (2025)

    work page 2025

  23. [23]

    Journal of The Society for Industrial and Applied Mathematics (SIAM)10(1957)

    Munkres, J.: Algorithms for the assignment and transportation problems. Journal of The Society for Industrial and Applied Mathematics (SIAM)10(1957)

    work page 1957

  24. [24]

    Transactions on Ma- chine Learning Research (2024)

    Oquab, M., Darcet, T., Moutakanni, T., Vo, H.V., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., Assran, M., Ballas, N., Galuba, W., Howes, R., Huang, P.Y., Li, S.W., Misra, I., Rabbat, M., Sharma, V., Syn- naeve, G., Xu, H., Jegou, H., Mairal, J., Labatut, P., Joulin, A., Bojanowski, P.: DINOv2: Learning robust visual fe...

    work page 2024

  25. [25]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (June 2025)

    Perrett, T., Darkhalil, A., Sinha, S., Emara, O., Pollard, S., Parida, K., Liu, K., Gatti, P., Bansal, S., Flanagan, K., Chalk, J., Zhu, Z., Guerrier, R., Abdelazim, F., Zhu, B., Moltisanti, D., Wray, M., Doughty, H., Damen, D.: Hd-epic: A highly- detailed egocentric video dataset. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern ...

    work page 2025

  26. [26]

    In: 2025 International Conference on 3D Vision (3DV) (2025)

    Plizzari, C., Goel, S., Perrett, T., Chalk, J., Kanazawa, A., Damen, D.: Spatial cog- nition from egocentric video: Out of sight, not out of mind. In: 2025 International Conference on 3D Vision (3DV) (2025)

    work page 2025

  27. [27]

    In: ProceedingsoftheIEEE/CVFConferenceonComputerVisionandPatternRecog- nition (CVPR) (2022)

    Price, W., Vondrick, C., Damen, D.: Unweavenet: Unweaving activity stories. In: ProceedingsoftheIEEE/CVFConferenceonComputerVisionandPatternRecog- nition (CVPR) (2022)

    work page 2022

  28. [28]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (2023)

    Qian, S., Fouhey, D.F.: Understanding 3d object interaction from a single image. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (2023)

    work page 2023

  29. [29]

    In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR) (2022)

    Rajasegaran, J., Pavlakos, G., Kanazawa, A., Malik, J.: Tracking people by pre- dicting 3d appearance, location and pose. In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR) (2022)

    work page 2022

  30. [30]

    Chalk et al

    Ravi, N., Gabeur, V., Hu, Y.T., Hu, R., Ryali, C., Ma, T., Khedr, H., Rädle, R., Rolland, C., Gustafson, L., Mintun, E., Pan, J., Alwala, K.V., Carion, N., 18 J. Chalk et al. Wu, C.Y., Girshick, R., Dollar, P., Feichtenhofer, C.: SAM 2: Segment anything in images and videos. In: The Thirteenth International Conference on Learning Representations (2025)

    work page 2025

  31. [31]

    In: European Conference on Computer Vision (ECCV) (2016)

    Ristani, E., Solera, F., Zou, R., Cucchiara, R., Tomasi, C.: Performance measures and a data set for multi-target, multi-camera tracking. In: European Conference on Computer Vision (ECCV) (2016)

    work page 2016

  32. [32]

    In: Proceedings of the Fourteenth Interna- tional Conference on Artificial Intelligence and Statistics (AISTATS)

    Ross, S., Gordon, G., Bagnell, D.: A reduction of imitation learning and structured prediction to no-regret online learning. In: Proceedings of the Fourteenth Interna- tional Conference on Artificial Intelligence and Statistics (AISTATS). Proceedings of Machine Learning Research, vol. 15 (2011)

    work page 2011

  33. [33]

    In: European Conference on Computer Vision (ECCV) (2022)

    Stearns, C., Rempe, D., Li, J., Ambruş, R., Zakharov, S., Guizilini, V., Yang, Y., Guibas, L.J.: Spot: Spatiotemporal modeling for 3d object tracking. In: European Conference on Computer Vision (ECCV) (2022)

    work page 2022

  34. [34]

    Advances in Neural Information Processing Systems (NeurIPS)36(2023)

    Tang, H., Liang, K.J., Grauman, K., Feiszli, M., Wang, W.: Egotracks: A long- term egocentric visual object tracking dataset. Advances in Neural Information Processing Systems (NeurIPS)36(2023)

    work page 2023

  35. [35]

    IEEE Robotics and Automation Letters (RA-L)7(2022)

    Wang, X., Fu, C., Li, Z., Lai, Y., He, J.: Deepfusionmot: A 3d multi-object tracking framework based on camera-lidar fusion with deep association. IEEE Robotics and Automation Letters (RA-L)7(2022)

    work page 2022

  36. [36]

    International Conference on Intelligent Robots and Systems (IROS) (2020)

    Weng, X., Wang, J., Held, D., Kitani, K.: 3D Multi-Object Tracking: A Baseline and New Evaluation Metrics. International Conference on Intelligent Robots and Systems (IROS) (2020)

    work page 2020

  37. [37]

    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 35(2012)

    Yang, Y., Ramanan, D.: Articulated human detection with flexible mixtures of parts. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 35(2012)

    work page 2012

  38. [38]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021)

    Yin, T., Zhou, X., Krahenbuhl, P.: Center-based 3d object detection and tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021)

    work page 2021

  39. [39]

    In: European Conference on Computer Vision (ECCV) (2022)

    Zhang, Y., Sun, P., Jiang, Y., Yu, D., Weng, F., Yuan, Z., Luo, P., Liu, W., Wang, X.: Bytetrack: Multi-object tracking by associating every detection box. In: European Conference on Computer Vision (ECCV) (2022)

    work page 2022

  40. [40]

    what” and “where

    Zhao, Y., Ma, H., Kong, S., Fowlkes, C.: Instance tracking in 3d scenes from ego- centric videos. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2024) Whareformer: Learning to Track What is Where in Long Egocentric Videos 19 A Additional qualitative examples We provide video-based qualitative results for all ...

    work page 2024

This paper was first reviewed by grok-4.5 on July 10, 2026.