Pith. sign in

REVIEW 3 major objections 6 minor 53 references

DVPSFormer claims that panoptic segmentation queries can double as an explicit discretization of the scene, letting a single online model decode dense metric depth in one cross-attention pass and track instances with majority-voted class re

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 00:36 UTC pith:WMKIOPBE

load-bearing objection Solid, fast, SOTA online DVPS paper whose main conceptual claim (ESD) is plausible but not fully pinned down against a simpler depth head. the 3 major comments →

arxiv 2607.26165 v1 pith:WMKIOPBE submitted 2026-07-28 cs.CV

DVPSFormer: Efficient Online Depth-aware Video Panoptic Segmentation for Autonomous Driving

Yung-Hsu Yang , Luigi Piccinelli , Siyuan Li , Mattia Segu , Lei Ke , Martin Danelljan , Yuqian Fu , Zuria Bauer
show 3 more authors
Fisher Yu Hermann Blum Marc Pollefeys
This is my paper
classification cs.CV
keywords depth-aware video panoptic segmentationonline instance trackingexplicit scene discretizationmetric depth estimationtransformer queriesonline majority votingautonomous drivingpanoptic segmentation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Depth-aware video panoptic segmentation asks one model to output panoptic masks, metric depth, and instance trajectories per frame. The paper argues this can be done in a single online pass: the transformer queries that produce the segmentation masks are treated as an explicit discrete description of the scene, and depth is decoded from them by one cross-attention layer followed by an MLP. Tracking stays online by matching query-derived embeddings across frames and correcting class labels with a majority vote over the track's recent history. The resulting model reports higher depth-aware video panoptic quality than all prior methods on Cityscapes-DVPS and SemKITTI-DVPS, while running about 18x faster than the prior 20-frame-association pipeline. This suggests that for autonomous driving, joint 4D understanding does not require separate depth and tracking stages.

Core claim

The paper's central claim is that semantic segmentation can be repurposed as an explicit scene discretization that supports both metric depth estimation and instance tracking, not just classification. Specifically, the final transformer queries that produce panoptic masks are treated as discrete scene tokens; a depth head applies cross-attention between the pixel representation and these tokens (the discrete-to-continuous, D2C, transformation), followed by an MLP and exponential map, to yield a full metric depth map in a single pass. Because the depth loss flows back through these same queries, depth supervision directly sharpens segmentation. For tracking, a lightweight MLP projects the que

What carries the argument

The load-bearing mechanism is explicit scene discretization (ESD): the set of panoptic segmentation queries, which already encode foreground objects and background stuff, is treated as a discrete codebook of the scene. A discrete-to-continuous (D2C) depth head then computes D = exp(MLP_esd(CA_esd(E_d, MLP_depth(q_i)))), i.e. it uses the high-resolution pixel representation as the query and the segmentation queries as keys/values in a single cross-attention layer, followed by an MLP to log-depth. This replaces per-mask depth estimation with a one-pass, full-image depth map, and makes the depth loss directly refine the segmentation queries. The second mechanism is online majority voting (OMV),

Load-bearing premise

The central premise is that the panoptic segmentation queries, trained mostly to separate masks and classes, also encode enough geometric scene structure that a single cross-attention layer can turn them into accurate dense metric depth for every pixel, including background and unannotated areas.

What would settle it

A decisive test would be to feed the depth head with permuted or randomly re-initialized segmentation queries and measure depth accuracy: if dense depth degrades sharply, the queries are the genuine carrier of geometry; if it stays accurate, the ESD coupling is not doing the work. Or, on a dataset with dense ground-truth depth, compute depth error separately for correctly masked, misclassified, and unmasked regions to see whether the discretization truly covers the whole scene.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • A single forward pass per frame can deliver panoptic segmentation, dense metric depth, and instance identities, removing the separate depth-estimation and tracking stages of earlier DVPS systems.
  • Because the depth loss flows through the same queries that generate masks, geometric supervision directly improves panoptic quality, not just depth accuracy.
  • Online, class-agnostic association with query embeddings outperforms offline tracking that observes future frames on long sequences, making real-time deployment feasible.
  • Filtering invalid points from sparse LiDAR-projected annotations during point-sampled mask training enables full-resolution training on a single 24GB GPU, a practical requirement for high-resolution driving datasets.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The ESD principle is not specific to depth: any mask transformer could, in principle, decode other dense per-pixel quantities (normals, occupancy, flow) by treating its segmentation queries as a scene codebook, so the paper may point to a general architecture pattern.
  • The class-agnostic association plus majority voting could be extended to open-set tracking, where the model corrects or even discovers class labels over time without retraining, as long as the track is stable.
  • The reported 46.9 FPS is at 384x1280 with a ResNet-50 backbone; the real-time margin on embedded hardware or with stronger backbones is not evaluated, so the practical deployment headroom remains an open question.
  • The depth head's completeness relies on the segmentation queries covering the whole scene; on domains without dense panoptic labels (e.g., open-world or indoor scenes with many unannotated objects), the discrete representation may miss regions and the depth decoder would likely need extra capacity.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes DVPSFormer, a unified online transformer architecture for depth-aware video panoptic segmentation (DVPS). The method builds on Mask2Former-style panoptic queries, which are reused as an 'explicit scene discretization' to decode metric depth in a single cross-attention pass (ESD, Eq. 3). Tracking is performed online with a lightweight query-based tracking head trained with quasi-dense similarity learning, and an online majority voting (OMV) mechanism refines class predictions from temporal context. Experiments on Cityscapes-DVPS and SemKITTI-DVPS report new state-of-the-art DVPQ scores (55.7 and 46.3 with ResNet-50) and a large speed advantage over Uni-DVPS (e.g., 46.9 FPS vs 2.6 FPS for 20-frame association on SemKITTI-DVPS).

Significance. If the results hold, this is a practically valuable contribution: a single online model that simultaneously outputs panoptic segmentation, metric depth, and instance trajectories at near-real-time rates, with consistent gains over prior multi-stage and offline systems. The paper is clearly written, the ablations are systematic, and the code/models are promised. The conceptual idea of reusing segmentation queries as explicit scene discretization for depth is interesting and connects to the iDisc internal-discretization line. However, the central novelty claim is not yet fully isolated because the ESD head is not compared against a simple per-pixel MLP head of comparable capacity, and all headline numbers are single-run estimates without error bars. With proper controls and statistical reporting, this could be a strong benchmark paper.

major comments (3)
  1. [Sec. 3.2 / Table 4, Eq. (3)] The attribution of the ESD gain to the query-based 'explicit scene discretization' is not established. Table 4 compares ESD against the Uni-DVPS depth pipeline (row 2) and removes ESD from the full model (row 5), but no within-model control is run against a per-pixel MLP depth head on E_d with comparable parameter count, same training schedule, and same depth supervision. The +1.7 DPQ / +1.8 PQ could come from added head capacity or from joint depth training, not from the segmentation queries. Please add this MLP baseline, report parameter counts, and show whether the query cross-attention is actually load-bearing for depth quality and for the induced segmentation gain.
  2. [Tables 1, 2, 4, 5] All headline numbers are single-run point estimates with no standard deviations or seed sweeps. The Cityscapes average margin over Multiformer is only 0.9 DVPQ (55.7 vs 54.8), so the 'new state-of-the-art' claim is statistically fragile as presented. Please report at least three seeds for the main tables (or a justified single-run statement with variance estimates), and specify the FPS measurement protocol in Table 2 — batch size, warm-up, precision, and whether preprocessing and I/O are included — since the 18x speed-up is a central claim.
  3. [Table 1 / Sec. 4.4] The Multiformer baseline is reported only as a single averaged row (54.8 / 37.4 / 67.4) with no per-window-size values, while all other methods show complete k sweeps. Because the averaged DVPQ is used to support the SOTA claim, the comparison is not fully verifiable. Please provide the full breakdown for Multiformer or clearly state the source and protocol; otherwise the 0.9-point advantage cannot be checked.
minor comments (6)
  1. [Sec. 4 header / Table 1 caption] Typos: 'Experiements' should be 'Experiments'; 'RestNet-50' should be 'ResNet-50'.
  2. [Sec. 3.2, Eq. (3)] Equation (3) uses q_i, but the text says the depth head uses the output of the last decoder layer q_{l-1}. Please align the notation.
  3. [Sec. 4.3] The paper says 'same training schedule' as Uni-DVPS, but the exact number of epochs, learning-rate schedule, and warm-up are not stated. Please include these details for reproducibility.
  4. [Sec. 3.3 / Table 5] In the tracking ablations, the captions do not fully define 'Query' and 'SL'. Also, OMV uses the past five frames, but the behavior for tracks younger than five frames is not described; please clarify initialization.
  5. [Eq. (9)] The DVPQ definition has mismatched parentheses in the PQ expression and the concatenation operator is not defined. A cleaner formal definition would improve the paper.
  6. [Fig. 4] The annotation-validity percentages (e.g., COCO ~99%, Cityscapes-DVPS ~70%, SemKITTI-DVPS ~40%) should be specified as how they are computed over pixels/annotations, since they motivate the point-sampling modification.

Circularity Check

0 steps flagged

No significant circularity: the paper's claims are empirical benchmark results with training-based components and external comparisons.

full rationale

DVPSFormer's central claims are direct benchmark results on Cityscapes-DVPS and SemKITTI-DVPS measured with DVPQ, an external metric defined in Eq. (9) from prior work. The ESD depth head in Eq. (3) is a network module trained with the depth loss in Eq. (6) and evaluated on held-out frames; it is not a fitted parameter renamed as a prediction. The panoptic segmentation, tracking, and majority-voting components are also trained with the stated losses (Eqs. (2), (7), (8)) and validated by ablations in Tables 4 and 5. The paper does not derive its state-of-the-art claim from its own assumptions; it compares against external prior methods under common benchmarks. Self-citations to iDisc, QDTrack, and TETer provide architectural inspiration and prior techniques but are not load-bearing evidence for the reported DVPS numbers. The absence of a within-model MLP depth-head baseline is a legitimate ablation/correctness concern, but it does not make the derivation circular, since no equation reduces to its own input and no fitted value is presented as an independent prediction. Therefore, no circular step can be exhibited from the paper's text.

Axiom & Free-Parameter Ledger

8 free parameters · 5 axioms · 0 invented entities

All ledger entries are standard supervised-learning hyperparameters and benchmark-domain assumptions; none are fitted to the target DVPQ numbers. The main conceptual risk is the ESD premise that segmentation queries encode enough geometry for one-pass depth decoding, which is checked only through ablations on the two datasets. No new physical entities are introduced.

free parameters (8)
  • Panoptic loss weights (lambda_ce, lambda_dice, lambda_cls) = 5.0, 5.0, 2.0
    Taken from Mask2Former/Uni-DVPS; set by hand and affect segmentation quality and the balance with other losses.
  • Depth loss weights (alpha, lambda_depth, lambda_abs) = 0.15, 100.0, 10.0
    Hand-set for scale-invariant and absolute-relative depth losses; control the depth/segmentation trade-off.
  • Tracking loss weight lambda_embed = 0.25
    Balances the multi-positive similarity loss and the cosine auxiliary loss for tracking embeddings.
  • Association similarity threshold = 0.3
    Heuristic threshold deciding whether a query matches an active track or starts a new track; directly affects ID switches and OMV input.
  • OMV temporal window = 5 frames
    Hand-selected majority-vote history; controls how much temporal smoothing is applied to class labels.
  • Point-sampling size K
    Inherited from Mask2Former but not reported; influences mask-loss behavior and training efficiency.
  • Transformer decoder layers l
    Number of round-robin decoder layers is not stated; presumably taken from Mask2Former defaults.
  • Number of unified queries N
    Query count is not specified; it determines ESD representation capacity and tracking association complexity.
axioms (5)
  • domain assumption Panoptic segmentation queries are a sufficient discrete scene representation for dense metric depth decoding
    Core ESD premise; it relies on depth supervision shaping the same queries that are primarily trained for masks and classes. Supported only by ablations on two datasets. See Eq. (3) and Sec. 3.2.
  • domain assumption Cityscapes-DVPS and SemKITTI-DVPS ground-truth depth is accurate enough for training and DVPQ evaluation
    The paper acknowledges sparse/noisy LiDAR-derived and stereo-derived annotations and filters invalid points, but residual ground-truth error affects every reported DVPQ number. See Sec. 4.1 and Sec. 4.5.
  • domain assumption DVPQ with depth thresholds lambda and windows k is a valid measure of 4D panoptic understanding
    All state-of-the-art comparisons use this metric, so 'better' is defined by DVPQ. The metric gates panoptic matches by depth accuracy. See Sec. 4.2.
  • domain assumption The training schedule, pretrained ResNet-50 backbone, and data splits match Uni-DVPS
    Stated for fairness in Sec. 4.3 but not independently verifiable from the paper alone, since no training logs or checkpoints are provided.
  • domain assumption Quasi-dense similarity training produces discriminative tracking queries for online association
    The method relies on MOT literature (QDTrack and related works) and is validated by ablation row 3 versus row 2 in Table 5.

pith-pipeline@v1.3.0-alltime-deepseek · 13369 in / 18914 out tokens · 183765 ms · 2026-08-01T00:36:38.299906+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of DVPSFormer: Efficient Online Depth-aware Video Panoptic Segmentation for Autonomous Driving." pith.science (2026). https://pith.science/paper/WMKIOPBE

@misc{pith2026260726165,
  author       = {Pith},
  title        = {Pith review of: DVPSFormer: Efficient Online Depth-aware Video Panoptic Segmentation for Autonomous Driving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WMKIOPBE}},
  note         = {Machine review of arXiv:2607.26165}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Safe autonomous navigation requires a holistic understanding of dynamic environments, necessitating the simultaneous estimation of metric depth, semantic segmentation, and instance trajectories. While depth-aware video panoptic segmentation (DVPS) unifies these tasks, existing approaches often rely on computationally expensive, multi-stage pipelines or offline tracking, rendering them unsuitable for real-time decision-making. To address this, we propose DVPSFormer, a unified online architecture designed for efficient 4D scene understanding. Central to our approach is explicit scene discretization (ESD), a novel mechanism that leverages segmentation queries to represent foreground and background regions, enabling a discrete-to-continuous (D2C) depth head to decode metric depth in a single pass. This tightly couples semantic and geometric learning while significantly reducing latency. Furthermore, we propose an online majority voting (OMV) mechanism that exploits temporal consistency to refine classification during instance tracking. DVPSFormer establishes a new state-of-the-art on the Cityscapes-DVPS and SemKITTI-DVPS benchmarks, offering a streamlined solution for online robotic perception. Code and models are available at https://royyang0714.github.io/DVPSFormer.

Figures

Figures reproduced from arXiv: 2607.26165 by Fisher Yu, Hermann Blum, Lei Ke, Luigi Piccinelli, Marc Pollefeys, Martin Danelljan, Mattia Segu, Siyuan Li, Yung-Hsu Yang, Yuqian Fu, Zuria Bauer.

Figure 1
Figure 1. Figure 1: Intuition on DVPSFormer. Compared to the previous state-of-the-art method [22], DVPSFormer is an efficient and online architecture for autonomous driving. We treat the segmentation pipeline as the explicit scene discretization process to estimate metric depth in one pass, and design the online tracking pipeline with majority voting to obtain better segmentation. However, the main challenge of deploying DVP… view at source ↗
Figure 2
Figure 2. Figure 2: Model Architecture. DVPSFormer is an online unified architecture for DVPS. We treat the segmentation pipeline as an explicit scene discretization process and utilize a discrete-to-continuous depth head to decode MMDE and track instances with representative tracking queries, which are trained using similarity learning. We plot three transformer decoder layers TrDi in the figures, where the decoder operates … view at source ↗
Figure 4
Figure 4. Figure 4: Online majority voting (OMV) and point sampling. The upper part illustrates our proposed OMV, which conducts class-agnostic instance association and refines the current frame classification based on the majority of the trajectory classes. The lower part shows the annotation difference among the COCO [27] and DVPS datasets [40], where black color stands for no ground truth. described as follows: D = exp(MLP… view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative results of DVPSFormer. The upper part is Cityscapes-DVPS, while the lower part is SemKITTI-DVPS. We show the sequence with input RGB images, video panoptic segmentation results (VPS), monocular metric depth estimation (Depth), and unprojected 3D prediction. Input RGB Uni-DVPS Ours [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Qualitative analyses. We compare Uni-DVPS with our method on Cityscapes-DVPS for both segmentation and absolute relative error for depth The red boxes highlight the inaccuracies of the previous SOTA. 4.7. Qualitative Results In [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

53 extracted references · 5 linked inside Pith

  1. [1]

    4d- former: Multimodal4dpanopticsegmentation

    Ali Athar, Enxu Li, Sergio Casas, and Raquel Urtasun. 4d- former: Multimodal4dpanopticsegmentation. InConference on Robot Learning, 2023. 1

    2023

  2. [2]

    Fantrack: 3d multi- object tracking with feature association network

    Erkan Baser, Venkateshwaran Balasubramanian, Prarthana Bhattacharyya, and Krzysztof Czarnecki. Fantrack: 3d multi- object tracking with feature association network. InIEEE Intelligent Vehicles Symposium (IV), 2019. 1

    2019

  3. [3]

    Semantickitti: Adatasetforsemanticsceneunderstandingoflidarsequences

    Jens Behley, Martin Garbade, Andres Milioto, Jan Quenzel, SvenBehnke,CyrillStachniss,andJurgenGall. Semantickitti: Adatasetforsemanticsceneunderstandingoflidarsequences. InICCV, 2019. 2, 6

    2019

  4. [4]

    Depth pro: Sharp monocular metric depth in less than a second.arXiv preprint arXiv:2410.02073, 2024

    Aleksei Bochkovskii, Amaël Delaunoy, Hugo Germain, Mar- cel Santos, Yichao Zhou, Stephan R Richter, and Vladlen Koltun. Depth pro: Sharp monocular metric depth in less than a second.arXiv preprint arXiv:2410.02073, 2024. 2

    Pith/arXiv arXiv 2024

  5. [5]

    Cafuser: Condition-aware multimodal fusion for robust semantic perception of driving scenes.IEEE Robotics and Automation Letters, 2025

    Tim Brödermann, Christos Sakaridis, Yuqian Fu, and Luc Van Gool. Cafuser: Condition-aware multimodal fusion for robust semantic perception of driving scenes.IEEE Robotics and Automation Letters, 2025. 1

    2025

  6. [6]

    Dgfusion: Depth-guidedsensor fusion for robust semantic perception.IEEE Robotics and Automation Letters, 2026

    Tim Brödermann, Christos Sakaridis, Luigi Piccinelli, Wim Abbeloos,andLucVanGool. Dgfusion: Depth-guidedsensor fusion for robust semantic perception.IEEE Robotics and Automation Letters, 2026. 1

    2026

  7. [7]

    End- to-end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End- to-end object detection with transformers. InECCV, 2020. 3

    2020

  8. [8]

    Panoptic-deeplab: A simple, strong, and fast baseline for bottom-up panoptic segmentation

    Bowen Cheng, Maxwell D Collins, Yukun Zhu, Ting Liu, Thomas S Huang, Hartwig Adam, and Liang-Chieh Chen. Panoptic-deeplab: A simple, strong, and fast baseline for bottom-up panoptic segmentation. InCVPR, 2020. 2

    2020

  9. [9]

    Per- pixel classification is not all you need for semantic segmenta- tion.NeurIPS, 2021

    Bowen Cheng, Alex Schwing, and Alexander Kirillov. Per- pixel classification is not all you need for semantic segmenta- tion.NeurIPS, 2021. 2, 3

    2021

  10. [10]

    Masked-attention mask trans- former for universal image segmentation

    BowenCheng,IshanMisra,AlexanderGSchwing,Alexander Kirillov, and Rohit Girdhar. Masked-attention mask trans- former for universal image segmentation. InCVPR, 2022. 1, 2, 3, 4, 5

    2022

  11. [11]

    The cityscapes dataset for semantic urban scene understanding

    Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. InCVPR, 2016. 2, 5

    2016

  12. [12]

    Depth map prediction from a single image using a multi-scale deep network.NeurIPS, 2014

    David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from a single image using a multi-scale deep network.NeurIPS, 2014. 1, 2, 5

    2014

  13. [13]

    Cc-3dt: Panoramic 3d object tracking via cross-camera fusion

    Tobias Fischer, Yung-Hsu Yang, Suryansh Kumar, Min Sun, and Fisher Yu. Cc-3dt: Panoramic 3d object tracking via cross-camera fusion. InConference on Robot Learning, 2022. 2, 5

    2022

  14. [14]

    Qdtrack: Quasi-dense similarity learning for appearance-only multiple object tracking.T-PAMI, 2023

    Tobias Fischer, Thomas E Huang, Jiangmiao Pang, Linlu Qiu, Haofeng Chen, Trevor Darrell, and Fisher Yu. Qdtrack: Quasi-dense similarity learning for appearance-only multiple object tracking.T-PAMI, 2023. 2, 3, 5

    2023

  15. [15]

    Panopticdepth: A unified framework for depth-aware panoptic segmentation

    NaiyuGao,FeiHe,JianJia,YanhuShan,HaoyangZhang,Xin Zhao, and Kaiqi Huang. Panopticdepth: A unified framework for depth-aware panoptic segmentation. InCVPR, 2022. 1, 4

    2022

  16. [16]

    Are we ready for autonomous driving? the kitti vision benchmark suite

    Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. InCVPR, 2012. 6

    2012

  17. [17]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InCVPR, 2016. 6

    2016

  18. [18]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Gir- shick. Mask r-cnn. InICCV, 2017. 2, 5

    2017

  19. [19]

    Monocular quasi-dense 3d object tracking.T-PAMI, 2022

    Hou-NingHu,Yung-HsuYang,TobiasFischer,TrevorDarrell, Fisher Yu, and Min Sun. Monocular quasi-dense 3d object tracking.T-PAMI, 2022. 2, 5

    2022

  20. [20]

    Metric3d v2: A versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation.arXiv preprint arXiv:2404.15506, 2024

    Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai, Xiaoxiao Long, Hao Chen, Kaixuan Wang, Gang Yu, Chunhua Shen, and Shaojie Shen. Metric3d v2: A versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation.arXiv preprint arXiv:2404.15506, 2024. 2

    Pith/arXiv arXiv 2024

  21. [21]

    Computervisionforautonomousvehicles: Problems,datasets and state of the art.Foundations and Trends in Computer Graphics and Vision, 2020

    Joel Janai, Fatma Güney, Aseem Behl, and Andreas Geiger. Computervisionforautonomousvehicles: Problems,datasets and state of the art.Foundations and Trends in Computer Graphics and Vision, 2020. 1

    2020

  22. [22]

    Uni-dvps: Unified model for depth-aware video panoptic segmentation.IEEE Robotics and Automation Letters (RA-L), 2024

    Kim Ji-Yeon, Oh Hyun-Bin, Kwon Byung-Ki, Dahun Kim, Yongjin Kwon, and Tae-Hyun Oh. Uni-dvps: Unified model for depth-aware video panoptic segmentation.IEEE Robotics and Automation Letters (RA-L), 2024. 1, 2, 3, 4, 5, 6, 7

    2024

  23. [23]

    Eager- mot: 3d multi-object tracking via sensor fusion

    Aleksandr Kim, Aljoša Ošep, and Laura Leal-Taixé. Eager- mot: 3d multi-object tracking via sensor fusion. InIEEE InternationalConferenceonRoboticsandAutomation(ICRA),

  24. [24]

    Video panoptic segmentation

    Dahun Kim, Sanghyun Woo, Joon-Young Lee, and In So Kweon. Video panoptic segmentation. InCVPR, 2020. 2, 6

    2020

  25. [25]

    Semantic hierarchy- guided adversarial attack for autonomous driving.IEEE Robotics and Automation Letters, 2025

    Gwangbin Kim and SeungJun Kim. Semantic hierarchy- guided adversarial attack for autonomous driving.IEEE Robotics and Automation Letters, 2025. 1

    2025

  26. [26]

    Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980,

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980,

    Pith/arXiv arXiv

  27. [27]

    Panoptic segmentation

    Alexander Kirillov, Kaiming He, Ross Girshick, Carsten Rother, and Piotr Dollár. Panoptic segmentation. InCVPR,

  28. [28]

    Tracking every thing in the wild

    SiyuanLi,MartinDanelljan,HenghuiDing,ThomasEHuang, and Fisher Yu. Tracking every thing in the wild. InECCV,

  29. [29]

    Fully convolutional networks for panoptic segmentation

    Yanwei Li, Hengshuang Zhao, Xiaojuan Qi, Liwei Wang, Zeming Li, Jian Sun, and Jiaya Jia. Fully convolutional networks for panoptic segmentation. InCVPR, 2021. 2

    2021

  30. [30]

    Object-centric learning with slot attention.NeurIPS, 2020

    Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, AravindhMahendran,GeorgHeigold,JakobUszkoreit,Alexey Dosovitskiy, and Thomas Kipf. Object-centric learning with slot attention.NeurIPS, 2020. 2, 4

    2020

  31. [31]

    V-net: Fully convolutional neural networks for volumetric medical image segmentation

    Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In3DV, 2016. 5

    2016

  32. [32]

    Scalableparallelprogrammingwithcuda: Iscudatheparallel programming model that application developers have been waiting for?Queue, 2008

    JohnNickolls,IanBuck,MichaelGarland,andKevinSkadron. Scalableparallelprogrammingwithcuda: Iscudatheparallel programming model that application developers have been waiting for?Queue, 2008. 6

    2008

  33. [33]

    Quasi-densesimilaritylearning for multiple object tracking

    Jiangmiao Pang, Linlu Qiu, Xia Li, Haofeng Chen, Qi Li, TrevorDarrell,andFisherYu. Quasi-densesimilaritylearning for multiple object tracking. InCVPR, 2021. 2, 3, 5

    2021

  34. [34]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zem- ing Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. NeurIPS, 2019. 6

    2019

  35. [35]

    Monodvps: A self- supervised monocular depth estimation approach to depth- aware video panoptic segmentation

    Andra Petrovai and Sergiu Nedevschi. Monodvps: A self- supervised monocular depth estimation approach to depth- aware video panoptic segmentation. InWACV, 2023. 1, 2, 6

    2023

  36. [36]

    iDisc: Internal discretization for monocular depth estimation

    Luigi Piccinelli, Christos Sakaridis, and Fisher Yu. iDisc: Internal discretization for monocular depth estimation. In CVPR, 2023. 2, 3, 4

    2023

  37. [37]

    Unidepth: Universalmonocularmetricdepthestimation

    Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. Unidepth: Universalmonocularmetricdepthestimation. InCVPR,2024. 2

    2024

  38. [38]

    UniK3D: Universal camera monocular 3d estimation

    Luigi Piccinelli, Christos Sakaridis, Mattia Segu, Yung-Hsu Yang,SiyuanLi,WimAbbeloos,andLucVanGool. UniK3D: Universal camera monocular 3d estimation. InCVPR, 2025

    2025

  39. [39]

    UniDepthV2: Universal monocular metric depth estimation made simpler.arXiv:2502.20110, 2025

    Luigi Piccinelli, Christos Sakaridis, Yung-Hsu Yang, Mat- tia Segu, Siyuan Li, Wim Abbeloos, and Luc Van Gool. UniDepthV2: Universal monocular metric depth estimation made simpler.arXiv:2502.20110, 2025. 2

    Pith/arXiv arXiv 2025

  40. [40]

    Vip-deeplab: Learning visual perception with depth-aware video panoptic segmentation

    Siyuan Qiao, Yukun Zhu, Hartwig Adam, Alan Yuille, and Liang-Chieh Chen. Vip-deeplab: Learning visual perception with depth-aware video panoptic segmentation. InCVPR,

  41. [41]

    Balancing shared and task-specific repre- sentations: A hybrid approach to depth-aware video panoptic segmentation

    Kurt HW Stolle. Balancing shared and task-specific repre- sentations: A hybrid approach to depth-aware video panoptic segmentation. InWACV, 2025. 6

    2025

  42. [42]

    Attention is all you need.NeurIPS, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.NeurIPS, 2017. 1, 2, 3

    2017

  43. [43]

    A good foundation is worth many labels: Label-efficient panoptic segmentation.IEEE Robotics and Automation Letters, 2024

    Niclas Vödisch, Kürsat Petek, Markus Käppeler, Abhinav Valada, and Wolfram Burgard. A good foundation is worth many labels: Label-efficient panoptic segmentation.IEEE Robotics and Automation Letters, 2024. 1

    2024

  44. [44]

    Fastdepth: Fast monocular depth estima- tion on embedded systems

    Diana Wofk, Fangchang Ma, Tien-Ju Yang, Sertac Karaman, and Vivienne Sze. Fastdepth: Fast monocular depth estima- tion on embedded systems. InInternational Conference on Robotics and Automation (ICRA), 2019. 1

    2019

  45. [45]

    In defense of online models for video instance segmentation

    Junfeng Wu, Qihao Liu, Yi Jiang, Song Bai, Alan Yuille, and Xiang Bai. In defense of online models for video instance segmentation. InECCV, 2022. 2, 3

    2022

  46. [46]

    Efficientdps: Efficient and end-to-end depth- aware panoptic segmentation

    Shengkai Wu, Liangliang Ren, Linfeng Gao, Yupeng Li, and Wenyu Liu. Efficientdps: Efficient and end-to-end depth- aware panoptic segmentation. InICRA, 2024. 1

    2024

  47. [47]

    Upsnet: A unified panoptic segmentation network

    Yuwen Xiong, Renjie Liao, Hengshuang Zhao, Rui Hu, Min Bai, Ersin Yumer, and Raquel Urtasun. Upsnet: A unified panoptic segmentation network. InCVPR, 2019. 2

    2019

  48. [48]

    Video instance segmentation

    Linjie Yang, Yuchen Fan, and Ning Xu. Video instance segmentation. InICCV, 2019. 1, 2

    2019

  49. [49]

    k-means mask transformer

    Qihang Yu, Huiyu Wang, Siyuan Qiao, Maxwell Collins, Yukun Zhu, Hartwig Adam, Alan Yuille, and Liang-Chieh Chen. k-means mask transformer. InECCV, 2022. 2

    2022

  50. [50]

    Polyphonicformer: unified query learning for depth-aware video panoptic segmentation

    Haobo Yuan, Xiangtai Li, Yibo Yang, Guangliang Cheng, Jing Zhang, Yunhai Tong, Lefei Zhang, and Dacheng Tao. Polyphonicformer: unified query learning for depth-aware video panoptic segmentation. InECCV, 2022. 1, 2, 4, 5, 6, 7

    2022

  51. [51]

    K-net: Towards unified image segmentation.NeurIPS,

    WenweiZhang, JiangmiaoPang, KaiChen, andChenChange Loy. K-net: Towards unified image segmentation.NeurIPS,

  52. [52]

    Does computer vision matter for action?Science Robotics, 2019

    Brady Zhou, Philipp Krähenbühl, and Vladlen Koltun. Does computer vision matter for action?Science Robotics, 2019. 1

    2019

  53. [53]

    Deformabledetr: Deformabletransformersfor end-to-endobjectdetection.arXivpreprintarXiv:2010.04159,

    Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, andJifengDai. Deformabledetr: Deformabletransformersfor end-to-endobjectdetection.arXivpreprintarXiv:2010.04159,

    Pith/arXiv arXiv 2010