Contour Errors: Ego-Centric Matching for 3D Multi-Object Tracking Performance Evaluation

Abhinav Valada; Mario Berk; Sharang Kaul; Simon Bultmann

REVIEW 5 major objections 5 minor 33 references

Contour Errors: Ego-Centric Matching for 3D Multi-Object Tracking Performance Evaluation

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A contour-based matching distance for 3D tracking keeps matches that IoU rejects, cutting functional failures by up to 80 percent.

desk verdict A plausible ego-centric Hausdorff-style matching metric whose headline safety-critical claim is undermined by a filter that shares the metric's geometry; still worth a serious referee. read the letter →

arxiv 2506.04122 v4 pith:6YSJJOIU submitted 2025-06-04 cs.CV

Sharang Kaul , Simon Bultmann , Mario Berk , Abhinav Valada This is my paper

classification cs.CV

keywords multi-objecttracking3Dboundingboxego-centricevaluationHausdorffdistancematchingcriterionautonomousdrivingperformance

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the standard matching criteria for 3D multi-object tracking, volumetric Intersection-over-Union (IoU) and Center-Point Distance (CPD), are unreliable from the perspective of the ego vehicle, and it introduces Contour Error (CE) as a replacement. CE is a symmetric max-min Hausdorff-style distance computed only on the bounding-box corners nearest to the ego vehicle, so it penalizes yaw and shape errors that CPD ignores without collapsing on minor rotations the way IoU does. On the nuScenes dataset the authors report that 33–47% of car, pedestrian, and truck matches accepted by CE are rejected by IoU at the standard threshold, while almost no IoU-valid match fails CE. In safety-critical scenes selected by a time-to-collision filter, CE reduces functional false positives and false negatives by 80% at close range and 60% at far range compared to IoU. The consequence is that the matching criterion itself, not just the tracker, controls apparent tracking quality.

What carries the argument

The central object is the Contour Error distance, a hand-crafted ego-centric variant of the Hausdorff distance applied to sparse corner geometry. For each candidate match, the metric selects the k corners of each bounding box that are closest to the ego vehicle's position, then takes the maximum over those corners of the minimum Euclidean distance to the other box's surface, symmetrically in both directions. Restricting to the ego-nearest corners is what carries the argument: it makes the measure perspective-aware, so it focuses on the side of the object that faces the ego vehicle and matters for collision. This one change yields the desired graded sensitivity between IoU and CPD, and the distance matrix is then fed to the Hungarian algorithm for global assignment.

What would settle it

Re-run the nuScenes evaluation with the safety filter replaced by a planner-based or learned criticality model; if CE's reported 80% and 60% functional-failure reductions against IoU shrink or reverse, the advantage is an artifact of the filter. A complementary check is to have human raters label a set of cut-in and intersection matches: if many CE-accepted, IoU-rejected matches are judged incorrect, then CE is too permissive.

Watch

Extended reading notes

Core claim

The central discovery is that a matching criterion can be made ego-centric by restricting the comparison to the k corners of each box that face the ego vehicle. The paper defines CE as \max\left(\max_{p\in P'_j}\min_{x\in X_i}\lVert p-x\rVert,\ \max_{g\in G'_i}\min_{y\in Y_j}\lVert g-y\rVert\right), where $G'_i$ and $P'_j$ are the three (2D) or six (3D) ego-nearest corners of the ground-truth and predicted boxes, and a match holds when the value is below a category-specific threshold. Empirically, this criterion sits between IoU and CPD: it is orientation-aware enough to flag yaw misalignments that CPD misses, yet stable enough to accept near-correct matches that IoU's volume overlap rejects. The paper demonstrates this with threshold-independent scatter plots, tuned HOTA evaluations on nuScenes, and an ego-centric AP comparison of detectors on KITTI, and it argues that CE's main payoff appears once evaluation is restricted to safety-critical interactions.

Load-bearing premise

The load-bearing premise is that the hand-crafted safety-criticality filter—time-to-collision of 5 seconds or less, or lateral velocity of at least 2 m/s within 12 meters and inside a 30-meter ego radius, followed by an 11-frame analysis window—correctly identifies which mismatches are dangerous.

Editorial extensions

If this is right

Open-loop 3D MOT evaluation should report results under more than one matching criterion, because IoU and CE disagree on a large share of matches near safety-critical thresholds.
Switching the association cost from IoU to CE changes the false-positive and false-negative counts of the same tracker, so tracker scores and rankings depend on the matching criterion as much as on the tracker.
Ego-centric CE-AP reshuffles detector rankings on KITTI compared to IoU-AP, revealing near-field and far-field strengths that IoU-based benchmarking masks.
Category-specific CE thresholds (0.75 m for pedestrians, 2.5 m for cars, 4.0 m for trucks) provide a concrete starting point for standardizing safety-oriented evaluation on nuScenes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If CE becomes a standard association cost, trackers tuned to it may learn to align the ego-facing side of a box rather than the whole volume, which could yield geometry that is more directly usable by downstream planners.
The k-nearest-corner idea transfers naturally to other ego-perspective problems, such as evaluating 2D-3D fusion or motion forecasting, where measuring distance on the visible corners may be more meaningful than full-shape similarity.
The high disagreement rate between CE and IoU implies that current benchmarks may undercount true positives in near-miss scenarios, so reported false-negative rates on existing leaderboards could be inflated; a re-evaluation with CE or a hybrid criterion would test this directly.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: the CE metric is a modest but real extension of Hausdorff distance — symmetric max-min over the few bounding-box corners nearest the ego. That specific construction is not in the cited literature, and the paper is honest that any single threshold is arbitrary. The best material is threshold-independent: scatter plots and correlation tables across CE thresholds, plus the KITTI detector comparison. Those are useful and give CE a defensible place as a secondary ego-centric metric.

The soft spots are real. Eq. (1) and Algorithm 1 do not agree: Eq. (1) minimizes over surface points of the boxes, the algorithm minimizes over corner sets, and the 2D/3D corner count drifts between three and six. The abstract's '75% pedestrian' number is not in Table II, which says 33.1% at the fitted threshold. And Table I puts Corr(CE, CPD) above 0.96 for all three classes, so CE is mostly a re-parameterization of center-point distance with an orientation-sensitive twist; the paper argues they differ in edge cases, which is fair, but not a new independent signal.

The load-bearing weakness is Sec. IV-B. The functional-failure analysis uses a kinematic filter (TTC <=5 s or lateral velocity >=2 m/s within 12 m, inside 30 m) and then labels CE's extra associations as functional TPs without checking against nuScenes track IDs or any independent source of truth. The filter rewards exactly the ego-proximity geometry that CE is constructed to reward. So the 80%/60% reduction is quite plausibly a selection artifact. The threshold-independent scatter plots mitigate the circularity of the tuned thresholds, but they do not rescue the specific claim about functional failures.

Citations look fine: Hausdorff, Chamfer, ego-centric IoU, and the KITTI/nuScenes evaluation baselines are all there. No code is released, which is a smaller issue but worth noting for reproducibility.

For whom: anyone building 3D MOT evaluation criteria, or benchmarking trackers in safety-critical settings, should read it, with caution. It deserves a serious referee — the metric is plausible and the threshold-independent analysis is a real contribution — but the referee should require instance-level validation of the functional-failure claims and a fix of the Eq./Algorithm mismatch before acceptance.

Referee Report

5 major / 5 minor

Summary. The paper proposes Contour Error (CE), an ego-centric matching criterion for 3D multi-object tracking evaluation. CE is defined as a symmetric max-min Hausdorff-type distance between the k ego-nearest corners of ground-truth and predicted bounding boxes. The authors evaluate CE against IoU, CPD, and other baselines using the HOTA protocol on nuScenes and KITTI, and report that at standard IoU thresholds 33-47% of CE-valid matches are rejected by IoU while almost no IoU-valid matches fail CE. They also claim a reduction of functional failures by 80% at close ranges and 60% at far ranges, based on a kinematics-based safety-criticality filter.

Significance. If substantiated, an ego-centric matching metric could be valuable for safety-oriented evaluation of 3D MOT, because IoU over-penalizes yaw misalignment and CPD ignores orientation. The paper contributes a threshold-independent scatter-plot analysis that is useful for understanding metric behavior. However, the current manuscript does not yet establish the claimed advantages: the metric definition is internally inconsistent, the headline numbers conflict with the reported tables, and the main empirical claims rely on thresholds tuned on the test set and a safety-critical filter that may select for the behavior the metric is designed to exhibit. The high correlation between CE and CPD (r>0.96) also raises a question about the incremental contribution over CPD.

major comments (5)

[Sec. III, Eq. (1) and Algorithm 1] The metric definition is internally inconsistent. Equation (1) defines distances from selected corners p in P'_j and g in G'_i to points on the opposite bounding-box surfaces (x in X_i, y in Y_j), but Algorithm 1 (lines 6 and 11) computes the nearest point by taking argmin over the corner sets G_i and P_j, not over the surfaces. These two definitions yield different distances: a corner-to-surface distance can be small for a slightly rotated box, whereas a corner-to-corner distance will be much larger. The number of selected corners is also inconsistent: the text after Eq. (1) says "subset of three corners" for both directions, while Algorithm 1 says "three and six closest corners" in 2D and 3D. The authors must specify the metric unambiguously and ensure the algorithm implements exactly that definition, because the metric is the paper's central contribution.
[Abstract and Sec. IV-A, Table II] The abstract states that "47% of car and 75% of pedestrian CE-valid matches are rejected by IoU," but Table II reports 43.6% for car (CE≤threshold and IoU≤threshold), 33.1% for pedestrian, and 47.7% for truck. The pedestrian figure is more than a factor of two too high, and the car figure does not match 47%. These numbers are the primary quantitative claims of the paper, so the abstract must be corrected to match the table, or the table must be recomputed if the abstract values are the intended result.
[Sec. IV-A] The category-specific CE thresholds are optimized on the same nuScenes validation set used to demonstrate CE's advantage: the text says "we optimize separate thresholds for each object category" and "sensitivity analysis to maximize tracking performance (mHOTA and Recall)." The rejection percentages in Table II and the functional-failure reductions in Table III are then reported at these fitted thresholds. This makes the quantitative comparisons vulnerable to overfitting and circularity. The authors should use a held-out validation split or report results as a continuous function of threshold, with the scatter plots (Figs. 3 and 4) as the primary threshold-independent evidence.
[Sec. IV-B, Table III] The functional-failure evaluation is potentially circular. The safety-criticality filter (TTC ≤ 5 s, lateral velocity ≥ 2 m/s within 12 m, inside a 30 m ego radius, followed by an 11-frame window) uses proximity and kinematic criteria that are exactly aligned with the ego-centric behavior CE is designed to reward. The reported reductions (e.g., 58→11 functional failures in 0-10 m, 397→96 in 10-20 m, 1279→510 in 20-30 m) are computed on this filtered subset without any external ground truth (e.g., nuScenes track IDs) to confirm that CE-matched pairs are correct associations and that IoU-rejected pairs are truly dangerous. The claim that CE reduces functional failures by 80% at close ranges and 60% at far ranges is therefore not yet established. The authors should validate matches against instance-level ground truth or demonstrate robustness to alternative filter definitions.
[Sec. IV-A, Table I] The correlation between CE and CPD exceeds 0.96 for all object categories, which indicates that CE carries little additional information beyond CPD in the majority of cases. The manuscript argues that the metrics are "conceptually distinct" and supports this with edge-case examples, but the aggregate experiments in Tables III-V compare CE and CPD on data where the two metrics largely agree. To substantiate the claim that CE is more reliable than CPD, the authors should quantify the fraction of match decisions on which CE and CPD disagree and show, using ground-truth associations, that CE's decisions are the correct ones in those disagreement cases.

minor comments (5)

[Throughout] There are several typos, including "we introduce2D and 3D Contour Errors" (missing space) and "To access it" (should be "To assess it") in Sec. IV-E. The submitted PDF also contains stray "INTERNAL" watermarks and formatting artifacts that should be removed.
[Fig. 5 caption and Sec. IV-B] The Figure 5 caption says "Each row shows two seconds of motion," but the text in Sec. IV-B describes an 11-frame (5 s) analysis window. Please make the temporal duration consistent between the caption and the text.
[Sec. IV-E and Fig. 6] The KITTI evaluation reports only detection average precision (IoU-AP vs CE-AP) and does not include tracking metrics, despite the paper's stated focus on multi-object tracking. Clarify the scope or add tracking results on KITTI.
[Table II] The column headers use mixed symbols (≤ and >) without an explicit explanation of which condition corresponds to a 'reliable match' versus a 'contour-based match.' Please spell out the four conditions in the text or table caption for clarity.
[References] Several references contain formatting errors, such as stray spacing in [26] and inconsistent presentation of author lists. The reference list should be carefully proofread.

Circularity Check

2 steps flagged · score 5.0 of 10

CE's 'functional' advantage rests on self-labeled matches and thresholds fitted on the same nuScenes data; scatter plots and KITTI AP provide independent support.

self definitional [Sec. IV-A, Table II discussion]
"We determine these missed IoU matches as critical contour-based matches, as shown in Tab. II. We evaluate them as true associations from the perspective of AD driving functionality by utilizing the geometric properties of CEs (e.g., relative distance and orientation to the ego vehicle)."

The paper defines a set of matches by the CE threshold, then labels those matches as 'critical contour-based matches' and 'true associations' using CE's own geometric properties (relative distance/orientation). The conclusion that CE-valid matches are correct associations that IoU wrongly discards is therefore built into the labeling convention, not established by an external ground truth (e.g., nuScenes track IDs). The safety filter in Sec. IV-B supplies kinematics, but the actual functional TP/failure counts (Tab. III) are threshold-based counts of CE/IoU/CPD matches, so the superiority of CE on 'functional' matches is partly self-definitional.
fitted input called prediction [Sec. IV-A, threshold selection and Tab. II]
"To determine object-category-specific optimal thresholds for contour error (CE) association, we conduct a sensitivity analysis to maximize tracking performance (mHOTA and Recall)... This methodology yields the following optimal CE thresholds: 0.75 m for pedestrians, 2.5 m for cars, and 4.0 m for trucks."

The CE thresholds are optimized on the same nuScenes validation data that is later used to report '33.1%, 43.6%, and 47.7% of all matches within these CE thresholds do not satisfy the IoU matching criteria' and the abstract's '47% of car and 75% of pedestrian CE-valid matches are rejected by IoU'. These rejection rates are properties of the fitted operating point, not independent predictions. The paper's threshold-independent scatter plots (Figs. 3-4) partially mitigate this, but the headline quantitative claims are computed at thresholds chosen to maximize CE's HOTA/Recall on the same data.

full rationale

The paper's core metric (Eq. 1) is a concrete, well-defined distance (symmetric max-min Hausdorff over the k ego-nearest corners), and the threshold-independent scatter plots (Figs. 3-4) and the KITTI CE-AP comparison (Fig. 6) are legitimate external evidence that CE behaves differently from IoU/CPD. However, the central quantitative claims of the abstract and Sec. IV-A—'47% of car and 75% of pedestrian CE-valid matches rejected by IoU' and '33.1%, 43.6%, 47.7%' in Table II—are computed at CE thresholds fitted on the same nuScenes data to maximize mHOTA/Recall, and the paper then labels CE-valid matches as 'true associations' by using CE's own geometric properties. The 'functional failure' reductions (80%/60%) are additionally computed on a subset biased toward close, yaw-misaligned, high-lateral-velocity events (Secs. IV-B/IV-C), so the selection reinforces CE's design. No instance-level ground truth (track IDs) is used to verify that the CE-accepted associations are the correct physical object; the 'functional TP' label is assigned by the metric itself. These issues make the headline evaluation partially circular, though not the metric definition. The abstract's 75% pedestrian figure also conflicts with Table II's 33.1%, an internal inconsistency that further weakens the quantitative claim.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical entities. The free parameters are the CE thresholds and the safety filter, both fitted or hand-chosen, and the k-corner selection rule. These carry much of the reported advantage.

free parameters (3)

CE association threshold per object category = 0.75 m (pedestrian), 2.5 m (car), 4.0 m (truck)
Tuned via sensitivity analysis to maximize mHOTA and recall, with an upper bound at 50% of the object diagonal (Sec. IV-A). These thresholds determine the headline rejection percentages.
Safety-critical filter thresholds = TTC <= 5 s, lateral velocity >= 2 m/s within 12 m, 30 m ego radius, minimum 10 frames, yaw error > 10 deg
Hand-crafted rules in Sec. IV-B and IV-C that define 'functional failures' and select the evaluation subset; the reported advantage depends on them.
Number of ego-nearest corners k = 3 in 2D, 6 in 3D
A design choice stated in Sec. III and Algorithm 1 without derivation.

assumptions (4)

standard math Hausdorff distance and symmetric max-min distance are valid shape similarity measures
CE is defined as a modified Hausdorff distance; the paper assumes the metric properties without proof.
domain assumption The corners closest to the ego vehicle are the most safety-relevant
Sec. III states the ego-centric version focuses on the nearest bounding box corners relative to the ego center; this is plausible but unvalidated.
domain assumption The safety-critical filter correctly labels dangerous mismatches
Sec. IV-B uses TTC and lateral velocity thresholds to remove 'benign' mismatches; this filter is not independently validated.
ad hoc to paper The 50% diagonal upper bound prevents physically implausible associations
Sec. IV-A imposes this bound to constrain threshold selection; it is arbitrary and shapes the results.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Contour Errors: Ego-Centric Matching for 3D Multi-Object Tracking Performance Evaluation." pith.science (2026). https://pith.science/paper/6YSJJOIU

@misc{pith2026250604122,
  author       = {Pith},
  title        = {Pith review of: Contour Errors: Ego-Centric Matching for 3D Multi-Object Tracking Performance Evaluation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6YSJJOIU}},
  note         = {Machine review of arXiv:2506.04122}
}

read the original abstract

Open-loop performance evaluation of 3D multi-object tracking in autonomous driving requires matching criteria that effectively penalize translational, shape, and orientation errors from the ego vehicle perspective. The prevailing criteria for determining true positives are Intersection over Union (IoU) and Centre-Point Distances (CPD). When IoU is extended from the 2D image plane to 3D volumetric overlap, it often falls below its acceptance threshold even with minor yaw misalignments, whereas CPD disregards orientation entirely. To address this limitation, we propose Contour Errors (CE) as an ego-centric criterion that employs Hausdorff-type reasoning to sparse bounding-box corner geometry by selecting the k-nearest ego-centric corners. This method provides a graded orientation sensitivity between the extremes of IoU, which overpenalizes, and CPD, which is orientation-blind. We evaluate Contour Errors against six baselines using the HOTA evaluation protocol on the nuScenes dataset, conditioned on proximity, yaw error, and a confidence threshold. At the standard IoU vehicle threshold, 47% of car and 75% of pedestrian CE-valid matches are rejected by IoU despite close contour proximity, while fewer than 0.1% of IoU-valid matches fail CE. These results establish the ego-centric matching criterion as a primary factor for improving open-loop perception evaluation in safety-critical autonomous driving.

Figures

Figures reproduced from arXiv: 2506.04122 by the authors.

**Figure 1.** Motivation for ego-centric Contour Error (CE) association. This figure illustrates scenarios where ego-centric CE association provides more robust [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

**Figure 2.** Computation of the ego-centric Contour Error (CE) metric for MOT [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Scatter plots of IoU vs. CE for all matches within [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Scatter plots of CPD vs. CE for all matches within [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

**Figure 5.** Figure 5: Qualitative comparison of our ego-centric Contour Error (CE) and object-centric metrics (IoU/CPD) in four different safety-critical interactions. Each [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

**Figure 6.** Figure 6: IoU-AP and Contour Error AP (CE-AP) across distance breakdowns for the car category on the KITTI validation set. CE-AP provides better [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

33 extracted references · 28 canonical work pages

[1]

Large scale interactive motion forecasting for autonomous driving : The waymo open motion dataset,

S. Ettinger,et al., “Large scale interactive motion forecasting for autonomous driving : The waymo open motion dataset,”arXiv preprint arXiv:2104.10133, 2021

arXiv 2021
[2]

nuscenes: A multimodal dataset for autonomous driving,

H. Caesar,et al., “nuscenes: A multimodal dataset for autonomous driving,” inProc. of the IEEE Conf. on Comp. Vis. and Pattern Recog., 2020, pp. 11 621–11 631

work page 2020
[3]

Robust object detection using knowledge graph embeddings,

C. Lang, A. Braun, and A. Valada, “Robust object detection using knowledge graph embeddings,” inDAGM German Conference on Pattern Recognition, 2022, pp. 445–461

work page 2022
[4]

Self-supervised multi-object tracking for autonomous driving from consistency across timescales,

C. Lang, A. Braun, L. Schillingmann, and A. Valada, “Self-supervised multi-object tracking for autonomous driving from consistency across timescales,”IEEE Robotics and Automation Letters, 2023

work page 2023
[5]

A survey on performance metrics for object-detection algorithms,

R. Padilla, S. L. Netto, and E. A. Da Silva, “A survey on performance metrics for object-detection algorithms,” inInt. conf. on systems, signals and image processing, 2020, pp. 237–242

work page 2020
[6]

Evaluating multiple object tracking performance: the clear mot metrics,

K. Bernardin and R. Stiefelhagen, “Evaluating multiple object tracking performance: the clear mot metrics,”EURASIP Journal on Image and Video Processing, vol. 2008, pp. 1–10, 2008

2008
[7]

Tracking the Trackers: An Analysis of the State of the Art in Multiple Object Tracking

L. Leal-Taixé, A. Milan, K. Schindler, D. Cremers, I. Reid, and S. Roth, “Tracking the trackers: an analysis of the state of the art in multiple object tracking,”arXiv preprint arXiv:1704.02781, 2017

work page Pith review arXiv 2017
[8]

3d multi-object tracking using graph neural networks with cross-edge modality attention,

M. Büchner and A. Valada, “3d multi-object tracking using graph neural networks with cross-edge modality attention,”IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 9707–9714, 2022

work page 2022

Show all 33 references

[9]

Revisiting 3d object detection from an egocentric perspective,

B. Deng, C. R. Qi, M. Najibi, T. Funkhouser, Y . Zhou, and D. Anguelov, “Revisiting 3d object detection from an egocentric perspective,”Proc. of the Conf. on Neural Information Processing Systems, 2021

2021
[10]

Ec-iou: Orienting safety for object detectors via ego-centric intersection-over-union,

B. H.-C. Liao, C.-H. Cheng, H. Esen, and A. Knoll, “Ec-iou: Orienting safety for object detectors via ego-centric intersection-over-union,” arXiv preprint arXiv:2403.15474, 2024

2024 arXiv
[11]

Weighted intersection over union (wiou) for evaluating image segmentation,

Y .-J. Cho, “Weighted intersection over union (wiou) for evaluating image segmentation,”Pattern Recognition Letters, vol. 185, 2024

2024
[12]

Evcenternet: Uncertainty estimation for object detection using evidential learning,

M. R. Nallapareddy, K. Sirohi, P. L. Drews, W. Burgard, C.-H. Cheng, and A. Valada, “Evcenternet: Uncertainty estimation for object detection using evidential learning,” inInt. Conf. on Intelligent Robots and Systems, 2023, pp. 5699–5706

2023
[13]

On hyperbolic embeddings in object detection,

C. Lang, A. Braun, L. Schillingmann, and A. Valada, “On hyperbolic embeddings in object detection,” inDAGM German Conference on Pattern Recognition, 2022, pp. 462–476

2022
[14]

Are we ready for autonomous driving? the kitti vision benchmark suite,

A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? the kitti vision benchmark suite,” inProc. of the IEEE Conf. on Comp. Vis. and Pattern Recog., 2012, pp. 3354–3361

2012
[15]

Scalability in perception for autonomous driving: Waymo open dataset,

P. Sun,et al., “Scalability in perception for autonomous driving: Waymo open dataset,” inProc. of the IEEE Conf. on Comp. Vis. and Pattern Recog., 2020, pp. 2446–2454

2020
[16]

Addressing the gaps of iou loss in 3d object detection with iiou,

N. Ravi and M. El-Sharkawy, “Addressing the gaps of iou loss in 3d object detection with iiou,”Future Internet, 2023

2023
[17]

Shape-iou: More accurate metric considering bounding box shape and scale,

H. Zhang and S. Zhang, “Shape-iou: More accurate metric considering bounding box shape and scale,”arXiv preprint arXiv:2312.17663, 2023

2023 arXiv
[18]

Com- paring images using the hausdorff distance,

D. P. Huttenlocher, G. A. Klanderman, and W. J. Rucklidge, “Com- paring images using the hausdorff distance,”IEEE Trans. on pattern analysis and machine intelligence, vol. 15, no. 9, pp. 850–863, 2002

2002
[19]

A modified hausdorff distance for object matching,

M.-P. Dubuisson and A. K. Jain, “A modified hausdorff distance for object matching,” inProc. of the IEEE Conf. on Comp. Vis. and Pattern Recog., vol. 1, 1994, pp. 566–568

1994
[20]

Balanced chamfer distance as a comprehensive metric for point cloud completion,

T. Wu, L. Pan, J. Zhang, T. Wang, Z. Liu, and D. Lin, “Balanced chamfer distance as a comprehensive metric for point cloud completion,” Adv. in Neural Information Processing Systems, 2021

2021
[21]

Deepsdf: Learning continuous signed distance functions for shape representation,

J. J. Park, P. Florence, J. Straub, R. Newcombe, and S. Lovegrove, “Deepsdf: Learning continuous signed distance functions for shape representation,” inProc. of the IEEE Conf. on Comp. Vis. and Pattern Recog., 2019, pp. 165–174

2019
[22]

A novel benchmarking paradigm and a scale-and motion- aware model for egocentric pedestrian trajectory prediction,

A. Rasouli, “A novel benchmarking paradigm and a scale-and motion- aware model for egocentric pedestrian trajectory prediction,” in Int. Conf. on Robotics and Automation, 2024, pp. 5630–5636

2024
[23]

Evaluation metrics for object detection for autonomous systems,

A. Badithela, T. Wongpiromsarn, and R. M. Murray, “Evaluation metrics for object detection for autonomous systems,”arXiv preprint arXiv:2210.10298, 2022

2022 arXiv
[24]

Object criticality for safer navigation,

A. Ceccarelli and L. Montecchi, “Object criticality for safer navigation,” arXiv preprint arXiv:2406.10232, 2024

2024 arXiv
[25]

Learning to evaluate perception models using planner-centric metrics,

J. Philion, A. Kar, and S. Fidler, “Learning to evaluate perception models using planner-centric metrics,” inProc. of the IEEE Conf. on Comp. Vis. and Pattern Recog., 2020, pp. 14 055–14 064

2020
[26]

Usc: Uncompromising spatial constraints for safety-oriented 3d object detectors in autonomous driving,

B. Hsuan-Cheng Liao, C.-H. Cheng, H. Esen, and A. Knoll, “Usc: Uncompromising spatial constraints for safety-oriented 3d object detectors in autonomous driving,”arXiv preprint:2209.10368, 2022

2022 arXiv
[27]

Safety-aware metrics for object detectors in autonomous driving

A. Ceccarelli and L. Montecchi, “Safety-aware metrics for object detectors in autonomous driving.”arXiv preprint:2203.02205, 2022

2022 arXiv
[28]

Injecting planning-awareness into prediction and detection evaluation,

B. Ivanovic and M. Pavone, “Injecting planning-awareness into prediction and detection evaluation,” inIEEE Intelligent V ehicles Symposium, 2022, pp. 821–828

2022
[29]

Ab3dmot: A baseline for 3d multi-object tracking and new evaluation metrics,

X. Weng, J. Wang, D. Held, and K. Kitani, “Ab3dmot: A baseline for 3d multi-object tracking and new evaluation metrics,”arXiv preprint arXiv:2008.08063, 2020

2008 arXiv
[30]

Criticality metric for the safety validation of automated driving using model predictive trajectory optimization,

P. Junietz, F. Bonakdar, B. Klamann, and H. Winner, “Criticality metric for the safety validation of automated driving using model predictive trajectory optimization,” inInt. conf. on intelligent transportation systems, 2018, pp. 60–65

2018
[31]

Pointpillars: Fast encoders for object detection from point clouds,

A. H. Lang, S. V ora, H. Caesar, L. Zhou, J. Yang, and O. Beijbom, “Pointpillars: Fast encoders for object detection from point clouds,” in Proc. of the IEEE Conf. on Comp. Vis. and Pattern Recog., 2019

2019
[32]

Pointrcnn: 3d object proposal generation and detection from point cloud,

S. Shi, X. Wang, and H. Li, “Pointrcnn: 3d object proposal generation and detection from point cloud,” inProc. of the IEEE Conf. on Comp. Vis. and Pattern Recog., 2019

2019
[33]

Pv-rcnn: Point-voxel feature set abstraction for 3d object detection,

S. Shi,et al., “Pv-rcnn: Point-voxel feature set abstraction for 3d object detection,” inProc. of the IEEE Conf. on Comp. Vis. and Pattern Recog., 2020, pp. 10 529–10 538

2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Large scale interactive motion forecasting for autonomous driving : The waymo open motion dataset,

S. Ettinger,et al., “Large scale interactive motion forecasting for autonomous driving : The waymo open motion dataset,”arXiv preprint arXiv:2104.10133, 2021

arXiv 2021

[2] [2]

nuscenes: A multimodal dataset for autonomous driving,

H. Caesar,et al., “nuscenes: A multimodal dataset for autonomous driving,” inProc. of the IEEE Conf. on Comp. Vis. and Pattern Recog., 2020, pp. 11 621–11 631

work page 2020

[3] [3]

Robust object detection using knowledge graph embeddings,

C. Lang, A. Braun, and A. Valada, “Robust object detection using knowledge graph embeddings,” inDAGM German Conference on Pattern Recognition, 2022, pp. 445–461

work page 2022

[4] [4]

Self-supervised multi-object tracking for autonomous driving from consistency across timescales,

C. Lang, A. Braun, L. Schillingmann, and A. Valada, “Self-supervised multi-object tracking for autonomous driving from consistency across timescales,”IEEE Robotics and Automation Letters, 2023

work page 2023

[5] [5]

A survey on performance metrics for object-detection algorithms,

R. Padilla, S. L. Netto, and E. A. Da Silva, “A survey on performance metrics for object-detection algorithms,” inInt. conf. on systems, signals and image processing, 2020, pp. 237–242

work page 2020

[6] [6]

Evaluating multiple object tracking performance: the clear mot metrics,

K. Bernardin and R. Stiefelhagen, “Evaluating multiple object tracking performance: the clear mot metrics,”EURASIP Journal on Image and Video Processing, vol. 2008, pp. 1–10, 2008

2008

[7] [7]

Tracking the Trackers: An Analysis of the State of the Art in Multiple Object Tracking

L. Leal-Taixé, A. Milan, K. Schindler, D. Cremers, I. Reid, and S. Roth, “Tracking the trackers: an analysis of the state of the art in multiple object tracking,”arXiv preprint arXiv:1704.02781, 2017

work page Pith review arXiv 2017

[8] [8]

3d multi-object tracking using graph neural networks with cross-edge modality attention,

M. Büchner and A. Valada, “3d multi-object tracking using graph neural networks with cross-edge modality attention,”IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 9707–9714, 2022

work page 2022