{"id":"e23ab284-c0c7-4cf5-8854-af68ea69ae71","arxiv_id":"2506.04122","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A modified Hausdorff distance on ego-nearest bounding box corners is proposed as a 3D tracking matching criterion, showing more robust matches than IoU or center-point distance.","lead":"A new matching metric, Contour Error, judges whether a predicted 3D box matches the true object by measuring distances between the bounding box corners closest to the self-driving car. The authors claim it captures orientation and shape errors better than standard IoU overlap or center-point distance, potentially making autonomous driving evaluation more safety-relevant.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The functional-failure evaluation in Sec. IV-B lacks an external ground truth for match correctness; the criticality filter is defined by proximity/kinematics that CE directly rewards, so the claimed 80%/60% reductions may be a selection artifact.","rationale":"The reader's weakest-assumption analysis identifies the safety-criticality filter in Sec. IV-B as the load-bearing premise, and my independent reading converges on the same point. The paper's headline quantitative claim is the 80%/60% reduction in functional failures, but that reduction is computed on a scene subset selected by kinematic rules (TTC <= 5 s, lateral velocity >= 2 m/s within 12 m, 30 m ego radius). These rules are not validated against any authoritative notion of dangerous mismatch; they are simply chosen to isolate safety-relevant cases. Since CE is itself defined as a distance between the ego vehicle and nearby box corners, the filter preferentially retains cases where CE will be small for spatially close pairs, making it unsurprising that CE 'recovers' more matches than IoU. Meanwhile, CPD also performs well in Table III because it, too, is a distance metric, further suggesting that the advantage is distance-driven rather than shape-orientation-driven. The absence of instance-level association ground truth is the critical gap: without knowing whether a CE-matched pair is the same object, one cannot tell if CE is correctly saving true positives or merely admitting geometrically near false positives. The internal inconsistency between the abstract (75% pedestrian) and Table II (33.1%) is a secondary reliability concern but does not change the central objection. The proposed concrete test—using nuScenes instance IDs as labels and recomputing same-instance FP/FN both globally and on the filtered subset—would settle whether the claimed advantage is genuine. Until such a test is performed or the authors provide an alternative external validation, the conditional verdict is appropriate, and I see no reason to change it based on this stress-test pass.","tokens_in":13266,"tokens_out":3282,"duration_ms":33476,"concrete_test":"Recompute the matching evaluation on the nuScenes validation set using ground-truth instance IDs as the definitive label for correct association. For each metric (3D IoU at 0.4, CPD at 2.5 m, CE at 2.5 m), compute the Hungarian assignment per frame, label each TP/FP/FN by whether the matched pair shares the same ground-truth instance ID, and then recompute functional failure rates (a) on the full set and (b) on the Sec. IV-B filtered scenes. If CE's advantage over IoU/CPD in reducing same-instance FP/FN does not reproduce, the 80%/60% claim is an artifact of the hand-designed criticality filter rather than a property of the metric. This test requires no new data—only the specified tracker outputs and the nuScenes instance labels.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central empirical claim—that CE reduces functional failures by 80% at close ranges and 60% at far ranges compared to IoU—depends on the definition of a 'functional' true positive and false positive. In Sec. IV-B, a safety-critical failure is defined by TTC <= 5 s or lateral velocity >= 2 m/s within 12 m, inside a 30 m ego radius, and the retained pairs are then scored by a criticality score that favors imminent longitudinal impact and high-speed lateral maneuvers. These criteria are geometric/kinematic, and they are exactly the conditions under which CE—an ego-centric distance over the nearest bounding-box corners—will naturally associate nearby boxes with small position error. IoU rejects many such pairs because of yaw misalignment, and CPD ignores orientation, but the paper labels CE's additional associations as 'functional TPs' without verifying that the predicted box actually corresponds to the same physical object. No instance-level ground truth (e.g., nuScenes track IDs) is used to confirm that the CE-matched pairs are correct associations and the IoU-rejected pairs are truly dangerous false negatives. The reported reduction in functional failures is therefore potentially a consequence of the filter rewarding the behavior that CE is explicitly designed to exhibit, rather than evidence that CE is a more reliable matching criterion. Additionally, the abstract's '75% of pedestrian CE-valid matches rejected by IoU' conflicts with Table II's 33.1%, further weakening confidence in the reported statistics. The central claim would be supported only if CE's advantage persists when match correctness is judged by an independent source of truth, not by proximity-based heuristics.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Contour Error (CE), an ego-centric matching criterion for 3D multi-object tracking evaluation. CE is defined as a symmetric max-min Hausdorff-type distance between the k ego-nearest corners of ground-truth and predicted bounding boxes. The authors evaluate CE against IoU, CPD, and other baselines using the HOTA protocol on nuScenes and KITTI, and report that at standard IoU thresholds 33-47% of CE-valid matches are rejected by IoU while almost no IoU-valid matches fail CE. They also claim a reduction of functional failures by 80% at close ranges and 60% at far ranges, based on a kinematics-based safety-criticality filter.","tokens_in":13600,"tokens_out":6595,"duration_ms":58570,"significance":"If substantiated, an ego-centric matching metric could be valuable for safety-oriented evaluation of 3D MOT, because IoU over-penalizes yaw misalignment and CPD ignores orientation. The paper contributes a threshold-independent scatter-plot analysis that is useful for understanding metric behavior. However, the current manuscript does not yet establish the claimed advantages: the metric definition is internally inconsistent, the headline numbers conflict with the reported tables, and the main empirical claims rely on thresholds tuned on the test set and a safety-critical filter that may select for the behavior the metric is designed to exhibit. The high correlation between CE and CPD (r>0.96) also raises a question about the incremental contribution over CPD.","major_comments":[{"comment":"The metric definition is internally inconsistent. Equation (1) defines distances from selected corners p in P'_j and g in G'_i to points on the opposite bounding-box surfaces (x in X_i, y in Y_j), but Algorithm 1 (lines 6 and 11) computes the nearest point by taking argmin over the corner sets G_i and P_j, not over the surfaces. These two definitions yield different distances: a corner-to-surface distance can be small for a slightly rotated box, whereas a corner-to-corner distance will be much larger. The number of selected corners is also inconsistent: the text after Eq. (1) says \"subset of three corners\" for both directions, while Algorithm 1 says \"three and six closest corners\" in 2D and 3D. The authors must specify the metric unambiguously and ensure the algorithm implements exactly that definition, because the metric is the paper's central contribution.","section":"Sec. III, Eq. (1) and Algorithm 1"},{"comment":"The abstract states that \"47% of car and 75% of pedestrian CE-valid matches are rejected by IoU,\" but Table II reports 43.6% for car (CE≤threshold and IoU≤threshold), 33.1% for pedestrian, and 47.7% for truck. The pedestrian figure is more than a factor of two too high, and the car figure does not match 47%. These numbers are the primary quantitative claims of the paper, so the abstract must be corrected to match the table, or the table must be recomputed if the abstract values are the intended result.","section":"Abstract and Sec. IV-A, Table II"},{"comment":"The category-specific CE thresholds are optimized on the same nuScenes validation set used to demonstrate CE's advantage: the text says \"we optimize separate thresholds for each object category\" and \"sensitivity analysis to maximize tracking performance (mHOTA and Recall).\" The rejection percentages in Table II and the functional-failure reductions in Table III are then reported at these fitted thresholds. This makes the quantitative comparisons vulnerable to overfitting and circularity. The authors should use a held-out validation split or report results as a continuous function of threshold, with the scatter plots (Figs. 3 and 4) as the primary threshold-independent evidence.","section":"Sec. IV-A"},{"comment":"The functional-failure evaluation is potentially circular. The safety-criticality filter (TTC ≤ 5 s, lateral velocity ≥ 2 m/s within 12 m, inside a 30 m ego radius, followed by an 11-frame window) uses proximity and kinematic criteria that are exactly aligned with the ego-centric behavior CE is designed to reward. The reported reductions (e.g., 58→11 functional failures in 0-10 m, 397→96 in 10-20 m, 1279→510 in 20-30 m) are computed on this filtered subset without any external ground truth (e.g., nuScenes track IDs) to confirm that CE-matched pairs are correct associations and that IoU-rejected pairs are truly dangerous. The claim that CE reduces functional failures by 80% at close ranges and 60% at far ranges is therefore not yet established. The authors should validate matches against instance-level ground truth or demonstrate robustness to alternative filter definitions.","section":"Sec. IV-B, Table III"},{"comment":"The correlation between CE and CPD exceeds 0.96 for all object categories, which indicates that CE carries little additional information beyond CPD in the majority of cases. The manuscript argues that the metrics are \"conceptually distinct\" and supports this with edge-case examples, but the aggregate experiments in Tables III-V compare CE and CPD on data where the two metrics largely agree. To substantiate the claim that CE is more reliable than CPD, the authors should quantify the fraction of match decisions on which CE and CPD disagree and show, using ground-truth associations, that CE's decisions are the correct ones in those disagreement cases.","section":"Sec. IV-A, Table I"}],"minor_comments":[{"comment":"There are several typos, including \"we introduce2D and 3D Contour Errors\" (missing space) and \"To access it\" (should be \"To assess it\") in Sec. IV-E. The submitted PDF also contains stray \"INTERNAL\" watermarks and formatting artifacts that should be removed.","section":"Throughout"},{"comment":"The Figure 5 caption says \"Each row shows two seconds of motion,\" but the text in Sec. IV-B describes an 11-frame (5 s) analysis window. Please make the temporal duration consistent between the caption and the text.","section":"Fig. 5 caption and Sec. IV-B"},{"comment":"The KITTI evaluation reports only detection average precision (IoU-AP vs CE-AP) and does not include tracking metrics, despite the paper's stated focus on multi-object tracking. Clarify the scope or add tracking results on KITTI.","section":"Sec. IV-E and Fig. 6"},{"comment":"The column headers use mixed symbols (≤ and >) without an explicit explanation of which condition corresponds to a 'reliable match' versus a 'contour-based match.' Please spell out the four conditions in the text or table caption for clarity.","section":"Table II"},{"comment":"Several references contain formatting errors, such as stray spacing in [26] and inconsistent presentation of author lists. The reference list should be carefully proofread.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the journal and the general idea of an ego-centric matching metric is worth exploring. However, the central empirical claims are not yet supported: the metric definition is ambiguous, the abstract numbers conflict with the tables, and the evaluation procedure (threshold tuning on the test set and a safety-critical filter that may be circular) does not convincingly establish the advantages over CPD. I would not reject the paper, because the issues are fixable within the manuscript's scope, but a major revision is needed to clarify the metric, correct the numbers, and add ground-truth-validated or threshold-robust experiments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the CE metric is a modest but real extension of Hausdorff distance — symmetric max-min over the few bounding-box corners nearest the ego. That specific construction is not in the cited literature, and the paper is honest that any single threshold is arbitrary. The best material is threshold-independent: scatter plots and correlation tables across CE thresholds, plus the KITTI detector comparison. Those are useful and give CE a defensible place as a secondary ego-centric metric.\n\nThe soft spots are real. Eq. (1) and Algorithm 1 do not agree: Eq. (1) minimizes over surface points of the boxes, the algorithm minimizes over corner sets, and the 2D/3D corner count drifts between three and six. The abstract's '75% pedestrian' number is not in Table II, which says 33.1% at the fitted threshold. And Table I puts Corr(CE, CPD) above 0.96 for all three classes, so CE is mostly a re-parameterization of center-point distance with an orientation-sensitive twist; the paper argues they differ in edge cases, which is fair, but not a new independent signal.\n\nThe load-bearing weakness is Sec. IV-B. The functional-failure analysis uses a kinematic filter (TTC <=5 s or lateral velocity >=2 m/s within 12 m, inside 30 m) and then labels CE's extra associations as functional TPs without checking against nuScenes track IDs or any independent source of truth. The filter rewards exactly the ego-proximity geometry that CE is constructed to reward. So the 80%/60% reduction is quite plausibly a selection artifact. The threshold-independent scatter plots mitigate the circularity of the tuned thresholds, but they do not rescue the specific claim about functional failures.\n\nCitations look fine: Hausdorff, Chamfer, ego-centric IoU, and the KITTI/nuScenes evaluation baselines are all there. No code is released, which is a smaller issue but worth noting for reproducibility.\n\nFor whom: anyone building 3D MOT evaluation criteria, or benchmarking trackers in safety-critical settings, should read it, with caution. It deserves a serious referee — the metric is plausible and the threshold-independent analysis is a real contribution — but the referee should require instance-level validation of the functional-failure claims and a fix of the Eq./Algorithm mismatch before acceptance.","headline":"A plausible ego-centric Hausdorff-style matching metric whose headline safety-critical claim is undermined by a filter that shares the metric's geometry; still worth a serious referee.","tokens_in":14101,"tokens_out":4322,"would_cite":true,"duration_ms":42735,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A contour-based matching distance for 3D tracking keeps matches that IoU rejects, cutting functional failures by up to 80 percent.","keywords":["multi-object tracking","3D bounding box","ego-centric evaluation","Hausdorff distance","matching criterion","autonomous driving","performance evaluation"],"falsifier":"Re-run the nuScenes evaluation with the safety filter replaced by a planner-based or learned criticality model; if CE's reported 80% and 60% functional-failure reductions against IoU shrink or reverse, the advantage is an artifact of the filter. A complementary check is to have human raters label a set of cut-in and intersection matches: if many CE-accepted, IoU-rejected matches are judged incorrect, then CE is too permissive.","tokens_in":13087,"feed_emoji":"🚗","tokens_out":8387,"duration_ms":68884,"temperature":0.7,"pith_summary":"The paper claims that the standard matching criteria for 3D multi-object tracking, volumetric Intersection-over-Union (IoU) and Center-Point Distance (CPD), are unreliable from the perspective of the ego vehicle, and it introduces Contour Error (CE) as a replacement. CE is a symmetric max-min Hausdorff-style distance computed only on the bounding-box corners nearest to the ego vehicle, so it penalizes yaw and shape errors that CPD ignores without collapsing on minor rotations the way IoU does. On the nuScenes dataset the authors report that 33–47% of car, pedestrian, and truck matches accepted by CE are rejected by IoU at the standard threshold, while almost no IoU-valid match fails CE. In safety-critical scenes selected by a time-to-collision filter, CE reduces functional false positives and false negatives by 80% at close range and 60% at far range compared to IoU. The consequence is that the matching criterion itself, not just the tracker, controls apparent tracking quality.","feed_headline":"Ego-centric contour metric cuts 3D tracking failures by 80%","feed_subtitle":"A Hausdorff-style contour measure keeps ego-close matches that IoU rejects, improving safety-critical MOT evaluation.","key_machinery":"The central object is the Contour Error distance, a hand-crafted ego-centric variant of the Hausdorff distance applied to sparse corner geometry. For each candidate match, the metric selects the k corners of each bounding box that are closest to the ego vehicle's position, then takes the maximum over those corners of the minimum Euclidean distance to the other box's surface, symmetrically in both directions. Restricting to the ego-nearest corners is what carries the argument: it makes the measure perspective-aware, so it focuses on the side of the object that faces the ego vehicle and matters for collision. This one change yields the desired graded sensitivity between IoU and CPD, and the distance matrix is then fed to the Hungarian algorithm for global assignment.","core_discovery":"The central discovery is that a matching criterion can be made ego-centric by restricting the comparison to the k corners of each box that face the ego vehicle. The paper defines CE as \\max\\left(\\max_{p\\in P'_j}\\min_{x\\in X_i}\\lVert p-x\\rVert,\\ \\max_{g\\in G'_i}\\min_{y\\in Y_j}\\lVert g-y\\rVert\\right), where $G'_i$ and $P'_j$ are the three (2D) or six (3D) ego-nearest corners of the ground-truth and predicted boxes, and a match holds when the value is below a category-specific threshold. Empirically, this criterion sits between IoU and CPD: it is orientation-aware enough to flag yaw misalignments that CPD misses, yet stable enough to accept near-correct matches that IoU's volume overlap rejects. The paper demonstrates this with threshold-independent scatter plots, tuned HOTA evaluations on nuScenes, and an ego-centric AP comparison of detectors on KITTI, and it argues that CE's main payoff appears once evaluation is restricted to safety-critical interactions.","pith_inferences":["If CE becomes a standard association cost, trackers tuned to it may learn to align the ego-facing side of a box rather than the whole volume, which could yield geometry that is more directly usable by downstream planners.","The k-nearest-corner idea transfers naturally to other ego-perspective problems, such as evaluating 2D-3D fusion or motion forecasting, where measuring distance on the visible corners may be more meaningful than full-shape similarity.","The high disagreement rate between CE and IoU implies that current benchmarks may undercount true positives in near-miss scenarios, so reported false-negative rates on existing leaderboards could be inflated; a re-evaluation with CE or a hybrid criterion would test this directly."],"forward_implications":["Open-loop 3D MOT evaluation should report results under more than one matching criterion, because IoU and CE disagree on a large share of matches near safety-critical thresholds.","Switching the association cost from IoU to CE changes the false-positive and false-negative counts of the same tracker, so tracker scores and rankings depend on the matching criterion as much as on the tracker.","Ego-centric CE-AP reshuffles detector rankings on KITTI compared to IoU-AP, revealing near-field and far-field strengths that IoU-based benchmarking masks.","Category-specific CE thresholds (0.75 m for pedestrians, 2.5 m for cars, 4.0 m for trucks) provide a concrete starting point for standardizing safety-oriented evaluation on nuScenes."],"supporting_citations":[{"why":"Provides the nuScenes dataset used for the main HOTA evaluation and for category-specific threshold tuning.","marker":"[2]"},{"why":"Defines the IoU baseline matching criterion that CE is compared against.","marker":"[5]"},{"why":"Defines the Center-Point Distance baseline for tracking association.","marker":"[6]"},{"why":"Supplies the KITTI benchmark, its IoU thresholds, and the detector comparison used for CE-AP.","marker":"[14]"},{"why":"Introduces the Hausdorff distance that the CE formulation is built on.","marker":"[18]"},{"why":"Supplies the baseline tracker whose predictions are evaluated on both datasets.","marker":"[29]"},{"why":"Defines time-to-collision criticality used in the safety filter that selects functional-failure scenes.","marker":"[30]"}],"fun_headline_variants":["Ego-centric contour matching rescues 47% of car matches from IoU rejection","Contour Errors: ego-centric criterion that sees what IoU misses","New metric keeps ego-close matches that IoU wrongly rejects","Hausdorff-style contour metric for safety-critical 3D tracking evaluation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the hand-crafted safety-criticality filter—time-to-collision of 5 seconds or less, or lateral velocity of at least 2 m/s within 12 meters and inside a 30-meter ego radius, followed by an 11-frame analysis window—correctly identifies which mismatches are dangerous.","fun_headline_variants_meta":{"raw":{"variants":["Ego-centric contour matching rescues 47% of car matches from IoU rejection","Contour Errors: ego-centric criterion that sees what IoU misses","New metric keeps ego-close matches that IoU wrongly rejects","Hausdorff-style contour metric for safety-critical 3D tracking evaluation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000719,"raw_usage":{"total_tokens":3274,"prompt_tokens":1037,"completion_tokens":2237,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":653,"completion_tokens_details":{"reasoning_tokens":2157}},"tokens_in":653,"tokens_out":2237,"duration_ms":17074,"temperature":1.0,"reasoning_tokens":2157,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T10:46:45.628947+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the nuScenes evaluation with the safety filter replaced by a planner-based or learned criticality model; if CE's reported 80% and 60% functional-failure reductions against IoU shrink or reverse, the advantage is an artifact of the filter. A complementary check is to have human raters label a set of cut-in and intersection matches: if many CE-accepted, IoU-rejected matches are judged incorrect, then CE is too permissive.","supporting_citations":[{"cited_title":"A survey on performance metrics for object-detection algorithms,","cited_arxiv_id":null,"evidence_quote":"Defines the IoU baseline matching criterion that CE is compared against."},{"cited_title":"Are we ready for autonomous driving? the kitti vision benchmark suite,","cited_arxiv_id":null,"evidence_quote":"Supplies the KITTI benchmark, its IoU thresholds, and the detector comparison used for CE-AP."},{"cited_title":"Com- paring images using the hausdorff distance,","cited_arxiv_id":null,"evidence_quote":"Introduces the Hausdorff distance that the CE formulation is built on."},{"cited_title":"nuscenes: A multimodal dataset for autonomous driving,","cited_arxiv_id":null,"evidence_quote":"Provides the nuScenes dataset used for the main HOTA evaluation and for category-specific threshold tuning."},{"cited_title":"Criticality metric for the safety validation of automated driving using model predictive trajectory optimization,","cited_arxiv_id":null,"evidence_quote":"Defines time-to-collision criticality used in the safety filter that selects functional-failure scenes."}],"review_version":1}