pith. sign in

arxiv: 2507.13706 · v2 · submitted 2025-07-18 · 💻 cs.CV · math.ST· stat.TH

GOSPA and T-GOSPA quasi-metrics for evaluation of multi-object tracking algorithms

\'Angel F. Garc\'ia-Fern\'andez , Jinhao Gu , Lennart Svensson , Yuxuan Xia , Jan Krej\v{c}\'i , Oliver Kost , Ond\v{r}ej Straka This is my paper

Pith reviewed 2026-05-19 03:52 UTC · model grok-4.3

classification 💻 cs.CV math.STstat.TH
keywords multi-object trackingperformance evaluationquasi-metricsGOSPAT-GOSPAmissed detectionsfalse alarmstrajectory evaluation
0
0 comments X

The pith

Quasi-metrics extend GOSPA and T-GOSPA to allow unequal penalties for missed and false objects plus asymmetric localization costs in multi-object tracking.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces two quasi-metrics for evaluating multi-object tracking algorithms. One extends the generalised optimal subpattern assignment metric to compare sets of objects, while the other extends the trajectory version to compare sets of trajectories. Both retain costs for localisation error, false objects and missed objects, with the trajectory version also charging for track switches. They differ from prior versions by permitting different penalties for missed versus false objects and by allowing localisation costs that need not be symmetric. A reader would care because many applications treat these two error types unequally, and the new measures still support derivation of similarity scores for algorithm comparison.

Core claim

The central claim is that the GOSPA and T-GOSPA formulations can be relaxed into quasi-metrics by removing the symmetry requirement on localisation costs and by allowing separate costs for the number of missed objects and the number of false objects, while the resulting functions continue to satisfy the axioms of quasi-metrics. These measures apply directly to sets of objects and to sets of trajectories. The paper shows how to convert the quasi-metrics into similarity scores and demonstrates their use by evaluating several Bayesian multi-object tracking algorithms in simulation.

What carries the argument

The GOSPA quasi-metric and T-GOSPA quasi-metric, which extend the originals by replacing symmetric localisation costs with possibly asymmetric ones and by decoupling the penalty for missed objects from the penalty for false objects.

If this is right

  • Tracking algorithms can be scored with penalties that reflect the higher cost of missing a real object compared with reporting an extra one.
  • Localisation errors can be measured with directed distances that treat overestimation and underestimation differently.
  • Similarity scores for overall performance can be obtained directly from either quasi-metric.
  • The trajectory quasi-metric continues to penalise track switches while adding the new flexibility in miss and false-object costs.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Evaluators in safety-critical domains could use the new measures to produce rankings that better match operational risk profiles.
  • Algorithm designers might derive assignment rules optimised for the asymmetric costs instead of the original symmetric ones.
  • Benchmarks built on these quasi-metrics could reveal whether existing trackers remain competitive once miss and false penalties are allowed to differ.

Load-bearing premise

The chosen cost functions for localisation, misses and false objects must still satisfy the mathematical properties required of a quasi-metric.

What would settle it

A specific collection of object positions or trajectories for which the computed value of one of the proposed quasi-metrics violates the triangle inequality.

Figures

Figures reproduced from arXiv: 2507.13706 by \'Angel F. Garc\'ia-Fern\'andez, Jan Krej\v{c}\'i, Jinhao Gu, Lennart Svensson, Oliver Kost, Ond\v{r}ej Straka, Yuxuan Xia.

Figure 1
Figure 1. Figure 1: Two estimated sets of objects y1 and y2 of the ground truth x. The dashed lines represent assignments between the elements of the ground truth and the estimate. Estimate y1 has two properly detected objects with localisation errors ∆1 and ∆2 and a false object. Estimate y2 has one properly detected object with localisation error ∆1 plus a missed object. The proof of this lemma follows the proof of how to w… view at source ↗
Figure 2
Figure 2. Figure 2: Two estimated sets of trajectories Y1 and Y2 of the ground truth X. The dashed lines represent assignments between the elements of the ground truth and the estimate. Estimate Y1 has five correct detections with localisation error ∆1, a false object and a track switch. Estimate Y2 has four properly detected objects with localisation error ∆1 and a missed object. detected objects with localisation error ∆1, … view at source ↗
Figure 3
Figure 3. Figure 3: Scenario considered the simulations. All objects appear at time [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Overall, the q-metric value decreases with time since [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 5
Figure 5. Figure 5: RMS-T-GOSPA q-metric decomposition across time. A change [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
read the original abstract

This paper introduces two quasi-metrics for performance assessment of multi-object tracking (MOT) algorithms. One quasi-metric is an extension of the generalised optimal subpattern assignment (GOSPA) metric and measures the discrepancy between sets of objects. The other quasi-metric is an extension of the trajectory GOSPA (T-GOSPA) metric and measures the discrepancy between sets of trajectories. Similar to the GOSPA-based metrics, these quasi-metrics include costs for localisation error for properly detected objects, the number of false objects and the number of missed objects. The T-GOSPA quasi-metric also includes a track switching cost. Differently from the GOSPA and T-GOSPA metrics, the proposed quasi-metrics have the flexibility of penalising missed and false objects with different costs, and the localisation costs are not required to be symmetric. We also explain how to obtain similarity score functions based on these quasi-metrics. The performance of several Bayesian MOT algorithms is assessed with the T-GOSPA quasi-metric via simulations.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 3 minor

Summary. The paper introduces two quasi-metrics for multi-object tracking evaluation: a GOSPA quasi-metric for sets of objects and a T-GOSPA quasi-metric for sets of trajectories. These extend prior GOSPA-based metrics by permitting asymmetric costs for missed versus false objects and non-symmetric localization costs, while retaining costs for localization error, false objects, missed objects, and (in the trajectory case) track switches. The manuscript supplies explicit definitions in Sections 3 and 4, self-contained elementary proofs that the functions satisfy non-negativity, identity of indiscernibles, and the triangle inequality under stated conditions on the underlying cost functions, a method for deriving similarity scores, and simulation results comparing several Bayesian MOT algorithms using the T-GOSPA quasi-metric.

Significance. If the constructions hold, the work supplies practically useful flexibility in MOT scoring by allowing application-specific error penalties, which is valuable when misses and false alarms carry unequal operational costs. Credit is due for the direct, self-contained proofs of the quasi-metric axioms in Sections 3 and 4; these are elementary yet address the central claim without hidden symmetry assumptions.

minor comments (3)
  1. [§3] §3, definition of the GOSPA quasi-metric: the precise statement of the conditions required on the localization cost function (to guarantee the triangle inequality) could be stated more explicitly as a numbered assumption to aid verification.
  2. [Simulations] Simulation section: the choice of numerical values for the miss and false-object penalties in the reported experiments should be justified or tabulated so that readers can reproduce the relative weighting used.
  3. [Abstract] Abstract: the claim that the functions are quasi-metrics would be strengthened by a one-sentence reference to the proofs provided later in the paper.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their positive assessment of the manuscript and for recommending minor revision. The referee's summary accurately reflects the paper's contributions on extending GOSPA and T-GOSPA to quasi-metrics with asymmetric missed/false penalties and non-symmetric localization costs, along with the explicit definitions, elementary proofs of the quasi-metric properties, similarity score derivation, and simulation results.

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The paper defines the GOSPA and T-GOSPA quasi-metrics via explicit constructions in Sections 3 and 4 that extend prior GOSPA/T-GOSPA metrics by allowing asymmetric miss/false penalties and non-symmetric localisation costs. It supplies direct, elementary proofs that the triangle inequality, non-negativity and identity-of-indiscernibles hold under the stated conditions on the cost functions; these proofs are self-contained and do not reduce any claimed property to a fitted parameter, self-referential equation or load-bearing self-citation. The central result is therefore a definitional extension whose validity rests on independent mathematical verification rather than circular reduction to its own inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The paper rests on standard definitions of quasi-metrics and the assumption that the proposed cost functions preserve the necessary properties while adding flexibility; no invented entities or fitted parameters are mentioned in the abstract.

axioms (1)
  • domain assumption The defined functions qualify as quasi-metrics
    Abstract presents them as quasi-metrics without detailing the verification steps.

pith-pipeline@v0.9.0 · 5757 in / 1146 out tokens · 31333 ms · 2026-05-19T03:52:06.793365+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

41 extracted references · 41 canonical work pages

  1. [1]

    Multi-object tracking in traffic environments: A systematic literature review,

    D. M. Jiménez-Bravo, A. Lozano Murciego, A. Sales Mendes, H. Sánchez San Blás, and J. Bajo, “Multi-object tracking in traffic environments: A systematic literature review,” Neurocomputing, vol. 494, pp. 43–55, 2022

    work page 2022

  2. [2]

    Bayesian tracking in underwater wireless sensor networks with port-starboard ambiguity,

    P. Braca, P. Willett, K. LePage, S. Marano, and V . Matta, “Bayesian tracking in underwater wireless sensor networks with port-starboard ambiguity,” IEEE Transactions on Signal Processing , vol. 62, no. 7, pp. 1864–1878, Apr. 2014

    work page 2014

  3. [3]

    A new multi-target tracking algorithm for a large number of orbiting objects,

    E. Delande, J. Houssineau, J. Franco, C. Frueh, D. Clark, and M. Jah, “A new multi-target tracking algorithm for a large number of orbiting objects,” Advances in Space Research , vol. 64, pp. 645–667, 2019

    work page 2019

  4. [4]

    Blackman and R

    S. Blackman and R. Popoli, Design and Analysis of Modern Tracking Systems. Artech House, 1999

    work page 1999

  5. [5]

    Evaluating multi- object tracking,

    K. Smith, D. Gatica-Perez, J. Odobez, and S. Ba, “Evaluating multi- object tracking,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition , 2005, pp. 36–36

    work page 2005

  6. [6]

    T. M. Apostol, Mathematical Analysis. Addison Wesley, 1974

    work page 1974

  7. [7]

    A new metric between distributions of point processes,

    D. Schuhmacher and A. Xia, “A new metric between distributions of point processes,” Advances in Applied Probability , vol. 40, no. 3, pp. 651–672, Sep. 2008

    work page 2008

  8. [8]

    A consistent metric for performance evaluation of multi-object filters,

    D. Schuhmacher, B.-T. V o, and B.-N. V o, “A consistent metric for performance evaluation of multi-object filters,” IEEE Transactions on Signal Processing, vol. 56, no. 8, pp. 3447–3457, Aug. 2008

    work page 2008

  9. [9]

    Metrics for evaluating feature-based mapping performance,

    P. Barrios, M. Adams, K. Leung, F. Inostroza, G. Naqvi, and M. E. Orchard, “Metrics for evaluating feature-based mapping performance,” IEEE Transactions on Robotics, vol. 33, no. 1, pp. 198–213, Feb. 2017

    work page 2017

  10. [10]

    A comparison of multi-object sub-pattern linear assignment metrics,

    P. Barrios and M. Adams, “A comparison of multi-object sub-pattern linear assignment metrics,” in 12th International Conference on Control, Automation and Information Sciences , 2023, pp. 712–718

    work page 2023

  11. [11]

    Multitarget miss distance via optimal assignment,

    J. R. Hoffman and R. P. S. Mahler, “Multitarget miss distance via optimal assignment,” IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans , vol. 34, no. 3, pp. 327–336, May 2004

    work page 2004

  12. [12]

    A complete optimal subpattern assignment (COSPA) metric,

    T. Vu, “A complete optimal subpattern assignment (COSPA) metric,” in 23rd International Conference on Information Fusion , 2020, pp. 1–8

    work page 2020

  13. [13]

    Gen- eralized optimal sub-pattern assignment metric,

    A. S. Rahmathullah, A. F. García-Fernández, and L. Svensson, “Gen- eralized optimal sub-pattern assignment metric,” in 20th International Conference on Information Fusion , 2017, pp. 1–8

    work page 2017

  14. [14]

    Evaluating multiple object tracking performance: The CLEAR MOT metrics,

    K. Bernardin and R. Stiefelhagen, “Evaluating multiple object tracking performance: The CLEAR MOT metrics,” EURASIP Journal on Image and Video Processing, vol. 2008, pp. 1–10, 2008

    work page 2008

  15. [15]

    nuScenes: A multimodal dataset for autonomous driving,

    H. Caesar et al. , “nuScenes: A multimodal dataset for autonomous driving,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 11 618–11 628

    work page 2020

  16. [16]

    HOTA: A higher order metric for evaluating multi- object tracking,

    J. Luiten et al. , “HOTA: A higher order metric for evaluating multi- object tracking,” International Journal of Computer Vision , pp. 1–31, 2020

    work page 2020

  17. [17]

    A solution for large-scale multi-object tracking,

    M. Beard, B. T. V o, and B. V o, “A solution for large-scale multi-object tracking,” IEEE Transactions on Signal Processing , vol. 68, pp. 2754– 2769, 2020

    work page 2020

  18. [18]

    A time- weighted metric for sets of trajectories to assess multi-object tracking algorithms,

    A. F. García-Fernández, A. S. Rahmathullah, and L. Svensson, “A time- weighted metric for sets of trajectories to assess multi-object tracking algorithms,” in 24th International Conference on Information Fusion , 2021, pp. 1–8

    work page 2021

  19. [19]

    A metric for sets of trajectories that is practical and mathematically consistent,

    J. Bento and J. J. Zhu, “A metric for sets of trajectories that is practical and mathematically consistent,” 2018. [Online]. Available: https://arxiv.org/abs/1601.03094

    work page arXiv 2018

  20. [20]

    A metric on the space of finite sets of trajectories for evaluation of multi-target tracking algorithms,

    A. F. García-Fernández, A. S. Rahmathullah, and L. Svensson, “A metric on the space of finite sets of trajectories for evaluation of multi-target tracking algorithms,” IEEE Transactions on Signal Processing , vol. 68, pp. 3917–3928, 2020

    work page 2020

  21. [21]

    Fast computation of the TGOSPA metric for multiple target tracking via unbalanced optimal transport,

    V . N. Wernholm, A. Wärnsäter, and A. Ringh, “Fast computation of the TGOSPA metric for multiple target tracking via unbalanced optimal transport,” IEEE Control Systems Letters , vol. 9, pp. 498–503, 2025

    work page 2025

  22. [22]

    Skolnik, Introduction to Radar Systems

    M. Skolnik, Introduction to Radar Systems . McGraw-Hill, 2001

    work page 2001

  23. [23]

    On quasi-metric spaces,

    W. A. Wilson, “On quasi-metric spaces,” American Journal of Mathe- matics, vol. 53, no. 3, pp. 675–684, Jul. 1931

    work page 1931

  24. [24]

    Quasi-metric and metric spaces,

    V . Schroeder, “Quasi-metric and metric spaces,” Conformal Geometry and Dynamics, vol. 10, pp. 355–360, Dec. 2006

    work page 2006

  25. [25]

    A family of tractable graph metrics,

    J. Bento and S. Ioannidis, “A family of tractable graph metrics,” Applied Network Science, vol. 4, no. 107, 2019

    work page 2019

  26. [26]

    Quasi-metrics, similarities and searches: aspects of geometry of protein datasets,

    A. Stojmirovi ´c, “Quasi-metrics, similarities and searches: aspects of geometry of protein datasets,” Ph.D. dissertation, Victoria University of Wellington, 2005

    work page 2005

  27. [27]

    Optimal goal-reaching reinforcement learning via quasimetric learning,

    T. Wang, A. Torralba, P. Isola, and A. Zhang, “Optimal goal-reaching reinforcement learning via quasimetric learning,” in Proceedings of the 40th International Conference on Machine Learning , 2023, pp. 36 411– 36 430

    work page 2023

  28. [28]

    On the learning and learnability of quasimetrics,

    T. Wang and P. Isola, “On the learning and learnability of quasimetrics,” in International Conference on Learning Representations , 2022

    work page 2022

  29. [29]

    R. P. S. Mahler, Advances in Statistical Multisource-Multitarget Infor- mation Fusion. Artech House, 2014

    work page 2014

  30. [30]

    Multiple target tracking based on sets of trajectories,

    A. F. García-Fernández, L. Svensson, and M. R. Morelande, “Multiple target tracking based on sets of trajectories,” IEEE Transactions on Aerospace and Electronic Systems , vol. 56, no. 3, pp. 1685–1707, Jun. 2020

    work page 2020

  31. [31]

    Metrics for performance evaluation of elliptic extended object tracking methods,

    S. Yang, M. Baum, and K. Granström, “Metrics for performance evaluation of elliptic extended object tracking methods,” in IEEE Table I: RMS-TGOSPA q-metric errors across all time steps T-PMBM ρ = PMBM ρ = PMB ρ = GLMB ρ = pD λ C 0.3 0.5 0.7 0.3 0.5 0.7 0.3 0.5 0.7 0.3 0.5 0.7 0.8 15 4.57 4.28 3.98 7.46 6.89 6.26 7.43 6.95 6.43 8.83 7.85 6.73 20 4.79 4.50 ...

    work page 2016

  32. [32]

    Poisson multi-Bernoulli mixtures for sets of trajectories,

    K. Granström, L. Svensson, Y . Xia, J. Williams, and A. F. García- Fernández, “Poisson multi-Bernoulli mixtures for sets of trajectories,” IEEE Transactions on Aerospace and Electronic Systems, vol. 61, no. 2, pp. 5178–5194, 2025

    work page 2025

  33. [33]

    Marginal multi-Bernoulli filters: RFS derivation of MHT, JIPDA and association-based MeMBer,

    J. L. Williams, “Marginal multi-Bernoulli filters: RFS derivation of MHT, JIPDA and association-based MeMBer,” IEEE Transactions on Aerospace and Electronic Systems , vol. 51, no. 3, pp. 1664–1687, July 2015

    work page 2015

  34. [34]

    Poisson multi-Bernoulli mixture filter: direct derivation and imple- mentation,

    A. F. García-Fernández, J. L. Williams, K. Granström, and L. Svensson, “Poisson multi-Bernoulli mixture filter: direct derivation and imple- mentation,” IEEE Transactions on Aerospace and Electronic Systems , vol. 54, no. 4, pp. 1883–1901, Aug. 2018

    work page 1901

  35. [35]

    Labeled random finite sets and multi-object conjugate priors,

    B. T. V o and B. N. V o, “Labeled random finite sets and multi-object conjugate priors,” IEEE Transactions on Signal Processing , vol. 61, no. 13, pp. 3460–3475, July 2013

    work page 2013

  36. [36]

    Trajectory Poisson multi-Bernoulli filters,

    A. F. García-Fernández, L. Svensson, J. L. Williams, Y . Xia, and K. Granström, “Trajectory Poisson multi-Bernoulli filters,” IEEE Trans- actions on Signal Processing , vol. 68, pp. 4933–4945, 2020

    work page 2020

  37. [37]

    An algorithm for ranking all the assignments in order of increasing cost

    K. G. Murty, “An algorithm for ranking all the assignments in order of increasing cost.” Operations Research, vol. 16, no. 3, pp. 682–687, 1968

    work page 1968

  38. [38]

    Bar-Shalom, T

    Y . Bar-Shalom, T. Kirubarajan, and X. R. Li, Estimation with Applica- tions to Tracking and Navigation . John Wiley & Sons, Inc., 2001

    work page 2001

  39. [39]

    Probabilistic GOSPA: a metric for performance evaluation of multi-object filters with uncertainties,

    Y . Xia, A. F. García-Fernández, J. Karlsson, K.-C. Chang, T. Yuan, and L. Svensson, “Probabilistic GOSPA: a metric for performance evaluation of multi-object filters with uncertainties,”accepted in IEEE Transactions on Aerospace and Electronic Systems , 2025

    work page 2025

  40. [40]

    Probabilistic trajectory GOSPA: A metric for uncertainty- aware multi-object tracking performance evaluation,

    Y . Xia, A. F. García-Fernández, J. Karlsson, Y . Ge, L. Svensson, and T. Yuan, “Probabilistic trajectory GOSPA: A metric for uncertainty- aware multi-object tracking performance evaluation,” 2025. [Online]. Available: https://arxiv.org/abs/2506.15148

    work page arXiv 2025

  41. [41]

    GOSPA and T-GOSPA quasi-metrics for evaluation of multi-object tracking algorithms

    C. S. Kubrusly, The Elements of Operator Theory . Springer Science + Business Media, 2011. Supplemental material: “GOSPA and T-GOSPA quasi-metrics for evaluation of multi-object tracking algorithms” APPENDIX A We prove the triangle inequality for the GOSPA q-metric. In particular we prove that for any set Z, the following inequality holds d(c,ρ) p (X, Y )...

    work page 2011