{"id":"fe8fb24c-3a76-4d93-bc0a-befbe4ba0d3f","arxiv_id":"2412.11482","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":3,"one_line_summary":"P-GOSPA is a true metric on multi-Bernoulli densities that reduces to GOSPA for deterministic sets and decomposes into localization, existence mismatch, missed detection, and false detection errors.","lead":"This paper defines P-GOSPA, a metric that measures the distance between two multi-Bernoulli densities, which are the probabilistic outputs of multi-object tracking filters. It generalizes the widely used GOSPA metric so that uncertainty in object existence and state estimates is penalized, not just point estimates.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The triangle-inequality proof in Appendix A uses a false subadditivity step at Eq. (32): p-th powers are not subadditive, so the metric property is not established as written, although a repair via Remark 2 may exist.","rationale":"I read the paper as establishing P-GOSPA as a metric on multi-Bernoulli densities, with the triangle inequality proof in Appendix A as the critical support. The reader's weakest assumption is that the user-selected base distance must be a metric; this is a real usage condition, but it is explicitly stated in Definition 1 and satisfied in all presented examples, so it is not the most fragile point. The more serious issue is internal to the proof: Eq. (32) asserts a subadditivity of p-th powers that is false, and a concrete single-Bernoulli instance contradicts it. This means the proof of the central metric property is invalid as written. I do not claim the theorem itself is false: the Bernoulli-level metric in Eq. (4) appears to satisfy the triangle inequality, and the augmented-set interpretation in Remark 2 offers a credible route to a valid proof via the known GOSPA metric theorem. However, that route is not developed in the paper, and the direct proof currently contains an erroneous inequality. Because the mathematical contribution is likely salvageable but the submitted argument has a genuine gap, the conditional verdict remains appropriate. The required revision is more specific and more urgent than the practical concerns listed by the reader: either replace the flawed step with the correct p-th-root Minkowski argument or prove the metric property through the Remark 2 construction.","tokens_in":18092,"tokens_out":21160,"duration_ms":177263,"concrete_test":"Verify Eq. (32) on the single-pair counterexample: p=2, alpha=2, c=1, r_x=r_y=r_z=0.5, and point-mass densities whose Euclidean distances are d_xz=0.6, d_yz=0.6, d_xy=1.1. Compute the claimed component inequality: left side 0.5*min(1.1,1)^2=0.5, right side 0.5*(0.6^2+0.6^2)=0.36; the inequality fails. Then settle whether the theorem survives by re-deriving the triangle inequality from the Bernoulli-level metric (4) and the augmenting construction of Remark 2 (append zero-existence Bernoullis and invoke the standard GOSPA metric theorem). If the re-derivation succeeds, the flaw is a proof gap; if it fails, the metric claim is false.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that Eq. (2) satisfies the triangle inequality. The proof for multi-Bernoulli densities breaks at the step from Eq. (31) to Eq. (32). The authors write that 'using the triangle inequality on Bernoulli densities (14)' gives, inside the outer p-th root, a sum of the p-th powers B_XZ,i + B_YZ,i for each matched pair. But the Bernoulli triangle inequality is B_XY,i^(1/p) <= B_XZ,i^(1/p) + B_YZ,i^(1/p), which upon taking p-th powers yields (B_XZ,i^(1/p)+B_YZ,i^(1/p))^p, not the smaller quantity B_XZ,i + B_YZ,i. For p>1 these are not equivalent. The displayed inequality is false already in the single-Bernoulli case: take p=2, alpha=2, c=1, r_x=r_y=r_z=0.5, and choose single-object densities with d(p_x,p_z)=d(p_y,p_z)=0.6 and d(p_x,p_y)=1.1, realizable with three point masses in the plane. Then B_XY=0.5*min(1.1,1)^2=0.5, while B_XZ+B_YZ=0.5*(0.6^2+0.6^2)=0.36. Eq. (32) would assert d(f_X,f_Y) <= (B_XZ+B_YZ)^(1/2)=0.6, whereas the true left side is sqrt(0.5)≈0.707. Thus the multi-Bernoulli triangle inequality is not proven by Appendix A as written. This is load-bearing because the metric property is the paper's main theoretical contribution. The theorem may still be true: Remark 2 reduces P-GOSPA to GOSPA on augmented Bernoulli sets, and if the Bernoulli-level distance (4) is a metric, the standard GOSPA triangle inequality would prove it. But the submitted proof needs correction.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper defines P-GOSPA in Eq. (2), a distance between multi-Bernoulli densities obtained by extending GOSPA's assignment cost to Bernoulli components with existence probabilities and single-object densities. It claims P-GOSPA is a metric (Appendix A), that for alpha=2 it decomposes into expected localization, existence-probability mismatch, missed-detection, and false-detection terms (Proposition 2), and it illustrates the metric on two examples and on PMBM/PMB filtering simulations. The manuscript also proves an upper-bound relationship to the Wasserstein distance between Bernoulli densities (Proposition 1).","tokens_in":18426,"tokens_out":19134,"duration_ms":164406,"significance":"P-GOSPA addresses a real need: GOSPA evaluates point estimates only, while multi-object filters output densities carrying existence and state uncertainty. If the metric property is established, P-GOSPA would be an analytically tractable, parameter-free (modulo c, p, alpha) distance on MB densities that reduces to GOSPA for Dirac/point-mass components and admits the interpretable alpha=2 decomposition. The assignment formulation in Proposition 2 gives a practical computational route, and Proposition 1's Wasserstein connection is a useful sanity check. The main obstacle is that the submitted proof of the triangle inequality contains a false step, so the paper's central claim is not yet established as written.","major_comments":[{"comment":"The step from Eq. (31) to Eq. (32) is invalid. Denote by C_i the i-th summand on the right of Eq. (31) (the p-th power of the Bernoulli distance between the i-th pair), and by A_i and B_i the analogous summands involving (X,Z) and (Y,Z). The Bernoulli triangle inequality (14) gives C_i^{1/p} <= A_i^{1/p}+B_i^{1/p}, which after raising to the p-th power is C_i <= (A_i^{1/p}+B_i^{1/p})^p, not C_i <= A_i+B_i. The displayed inequality in Eq. (32) is therefore false in general: for p=2, alpha=2, c=1, r_x=r_y=r_z=0.5, and single-object densities with d(p_x,p_z)=d(p_y,p_z)=0.6 and d(p_x,p_y)=1.1 (realizable by three point masses in the plane), one has C_XY=0.5*1^2=0.5 whereas A_XZ+B_YZ=0.5*(0.6^2+0.6^2)=0.36, so Eq. (32) would assert sqrt(0.5) <= 0.6. Since this step is used in all three cases of the multi-Bernoulli proof, the triangle inequality is not proven by the submitted text. A correct argument may be available either by applying Minkowski's inequality to the sequences (A_i^{1/p}) and (B_i^{1/p}) before summing, or by combining Remark 2 with the standard GOSPA metric property, but the proof as written must be revised.","section":"Appendix A, Eq. (32)"},{"comment":"The PMBM filter's posterior is a mixture of MB densities, not an MB density, so Eq. (2) does not directly apply. The paper computes P-GOSPA between each MB component and the ground truth and then takes a weighted sum. This is an ad hoc extension; a weighted sum of metrics is not automatically a metric, and no definition or properties are given for P-GOSPA on MBM densities. If the authors wish to claim that P-GOSPA evaluates PMBM (and hence 'multi-object filters with uncertainties' generally), this extension must be defined and analyzed; otherwise the PMBM results should be labeled as a heuristic expected-P-GOSPA rather than as the metric itself.","section":"Section IV, Fig. 7"}],"minor_comments":[{"comment":"The metric property of P-GOSPA is only guaranteed when the base distance d(p_x,p_y) is a metric; the paper uses Wasserstein and Hellinger distances, which are metrics, but a sentence explicitly warning against non-metric divergences such as KL divergence would prevent misuse.","section":"Section II, Definition 1"},{"comment":"The text refers to 'Section II.II-A' in the second paragraph of Section II; this should be 'Section II-A'.","section":"Section II-A"},{"comment":"Reference [12] contains a typo: 'García-Femández' should be 'García-Fernández'.","section":"References"},{"comment":"The simulations report RMS errors over 100 Monte Carlo runs without error bars or variance information; a brief indication of variability would strengthen the comparison between PMBM and PMB.","section":"Section IV"}],"recommendation":"major_revision","confidential_remarks":"The Appendix A gap is significant but likely fixable, since the intended result can probably be obtained through the Remark 2 route or a corrected Minkowski argument. The paper's core idea is interesting and relevant for the multi-object tracking community. I would not reject on the current evidence, but the triangle-inequality proof must be corrected before the metric claim can be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"P-GOSPA is a genuinely useful idea: an analytical metric on multi-Bernoulli densities that reduces to GOSPA when existence probabilities are one and the densities are point masses, with a clean four-term decomposition at alpha=2. The paper is also careful about interpretation, and the Remark 2 reading of the metric as GOSPA on augmented Bernoulli elements is a nice sanity check. That said, the proof of the triangle inequality in Appendix A has a real gap. The step from Eq. (31) to (32) uses the claim that the p-th power of the Bernoulli metric is subadditive, i.e., that (a+b)^p <= a^p + b^p, which is false for p>1. The stress-test counterexample is correct: take equal existence probabilities 0.5, c=1, p=2, and single-object distances 0.6, 0.6, and 1.1; then d_XY^2 = 0.5 while d_XZ^2 + d_YZ^2 = 0.36, so the displayed inequality fails. The metric may still be a metric – Remark 2 suggests a correct proof via GOSPA on sets of Bernoulli densities with the Bernoulli distance as base – but the submitted proof is not sound as written. Since the metric property is the paper's main theoretical contribution, this needs to be fixed before publication. The rest is in good shape: the definition is clear, the assignment-set reformulation in Proposition 2 checks out, and the simulations, while lacking error bars and code, illustrate the behavior. The PMBM evaluation aggregates over MB components with a weighted sum that isn't part of the metric definition, so that part is more heuristic. For the target audience of multi-object tracking researchers, this is a worthwhile contribution that deserves peer review, but the authors should either repair the appendix or replace it with the augmented-set argument.","headline":"A useful, clearly-written extension of GOSPA to multi-Bernoulli densities with a nice alpha=2 decomposition, but the triangle-inequality proof in Appendix A has a real gap that needs fixing before the metric claim can stand.","tokens_in":19093,"tokens_out":5286,"would_cite":true,"duration_ms":43803,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"P-GOSPA is a metric on multi-Bernoulli densities that extends GOSPA to full probabilistic multi-object representations, reducing to GOSPA when all existence probabilities are one and densities are point masses.","keywords":["multi-object tracking","performance evaluation","random finite sets","multi-Bernoulli process","GOSPA","Wasserstein distance","metric"],"falsifier":"Compute Eq. (2) for three Bernoulli densities in one dimension, e.g., existence probabilities $0.2, 0.5, 0.8$ at point masses $0, 1, 2$, with $c=1$, $\\alpha=1$, $p=1$ and Euclidean distance as the base metric; a violation of $d(f_X,f_Y)\\le d(f_X,f_Z)+d(f_Z,f_Y)$ would refute the metric claim. The same computation with a non-metric base such as KL divergence would show that the triangle inequality rests on the base being a metric.","tokens_in":17823,"feed_emoji":"🎯","tokens_out":9112,"duration_ms":74834,"temperature":0.7,"pith_summary":"Multi-object tracking filters output more than point estimates: Bayesian filters such as the Poisson multi-Bernoulli mixture filter produce entire multi-Bernoulli densities, complete with existence probabilities and per-object uncertainties. Existing metrics like GOSPA compare only deterministic sets of estimated points, discarding this uncertainty. This paper proposes P-GOSPA, a metric on the space of multi-Bernoulli densities, defined by an assignment problem that pairs Bernoulli components and charges a cut-off base distance on single-object densities, an existence-probability mismatch cost, and costs for unassigned components. The paper proves P-GOSPA is a true metric, with the triangle inequality being the nontrivial part, and shows that with parameter $\\alpha=2$ it decomposes into expected localization error, existence-probability mismatch error, expected missed detection error, and expected false detection error. If correct, this gives practitioners a tractable, interpretable way to compare filters on their full posterior densities rather than on extracted estimates.","feed_headline":"New metric P-GOSPA scores tracking filters with uncertainty intact","feed_subtitle":"Extends GOSPA to multi-Bernoulli densities and splits error into localization, existence, missed and false detections.","key_machinery":"The central object is the P-GOSPA formula (2), an assignment problem over Bernoulli components. Its two building blocks are the cut-off metric $d^{(c)}(p_x,p_y)=\\min(d(p_x,p_y),c)$ on single-object densities and the per-pair cost $\\min(r_x,r_y)d^{(c)}(\\cdot,\\cdot)^p+|r_x-r_y|c^p/\\alpha$; unassigned components carry $c^p/\\alpha$ times their existence probability. The proof of the triangle inequality hinges on first proving the Bernoulli case (Lemma 1) and then gluing the general case with Minkowski's inequality, using permutations to match component counts. Proposition 2, valid for $\\alpha=2$, converts the permutation form into an assignment-set form that yields the four-term error decomposition.","core_discovery":"The central claim is that the P-GOSPA expression in Eq. (2), defined for two multi-Bernoulli densities parameterized by $\\{(r_x^i,p_x^i)\\}_{i=1}^{n_X}$ and $\\{(r_y^j,p_y^j)\\}_{j=1}^{n_Y}$, is a metric on the space of such densities. Given a metric $d$ between single-object densities and its cut-off $d^{(c)}=\\min(d,c)$, P-GOSPA minimizes over permutations $\\pi$ the sum of $\\min(r_x^i,r_y^{\\pi(i)}) d^{(c)}(p_x^i,p_y^{\\pi(i)})^p + |r_x^i-r_y^{\\pi(i)}| c^p/\\alpha$ for the associated $n_X$ components, plus $c^p/\\alpha$ times the sum of the existence probabilities of the $n_Y-n_X$ unassigned components, with the formula symmetrized when $n_X>n_Y$. Non-negativity, definiteness, and symmetry follow directly from the definition; the triangle inequality is proved in Appendix A by first handling the Bernoulli case and then applying an extended Minkowski inequality. For $\\alpha=2$ the metric can be rewritten, as Proposition 2, into a 2D assignment problem whose four components are exactly the expected localization error, existence-probability mismatch, expected missed detection, and expected false detection errors. The paper also shows P-GOSPA reduces to the ordinary GOSPA metric when all existence probabilities are one and all single-object densities are Dirac deltas.","pith_inferences":["A trajectory-valued analogue is the next natural step: the paper explicitly leaves sets of trajectories as future work, and a P-GOSPA over finite sets of trajectories with existence probabilities would let track-level uncertainty be scored in the same four-term decomposition.","The parameter $\\alpha$ gives an application-tunable dial between location and existence disagreement: because both the mismatch cost and the unassigned-component cost scale as $c^p/\\alpha$, lowering $\\alpha$ raises the price of disagreeing about whether an object exists relative to where it is, without leaving the metric family.","One could stress-test the metric's usefulness by comparing P-GOSPA rankings against expected losses computed by Monte Carlo sampling of the posterior: if a filter's P-GOSPA error does not correlate with its sampled-average GOSPA error, the closed-form metric would still be a distance but a less informative score for filter selection."],"forward_implications":["With $\\alpha=2$, P-GOSPA decomposes into exactly four expected-error terms—localization, existence-probability mismatch, missed detection, and false detection—so evaluation output can be attributed to causes.","P-GOSPA is computable in closed analytical form via a 2D assignment problem, whereas the average GOSPA of two independent multi-Bernoulli densities has no general analytical expression.","When all existence probabilities equal one and all single-object densities are Dirac deltas, P-GOSPA reduces to GOSPA, making it a strict generalisation that inherits GOSPA's interpretability.","P-GOSPA can be used to measure approximation error inside a filter, e.g., the loss incurred by collapsing a Poisson multi-Bernoulli mixture posterior to a single multi-Bernoulli density, which the simulation quantifies.","For $\\alpha=2$, each unassigned Bernoulli component with existence probability $r$ contributes $r c^p/2$, so missed and false detections are penalized in proportion to the probability that the object exists."],"supporting_citations":[{"why":"Defines GOSPA, the deterministic-set metric P-GOSPA generalizes and reduces to when existence probabilities are one and densities are Dirac deltas.","marker":"[11]"},{"why":"Supplies the random finite set and multi-Bernoulli formalism, including the set integral that the Wasserstein interpretation and Proposition 1 use.","marker":"[6]"},{"why":"Provides the extension of Minkowski's inequality to sequences of different lengths, the main technical tool in the Appendix A triangle-inequality proof.","marker":"[25]"},{"why":"Gives the closed-form 2-Wasserstein distance between Gaussian densities used as the base metric in the examples and simulations.","marker":"[22]"},{"why":"Defines the PMBM filter whose posterior densities are evaluated; the simulation uses P-GOSPA on its MB components and measures MB approximation error.","marker":"[17]"},{"why":"Defines the track-oriented PMB filter that P-GOSPA evaluates as an approximation of the PMBM posterior in the simulation comparison.","marker":"[18]"},{"why":"Introduces Q-OSPA, the earlier attempt to fold existence probabilities into OSPA, which P-GOSPA supersedes because Q-OSPA fails the definiteness property.","marker":"[15]"}],"fun_headline_variants":["P-GOSPA: metric for uncertain multi-object tracking","New metric scores uncertain tracking filters","P-GOSPA decomposes tracking error four ways","Uncertainty-aware metric for multi-object filters","P-GOSPA extends GOSPA to probabilistic filters"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"For P-GOSPA to be a metric, the user-chosen base distance $d(p_x,p_y)$ between single-object densities must itself be a metric, because the cut-off of a metric is a metric but the cut-off of a non-metric divergence is not guaranteed to satisfy the triangle inequality; the paper's examples all use metric bases such as Wasserstein or Hellinger distances.","fun_headline_variants_meta":{"raw":{"variants":["P-GOSPA: metric for uncertain multi-object tracking","New metric scores uncertain tracking filters","P-GOSPA decomposes tracking error four ways","Uncertainty-aware metric for multi-object filters","P-GOSPA extends GOSPA to probabilistic filters"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000278,"raw_usage":{"total_tokens":1673,"prompt_tokens":984,"completion_tokens":689,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":600,"completion_tokens_details":{"reasoning_tokens":616}},"tokens_in":600,"tokens_out":689,"duration_ms":6097,"temperature":1.0,"reasoning_tokens":616,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:54:31.522134+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute Eq. (2) for three Bernoulli densities in one dimension, e.g., existence probabilities $0.2, 0.5, 0.8$ at point masses $0, 1, 2$, with $c=1$, $\\alpha=1$, $p=1$ and Euclidean distance as the base metric; a violation of $d(f_X,f_Y)\\le d(f_X,f_Z)+d(f_Z,f_Y)$ would refute the metric claim. The same computation with a non-metric base such as KL divergence would show that the triangle inequality rests on the base being a metric.","supporting_citations":[{"cited_title":"Gen- eralized optimal sub-pattern assignment metric,","cited_arxiv_id":null,"evidence_quote":"Defines GOSPA, the deterministic-set metric P-GOSPA generalizes and reduces to when existence probabilities are one and densities are Dirac deltas."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the random finite set and multi-Bernoulli formalism, including the set integral that the Wasserstein interpretation and Proposition 1 use."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the extension of Minkowski's inequality to sequences of different lengths, the main technical tool in the Appendix A triangle-inequality proof."},{"cited_title":"The distance between two random vectors with given dispersion matrices,","cited_arxiv_id":null,"evidence_quote":"Gives the closed-form 2-Wasserstein distance between Gaussian densities used as the base metric in the examples and simulations."},{"cited_title":"Poisson multi-Bernoulli mixture filter: direct derivation and implementa- tion,","cited_arxiv_id":null,"evidence_quote":"Defines the PMBM filter whose posterior densities are evaluated; the simulation uses P-GOSPA on its MB components and measures MB approximation error."},{"cited_title":"Marginal multi-Bernoulli filters: RFS derivation of MHT, JIPDA, and association-based MeMBer,","cited_arxiv_id":null,"evidence_quote":"Defines the track-oriented PMB filter that P-GOSPA evaluates as an approximation of the PMBM posterior in the simulation comparison."},{"cited_title":"A track quality based metric for evaluating performance of multitarget filters,","cited_arxiv_id":null,"evidence_quote":"Introduces Q-OSPA, the earlier attempt to fold existence probabilities into OSPA, which P-GOSPA supersedes because Q-OSPA fails the definiteness property."}],"review_version":1}