{"id":"1125d4eb-ae2c-487d-8b8e-5cecb8236ffa","arxiv_id":"2506.15148","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"The paper introduces PTGOSPA, a probabilistic trajectory GOSPA metric for evaluating multi-object tracking algorithms that output trajectory estimates with track-level uncertainties.","lead":"This paper defines a new way to measure how well a multi-object tracking algorithm performs when the algorithm reports uncertainties about each track. The new metric combines two existing metrics so that missed objects, false tracks, localization errors, uncertainty mismatch, and track switches are all penalized in one interpretable score.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"PTGOSPA and its LP relaxation are asserted to be metrics by analogy to TGOSPA, but no proof is supplied; metricity is the central unverified claim.","rationale":"The reader's conditional verdict is appropriate. The paper's own text explicitly defers the metric proof to analogy with TGOSPA, both for Definition 3 and for Proposition 1, providing no detailed argument. This is an omitted proof of the central claim. I checked the natural augmented-metric construction: defining a dummy element ∅ with distance (r c^p/2)^{1/p} to a Bernoulli density (r,p) makes the per-step cost look like the p-th power of a metric, so the TGOSPA composition argument probably goes through. But the paper itself does not establish this, and the existence-probability-weighted unassignment cost is a genuine departure from TGOSPA, so the analogy is not merely cosmetic. The simulation study is illustrative rather than a proof, and no code is provided. Therefore the verdict should remain CONDITIONAL: the metric claim should be accepted only after the full proof (or a carefully verified counterexample search) is supplied. If the proof turns out to fail, the verdict would need to move to REJECT. My proposed concrete test is an analytical re-derivation of the TGOSPA proof with the PTGOSPA cost, which would settle the issue definitively.","tokens_in":10929,"tokens_out":20194,"duration_ms":195924,"concrete_test":"Re-derive the proof of [13, App. B] line by line with the PGOSPA per-step cost (Eqs. 9-12) and the LP objective (Eq. 18). In particular, prove or disprove that D_B((r,p),∅) = (r c^p/2)^{1/p} and D_B((r,p),(s,q)) = (min(r,s) min(db(p,q),c)^p + |r-s|c^p/2)^{1/p} is a metric on B∪{∅}, and that composing two feasible trajectory assignments (or two feasible soft assignment matrices) yields a feasible third assignment whose cost is bounded by the sum of the two costs in the Minkowski sense. If any inequality fails, exhibit the explicit triple of sets of Bernoulli sequences violating the triangle inequality; if all hold, write up the completed proof as an appendix.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III-B defines PTGOSPA in Definition 3 (Eq. 8) and then states without proof: \"The proof that PTGOSPA is a mathematically well-defined metric follows analogously to that of TGOSPA.\" Section III-C makes the same move for Proposition 1: the LP relaxation (Eq. 18) is declared a metric by analogy to [13, App. B]. The TGOSPA metric proof rests on per-time-step costs that are the p-th power of a metric on the augmented object space (with a dummy element for unassigned objects) plus a switching cost that is itself a metric on assignments; the triangle inequality for the whole objective then follows by a gluing/composition argument over time. PTGOSPA changes the per-step object space to Bernoulli densities and replaces the constant unassignment penalty c^p/2 by the existence-probability-weighted penalty r c^p/2. The paper does not verify that the resulting PGOSPA base cost, Eq. (5), with these weighted dummy costs defines a metric on the augmented space B∪{∅}, nor that the composition lemma from [13] survives this replacement. Since metricity is the paper's central claim, this omission is load-bearing: if the gluing argument fails at any step, Definition 3 or Prop 1 may not satisfy the triangle inequality, and the contribution collapses to an unproven heuristic.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript defines PTGOSPA, a proposed metric on finite sets of time sequences of Bernoulli densities, by combining trajectory-level assignments with per-time-step PGOSPA costs and a switching cost. It states that PTGOSPA and its linear programming relaxation are metrics (Definition 3, Eq. (8), and Proposition 1, Eq. (18)), formulates an equivalent binary linear program (Lemma 1), derives a five-term decomposition, and illustrates the quantities on a TPMBM/TPMB simulation. The central claim is the metric property of Eqs. (8) and (18), but no proof is given in the paper.","tokens_in":11172,"tokens_out":11515,"duration_ms":122022,"significance":"If the metric property is correct, PTGOSPA would be a valuable uncertainty-aware generalization of TGOSPA, with a clear decomposition and a polynomial-time computable relaxation. The paper deserves credit for the explicit construction, the interpretable decomposition, and the simulation-based demonstration that uncertainty information changes the error profile in a meaningful way. The significance is, however, conditional: the main theorem-like claims, metricity of Definition 3 and Proposition 1, are asserted by analogy and are not demonstrated in the manuscript.","major_comments":[{"comment":"The claim that d(X,Y) is a metric is not supported. The sentence 'The proof that PTGOSPA is a mathematically well-defined metric follows analogously to that of TGOSPA' is not a proof. The TGOSPA proof in [13] relies on a per-time-step cost that is the p-th power of a metric on an augmented object space with a constant dummy distance c/2^{1/p}. In PTGOSPA the unassignment cost is r c^p/2, which depends on the existence probability of the Bernoulli component. This can be interpreted as a distance to a dummy Bernoulli component with r=0, but then the triangle inequality for the augmented space and the gluing argument over time must be verified explicitly. The inequality in (13) is necessary but not sufficient. Please provide a complete proof, or a precise reference to a proof that covers this weighted-dummy case.","section":"Section III.B, Definition 3 (Eq. (8))"},{"comment":"The same problem occurs for the LP relaxation. The statement that the proof 'follows analogously' to [13, App. B] is not adequate, because the cost matrix D^k in (16) contains existence-probability-weighted dummy entries and zeros for inactive pairs; the relaxation objective in (18) is not the same object as in [13, Eq. (23)]. The metricity of the relaxation, in particular the triangle inequality for soft assignments, needs to be proved, or the claim should be downgraded to a conjecture.","section":"Section III.C, Proposition 1 (Eq. (18))"},{"comment":"The equivalence between Definition 3 and the binary linear program requires an additional observation that is not stated: if two nonempty Bernoulli densities have d_b >= c, then by (5), d_P(x,y)^p = (x(r)+y(r)) c^p/2, so assigning them costs exactly the same as leaving both unassigned. Without this observation, the restriction in (10) to d_b < c and the unqualified entries d_P(x_i^k,y_j^k)^p in D^k are not obviously consistent. Please add this verification.","section":"Section III.D / Lemma 1 (Eqs. (15) and (19))"}],"minor_comments":[{"comment":"The notation X(θ~^k(π^k)) is undefined; earlier only X(π~^k) was defined. Please clarify what the tilde denotes.","section":"Section III.A, Eq. (9)"},{"comment":"In Eqs. (14a)-(14b), 'n nY' and 'n nX' should read n_Y and n_X; in Eq. (16), the condition 'y_i^k' should read 'y_j^k'.","section":"Section III.C, Eqs. (14a)-(14b) and (16)"},{"comment":"The paper allows r=0 in the space B, but PGOSPA in Section II.B assumes r>0 for all Bernoulli components. Since components with r=0 are indistinguishable from the empty density, the authors should state whether Bernoulli sequences are restricted to r>0 or whether the metric space is quotiented accordingly, so that identity of indiscernibles holds.","section":"Sections II.B and III.A"},{"comment":"The qualitative interpretation of the difference between TGOSPA and PTGOSPA missed/existence-mismatch errors near object death is plausible but is supported only by single averaged curves; reporting Monte Carlo variability would strengthen the claim.","section":"Section IV"}],"recommendation":"major_revision","confidential_remarks":"The main barrier is the unproved metricity claim. If the authors supply a complete proof for Definition 3 and Proposition 1, and fix the small consistency gaps, I would support acceptance. The construction is a natural extension of TGOSPA and the simulation study is appropriate for the journal's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper takes the trajectory-GOSPA program one step further: it combines PGOSPA with TGOSPA to get a metric on sets of time sequences of Bernoulli densities, so that track-level uncertainties (existence probabilities and density shapes) enter the evaluation. That is a real gap in the literature, and the construction is a direct, sensible combination of two existing ideas. The five-term decomposition (localization, existence-probability mismatch, missed/false detections, and switches) is clearly derived and seems algebraically right. The LP relaxation is also a useful practical contribution, since it is computable in polynomial time. The simulation comparing TPMBM and TPMB filters is illustrative and gives a nice insight: at object death, PTGOSPA shifts the error from missed detections to existence-probability mismatch, which is arguably the right way to think about uncertainty-aware evaluation.\n\nThat said, there is a load-bearing gap. The paper never proves that PTGOSPA is a metric. Section III-B says the proof follows analogously to TGOSPA, and Section III-C says the same for the LP relaxation. But the per-time-step cost here is not the GOSPA cost; it is built from PGOSPA's existence-probability-weighted penalties. Whether the triangle inequality survives that replacement is exactly the thing that needs checking, and the authors do not check it. The issue is compounded because PGOSPA itself is an unreviewed preprint, so the chain of support is thinner than a citation to a published theorem. The metricity claim is not a side remark; it is the paper's central assertion. If the analogy fails, the contribution becomes an unproven heuristic.\n\nOther concerns are minor. The simulation is a single scenario, and no code is provided, so the empirical demonstration is more illustrative than definitive. The paper is honest about what it does and does not prove, which I appreciate, but for a metrics paper the main theorem needs a full proof or at least a rigorous reduction to the TGOSPA/PGOSPA theorems.\n\nI would send this to a serious referee: the contribution is relevant to anyone using trajectory filters (TPMB, TPMBM) or designing MOT evaluation metrics, and the gap is fixable. The right outcome is major revision with a request for a complete metricity proof. I would not desk-reject it, but I also would not accept it as is.\n\nRecommendation: peer review, major revision.","headline":"A natural and useful extension of TGOSPA to track-level uncertainties, but the central metricity claim is asserted by analogy rather than proven, so the paper needs a serious revision before the main result can be accepted.","tokens_in":11721,"tokens_out":3919,"would_cite":true,"duration_ms":43405,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PTGOSPA extends tracking error metrics to trajectories reported as uncertain Bernoulli densities.","keywords":["multi-object tracking","performance evaluation","trajectory GOSPA","probabilistic GOSPA","Bernoulli densities","linear programming relaxation","track switching","uncertainty-aware evaluation"],"falsifier":"Brute-force a small triple of sets of Bernoulli sequences with $K=2$ or $3$ time steps and small $n_X, n_Y$, evaluating Definition 3 by enumerating all assignment vectors $\\pi^k$; if for any $c, \\gamma, p$ the values violate $d(X,Z) \\le d(X,Y) + d(Y,Z)$, the metric claim is false. The same check on Eq. (18) would test the relaxation claim.","tokens_in":10707,"feed_emoji":"🎯","tokens_out":6816,"duration_ms":63492,"temperature":0.7,"pith_summary":"Multi-object tracking algorithms increasingly output not just state estimates but track-level probabilities, and existing metrics like trajectory GOSPA collapse those to point estimates before scoring. This paper proposes probabilistic trajectory GOSPA (PTGOSPA), a metric on sets of time sequences of Bernoulli densities: each estimated trajectory is a sequence of existence probabilities and single-object densities. PTGOSPA is formulated as a multidimensional assignment problem, and the paper argues its linear programming relaxation is also a metric and computable in polynomial time. The metric decomposes into expected localization error, existence-probability mismatch, expected missed and false detection error, and track switch error, so a practitioner can see not only how far off a tracker is but why. If the metric property holds, it gives a principled, uncertainty-aware way to compare and tune trackers without discarding their posterior information.","feed_headline":"PTGOSPA turns uncertain track estimates into a true error metric","feed_subtitle":"Extends trajectory GOSPA to Bernoulli densities and splits error into localization, existence, missed/false detection, and switch costs.","key_machinery":"The machinery is a trajectory-level assignment problem lifted into the space of Bernoulli densities. Each trajectory is a sequence of Bernoulli densities, each described by an existence probability $r$ and a single-object density $p(\\cdot)$; at every time step $k$, a binary assignment vector $\\pi^k$ pairs estimated trajectories to true trajectories, and the per-step cost (9) is built from the PGOSPA pair cost $d_P(x_i^k,y_j^k)^p$ with the assignment feasibility condition $d_b(x_i^k(p), y_j^k(p)) < c$, plus expected missed and false detection terms weighted by existence probabilities. Between time steps, the switching cost (11) charges $\\gamma^p$ for full assignment changes and $\\gamma^p/2$ for assignment-to-unassignment changes. Replacing binary assignments by a doubly stochastic relaxation gives the LP formulation (18), whose cost matrix $D^k_{X,Y}$ encodes the same Bernoulli-level distances and absence costs. This construction is what allows the final error to split into five named components.","core_discovery":"The central claim is that the distance between two sets of trajectory estimates can be defined at the level of Bernoulli densities while still obeying the metric axioms, so that uncertainty enters the evaluation itself rather than being thrown away before scoring. Concretely, PTGOSPA assigns entire estimated trajectories to true trajectories over the evaluation window; at each time step the per-pair cost is the PGOSPA distance between the two Bernoulli densities, with assignment allowed only when the base density distance is below the cutoff $c$, and a switching cost penalizes changes in trajectory-level assignment between consecutive steps. The resulting cost (Definition 3, Eq. (8)) is claimed to be a true metric on the space of sets of time sequences of Bernoulli densities, and its linear programming relaxation (Eq. (18)), which allows soft assignments, is claimed to be a metric as well. The proof is presented as following analogously to the TGOSPA proof, with TGOSPA recovered as the special case where all existence probabilities are one and single-object densities are Dirac deltas. The simulation comparison of TPMBM and TPMB filters shows the two metrics largely agree in trend but diverge when objects die, where PTGOSPA charges the error to existence-probability mismatch instead of missed detection.","pith_inferences":["Because PGOSPA is built on the $p$-Wasserstein distance between Bernoulli densities, PTGOSPA's localization term inherits that Wasserstein interpretation; one testable consequence is that sensitivity to overconfident tracks is governed by the min-existence weighting $\\min(r_x,r_y)$.","The switching cost, applied to trajectory-level assignments through soft matrices $W_k$, could be adapted to penalize identity switches with unequal costs for birth versus swap, a direction the paper only hints at through the $\\gamma^p/2$ half-switch.","The metric could serve as a training loss or reward for learning-based MOT, since the LP relaxation is differentiable in the density parameters; the paper mentions quantifying filter approximation errors, not end-to-end learning.","Whether the triangle inequality really holds for the full assignment formulation depends on the proof-by-analogy from TGOSPA; a direct verification on random triples of small trajectory sets would settle it before wide deployment."],"forward_implications":["Two trackers can be compared on their full posterior output rather than on point estimates extracted from them, so evaluation no longer discards track-level uncertainty.","The binary and relaxed formulations give a polynomial-time computable lower bound, letting large evaluation problems be solved approximately with soft assignment.","Decomposed PTGOSPA reports expected localization, existence mismatch, missed and false detection, and track switch errors separately, so a poor score can be traced to a specific failure mode.","TGOSPA is a special case, meaning existing TGOSPA deployments can be upgraded to PTGOSPA without changing the ground truth representation.","The simulation shows that uncertainty-aware scoring changes the qualitative diagnosis at object death, shifting the reported error from missed detection to existence probability mismatch."],"supporting_citations":[{"why":"Supplies the trajectory-level assignment formulation, the switching cost, and the metric proof that PTGOSPA claims to follow.","marker":"[13]"},{"why":"Defines the Bernoulli-density pair cost $d_P$ and its decomposition used in PTGOSPA's per-step cost.","marker":"[16]"},{"why":"Provides the base GOSPA metric with $\\alpha=2$ and the single-pair cost that TGOSPA and PTGOSPA build on.","marker":"[9]"},{"why":"One of the two filters evaluated with PTGOSPA and the source of trajectory Bernoulli densities.","marker":"[20]"},{"why":"The second filter used in the simulation study.","marker":"[22]"},{"why":"Supplies the simulation scenario with six objects moving into close proximity.","marker":"[26]"}],"fun_headline_variants":["PTGOSPA turns track uncertainty into a verifiable metric","New metric scores uncertain track estimates truthfully","PTGOSPA gives uncertain tracks a true metric, not a hack","Uncertain tracks now get a real error metric: PTGOSPA","Track uncertainty is now part of the score, says PTGOSPA"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the PTGOSPA construction in Definition 3, and its linear programming relaxation, satisfy the triangle inequality; the paper asserts this by analogy with TGOSPA rather than giving a proof, so if the analogy fails the central claim collapses.","fun_headline_variants_meta":{"raw":{"variants":["PTGOSPA turns track uncertainty into a verifiable metric","New metric scores uncertain track estimates truthfully","PTGOSPA gives uncertain tracks a true metric, not a hack","Uncertain tracks now get a real error metric: PTGOSPA","Track uncertainty is now part of the score, says PTGOSPA"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000197,"raw_usage":{"total_tokens":1365,"prompt_tokens":948,"completion_tokens":417,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":564,"completion_tokens_details":{"reasoning_tokens":329}},"tokens_in":564,"tokens_out":417,"duration_ms":4528,"temperature":1.0,"reasoning_tokens":329,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:41:29.204321+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Brute-force a small triple of sets of Bernoulli sequences with $K=2$ or $3$ time steps and small $n_X, n_Y$, evaluating Definition 3 by enumerating all assignment vectors $\\pi^k$; if for any $c, \\gamma, p$ the values violate $d(X,Z) \\le d(X,Y) + d(Y,Z)$, the metric claim is false. The same check on Eq. (18) would test the relaxation claim.","supporting_citations":[{"cited_title":"A metric on the space of finite sets of trajectories for evaluation of multi- target tracking algorithms,","cited_arxiv_id":null,"evidence_quote":"Supplies the trajectory-level assignment formulation, the switching cost, and the metric proof that PTGOSPA claims to follow."},{"cited_title":"Probabilistic GOSPA: A Metric for Performance Evaluation of Multi-Object Filters with Uncertainties","cited_arxiv_id":"2412.11482","evidence_quote":"Defines the Bernoulli-density pair cost $d_P$ and its decomposition used in PTGOSPA's per-step cost."},{"cited_title":"Gen- eralized optimal sub-pattern assignment metric,","cited_arxiv_id":null,"evidence_quote":"Provides the base GOSPA metric with $\\alpha=2$ and the single-pair cost that TGOSPA and PTGOSPA build on."},{"cited_title":"Poisson multi-Bernoulli mixtures for sets of trajectories,","cited_arxiv_id":null,"evidence_quote":"One of the two filters evaluated with PTGOSPA and the source of trajectory Bernoulli densities."},{"cited_title":"Trajectory Poisson multi-Bernoulli filters,","cited_arxiv_id":null,"evidence_quote":"The second filter used in the simulation study."},{"cited_title":"Markov chain Monte Carlo multiscan data association for sets of trajectories,","cited_arxiv_id":null,"evidence_quote":"Supplies the simulation scenario with six objects moving into close proximity."}],"review_version":2}