Selection-Induced Contraction of Innovation Statistics in Gated Kalman Filters

Barak Or

REVIEW 3 major objections 4 minor 21 references

Selection-Induced Contraction of Innovation Statistics in Gated Kalman Filters

T0 review · 3 major / 4 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Even in a perfectly matched Kalman filter, validation gating and nearest-neighbor selection push the observed innovation energy below its nominal chi-square reference.

desk verdict The gate-conditioned covariance contraction is a solid, useful result; the NN impossibility proof needs a whitening argument to match the Mahalanobis selection rule it actually uses. read the letter →

arxiv 2512.18508 v3 pith:XLY2XIA7 submitted 2025-12-20 stat.ME cs.AIcs.SYeess.SPeess.SY

Barak Or This is my paper · ORCID

classification stat.MEcs.AIcs.SYeess.SPeess.SY MSC 62M2062H1060E05

keywords KalmanfilteringvalidationgatingnormalizedinnovationsquaredNISnearest-neighborassociationcovariancecontractionchi-squaretruncationorderstatistics

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the standard practice of validating Kalman-filter innovations with elliptical gates and then selecting the nearest-neighbor measurement makes post-selection innovation statistics deviate from the nominal Gaussian and chi-square references, even when the model is exact. It proves that conditioning on the gate contracts the innovation covariance by a deterministic factor gamma(tau,m) < 1, so the mean normalized innovation squared becomes m*gamma instead of m. It then shows that choosing the minimum-norm innovation among several in-gate candidates imposes a further contraction, an unavoidable energy loss driven by order statistics, so nominal innovation energy cannot be preserved under nontrivial gating plus NN association. Closed-form two-dimensional results quantify the bias for common gate probabilities, and the paper recommends replacing nominal NIS references with gate-aware ones to avoid self-reinforcing gate tightening in adaptive filters.

What carries the argument

The load-bearing object is the whitening map u = S^{-1/2} nu, which turns the ellipsoidal gate A = {||u||^2 <= tau} into a ball centered at the origin. Because a spherical Gaussian is rotationally invariant, the conditional second moment of u given A must be a scalar multiple of the identity, gamma(tau,m) I_m; converting back to nu gives the contraction E[nu nu^T|A] = gamma(tau,m) S. The second mechanism is the order-statistic inequality E[min_i X_i] < E[X] for a non-degenerate nonnegative X, which yields the energy contraction under NN association.

What would settle it

Simulate a linear-Gaussian Kalman filter with exact model parameters, gate with tau chosen for a given acceptance probability Pg, and record the NIS of accepted measurements: the empirical mean should equal m*gamma(tau,m) (for m=2, gamma = 1 + (1-Pg) ln(1-Pg)/Pg). Adding NN selection from M=2 independent in-gate measurements should reduce it further. A data set where the accepted-NIS mean equals or exceeds the nominal m, or where NN selection does not reduce the mean below the gate-conditioned value, would contradict the central claims.

Watch

Extended reading notes

Core claim

At the paper's center is the identity E[nu nu^T | A] = gamma(tau,m) S, where nu is the innovation, S its covariance, A the Mahalanobis validation event, and gamma is the relative mean of a truncated chi-square. Since 0 < gamma < 1, the gate alone contracts the innovation covariance without rotating it or shifting the mean. Adding NN association amounts to picking the smallest of M squared norms, which by the basic mean-of-minimum inequality strictly reduces expected energy whenever M > 1 and the gate-conditioned distribution is non-degenerate. The combination makes post-selection innovation energy strictly less than tr(S), so nominal chi-square references cannot be preserved, and the same me

Load-bearing premise

The proof that nearest-neighbor association makes energy loss unavoidable assumes the in-gate measurements are independent draws from the same post-gate conditional distribution and that the selection norm equals the norm in which innovation energy is measured.

Editorial extensions

If this is right

Post-gate NIS has mean m*gamma(tau,m) rather than m; consistency tests comparing to m will flag a healthy filter as overconfident.
A gate-aware normalization Z_corr = Z/gamma restores mean m and allows reusing chi-square intuition without modifying the filter or gating logic.
Adaptive covariance-tuning schemes that ignore the gate bias will underestimate innovation covariance, tighten the gate, and increase missed detections.
With NN association the contraction grows with M, the number of in-gate candidates, exactly in regimes where NN association is theoretically suboptimal.
In two dimensions the contraction factor is closed-form: gamma(Pg,2) = 1 + (1 - Pg) ln(1 - Pg)/Pg.
Systematically reduced innovation energy in post-gate diagnostics does not by itself indicate filter mis-tuning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The independence premise behind the NN contraction is fragile: real in-gate candidates often originate from other targets or clutter, so they are not draws from the same conditional innovation distribution, meaning the observable contraction magnitude in operational scenarios may differ.
The paper's correction factor suggests a simple experiment—compare the empirical mean NIS of accepted measurements in an ideal simulation against m*gamma and against the post-NN value; a mismatch would flag either an incorrect gate threshold or an unwhitened selection metric.
For designers, the result implies that preserving nominal diagnostics requires either reporting the NIS after dividing by gamma or ensuring the association metric is the same Mahalanobis norm used for the NIS; otherwise the selection bias will be mistaken for model error.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this paper has one clean, useful result—the gate-conditioned innovation moments under ellipsoidal gating—and one result that is plausible but not proven as written, the NN energy contraction. The gating part (Proposition 2) is correct: after the standard whitening and rotational invariance argument, E[νν'|A] = γ(τ,m)S with the explicit χ² truncation factor. That's a nice, practical correction for anyone using NIS diagnostics after gating. The closed forms for m=2 are a good addition, and the gate-aware normalization in Eq. (15) is a sensible fix.

The soft spot is in Section V. Proposition 3 is stated for selection by minimum Euclidean norm, but nearest-neighbor association is defined in Eq. (17) as minimum NIS, i.e., Mahalanobis distance. Corollary 2 then applies the Euclidean-norm order-statistic result to the NIS-based rule without justification. The NIS-minimizing candidate is not necessarily the Euclidean-minimum candidate, so the written proof doesn't establish the impossibility claim for the rule the paper actually cares about. I'm fairly confident a whitening argument repairs this: in u = S^{-1/2}ν coordinates, NIS = ||u||² and the conditional distribution is isotropic, so the expected trace contraction follows. But the paper doesn't supply that argument, and the claim that Proposition 3 is distribution-free is misleading in this application—it relies on Gaussian isotropy to translate energy contraction back to the original coordinates.

There are also some smaller issues: the proof refers to 'Proposition 5.1' which doesn't exist, and the independence assumption across in-gate candidates is strong in realistic multi-target/clutter scenarios. If candidates come from other targets, they aren't i.i.d. draws from the same post-gate distribution.

Still, the gating result alone is worth publishing. This is a paper for the tracking/Kalman filtering community, especially people using innovation-based consistency tests and adaptive tuning. It deserves serious peer review; the NN section needs a repair but the core insight is likely correct.

Referee Report

3 major / 4 minor

Summary. The paper examines how validation gating and nearest-neighbor (NN) association affect the statistical properties of innovations in a linear-Gaussian Kalman filter. It shows that conditioning on the ellipsoidal gate event A={ν^T S^{-1}ν≤τ} yields E[ν|A]=0 and E[νν^T|A]=γ(τ,m)S, with γ(τ,m)=E[Z|Z≤τ]/m for Z~χ²_m, so post-gate NIS has mean mγ instead of m. It then argues that NN association, which selects the in-gate candidate with the smallest normalized innovation, adds an order-statistic contraction, leading to the claimed impossibility of preserving nominal innovation energy under gating plus NN association. Closed-form 2D expressions for γ and the post-gate NIS mean are provided.

Significance. If the results are correct, the paper gives a clean, parameter-free account of a selection-induced bias that practitioners often attribute to model mismatch or mis-tuning. Proposition 2 and the 2D closed forms are derived rigorously via whitening and rotational invariance, and the paper is self-contained. The practical message — that post-gate diagnostics should be compared to gate-conditioned references — is useful and clearly stated. The NN-association part is the weakest link: as written, the proof of the impossibility result applies to Euclidean-norm selection, not to the Mahalanobis/NIS selection rule that the paper itself defines. The gap is repairable with a whitening argument, but the current proof is incomplete. Once repaired, the contribution would be a solid, if modest, theoretical clarification of a known phenomenon.

major comments (3)

[Section V.B, Eqs. (17)–(19)] The central impossibility result is not proven for the NN rule that the paper actually defines. Eq. (17) sets i* = argmin_i ||ν^(i)||_{S^{-1}} (minimum NIS), but Proposition 3 states and proves the contraction inequality for i* = argmin_i ||ν^(i)|| (Euclidean norm). Corollary 2 then applies Proposition 3 to the NIS-minimizer without any bridging argument. The two minimizers coincide only when S is a scalar multiple of the identity, which is not assumed. The gap can be repaired: in coordinates u = S^{-1/2}ν, the gate event is {||u||^2≤τ}, u|A is rotationally invariant, and the NIS-minimizer is the Euclidean-minimizer in u-space. A short argument using the uniform distribution of the selected direction gives E[||ν^(i*)||^2|A] = E[R_(1)^2] tr(S)/m < γ(τ,m) tr(S), where R_(1)^2 is the first order statistic of M independent truncated χ²_m variables. This argument should be supplied; without i
[Proposition 3 / Corollary 2 (Section V)] The 'unavoidable' contraction is proved only under the premise that the M gated candidates are independent samples from the same conditional distribution (ν|A). In operational multi-target tracking, in-gate candidates may originate from clutter or from different targets and need not share the target innovation distribution. The claim in the abstract and conclusion that NN association introduces 'an unavoidable energy contraction' is therefore broader than the theorem's assumptions. Please qualify the statement in the abstract and the theorem, or add a discussion of what can happen when candidates are not i.i.d. draws from (ν|A), e.g., when one candidate is the true target and the others are clutter. This is a scope issue rather than a mathematical error, but it is load-bearing for the practical conclusions.
[Proposition 3, first paragraph, Section V.B] The claim that Proposition 3 is 'distribution-free' and does not invoke Gaussianity is misleading in context. The inequality (18) is indeed distribution-free for Euclidean-norm selection, but Corollary 2's application to the NIS-based NN rule requires the rotational invariance of u|A in whitened coordinates, which is a property of the Gaussian/ellipsoidally symmetric model, not of arbitrary conditional distributions. The text should either drop the 'distribution-free' claim or explicitly separate the Euclidean-norm result from the Gaussian/ellipsoidal-symmetry argument needed for the NIS rule.

minor comments (4)

[Corollary 2 proof (Section V.B)] Eq. (19) is written as an unconditional expectation E[||ν^(i*)||^2], but ν^(i*) is only defined on the acceptance event A. The proof then reinterprets it as E[||ν^(i*)||^2 | A]. Please state the theorem conditionally, or define ν^(i*) arbitrarily on A^c, to make the notation unambiguous.
[Sections V.B and V.C] The proof of Corollary 2 and the discussion in Section V.C refer to 'Proposition 5.1' and 'Eq. (18)' as 'Proposition 5.1'. The actual proposition is numbered 3, and Eq. (18) is in Section V.B. Please renumber or fix the cross-references.
[Section V.A, Eq. (17)] The notation ||ν||_{S^{-1}} is used without an explicit definition; it would be helpful to state that ||ν||_{S^{-1}} = (ν^T S^{-1} ν)^{1/2}. This is standard but should be spelled out for clarity.
[Section IV.C] The correction Z_corr = Z/γ in Eq. (15) makes the mean equal to m, but the distribution of Z_corr|A is not χ²_m. The sentence 'allows classical chi-square intuition to be reused' is too strong; it should say that the first moment is calibrated, or that higher-order diagnostics still require the truncated distribution.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the gating and NN contraction results are self-contained mathematical consequences of the stated Gaussian and order-statistic assumptions.

full rationale

The paper's derivation chain is self-contained. Proposition 2 derives E[νν^T|A] = γ(τ,m)S using the whitening transformation u = S^{-1/2}ν and rotational invariance; γ is explicitly computed as (1/m)E[Z|Z≤τ] for Z∼χ²_m, not fitted or assumed from the conclusion. Corollary 1 follows by taking the trace, and the closed-form 2-D expressions in Section VI are direct evaluations of these formulas. Proposition 3 is a standard order-statistic inequality for i.i.d. nonnegative variables, and Corollary 2 applies it to the gate-conditioned Gaussian innovation; although the application to the actual NIS-based NN rule of Eq. (17) has a Euclidean-vs-Mahalanobis norm-matching gap, that is a correctness/rigor issue, not circularity. The author's prior references [13]–[15] appear only in the related-work discussion and are not load-bearing for any proof. There are no fitted parameters renamed as predictions, no uniqueness results imported from the author's own prior theorems, and no ansatz smuggled in via citation. The central claims either follow from the explicitly stated linear-Gaussian assumptions or are elementary order-statistic facts.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper's claims rest on standard Gaussian and order-statistic facts plus the domain restriction to clutter-free linear-Gaussian tracking. No fitted parameters. The i.i.d.-candidate assumption is the most application-specific premise.

assumptions (4)

domain assumption Innovations are zero-mean Gaussian with known covariance S (ν ~ N(0,S))
Stated in Section II; the entire analysis is restricted to linear-Gaussian Kalman assumptions.
domain assumption Gated candidates are independent draws from the same conditional distribution (ν|A)
Proposition 3; needed for the order-statistic contraction. Fails with multi-target or clutter scenarios.
standard math Rotational invariance of the Gaussian and the lemma that matrices commuting with all orthogonal matrices are scalar multiples of the identity
Used in Appendix A to show E[uu^T|A] is proportional to I.
standard math Order-statistic inequality E[min_i X_i] ≤ E[X] for i.i.d. nonnegative X
Used in Proposition 3 proof.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Selection-Induced Contraction of Innovation Statistics in Gated Kalman Filters." pith.science (2026). https://pith.science/paper/XLY2XIA7

@misc{pith2026251218508,
  author       = {Pith},
  title        = {Pith review of: Selection-Induced Contraction of Innovation Statistics in Gated Kalman Filters},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XLY2XIA7}},
  note         = {Machine review of arXiv:2512.18508}
}

read the original abstract

Validation gating is a fundamental component of classical Kalman-based tracking systems. Only measurements whose normalized innovation squared (NIS) falls below a prescribed threshold are considered for state update. While this procedure is statistically motivated by the chi-square distribution, it implicitly replaces the unconditional innovation process with a conditionally observed one, restricted to the validation event. This paper shows that innovation statistics computed after gating converge to gate-conditioned rather than nominal quantities. Under classical linear--Gaussian assumptions, we derive exact expressions for the first- and second-order moments of the innovation conditioned on ellipsoidal gating, and show that gating induces a deterministic, dimension-dependent contraction of the innovation covariance. The analysis is extended to NN association, which is shown to act as an additional statistical selection operator. We prove that selecting the minimum-norm innovation among multiple in-gate measurements introduces an unavoidable energy contraction, implying that nominal innovation statistics cannot be preserved under nontrivial gating and association. Closed-form results in the two-dimensional case quantify the combined effects and illustrate their practical significance.

Figures

Figures reproduced from arXiv: 2512.18508 by the authors.

**Figure 2.** Effect of NN association on normalized innovation squared (NIS) [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 1 linked inside Pith

[1]

A new approach to linear filtering and prediction problems,

R. E. Kalman, “A new approach to linear filtering and prediction problems,”Journal of basic Engineering, vol. 82, no. 1, pp. 35–45, 1960

1960
[2]

New results in linear filtering and prediction theory,

R. E. Kalman and R. S. Bucy, “New results in linear filtering and prediction theory,”Journal of basic engineering, vol. 83, no. 1, pp. 95– 108, 1961

1961
[3]

A. H. Jazwinski,Stochastic processes and filtering theory. Courier Corporation, 2007

2007
[4]

Bar-Shalom, X

Y . Bar-Shalom, X. R. Li, and T. Kirubarajan,Estimation with applica- tions to tracking and navigation: theory algorithms and software. John Wiley & Sons, 2001

2001
[5]

R. G. Brown and P. Y . Hwang,Introduction to random signals and applied Kalman filtering. Wiley New York, Third Edtion, 1992

1992
[6]

On the identification of variances and adaptive kalman filtering,

R. Mehra, “On the identification of variances and adaptive kalman filtering,”IEEE Transactions on automatic control, vol. 15, no. 2, pp. 175–184, 1970

1970
[7]

Adaptive adjustment of noise covariance in kalman filter for dynamic state estimation,

S. Akhlaghi, N. Zhou, and Z. Huang, “Adaptive adjustment of noise covariance in kalman filter for dynamic state estimation,” in2017 IEEE Power & Energy Society General Meeting. IEEE, 2017, pp. 1–5

2017
[8]

A variational bayesian based strong tracking interpolatory cubature kalman filter for maneuvering target tracking,

J. Wang, T. Zhang, X. Xu, and Y . Li, “A variational bayesian based strong tracking interpolatory cubature kalman filter for maneuvering target tracking,”IEEE Access, vol. 6, pp. 52 544–52 560, 2018

2018

Show all 21 references

[9]

A novel multiple-model adaptive kalman filter for an unknown measurement loss probability,

W. Youn, N. Y . Ko, S. Gadsden, and H. Myung, “A novel multiple-model adaptive kalman filter for an unknown measurement loss probability,” IEEE Transactions on Instrumentation and Measurement, 2020

2020
[10]

Ramachandra,Kalman filtering techniques for radar tracking

K. Ramachandra,Kalman filtering techniques for radar tracking. CRC Press, 2018

2018
[11]

Discrete-time state estimation of analog double integrators,

S. Shats, B. Bobrovsky, and U. Shaked, “Discrete-time state estimation of analog double integrators,”IEEE transactions on aerospace and electronic systems, vol. 24, no. 6, pp. 670–677, 1988

1988
[12]

The future of automotive localization algorithms: Available, reliable, and scalable localization: Anywhere and anytime,

R. Karlsson and F. Gustafsson, “The future of automotive localization algorithms: Available, reliable, and scalable localization: Anywhere and anytime,”IEEE signal processing magazine, vol. 34, no. 2, pp. 60–69, 2017

2017
[13]

Kalman filtering with adaptive step size using a covariance-based criterion,

B. Or, B.-Z. Bobrovsky, and I. Klein, “Kalman filtering with adaptive step size using a covariance-based criterion,”IEEE Transactions on Instrumentation and Measurement, vol. 70, pp. 1–10, 2021

2021
[14]

A hybrid model and learning-based adaptive navi- gation filter,

B. Or and I. Klein, “A hybrid model and learning-based adaptive navi- gation filter,”IEEE Transactions on Instrumentation and Measurement, vol. 71, pp. 1–11, 2022

2022
[15]

Learning vehicle trajectory uncertainty,

——, “Learning vehicle trajectory uncertainty,”Engineering Applica- tions of Artificial Intelligence, vol. 122, p. 106101, 2023

2023
[16]

Online tests of kalman filter consistency,

R. Pich ´e, “Online tests of kalman filter consistency,”International Journal of Adaptive Control and Signal Processing, vol. 30, no. 1, pp. 115–124, 2016. 9

2016
[17]

Covariance estimation and gaussianity assessment for state and measurement noise,

J. Dun ´ık, O. Kost, O. Straka, and E. Blasch, “Covariance estimation and gaussianity assessment for state and measurement noise,”Journal of Guidance, Control, and Dynamics, vol. 43, no. 1, pp. 132–139, 2020

2020
[18]

Noise covariance ma- trices in state-space models: A survey and comparison of estimation methods—part i,

J. Dun ´ık, O. Straka, O. Kost, and J. Havl ´ık, “Noise covariance ma- trices in state-space models: A survey and comparison of estimation methods—part i,”International Journal of Adaptive Control and Signal Processing, vol. 31, no. 11, pp. 1505–1543, 2017

2017
[19]

Kalman filter auto-tuning through enforcing chi-squared normalized error distri- butions with bayesian optimization,

Z. Chen, H. Biggie, N. Ahmed, S. Julier, and C. Heckman, “Kalman filter auto-tuning through enforcing chi-squared normalized error distri- butions with bayesian optimization,”arXiv preprint arXiv:2306.07225, 2023

2023 arXiv
[20]

An algorithm for tracking multiple targets,

D. Reid, “An algorithm for tracking multiple targets,”IEEE transactions on Automatic Control, vol. 24, no. 6, pp. 843–854, 2003

2003
[21]

Tracking and data association,

Y . Bar-Shalom, T. E. Fortmann, and P. G. Cable, “Tracking and data association,” 1990. Barak Or(Member, IEEE) received a B.Sc. degree in aerospace engineering (2016), a B.A. degree (cum laude) in economics and management (2016), and an M.Sc. degree in aerospace engineering (2...

1990

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.

[1] [1]

A new approach to linear filtering and prediction problems,

R. E. Kalman, “A new approach to linear filtering and prediction problems,”Journal of basic Engineering, vol. 82, no. 1, pp. 35–45, 1960

1960

[2] [2]

New results in linear filtering and prediction theory,

R. E. Kalman and R. S. Bucy, “New results in linear filtering and prediction theory,”Journal of basic engineering, vol. 83, no. 1, pp. 95– 108, 1961

1961

[3] [3]

A. H. Jazwinski,Stochastic processes and filtering theory. Courier Corporation, 2007

2007

[4] [4]

Bar-Shalom, X

Y . Bar-Shalom, X. R. Li, and T. Kirubarajan,Estimation with applica- tions to tracking and navigation: theory algorithms and software. John Wiley & Sons, 2001

2001

[5] [5]

R. G. Brown and P. Y . Hwang,Introduction to random signals and applied Kalman filtering. Wiley New York, Third Edtion, 1992

1992

[6] [6]

On the identification of variances and adaptive kalman filtering,

R. Mehra, “On the identification of variances and adaptive kalman filtering,”IEEE Transactions on automatic control, vol. 15, no. 2, pp. 175–184, 1970

1970

[7] [7]

Adaptive adjustment of noise covariance in kalman filter for dynamic state estimation,

S. Akhlaghi, N. Zhou, and Z. Huang, “Adaptive adjustment of noise covariance in kalman filter for dynamic state estimation,” in2017 IEEE Power & Energy Society General Meeting. IEEE, 2017, pp. 1–5

2017

[8] [8]

A variational bayesian based strong tracking interpolatory cubature kalman filter for maneuvering target tracking,

J. Wang, T. Zhang, X. Xu, and Y . Li, “A variational bayesian based strong tracking interpolatory cubature kalman filter for maneuvering target tracking,”IEEE Access, vol. 6, pp. 52 544–52 560, 2018

2018