REVIEW 2 major objections 8 minor 2 cited by
Data-Driven and Stealthy Deactivation of Safety Filters
T0 review · 2 major / 8 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read A false-data injection attack can deactivate safety filters using only observed input–output data.
desk verdict A clean data-driven safety-filter attack with a real proof gap: the estimated safe set is a cover, not an inner approximation, of the true latent safe set. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the pair made of a surrogate state-space model in a latent coordinate space and a homeomorphism $T$ that links that space to the true observer state space through $\tilde{f}\circ T = T\circ \hat{f}$. Under this topological equivalence, Proposition 7 and Lemma 8 transfer safety regions between the two spaces, so biasing the latent state $z$ into an estimated safe set $Z$ approximating $T(S)$ biases the true estimate $\hat{x}$ into the actual safe set $S$. The online attack policy (15) is the engine: at each step it solves $$\max_{y_a} \nabla\tilde{h}_S(z_k)^\top \frac{\partial\tilde{f}}{\partial y}\Big|_{z=z_k} y_a \quad \text{subject to } \|y_a - \tilde{h}(z_k)\| \le \tilde{\delta},$$ which is exactly the same policy as the ideal model-based attack, with the identified model and estimated safe set in place of the true ones. The chain rule in (17) shows that this objective measures how much the false measurement locally increases the estimated safety margin. In the linear-observer special case, the latent safe set can be chosen as a convex hull or a minimum covering ellipse, both of which preserve the shape of the true safe set because the homeomorphism is a similarity transform.
What would settle it
Run the same attack on an inverted pendulum whose observer is nonlinear, or with deliberately weak excitation during data collection. If the residual exceeds the detector threshold or the true state does not leave the safe set while the estimate stays inside, the coordinate-change premise is the part that gives way.
Extended reading notes
Core claim
The central claim is that the model-based deactivation attack from earlier work can be replaced by a data-driven attack with no loss of effectiveness. The attacker's offline phase uses the collected input–output sequence $(u, y, \hat{y})$ to identify a state-space model whose output reproduces the observer's predicted measurement, then fits a covering ellipse around the latent trajectory induced by the identified model and uses that ellipse as an estimated safe region; the largest residual in the data gives a lower bound on the detector threshold. Online, the attacker solves a small optimization at each step that selects a false measurement driving the latent state toward the interior of the estimated safe region. The safety transfer is justified by topological equivalence: if the identified dynamics and the true observer dynamics are related by a homeomorphism $T$, then Lemma 8 shows that latent trajectories inside $T(S)$ correspond exactly to estimated trajectories inside $S$, and in the linear case this homeomorphism is a similarity transform that standard subspace identification delivers. The experimental demonstration on an inverted pendulum with a control-barrier-function safety filter confirms the claim: the true state exits the safe set, the perceived state remains inside, and the residual never exceeds the noise level.
Load-bearing premise
The load-bearing premise is that the learned surrogate model is the true observer dynamics up to a fixed change of coordinates; if the training data does not reveal that relationship, the estimated latent safe region need not correspond to any real safety region.
Editorial extensions
If this is right
- An adversary with passive access to the observer's inputs and outputs can deactivate a safety filter: model knowledge, safety-set knowledge, and observer-gain knowledge are all replaceable by data.
- The attack stays stealthy against threshold-based anomaly detectors because the injected measurements are constrained to a lower bound $\tilde{\delta}$ of the true detection threshold.
- Because the attack only assumes the filter projects onto a safe control set of the form (7), it is not limited to control-barrier-function filters.
- In the linear case, any identification method that returns a model similar to the true observer dynamics is sufficient; the exact method used offline is not critical.
- The inverted-pendulum experiment shows that a coverage ellipse fitted to a passively observed latent trajectory is a good enough stand-in for the true safe set to drive the state out of safety.
Reading between the lines
- A consequence left implicit is that the security of a safety filter depends on the secrecy and integrity of the observer's measurement channel; protecting that channel, or deliberately making the observer's internal coordinates hard to identify, would be a direct defensive countermeasure.
- A testable extension would apply the same latent-space safe-set estimation to nonlinear observers using nonlinear system identification; the paper's topological-equivalence argument is stated for general homeomorphisms, but the experiments only cover the linear-similarity case.
- The observed bias in the residual after the attack could be exploited: even if the magnitude stays under the threshold, a defender who monitors the residual's direction or its mean could detect the deactivation attempt.
- A quantitative next step is to measure how attack success degrades as the identification error or the distance between the covering ellipse and $T(S)$ grows, since Lemma 8 is exact only under perfect topological equivalence.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies false-data injection attacks against safety filters. The adversary can observe and modify the sensor measurement y, can observe the control input u, and observes the observer's predicted output ˆy. The attack has an offline phase in which the adversary identifies a latent state-space model of the observer from data, estimates a safe set in the latent space as a convex hull or minimum-volume ellipse covering the latent trajectory, and estimates a lower bound on the detector threshold. In the online phase, the adversary injects a false measurement ya computed by (15) to bias the latent state toward the interior of the estimated safe set. The paper presents theoretical results (Proposition 7, Lemma 8, Lemmas 11–12) linking topological equivalence of the identified and true observer dynamics to preservation of safety, and validates the attack in a simulation of an inverted pendulum with a CBF safety filter.
Significance. The contribution is a data-driven relaxation of the model-based attack of Arnström and Teixeira (2024), requiring no a priori knowledge of dynamics, safe set, or observer gain. The paper is clearly written, includes reproducible code, and the simulation is a forward test of the learned quantities. If the theoretical gap identified below is closed, the result would be significant for the security analysis of safety filters. As it stands, the central claim rests on assumptions that are not formally established.
major comments (2)
- [Section 3.2.1 and Section 3.1.2] The proof chain from z ∈ int(Z) to x̂ ∈ int(S) has a missing link. Z is defined as a cover of the latent trajectory (convex hull or minimum covering ellipse via (13)); Lemma 8 shows only that the trajectory points lie in T(S), not that Z ⊆ T(S). Since a minimal covering ellipse of points inside an ellipse can protrude outside it, the attack may move z into Z \ T(S). The numerical experiment does not verify Z ⊆ T(S) or compare Z with the true T(S). To establish the central claim, the authors should either compute an inner approximation of T(S) (e.g., an inscribed ellipse or a contracted hull) or state and verify conditions under which the cover is contained in T(S).
- [Section 3.1.1 and linear special case after Lemma 8] The assertion that N4SID identifies a model topologically equivalent to the observer is not formalized. Topological equivalence (Definition 6) is the load-bearing assumption for Lemma 8 and Proposition 7, but the paper does not state the identification conditions (persistency of excitation, correct model order, noise-free or asymptotically consistent estimation) under which the identified model is guaranteed to be equivalent. In the experiment the observer is a linearized Kalman filter, so the equivalence may hold approximately, but no verification or residual-based check is provided. The authors should either provide a precise theorem with sufficient conditions for the identifier used, or explicitly characterize the attack as heuristic under ideal identification.
minor comments (8)
- [Eq. (15)] In Eq. (15), the constraint is missing the opening norm symbol; it should read ‖ya − ˜h(zk)‖ ≤ ˜δ.
- [Lemma 8] In Lemma 8, there is a typo: 'assume and that system' should be 'assume that system'.
- [Notation] In the Notation section, 'super set' should be 'super-level set'.
- [Definition 6] In Definition 6, the domain of y is stated as R^{nx}; it should be R^{ny}.
- [Algorithm 1] In Algorithm 1, the instruction 'replace yk with ya by solving (15)' is imprecise; the attacker should compute ya by solving the optimization (15) and then set yk = ya.
- [Figure 5] In Figure 5, the caption 'Latent trajectory ∂Z' is ambiguous; please specify what is plotted (e.g., the latent trajectory and the boundary ∂Z).
- [Section 3.2.1] The statement that U_S(x) ⊂ U_S(˜x) when x is closer to ∂S than ˜x is used as a heuristic; it would be helpful to state this as an explicit assumption or to provide a proof for the class of safe sets used.
- [Eq. (8)] In Eq. (8), the optimization variable is written as ya, which is the same symbol as the false measurement; consider using a different variable for clarity.
Circularity Check
No significant circularity: the data-driven attack is evaluated by a forward simulation, and the self-citation to the authors' prior policy is a transparent, non-load-bearing building block.
full rationale
The derivation chain is not circular. The adversary identifies a state-space model and a latent safe set Z from offline data D, but the central claim (deactivation of the safety filter) is tested by a forward rollout of Algorithm 1 on the inverted-pendulum example, not by re-fitting the learned parameters to force the outcome. The attack policy (15) is explicitly adapted from the authors' earlier model-based policy (8), and the direction of the bias is re-derived in the present paper through the chain-rule expression in (17); thus the prior work is a transparent building block rather than an unverified load-bearing premise. The lower-bound threshold argument in Lemma 13 is a direct consequence of (14). The only substantive weakness is a proof gap, not circularity: Z is constructed as a cover of the sampled latent trajectory, while the argument that biasing z into int(Z) biases x̂ into int(S) requires Z to behave like T(S); Lemmas 11 and 12 only preserve the shape of T(S), and the paper does not establish int(Z) ⊆ int(T(S)). This is a correctness risk in the theoretical claim, but it does not reduce the central result to an input by construction, and the simulation remains an independent forward test.
Assumptions & free parameters
free parameters (4)
- identified state-space matrices (A, Bu, By, K, C)
- latent state dimension nz =
2
- covering ellipse parameters (Q, v)
- threshold scaling constant gamma =
not specified
assumptions (7)
- domain assumption The adversary can observe and modify the measurement y (Assumption 1).
- domain assumption The adversary can observe the control action u (Assumption 2).
- domain assumption The adversary can observe the predicted measurement y-hat (Assumption 3).
- domain assumption The anomaly detector is of the form ||r|| > delta with a norm threshold (Equation 5).
- domain assumption The safety filter is a projection onto a safe control set (Equation 7).
- ad hoc to paper The identified model is topologically equivalent to the observer dynamics.
- domain assumption The detector was not triggered during data collection.
Cite this review
Pith. "Pith review of Data-Driven and Stealthy Deactivation of Safety Filters." pith.science (2026). https://pith.science/paper/ZPQCLKOL
@misc{pith2026241201346,
author = {Pith},
title = {Pith review of: Data-Driven and Stealthy Deactivation of Safety Filters},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZPQCLKOL}},
note = {Machine review of arXiv:2412.01346}
}
read the original abstract
Safety filters ensure that control actions that are executed are always safe, no matter the controller in question. Previous work has proposed a simple and stealthy false-data injection attack for deactivating such safety filters. This attack injects false sensor measurements to bias state estimates toward the interior of a safety region, making the safety filter accept unsafe control actions. The attack does, however, require the adversary to know the dynamics of the system, the safety region used in the safety filter, and the observer gain. In this work we relax these requirements and show how a similar data-injection attack can be performed when the adversary only observes the input and output of the observer that is used by the safety filter, without any a priori knowledge about the system dynamics, safety region, or observer gain. In particular, the adversary uses the observed data to identify a state-space model that describes the observer dynamics, and then approximates a safety region in the identified embedding. We exemplify the data-driven attack on an inverted pendulum, where we show how the attack can make the system leave a safe set, even when a safety filter is supposed to stop this from happening.
Figures
Figures from the paper (2 more)
Forward citations
Cited by 2 Pith papers
-
Secure Safety Filter: Towards Safe Flight Control under Sensor Attacks
A secure safety filter combines secure state reconstruction under bounded noise with a robust control barrier function QP, and tests on a quadrotor show it can keep the drone safe when one sensor stream is spoofed.
-
Secure Safety Filter Design for Sampled-data Nonlinear Systems under Sensor Spoofing Attacks
Secure safety filters for sampled-data nonlinear systems under sensor spoofing are constructed from exact and relaxed observability maps, with safety guaranteed under sparse-observability and control barrier function ...
Reference graph
Works this paper leans on
-
[5]
Linear and nonlinear system identificati on under ℓ1-and group-Lasso regular- ization via L-BFGS-B
Alberto Bemporad. Linear and nonlinear system identificati on under ℓ1-and group-Lasso regular- ization via L-BFGS-B. arXiv preprint arXiv:2403.03827,
-
[7]
Daniel Gedon, Niklas Wahlstr¨ om, Thomas B Sch¨ on, and Lennart Ljung
doi: 10.1109/MCS.2023.3310302. Daniel Gedon, Niklas Wahlstr¨ om, Thomas B Sch¨ on, and Lennart Ljung. Deep state space models for nonlinear system identification. IF AC-PapersOnLine, 54(7):481–486,
arXiv 2023
-
[14]
Kim P Wabersich, Andrew J Taylor, Jason J Choi, Koushil Sreen ath, Claire J Tomlin, Aaron D Ames, and Melanie N Zeilinger. Data-driven safety filters: H amilton-jacobi reachability, control barrier functions, and predictive methods for uncertain systems. IEEE Control Systems Magazine, 43(5):137–177, 2023a. Kim P . Wabersich, Andrew J. Taylor, Jason J. Choi...
-
[1960]
Distrib uted quickest detection of cyber- attacks in smart grid
Mehmet Necip Kurt, Y asin Yılmaz, and Xiaodong Wang. Distrib uted quickest detection of cyber- attacks in smart grid. IEEE Transactions on Information F orensics and Security , 13(8):2015– 2030,
work page 2015
-
[1987]
Jeremy Coulson, John Lygeros, and Florian D¨ orfler
doi: 10.1109/TAC.1987.1104658. Jeremy Coulson, John Lygeros, and Florian D¨ orfler. Data-en abled predictive control: In the shal- lows of the DeePC. In 2019 18th European Control Conference (ECC) , pages 307–312. IEEE,
-
[1994]
doi: https://doi.org/10.1016/0005-1098(94)90230-5
ISSN 0005-1098. doi: https://doi.org/10.1016/0005-1098(94)90230-5. Sp ecial issue on statistical signal processing and control. Peter V an Overschee and BL0888 De Moor. Subspace identification for linear systems: The- ory—Implementation—Applications . Springer Science & Business Media,
-
[2012]
Linear model predictive safety certification for learning- based control
Kim P Wabersich and Melanie N Zeilinger. Linear model predictive safety certification for learning- based control. In 2018 IEEE Conference on Decision and Control (CDC) , pages 7130–7135. IEEE,
work page 2018
-
[2017]
doi: 10.1109/TAC.2016.2638961. Anuradha M. Annaswamy, Karl H. Johansson, and George J. Papp as. Control for Societal-scale Challenges: Road Map 2030 . IEEE Control Systems Society, May
arXiv 2016
Show all 14 references
-
[2018]
Lennart Ljung
doi: 10.1109/TIFS.2018.2800908. Lennart Ljung. System identification (2nd ed.): theory for the user . Prentice Hall PTR, USA,
2018
-
[2019]
Peter V an Overschee and Bart De Moor
doi: 10.23919/ACC.2019.8814731. Peter V an Overschee and Bart De Moor. N4SID: Subspace algori thms for the identification of combined deterministic-stochastic systems. Automatica, 30(1):75–93,
2019
-
[2021]
On the confidentiality of linear anomaly detector states
David Umsonst, Ehsan Nekouei, Andr´ e Teixeira, and Henrik S andberg. On the confidentiality of linear anomaly detector states. In 2019 American Control Conference (ACC) , pages 397–403,
2019
-
[2022]
Somil Bansal, Mo Chen, Sylvia Herbert, and Claire J Tomlin
doi: 10.1109/TAC.2022.3176430. Somil Bansal, Mo Chen, Sylvia Herbert, and Claire J Tomlin. H amilton-jacobi reachability: A brief overview and recent advances. In 2017 IEEE 56th Annual Conference on Decision and Control (CDC), pages 2242–2253. IEEE,
2022
-
[2023]
Teixeira
Daniel Arnstr¨ om and Andr´ e M.H. Teixeira. Stealthy deactivation of safety filters. In 2024 European Control Conference (ECC), pages 3077–3082,
2024
-
[2024]
Daniel Arnstr¨ om, Alberto Bemporad, and Daniel Axehill
doi: 10.23919/ECC64448.2024.105 90978. Daniel Arnstr¨ om, Alberto Bemporad, and Daniel Axehill. A d ual active-set solver for embedded quadratic programming using recursive LDLT updates. IEEE Transactions on Automatic Control, 67(8):4362–4369,
2024
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.