Pith. sign in

REVIEW 2 major objections 8 minor 2 cited by

Data-Driven and Stealthy Deactivation of Safety Filters

T0 review · 2 major / 8 minor · reviewed 2026-08-12 · deepseek-v4-flash

Pith's one-line read A false-data injection attack can deactivate safety filters using only observed input–output data.

desk verdict A clean data-driven safety-filter attack with a real proof gap: the estimated safe set is a cover, not an inner approximation, of the true latent safe set. read the letter →

arxiv 2412.01346 v1 pith:ZPQCLKOL submitted 2024-12-02 eess.SY cs.SY

classification eess.SYcs.SY
keywords safetyfiltersfalsedatainjectionattacksstealthysystemidentificationcontrolbarrierfunctionscyber-physicalsecuritystateestimationinvertedpendulum
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Safety filters are supposed to be the last layer that stops unsafe control commands from reaching a physical system. The paper claims that this layer can be switched off by an adversary who knows nothing about the system dynamics, the safety region, or the observer gain, and who only watches the inputs and outputs of the observer inside the filter. The adversary learns a surrogate state-space model from the observed data, draws an approximate safety region around the resulting latent trajectory, and then injects false measurements that push the filter's state estimate deeper into that region. On an inverted pendulum equipped with a control-barrier-function safety filter, the attack makes the true state leave the safe set while the filter's estimate stays inside and the anomaly detector stays quiet.

What carries the argument

The load-bearing mechanism is the pair made of a surrogate state-space model in a latent coordinate space and a homeomorphism $T$ that links that space to the true observer state space through $\tilde{f}\circ T = T\circ \hat{f}$. Under this topological equivalence, Proposition 7 and Lemma 8 transfer safety regions between the two spaces, so biasing the latent state $z$ into an estimated safe set $Z$ approximating $T(S)$ biases the true estimate $\hat{x}$ into the actual safe set $S$. The online attack policy (15) is the engine: at each step it solves $$\max_{y_a} \nabla\tilde{h}_S(z_k)^\top \frac{\partial\tilde{f}}{\partial y}\Big|_{z=z_k} y_a \quad \text{subject to } \|y_a - \tilde{h}(z_k)\| \le \tilde{\delta},$$ which is exactly the same policy as the ideal model-based attack, with the identified model and estimated safe set in place of the true ones. The chain rule in (17) shows that this objective measures how much the false measurement locally increases the estimated safety margin. In the linear-observer special case, the latent safe set can be chosen as a convex hull or a minimum covering ellipse, both of which preserve the shape of the true safe set because the homeomorphism is a similarity transform.

What would settle it

Run the same attack on an inverted pendulum whose observer is nonlinear, or with deliberately weak excitation during data collection. If the residual exceeds the detector threshold or the true state does not leave the safe set while the estimate stays inside, the coordinate-change premise is the part that gives way.

Watch

Extended reading notes

Core claim

The central claim is that the model-based deactivation attack from earlier work can be replaced by a data-driven attack with no loss of effectiveness. The attacker's offline phase uses the collected input–output sequence $(u, y, \hat{y})$ to identify a state-space model whose output reproduces the observer's predicted measurement, then fits a covering ellipse around the latent trajectory induced by the identified model and uses that ellipse as an estimated safe region; the largest residual in the data gives a lower bound on the detector threshold. Online, the attacker solves a small optimization at each step that selects a false measurement driving the latent state toward the interior of the estimated safe region. The safety transfer is justified by topological equivalence: if the identified dynamics and the true observer dynamics are related by a homeomorphism $T$, then Lemma 8 shows that latent trajectories inside $T(S)$ correspond exactly to estimated trajectories inside $S$, and in the linear case this homeomorphism is a similarity transform that standard subspace identification delivers. The experimental demonstration on an inverted pendulum with a control-barrier-function safety filter confirms the claim: the true state exits the safe set, the perceived state remains inside, and the residual never exceeds the noise level.

Load-bearing premise

The load-bearing premise is that the learned surrogate model is the true observer dynamics up to a fixed change of coordinates; if the training data does not reveal that relationship, the estimated latent safe region need not correspond to any real safety region.

Editorial extensions

If this is right

  • An adversary with passive access to the observer's inputs and outputs can deactivate a safety filter: model knowledge, safety-set knowledge, and observer-gain knowledge are all replaceable by data.
  • The attack stays stealthy against threshold-based anomaly detectors because the injected measurements are constrained to a lower bound $\tilde{\delta}$ of the true detection threshold.
  • Because the attack only assumes the filter projects onto a safe control set of the form (7), it is not limited to control-barrier-function filters.
  • In the linear case, any identification method that returns a model similar to the true observer dynamics is sufficient; the exact method used offline is not critical.
  • The inverted-pendulum experiment shows that a coverage ellipse fitted to a passively observed latent trajectory is a good enough stand-in for the true safe set to drive the state out of safety.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A consequence left implicit is that the security of a safety filter depends on the secrecy and integrity of the observer's measurement channel; protecting that channel, or deliberately making the observer's internal coordinates hard to identify, would be a direct defensive countermeasure.
  • A testable extension would apply the same latent-space safe-set estimation to nonlinear observers using nonlinear system identification; the paper's topological-equivalence argument is stated for general homeomorphisms, but the experiments only cover the linear-similarity case.
  • The observed bias in the residual after the attack could be exploited: even if the magnitude stays under the threshold, a defender who monitors the residual's direction or its mean could detect the deactivation attempt.
  • A quantitative next step is to measure how attack success degrades as the identification error or the distance between the covering ellipse and $T(S)$ grows, since Lemma 8 is exact only under perfect topological equivalence.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 8 minor

Summary. The paper studies false-data injection attacks against safety filters. The adversary can observe and modify the sensor measurement y, can observe the control input u, and observes the observer's predicted output ˆy. The attack has an offline phase in which the adversary identifies a latent state-space model of the observer from data, estimates a safe set in the latent space as a convex hull or minimum-volume ellipse covering the latent trajectory, and estimates a lower bound on the detector threshold. In the online phase, the adversary injects a false measurement ya computed by (15) to bias the latent state toward the interior of the estimated safe set. The paper presents theoretical results (Proposition 7, Lemma 8, Lemmas 11–12) linking topological equivalence of the identified and true observer dynamics to preservation of safety, and validates the attack in a simulation of an inverted pendulum with a CBF safety filter.

Significance. The contribution is a data-driven relaxation of the model-based attack of Arnström and Teixeira (2024), requiring no a priori knowledge of dynamics, safe set, or observer gain. The paper is clearly written, includes reproducible code, and the simulation is a forward test of the learned quantities. If the theoretical gap identified below is closed, the result would be significant for the security analysis of safety filters. As it stands, the central claim rests on assumptions that are not formally established.

major comments (2)
  1. [Section 3.2.1 and Section 3.1.2] The proof chain from z ∈ int(Z) to x̂ ∈ int(S) has a missing link. Z is defined as a cover of the latent trajectory (convex hull or minimum covering ellipse via (13)); Lemma 8 shows only that the trajectory points lie in T(S), not that Z ⊆ T(S). Since a minimal covering ellipse of points inside an ellipse can protrude outside it, the attack may move z into Z \ T(S). The numerical experiment does not verify Z ⊆ T(S) or compare Z with the true T(S). To establish the central claim, the authors should either compute an inner approximation of T(S) (e.g., an inscribed ellipse or a contracted hull) or state and verify conditions under which the cover is contained in T(S).
  2. [Section 3.1.1 and linear special case after Lemma 8] The assertion that N4SID identifies a model topologically equivalent to the observer is not formalized. Topological equivalence (Definition 6) is the load-bearing assumption for Lemma 8 and Proposition 7, but the paper does not state the identification conditions (persistency of excitation, correct model order, noise-free or asymptotically consistent estimation) under which the identified model is guaranteed to be equivalent. In the experiment the observer is a linearized Kalman filter, so the equivalence may hold approximately, but no verification or residual-based check is provided. The authors should either provide a precise theorem with sufficient conditions for the identifier used, or explicitly characterize the attack as heuristic under ideal identification.
minor comments (8)
  1. [Eq. (15)] In Eq. (15), the constraint is missing the opening norm symbol; it should read ‖ya − ˜h(zk)‖ ≤ ˜δ.
  2. [Lemma 8] In Lemma 8, there is a typo: 'assume and that system' should be 'assume that system'.
  3. [Notation] In the Notation section, 'super set' should be 'super-level set'.
  4. [Definition 6] In Definition 6, the domain of y is stated as R^{nx}; it should be R^{ny}.
  5. [Algorithm 1] In Algorithm 1, the instruction 'replace yk with ya by solving (15)' is imprecise; the attacker should compute ya by solving the optimization (15) and then set yk = ya.
  6. [Figure 5] In Figure 5, the caption 'Latent trajectory ∂Z' is ambiguous; please specify what is plotted (e.g., the latent trajectory and the boundary ∂Z).
  7. [Section 3.2.1] The statement that U_S(x) ⊂ U_S(˜x) when x is closer to ∂S than ˜x is used as a heuristic; it would be helpful to state this as an explicit assumption or to provide a proof for the class of safe sets used.
  8. [Eq. (8)] In Eq. (8), the optimization variable is written as ya, which is the same symbol as the false measurement; consider using a different variable for clarity.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the data-driven attack is evaluated by a forward simulation, and the self-citation to the authors' prior policy is a transparent, non-load-bearing building block.

full rationale

The derivation chain is not circular. The adversary identifies a state-space model and a latent safe set Z from offline data D, but the central claim (deactivation of the safety filter) is tested by a forward rollout of Algorithm 1 on the inverted-pendulum example, not by re-fitting the learned parameters to force the outcome. The attack policy (15) is explicitly adapted from the authors' earlier model-based policy (8), and the direction of the bias is re-derived in the present paper through the chain-rule expression in (17); thus the prior work is a transparent building block rather than an unverified load-bearing premise. The lower-bound threshold argument in Lemma 13 is a direct consequence of (14). The only substantive weakness is a proof gap, not circularity: Z is constructed as a cover of the sampled latent trajectory, while the argument that biasing z into int(Z) biases x̂ into int(S) requires Z to behave like T(S); Lemmas 11 and 12 only preserve the shape of T(S), and the paper does not establish int(Z) ⊆ int(T(S)). This is a correctness risk in the theoretical claim, but it does not reduce the central result to an input by construction, and the simulation remains an independent forward test.

Assumptions & free parameters 4 free parameters · 7 assumptions · 0 invented entities

The attack rests on the attacker's observability assumptions and the existence of a topological equivalence between the learned and true observer dynamics. The free parameters are the identified model, the safe-set approximation, and the threshold scaling, all fitted from data or chosen by the adversary.

free parameters (4)
  • identified state-space matrices (A, Bu, By, K, C)
    These matrices are fitted to the observed data D using N4SID in Section 3.1.1. They define the learned observer model and are the core of the attack.
  • latent state dimension nz = 2
    The latent state dimension is chosen by the adversary (Section 4) and is a free design choice for the system identification.
  • covering ellipse parameters (Q, v)
    The minimum covering ellipse is computed by solving SDP (13) in Section 3.1.2, fitting Q and v to the latent trajectory.
  • threshold scaling constant gamma = not specified
    The paper defines gamma in (0,1] in Section 3.1.3 to lower-bound the detector threshold, but does not specify its value in the experiment. It is an adversary-chosen tuning parameter.
assumptions (7)
  • domain assumption The adversary can observe and modify the measurement y (Assumption 1).
    This is the attacker capability that motivates the entire paper; it is stated in Section 2.1.
  • domain assumption The adversary can observe the control action u (Assumption 2).
    Needed for the data-driven identification; stated in Section 2.1.
  • domain assumption The adversary can observe the predicted measurement y-hat (Assumption 3).
    Justified as equivalent to observing the residual; stated in Section 2.1.
  • domain assumption The anomaly detector is of the form ||r|| > delta with a norm threshold (Equation 5).
    The whole stealthiness argument relies on this detector form; introduced in Section 2.
  • domain assumption The safety filter is a projection onto a safe control set (Equation 7).
    Covers CBF-based, predictive, and reachability safety filters; stated in Section 2.
  • ad hoc to paper The identified model is topologically equivalent to the observer dynamics.
    The theory in Section 3.1.2 requires this equivalence, but it is not verified in the experiment. It is assumed for the linear case through similarity transforms.
  • domain assumption The detector was not triggered during data collection.
    Used in Section 3.1.3 to lower-bound the threshold; if violated, the bound is invalid.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Data-Driven and Stealthy Deactivation of Safety Filters." pith.science (2026). https://pith.science/paper/ZPQCLKOL

@misc{pith2026241201346,
  author       = {Pith},
  title        = {Pith review of: Data-Driven and Stealthy Deactivation of Safety Filters},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZPQCLKOL}},
  note         = {Machine review of arXiv:2412.01346}
}
read the original abstract

Safety filters ensure that control actions that are executed are always safe, no matter the controller in question. Previous work has proposed a simple and stealthy false-data injection attack for deactivating such safety filters. This attack injects false sensor measurements to bias state estimates toward the interior of a safety region, making the safety filter accept unsafe control actions. The attack does, however, require the adversary to know the dynamics of the system, the safety region used in the safety filter, and the observer gain. In this work we relax these requirements and show how a similar data-injection attack can be performed when the adversary only observes the input and output of the observer that is used by the safety filter, without any a priori knowledge about the system dynamics, safety region, or observer gain. In particular, the adversary uses the observed data to identify a state-space model that describes the observer dynamics, and then approximates a safety region in the identified embedding. We exemplify the data-driven attack on an inverted pendulum, where we show how the attack can make the system leave a safe set, even when a safety filter is supposed to stop this from happening.

Figures

Figures reproduced from arXiv: 2412.01346 by the authors.

Figure 1
Figure 1. Overview of the system architecture considered in Arnstr ¨om and Teixeira (2024) and in this paper. The safety filter produces a safe control action u given a desired control uc and estimated state xˆ. An adversary tries to deactivate this filter through false-data injections on the communication channel between the sensors and the observer by replacing the true measurement y with y a [PITH_FULL_IMAGE:figures/full… view at source ↗
Figure 2
Figure 2. The pipeline for the offline phase of the data-drive [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Inverted pendulum with mass m and length l. The angle θ is con￾trolled by applying the torque u. which is equivalent to the policy in (8), but with an estimated system dynamics ( ˜f and h˜), an estimated safe set (h˜ S), and a lower-bound ˜δ of δ. Moreover, the adversary updates the latent state z by using the identified model of the form (10) with u, y a and yˆ, resulting in the update zk+1 = ˜f(zk, uk, ya (zk)) + … view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Latent trajectory from simulating (9) with the collected data D. is d dt h θ ˙θ i = h ˙θ g l sin θ i + h 0 1 ml2 i u, with parameter values m = 2 kg, l = 1 m , and g = 10 m/s2 (the same as in Wabersich et al. (2023b); Alan et al. (2023)). Moreover, the applied torque u…
Figure 6
Figure 6. Figure 6: The actual and perceived state trajectories, and the corresponding residual r, when the false-data attack in Algorithm 1 is initiated at t = 1. Consequently, the state leaves the safe set. the attack at t = 1 second, which, as in shown in [PITH_FULL_IMAGE:figures/full…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Secure Safety Filter: Towards Safe Flight Control under Sensor Attacks

    cs.RO 2025-05 conditional novelty 6.0 of 10

    A secure safety filter combines secure state reconstruction under bounded noise with a robust control barrier function QP, and tests on a quadrotor show it can keep the drone safe when one sensor stream is spoofed.

  2. Secure Safety Filter Design for Sampled-data Nonlinear Systems under Sensor Spoofing Attacks

    eess.SY 2025-05 conditional novelty 6.0 of 10

    Secure safety filters for sampled-data nonlinear systems under sensor spoofing are constructed from exact and relaxed observability maps, with safety guaranteed under sparse-observability and control barrier function ...

Reference graph

Works this paper leans on

14 extracted references · 7 canonical work pages · cited by 2 Pith papers

  1. [5]

    Linear and nonlinear system identificati on under ℓ1-and group-Lasso regular- ization via L-BFGS-B

    Alberto Bemporad. Linear and nonlinear system identificati on under ℓ1-and group-Lasso regular- ization via L-BFGS-B. arXiv preprint arXiv:2403.03827,

  2. [7]

    Daniel Gedon, Niklas Wahlstr¨ om, Thomas B Sch¨ on, and Lennart Ljung

    doi: 10.1109/MCS.2023.3310302. Daniel Gedon, Niklas Wahlstr¨ om, Thomas B Sch¨ on, and Lennart Ljung. Deep state space models for nonlinear system identification. IF AC-PapersOnLine, 54(7):481–486,

  3. [14]

    Data-driven safety filters: H amilton-jacobi reachability, control barrier functions, and predictive methods for uncertain systems

    Kim P Wabersich, Andrew J Taylor, Jason J Choi, Koushil Sreen ath, Claire J Tomlin, Aaron D Ames, and Melanie N Zeilinger. Data-driven safety filters: H amilton-jacobi reachability, control barrier functions, and predictive methods for uncertain systems. IEEE Control Systems Magazine, 43(5):137–177, 2023a. Kim P . Wabersich, Andrew J. Taylor, Jason J. Choi...

  4. [1960]

    Distrib uted quickest detection of cyber- attacks in smart grid

    Mehmet Necip Kurt, Y asin Yılmaz, and Xiaodong Wang. Distrib uted quickest detection of cyber- attacks in smart grid. IEEE Transactions on Information F orensics and Security , 13(8):2015– 2030,

  5. [1987]

    Jeremy Coulson, John Lygeros, and Florian D¨ orfler

    doi: 10.1109/TAC.1987.1104658. Jeremy Coulson, John Lygeros, and Florian D¨ orfler. Data-en abled predictive control: In the shal- lows of the DeePC. In 2019 18th European Control Conference (ECC) , pages 307–312. IEEE,

  6. [1994]

    doi: https://doi.org/10.1016/0005-1098(94)90230-5

    ISSN 0005-1098. doi: https://doi.org/10.1016/0005-1098(94)90230-5. Sp ecial issue on statistical signal processing and control. Peter V an Overschee and BL0888 De Moor. Subspace identification for linear systems: The- ory—Implementation—Applications . Springer Science & Business Media,

  7. [2012]

    Linear model predictive safety certification for learning- based control

    Kim P Wabersich and Melanie N Zeilinger. Linear model predictive safety certification for learning- based control. In 2018 IEEE Conference on Decision and Control (CDC) , pages 7130–7135. IEEE,

  8. [2017]

    Anuradha M

    doi: 10.1109/TAC.2016.2638961. Anuradha M. Annaswamy, Karl H. Johansson, and George J. Papp as. Control for Societal-scale Challenges: Road Map 2030 . IEEE Control Systems Society, May

Show all 14 references
  1. [2018]

    Lennart Ljung

    doi: 10.1109/TIFS.2018.2800908. Lennart Ljung. System identification (2nd ed.): theory for the user . Prentice Hall PTR, USA,

  2. [2019]

    Peter V an Overschee and Bart De Moor

    doi: 10.23919/ACC.2019.8814731. Peter V an Overschee and Bart De Moor. N4SID: Subspace algori thms for the identification of combined deterministic-stochastic systems. Automatica, 30(1):75–93,

  3. [2021]

    On the confidentiality of linear anomaly detector states

    David Umsonst, Ehsan Nekouei, Andr´ e Teixeira, and Henrik S andberg. On the confidentiality of linear anomaly detector states. In 2019 American Control Conference (ACC) , pages 397–403,

  4. [2022]

    Somil Bansal, Mo Chen, Sylvia Herbert, and Claire J Tomlin

    doi: 10.1109/TAC.2022.3176430. Somil Bansal, Mo Chen, Sylvia Herbert, and Claire J Tomlin. H amilton-jacobi reachability: A brief overview and recent advances. In 2017 IEEE 56th Annual Conference on Decision and Control (CDC), pages 2242–2253. IEEE,

  5. [2023]

    Teixeira

    Daniel Arnstr¨ om and Andr´ e M.H. Teixeira. Stealthy deactivation of safety filters. In 2024 European Control Conference (ECC), pages 3077–3082,

  6. [2024]

    Daniel Arnstr¨ om, Alberto Bemporad, and Daniel Axehill

    doi: 10.23919/ECC64448.2024.105 90978. Daniel Arnstr¨ om, Alberto Bemporad, and Daniel Axehill. A d ual active-set solver for embedded quadratic programming using recursive LDLT updates. IEEE Transactions on Automatic Control, 67(8):4362–4369,

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.