{"id":"eca487c9-340e-4ea5-8e7b-bb95aa1b5bcb","arxiv_id":"2412.01346","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A data-driven false-data injection attack that identifies the observer dynamics and approximates the safety region to stealthily deactivate safety filters.","lead":"This paper shows how an attacker can disable a safety filter using only recorded input/output data, without knowing the system model or safety region. The attack learns a model of the observer and a safe region, then injects fake sensor measurements to trick the filter into accepting unsafe actions.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The data-only safe-set cover is not shown to lie inside T(S); biasing z into the learned ellipse can push the true estimate outside S even when the identified model is exactly equivalent.","rationale":"The reader's conditional verdict is reasonable, but the weakest point is not only the topological-equivalence assumption on the identified model. Even if N4SID returns an exactly equivalent realization, the attack objective (15) maximizes h̃_S, and h̃_S is defined by a finite-data cover of the latent trajectory. Since this cover is an outer approximation of observed safe points, it need not be an inner approximation of T(S). Proposition 7 and Lemma 8 only map z∈T(S) to x̂∈S; they say nothing about z∈Z\\T(S). Lemmas 11 and 12 do not close this gap. The concern is concrete and fixable: add a condition ensuring Z⊆T(S), or verify the containment in the experiment. Because the paper provides only a single successful simulation and the theoretical justification has this unverified link, conditional acceptance with the containment check is appropriate. The code is promised and the experimental setup is standard; the issue is not reproducibility but a missing proof step and an untested failure mode.","tokens_in":11368,"tokens_out":11022,"duration_ms":111836,"concrete_test":"Using the released code, compute the true linear observer realization and the N4SID realization, and recover the similarity transform T between x̂ and z from the identified matrices. For the CBF h_S in Section 4, compute the exact ellipse T(S). Then solve (13) on the offline latent trajectory and test whether Z⊆T(S) by sampling the boundary of Z and checking h_S(T^{-1}(z))≥0. Then run Algorithm 1 from several initial conditions inside S; record whether any attack moves z into Z\\T(S) while the true state leaves S. If such a case exists, the safe-set approximation is the load-bearing failure; if all successful attacks enter Z∩T(S) and Z⊆T(S) on the experimental data, the authors should still state this containment as an explicit assumption or verified condition.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section 3.2.1 argues that the attack biases z into the interior of the estimated set Z and then invokes Proposition 7 to conclude that x̂ is biased into the interior of S. This chain is missing a link. In Section 3.1.2, Z is defined as a cover of the offline latent trajectory (convex hull or minimum-volume ellipse from (13)), not as a subset of T(S). Lemmas 11 and 12 only show that, if f̂ is linear and S is a polyhedron/ellipse, then T(S) is also a polyhedron/ellipse; they do not show that the computed Z is contained in T(S). A minimum covering ellipse of points inside an ellipse can protrude outside that ellipse: two points on a small arc of a disk boundary have a minimal enclosing circle whose center lies inside the disk but whose far side extends beyond the boundary. Consequently, Algorithm 1 can move z into Z\\T(S), and the proof link from z∈int(Z) to x̂∈int(S), and hence to deactivation in Definition 2, is broken. The numerical experiment does not test this: no exact T(S) is computed, and no persistency-of-excitation, boundary-coverage, or inner-approximation condition is imposed. Thus even granting perfect topological equivalence of the identified model, the paper's own theory does not establish the central claim; the single simulation may rely on favorable data coverage.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies false-data injection attacks against safety filters. The adversary can observe and modify the sensor measurement y, can observe the control input u, and observes the observer's predicted output ˆy. The attack has an offline phase in which the adversary identifies a latent state-space model of the observer from data, estimates a safe set in the latent space as a convex hull or minimum-volume ellipse covering the latent trajectory, and estimates a lower bound on the detector threshold. In the online phase, the adversary injects a false measurement ya computed by (15) to bias the latent state toward the interior of the estimated safe set. The paper presents theoretical results (Proposition 7, Lemma 8, Lemmas 11–12) linking topological equivalence of the identified and true observer dynamics to preservation of safety, and validates the attack in a simulation of an inverted pendulum with a CBF safety filter.","tokens_in":11662,"tokens_out":8480,"duration_ms":73222,"significance":"The contribution is a data-driven relaxation of the model-based attack of Arnström and Teixeira (2024), requiring no a priori knowledge of dynamics, safe set, or observer gain. The paper is clearly written, includes reproducible code, and the simulation is a forward test of the learned quantities. If the theoretical gap identified below is closed, the result would be significant for the security analysis of safety filters. As it stands, the central claim rests on assumptions that are not formally established.","major_comments":[{"comment":"The proof chain from z ∈ int(Z) to x̂ ∈ int(S) has a missing link. Z is defined as a cover of the latent trajectory (convex hull or minimum covering ellipse via (13)); Lemma 8 shows only that the trajectory points lie in T(S), not that Z ⊆ T(S). Since a minimal covering ellipse of points inside an ellipse can protrude outside it, the attack may move z into Z \\ T(S). The numerical experiment does not verify Z ⊆ T(S) or compare Z with the true T(S). To establish the central claim, the authors should either compute an inner approximation of T(S) (e.g., an inscribed ellipse or a contracted hull) or state and verify conditions under which the cover is contained in T(S).","section":"Section 3.2.1 and Section 3.1.2"},{"comment":"The assertion that N4SID identifies a model topologically equivalent to the observer is not formalized. Topological equivalence (Definition 6) is the load-bearing assumption for Lemma 8 and Proposition 7, but the paper does not state the identification conditions (persistency of excitation, correct model order, noise-free or asymptotically consistent estimation) under which the identified model is guaranteed to be equivalent. In the experiment the observer is a linearized Kalman filter, so the equivalence may hold approximately, but no verification or residual-based check is provided. The authors should either provide a precise theorem with sufficient conditions for the identifier used, or explicitly characterize the attack as heuristic under ideal identification.","section":"Section 3.1.1 and linear special case after Lemma 8"}],"minor_comments":[{"comment":"In Eq. (15), the constraint is missing the opening norm symbol; it should read ‖ya − ˜h(zk)‖ ≤ ˜δ.","section":"Eq. (15)"},{"comment":"In Lemma 8, there is a typo: 'assume and that system' should be 'assume that system'.","section":"Lemma 8"},{"comment":"In the Notation section, 'super set' should be 'super-level set'.","section":"Notation"},{"comment":"In Definition 6, the domain of y is stated as R^{nx}; it should be R^{ny}.","section":"Definition 6"},{"comment":"In Algorithm 1, the instruction 'replace yk with ya by solving (15)' is imprecise; the attacker should compute ya by solving the optimization (15) and then set yk = ya.","section":"Algorithm 1"},{"comment":"In Figure 5, the caption 'Latent trajectory ∂Z' is ambiguous; please specify what is plotted (e.g., the latent trajectory and the boundary ∂Z).","section":"Figure 5"},{"comment":"The statement that U_S(x) ⊂ U_S(˜x) when x is closer to ∂S than ˜x is used as a heuristic; it would be helpful to state this as an explicit assumption or to provide a proof for the class of safe sets used.","section":"Section 3.2.1"},{"comment":"In Eq. (8), the optimization variable is written as ya, which is the same symbol as the false measurement; consider using a different variable for clarity.","section":"Eq. (8)"}],"recommendation":"major_revision","confidential_remarks":"The manuscript addresses a timely security question and includes a reproducible simulation, which is commendable. The main theoretical guarantees, however, are incomplete in a way that affects the central claim: the estimated latent safe set is not shown to be an inner approximation of the true latent safe set, and the identification step lacks formal conditions for topological equivalence. These issues are likely fixable with additional assumptions or by reframing the contribution as an empirical study. If the authors can close the gap, the paper would be a solid contribution; otherwise, it may be better suited to a conference venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe headline: this paper has a good idea and a real result, but the main proof has a hole. The attack—learn an observer model from input/output data, approximate the safety region in latent space with a covering ellipse, then bias the latent state into that ellipse—is a natural data-driven version of the authors' earlier model-based attack. The simulation on the inverted pendulum is clean and shows the attack working. But the theory does not connect the learned set to the true safety set in the way the authors claim.\n\nThe new part is genuinely new: no prior knowledge of dynamics, safety region, or observer gain. The writing is clear, Lemma 8 (latent trajectories map through a homeomorphism) and the linear Lemmas 11–12 are correct. Using topological equivalence to transfer safety sets is a nice framing. Code is promised.\n\nThe soft spot is the one in the stress-test note, and it's real. Section 3.2.1 says Z is \"a cover of points in T(S)\" and then argues that biasing z into int(Z) biases x̂ into int(S). That is a non-sequitur. A cover of points inside an ellipse can protrude outside it—the minimum enclosing ellipse of a few points on the boundary of a disk extends beyond the disk. So Algorithm 1 can push z into Z\\T(S), and then Proposition 7 gives no conclusion about x̂. Lemmas 11 and 12 only say T(S) is the same shape family; they don't give an inner approximation. To make the argument work the authors need either to construct Z as an inner approximation of T(S) (hard without knowing T) or to state the result as empirical: the attack works in the simulated scenario, not as a guarantee. The single experiment doesn't test the gap: no exact T(S) is computed, no boundary coverage or persistence of excitation is checked.\n\nOther issues are minor. The identified model's topological equivalence is assumed; the paper doesn't validate the fit or discuss transient mismatch of the latent state's initial condition. The residual bias after the attack is acknowledged and is fine.\n\nWho this is for: CPS security researchers and people building safety filters. It's a useful feasibility result and a good starting point for discussion. It deserves peer review, but the authors should be asked to fix the gap or explicitly reframe the claims as empirical.\n\nRecommendation: send it out, with a request for revision on the safe-set approximation issue.","headline":"A clean data-driven safety-filter attack with a real proof gap: the estimated safe set is a cover, not an inner approximation, of the true latent safe set.","tokens_in":12178,"tokens_out":4417,"would_cite":true,"duration_ms":39931,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A false-data injection attack can deactivate safety filters using only observed input–output data.","keywords":["safety filters","false data injection attacks","stealthy attacks","system identification","control barrier functions","cyber-physical system security","state estimation","inverted pendulum"],"falsifier":"Run the same attack on an inverted pendulum whose observer is nonlinear, or with deliberately weak excitation during data collection. If the residual exceeds the detector threshold or the true state does not leave the safe set while the estimate stays inside, the coordinate-change premise is the part that gives way.","tokens_in":11160,"feed_emoji":"🎯","tokens_out":11237,"duration_ms":92171,"temperature":0.7,"pith_summary":"Safety filters are supposed to be the last layer that stops unsafe control commands from reaching a physical system. The paper claims that this layer can be switched off by an adversary who knows nothing about the system dynamics, the safety region, or the observer gain, and who only watches the inputs and outputs of the observer inside the filter. The adversary learns a surrogate state-space model from the observed data, draws an approximate safety region around the resulting latent trajectory, and then injects false measurements that push the filter's state estimate deeper into that region. On an inverted pendulum equipped with a control-barrier-function safety filter, the attack makes the true state leave the safe set while the filter's estimate stays inside and the anomaly detector stays quiet.","feed_headline":"Data-only attack deactivates safety filters without model knowledge","feed_subtitle":"An adversary who watches the observer's inputs and outputs can push an inverted pendulum out of its safe set.","key_machinery":"The load-bearing mechanism is the pair made of a surrogate state-space model in a latent coordinate space and a homeomorphism $T$ that links that space to the true observer state space through $\\tilde{f}\\circ T = T\\circ \\hat{f}$. Under this topological equivalence, Proposition 7 and Lemma 8 transfer safety regions between the two spaces, so biasing the latent state $z$ into an estimated safe set $Z$ approximating $T(S)$ biases the true estimate $\\hat{x}$ into the actual safe set $S$. The online attack policy (15) is the engine: at each step it solves\n$$\\max_{y_a} \\nabla\\tilde{h}_S(z_k)^\\top \\frac{\\partial\\tilde{f}}{\\partial y}\\Big|_{z=z_k} y_a \\quad \\text{subject to } \\|y_a - \\tilde{h}(z_k)\\| \\le \\tilde{\\delta},$$\nwhich is exactly the same policy as the ideal model-based attack, with the identified model and estimated safe set in place of the true ones. The chain rule in (17) shows that this objective measures how much the false measurement locally increases the estimated safety margin. In the linear-observer special case, the latent safe set can be chosen as a convex hull or a minimum covering ellipse, both of which preserve the shape of the true safe set because the homeomorphism is a similarity transform.","core_discovery":"The central claim is that the model-based deactivation attack from earlier work can be replaced by a data-driven attack with no loss of effectiveness. The attacker's offline phase uses the collected input–output sequence $(u, y, \\hat{y})$ to identify a state-space model whose output reproduces the observer's predicted measurement, then fits a covering ellipse around the latent trajectory induced by the identified model and uses that ellipse as an estimated safe region; the largest residual in the data gives a lower bound on the detector threshold. Online, the attacker solves a small optimization at each step that selects a false measurement driving the latent state toward the interior of the estimated safe region. The safety transfer is justified by topological equivalence: if the identified dynamics and the true observer dynamics are related by a homeomorphism $T$, then Lemma 8 shows that latent trajectories inside $T(S)$ correspond exactly to estimated trajectories inside $S$, and in the linear case this homeomorphism is a similarity transform that standard subspace identification delivers. The experimental demonstration on an inverted pendulum with a control-barrier-function safety filter confirms the claim: the true state exits the safe set, the perceived state remains inside, and the residual never exceeds the noise level.","pith_inferences":["A consequence left implicit is that the security of a safety filter depends on the secrecy and integrity of the observer's measurement channel; protecting that channel, or deliberately making the observer's internal coordinates hard to identify, would be a direct defensive countermeasure.","A testable extension would apply the same latent-space safe-set estimation to nonlinear observers using nonlinear system identification; the paper's topological-equivalence argument is stated for general homeomorphisms, but the experiments only cover the linear-similarity case.","The observed bias in the residual after the attack could be exploited: even if the magnitude stays under the threshold, a defender who monitors the residual's direction or its mean could detect the deactivation attempt.","A quantitative next step is to measure how attack success degrades as the identification error or the distance between the covering ellipse and $T(S)$ grows, since Lemma 8 is exact only under perfect topological equivalence."],"forward_implications":["An adversary with passive access to the observer's inputs and outputs can deactivate a safety filter: model knowledge, safety-set knowledge, and observer-gain knowledge are all replaceable by data.","The attack stays stealthy against threshold-based anomaly detectors because the injected measurements are constrained to a lower bound $\\tilde{\\delta}$ of the true detection threshold.","Because the attack only assumes the filter projects onto a safe control set of the form (7), it is not limited to control-barrier-function filters.","In the linear case, any identification method that returns a model similar to the true observer dynamics is sufficient; the exact method used offline is not critical.","The inverted-pendulum experiment shows that a coverage ellipse fitted to a passively observed latent trajectory is a good enough stand-in for the true safe set to drive the state out of safety."],"supporting_citations":[{"why":"Defines the deactivation concept and the ideal model-based attack that this paper generalizes to the data-driven case; supplies the baseline policy (8) that (15) replaces.","marker":"Arnström and Teixeira (2024)"},{"why":"Provides the N4SID subspace identification method used offline to estimate the latent state-space model from the observed input–output data.","marker":"Van Overschee and De Moor (1994)"},{"why":"Supplies the forward innovation state-space form (10)–(11) and the statement that subspace identification returns a model related to a linear observer by a similarity transform.","marker":"Van Overschee and De Moor (2012)"},{"why":"Defines control-barrier-function quadratic-program safety filters of the form (7), the filter class the attack targets and instantiates in the experiment.","marker":"Ames et al. (2017)"},{"why":"Supplies the inverted pendulum model, safety constraint, and control barrier function used in the numerical experiment.","marker":"Alan et al. (2023)"}],"fun_headline_variants":["Data-driven attack on safety filters: no model needed, just data","Safety filter defeated by attacker who only watches inputs and outputs","Pendulum leaves safe set: data-only attack hides true state from filter","Stealthy attack deactivates safety filter using observed I/O alone","No system model, no observer gain: data suffices to break safety filter"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the learned surrogate model is the true observer dynamics up to a fixed change of coordinates; if the training data does not reveal that relationship, the estimated latent safe region need not correspond to any real safety region.","fun_headline_variants_meta":{"raw":{"variants":["Data-driven attack on safety filters: no model needed, just data","Safety filter defeated by attacker who only watches inputs and outputs","Pendulum leaves safe set: data-only attack hides true state from filter","Stealthy attack deactivates safety filter using observed I/O alone","No system model, no observer gain: data suffices to break safety filter"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000742,"raw_usage":{"total_tokens":3326,"prompt_tokens":978,"completion_tokens":2348,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":594,"completion_tokens_details":{"reasoning_tokens":2254}},"tokens_in":594,"tokens_out":2348,"duration_ms":17461,"temperature":1.0,"reasoning_tokens":2254,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T04:25:36.209355+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same attack on an inverted pendulum whose observer is nonlinear, or with deliberately weak excitation during data collection. If the residual exceeds the detector threshold or the true state does not leave the safe set while the estimate stays inside, the coordinate-change premise is the part that gives way.","supporting_citations":[],"review_version":1}