{"id":"726e85fe-6b28-453e-ad7f-bc81f7740a69","arxiv_id":"2511.00934","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"pacSTL composes PAC-bounded reachable sets with interval STL to compute spec-level robustness intervals that contain an unseen trajectory's robustness with probability ≥ 1−ε.","lead":"This paper presents pacSTL, a method that turns data-learned reachable sets into interval-valued robustness bounds for signal temporal logic specifications, so a robot can check safety rules under uncertainty. It reports a statistical guarantee that an unseen trajectory's robustness falls inside the computed interval, and demonstrates the approach on simulated and physical model vessels in collision-avoidance scenarios.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 2's proof equates trajectory containment in the reachable tube with containment at all signal times, but the tube is defined only at finitely many sampled time points while I-STL temporal operators quantify over continuous intervals; the implication is not established.","rationale":"I read the paper in good faith: the core idea—combining PAC reachable-tube guarantees with interval STL—is coherent and Theorem 2 would follow if the reachable tube covered the full time domain over which STL is evaluated. The reader's weakest assumption is the external distributional premise (μ_{X0}, μ_D estimated from lab experiments). That is real but inherited by any data-driven PAC method. A more load-bearing problem is internal: the tube is finite-time-sampled, while the STL semantics in Sec. III-A are continuous-time. The proof's key set-inclusion step is therefore not justified as written. This does not refute the framework—it can be repaired by explicitly restricting pacSTL to discrete-time traces or by adding inter-sample reachability guarantees—so the appropriate verdict remains CONDITIONAL, matching the reader. I do not agree that the distribution-shift concern is the single most load-bearing issue; the sampled/continuous mismatch attacks the central theorem directly and is testable. The concrete refinement check is feasible with the existing simulator and would settle whether the gap is material for the maritime experiments.","tokens_in":24853,"tokens_out":10616,"duration_ms":126047,"concrete_test":"Take one maritime scenario from Table II (e.g. S-S head-on), keep the same reachable tubes and the same atomic optimization routines as in the paper, but evaluate the I-STL specification on the same simulated trajectories with a time step of 0.05 s instead of 0.5 s. If any trajectory's continuous-time robustness h^φ(δ) falls outside the reported [h,h]^φ interval, the implication 'δ ∈ R ⇒ h^φ(δ) ∈ [h,h]^φ' used in Theorem 2 is false under the paper's own STL semantics. If all trajectories remain inside the interval, the discrete/continuous gap is benign for these specifications and the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central guarantee (Theorem 2, Eq. (24)) depends on the step P(h^φ(δ) ∈ [h,h]^φ) ≥ P(δ ∈ R), justified by 'if a trajectory is fully contained in the reachable tube R, then its robustness ... will be within [h,h]^φ'. However, R is defined as a finite collection {R_0, ..., R_T} (Sec. III-B), and the PAC guarantee in Eq. (9) is for trajectories lying in that finite tube. Meanwhile, the I-STL semantics used for temporal operators, e.g. Eq. (4) for G, take the minimum over the continuous interval t' ∈ [t+t1, t+t2]. Nothing in the construction ensures that a trajectory contained in R at each sampled τ is inside R_t at intermediate times, so the temporal min/max over unsampled times can lie outside the interval computed from the sampled R_t. Thus δ ∈ R does not imply h^φ(δ) ∈ [h,h]^φ under the continuous-time STL semantics the paper invokes. The framework could be rescued by explicitly adopting discrete-time signals and defining the tube at every time point appearing in the specification, but this is absent. This is a direct internal gap in the proof of the central claim, distinct from the distribution-mismatch caveat and from the separate issue the Reader identified in Theorem 3.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces pacSTL, a framework that combines PAC-bounded reachable tubes obtained via scenario optimization with Interval Signal Temporal Logic (I-STL) to compute robustness intervals for STL specifications together with a probabilistic guarantee. Atomic robustness bounds are obtained by solving optimization problems over the reachable sets, and these bounds are propagated through I-STL operators. Theorem 2 claims that, with probability at least 1−β, an unseen trajectory's STL robustness lies in the computed interval with probability at least 1−ε_R. A second theorem (Theorem 3) attempts to give a tighter guarantee using characteristic time points. The method is evaluated on maritime COLREGS encounter monitoring in simulation and on physical model vessels, and the paper reports real-time feasibility and code release.","tokens_in":25199,"tokens_out":7602,"duration_ms":85697,"significance":"If the correctness issues are resolved, the paper would be a useful contribution: it decouples the expensive data-driven reachability computation from STL evaluation, supports changing atomic propositions without recalibration, and demonstrates real-time monitoring on a physical testbed. The compositional structure and the explicit probabilistic interface are appealing, and the release of code plus real-world experiments are concrete strengths. The distributional caveat—guarantees are relative to the user-chosen μ_X0 and μ_D—is inherent and acknowledged. However, as written, the central guarantee has a formal gap due to the finite sampling of the reachable tube versus the continuous-time I-STL semantics, and Theorem 3 is not correct as stated. These issues affect the main theorem and the experimental claims that rely on Theorem 3, so they must be fixed before the paper can be accepted.","major_comments":[{"comment":"The proof of Theorem 3 relies on the assertion that P(h^φ(δ_t) ∧ h^φ(δ_t) ∈ [h,h]^φ) ≥ P(h^φ(δ) ∈ [h,h]^φ), but the robustness of the full trajectory is not determined by the two characteristic time points t and t of the computed interval. The final inequality P(h^φ(δ)∈[h,h]^φ) ≥ max(P(δ_t∈R_t), P(δ_t∈R_t)) does not follow. Consequently, the improved accuracies reported in Sec. VIII-C (e.g., ε_Rt = 0.039/0.038 at t_e) are not supported by a valid theorem. Either remove Theorem 3 or provide a correct proof under explicit assumptions, e.g. that the characteristic time points are fixed for all trajectories and that the specification robustness depends only on the value at those points.","section":"Theorem 3 (Eq. (30))"},{"comment":"Algorithm 1 is presented as computing the exact lower and upper bounds for the nonlinear orientation-halfplane robustness, but no proof of exactness is given. Lemma 1 and Theorem 2 require the solutions of (21) and (22) to be exact; if Algorithm 1 only evaluates endpoint cases with a clipping rule, its correctness for a nonlinear, possibly non-monotonic function is not obvious. A correctness proof, or a reference containing one, is needed to substantiate the nonlinear atomic proposition experiments in Sec. VIII.","section":"Appendix A1 (Algorithm 1)"}],"minor_comments":[{"comment":"The formula for the binomial tail inversion is typeset confusingly ('max_e { n e: ... }'). Please clarify the notation and define the variables (e.g., the candidate violation probability and the empirical count) explicitly.","section":"Eq. (8)"},{"comment":"The notation 'i∈{1,...,K}, t∈{0,...,T}' should explicitly state that t ranges over the time grid of the reachable tube, consistent with the discrete-time interpretation that the framework apparently uses.","section":"Sec. V, proof of Theorem 2"},{"comment":"The notation δ∈R^{6×T} for trajectories conflicts with the earlier continuous-time signal notation. State explicitly that the case study uses discrete-time trajectories with step Δt.","section":"Sec. VI-A"},{"comment":"Please clarify which accuracy quantities are plotted: ε_R (tube accuracy) versus ε_Rt (time-point accuracy). The caption currently says 'minimal and maximal time-point accuracies' but the figure also shows tube accuracies.","section":"Fig. 6 caption"}],"recommendation":"major_revision","confidential_remarks":"The paper relies heavily on two same-group preprints: [16] for the core PAC bound and [66] for the maritime predicates. The refereed versions, if any, should be cited or the results made self-contained. The time-discretization gap in Theorem 2 and the incorrect Theorem 3 should be addressed before the paper is considered further; the authors may be able to fix these by explicitly adopting a discrete-time STL semantics and removing or carefully re-stating the time-point guarantee."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: the composition is real, the maritime experiments are thorough, but the central theorem is shakier than the paper lets on. The proof of Theorem 2 relies on an implication that only holds if the signal and the tube are discrete-time; the paper states continuous-time STL semantics but defines the reachable tube as a finite collection of sampled sets. The stress-test note gets this right. Until the authors either adopt discrete-time semantics or define the tube over all times, the step from δ ∈ R to h^φ(δ) ∈ [h,hbar]^φ is not justified. That's a load-bearing gap, not a cosmetic one.\n\nThe novel contribution is the transfer of PAC guarantees from reachable sets to I-STL specification-level robustness intervals, with an optimization-based atomic bound that does not require re-sampling when atomic propositions change. That is useful, and the baseline comparison to direct scenario optimization quantifies the conservativeness. The real-world maritime results are a nice sanity check, and the paper is honest about distribution mismatch and the estimated disturbance model.\n\nThe reader's take on Theorem 3 is correct: as stated, containment at the characteristic time point does not imply the interval bound for the full specification; the union-bound argument would need a sum of violation probabilities, and the characteristic time point is defined for the interval computation, not for an individual trajectory. That theorem should be corrected or removed; the paper does not need it for the main claim.\n\nThe self-citations to [16] and [66] are legitimate given that they are the direct technical foundations; the paper is transparent about that.\n\nWho should read it: anyone working on probabilistic STL monitoring or data-driven reachability for temporal logic. It is an important combination even if the current form has a fixable gap. A serious referee should see it, with the explicit expectation that the discrete-time semantics issue is resolved before acceptance. My recommendation: send to peer review, but flag the Theorem 2 gap to the authors clearly.","headline":"A useful composition of PAC reachability and interval STL, but the central theorem has an unstated discrete-time assumption that conflicts with the continuous-time semantics; fixable but essential.","tokens_in":25675,"tokens_out":3572,"would_cite":false,"duration_ms":36205,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"pacSTL gives temporal-logic safety specs a probabilistic certificate: with confidence 1−β, an unseen trajectory's robustness lands inside the computed interval with probability at least 1−ε_R.","keywords":["Signal Temporal Logic","PAC bounds","reachability analysis","scenario optimization","interval STL","runtime monitoring","maritime navigation","probabilistic guarantees"],"falsifier":"Run a new batch of real-world trajectories in an environment that includes disturbances not captured in the training distribution (e.g., waves or currents), and count the fraction of those trajectories whose measured STL robustness falls outside the pacSTL interval. If that fraction systematically exceeds the reported ε_R (beyond what the confidence β allows), the claimed probabilistic containment is falsified for that operational domain.","tokens_in":24749,"feed_emoji":"⚓","tokens_out":4872,"duration_ms":51164,"temperature":0.7,"pith_summary":"pacSTL is a framework for checking Signal Temporal Logic (STL) specifications on systems whose future behavior is uncertain. Instead of sampling many trajectories to estimate satisfaction probability, it builds a PAC-bounded reachable tube from sample data—a set that contains a new trajectory with probability at least 1−ε_R, with confidence 1−β—and then computes the narrowest interval of STL robustness values consistent with that tube. The key guarantee, Theorem 2, transfers the reachable tube's probabilistic containment to the specification level: with confidence 1−β, the probability that an unseen trajectory's STL robustness lies in the computed interval is at least 1−ε_R. A time-point refinement (Theorem 3) tightens the bound by using the typically better per-time-set accuracies at the characteristic time points that determine the robustness interval. The result is a real-time monitoring tool that avoids online re-sampling when atomic propositions or specification parameters change, demonstrated on maritime collision-avoidance rules in simulation and on physical model vessels.","feed_headline":"Safety specs get probability certificates","feed_subtitle":"Reachability plus interval STL certifies where a new trajectory's robustness falls, without resampling.","key_machinery":"The central machinery is the composition of three ingredients: (1) PAC-bounded reachable tube estimates—convex sets (ellipsoids or zonotopes) fitted to sample trajectories via scenario optimization, with the holdout method and binomial tail inversion providing the accuracy ε_R and confidence β; (2) atomic robustness bounds computed by solving convex optimization problems that minimize and maximize the atomic robustness function h over each reachable set R_t, yielding interval inclusion functions [h_t, h_t]; and (3) Interval-STL (I-STL) semantics, which propagate these intervals through Boolean and temporal operators and can track the characteristic time points (the argmin/argmax of the lower","core_discovery":"The paper claims that PAC-bounded reachable set predictions can be composed with interval-valued STL semantics to yield a robustness interval with a formal probabilistic guarantee. Concretely, for any STL specification φ, if the atomic robustness bounds are obtained by solving min/max optimization problems of the robustness function h over each time-point reachable set R_t of a PAC-bounded reachable tube, and the I-STL semantics propagate these intervals through logical and temporal operators, then the resulting interval [h, h]^φ satisfies P( P( h^φ(δ) ∈ [h,h]^φ ) ≥ 1 − ε_R ) ≥ 1 − β. That is, the probability that an unseen trajectory's robustness lies in the computed interval is at least 1−","pith_inferences":["Beyond the paper: because pacSTL inherits the reachable tube's distributional assumptions, the strongest testable extension is to stress the method under environmental conditions not represented in the lab-estimated disturbance distribution (e.g., wave tank waves or currents) and empirically measure how often the real robustness leaves the computed interval.","The characteristic-time-point tracking suggests a natural closed-loop application the paper only hints at: a controller that steers the reachable sets at those critical time points to maximize the lower robustness bound would inherit the same PAC certificate, turning pacSTL into a synthesis tool rather than only a monitor.","The comparison with direct scenario optimization on robustness values quantifies a trade-off that could be explored analytically: pacSTL is more conservative because it optimizes over the entire reachable set, not just the robustness distribution; deriving the gap between the two interval widths as a function of set volume and robustness curvature is an open problem.","Since the guarantee is one-sided (containment in the set implies containment in the robustness interval, not conversely), the interval can be tightened by shrinking the reachable tube—e.g., by conditioning the tube on the current ego trajectory—a modification that preserves the theorem's proof structure."],"forward_implications":["Runtime monitors can now certify STL specifications with a probability bound without online trajectory sampling; each evaluation reduces to a few convex optimizations, taking about 0.15–0.6 seconds on a laptop.","Changing atomic propositions or specification parameters (e.g., time horizons, rule thresholds) requires no re-calibration or re-sampling, because the reachable tube is computed once per agent and the robustness bounds are derived by optimization.","The guarantee holds for any PAC-bounded set predictor—scenario optimization is just one instance—so the framework can be combined with other data-driven reachability methods that provide (ε, β) bounds.","The time-point refinement (Theorem 3) gives a practical accuracy improvement: since per-time-set accuracies are often much better than the tube-level accuracy, the robustness interval's certificate can be stated with a tighter ε without additional data.","In real-world maritime trials, the estimated disturbance distribution (captured by the bias term b) made the sim-to-real transfer successful: robustness intervals and trigger times remained similar, and the evasive maneuvers avoided collisions."],"fun_headline_variants":["Certify STL robustness via PAC reachability","PAC-bounded STL: robustness intervals with probability","Robustness intervals get PAC probability guarantees","No sampling: PAC-bounded robustness for STL"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The PAC guarantee holds only for the specific probability distributions over initial states and disturbances chosen by the user; if the real operational environment draws scenarios from a different distribution, the stated probability bound on the robustness interval may be violated.","fun_headline_variants_meta":{"raw":{"variants":["Certify STL robustness via PAC reachability","PAC-bounded STL: robustness intervals with probability","Robustness intervals get PAC probability guarantees","No sampling: PAC-bounded robustness for STL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000708,"raw_usage":{"total_tokens":3001,"prompt_tokens":693,"completion_tokens":2308,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":437,"completion_tokens_details":{"reasoning_tokens":2248}},"tokens_in":437,"tokens_out":2308,"duration_ms":18773,"temperature":1.0,"reasoning_tokens":2248,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T00:26:38.589313+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a new batch of real-world trajectories in an environment that includes disturbances not captured in the training distribution (e.g., waves or currents), and count the fraction of those trajectories whose measured STL robustness falls outside the pacSTL interval. If that fraction systematically exceeds the reported ε_R (beyond what the confidence β allows), the claimed probabilistic containment is falsified for that operational domain.","supporting_citations":[],"review_version":1}