{"id":"4290038d-fc9f-42a6-93c6-3fd4ba1d93ea","arxiv_id":"2603.25670","paper_version":4,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Uncertainty-guided label rebalancing (uLNR) lifts UAV safety-prediction F1 to 0.806 under 46:1 imbalance by probabilistically flipping high-uncertainty safe windows to unsafe.","lead":"U-Balance rebalances rare unsafe events in cyber-physical systems by using behavioral uncertainty to flip some safe-labeled telemetry windows before training a safety predictor. On a large UAV benchmark with a 46:1 imbalance, this raises F1 by over 14 points without synthesizing new samples.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"uLNR may inject systematic label noise rather than true boundary enrichment under only moderate uncertainty–safety correlation.","rationale":"The Reader correctly isolates the same soft spot: the semantic leap from “high behavioral uncertainty” to “treat as unsafe for training” under only moderate correlation and explicit distributional overlap. No mathematical contradiction exists, the ablations are honest, and the empirical gains are large and statistically clean on the given benchmark; therefore the verdict remains CONDITIONAL rather than REJECT. The concrete test above would decide whether the gain is genuine boundary enrichment or label-noise artifact, which is exactly the condition that must hold for the strongest claim to generalize beyond this UAV simulation set. No stronger internal inconsistency was found.","tokens_in":24276,"tokens_out":554,"duration_ms":6407,"concrete_test":"On the held-out test set, isolate the subset of originally safe windows that would have been flipped under the trained uncertainty predictor + τ = 3.0; compute (a) their true safety rate under the 1.5 m rule and (b) the safety predictor’s precision/recall when those windows are forced back to safe labels at training time. If >70 % of the flipped windows are truly safe and restoring their labels drops F1 by more than ~0.10, the load-bearing premise fails.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim (F1 0.806 via uLNR) rests on the premise in §3.2 / Eqs. 7–8 that safe-labeled windows with high z-scored uncertainty scores are informative decision-boundary samples that should be stochastically flipped to unsafe. The paper itself reports only a moderate point-biserial correlation (r_pb = 0.444 ± 0.014) and explicitly notes substantial overlap: many safe windows near maneuvers remain safe under the 1.5 m rule yet receive elevated uncertainty. At the chosen τ = 3.0, 1 140 safe windows (~1.7 % of training) are flipped, raising the unsafe ratio only to 2.9 %. If those flipped windows are predominantly true-safe (as ground-truth labels assert), uLNR is injecting structured label noise rather than useful minority enrichment; the large F1 gain over TimeMoE and over all standard rebalancers would then be an artifact of training on corrupted labels that happen to improve recall on this particular simulation distribution, not a general principle for CPS safety monitoring.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes U-Balance for CPS safety monitoring under extreme class imbalance. It trains a GatedMLP uncertainty predictor on distributional kinematic features of telemetry windows, then applies uncertainty-guided label rebalancing (uLNR) that stochastically flips high-uncertainty safe labels to unsafe (Eqs. 7–8, §3.2), and finally trains a Bi-LSTM safety predictor on the rebalanced set. On the Khatiri et al. UAV window-level benchmark (≈46:1 safe:unsafe), the authors report a moderate point-biserial correlation between uncertainty and safety (r_pb = 0.444), show that uLNR substantially outperforms early/late fusion of uncertainty (Table 1), and claim F1 = 0.806 ± 0.031, a 14.3 pp gain over the strongest baseline (TimeMoE), with large Mann-Whitney effect sizes, competitive latency, and ablations of GatedMLP, distributional features, τ, and 14 rebalancers.","tokens_in":24530,"tokens_out":1124,"duration_ms":14040,"significance":"If the central premise holds—that high-uncertainty safe windows are informative boundary samples rather than pure noise—the work offers a practical, non-synthetic rebalancing route for rare-event CPS safety monitoring and a concrete alternative to fusion-based use of uncertainty. Strengths include a large multi-seed evaluation (30 runs), proper non-parametric testing with Â12, honest reporting of the precision–recall trade-off with τ (Table 4), a broad baseline suite (14 methods plus 14 rebalancers), and explicit comparison of uLNR against early/late fusion. The adaptation of LNR to time-series CPS data via a GatedMLP on distributional kinematics is a clear methodological contribution within the paper’s scope.","major_comments":[{"comment":"§3.2 / Eqs. 7–8 and §5.1: The load-bearing premise is that safe-labeled windows with high z-scored uncertainty are informative decision-boundary samples worth flipping to unsafe. The paper itself reports only moderate correlation (r_pb = 0.444 ± 0.014) and substantial overlap (many safe windows near maneuvers remain safe under the 1.5 m rule). At the chosen τ = 3.0, 1 140 training windows (~1.7 %) are flipped, raising the unsafe ratio only to 2.9 % (Table 4). Without a direct check of how many flipped windows are true-safe under the ground-truth distance rule (or of performance when those flips are replaced by random flips of the same cardinality / same z-score ranks), it remains possible that uLNR injects structured label noise that happens to improve recall on this simulation distribution. A short diagnostic—fraction of flipped windows that violate the 1.5 m rule, or a random-flip cont","section":null},{"comment":"§4.2 / External validity: All results rest on a single Surrealist-generated PX4 UAV dataset with rule-based uncertainty and safety labels. The threats section notes that uncertainty can be auto-annotated elsewhere, but no second CPS domain, real-flight telemetry, or alternative safety definition is evaluated. Given that the F1 gain is large and the mechanism depends on the uncertainty–safety relationship, at least a limited transfer experiment (different windowing, different obstacle threshold, or another public CPS safety set with analogous uncertainty rules) is needed before the method can be recommended as a general CPS rebalancing technique.","section":null}],"minor_comments":[{"comment":"§3.1 / Eq. (1): The distributional feature vector is written as d_t ∈ R^16, but the four channels × four statistics are listed without an explicit concatenation operator; a one-line clarification would help.","section":null},{"comment":"Figure 3 caption and body: “Proprocessing” is misspelled; also “U-Balanc3” appears as a stray heading fragment before §3.1.","section":null},{"comment":"Table 2: Superialist is flight-level unsupervised reconstruction; a short note that its window-level comparison is therefore imperfect would avoid over-interpreting the large ΔF1.","section":null},{"comment":"§4.5 / Implementation: Hyperparameter grids are deferred to a repository; for archival completeness, the final chosen values for all baselines should appear in an appendix or table.","section":null},{"comment":"References and ACM template still contain placeholder conference metadata (Conference acronym ’XX, Woodstock, NY, 2018 dates).","section":null}],"recommendation":"major_revision","confidential_remarks":"The empirical package is stronger than many CPS-ML submissions (30 seeds, Mann-Whitney + Â12, broad ablations). The main risk is that the impressive F1 gain is an artifact of label noise that correlates with the simulation’s failure modes. If the authors add the random-flip / true-safety diagnostic and either a second dataset or a clear scope limitation, the paper becomes a solid contribution; without that, I would not recommend acceptance at a top venue."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful takeaway is simple: on the Khatiri UAV window task (46:1), driving LNR flip rates with a learned behavioral-uncertainty score lifts F1 to 0.806, beating TimeMoE by 14.3 pp and every standard rebalancer by at least 13 pp, with clean 30-seed Mann-Whitney results. That is a real empirical result, not a restatement of SMOTE or class weighting.\n\nWhat is new is the adaptation itself. They take Hu et al.’s LNR (image classification) and replace its generic boundary heuristic with a GatedMLP that scores distributional kinematic features (mean/std/min/max of heading and position). RQ0 is done properly: moderate point-biserial correlation (0.444), then a head-to-head showing uLNR crushes early/late fusion. Ablations of the gate, the distributional features, τ, and 14 rebalancers are honest; the τ table shows the precision-recall trade-off without spin. Inference cost stays competitive. Citations are current and the related-work framing is fair.\n\nThe soft spot is exactly the one the stress-test flags, and the paper itself surfaces it. Correlation is only moderate and many high-uncertainty windows remain truly safe under the 1.5 m rule. At τ=3 they flip 1 140 safe windows (~1.7 %). If those are mostly true-safe, uLNR is structured label noise that happens to help recall on this simulation distribution. That is a genuine load-bearing assumption, not a minor caveat. Single-dataset evaluation and code promised only on acceptance keep the claim conditional. No math contradiction, no circularity between uncertainty and safety labels, no invented theory.\n\nThis is for people who actually train safety monitors on rare-event CPS telemetry and are tired of synthetic samples that look wrong. It deserves a serious referee; the experiments are careful enough that the community should see the numbers and the assumption. I would bring it to reading group and would cite the method if I were working on the same imbalance problem. Send it out.","headline":"Solid empirical methods paper: uncertainty-driven LNR beats fusion and standard rebalancers by a large margin on one hard UAV safety task; the flip premise is the real soft spot, not a fatal flaw.","tokens_in":25203,"tokens_out":526,"would_cite":true,"duration_ms":8963,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Behavioral uncertainty can rebalance rare unsafe labels for CPS safety monitors without synthesizing data.","keywords":["Cyber-Physical Systems","Safety Monitoring","Label Rebalancing","Behavioral Uncertainty","Class Imbalance","UAV Telemetry","Time-Series Classification"],"falsifier":"On the same UAV windows, if a held-out evaluation showed that the windows flipped by uLNR at the chosen threshold are predominantly true-safe (not near-miss or future-unsafe) and that removing those flips collapses F1 back to the plain or fusion baselines, the central claim would fail.","tokens_in":25106,"feed_emoji":"🛸","tokens_out":600,"duration_ms":7154,"temperature":0.7,"pith_summary":"Safety predictors for cyber-physical systems fail when unsafe events are extremely rare, and standard rebalancing either invents unrealistic time-series samples or overfits the minority class. This paper argues that behavioral uncertainty—erratic control or rapid heading changes—correlates with safety risk and can be used to fix the imbalance. U-Balance first scores each telemetry window for uncertainty from simple distributional kinematic statistics via a gated MLP, then stochastically flips the labels of unusually uncertain safe windows to unsafe. A standard safety predictor trained on the resulting set reaches 0.806 F1 on a large UAV benchmark with a 46:1 imbalance, beating strong baselines by at least 14 percentage points. The claim matters because it turns an often-ignored operational signal into a practical lever for rare-event safety monitoring without fabricating data.","feed_headline":"Uncertainty flips rare unsafe labels for UAV safety monitors","feed_subtitle":"uLNR lifts F1 by 14 points on a 46:1 UAV benchmark without inventing synthetic flights","key_machinery":"Uncertainty-guided label rebalancing (uLNR): convert GatedMLP uncertainty scores of safe windows into z-scores, map them through a shifted tanh into flip probabilities, and stochastically relabel only those windows, thereby enriching the minority class with boundary samples without synthesizing new telemetry.","core_discovery":"On a large UAV safety-monitoring benchmark with extreme class imbalance, using predicted behavioral uncertainty to probabilistically relabel high-uncertainty safe windows as unsafe (uLNR) yields a safety predictor with F1 0.806, substantially better than early or late fusion of the same uncertainty signal and better than fourteen standard rebalancing methods and fourteen model baselines.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Uncertainty-guided rebalancing lifts UAV safety F1 to 0.806","uLNR flips high-uncertainty safe windows, +14.3 F1 points","Behavioral uncertainty finds the rare unsafe CPS labels","GatedMLP uncertainty rebalances 46:1 UAV safety data","No synthetic flights: uLNR beats fusion and rebalancers"],"cache_read_input_tokens":20224,"weakest_assumption_plain":"Safe-labeled windows that show unusually high behavioral uncertainty are informative boundary cases that should be treated as unsafe for training, even though the ground-truth safety rule still marks them safe and the uncertainty-safety correlation is only moderate.","fun_headline_variants_meta":{"raw":{"variants":["Uncertainty-guided rebalancing lifts UAV safety F1 to 0.806","uLNR flips high-uncertainty safe windows, +14.3 F1 points","Behavioral uncertainty finds the rare unsafe CPS labels","GatedMLP uncertainty rebalances 46:1 UAV safety data","No synthetic flights: uLNR beats fusion and rebalancers"]},"model":"grok-4.5","effort":"low","cost_usd":0.001796,"raw_usage":{"total_tokens":940,"prompt_tokens":865,"num_sources_used":0,"completion_tokens":75,"cost_in_usd_ticks":17960000,"prompt_tokens_details":{"text_tokens":865,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":0,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":865,"tokens_out":75,"duration_ms":1367,"temperature":1.0,"reasoning_tokens":0,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-13T18:04:34.240620+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On the same UAV windows, if a held-out evaluation showed that the windows flipped by uLNR at the chosen threshold are predominantly true-safe (not near-miss or future-unsafe) and that removing those flips collapses F1 back to the plain or fusion baselines, the central claim would fail.","supporting_citations":[],"review_version":1}