{"id":"1b121e2c-50d5-4ced-8446-c48889c482a2","arxiv_id":"2607.24608","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"In learned residual gyro correction, ensemble epistemic uncertainty separates nominal from structured sensor perturbations more clearly than heteroscedastic aleatoric uncertainty, while attributions stay gyro-dominated and stable.","lead":"A CNN that corrects gyroscope bias for spacecraft attitude estimation was tested with uncertainty and attribution tools under synthetic sensor noise. Epistemic uncertainty from model ensembles separates normal from degraded conditions more cleanly than data noise estimates, which motivates onboard fault monitoring.","discovery_kind":"new_application","skeptic_critique":{"model":"moonshotai/kimi-k3","headline":"Internal tension between the attribution result and the fault-detection claim: the dropout regime perturbs only star trackers, which the paper's own IG analysis shows the network essentially ignores.","rationale":"The reader's weakest assumption targeted external validity: synthetic perturbations, open-loop evaluation, proprietary simulator, and three hand-chosen intensities may not represent real sensor degradation or closed-loop behavior. That is a fair and standard concern for this genre, and the authors concede much of it in §3.3 and the conclusion (\"do not exhaust the space of possible distribution shifts... characterize uncertainty behavior under controlled distribution shifts rather than demonstrating general OOD detection capability\"). My concern is different and, I think, more load-bearing because it is internal: the paper's own attribution result (negligible star tracker contribution) predicts failure of its monitoring claim for the one perturbation family that affects only star trackers (dropout), which is also the fault class the introduction motivates the work with. The reader did not flag this cross-claim tension, hence \"partial\" agreement — I share the reader's scope concerns but identify a different weakest point. I do not change the verdict: CONDITIONAL remains right. The paper is a careful, descriptive empirical study with honest hedging, reasonable metrics (coverage, z-scores, entropy), and no circularity problem; the concern does not rise to reject-level because the aggregated separation results may well be driven by the gyro-affecting perturbations, which are legitimate. But the condition attached to acceptance should specifically include the per-perturbation-type breakdown (and the star tracker ablation the authors themselves propose in future work), because without it the FDIR-motivating claim is unsupported for the fault type that matters most. The §3.3 \"retrain the ensembles with these perturbations\" wording is a secondary ambiguity worth one clarifying sentence, since retraining on perturbed data would change the meaning of \"OOD\" entirely.","tokens_in":13276,"tokens_out":2094,"duration_ms":80356,"concrete_test":"Report the epistemic (and aleatoric) uncertainty distributions for the dropout-only regime separately, not aggregated: compute AUROC between ID and each of OOD-L/M/H dropout on u_epi(t), and repeat with star tracker inputs ablated (zeroed/held constant at inference). If AUROC for dropout-only regimes is near chance (~0.5) and the ablation does not change it, the fault-detection claim must be scoped to gyro-affecting perturbations and the attribution story is confirmed; if AUROC is high, the \"negligible star tracker contribution\" finding needs reinterpretation. Also clarify in one sentence whether ensembles were retrained on perturbed data or only evaluated on it (§3.3).","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim is that ensemble epistemic variance is a clean, separable ID/OOD signal suitable for downstream monitoring and fault detection. But the paper's two result strands pull against each other in a specific place. §3.3 defines three perturbation families: (a) AR(1) noise and vibration applied to gyro AND star trackers, (b) bias step/drift applied only to the gyro, and (c) dropout (hold-last-value) applied ONLY to the star trackers. Meanwhile, §4.5 reports that IG channel attribution for both µ and log σ² is dominated by gyro inputs — gyro-y especially — with star tracker channels contributing \"negligibly\" across all regimes. If the model truly places near-zero functional weight on star tracker channels, then a perturbation that only corrupts star tracker inputs should produce little change in µ across ensemble members, hence little epistemic signal. Yet the abstract/conclusion claim epistemic uncertainty separates nominal from perturbed regimes generally, motivating use in FDIR — and the motivating operational fault in §4.1 is precisely star tracker outage (\"even a 10 min star tracker outage causes ~1° attitude error\"). Either (i) epistemic separation under the dropout regime is weak, in which case the monitoring claim fails for the most operationally relevant fault type and the negligible-attribution finding predicts it; or (ii) separation is strong under dropout, in which case the \"star trackers contribute negligibly\" attribution claim is misleading (the model does depend on them, or ensemble variance is driven by something other than input sensitivity). The paper never reports per-perturbation-type separation — Figures 4 and 6 appear to aggregate regimes — so this cannot be resolved from the text. This is not a consensus-disagreement issue; it is an internal-consistency gap at the heart of the headline monitoring claim. Note also §3.3's ambiguous sentence \"we retrain the ensembles with these perturbations while maintaining the ground truth bias,\";—\"","agreement_with_reader":"partial"},"referee_report":{"model":"moonshotai/kimi-k3","summary":"The manuscript presents an empirical study of uncertainty and attribution behavior in a hybrid state-estimation pipeline. A 1-D CNN with a heteroscedastic Gaussian head (mean residual correction μ and log-variance log σ², trained with an NLL+α·MSE loss) predicts residual gyroscope bias corrections from gyro and star-tracker rate windows; the correction is subtracted from gyro measurements upstream of a flight-representative Gyro-Stellar Estimator (GSE), which is left unmodified. Training uses nominal data from a proprietary high-fidelity mission simulator (100 logs, ~3.24 h each); evaluation is open-loop at the residual level under synthetic structured perturbations (AR(1) noise, multi-tone vibration, gyro bias step/drift, star-tracker hold-last-value dropout) at three hand-set intensities (OOD-L/M/H). Epistemic uncertainty is the variance of μ across an ensemble of M=5 independently trained models. Reported findings: (i) the CNN reduces mean residual magnitude by ~90% vs. raw and ~82% vs. a mean-bias-removal baseline (Table 2); (ii) aleatoric uncertainty rises with perturbation intensity but distributions overlap and calibration shifts from under-dispersion (ID) to over-dispersion (OOD); (iii) epistemic variance separates ID from OOD regimes more cleanly; (iv) Integrated Gradients attribution for both μ and log σ² is dominated by gyroscope channels (gyro-y especially), with negligible star-tracker contribution, and temporal/channel attribution structure remains stable across","tokens_in":13598,"tokens_out":3652,"duration_ms":117146,"significance":"If the results hold, the paper offers a useful, honest characterization of how heteroscedastic aleatoric and ensemble-epistemic uncertainty behave under temporally structured sensor degradation in a hybrid deep-filter architecture — a regime (correlated, physically motivated perturbations rather than i.i.d. noise) that prior UQ work on learned estimation components has largely skipped. The joint attribution of both the mean correction μ and the log-variance head log σ², aggregated per axis and per regime, is a genuinely uncommon analysis and the attribution-stability finding (uncertainty magnitude rises without temporal or channel redistribution) is an interpretable, falsifiable observation. The correction gains are concretely quantified (Table 2: ~90% mean residual reduction vs. raw, ~82% vs. naive baseline, with consistent tail improvements). The manuscript is also commendably careful about its own limits: it states the perturbations do not exhaust operational shifts and that results characterize controlled distribution shift rather than general OOD detection. It does not, however, ship code or data (the simulator is proprietary), and the downstream FDIR claim is motivational, so","major_comments":[{"comment":"There is a direct contradiction about whether the perturbation data enters training. The second paragraph of §3.3 states 'The perturbation datasets are not used during training and are therefore out-of-distribution with respect to the training data,' but the final paragraph of §3.3 states 'we retrain the ensembles with these perturbations while maintaining the ground truth bias unchanged. This ensures that only the uncertainty awareness is tested to capture epistemic uncertainty through prediction disagreement.' If ensemble members are actually retrained on perturbed inputs, the OOD interpretation of Fig. 6 collapses, because epistemic separation rests on the perturbations being unseen. If (as I suspect) the intended meaning is that perturbed inputs are only passed through the already-trained ensembles at evaluation time, the sentence must be corrected, because as written it describes a","section":"§3.3 (Perturbation Modeling), final paragraph"},{"comment":"The manuscript's two result strands pull against each other at the point that matters most for the stated FDIR motivation. §3.3 defines the dropout perturbation as applying only to star trackers (hold-last-value); §4.5/Fig. 7 report that star tracker channels contribute 'negligibly' to both μ and log σ² across all regimes; and §4.1 motivates the whole study with a star tracker outage ('even a 10 min star tracker outage causes a ~1° attitude error'). If the network places near-zero functional weight on star tracker channels, a star-tracker-only corruption should produce little ensemble disagreement, i.e., little epistemic signal — so the clean separation in Fig. 6 may be driven entirely by the gyro-affecting perturbation families (AR noise, vibration, bias step/drift). Fig. 6 appears to pool all perturbation families into intensity levels OOD-L/M/H, so this cannot be checked from what is","section":"§4.4 / §4.5 / §3.3 (dropout), with §4.1 motivation"},{"comment":"The central comparative claim — that epistemic uncertainty separates ID from OOD regimes with 'very small overlap' while aleatoric distributions overlap — is supported only visually (Fig. 4 vs. Fig. 6). Given that the ensemble has only M=5 members (so per-sample epistemic variance estimates are themselves noisy), a quantitative separation metric is needed: e.g., AUROC or a thresholded detection rate of a simple epistemic-variance classifier between ID and each OOD level, ideally with confidence intervals across evaluation logs. This is a modest addition but it is load-bearing for the paper's strongest claim and for any downstream threshold-based monitoring use.","section":"§4.4 (Epistemic Uncertainty Behavior), Fig. 6"}],"minor_comments":[{"comment":"The IG baseline is never specified (zeros, feature means, or other), and no completeness check (sum of attributions vs. output difference from baseline) is reported. Since the channel-dominance conclusions in Fig. 7 depend on attribution magnitudes, please state the baseline and ideally report the completeness error.","section":"§3.4 (Attribution Analysis)"},{"comment":"Wording: 'the residuals are spread out over a wide range, which means they are under-dispersed' is confusing — wide z-scores indicate that the predicted σ is too small, i.e., the predictive distribution is under-dispersed (equivalently the standardized residuals are over-dispersed). Please rephrase for precision.","section":"§4.3, z-score discussion"},{"comment":"Eq. (2) defines epistemic uncertainty using only the spread of ensemble means, discarding the members' predicted variances. This is a reasonable choice given the stated goal of analyzing components separately, but please note explicitly that this is not the full predictive variance of a deep ensemble (Lakshminarayanan et al.) and that conclusions about 'epistemic' behavior are specific to this mean-variance definition.","section":"§3.2, Eq. (2)"},{"comment":"The dropout probability range is very narrow (6×10⁻⁴–8×10⁻⁴) compared to the wide ranges for the other parameters; please justify why probability is held nearly fixed while duration varies, or widen the sweep.","section":"Table 1"},{"comment":"Typo: 'Since, the onboard time is set at 8 Hz' — spurious comma.","section":"§4.1"},{"comment":"Typos: 'European Aerospace Agency' should be 'European Space Agency'; 'lead by Airbus' should be 'led by Airbus'.","section":"Acknowledgments"},{"comment":"Duplicated URL string in the Kechris et al. entry; also Ref. [5] is a project webpage (ESA Nebula) rather than a publication — consider citing a peer-reviewed source for SHAP-based XAI in spacecraft GNC if one exists.","section":"References [9], [5]"},{"comment":"Phrases such as 'epistemic uncertainty gives a clear signal that gets clearer as the distributional shift happens, showing that the models disagree more' are colloquial and slightly circular; suggest tightening to a quantitative statement once a separation metric is added.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The evaluation relies on a proprietary Airbus high-fidelity simulator and no code or data release is indicated, so independent reproduction will not be possible; the editor may wish to weigh this against the venue's reproducibility expectations. The work is clearly situated within the ESA GSTP AI4AOCS activity and the related-work positioning appears fair. The contribution is a careful empirical behavior study rather than a new method; if the venue expects methodological novelty, that is a fit question, but on scientific merit the paper is sound once the dropout/attribution tension and the training-data contradiction are resolved."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful core here is empirical and specific: a residual 1-D CNN with heteroscedastic heads plus a 5-member ensemble, trained only on nominal high-fidelity sim data, then probed under structured synthetic OOD (AR noise, vibration, gyro bias/drift, ST hold-last-value). Correction gains are real and quantified (~90% mean residual cut vs raw, ~82% vs naive mean removal). Aleatoric rises with intensity but overlaps and flips from under- to over-dispersion; epistemic variance separates ID from OOD more cleanly. IG on both µ and log σ² is stably gyro-dominated (especially gyro-y), with temporal peaks that barely move under perturbation. That joint picture is new for this hybrid GSE setting and worth having.\n\nWhat they do well: honest calibration diagnostics (coverage, entropy, z-scores), open-loop residual isolation so you can actually see the learned piece, and the explicit note that attributions are model reliance not physical causality. Citations are appropriate (Kendall/Gal, Lakshminarayanan, Sundararajan, KalmanNet-style work). Math is standard NLL+MSE and ensemble variance; no circularity.\n\nSoft spots, in proportion. The monitoring/FDIR motivation is ahead of the data. Figures 4 and 6 appear to pool perturbation families, so you cannot tell whether separation holds under ST-only dropout—the fault they themselves flag as operationally costly. Their own IG says ST channels contribute negligibly; if that is true, pure ST dropout should not move ensemble µ much. Either the dropout signal is weak (undercutting the claim) or the “negligible” attribution is incomplete. They never break it out. Open-loop residual evaluation and fully synthetic OOD on a proprietary simulator further limit how far the FDIR sentence can travel. Minor: α, window/stride, and intensity schedules are free parameters; no public code/data; no ST ablation despite the conclusion asking for one.\n\nWho it is for: people building uncertainty-aware hybrid GNC estimators who want concrete ID/OOD behavior numbers and attribution sanity checks, not a new UQ method. I would send it to referees. Tighten the per-regime tables, resolve the dropout/attribution tension, and dial back the FDIR claim to what the plots actually show. Engage if you work in this lane; skip if you need closed-loop or flight logs.","headline":"Solid applied empirical study of residual-gyro CNN uncertainty and IG attribution; the FDIR claim outruns the evidence because OOD results are aggregated and star-tracker dropout sits in tension with near-zero ST attribution.","tokens_in":14559,"tokens_out":591,"would_cite":false,"duration_ms":11884,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Epistemic ensemble disagreement separates nominal from perturbed gyro regimes more cleanly than learned aleatoric uncertainty in residual bias correction.","keywords":["hybrid state estimation","explainable AI","uncertainty quantification","gyro bias correction","deep ensembles","integrated gradients","aleatoric uncertainty","epistemic uncertainty"],"falsifier":"On real flight telemetry or closed-loop Gyro-Stellar Estimator runs with genuine sensor faults, if ensemble-variance distributions of nominal and degraded segments overlapped as heavily as the paper’s aleatoric distributions do, the claim that epistemic uncertainty is the better discriminator would be falsified.","tokens_in":14296,"feed_emoji":"🛰️","tokens_out":985,"duration_ms":33556,"temperature":0.7,"pith_summary":"Spacecraft attitude filters still leave residual gyroscope bias that a small external neural network can learn to cancel from gyro and star-tracker rates. This paper asks what that network’s uncertainties and explanations do when the sensors are hit by realistic, temporally structured degradation never seen in training. A 1-D CNN predicts both a mean rate correction and an input-dependent noise level; five independently trained copies supply an ensemble-disagreement (epistemic) signal. On simulator logs with AR noise, vibration, bias steps/drifts and star-tracker freezes of rising strength, aleatoric uncertainty grows but overlaps across regimes and is inconsistently calibrated, while ensemble variance stratifies cleanly and tightens the separation as the shift worsens. Gradient attributions show both the correction and the uncertainty are driven mainly by the gyroscope channels—especially the y-axis—and that this reliance pattern barely moves from clean to degraded data. The result matters because hybrid deep-filter pipelines can keep the classical estimator intact and still obtain a usable out-of-distribution monitor plus an account of which sensors are driving the elevated uncertainty.","feed_headline":"Ensemble disagreement flags bad gyros better than noise estimates","feed_subtitle":"Learned residual correctors keep relying on the same gyro axes while uncertainty rises under structured sensor faults","key_machinery":"A residual 1-D CNN that jointly outputs mean correction μ and log-variance log σ² (Gaussian NLL plus weighted MSE), a five-member independent ensemble whose prediction variance is treated as epistemic uncertainty, and Integrated Gradients applied separately to μ and log σ² under AR(1), multi-tone vibration, bias-step/drift and hold-last-value dropout perturbations of three intensities.","core_discovery":"Under structured sensor perturbations unseen in training, epistemic uncertainty from deep-ensemble disagreement gives a clearer, more separable signal of distributional shift than the network’s heteroscedastic aleatoric uncertainty for residual gyro-rate correction, while Integrated Gradients attributions for both the mean correction and the log-variance stay dominated by gyroscope inputs (especially gyro-y) and remain largely stable from nominal to perturbed regimes.","pith_inferences":["The same ensemble-disagreement monitor could be attached to other residual learners (star-tracker or actuator correctors) without touching the classical filter internals.","A controlled ablation that drops star-tracker channels at inference would test whether their near-zero attribution means they act only as a training-time reference rather than a true input dependence.","If the mid-window temporal peaks survive across different window lengths, the network may be learning a discrete-time bias estimator rather than a full dynamical model, which would simplify onboard assurance arguments.","Pairing the epistemic flag with a coverage check on standardized residuals could give operators a two-tier alert: shift detected versus confidence miscalibrated."],"forward_implications":["Downstream monitors can treat rising ensemble disagreement as the primary flag for non-nominal sensor conditions and treat aleatoric rise as a secondary intensity cue.","Stable gyro-dominated attribution maps can be used to report which input channels are responsible when uncertainty climbs.","Residual correctors can remain external to the classical estimator without losing explainability or an OOD signal.","Aleatoric heads trained only on nominal data should not be assumed calibrated under structured degradation; regime-aware checks are required.","Axis-specific mid-window temporal peaks that persist under perturbation indicate the network reuses fixed temporal structure rather than redistributing attention."],"fun_headline_variants":["Ensemble disagreement beats aleatoric noise for spotting gyro distribution shift","Epistemic uncertainty separates perturbed gyros better than learned noise estimates","Deep ensembles flag unseen gyro faults clearer than heteroscedastic uncertainty","Gyro-y still dominates attributions as epistemic signal tracks sensor perturbations","Aleatoric overlap stays high while ensemble disagreement rises under structured faults"],"cache_read_input_tokens":128,"weakest_assumption_plain":"The synthetic structured perturbations and open-loop residual tests on simulator logs are representative enough of real spacecraft sensor degradation and closed-loop estimator behavior to support claims about monitoring and fault detection.","fun_headline_variants_meta":{"raw":{"variants":["Ensemble disagreement beats aleatoric noise for spotting gyro distribution shift","Epistemic uncertainty separates perturbed gyros better than learned noise estimates","Deep ensembles flag unseen gyro faults clearer than heteroscedastic uncertainty","Gyro-y still dominates attributions as epistemic signal tracks sensor perturbations","Aleatoric overlap stays high while ensemble disagreement rises under structured faults"]},"model":"grok-4.5","effort":"low","cost_usd":0.00498,"raw_usage":{"total_tokens":1417,"prompt_tokens":826,"num_sources_used":0,"completion_tokens":90,"cost_in_usd_ticks":49804000,"prompt_tokens_details":{"text_tokens":826,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":501,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":826,"tokens_out":90,"duration_ms":8608,"temperature":1.0,"reasoning_tokens":501,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-31T10:43:51.194823+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On real flight telemetry or closed-loop Gyro-Stellar Estimator runs with genuine sensor faults, if ensemble-variance distributions of nominal and degraded segments overlapped as heavily as the paper’s aleatoric distributions do, the claim that epistemic uncertainty is the better discriminator would be falsified.","supporting_citations":[],"review_version":1}