Automating the Deep Space Network Data Systems; A Case Study in Adaptive Anomaly Detection through Agentic AI

2); (2) Pasadena City College; (3) Jet Propulsion Laboratory California Institute of Technology); Evan J. Chou (1; Harvey M. Soldan (3) ((1) University of California San Diego; Lisa S. Locke (3)

REVIEW 4 major objections 6 minor 11 references

Automating the Deep Space Network Data Systems; A Case Study in Adaptive Anomaly Detection through Agentic AI

T0 review · 4 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A prototype agentic-AI system detects and explains Deep Space Network anomalies

desk verdict A well-written case study of an agentic anomaly-detection prototype for the Deep Space Network, with one genuinely useful transmitter email parser, but the central detection claim is unsupported by any quantitative evaluation. read the letter →

arxiv 2508.21111 v1 pith:YYXI2AUG submitted 2025-08-28 cs.LG cs.AI

Evan J. Chou (1 , 2) , Lisa S. Locke (3) , Harvey M. Soldan (3) ((1) University of California San Diego , (2) Pasadena City College , (3) Jet Propulsion Laboratory California Institute of Technology) This is my paper

classification cs.LGcs.AI

keywords DeepSpaceNetworkanomalydetectionagenticAImultivariatetimeseriesreconstructionmodelsQ-learningLLMfine-tuningdiscrepancyreports

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper reports on a prototype anomaly-detection assistant for the Deep Space Network. Its central claim is that an agentic AI workflow can take multivariate time-series monitor data from DSN antennas and transmitters, reconstruct the expected signal with deep learning models, flag deviations as anomalies, verify them with a Q-learning agent, and auto-generate discrepancy reports through a fine-tuned language model. The authors argue that detecting and pinpointing true anomalies this way would help DSN engineers catch equipment degradation before it disrupts spacecraft communications. The results are explicitly preliminary: the models flagged many false positives, and the threshold mapping between reconstruction error and real anomalies is the fragile link.

What carries the argument

The load-bearing machinery is reconstruction-based anomaly detection: each deep model learns to reconstruct the normal multivariate time series, and the deviation between reconstructed and observed values is the anomaly signal. Around that sits the agentic workflow, which chains function calls for anomaly scoring, explanation, planning, and human feedback, and uses the fine-tuned language model as the reasoning agent and report generator. The unvalidated link that carries the whole chain is the hand-set threshold on reconstruction error; when the threshold moves, the false-positive rate moves with it.

What would settle it

Take a date range with known operator-confirmed discrepancy reports and run the trained pipeline at the chosen threshold; compute precision and recall at the flagged timestamps, and compare flagged events to actual maintenance records. If most flags do not coincide with confirmed equipment events—or if no threshold value yields both high precision and recall—the central claim that the system pinpoints true anomalies is not supported.

Watch

Extended reading notes

Core claim

The central claim is that a single orchestrated agent can close the DSN data-to-report loop: LSTM, GAN, and Time-Series Transformer models reconstruct normal antenna behavior from SSNR and PCNO features; reconstruction errors above a chosen threshold become anomaly candidates; a Q-learning verifier scores their severity from weather and feature context; and a low-rank-adaptation fine-tuned language model converts the survivors into human-readable discrepancy reports with corrective actions. A separate pipeline extracts and parses transmitter data from email into tabular form, connecting transmitter anomalies to antenna anomalies. The paper states the system is 'capable of detecting and pinpo

Load-bearing premise

The whole chain rests on the assumption that a hand-set threshold on reconstruction error marks real equipment faults; the paper reports that many flagged points were false positives and that lowering the threshold adds more, so if that error-to-anomaly mapping fails, the RL verifier and LLM reports inherit garbage.

Editorial extensions

If this is right

Engineers could hand the assistant a date range and receive a list of candidate anomalies with severity levels and suggested corrective actions, without manually inspecting raw telemetry.
The transmitter email-to-tabular pipeline gives antenna anomalies a second data source to cross-check, which should reduce ambiguous flags.
Human feedback on Q-learning classifications allows the system to improve without retraining the deep reconstruction models.
The modular workflow can swap in better-performing reconstruction models or add real-time streaming sources later.
If the threshold mapping holds, degradation trends that are invisible to human visual inspection become machine-detectable over long time windows.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The repeated false positives and end-of-sequence anomaly clustering suggest the current threshold is measuring model extrapolation error as much as equipment degradation; a walk-forward validation design with known maintenance events would separate the two.
The email-parsing pipeline could generalize to any operational system that reports status through human-readable email, not just DSN transmitters.
A deployable version would need a calibration step on labeled discrepancies before its RL verifier can be trusted; otherwise the human-feedback loop only learns whatever the threshold already decides.
The modular design implies the language model could eventually be replaced or upgraded without retraining the statistical detectors, so the agent's reasoning quality and the detector's accuracy can improve independently.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The reader's take is right: the central claim is unsupported. Section 4.4 admits many flagged points were false positives, Section 10 calls the results preliminary and not robust, and there are no precision/recall numbers, no ROC curves, no baseline comparisons anywhere. As a scientific claim—"capable of detecting and pinpointing true anomalies"—this paper does not hold up.

What is genuinely new and good: the transmitter email parsing pipeline. The authors built a working program to extract and parse DSN transmitter data from Outlook emails, separating JPL and CEC transmitters, and connecting that data to the antenna-based models. That fills a real operational gap, and it is a concrete, reproducible engineering contribution. The integration of LSTM, GAN, and TST reconstruction with Q-learning verification and a locally hosted LoRA-tuned Mistral-7B inside a LangGraph workflow is real work, clearly described. The paper is also refreshingly honest in its limitations section: it acknowledges no real-time capability, that the RL component may not be necessary, and that extensive retraining is needed. The self-citation to Bhargava 2023 is appropriate as prior pipeline work.

The load-bearing flaw is that the anomaly detection claim is asserted, not measured. The reconstruction-error-to-threshold mapping is hand-set, and the paper itself says more false positives appear when the threshold is lowered. Without comparing against ground-truth discrepancy reports or labeled anomalies, the downstream RL verification and LLM report generation are built on unvalidated flags. Hyperparameters are described qualitatively, error bars are absent, and no baseline (even a simple statistical method) is provided. This is not a minor omission; it is the core of the paper's stated purpose.

Who is this for? JPL engineers or operational teams wanting a template for integrating agentic AI with telemetry pipelines, and educators looking for a candid case study of a seventeen-week internship project. A reader expecting a validated detector will be disappointed.

I would not accept this as an archival research paper, but I would send it to a workshop or a systems-oriented venue with a request for major revision: reframe it as a system description and lessons learned, remove or substantially soften the detection claim, and add at least a small quantitative evaluation on labeled data. The parser and integration are real, and reviewer feedback could help the authors present those contributions honestly. It deserves referee time rather than a silent desk reject.

Referee Report

4 major / 6 minor

Summary. The paper describes an agentic AI system for anomaly detection in Deep Space Network (DSN) antenna and transmitter data. It combines three deep learning reconstruction models (LSTM, GAN, Time-Series Transformer) with threshold-based statistical anomaly flagging, a Q-learning verification layer, and a LoRA fine-tuned Mistral-7B LLM for discrepancy report generation, all orchestrated through a LangGraph workflow. It also presents a data pipeline for extracting and parsing DSN transmitter data from Outlook emails and tar.gz files. The central claim, stated in the abstract and Section 10, is that the resulting system is 'capable of detecting and pinpointing true anomalies.'

Significance. If the central claim were supported, the system could be a valuable assistant for JPL engineers, flagging degraded DSN equipment and auto-generating discrepancy reports. The engineering contributions are real: the transmitter email extraction/parsing pipeline addresses a concrete operational gap, and the local hosting of the fine-tuned LLM shows appropriate attention to data privacy. The overall architecture is coherent and thoughtfully assembled from existing ML components. However, the detection claim is not substantiated by any quantitative evaluation. The paper's own Section 4.4 reports many false positives and Section 10 admits that all statistical results were preliminary and models were not robust. Without precision/recall, ROC curves, or comparison against ground-truth discrepancy reports, the paper is an architecture and pipeline description rather than a validated anomaly detection study.

major comments (4)

[Section 4.4 and Section 10] No quantitative evaluation is reported for any component of the anomaly detection pipeline. Section 4.4 states that 'many timestamps and points outputted by the models were verified as false positives' and that larger numbers of false positives appear when the threshold is lowered. Section 10 concedes that 'all our statistical results were preliminary and the models were not robust.' No precision, recall, F1, ROC, or comparison to the available discrepancy-report ground truth is given for the LSTM, GAN, or TST models, nor for the Q-learning verifier or the LLM. Consequently, the abstract's claim that the system is 'capable of detecting and pinpointing true anomalies' is unsupported by the evidence in the manuscript.
[Section 4.4, Figures 7, 9, 11] The mapping from reconstruction error spikes to real equipment anomalies is the load-bearing step of the entire system, but it is validated only qualitatively. The anomaly threshold is hand-set; Section 4.4 notes that lowering the threshold increases false positives, yet no procedure for selecting the threshold, no sensitivity analysis, and no error bars are provided. Because the Q-learning verifier and the LLM report generator consume these flagged anomalies as inputs, any error in this error-to-anomaly mapping propagates through the entire agentic workflow. The threshold-and-error mapping must be validated against labeled anomalies before the downstream claims can be assessed.
[Section 5.1 and Section 5.3] The Q-learning verification subsystem is described but not evaluated. Section 5.1.1 gives a high-level description of the algorithm and reward structure, but no reward-function equations, no Q-table convergence behavior, no classification results, and no human-feedback experiments are reported. Section 5.3 itself states that 'the entire concept of reinforcement learning and how it applies to adaptive anomaly detection is still an uncertainty that came from very preliminary results.' Thus the claim of an 'adaptive anomaly classification' layer is not demonstrated.
[Section 6.2, Section 6.3, and Section 3.2.3] The agentic AI workflow is described only at the architectural level. No examples of generated discrepancy reports, no human evaluation of the LLM outputs, and no comparison to human-written reports are provided. Section 6.3 states that 'our current AI agent system and workflow is far from being a fully agentic system,' and Section 3.2.3 says the current transmitter program 'does not allow for the continuous input of data that is crucial for detecting anomalies in real-time.' These statements directly temper the abstract's suggestion of a deployed real-time agentic assistant. The 'complex reasoning' claim therefore remains an aspiration, not a demonstrated result.

minor comments (6)

[Section 3.1, Table 1] Please provide the exact number of rows, number of features, and the date range for the combined MIA/SNT dataset. 'Hundreds of thousands of data and more than a hundred features' is too vague for reproducibility.
[Section 4.1-4.3] Architectural hyperparameters (hidden sizes, number of layers, sequence lengths, training epochs) are given only for the LSTM. Please summarize all three models' settings in a table to enable replication.
[Figures 3 and 4] Axis labels are missing or unclear. Figure 3 claims a 'logarithmic graph of all the principal components and their cumulative explained variance' but the axes are not labeled; Figure 4's 'outputted results from PCA' similarly lacks axis descriptions.
[Section 5.1.1] The reward function is described in words only. A concrete equation for R(s,a) and the Bellman update would be more informative than the current text.
[Throughout] There are numerous typos and citation inconsistencies (e.g., Section 3.5.1 'diagonostics', Section 7.1 references are non-standard and some lack DOIs/arXiv IDs). A careful copyedit is needed.
[Section 6.1] The fine-tuning dataset size, number of training examples, and validation metrics for the LoRA fine-tuned Mistral-7B are not reported. Even a simple perplexity or human-rated correctness measure would strengthen the LLM section.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: detection is an explicit reconstruction-error-plus-threshold pipeline; the only self-citation (Bhargava 2023) is a disclosed data-import tool and does not load-bear the central claim.

full rationale

The manuscript does not present a mathematical derivation in which a target quantity is defined in terms of the predicted quantity or in which a fitted parameter is renamed a prediction. The anomaly detection step (Section 4) is an explicit reconstruction-error plus user-set threshold procedure: LSTM, GAN, and TST are trained to reconstruct MIA/SNT sequences and flagged errors are compared to a threshold; no claim is made that a threshold fitted on one subset predicts another. Section 4.4 explicitly reports false positives and sensitivity to threshold choice, and Section 10 concedes preliminary, non-robust results; these are evidence against the central claim but are not circularity. The only self-citation (Bhargava et al. 2023) is cited as a prior data-import utility for DSNPAR queries (Section 3.1); it is disclosed and is not load-bearing for the anomaly-detection or agentic-AI claim. External standards (PyTorch, Hugging Face, LoRA, LangGraph, Ollama, Mistral-7B) and a provided JPL discrepancy-report dataset are used as independent components. No cited uniqueness theorem or prior-work ansatz is invoked to force the system design. Hence no circular step meeting the quote-and-reduction standard is identifiable.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The system relies on standard ML training (backpropagation, Adam, Bellman updates) and on domain assumptions that reconstruction error spikes map to equipment anomalies and that SSNR and PCNO are the key features. No new physical or mathematical entities are postulated. The main unexamined assumption is that the agentic workflow and hand-set thresholds improve detection; no comparison or validation is provided.

free parameters (4)

anomaly_threshold = not reported
Central detection decisions depend on a threshold on reconstruction error; the threshold value and how it was chosen are not reported (Section 4.4).
isolation_forest_contamination = not reported
The fraction of outliers is a user-set input to Isolation Forest (Section 3.4.2), affecting how many candidates the deep models later see.
q_learning_rewards = not reported
Point-based rewards for low, medium, and high severity and for correct or incorrect actions are hand-assigned (Section 5.1.1), shaping the agent's verification behavior.
lstm_hyperparameters = hidden=64, lr=1e-4, wd=1e-5, dropout=0.2
Hidden size 64, two layers, learning rate 1e-4, weight decay 1e-5, dropout 0.2 (Section 4.1). These are standard hyperparameters but are not justified by any search or analysis.

assumptions (6)

standard math Q-learning Bellman update is a valid way to learn an optimal policy
Invoked in Section 5.1.1 to update the action-value function.
standard math Backpropagation with Adam converges to useful reconstruction models
Assumed throughout Section 4 for training LSTM, GAN, and TST models.
domain assumption Peaks in reconstruction error indicate equipment anomalies
The entire detection approach rests on this mapping, stated in Sections 4.1 to 4.4, but Section 4.4 reports many false positives.
domain assumption SSNR and PCNO best capture anomalies
Chosen in Section 3.3 based on visual inspection and PCA, then used as the features for anomaly detection without independent validation.
ad hoc to paper An agentic LangGraph workflow improves detection over component models
Implicit in Section 6.2; no comparison is provided between the full workflow and the individual models.
domain assumption Hand-set anomaly thresholds can separate true from false anomalies
Assumed in Section 4.4, where the authors also note that lowering the threshold increases false positives.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Automating the Deep Space Network Data Systems; A Case Study in Adaptive Anomaly Detection through Agentic AI." pith.science (2026). https://pith.science/paper/YYXI2AUG

@misc{pith2026250821111,
  author       = {Pith},
  title        = {Pith review of: Automating the Deep Space Network Data Systems; A Case Study in Adaptive Anomaly Detection through Agentic AI},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YYXI2AUG}},
  note         = {Machine review of arXiv:2508.21111}
}

read the original abstract

The Deep Space Network (DSN) is NASA's largest network of antenna facilities that generate a large volume of multivariate time-series data. These facilities contain DSN antennas and transmitters that undergo degradation over long periods of time, which may cause costly disruptions to the data flow and threaten the earth-connection of dozens of spacecraft that rely on the Deep Space Network for their lifeline. The purpose of this study was to experiment with different methods that would be able to assist JPL engineers with directly pinpointing anomalies and equipment degradation through collected data, and continue conducting maintenance and operations of the DSN for future space missions around our universe. As such, we have researched various machine learning techniques that can fully reconstruct data through predictive analysis, and determine anomalous data entries within real-time datasets through statistical computations and thresholds. On top of the fully trained and tested machine learning models, we have also integrated the use of a reinforcement learning subsystem that classifies identified anomalies based on severity level and a Large Language Model that labels an explanation for each anomalous data entry, all of which can be improved and fine-tuned over time through human feedback/input. Specifically, for the DSN transmitters, we have also implemented a full data pipeline system that connects the data extraction, parsing, and processing workflow all together as there was no coherent program or script for performing these tasks before. Using this data pipeline system, we were able to then also connect the models trained from DSN antenna data, completing the data workflow for DSN anomaly detection. This was all wrapped around and further connected by an agentic AI system, where complex reasoning was utilized to determine the classifications and predictions of anomalous data.

Figures

Figures reproduced from arXiv: 2508.21111 by the authors.

**Figure 3.** Logarithmic graph of all the principal components and their cumulative explained variance [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

**Figure 8.** Line plot of GAN reconstruction errors for DSS [PITH_FULL_IMAGE:figures/full_fig_p007_8.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

11 extracted references · 6 canonical work pages

[1]

(2018, September 3)

ADL. (2018, September 3). An introduction to Q-Learning: reinforcement learning. freeCodeCamp. https://www.freecodecamp.org/news/an- introduction-to-q-learning-reinforcement-learning-14ac0b4493cc/

work page 2018
[2]

(2023, July 28)

Bhargava, E., Sikorski, A., Soldan, H, & Locke, L. (2023, July 28). Improving the Deep Space Network Data Pipeline: A Case Study in Data Engineering and Machine Learning-based Anomaly Detection. NASA Jet Propulsion Laboratory

work page 2023
[3]

(2025, May 19)

Fejjari, A., Delavault, A., Camilleri, R., & Valentino, G. (2025, May 19). A Review of Anomaly Detection in Spacecraft Telemetry Data. University of Malta. https://www.mdpi.com/2076-3417/15/10/5653

work page 2025
[4]

(2014, June 10)

Goodfellow, J., Pouget-Abadie, J., et al. (2014, June 10). Generative Adversarial Nets. Université de Montréal. https://arxiv.org/pdf/1406.2661. High Performance Computing. (2025, April 11). JPL Information Technology. https://jplit.jpl.nasa.gov/it-services/high-performance-computing

arXiv 2014
[5]

(2021, June 17)

Hu, E., Shen, Y ., et al. (2021, June 17). LoRA: Low-Rank Adaptation of Large Language Models. Hugging Face. https://huggingface.co/papers/2106.09685

arXiv 2021
[6]

(2023, October 10)

Jiang, A., Sablayrolles, A., et al. (2023, October 10). Mistral 7B. Mistral AI. https://arxiv.org/pdf/2310.06825

arXiv 2023
[7]

(2017, January 30)

Kingma, Diederik & Ba, Jimmy Li. (2017, January 30). Adam: A Method for Stochastic Optimization. International Conference for Learning Representations. https://arxiv.org/pdf/1412.6980

arXiv 2017
[8]

(2024, February 12)

Murel, Jacob & Kavlakoglu, Eda. (2024, February 12). What is transfer learning? IBM. https://www.ibm.com/think/topics/transfer-learning

work page 2024

Show all 11 references

[9]

(2023, August 30)

Sooriyarachchi, Avinash. (2023, August 30). Efficient Fine-Tuning with LoRA: A Guide to Optimal Parameter Selection for Large Language Models. Databricks. https://www.databricks.com/blog/efficient-fine-tuning-lora-guide-llms

2023
[10]

(2019, September 23)

Staudemeyer, R., & Morris, E. (2019, September 23). Understanding LSTM – a tutorial into Long Short-Term Memory Recurrent Neural Networks. https://arxiv.org/pdf/1909.09586. Time Series Transformer. (n.d.). Hugging Face. https://huggingface.co/docs/transformers/model_doc/time_s...

2019 arXiv
[11]

(2022, June 17)

Xue, S., Chen, H., & Zheng, X. (2022, June 17). Detection and quantification of anomalies in communication networks based on LSTM-ARIMA combined model. Anhui University of Science and Technology. https://pmc.ncbi.nlm.nih.gov/articles/PMC9205417/

2022

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

(2018, September 3)

ADL. (2018, September 3). An introduction to Q-Learning: reinforcement learning. freeCodeCamp. https://www.freecodecamp.org/news/an- introduction-to-q-learning-reinforcement-learning-14ac0b4493cc/

work page 2018

[2] [2]

(2023, July 28)

Bhargava, E., Sikorski, A., Soldan, H, & Locke, L. (2023, July 28). Improving the Deep Space Network Data Pipeline: A Case Study in Data Engineering and Machine Learning-based Anomaly Detection. NASA Jet Propulsion Laboratory

work page 2023

[3] [3]

(2025, May 19)

Fejjari, A., Delavault, A., Camilleri, R., & Valentino, G. (2025, May 19). A Review of Anomaly Detection in Spacecraft Telemetry Data. University of Malta. https://www.mdpi.com/2076-3417/15/10/5653

work page 2025

[4] [4]

(2014, June 10)

Goodfellow, J., Pouget-Abadie, J., et al. (2014, June 10). Generative Adversarial Nets. Université de Montréal. https://arxiv.org/pdf/1406.2661. High Performance Computing. (2025, April 11). JPL Information Technology. https://jplit.jpl.nasa.gov/it-services/high-performance-computing

arXiv 2014

[5] [5]

(2021, June 17)

Hu, E., Shen, Y ., et al. (2021, June 17). LoRA: Low-Rank Adaptation of Large Language Models. Hugging Face. https://huggingface.co/papers/2106.09685

arXiv 2021

[6] [6]

(2023, October 10)

Jiang, A., Sablayrolles, A., et al. (2023, October 10). Mistral 7B. Mistral AI. https://arxiv.org/pdf/2310.06825

arXiv 2023

[7] [7]

(2017, January 30)

Kingma, Diederik & Ba, Jimmy Li. (2017, January 30). Adam: A Method for Stochastic Optimization. International Conference for Learning Representations. https://arxiv.org/pdf/1412.6980

arXiv 2017

[8] [8]

(2024, February 12)

Murel, Jacob & Kavlakoglu, Eda. (2024, February 12). What is transfer learning? IBM. https://www.ibm.com/think/topics/transfer-learning

work page 2024