REVIEW 3 major objections 9 minor 35 references
Galileo Project Observatory Class System Architecture
T0 review · 3 major / 9 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper reports that a two-subsystem observatory platform deployed at three sites, collecting roughly 350 GB of multimodal data per day, makes sustained transparent UAP monitoring feasible
desk verdict A real deployment report with useful architectural detail, undermined by unvalidated detection and an over-claimed fusion result. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is OCICP itself, a two-subsystem architecture. At the edge, an event-driven architecture built on ZeroMQ messaging connects data collectors for each sensor (Dalek infrared array, Alcor all-sky camera, acoustic system, magnetometer, ADS-B receiver) to a Data Fusion Engine that performs entity resolution and state estimation using the JDL data fusion model; Sentinel, a YOLOv5-based detector, supplies object detections and tracks. In post-processing, a raw data lake, commissioning and census operations, and a blinded science-operations workflow turn the edge data into analysis-ready products, with a file-naming and directory scheme that encodes site, processor, source, and date for provenance. The mechanism that carries the argument is the decoupling: the edge can collect and react in real time, while the post-processing side can scale, validate, and analyze without contaminating the raw record.
What would settle it
Count how often Sentinel identifies an object that ADS-B and visual inspection show to be a known aircraft or bird, and count how many transponder-equipped aircraft within the Dalek field of view it never detects; if either error rate is high, the claim that OCICP delivers calibrated, cross-verified monitoring data is falsified.
Extended reading notes
Core claim
The paper's central claim is that OCICP is not just a design but an operating system: it has been deployed at three observatory sites and is actively collecting long-term multimodal data. The system's advertised discovery, in the authors' terms, is that an event-driven edge architecture and a cloud/HPC post-processing environment can together provide the data provenance, calibration traceability, and reproducible workflows needed for a scientific census of aerial objects. The paper presents example products—infrared detections from the Dalek array, Alcor all-sky images, acoustic spectrograms, magnetometer comparisons with a standard geophysical observatory, and ADS-B aircraft counts—as evidence that the sensors are functioning and that the data flow is intact. It does not claim to have identified any anomalous object; it claims to have built the infrastructure on which such identification could eventually rest.
Load-bearing premise
The load-bearing premise is that Sentinel's treeline-masked, spline-smoothed detections are trustworthy enough to serve as scientifically sound event data, yet the paper offers no precision, recall, or ground-truth measurement to support that.
Editorial extensions
If this is right
- A multi-year, multimodal census of ordinary aerial objects becomes possible, giving researchers a statistical baseline against which true outliers must be judged.
- Independent researchers can rerun detections and classifications on the same raw and quick-look data, because the pipeline records calibration, provenance, and configuration alongside each file.
- Adding new observatory sites requires commissioning and calibration rather than redesign, since the edge/post-processing split is uniform across sites.
- Planned additions such as passive radar, optical spectroscopy, and autonomous PTZ response would extend the same architecture to more modalities without changing its core data paths.
Reading between the lines
- The open, provenance-stamped dataset could serve as a shared benchmark for aerial-object detection, letting outside groups measure their detectors against the same sky rather than against proprietary footage.
- The edge/post-processing split is not specific to UAP and could be transplanted to other long-term environmental monitoring programs that need calibrated, auditable, multimodal data.
- Because the paper does not report detector precision or recall, the most decisive early test of the platform's scientific value will be an independent validation of Sentinel's false-positive rate against ADS-B and bird-observation ground truth.
- A concrete next step the authors leave implicit would be to publish per-site, per-modality detection statistics so the research community can judge data quality before trusting anomaly candidates.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents the Observatory Class Integrated Computing Platform (OCICP), a two-subsystem architecture for multimodal UAP monitoring: an Edge Computing Subsystem for real-time acquisition, detection, tracking, and provenance management, and a Post-Processing Subsystem for commissioning, census, science operations, and effectiveness monitoring. It describes the system life-cycle and processes, the event-driven edge implementation (including the Sentinel/YOLOv5-based detection pipeline and the Data Fusion Engine), the post-processing data lake and blind-analysis workflow, and reports preliminary results from deployments at three observatory sites, including a data yield of approximately 350 GB per day and example products from IR, all-sky, acoustic, magnetometer, and ADS-B sensors. The central claim is that these deployments demonstrate that OCICP is a viable and scalable architecture for sustained, transparent, and scientifically rigorous monitoring operations.
Significance. If the claims are substantiated, OCICP would be a valuable open-data platform for UAP science, directly addressing the field's chronic lack of calibrated, well-provenanced, multimodal data. The paper's strengths include a clear system-life-cycle framework, explicit data-provenance conventions via file naming, a FAIR-aligned data-product structure, a planned blind-analysis scheme, and a level of architectural detail that supports replication by other groups. The deployment evidence (data volumes, example detections, calibration traceability for the magnetometer) credibly supports the narrower claim that the platform is operational and collects multimodal data. However, the broader assertions of scientific rigor and improved classification confidence are not yet backed by quantitative validation, and one of those assertions appears to be contradicted by the paper's own implementation-status statements.
major comments (3)
- [3.6 and 3.1] The claim in Section 3.6 that multimodal fusion "significantly improves the confidence level in event classification" is not supported by any measured classification performance, and it is internally inconsistent with Section 2.2.4, which states that real-time classification "is not implemented currently," and with Section 3.1, which states that "real-time classification of objects is not currently implemented." As written, this is an unsupported result rather than a design goal. Please either present classification performance metrics from a working implementation (e.g., accuracy, confusion matrix, or comparison against labeled events) or revise the sentence to describe a planned capability.
- [3.2 and Figure 10] The Sentinel/YOLOv5 detection pipeline is presented as the basis for cross-verification and for the "scientifically rigorous" claim in the opening of Section 3, but the paper reports no detection accuracy metrics. No precision, recall, false-positive rate, confusion matrix, or comparison against an annotated ground-truth set is given for the deployed detector, and the effects of treeline masking and spline smoothing on detection quality are not measured. Without such validation, the claim that OCICP produces scientifically sound data through its detection and cross-checking pipeline is not established. Please add at least a basic quantitative evaluation of the detector's performance at the deployed sites, or explicitly scope the scientific-rigor claim to the raw and calibrated data collection rather than to the detection products.
- [3 (opening)] The statement that deployments "demonstrate that OCICP is a viable and scalable architecture for sustained, transparent, and scientifically rigorous monitoring operations" goes beyond the evidence presented in the Results section. The evidence consists of data volumes and example products; scalability is asserted but not measured (no uptime statistics, no per-site consistency metrics, no stress tests of the event-driven architecture), and scientific rigor is asserted but not demonstrated through validation of the detection/classification chain. Please either provide quantitative system-effectiveness evidence (e.g., operational uptime, detection consistency across sites, data completeness rates) or temper the demonstration claim to reflect what has actually been shown.
minor comments (9)
- [2.2.2] Typo: "Raspiberry Pi Model 4" should be "Raspberry Pi Model 4."
- [3.6] Typo: "acoutstic monitoring" should be "acoustic monitoring."
- [1.1] Typo: "studing anomalous aerial phenomena" should be "studying anomalous aerial phenomena."
- [2.1.6] The function name "Target Periodization" appears to be a typo for "Target Prioritization," which is the term used in the preceding paragraph and elsewhere.
- [2.1.8] The functional area is called "Data Inspection" elsewhere, but this subsection begins "The Data Inspect functional area"; please make the naming consistent.
- [1.2] Typo: "OCIPC" should be "OCICP" in the sentence "The primary user community for OCIPC is the Research Team."
- [2.2.7 and Figure 6] The text says the DFE Dashboard screen capture is shown in Figure 5, but the figure caption and the subsequent paragraph refer to Figure 6; the cross-reference should be corrected.
- [Table 4] The phrase "in the from of" appears twice in the Alerts and Alarms and Informational Messages rows; it should be "in the form of."
- [Data Availability Statement] The Data Availability Statement says raw data "will be made available by the authors on request," which is weaker than the paper's earlier claim of making data openly available to the research community; please clarify the intended access model and any embargo period.
Circularity Check
No significant circularity: OCICP is a descriptive systems-engineering paper whose deployment and data-yield claims rest on in-paper operational reporting, not on self-citation or fitted predictions.
full rationale
This paper does not derive any quantitative scientific result from an input. It describes the architecture, lifecycle, implementation status, and preliminary data yield of the OCICP system. The central claims—that the system has been deployed at three sites, generates roughly 350 GB/day, and enables calibrated multimodal data collection—are supported by the in-paper descriptions in Sections 3.1–3.3, not by a mathematical derivation or by a fitted parameter. No equation is presented that reduces a prediction to its own input, and no fitted quantity is relabeled as a prediction. The cited prior works by the same group (e.g., Watters et al. [1], Cloete et al. [4], Domine et al. [12], Szenher et al. [22]) supply project context, design lineage, and instrument-specific commissioning details, but the present paper's viability and scalability claims do not logically reduce to those citations; they stand or fall on the reported deployments and data volumes. The skeptic's concern about missing detector precision/recall validation and the unsupported statement in Section 3.6 that fusion 'significantly improves the confidence level in event classification' is a legitimate evidentiary weakness, but it is a correctness and completeness gap, not circularity. The paper also explicitly discloses that real-time classification is not implemented, which further indicates that no fitted classifier output is being presented as a validated prediction. Therefore no circular step meets the evidence bar required by the review rules.
Assumptions & free parameters
free parameters (2)
- Magnetometer baseline adjustment =
not specified
- Treeline mask and spline smoothing parameters =
not specified
assumptions (4)
- domain assumption High-quality, well-calibrated multimodal data are the primary missing ingredient for scientific UAP investigation.
- domain assumption Event-driven architecture and the JDL data fusion model provide a sufficient foundation for scientifically valid multi-sensor data collection and analysis.
- domain assumption Planned data blinding can be applied losslessly to all OCICP measurement types without degrading the scientific signal.
- domain assumption YOLOv5-based Sentinel detections, after treeline masking and spline smoothing, are accurate enough for scientific use.
Cite this review
Pith. "Pith review of Galileo Project Observatory Class System Architecture." pith.science (2026). https://pith.science/paper/POIMHJBV
@misc{pith2026250600125,
author = {Pith},
title = {Pith review of: Galileo Project Observatory Class System Architecture},
year = {2026},
howpublished = {\url{https://pith.science/paper/POIMHJBV}},
note = {Machine review of arXiv:2506.00125}
}
read the original abstract
Scientific investigation of Unidentified Anomalous Phenomena (UAP) is limited by poor data quality and a lack of transparency. Existing data are often fragmented, uncalibrated, and missing critical metadata. To address these limitations, the authors present the Observatory Class Integrated Computing Platform (OCICP), a system designed for the systematic and scientific study of UAPs. OCICP employs multiple sensors to collect and analyze data on aerial phenomena. The OCICP system consists of two subsystems. The first is the Edge Computing Subsystem which is located within the observatory site. This subsystem performs real-time data acquisition, sensor optimization, and data provenance management. The second is the Post-Processing Subsystem which resides outside the observatory. This subsystem supports data analysis workflows, including commissioning, census operations, science operations, and system effectiveness monitoring. This design and implementation paper describes the system lifecycle, associated processes, design, implementation, and preliminary results of OCICP, emphasizing the ability of the system to collect comprehensive, calibrated, and scientifically sound data.
Figures
Figures from the paper (9 more)
Reference graph
Works this paper leans on
-
[1]
Watters, W.A.; Loeb, A.; Laukien, F.; Cloete, R.; Delacroix, A.; Dobroshinsky, S.; Horvath, B.; Kelderman, E.; Little, S.; Masson, E.; et al. The Scientific Investigation of Unidentified Aerial Phenomena (UAP) Using Multimodal Ground-Based Observatories. Journal of Astronomical Instrumentation 2023, p. 2340006. Publisher: World Scientific Publishing Co., ...
-
[2]
Overview of the Galileo Project
Loeb, A.; Laukien, F.H. Overview of the Galileo Project. Journal of Astronomical Instrumentation 2022, p. 2340003. Publisher: World Scientific Publishing Co., https://doi.org/10.1142/S2251171723400032
-
[3]
SETI Institute, 2024
work page 2024
-
[4]
Integrated Computing Platform for Detection and Tracking of Unidentified Aerial Phenomena (UAP)
Cloete, R.; Bridgham, P .; Dobroshinsky, S.; Ezell, C.; Fedorenko, A.; Laukien, F.; Little, S.; Loeb, A.; Masson, E.; Szenher, M.; et al. Integrated Computing Platform for Detection and Tracking of Unidentified Aerial Phenomena (UAP). Journal of Astronomical Instrumentation 2023. Publisher: World Scientific Publishing Co., https://doi.org/10.1142/S2251171...
-
[5]
OMG Systems Modeling Language, 2024
work page 2024
-
[6]
NASA Unidentified Anomalous Phenomena Independent Study Team Report, 2023
NASA. NASA Unidentified Anomalous Phenomena Independent Study Team Report, 2023
work page 2023
-
[7]
DOD Official Testifies Before Senate Subcommittee, 2023
work page 2023
-
[8]
DOD Working to Better Understand, Resolve Anomalous Phenomena, 2023
Vergun, D. DOD Working to Better Understand, Resolve Anomalous Phenomena, 2023
work page 2023
Show all 35 references
-
[9]
NASA Provides Coverage of Unidentified Anomalous Phenomena Meeting, 2023
Donaldson, A. NASA Provides Coverage of Unidentified Anomalous Phenomena Meeting, 2023
2023
-
[10]
Erforschung des Unidentified Aerial Phenomena an der JMU Würzburg, 2023
Kayal, H.; Greiner, T.; Kaiser, T.; Riegler, C. Erforschung des Unidentified Aerial Phenomena an der JMU Würzburg, 2023
2023
-
[11]
Initial Results From the First Field Expedition of UAPx to Study Unidentified Anomalous Phenomena, 2024
Szydagis, M.; Knuth, K.H.; Kugielsky, B.W.; Levy, C. Initial Results From the First Field Expedition of UAPx to Study Unidentified Anomalous Phenomena, 2024. arXiv:2312.00558
2024 arXiv
-
[12]
Commissioning Galileo Project’s All-Sky Infrared Camera Array for Aerial Detections
Domine, L.; Biswas, A.; Cloete, R.; Delacroix, A.; Fedorenko, A.; Jacaruso, L.; Kelderman, E.; Keto, E.; Little, S.; Loeb, A.; et al. Commissioning Galileo Project’s All-Sky Infrared Camera Array for Aerial Detections. Sensors 2024
2024
-
[13]
The power of events; Vol
Luckham, D. The power of events; Vol. 204, Addison-Wesley Reading, 2002
2002
-
[14]
Revisions to the JDL data fusion model
Steinberg, A.N.; Bowman, C.L.; White, F.E. Revisions to the JDL data fusion model. In Proceedings of the Sensor Fusion: Architectures, Algorithms, and Applications III. SPIE, March 1999, Vol. 3719, pp. 430–441. https://doi.org/10.1117/12.341367
1999 doi
-
[15]
Handbook of Multisensor Data Fusion: Theory and Practice, Second Edition; CRC Press, 2017
II, M.L.; Hall, D.; Llinas, J. Handbook of Multisensor Data Fusion: Theory and Practice, Second Edition; CRC Press, 2017
2017
-
[16]
ZeroMQ, 2023
ZeroMQ. ZeroMQ, 2023. Programmers: _:n12442
2023
-
[18]
SkyWatch: A Passive Multistatic Radar Network for the Measurement of Object Position and Velocity
Randall, M.; Delacroix, A.; Ezell, C.; Kelderman, E.; Little, S.; Loeb, A.; Masson, E.; Watters, W.A.; Cloete, R.; White, A. SkyWatch: A Passive Multistatic Radar Network for the Measurement of Object Position and Velocity. Journal of Astronomical Instrumentation 2022, p. 2340...
2022 doi
-
[19]
Automatic Dependent Surveillance-Broadcast (ADS-B) | Federal Aviation Administration, 2023
FAA. Automatic Dependent Surveillance-Broadcast (ADS-B) | Federal Aviation Administration, 2023. Version August 25, 2025 submitted to Sensors 33 of 33
2023
-
[20]
Raspberry Pi 4 Model B, 2024
Raspberry Pi. Raspberry Pi 4 Model B, 2024
2024
-
[21]
FlightAware Pro Stick® Plus | ADS-B SDR USB dongle, 2024
2024
-
[22]
A Hardware and Software Platform for Aerial Object Localization
Szenher, M.; Delacroix, A.; Keto, E.; Little, S.; Randall, M.; Watters, W.A.; Masson, E.; Cloete, R. A Hardware and Software Platform for Aerial Object Localization. Journal of Astronomical Instrumentation 2022, p. 2340002. Publisher: World Scientific Publishing Co., https://d...
2022 doi
-
[24]
YOLOv5, 2020
Jocher, G. YOLOv5, 2020. Publication Title: GitHub repository
2020
-
[25]
GStreamer: open source multimedia framework, 2024
2024
-
[26]
Python 3.0, 2008
Python Software Foundation. Python 3.0, 2008
2008
-
[27]
Spring Boot, 2014
Pivotal Software, I. Spring Boot, 2014
2014
-
[28]
Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions ; Addison-Wesley Professional, 2003
Hohpe, G.; Woolf, B. Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions ; Addison-Wesley Professional, 2003
2003
-
[29]
Design Patterns: Elements of Reusable Object-Oriented Software; Addison-Wesley Professional: Boston, MA, USA, 1994
Gamma, E.; Helm, R.; Johnson, R.; Vlissides, J. Design Patterns: Elements of Reusable Object-Oriented Software; Addison-Wesley Professional: Boston, MA, USA, 1994
1994
-
[30]
Core J2EE Patterns: Best Practices and Design Strategies, 2nd edition ed.; Prentice Hall PTR, 2003
Alur, D.; Crupi, J.; Malks, D. Core J2EE Patterns: Best Practices and Design Strategies, 2nd edition ed.; Prentice Hall PTR, 2003
2003
-
[31]
CesiumJS, 2011
CesiumGS. CesiumJS, 2011
2011
-
[32]
Hierarchical Data Format, version 5, 1997
The HDF Group. Hierarchical Data Format, version 5, 1997
1997
-
[33]
The FAIR Guiding Principles for scientific data management and stewardship
Wilkinson, M.D.; Dumontier, M.; Aalbersberg, I.J.; Appleton, G.; Axton, M.; Baak, A.; Blomberg, N.; Boiten, J.W.; Santos, L.B.d.S.; Bourne, P .E.; et al. The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data 2016, 3, 160018. Number: 1 Publ...
2016 doi
-
[34]
Data Lake Architecture: Designing the Data Lake and avoiding the garbage dump; Technics Publications, LLC, 2016
Inmon, B. Data Lake Architecture: Designing the Data Lake and avoiding the garbage dump; Technics Publications, LLC, 2016
2016
-
[35]
FAS Research Computing
-
[36]
Blind analysis: Hide results to seek the truth
MacCoun, R.; Perlmutter, S. Blind analysis: Hide results to seek the truth. Nature 2015, 526, 187–189
2015
-
[37]
Blind analysis in nuclear and particle physics
Klein, J.R.; Roodman, A. Blind analysis in nuclear and particle physics. Annu. Rev. Nucl. Part. Sci. 2005, 55, 141–163. Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) ...
2005
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.