Differential fuzz testing to detect tampering in sensor systems and its application to arms control authentication
Pith reviewed 2026-05-24 02:35 UTC · model grok-4.3
The pith
Physical differential fuzz testing detects tampering in radiation sensors by comparing output sequences from the same randomized input sequences.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Physical differential fuzz testing creates a baseline signature by fuzzing the parameter space of an untampered radiation measurement system with randomized inputs, including off-normal values, and records the resulting output time series. Applying the same input sequence to a tampered system yields a modified output sequence that raises an alarm, even when Poisson noise is present, through a dedicated comparison mechanism. The method simultaneously verifies all layers of the cyber-physical system and was shown to detect two classes of tamper attempts on a NaI spectrometer.
What carries the argument
physical differential fuzz testing, a challenge-response process that records baseline output sequences from randomized inputs and compares them to outputs from the same inputs on a suspect system
If this is right
- Tampering with environment variables, external libraries, firmware, or hardware becomes detectable in one integrated test.
- Radiation measurement equipment in nuclear weapon verification systems can be authenticated without relying solely on software integrity checks.
- Stochastic outputs from radiation detectors can still be compared reliably using the noise-handling mechanism.
- The approach provides a framework usable for authenticating other cyber-physical systems in safeguards and arms control.
Where Pith is reading between the lines
- The same randomized-input comparison could apply to non-radiation sensors if their outputs show consistent baseline behavior under repeated inputs.
- Longer input sequences or adaptive fuzzing ranges might improve detection sensitivity for subtle tampering.
- Combining this method with existing hashing techniques could create layered authentication that covers both static and dynamic states.
Load-bearing premise
A tampered system will produce a measurably different output sequence from the untampered baseline under the same randomized inputs, and the noise-handling comparison can reliably separate tampering from normal Poisson variation without excessive errors.
What would settle it
Running the fuzz test protocol on a known-tampered NaI spectrometer and finding that its output sequence matches the baseline within the noise threshold, or that the comparison mechanism flags untampered runs as tampered at high rates.
Figures
read the original abstract
In future nuclear arms control treaties, it will be necessary to authenticate the hardware and software components of verification measurement systems, i.e., to ensure these systems are functioning as intended and have not been tampered with by malicious actors. While methods such as source code hashing and static analysis can help verify the integrity of software components, they may not be capable of detecting tampering with environment variables, external libraries, or the firmware and hardware of radiation measurement systems. In this article, we introduce the concept of physical differential fuzz testing as a challenge-response-style tamper indicator that can holistically and simultaneously test all the above components in a cyber-physical system. In essence, we randomly sample (or "fuzz") the untampered system's parameter space, including both normal and off-normal parameter values, and consider the time series of outputs as the baseline signature of the system. Re-running the same input sequence on a untampered system will produce an output sequence consistent with this baseline, while running the same input sequence on a tampered system will produce a modified output sequence and raise an alarm. We then apply this concept to authenticating the radiation measurement equipment in nuclear weapon verification systems and conduct demonstration fuzz testing measurements with a sodium iodide (NaI) gamma ray spectrometer. Because there is Poisson noise in the measured output spectra, we also use a mechanism for comparing inherently noisy or stochastic fuzzing sequences. We show that physical differential fuzz testing can detect two types of tamper attempts, and conclude that it is a promising framework for authenticating future cyber-physical systems in nuclear arms control, safeguards, and beyond.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes physical differential fuzz testing as a challenge-response tamper detection method for cyber-physical sensor systems. It generates a baseline output signature by applying randomized input sequences (including off-normal values) to an untampered system, then compares subsequent runs of the same sequence; deviations indicate tampering. The approach is demonstrated on a NaI gamma-ray spectrometer to authenticate radiation measurement equipment for nuclear arms control verification, with a mechanism to handle Poisson noise in the output spectra. The central claim is that the method detects two types of tamper attempts and offers a holistic test beyond software integrity checks.
Significance. If the detection reliability can be quantified and the baseline-establishment issue resolved, the framework could provide a practical, hardware-level complement to code hashing for authenticating complex verification systems in arms control and safeguards. The use of real hardware demonstration and explicit handling of stochastic noise are positive elements, but the absence of metrics and the unaddressed adversarial baseline requirement limit the immediate significance.
major comments (2)
- [Introduction and application to radiation measurement equipment] Introduction and application sections: The method requires generating the baseline signature from a known-untampered system, yet no procedure is described for establishing or attesting this reference when the inspected party supplies the equipment in an arms-control authentication scenario; this assumption is load-bearing for the claimed applicability.
- [Demonstration with NaI spectrometer] Demonstration description (NaI spectrometer section): The claim that two tamper types are detected is presented without quantitative metrics, false-positive/negative rates, error analysis, or a full specification of the Poisson-noise comparison algorithm (e.g., distance metric, threshold, or statistical test), leaving the central detection result only qualitatively supported.
minor comments (2)
- [Abstract] Abstract: 'a untampered system' should read 'an untampered system'.
- [Method description] Notation for the input/output sequences and comparison mechanism could be formalized with equations to improve clarity and reproducibility.
Simulated Author's Rebuttal
We thank the referee for their constructive comments. We address each major comment below, agreeing that both points identify areas where the manuscript can be strengthened through clarification and additional detail.
read point-by-point responses
-
Referee: Introduction and application sections: The method requires generating the baseline signature from a known-untampered system, yet no procedure is described for establishing or attesting this reference when the inspected party supplies the equipment in an arms-control authentication scenario; this assumption is load-bearing for the claimed applicability.
Authors: We agree this is a substantive point for real-world applicability. The manuscript presents the differential fuzz testing method under the assumption that a trusted baseline can be obtained beforehand. In revision we will expand the introduction and application sections to outline a practical procedure, for example by having the inspecting party perform the initial baseline measurements in a controlled environment prior to equipment handover, or by combining the method with independent hardware attestation steps. This will make the load-bearing assumption explicit and address how it can be satisfied in an arms-control workflow. revision: yes
-
Referee: Demonstration description (NaI spectrometer section): The claim that two tamper types are detected is presented without quantitative metrics, false-positive/negative rates, error analysis, or a full specification of the Poisson-noise comparison algorithm (e.g., distance metric, threshold, or statistical test), leaving the central detection result only qualitatively supported.
Authors: We accept that the demonstration is currently qualitative. In the revised manuscript we will augment the NaI spectrometer section with quantitative support: repeated-trial false-positive and false-negative rates, an error analysis that incorporates the Poisson statistics of the spectra, and a complete description of the comparison algorithm (including the chosen distance metric, threshold selection, and statistical test). These additions will place the detection claims on a firmer quantitative footing while remaining consistent with the existing experimental data. revision: yes
Circularity Check
No significant circularity; method is an independent experimental concept
full rationale
The paper presents physical differential fuzz testing as a new challenge-response framework for tamper detection in cyber-physical systems. It defines the baseline signature explicitly as the output time series obtained by fuzzing an untampered unit, then compares subsequent runs against that reference. This is a definitional construction of the test procedure itself, not a derivation in which a claimed prediction or result reduces by construction to its own inputs. No equations, fitted parameters, or self-citations appear in the provided text that would create self-definitional, fitted-input, or uniqueness-imported circularity. The noise-handling comparison for Poisson statistics is described as an auxiliary mechanism rather than a load-bearing derivation. The paper is therefore self-contained against external benchmarks and receives the default non-circularity finding.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Tampering with any component of the cyber-physical system will produce a detectable deviation in the output time series for the same randomized input sequence.
Reference graph
Works this paper leans on
-
[1]
“New START Treaty,” retrieved Nov. 17, 2023 from https://www.nti.org/education-center/treaties- and-regimes/treaty-between-the-united-states-of-america-and-the-russian-federation-on- measures-for-the-further-reduction-and-limitation-of-strategic-offensive-arms/
work page 2023
-
[2]
A zero-knowledge protocol for nuclear warhead verifica- tion,
A. Glaser, B. Barak, and R. J. Goldston, “A zero-knowledge protocol for nuclear warhead verifica- tion,”Nature, vol. 510, no. 7506, pp. 497–502, 2014. 18/22
work page 2014
-
[3]
A physical zero-knowledge object- comparison system for nuclear warhead verification,
S. Philippe, R. J. Goldston, A. Glaser, and F. d’Errico, “A physical zero-knowledge object- comparison system for nuclear warhead verification,”Nature Communications, vol. 7, no. 1, p. 12890, 2016
work page 2016
-
[4]
Physical cryptographic verification of nuclear warheads,
R. S. Kemp, A. Danagoulian, R. R. Macdonald, and J. R. Vavrek, “Physical cryptographic verification of nuclear warheads,”Proceedings of the National Academy of Sciences, vol. 113, no. 31, pp. 8618– 8623, 2016
work page 2016
-
[5]
J. R. Vavrek, B. S. Henderson, and A. Danagoulian, “Experimental demonstration of an isotope- sensitive warhead verification technique using nuclear resonance fluorescence,”Proceedings of the National Academy of Sciences, vol. 115, no. 17, pp. 4363–4368, 2018
work page 2018
-
[6]
Nuclear disarmament verification via resonant phenomena,
J. J. Hecla and A. Danagoulian, “Nuclear disarmament verification via resonant phenomena,”Nature Communications, vol. 9, no. 1, p. 1259, 2018
work page 2018
-
[7]
A physically cryptographic warhead verification system using neutron induced nuclear resonances,
E. M. Engel and A. Danagoulian, “A physically cryptographic warhead verification system using neutron induced nuclear resonances,”Nature Communications, vol. 10, no. 1, p. 4433, 2019
work page 2019
-
[8]
Investigation into practical implementations of a zero knowledge protocol
P. Marleau and R. E. Krentz-Wee, “Investigation into practical implementations of a zero knowledge protocol.” Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States), Tech. Rep. SAND2017-1649, 2017
work page 2017
-
[9]
17, 2023 from https://www.nti.org/wp-content/uploads/2021/09/newstart annex inspections.pdf
“Annex on inspection activities to the protocol to the treaty between the United States of America and the Russian Federation on measures for the further reduction and limitation of strategic offensive arms,” retrieved Nov. 17, 2023 from https://www.nti.org/wp-content/uploads/2021/09/newstart annex inspections.pdf
work page 2023
-
[10]
Trusting embedded hardware and software in treaty verification systems
J. K. Brotz, “Trusting embedded hardware and software in treaty verification systems.” Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States), Tech. Rep. SAND2019- 5077C, 2019
work page 2019
-
[11]
Next-generation arms-control agreements based on emerging radiation detection technologies
M. C. Hamel, “Next-generation arms-control agreements based on emerging radiation detection technologies.” Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States), Tech. Rep. SAND2018-6183C, 2018
work page 2018
-
[12]
M. K¨utt and A. Glaser, “Vintage electronics for trusted radiation measurements and verified disman- tlement of nuclear weapons,”PLOS One, vol. 14, no. 10, p. e0224149, 2019
work page 2019
-
[13]
Increasing inspectability of hardware and software for arms control and nonproliferation regimes,
G. White, “Increasing inspectability of hardware and software for arms control and nonproliferation regimes,” Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States), Tech. Rep. UCRL-JC-144409, 2001. 19/22
work page 2001
-
[14]
Computer language choices in arms control and nonproliferation regimes,
——, “Computer language choices in arms control and nonproliferation regimes,” Lawrence Liv- ermore National Laboratory (LLNL), Livermore, CA (United States), Tech. Rep. UCRL-CONF- 212857, 2005
work page 2005
-
[15]
Equipment design best practices for authentication,
T. Weber, D. Maierhafer, R. Helguero, M. Coram, J. Benz, M. Williamson, A. Swift, and J. Warner, “Equipment design best practices for authentication,” Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States), Tech. Rep. SAND2023-05546, 2023
work page 2023
-
[16]
Software development and authentication for arms control information barriers,
N. Evans, “Software development and authentication for arms control information barriers,” in FM 2015: Formal Methods: 20th International Symposium, Oslo, Norway, June 24-26, 2015, Proceedings 20. Springer, 2015, pp. 581–584
work page 2015
-
[17]
J. Brotz, E. Connolly, E. Enger, M. Goliath, N. Grant, I. Hayes, S. Høibr˚aten, R. Hughes, S. Kaald, J. Knowleset al., “2022 Data Authentication Demonstration: Exploration of host and inspector confidence in hardware, software, and data,” Quadrilateral Nuclear Verification Partnership, Tech. Rep., 2023
work page 2022
-
[18]
LETTERPRESS post-simulation report,
“LETTERPRESS post-simulation report,” Quadrilateral Nuclear Verification Partnership, Tech. Rep., 2020, retrieved Dec. 11, 2023 from https://quad-nvp.info/wp-content/uploads/2020/11/LP- Post-Simulation-Report P2P.pdf
work page 2020
-
[19]
Verification of operating software for cooperative monitoring applications,
K. M. Tolk and R. K. Rembold, “Verification of operating software for cooperative monitoring applications,” Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States), Tech. Rep. SAND-97-1785C, 1997
work page 1997
-
[20]
Authentication of monitoring systems for non-proliferation and arms control,
R. T. Kouzes and J. L. Fuller, “Authentication of monitoring systems for non-proliferation and arms control,” inProc. Symposium on International Safeguards: Verification and Nuclear Material Security, International Atomic Energy Agency, Vienna, Austria, 2001
work page 2001
-
[21]
Defining the questions: a research agenda for nontraditional authentication in arms control,
D. K. Hauck, D. W. Mac Arthur, M. K. Smith, J. L. Thron, and K. Budlong-Sylvester, “Defining the questions: a research agenda for nontraditional authentication in arms control,” Los Alamos National Laboratory (LANL), Los Alamos, NM (United States), Tech. Rep. LA-UR-10-03785, 2010
work page 2010
-
[22]
Simultaneous authentication and certification of arms- control measurement systems,
D. MacArthur, D. Hauck, and J. Thron, “Simultaneous authentication and certification of arms- control measurement systems,” inProceedings of the Institute of Nuclear Materials Management 53 rd Annual Meeting, Orlando, 2012, pp. 15–19
work page 2012
-
[23]
G. White, “Tools for authentication,” Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States), Tech. Rep. LLNL-CONF-405315, 2008
work page 2008
-
[24]
Trends in hardware authentication,
——, “Trends in hardware authentication,” Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States), Tech. Rep. LLNL-CONF-674264, 2015. 20/22
work page 2015
-
[25]
An empirical study of the reliability of UNIX utilities,
B. P. Miller, L. Fredriksen, and B. So, “An empirical study of the reliability of UNIX utilities,” Communications of the ACM, vol. 33, no. 12, pp. 32–44, 1990, also appears (in German translation) as “Fatale Fehlertractigkeit: Eine Empirische Studie zur Zuverlassigkeit von UNIX-Utilities”, iX, March 1991
work page 1990
-
[26]
G. Klees, A. Ruef, B. Cooper, S. Wei, and M. Hicks, “Evaluating fuzz testing,” inProceedings of the 2018 ACM SIGSAC conference on computer and communications security, 2018, pp. 2123–2138
work page 2018
-
[27]
The relevance of classic fuzz testing: Have we solved this one?
B. P. Miller, M. Zhang, and E. R. Heymann, “The relevance of classic fuzz testing: Have we solved this one?”IEEE Transactions on Software Engineering, vol. 48, no. 6, pp. 2028–2039, 2020
work page 2028
- [28]
-
[29]
Differential testing for software,
W. M. McKeeman, “Differential testing for software,”Digital Technical Journal, vol. 10, no. 1, pp. 100–107, 1998
work page 1998
-
[30]
Finding and understanding bugs in C compilers,
X. Yang, Y . Chen, E. Eide, and J. Regehr, “Finding and understanding bugs in C compilers,” inProceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation, 2011, pp. 283–294
work page 2011
-
[31]
Remote inspection of adversary-controlled environments,
J. Tobisch, S. Philippe, B. Barak, G. Kaplun, C. Zenger, A. Glaser, C. Paar, and U. R ¨uhrmair, “Remote inspection of adversary-controlled environments,”Nature Communications, vol. 14, no. 1, p. 6566, 2023
work page 2023
-
[32]
Portal monitor for diversion safeguards,
W. Chambers, H. Atwater, P. Fehlau, R. Hastings, C. Henry, W. Kunz, T. Sampson, T. Whittlesey, and G. Worth, “Portal monitor for diversion safeguards,” Los Alamos National Laboratory (LANL), Los Alamos, NM (United States), Tech. Rep. LA-5681, 1974
work page 1974
-
[33]
Overview of the US-UK Portal Monitor for Authentication and Certification (PMAC) Project [slides],
A. Swift, “Overview of the US-UK Portal Monitor for Authentication and Certification (PMAC) Project [slides],” Oak Ridge Y-12 Plant (Y-12), Oak Ridge, TN (United States), Tech. Rep., 2020
work page 2020
-
[34]
Testing Consistency of Two Histograms
F. C. Porter, “Testing consistency of two histograms,”arXiv preprint arXiv:0804.0380, 2008
work page internal anchor Pith review Pith/arXiv arXiv 2008
-
[35]
Trusted Radiation Identification System,
K. Seager, D. Mitchell, T. Laub, K. Tolk, R. Lucero, and K. Insch, “Trusted Radiation Identification System,” inProceedings of the 42nd Annual INMM Meeting, 2001
work page 2001
-
[36]
C. R. Harris, K. J. Millman, S. J. van der Walt, R. Gommers, P. Virtanen, D. Cournapeau, E. Wieser, J. Taylor, S. Berg, N. J. Smith, R. Kern, M. Picus, S. Hoyer, M. H. van Kerkwijk, M. Brett, A. Haldane, J. Fern ´andez del R ´ıo, M. Wiebe, P. Peterson, P. G ´erard-Marchant, K. Sheppard, T. Reddy, W. Weckesser, H. Abbasi, C. Gohlke, and T. E. Oliphant, “Ar...
work page 2020
-
[37]
To kill a centrifuge: A technical analysis of what Stuxnet’s creators tried to achieve,
R. Langner, “To kill a centrifuge: A technical analysis of what Stuxnet’s creators tried to achieve,” The Langner Group, vol. 37, 2013. 21/22
work page 2013
-
[38]
Zetter,Countdown to Zero Day: Stuxnet and the launch of the world’s first digital weapon
K. Zetter,Countdown to Zero Day: Stuxnet and the launch of the world’s first digital weapon. Crown Publishing Group, 2014
work page 2014
-
[39]
The science behind the V olkswagen emissions scandal,
Q. Schiermeier, “The science behind the V olkswagen emissions scandal,”Nature, vol. 9, p. 24, 2015
work page 2015
-
[40]
‘Underhanded C’ contest highlights challenges for nuclear arms control verification technologies,
“‘Underhanded C’ contest highlights challenges for nuclear arms control verification technologies,” retrieved Dec. 5, 2023 from https://www.nti.org/news/underhanded-c-contest-highlights-challenges- nuclear-arms-control-verification-technologies/
work page 2023
-
[41]
Effect of temperature on the performance of a CZT radiation detector,
S. Park, J. Ha, J. Lee, H. Kim, Y . Cho, S. Cheon, and D. Hong, “Effect of temperature on the performance of a CZT radiation detector,”Journal of the Korean Physical Society, vol. 56, no. 4, pp. 1079–1082, 2010
work page 2010
-
[42]
Reflections on trusting trust,
K. Thompson, “Reflections on trusting trust,”Communications of the ACM, vol. 27, no. 8, pp. 761–763, 1984
work page 1984
-
[43]
Timeline of the xz open source attack,
R. Cox, “Timeline of the xz open source attack,” retrieved Apr. 2, 2024 from https://research.swtch. com/xz-timeline
work page 2024
-
[44]
Non-RF Chain of Custody Item Monitor (CoCIM) development report,
J. K. Brotz, J. R. Wade, and S. R. Schwartz, “Non-RF Chain of Custody Item Monitor (CoCIM) development report,” Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States), Tech. Rep. SAND2017-9716, 2017
work page 2017
-
[45]
Information barriers—a historical perspective,
D. Close, D. MacArthur, and N. Nicholas, “Information barriers—a historical perspective,” Los Alamos National Laboratory (LANL), Los Alamos, NM (United States), Tech. Rep. LA-UR-01-2180, 2001
work page 2001
-
[46]
D. M. Chambers and M. David, “UK-Norway Initiative: Research into information barriers to allow warhead attribute verification without release of sensitive or proliferative information,” in Proceedings of the 51 st Annual Meeting of the Institute of Nuclear Materials Management, Baltimore, MD, 2010
work page 2010
-
[47]
Nuclear warhead verification: A review of attribute and template systems,
J. Yan and A. Glaser, “Nuclear warhead verification: A review of attribute and template systems,” Science & Global Security, vol. 23, no. 3, pp. 157–170, 2015
work page 2015
-
[48]
M. K ¨utt, M. G ¨ottsche, and A. Glaser, “Information barrier experimental: Toward a trusted and open-source computing platform for nuclear warhead verification,”Measurement, vol. 114, pp. 185–190, 2018
work page 2018
-
[49]
Development of a general, modular, reprogrammable information barrier for arms control applications,
J. K. Brotz, J. K. Polack, R. Helguero, M. Hamel, T. Weber, and P. Marleau, “Development of a general, modular, reprogrammable information barrier for arms control applications,” Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States), Tech. Rep. SAND2023- 03283A, 2023. 22/22
work page 2023
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.