Scalar and Vector Airborne Platform Calibration Using Quantum and Classical Magnetometers and Inertial Sensors
Pith reviewed 2026-05-08 16:11 UTC · model grok-4.3
The pith
Vector airborne magnetometer calibration is first-order sensitive to attitude errors, while scalar models are robust to misalignment.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Vector calibration models are intrinsically first-order sensitive to attitude errors, irrespective of the accuracy of the magnetic field measurements. These errors arise from inaccurate representation of the background field direction in the body frame, and can amplify small orientation errors into noticeable calibration residuals. Scalar calibration models remain robust to misalignment.
What carries the argument
The vector calibration model and its dependence on accurate body-frame representation of the background magnetic field direction, which turns small attitude errors into first-order residuals.
If this is right
- High-accuracy independent attitude references such as inertial navigation systems become essential for any vector-based airborne compensation scheme.
- Quantum NV sensors can still improve scalar calibration accuracy and simplify sensor placement and synchronization even if full vector models remain limited by attitude knowledge.
- The standard scalar OPM plus fluxgate configuration retains practical value because its robustness to misalignment reduces reliance on perfect attitude data.
- Platform magnetic compensation for anomaly navigation will continue to favor scalar-derived fields unless attitude accuracy improves substantially.
Where Pith is reading between the lines
- The first-order sensitivity may appear in other moving-platform vector magnetic applications such as drone-based surveys or underwater vehicles where orientation is imperfectly known.
- Hybrid systems could use NV vector data selectively for scalar corrections while falling back to robust scalar models for the main compensation step.
- Testing whether advanced INS fusion can reduce the observed residuals to second-order levels would directly test the practical severity of the attitude bottleneck.
Load-bearing premise
The evaluation assumes realistic sensor models and flight trajectories correctly capture how attitude errors interact with calibration residuals without missing important platform dynamics or sensor correlations.
What would settle it
Flight data showing whether vector calibration residuals grow linearly with small, measured orientation errors when the background field direction is represented inaccurately in the body frame, or whether residuals stay small regardless of those errors.
Figures
read the original abstract
Airborne magnetometry requires rigorous calibration to isolate geomagnetic signals from sensor errors and platform magnetic fields. This magnetic compensation is needed for applications like geophysical exploration and magnetic anomaly navigation. The standard approach utilizes a quantum scalar Optically Pumped Magnetometer (OPM) and a less sensitive fluxgate vector sensor for attitude information. This configuration typically results in a scalar approximation of the platform field. Advancements in high-sensitivity Diamond Nitrogen-Vacancy (NV) vector magnetometers now enable a re-evaluation of the standard hardware configuration and full vector calibration models. We show through rigorous theoretical analysis that scalar calibration models are robust to misalignment. Vector calibration models, however, are intrinsically first-order sensitive to attitude errors, irrespective of the accuracy of the magnetic field measurements. These errors arise from inaccurate representation of the background field direction in the body frame, and can amplify small orientation errors into noticeable calibration residuals. Using realistic sensor models and flight trajectories, we evaluate different sensor configurations for magnetic calibration and assess the use of onboard Inertial Navigation Systems (INS) as an independent attitude reference to enable stable compensation. Our results suggest that quantum vector magnetometers like NV sensors are not sufficient to solve the attitude bottleneck for airborne vector magnetic calibration. However, as a single sensor capable of providing both absolute field and directional measurements, they may offer benefits regarding sensor placement, synchronization, and scalar calibration accuracy.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper claims that scalar calibration models for airborne magnetometry are robust to misalignment, while vector calibration models are intrinsically first-order sensitive to attitude errors arising from inaccurate body-frame representation of the background field direction, irrespective of magnetic measurement accuracy. This is supported by a rigorous theoretical analysis, followed by simulations using realistic sensor models (including quantum NV vector magnetometers and classical fluxgates) and flight trajectories to evaluate sensor configurations and the role of onboard INS as an attitude reference. The results indicate that quantum vector magnetometers do not resolve the attitude bottleneck for vector calibration but may provide benefits in sensor placement and scalar accuracy.
Significance. If the first-order sensitivity result is confirmed, the work would meaningfully inform hardware choices and calibration strategies for airborne geophysical exploration and magnetic anomaly navigation, highlighting limitations of vector sensors without precise attitude data. The timely focus on NV-center quantum magnetometers and the use of realistic trajectory simulations add practical value, though the absence of explicit derivations limits immediate impact.
major comments (2)
- [Theoretical Analysis] Theoretical Analysis section: The central claim that vector models are intrinsically first-order sensitive to attitude errors requires an explicit first-order expansion of the calibration residual (including hard/soft iron, scale factors, non-orthogonality, and projection onto the measured vector) to demonstrate that the leading term in δθ survives without cancellation by calibration parameters or field projection; the abstract isolates this as the source of error amplification, but without the linearized equations it is unclear if the O(δθ) term is load-bearing or reduces to O(δθ²).
- [Simulation results] Simulation results (flight trajectories and sensor models): The evaluation of first-order sensitivity and configuration comparisons relies on post-hoc model assumptions for platform dynamics and sensor correlations; if these omit key interactions between attitude errors and vector residuals, the claimed amplification may not hold, undermining the conclusion that NV sensors are insufficient to solve the attitude bottleneck.
minor comments (2)
- [Abstract] The abstract and introduction could more clearly distinguish the scalar approximation of the platform field from the full vector model to aid readers unfamiliar with airborne magnetometry conventions.
- [Introduction] Notation for body-frame vs. navigation-frame fields and attitude errors should be defined consistently at first use to improve readability of the theoretical claims.
Simulated Author's Rebuttal
We thank the referee for the constructive and detailed review of our manuscript. The comments highlight important areas for improving the rigor of the theoretical derivation and the transparency of the simulation assumptions. We address each major comment below and will incorporate revisions to strengthen the presentation of our results on scalar versus vector calibration robustness.
read point-by-point responses
-
Referee: Theoretical Analysis section: The central claim that vector models are intrinsically first-order sensitive to attitude errors requires an explicit first-order expansion of the calibration residual (including hard/soft iron, scale factors, non-orthogonality, and projection onto the measured vector) to demonstrate that the leading term in δθ survives without cancellation by calibration parameters or field projection; the abstract isolates this as the source of error amplification, but without the linearized equations it is unclear if the O(δθ) term is load-bearing or reduces to O(δθ²).
Authors: We agree that an explicit first-order expansion is required to fully substantiate the claim. The manuscript's theoretical analysis derives the sensitivity from the structure of the vector calibration equations, where attitude errors enter through the body-frame representation of the background field direction. However, we acknowledge that the linearized form was not shown in full detail. In the revised manuscript, we will add a dedicated derivation subsection presenting the first-order Taylor expansion of the calibration residual. This will explicitly include terms for hard/soft iron, scale factors, non-orthogonality, and vector projection, demonstrating that the leading O(δθ) contribution from the field-direction mismatch does not cancel and remains load-bearing. revision: yes
-
Referee: Simulation results (flight trajectories and sensor models): The evaluation of first-order sensitivity and configuration comparisons relies on post-hoc model assumptions for platform dynamics and sensor correlations; if these omit key interactions between attitude errors and vector residuals, the claimed amplification may not hold, undermining the conclusion that NV sensors are insufficient to solve the attitude bottleneck.
Authors: The simulations are constructed using established sensor models and trajectory generators drawn from the airborne magnetometry literature, with explicit incorporation of attitude-error coupling into the vector residuals via the body-frame field transformation. To address the concern, we will expand the relevant section with a more complete description of the model equations, including the precise interaction terms between δθ and the calibration residuals, and add supplementary analyses that vary platform dynamics and correlation assumptions. These additions will confirm that the first-order amplification persists and that NV vector measurements alone do not resolve the attitude limitation. revision: partial
Circularity Check
Theoretical derivation of first-order attitude sensitivity is self-contained from coordinate transformations and error models.
full rationale
The paper derives the claim that vector calibration models are intrinsically first-order sensitive to attitude errors from first-principles analysis of body-frame field representation and misalignment effects, without reducing the result to a fitted parameter, self-citation chain, or input defined by the output itself. Simulations rely on independent realistic sensor models and trajectories rather than using the target residuals to define the sensitivity. No load-bearing step matches the enumerated circularity patterns; the analysis remains externally falsifiable via the stated assumptions on sensor and platform dynamics.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Standard error models for OPM, fluxgate, and NV magnetometers plus platform magnetic interference are accurate representations of real hardware.
Reference graph
Works this paper leans on
-
[1]
Magnetic Navigation on an F-16 Aircraft Using Online Calibration,
A. J. Canciani, “Magnetic Navigation on an F-16 Aircraft Using Online Calibration,”IEEE Transactions on Aerospace and Electronic Systems, vol. 58, pp. 420–434, Feb. 2022
2022
-
[2]
M. Muradoglu, M. T. Johnsson, N. M. Wilson, Y . Cohen, D. Shin, T. Navickas, T. Pyragius, D. Thompson, S. I. Moore, M. T. Rahman, A. Walker, I. Dutta, S. Bijjahalli, J. Berlocher, M. R. Hush, R. P. Ander- son, S. S. Szigeti, and M. J. Biercuk, “Quantum-assured magnetic nav- igation achieves positioning accuracy better than a strategic-grade INS in airborn...
-
[3]
Optically pumped magnetometers: From quantum origins to multi-channel mag- netoencephalography,
T. M. Tierney, N. Holmes, S. Mellor, J. D. L ´opez, G. Roberts, R. M. Hill, E. Boto, J. Leggett, V . Shah, M. J. Brookes,et al., “Optically pumped magnetometers: From quantum origins to multi-channel mag- netoencephalography,”NeuroImage, vol. 199, pp. 598–608, 2019
2019
-
[4]
Magnetoencephalography with optically pumped magne- tometers (opm-meg): the next generation of functional neuroimaging,
M. J. Brookes, J. Leggett, M. Rea, R. M. Hill, N. Holmes, E. Boto, and R. Bowtell, “Magnetoencephalography with optically pumped magne- tometers (opm-meg): the next generation of functional neuroimaging,” Trends in neurosciences, vol. 45, no. 8, pp. 621–634, 2022
2022
-
[5]
Diamond magnetometer enhanced by ferrite flux concentrators,
I. Fescenko, A. Jarmola, I. Savukov, P. Kehayias, J. Smits, J. Damron, N. Ristoff, N. Mosavian, and V . M. Acosta, “Diamond magnetometer enhanced by ferrite flux concentrators,”Physical Review Research, vol. 2, p. 023394, June 2020. arXiv:1911.05070 [physics]
-
[6]
Integrated and Portable Magnetometer Based on Nitrogen-Vacancy Ensembles in Diamond,
F. M. St ¨urner, A. Brenneis, T. Buck, J. Kassel, R. R ¨olver, T. Fuchs, A. Savitsky, D. Suter, J. Grimmel, S. Hengesbach, M. F ¨ortsch, K. Naka- mura, H. Sumiya, S. Onoda, J. Isoya, and F. Jelezko, “Integrated and Portable Magnetometer Based on Nitrogen-Vacancy Ensembles in Diamond,”Advanced Quantum Technologies, vol. 4, p. 2000111, Apr. 2021
2021
-
[7]
Who Let the Diamonds Out?,
V . Halde, O. Bernard, M. Brochu, L. Dufresne, N. Fleury, K. Johnson, B. Moffett, and D. Roy-Guay, “Who Let the Diamonds Out?,” 2025. Version Number: 2
2025
-
[8]
L. Childress, V . Halde, K. Johnson, A. Lowther, D. Roy-Guay, R. Ruhlmann, and A. Solyom, “Bias-field-free operation of nitrogen- vacancy ensembles in diamond for accurate vector magnetometry,”PRX Quantum, vol. 6, p. 040364, Dec. 2025. arXiv:2505.24574 [quant-ph]
-
[9]
QDsiM: A Noise-Aware Simulation Toolkit for Quantum Diamond Microscope,
S. Pandey, A. Magapu, P. Anand, A. Khandelwal, and M. G. Chandra, “QDsiM: A Noise-Aware Simulation Toolkit for Quantum Diamond Microscope,” Jan. 2026. arXiv:2601.11649 [quant-ph]
-
[10]
Distributed Quantum Magnetic Sensing for Infrastructure-free Geo-localization,
T. Le, S. Guo, and J. Liu, “Distributed Quantum Magnetic Sensing for Infrastructure-free Geo-localization,” Dec. 2025. arXiv:2512.11300 [quant-ph]
-
[11]
An Analysis of the Benefits and Difficulties of Aerial Magnetic Vector Navigation,
A. J. Canciani and C. J. Brennan, “An Analysis of the Benefits and Difficulties of Aerial Magnetic Vector Navigation,”IEEE Transactions on Aerospace and Electronic Systems, vol. 56, pp. 4161–4176, Dec. 2020
2020
-
[12]
Imaging current flow and injection in scalable graphene devices through nv-magnetometry,
K. Dockx, M. Buscema, S. Kumar, T. van Ree, A. Mohtashami, L. van Dooren, G. Bulgarini, R. van Rijn, C. I. Osorio, and T. van der Sar, “Imaging current flow and injection in scalable graphene devices through nv-magnetometry,” 2025
2025
-
[13]
Quantum sensors for enhanced positioning and navigation: a comprehensive review,
H. Li, S. Zaminpardaz, A. Kealy, A. D. Greentree, E. Rubinov, B. Gib- son, and S. Choy, “Quantum sensors for enhanced positioning and navigation: a comprehensive review,”GPS Solutions, vol. 30, p. 62, Feb. 2026
2026
-
[14]
Derivation and Ex- tensions of the Tolles-Lawson Model for Aeromagnetic Compensation,
A. R. Gnadt, A. B. Wollaber, and A. P. Nielsen, “Derivation and Ex- tensions of the Tolles-Lawson Model for Aeromagnetic Compensation,” Dec. 2022. arXiv:2212.09899
-
[15]
Identification and Evaluation of Magnetic-Field Sources of Magnetic Airborne Detector Equipped Aircraft,
P. Leliak, “Identification and Evaluation of Magnetic-Field Sources of Magnetic Airborne Detector Equipped Aircraft,”IRE Transactions on Aeronautical and Navigational Electronics, vol. ANE-8, pp. 95–105, Sept. 1961
1961
-
[16]
A Modified Tolles–Lawson Model Robust to the Errors of the Three-Axis Strapdown Magnetometer,
Q. Han, Z. Dou, X. Tong, X. Peng, and H. Guo, “A Modified Tolles–Lawson Model Robust to the Errors of the Three-Axis Strapdown Magnetometer,”IEEE Geoscience and Remote Sensing Letters, vol. 14, pp. 334–338, Mar. 2017
2017
-
[17]
Applications of quantum sensing to aerial magnetic navigation,
M. Juki ´c, L. Visseren, A. Vijn, and R. Tan, “Applications of quantum sensing to aerial magnetic navigation,” inQuantum Technologies for Defence and Security, vol. 13202, pp. 98–116, SPIE, 2024
2024
-
[18]
A. R. Gnadt,Advanced Aeromagnetic Compensation Models for Air- borne Magnetic Anomaly Navigation. PhD thesis, Massachusetts Insti- tute of Technology, 2022
2022
-
[19]
M. Moradi, Z.-M. Zhai, A. Nielsen, and Y .-C. Lai, “Random forests for detecting weak signals and extracting physical information: a case study of magnetic navigation,” Feb. 2024. arXiv:2402.14131 [physics]
-
[20]
An Aeromagnetic Compensation Algorithm Based on a Temporal Convolutional Network,
H. Wang and B. Zuo, “An Aeromagnetic Compensation Algorithm Based on a Temporal Convolutional Network,”Applied Sciences, vol. 15, p. 3105, Mar. 2025
2025
-
[21]
Shipboard calibration of a diamond nitrogen vacancy magnetic field sensor,
P. Frontera, S. Alessandrini, and J. Stetson, “Shipboard calibration of a diamond nitrogen vacancy magnetic field sensor,” in2018 IEEE/ION Position, Location and Navigation Symposium (PLANS), (Monterey, CA), pp. 497–504, IEEE, Apr. 2018
2018
-
[22]
Maritime magnetic anomaly mapping with a diamond nitrogen vacancy sensor,
T. Fleig and P. Frontera, “Maritime magnetic anomaly mapping with a diamond nitrogen vacancy sensor,” in2018 IEEE/ION Position, Location and Navigation Symposium (PLANS), (Monterey, CA), pp. 1107–1112, IEEE, Apr. 2018
2018
-
[23]
Star trackers for attitude determination,
C. Liebe, “Star trackers for attitude determination,”IEEE Aerospace and Electronic Systems Magazine, vol. 10, no. 6, pp. 10–16, 1995
1995
-
[24]
INS- Based Aeromagnetic Compensation: Sliding Window Optimization for Enhanced Accuracy,
X. Liu, J. Liu, J. Zhang, W. Zhu, Q. Zhang, and G. Fang, “INS- Based Aeromagnetic Compensation: Sliding Window Optimization for Enhanced Accuracy,”IEEE Access, vol. 13, pp. 138589–138600, 2025
2025
-
[25]
International Geomagnetic Reference Field: the thir- teenth generation,
P. Alkenet al., “International Geomagnetic Reference Field: the thir- teenth generation,”Earth, Planets and Space, vol. 73, p. 49, Feb. 2021
2021
-
[26]
On the Road with a Diamond Magnetometer,
S. M. Graham, A. J. Newman, C. J. Stephen, A. M. Edmonds, D. J. Twitchen, M. L. Markham, and G. W. Morley, “On the Road with a Diamond Magnetometer,” Jan. 2024. arXiv:2401.16090 [cond-mat, physics:physics, physics:quant-ph]
-
[27]
E. Abe and K. Sasaki, “Tutorial: Magnetic resonance with nitrogen- vacancy centers in diamond—microwave engineering, materials science, and magnetometry,”Journal of Applied Physics, vol. 123, p. 161101, Apr. 2018. arXiv:1802.07857 [cond-mat]
-
[28]
Applications of quantum sensing to aerial magnetic navigation,
M. Jukic, L. Visseren, A. Vijn, and R. Tan, “Applications of quantum sensing to aerial magnetic navigation,” inQuantum Technologies for Defence and Security(G. Sorelli, S. Ducci, and S. Schwartz, eds.), (Edinburgh, United Kingdom), p. 39, SPIE, Nov. 2024
2024
-
[29]
Airborne magnetic anomaly navigation with neural-network- augmented online calibration,
A. Hager, S. Nebendahl, A. Klushyn, J. Krauser, T. H. Bryne, and T. A. Johansen, “Airborne magnetic anomaly navigation with neural-network- augmented online calibration,” 2026
2026
-
[30]
Long-Term Stability Assessment of Quantum Diamond Magnetometers in Low Earth Orbit,
Y . Beerden, R. Vandebosch, A. Ermakova, M. Nesladek, and J. Hruby, “Long-Term Stability Assessment of Quantum Diamond Magnetometers in Low Earth Orbit,” inIGARSS 2024 - 2024 IEEE International Geoscience and Remote Sensing Symposium, (Athens, Greece), pp. 465– 468, IEEE, July 2024
2024
-
[31]
Micrometer-scale magnetic imaging of geological samples using a quantum diamond microscope,
D. R. Glenn, R. R. Fu, P. Kehayias, D. Le Sage, E. A. Lima, B. P. Weiss, and R. L. Walsworth, “Micrometer-scale magnetic imaging of geological samples using a quantum diamond microscope,”Geochem- istry, Geophysics, Geosystems, vol. 18, no. 8, pp. 3254–3267, 2017
2017
-
[32]
The hunt for mineral resources with quantum magnetometers,
R. Stolz, M. Schiffler, M. Becken, M. Schneider, and G. Chubak, “The hunt for mineral resources with quantum magnetometers,”tm - Technisches Messen, vol. 91, no. 1, pp. 41–50, 2024
2024
-
[33]
Toward quantitative bio-sensing with nitrogen–vacancy center in diamond,
T. Zhang, G. Pramanik, K. Zhang, M. Gulka, L. Wang, J. Jing, F. Xu, Z. Li, Q. Wei, P. Cigler, and Z. Chu, “Toward quantitative bio-sensing with nitrogen–vacancy center in diamond,”ACS Sensors, vol. 6, no. 6, pp. 2077–2107, 2021. PMID: 34038091
2077
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.