Orthogonal Discrepancy Kernels for Learning with Partial Physics

Lawrence Bull; Swapnil Manna; Timothy J. Rogers

arxiv: 2606.21199 · v1 · pith:SNPP34QHnew · submitted 2026-06-19 · 📊 stat.ML · cs.LG· eess.SP

Orthogonal Discrepancy Kernels for Learning with Partial Physics

Swapnil Manna , Timothy J. Rogers , Lawrence Bull This is my paper

Pith reviewed 2026-06-26 12:48 UTC · model grok-4.3

classification 📊 stat.ML cs.LGeess.SP

keywords orthogonal Gaussian processesdiscrepancy learningsystem identificationsemi-parametric modelsincomplete physicsnonlinear systemskernel methods

0 comments

The pith

Orthogonal Gaussian process regression decouples discrepancy functions from physics-based components in nonlinear system identification.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces a semi-parametric framework for nonlinear system identification that decouples discrepancy functions from physics-based components. Orthogonal Gaussian process regression balances sparse parameter selection in the white box with discrepancy learning in the black box. This produces interpretable models from incomplete physics. A sympathetic reader would care because it combines known physical structure with data-driven corrections while preserving the ability to inspect which parameters come from physics.

Core claim

Orthogonal Gaussian process regression balances sparse parameter selection (the white box) with discrepancy learning (the black box) to produce interpretable models from incomplete physics by decoupling discrepancy functions from physics-based components.

What carries the argument

Orthogonal discrepancy kernels, which enforce orthogonality between the physics-based kernel and the discrepancy kernel inside Gaussian process regression to separate the two components.

If this is right

Sparse selection identifies which physical parameters remain active when the model is only partially known.
Discrepancy learning captures missing dynamics without altering the recovered physics terms.
The resulting models remain interpretable because the physics and discrepancy parts stay separated by construction.
The framework applies directly to engineering systems where some governing equations are known but others are not.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The orthogonality constraint may reduce the need for manual kernel design when extending the method to new physical domains.
Similar decoupling could be tested in other kernel-based or neural hybrid models that mix known equations with learned corrections.
Real-time control applications might benefit if the separation allows faster updates to only the discrepancy part.

Load-bearing premise

Discrepancy functions can be effectively decoupled from physics-based components through orthogonality in Gaussian process regression without introducing significant bias or loss of accuracy in the combined model.

What would settle it

A numerical test on a system with known full physics plus an added discrepancy term, checking whether the orthogonal method recovers the exact physics parameters and matches the accuracy of a non-orthogonal joint model.

Figures

Figures reproduced from arXiv: 2606.21199 by Lawrence Bull, Swapnil Manna, Timothy J. Rogers.

**Figure 1.** Figure 1: The acceleration response y¨ of a numerically simulated system with cubic nonlinearity. Showing oscillations through time t. We apply both, (i) the standard squared-exponential kernel (Non-Orthogonal) GP with SINDy and (ii) the Orthogonal kernel GP with SINDy to our setup and present the results for both complete and incomplete dictionaries. System Identification using Complete Dictionary: Our dictionary … view at source ↗

**Figure 2.** Figure 2: System Identification using Non-Orthogonal GP Grey Box Modeling for Complete Dictionary: (left panel) parameters identified via sparse regression using the white box; (right panel) the black-box discrepancy model, which greedily learns the cubic spring basis function (note the different scales of the response) [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Figure 3: System Identification using Orthogonal GP Grey Box Modeling for Complete Dictionary: (left panel) parameters identified via sparse regression using the white box; (right panel) the black-box discrepancy model (note the low scale of the response) [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

**Figure 4.** Figure 4: and [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗

**Figure 5.** Figure 5: System Identification using Non-Orthogonal GP Grey Box Modeling for Partial Dictionary: (left panel) parameters identified via sparse regression using the white box; (right panel) the black-box discrepancy model [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗

read the original abstract

We introduce a semi-parametric framework for nonlinear system identification, which decouples discrepancy functions from physics-based components. Orthogonal Gaussian process regression balances sparse parameter selection (the white box) with discrepancy learning (the black box) to produce interpretable models from incomplete physics.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper uses orthogonal GPs to separate physics parameters from discrepancy terms in nonlinear system ID, but the advance looks incremental and the abstract gives little to judge the actual performance.

read the letter

The main point is a semi-parametric GP method that applies orthogonality to keep sparse physics parameters distinct from a learned discrepancy function. This targets nonlinear system identification when the physics model is incomplete.

The construction follows standard identifiability tricks for semi-parametric models, and the stress-test note is correct that nothing in the abstract creates an obvious contradiction or circularity. The framing around interpretability for engineering use is reasonable and matches what people in this area often want.

What is actually new is the explicit kernel-level combination for this balance, though it reads as an extension of existing discrepancy GP work rather than a sharp departure. The paper does a clean job stating the goal of balancing white-box and black-box parts without overclaiming broad impact.

The soft spots are the missing details. No derivations or experiments are visible, so it is impossible to check whether the orthogonality actually limits bias or loss of accuracy in the combined model. The weakest assumption in the reader's note lands: decoupling through orthogonality is plausible but untested here. Novelty is also hard to pin down without the citation comparison.

This is for people already working on Gaussian processes for dynamical systems or physics-informed identification. A reader who needs a practical tweak for partial physics models might pick up a useful idea, but it is not essential reading for the wider field.

It deserves a serious referee because the idea is coherent and the application is concrete. The evidence is thin from what is supplied, but the internal logic holds and the topic is worth proper review.

Referee Report

1 major / 0 minor

Summary. The paper introduces a semi-parametric framework for nonlinear system identification that decouples discrepancy functions from physics-based components via orthogonal Gaussian process regression. This is claimed to balance sparse parameter selection in the white-box physics model with black-box discrepancy learning, yielding interpretable models from incomplete physics.

Significance. The proposed use of orthogonality to enforce identifiability between parametric physics terms and nonparametric discrepancy is a standard device in semi-parametric statistics and could, if rigorously developed with supporting derivations and experiments, offer a principled way to hybridize white- and black-box modeling. No such development, theorems, or empirical results are visible in the supplied manuscript, so significance cannot be assessed.

major comments (1)

No derivations, theorems, algorithms, or experimental sections are present. The central claim that orthogonality 'balances' the two components and produces interpretable models cannot be evaluated without the technical content promised by the abstract.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their review. The major comment correctly identifies that the supplied manuscript contains only the abstract and lacks the promised technical content. We address this point below and will revise the manuscript to include the missing sections.

read point-by-point responses

Referee: No derivations, theorems, algorithms, or experimental sections are present. The central claim that orthogonality 'balances' the two components and produces interpretable models cannot be evaluated without the technical content promised by the abstract.

Authors: We agree with this assessment. The version of the manuscript under review does not contain derivations of the orthogonal Gaussian process regression, theorems establishing identifiability via orthogonality, an explicit algorithm, or any experimental results. Without these elements the central claims cannot be evaluated. In the revised manuscript we will add: (i) full derivations showing how the orthogonality constraint decouples the parametric physics terms from the nonparametric discrepancy, (ii) theorems on identifiability and consistency, (iii) a complete algorithm description, and (iv) empirical studies on nonlinear system identification benchmarks that quantify the claimed balance between sparse parameter selection and discrepancy learning. revision: yes

Circularity Check

0 steps flagged

No circularity; derivation self-contained against external benchmarks

full rationale

The provided abstract and skeptic analysis contain no equations, theorems, or derivations. The framework is described as using standard orthogonality for identifiability in semi-parametric GP models, with no visible self-definition, fitted-input-as-prediction, or self-citation load-bearing steps. Without access to specific equations in the full text, no reduction to inputs by construction can be exhibited. This is the expected honest non-finding when no load-bearing claims are inspectable.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only; no free parameters, axioms, or invented entities can be extracted or audited from the provided text.

pith-pipeline@v0.9.1-grok · 5561 in / 1034 out tokens · 23425 ms · 2026-06-26T12:48:32.990933+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

19 extracted references · 3 canonical work pages

[1]

Messenger and David M

Weak SINDy for partial differential equations , journal =. 2021 , issn =. doi:10.1016/j.jcp.2021.110525 , url =

work page doi:10.1016/j.jcp.2021.110525 2021
[2]

Sparse identification of nonlinear dynamical systems via reweighted _1 -regularized least squares , volume=

Sparse identification of nonlinear dynamical systems via reweighted l1-regularized least squares , journal =. 2021 , issn =. doi:10.1016/j.cma.2020.113620 , url =

work page doi:10.1016/j.cma.2020.113620 2021
[3]

Mechanical Systems and Signal Processing , volume=

Equation discovery for nonlinear dynamical systems: A Bayesian viewpoint , author=. Mechanical Systems and Signal Processing , volume=. 2021 , publisher=

2021
[4]

Mechanical Systems and Signal Processing , volume=

On spike-and-slab priors for Bayesian equation discovery of nonlinear dynamical systems via sparse linear regression , author=. Mechanical Systems and Signal Processing , volume=. 2021 , publisher=

2021
[5]

Proceedings of the National Academy of Sciences , volume=

Discovering governing equations from data by sparse identification of nonlinear dynamical systems , author=. Proceedings of the National Academy of Sciences , volume=. 2016 , publisher=

2016
[6]

arXiv preprint arXiv:2004.08424 , year=

Pysindy: a python package for the sparse identification of nonlinear dynamics from data , author=. arXiv preprint arXiv:2004.08424 , year=

arXiv 2004
[7]

Proceedings of the Royal Society A , volume=

SINDy-PI: a robust algorithm for parallel implicit sparse identification of nonlinear dynamics , author=. Proceedings of the Royal Society A , volume=. 2020 , publisher=

2020
[8]

Proceedings of the Royal Society A , volume=

Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control , author=. Proceedings of the Royal Society A , volume=. 2022 , publisher=

2022
[9]

Multiscale Modeling & Simulation , volume=

Weak SINDy: Galerkin-based data-driven model selection , author=. Multiscale Modeling & Simulation , volume=. 2021 , publisher=

2021
[10]

Physical Review Research , volume=

SINDy-BVP: Sparse identification of nonlinear dynamics for boundary value problems , author=. Physical Review Research , volume=. 2021 , publisher=

2021
[11]

arXiv preprint arXiv:2408.08062 , year=

BINDy--Bayesian identification of nonlinear dynamics with reversible-jump Markov-chain Monte-Carlo , author=. arXiv preprint arXiv:2408.08062 , year=

arXiv
[12]

Nature Reviews Physics , volume=

Physics-informed machine learning , author=. Nature Reviews Physics , volume=. 2021 , publisher=

2021
[13]

Data-Centric Engineering , volume=

A spectrum of physics-informed Gaussian processes for regression in engineering , author=. Data-Centric Engineering , volume=. 2024 , publisher=

2024
[14]

Proceedings of the Technical Report in Preparation , year=

Automatic relevance determination for neural networks , author=. Proceedings of the Technical Report in Preparation , year=
[15]

, author=

The No-U-Turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. , author=. JMLR , volume=
[16]

Journal of the Royal Statistical Society: Series B (Statistical Methodology) , volume=

Bayesian calibration of computer models , author=. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , volume=. 2001 , publisher=

2001
[17]

2006 , publisher=

Gaussian processes for machine learning , author=. 2006 , publisher=

2006
[18]

Roshan Joseph , journal =

Matthew Plumlee and V. Roshan Joseph , journal =. ORTHOGONAL GAUSSIAN PROCESS MODELS , urldate =
[19]

Rasmussen, Carl Edward and Williams, Christopher K. I. , title =. 2005 , month =. doi:10.7551/mitpress/3206.001.0001 , url =

work page doi:10.7551/mitpress/3206.001.0001 2005

[1] [1]

Messenger and David M

Weak SINDy for partial differential equations , journal =. 2021 , issn =. doi:10.1016/j.jcp.2021.110525 , url =

work page doi:10.1016/j.jcp.2021.110525 2021

[2] [2]

Sparse identification of nonlinear dynamical systems via reweighted _1 -regularized least squares , volume=

Sparse identification of nonlinear dynamical systems via reweighted l1-regularized least squares , journal =. 2021 , issn =. doi:10.1016/j.cma.2020.113620 , url =

work page doi:10.1016/j.cma.2020.113620 2021

[3] [3]

Mechanical Systems and Signal Processing , volume=

Equation discovery for nonlinear dynamical systems: A Bayesian viewpoint , author=. Mechanical Systems and Signal Processing , volume=. 2021 , publisher=

2021

[4] [4]

Mechanical Systems and Signal Processing , volume=

On spike-and-slab priors for Bayesian equation discovery of nonlinear dynamical systems via sparse linear regression , author=. Mechanical Systems and Signal Processing , volume=. 2021 , publisher=

2021

[5] [5]

Proceedings of the National Academy of Sciences , volume=

Discovering governing equations from data by sparse identification of nonlinear dynamical systems , author=. Proceedings of the National Academy of Sciences , volume=. 2016 , publisher=

2016

[6] [6]

arXiv preprint arXiv:2004.08424 , year=

Pysindy: a python package for the sparse identification of nonlinear dynamics from data , author=. arXiv preprint arXiv:2004.08424 , year=

arXiv 2004

[7] [7]

Proceedings of the Royal Society A , volume=

SINDy-PI: a robust algorithm for parallel implicit sparse identification of nonlinear dynamics , author=. Proceedings of the Royal Society A , volume=. 2020 , publisher=

2020

[8] [8]

Proceedings of the Royal Society A , volume=

Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control , author=. Proceedings of the Royal Society A , volume=. 2022 , publisher=

2022

[9] [9]

Multiscale Modeling & Simulation , volume=

Weak SINDy: Galerkin-based data-driven model selection , author=. Multiscale Modeling & Simulation , volume=. 2021 , publisher=

2021

[10] [10]

Physical Review Research , volume=

SINDy-BVP: Sparse identification of nonlinear dynamics for boundary value problems , author=. Physical Review Research , volume=. 2021 , publisher=

2021

[11] [11]

arXiv preprint arXiv:2408.08062 , year=

BINDy--Bayesian identification of nonlinear dynamics with reversible-jump Markov-chain Monte-Carlo , author=. arXiv preprint arXiv:2408.08062 , year=

arXiv

[12] [12]

Nature Reviews Physics , volume=

Physics-informed machine learning , author=. Nature Reviews Physics , volume=. 2021 , publisher=

2021

[13] [13]

Data-Centric Engineering , volume=

A spectrum of physics-informed Gaussian processes for regression in engineering , author=. Data-Centric Engineering , volume=. 2024 , publisher=

2024

[14] [14]

Proceedings of the Technical Report in Preparation , year=

Automatic relevance determination for neural networks , author=. Proceedings of the Technical Report in Preparation , year=

[15] [15]

, author=

The No-U-Turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. , author=. JMLR , volume=

[16] [16]

Journal of the Royal Statistical Society: Series B (Statistical Methodology) , volume=

Bayesian calibration of computer models , author=. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , volume=. 2001 , publisher=

2001

[17] [17]

2006 , publisher=

Gaussian processes for machine learning , author=. 2006 , publisher=

2006

[18] [18]

Roshan Joseph , journal =

Matthew Plumlee and V. Roshan Joseph , journal =. ORTHOGONAL GAUSSIAN PROCESS MODELS , urldate =

[19] [19]

Rasmussen, Carl Edward and Williams, Christopher K. I. , title =. 2005 , month =. doi:10.7551/mitpress/3206.001.0001 , url =

work page doi:10.7551/mitpress/3206.001.0001 2005