pith. sign in

arxiv: 2605.30213 · v1 · pith:C56IVJWQnew · submitted 2026-05-28 · 💻 cs.LG

Faithful Embeddings of Irregular and Asynchronous Data for Online Log-NCDEs

Benjamin Walker , Alexandre Bloch , Lingyi Yang , Sam Morley , Terry Lyons This is my paper

Pith reviewed 2026-06-29 08:56 UTC · model grok-4.3

classification 💻 cs.LG
keywords Log-NCDEirregular time seriesasynchronous datacontinuous injective embeddinglog-signatureneural controlled differential equationsonline computationuniversality transfer
0
0 comments X

The pith

Compact-set universality transfers from model input space to irregular data space if the embedding is continuous and injective.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that for continuous-time models, one can skip the usual step of interpolating or imputing irregular observations to create a continuous path. Instead, a mapping from the raw data points to the model's input space needs only to be continuous and injective for universality properties to carry over under mild conditions. The authors construct exactly such a mapping for Log-NCDEs by treating observations as increments and directly building log-signatures over chosen intervals using a rectilinear control path. This produces interval summaries that support online evaluation and remain stable even when observations arrive asynchronously or sparsely. A sympathetic reader cares because real-world time series rarely arrive on a regular grid, and removing the reconstruction step reduces both modeling choices and sensitivity to them.

Core claim

Under mild conditions, compact-set universality on the model input space transfers to the data space whenever the embedding from data to input is continuous and injective. Guided by this result, and building on the rectilinear control path for Neural Controlled Differential Equations (NCDEs), the authors introduce a continuous and injective embedding for Log-NCDEs that records observations as increments and composes them over arbitrary query intervals to directly form log-signatures.

What carries the argument

Continuous and injective embedding from observed data to model input space, implemented via log-signatures of increments under rectilinear control paths.

If this is right

  • Log-NCDEs become insensitive to the particular choice of interpolation or imputation used to reconstruct paths from discrete observations.
  • The model can produce interval-level summaries directly from raw increments without first filling in values between observation times.
  • Computation can proceed online because log-signatures are composed incrementally over successive query intervals.
  • The representation remains accurate and robust when tested on both synthetic controlled dynamics and real-world time-series datasets with irregular and sparse sampling.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same transfer principle could be used to design faithful embeddings for other continuous-time architectures that currently rely on path reconstruction.
  • In domains with streaming sensor data, the online property might allow models to update predictions as soon as new observations arrive without re-processing the entire history.
  • One could explore whether relaxing injectivity while preserving continuity still yields useful approximation guarantees in practice.

Load-bearing premise

The mapping that turns sequences of discrete observations into the model's continuous input path must be both continuous and injective.

What would settle it

A counter-example where a continuous but non-injective embedding is used and the resulting Log-NCDE fails to approximate functions that the underlying model can approximate on the input space, or an empirical test on irregular data where the proposed embedding performs no better than standard interpolation methods.

Figures

Figures reproduced from arXiv: 2605.30213 by Alexandre Bloch, Benjamin Walker, Lingyi Yang, Sam Morley, Terry Lyons.

Figure 1
Figure 1. Figure 1: Faithful representations of irregular data. Imputation and interpolation complete the data before modelling, introducing values or dynamics that may be unrealistic. Our approach records the observed stream as time-stamped increments and groups them into interval summaries, giving a faithful (continuous and injective) representation of the discrete and irregular data. GRU-ODE-Bayes [Rubanova et al., 2019, D… view at source ↗
Figure 2
Figure 2. Figure 2: Schematic of a Log-SLiCE. Irregular observations are first summarised by interval log￾signatures Φα,β = log(S(X)[α,β]). The iterated Lie brackets of the matrices Ai θ , represented by A¯ θ, map each log-signature to a Log-ODE flow. These interval-wise hidden-state updates are composed using a parallel associative scan. The resulting hidden states hri are decoded to produce estimates of the output values yr… view at source ↗
Figure 3
Figure 3. Figure 3: Constructing interval log-signatures from irregular observations. Each observation is converted into a Lie-algebra increment containing value increments and observation-count increments. Continuously observed variables, such as time, contribute signature factors over gaps between observations. Event and gap factors are composed over each query interval using a parallel associative scan with the tensor prod… view at source ↗
Figure 4
Figure 4. Figure 4: Heatmap showing how well trained sinusoid models generalise across data sampling regimes. The mean and standard deviation are computed over 5 random seeds. 101 Number of intervals 10−6 10−5 Test MSE Test Error vs Number of Intervals Level 1 Level 2 [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Average test MSE against number of query intervals used for linear system driven by Brownian motion over 5 seeds. As the system is driven by area information, we see a much lower MSE for when using level 2 log-signatures as we would expect, especially for fewer query intervals. In [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Observation patterns and signal example for synchronous regular regime. 0.75 0.50 0.25 0.00 0.25 0.50 0.75 channel 0 true signal observations query targets 0 2 4 6 8 10 time 1.0 0.5 0.0 0.5 1.0 channel 1 true signal observations query targets (a) True underlying signal, observation, and query targets. 0 2 4 6 8 10 time ch 0 ch 1 Observation pattern (b) Observation pattern of the two channels [PITH_FULL_IM… view at source ↗
Figure 7
Figure 7. Figure 7: Observation patterns and signal example for synchronous irregular regime. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Observation patterns and signal example for asynchronous irregular regime. For each sequence, we also store the first observed value in each channel. This vector is passed to the input￾dependent initialisation of the controlled differential equation after a tanh transformation. This gives the model access to the starting observed level of each channel while the subsequent path records increments from one o… view at source ↗
Figure 9
Figure 9. Figure 9: Observation patterns and signal example for asynchronous sparse regime. 0 20 40 60 80 100 Epoch 0.00 0.05 0.10 0.15 0.20 0.25 MSE Mean train loss sync_regular sync_irregular async_irregular async_sparse 0 20 40 60 80 100 Epoch 0.00 0.02 0.04 0.06 0.08 0.10 0.12 MSE Mean test loss sync_regular sync_irregular async_irregular async_sparse [PITH_FULL_IMAGE:figures/full_fig_p026_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Training/test loss over epochs for the synthetic coupled task. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Prediction examples for model trained on synchronous regular data. After training one model on each regime, we evaluate each trained model on test sets from all four regimes. This gives the cross-regime matrix seen in [PITH_FULL_IMAGE:figures/full_fig_p027_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Prediction examples for model trained on synchronous irregular data. 0 5 10 −0.5 0.0 0.5 sample 0 channel 0 true pred 0 5 10 −1 0 1 channel 1 2.5 5.0 7.5 10.0 −0.5 0.0 0.5 sample 1 2.5 5.0 7.5 10.0 −0.5 0.0 0.5 [PITH_FULL_IMAGE:figures/full_fig_p028_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Prediction examples for model trained on asynchronous irregular data. 28 [PITH_FULL_IMAGE:figures/full_fig_p028_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Prediction examples for model trained on asynchronous sparse data. The two degree-one vector fields are fixed linear maps V1 = 0.15  −0.5 −1.0 1.0 −0.5  , V2 = 0.15  −0.2 0.8 0.3 −0.7  . The use of non-commuting matrices here ensures that second-order information cannot be removed without changing the solution. Although these Brownian streams are four-dimensional, our target system here is driven only… view at source ↗
Figure 15
Figure 15. Figure 15: Prediction of a linear system driven by Brownian motion using a model trained with only increments (level 1) data. Three examples are taken from the test set with 16 intervals. repeat the experiment over five random seeds. Like the sinusoid task, the training objective is the masked mean squared error over non-padded intervals [PITH_FULL_IMAGE:figures/full_fig_p030_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Prediction of a linear system driven by Brownian motion using a model trained with both increments and area (level 2) data. Three examples are taken from the test set with 16 intervals. C.3 UEA experiments We follow the UEA multivariate time series classification archive (UEA-MTSCA) [Bagnall et al., 2018] protocol of Walker et al. [2024] and the SLiCE protocol of Walker et al. [2025]. The benchmark consis… view at source ↗
read the original abstract

Continuous-time models are a natural choice for irregular and asynchronous data. A central design choice is how to embed discrete observations into continuous time. Interpolation- and imputation-based embeddings reconstruct a continuous observation path, making the model sensitive to the choice of reconstruction. We show that this reconstruction step is unnecessary; under mild conditions, compact-set universality on the model input space transfers to the data space whenever the embedding from data to input is continuous and injective. Guided by this result, and building on the rectilinear control path for Neural Controlled Differential Equations (NCDEs), we introduce a continuous and injective embedding for Log-NCDEs, a universal class of continuous-time models. Our approach records observations as increments and composes them over arbitrary query intervals to directly form log-signatures. This provides interval-level summaries without first interpolating the observed variables, while supporting online computation. Experiments on synthetic controlled dynamics and real-world time-series datasets show that the representation is accurate, efficient, and robust to irregular, asynchronous, and sparse observations.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 1 minor

Summary. The paper claims that under mild conditions, compact-set universality on the model input space transfers to the data space whenever the embedding from data to input is continuous and injective. Guided by this, the authors introduce a continuous injective embedding for Log-NCDEs based on rectilinear control paths and log-signatures of observation increments, enabling online interval-level summaries without interpolation. Experiments on synthetic controlled dynamics and real-world time-series datasets are said to show accuracy, efficiency, and robustness to irregular, asynchronous, and sparse observations.

Significance. If the universality transfer result holds with the stated conditions, the work would supply a principled alternative to interpolation-based embeddings in continuous-time models, with direct benefits for online processing of irregular data. The focus on log-signature increments as interval summaries is a concrete technical contribution.

major comments (1)
  1. [Abstract] Abstract (central claim): the statement that compact-set universality transfers whenever the embedding is continuous and injective (under mild conditions) requires the mild conditions to guarantee that the inverse is continuous on the image of the embedding. Without a continuous inverse, functions of the form h ∘ φ need not be dense in C(K) for compact K in data space. The manuscript should state the mild conditions explicitly and verify they yield a topological embedding.
minor comments (1)
  1. The abstract references experimental results on synthetic and real-world datasets but provides no quantitative metrics, baselines, or controls; adding one or two key performance numbers would improve readability.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive comment on the central claim. We agree that the abstract requires clarification on the precise conditions and will revise accordingly to ensure the statement is accurate.

read point-by-point responses

  1. Referee: [Abstract] Abstract (central claim): the statement that compact-set universality on the model input space transfers to the data space whenever the embedding from data to input is continuous and injective (under mild conditions) requires the mild conditions to guarantee that the inverse is continuous on the image of the embedding. Without a continuous inverse, functions of the form h ∘ φ need not be dense in C(K) for compact K in data space. The manuscript should state the mild conditions explicitly and verify they yield a topological embedding.

    Authors: We appreciate this observation. The paper's main theorem (Section 3) states the universality transfer result under the assumption that the embedding is a topological embedding, which explicitly includes continuity of the inverse on the image. The phrase 'mild conditions' in the abstract is intended to refer to those ensuring the embedding property, but we acknowledge the summary is imprecise. In revision we will (i) restate the abstract to specify that the embedding must be a topological embedding and (ii) add a short remark or appendix verification confirming that the rectilinear log-signature construction satisfies the embedding conditions on the relevant compact sets of observation sequences. revision: yes

Circularity Check

0 steps flagged

No circularity: universality transfer is a general topological claim independent of paper inputs

full rationale

The paper's core claim is that compact-set universality transfers from model input space to data space whenever the embedding φ: data space → input space is continuous and injective (under mild conditions). This is stated as a general property of such maps rather than a derivation that reduces to its own fitted quantities or definitions by construction. No equations, parameters, or predictions in the provided abstract or description involve fitting a quantity to data and then renaming a related output as a 'prediction.' The new Log-NCDE embedding is motivated by the result but does not enter the statement of the transfer theorem itself. No self-citation load-bearing steps, uniqueness theorems imported from prior author work, or ansatzes smuggled via citation are present. The derivation chain is therefore self-contained against external mathematical benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Review performed on abstract only; full manuscript text referenced but not supplied in query, preventing exhaustive extraction of parameters or background assumptions.

axioms (1)
  • domain assumption Compact-set universality transfers to data space under continuous and injective embedding
    Central sufficient condition stated in abstract for the main theoretical result

pith-pipeline@v0.9.1-grok · 5707 in / 1061 out tokens · 22765 ms · 2026-06-29T08:56:25.541997+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

3 extracted references · 2 canonical work pages

  1. [1]

    Long short-term memory

    ISSN 0899-7667. doi: 10.1162/neco.1997.9.8.1735. URL https://doi.org/10.1162/neco.1997. 9.8.1735. Christian Holberg and Cristopher Salvi. Exact gradients for stochastic spiking neural networks driven by rough signals. InProceedings of the 38th Conference on Neural Information Processing Systems (NeurIPS), 2024. Blanka Horvath, Maud Lemercier, Chong Liu, T...

    work page doi:10.1162/neco.1997.9.8.1735 1997

  2. [2]

    URLhttps://doi.org/10.1137/23M161077X

    doi: 10.1137/23M161077X. URLhttps://doi.org/10.1137/23M161077X. Sheo Yon Jhin, Jaehoon Lee, Minju Jo, Seungji Kook, Jinsung Jeon, Jihyeon Hyeong, Jayoung Kim, and Noseong Park. EXIT: Extrapolation and interpolation-based neural controlled differential equations for time-series classification and forecasting. InProceedings of the ACM Web Conference 2022, W...

    work page doi:10.1137/23m161077x 2022

  3. [3]

    The same partition is used for all samples within a run

    After rescaling by2048, this gives a random partition 0 =q 0 < q1 <· · ·< q m = 1. The same partition is used for all samples within a run. The target associated with interval [qk, qk+1) is the simulated state at the right endpoint, yk =X(q k+1)∈R 2. Although the model predicts endpoint values on a query grid that is separate from the fine input grid, the...

    2048