Heterogeneous Relational Kernel Learning

Andre T. Nguyen; Edward Raff

REVIEW 4 major objections 5 minor 14 references

Heterogeneous Relational Kernel Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Saving every BIC score during kernel search yields an interpretable embedding for heterogeneous time series.

desk verdict A simple, genuinely novel embedding from BIC histories that works on the chosen experiments, with a thin evaluation and an overclaim about DTW. read the letter →

arxiv 1908.09219 v1 pith:J4CMZJXX submitted 2019-08-24 cs.LG stat.ML

Andre T. Nguyen , Edward Raff This is my paper · ORCID

classification cs.LGstat.ML

keywords relationalkernellearningheterogeneoustimeseriesGaussianprocessesembeddingclusteringanomalydetectionBayesianinformationcriterioninterpretability

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces Heterogeneous Relational Kernel Learning (HRKL), a method for turning an existing relational kernel learning (RKL) search into an embedding for time series that do not share a single global structure. The key move is to keep the Bayesian Information Criterion score of every kernel evaluated on every time series, arrange these scores in a matrix, standardize each row, and treat each row as a vector representation of that time series. Because each dimension of the vector is tied to one interpretable kernel, the embedding has a built-in meaning: two series are similar when the same kernels fit both well and the same kernels fit both poorly. The authors demonstrate that this representation supports clustering, pattern discovery, and anomaly detection on data sets where a shared-kernel assumption fails, at practically the same computational cost as running RKL once.

What carries the argument

The load-bearing object is the BIC history matrix $B$, a $J \times K$ array whose entry $B_{j,k}$ is the Bayesian Information Criterion of a Gaussian process model specified by kernel $k$ and optimized for time series $d_j$. Row-wise standardization, subtracting the row mean and dividing by the row standard deviation, converts each row into the time series embedding. The embedding carries meaning because each column is associated with one kernel from the compositional kernel language, so a high standardized value means that kernel explains that series especially well. The argument runs through this matrix: it converts the bookkeeping of a single model search into a geometric representation, and clustering the rows gives sub-populations without any additional model fits.

What would settle it

Generate time series with the same six structural classes used in the synthetic experiment but vary the length and noise level within each class, then run HRKL; if the clusters separate by length or noise rather than by structural class, the embedding is capturing nuisance factors instead of structure.

Watch

Extended reading notes

Core claim

The paper's central claim is that the BIC history matrix, the intermediate output of an RKL search that previous work discarded, already contains enough information to represent each time series by its response to the whole library of candidate kernels. After each row is standardized to zero mean and unit variance, the row encodes a profile: which kernels explain that series unusually well and which explain it unusually poorly. Clustering these profiles separates time series by shared structure rather than by shared parameter values, and the best-scoring kernel for each cluster gives a human-readable description of the sub-population. The paper further claims this is essentially free, because the matrix is produced by a single RKL run and the subsequent clustering cost is small relative to the cubic cost of fitting the Gaussian process models.

Load-bearing premise

The embedding is only as meaningful as the assumption that a time series's standardized BIC scores against a fixed set of kernels reflect its structure rather than nuisance properties like length, noise level, or amplitude; the paper does not directly test this.

Editorial extensions

If this is right

HRKL can identify sub-populations in heterogeneous time series and produce a separate interpretable kernel description for each, a task that relational kernel learning and semi-relational kernel learning cannot do without a combinatorial search.
The same RKL run that would find one shared kernel also yields the embedding, so clustering, pattern discovery, and anomaly detection come at a measured extra cost of about 0.07 percent relative to all kernel evaluations.
Each dimension of the embedding is interpretable, so the clusters it produces can be explained in natural language, for example as a linear function or a periodic function whose shape changes smoothly.
On the synthetic clustering benchmark, HRKL separates structural classes such as sine waves, lines, step functions, and sinc functions with a V-measure of 0.836, well above the DTW, SAX BoP, and k-Shape baselines.
On the corrupted gait data, HRKL achieves a perfect separation of corrupted from uncorrupted time series, while the baseline methods do not separate them cleanly.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper: because the embedding is built from model-selection scores, the same construction should work with other selection criteria, such as AIC or cross-validated likelihood; the paper only explores BIC.
Beyond the paper: the per-row standardization makes the embedding a within-series shape profile, so series length and noise level could in principle dominate structure; testing HRKL on series of varying lengths within one structural class would settle this.
Beyond the paper: the embedding could be used for more than clustering, such as low-dimensional visualization of large time-series collections or as a feature representation for supervised tasks; the paper does not explore those uses.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this paper does one simple thing that hasn't been done – save the BIC score for every kernel–series pair from a relational kernel search and treat the standardized row as an embedding – and that idea is worth knowing. The paper explains the method cleanly and shows on synthetic data that it clusters by structure rather than parameter values, where DTW and SAX BoP fail. The computational cost argument is honest: the extra work is clustering plus saving a matrix, which is negligible next to GP fits.

Where it gets soft: the pattern discovery section says HRKL and DTW 'both perform best,' but Table 2 gives DTW a V-measure of 1.000 and HRKL 0.821. That is not a tie. The anomaly detection result is perfect, but it is one run on 15 series with three corrupted; no error bars, no repeated runs, and no statistical testing anywhere. The kernel grammar is described as 'all non-redundant structures,' but the redundancy rule is not specified, and no code is provided, so exact reproduction is not possible.

The bigger open question, which the paper does not address, is what the standardized BIC profile actually encodes. The BIC penalty term includes |M| log n, so the penalty differences among kernels scale with series length. After row standardization, the profile may still depend on n as well as on structure. All three experiments use series of equal length, so this confound is never tested. This is not a refutation – the embedding could still behave well on mixed-length data – but it is a load-bearing assumption that should be stated and tested.

The 'circularity' concern does not land: the kernels are fit per series, clustering uses only the scores, and the final cluster kernel is chosen afterwards. That is just ordinary bootstrap-like thinking, not circularity.

So: the central idea is plausible and the paper gives it a fair first evaluation. It deserves a careful referee, though not a spotless one. The authors should fix the DTW overclaim, add error bars or repeated runs, release code, and either test or clearly limit the claim with respect to series length. With those changes it would be a solid workshop or short-conference paper. As it stands, it is a useful but preliminary contribution.

Referee Report

4 major / 5 minor

Summary. The paper proposes Heterogeneous Relational Kernel Learning (HRKL), an extension of relational kernel learning (RKL) for time series. During an RKL search, the method records the BIC score of every kernel evaluated on every time series, forming a J-by-K BIC history matrix. After row-standardizing this matrix, each row is used as an embedding of the corresponding time series. The authors argue that because each column corresponds to an interpretable kernel, the embedding is interpretable, and they demonstrate its use for clustering, pattern discovery, and anomaly detection. They further claim that HRKL adds practically no computational cost over a single RKL fit. The central claim is that standardized BIC profiles provide a meaningful measure of structural similarity that allows RKL-style models to handle heterogeneous sets of time series without extra model fitting.

Significance. If the central claim holds, the contribution is genuinely useful: it repurposes intermediate results that are normally discarded, yielding an interpretable embedding at negligible additional cost. The method is simple, clearly specified in Algorithm 1, and the reuse of BIC history is an elegant idea. The experimental results are suggestive, especially the perfect separation in the gait anomaly detection task. The paper also states its computational overhead honestly and quantitatively. However, the evidence is not yet sufficient. The embedding's interpretability rests on an untested comparability assumption about BIC scores across series of different lengths and noise levels, the experiments are single-run with no error bars or significance tests, and one of the three application claims (pattern discovery) is contradicted by the paper's own table, where DTW achieves V-measure 1.000 versus HRKL's 0.821. These issues are fixable, but they are load-bearing for the paper's central claims rather than mere presentation.

major comments (4)

[Section 3 and Section 2.2 (BIC definition)] The central interpretability claim rests on the assumption that row-standardized BIC profiles encode shared structure rather than series-specific nuisance attributes. The BIC, defined in Section 2.2 as -2 log p(D|M) + |M| log n, has a penalty term that scales with log n_j, so longer series receive systematically different penalty increments between complex and simple kernels. Standardizing each row by subtracting its mean and dividing by its standard deviation does not remove this dependence, and the marginal likelihood scale also depends on noise level and other series-specific factors. The synthetic and Google Trends experiments use series of equal length, and the gait experiment does not systematically vary length while holding structure fixed. Please add a controlled experiment that varies series length and noise level within the same structural class and checks whether the standardized BIC profile remains stable. Without such a test, the embedding's structural semantics, which are the foundation of the method, are not established.
[Tables 1-3 and Sections 4.1-4.3] All quantitative results come from single runs with no error bars, no repeated trials, and no statistical significance tests. This is particularly problematic for the anomaly detection experiment, where Section 4.3.1 states that one time series from each class is 'randomly selected' for corruption; a single random draw cannot support the claim that HRKL 'significantly outperforms' all baselines. Please report means and standard deviations over multiple random corruptions and repeated clustering runs, or at least provide a sensitivity analysis, so the reader can assess whether the reported margins are stable.
[Section 4.2 and Table 2] The text contains a direct contradiction about the pattern discovery result. Section 4.2 says 'We find HRKL and DTW both perform best and discover the same overall structures,' while Section 4.2.3 says 'DTW performs the best, followed by HRKL and k-Shape with a tie.' Table 2 shows DTW with V-measure 1.000 and HRKL with V-measure 0.821, tied with k-Shape. The correct statement is that DTW outperforms HRKL on this task. This overstatement should be corrected, since pattern discovery is one of the three headline applications.
[Section 4.3 and Table 3] The claim in Section 4.3.3 that 'SAX BoP also clearly separates the corrupted data from the uncorrupted data' is inconsistent with Table 3, where SAX BoP attains V-measure 0.316. If the intended claim is that the dendrogram in Figure 6b visually shows a separation, that is not a quantitative guarantee and should be stated as such, or the text should be tempered to match the table.

minor comments (5)

[Algorithm 1] Algorithm 1 line 9 says 'cluster(B)', but Section 3 states that the standardized matrix, not the raw BIC matrix, is the embedding. Please make the standardization step explicit in the algorithm.
[Section 1] There is a typo: 'has to potential' should be 'has the potential'.
[Figure 3 caption] The caption reads 'The data can is divided into four structural subsets'; this should be 'The data can be divided'.
[Section 4.1.1 and Figure 2] The text and figure use 'Heavyside' where the standard spelling is 'Heaviside'.
[References] Reference [14] is cited in text as 'Yunseong et al.' but the reference entry lists the first author as 'Hwang Yunseong'; please make the citation style consistent with the reference entry.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: HRKL's embedding is built from independent per-series kernel BIC scores, and clustering is unsupervised.

full rationale

The paper's derivation chain is self-contained. HRKL saves the BIC value for every kernel-series pair evaluated during an RKL search (Section 3, Algorithm 1), row-standardizes these values, and uses the resulting matrix as an embedding. The kernels are optimized independently for each time series, and the clustering step operates only on the resulting BIC scores without any label or outcome being fitted. The later selection of a kernel for each discovered cluster is a post-hoc descriptive step, not a prediction fitted to the same data in a way that forces the clustering result. No parameter is fitted to a subset and then used to predict a closely related quantity. The paper contains no self-citations that carry a load-bearing uniqueness or ansatz argument; citations to prior work (Duvenaud et al., Lloyd et al., Yunseong et al.) are external and used to define the kernel language and RKL/SRKL baselines. The residual concern that row-standardized BIC profiles may encode series length or noise rather than structure is a substantive empirical validity question, not a circularity, because the embedding is defined from BIC scores rather than from the target labels or cluster assignments.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The method introduces no new entities or fitted constants beyond the GP hyperparameters already present in RKL. The kernel grammar, standardization, and clustering choices are domain assumptions rather than free parameters. The BIC criterion is standard model selection. The key unexamined assumption is that row-standardized BIC profiles are comparable across series.

assumptions (4)

standard math BIC is an appropriate model selection criterion for comparing kernels of different complexity.
Used throughout to rank kernels and to select cluster kernels (Sections 2.2 and 3).
domain assumption The kernel grammar of SE, LIN, and PER with the listed compositions generates a sufficiently rich set of 87 kernels to discriminate the structural classes in the experiments.
The list is chosen by the authors and is not derived from the data; Section 4.1.3.
domain assumption Gaussian process marginal likelihoods and their gradients can be computed reliably for each time series.
Required by the BIC computation in Algorithm 1; the paper cites Rasmussen and Williams for this.
domain assumption Per-row standardization of BIC values makes rows comparable across time series.
Introduced in Section 3 without theoretical justification.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Heterogeneous Relational Kernel Learning." pith.science (2026). https://pith.science/paper/J4CMZJXX

@misc{pith2026190809219,
  author       = {Pith},
  title        = {Pith review of: Heterogeneous Relational Kernel Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/J4CMZJXX}},
  note         = {Machine review of arXiv:1908.09219}
}

read the original abstract

Recent work has developed Bayesian methods for the automatic statistical analysis and description of single time series as well as of homogeneous sets of time series data. We extend prior work to create an interpretable kernel embedding for heterogeneous time series. Our method adds practically no computational cost compared to prior results by leveraging previously discarded intermediate results. We show the practical utility of our method by leveraging the learned embeddings for clustering, pattern discovery, and anomaly detection. These applications are beyond the ability of prior relational kernel learning approaches.

Figures

Figures reproduced from arXiv: 1908.09219 by the authors.

**Figure 2.** Dataset consisting of 60 time series with heteroge [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Standardized search volume time series from [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 5.** Figure 5: PhysioNet Gait in Aging and Disease dataset which [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

**Figure 4.** Figure 4: Dendrogram visualizing the HAC clustering of the [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 6.** Figure 6: Dendrogram visualizing the HAC clustering of the [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

14 extracted references · 14 canonical work pages

[1]

The Field Guide to Data Science

Booz Allen Hamilton. The Field Guide to Data Science. page 110, 2013

work page 2013
[2]

Density-Based Clustering Based on Hierarchical Density Estimates

Ricardo J G B Campello, Davoud Moulavi, and Joerg Sander. Density-Based Clustering Based on Hierarchical Density Estimates. pages 160–172, 2013

work page 2013
[3]

Tenenbaum, and Zoubin Ghahramani

David Duvenaud, James Robert Lloyd, Roger Grosse, Joshua B. Tenenbaum, and Zoubin Ghahramani. Structure Discovery in Nonparametric Regression through Compositional Kernel Search. 28, 2013

work page 2013
[4]

Finding structural similarity in time series data using bag-of-patterns representation

Jessica Lin and Yuan Li. Finding structural similarity in time series data using bag-of-patterns representation. Lecture Notes in Computer Science (including sub- series Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) , 5566 LNCS:461–477, 2009

work page 2009
[5]

Tenenbaum, and Zoubin Ghahramani

James Robert Lloyd, David Duvenaud, Roger Grosse, Joshua B. Tenenbaum, and Zoubin Ghahramani. Automatic Construction and Natural-Language Description of Nonparametric Regression Models. 2014

work page 2014
[6]

Fast and Accurate Time-Series Clustering

John Paparrizos and Luis Gravano. Fast and Accurate Time-Series Clustering. ACM Transactions on Database Systems , 42(2):1–49, 2017. (a) HRKL (b) SAX BoP (c) DTW Figure 6: Dendrogram visualizing the HAC clustering of the corrupted gait data found using (a) HRKL, (b) SAX BoP, and (c) DTW. The leaf labels correspond to the grouping labels from Figure 5

work page 2017
[7]

{O}ccam’s Razor

Carl Edward Rasmussen and Zoubin Ghahramani. {O}ccam’s Razor. In Advances in Neural Information Processing Systems 13 , 2001

work page 2001
[8]

Rasmussen and Williams - Gaussian Processes for Machine Learning

C E Rasmussen, C K I Williams, Gaussian Processes, M I T Press, and Michael I Jordan. Rasmussen and Williams - Gaussian Processes for Machine Learning . 2006

work page 2006

Show all 14 references

[9]

V-Measure : A conditional entropy-based external cluster evaluation measure

Andrew Rosenberg and Julia Hirschberg. V-Measure : A conditional entropy-based external cluster evaluation measure. (June):410–420, 2007

2007
[10]

FastDTW : Toward Accurate Dynamic Time Warping in Linear Time and Space

Stan Salvador and Philip Chan. FastDTW : Toward Accurate Dynamic Time Warping in Linear Time and Space. Time, 11(5):70–80, 2004

2004
[11]

Estimating the Dimension of a Model

Gideon Schwarz. Estimating the Dimension of a Model. The Annals of Statistics, 6(2):461–464, 1978. 7 MileTS ’19, August 5th, 2019, Anchorage, Alaska, USA Andre T. Nguyen and Edward Raff

1978
[12]

In Preparation

Eli Weinstein and Jeff Miller. In Preparation
[13]

Gaussian Process Kernels for Pattern Discovery and Extrapolation

Andrew Gordon Wilson and Ryan Prescott Adams. Gaussian Process Kernels for Pattern Discovery and Extrapolation. 28, 2013

2013
[14]

Automatic Construction of Non Parametric Relational Regression Models for Multiple Time Series

Hwang Yunseong, Tong Anh, and Choi Jaesik. Automatic Construction of Non Parametric Relational Regression Models for Multiple Time Series. Icml, 48, 2016. ACKNOWLEDGEMENTS Special thanks to Drew Farris for his support of this work and to Eli N. Weinstein for interesting conver...

2016

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

The Field Guide to Data Science

Booz Allen Hamilton. The Field Guide to Data Science. page 110, 2013

work page 2013

[2] [2]

Density-Based Clustering Based on Hierarchical Density Estimates

Ricardo J G B Campello, Davoud Moulavi, and Joerg Sander. Density-Based Clustering Based on Hierarchical Density Estimates. pages 160–172, 2013

work page 2013

[3] [3]

Tenenbaum, and Zoubin Ghahramani

David Duvenaud, James Robert Lloyd, Roger Grosse, Joshua B. Tenenbaum, and Zoubin Ghahramani. Structure Discovery in Nonparametric Regression through Compositional Kernel Search. 28, 2013

work page 2013

[4] [4]

Finding structural similarity in time series data using bag-of-patterns representation

Jessica Lin and Yuan Li. Finding structural similarity in time series data using bag-of-patterns representation. Lecture Notes in Computer Science (including sub- series Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) , 5566 LNCS:461–477, 2009

work page 2009

[5] [5]

Tenenbaum, and Zoubin Ghahramani

James Robert Lloyd, David Duvenaud, Roger Grosse, Joshua B. Tenenbaum, and Zoubin Ghahramani. Automatic Construction and Natural-Language Description of Nonparametric Regression Models. 2014

work page 2014

[6] [6]

Fast and Accurate Time-Series Clustering

John Paparrizos and Luis Gravano. Fast and Accurate Time-Series Clustering. ACM Transactions on Database Systems , 42(2):1–49, 2017. (a) HRKL (b) SAX BoP (c) DTW Figure 6: Dendrogram visualizing the HAC clustering of the corrupted gait data found using (a) HRKL, (b) SAX BoP, and (c) DTW. The leaf labels correspond to the grouping labels from Figure 5

work page 2017

[7] [7]

{O}ccam’s Razor

Carl Edward Rasmussen and Zoubin Ghahramani. {O}ccam’s Razor. In Advances in Neural Information Processing Systems 13 , 2001

work page 2001

[8] [8]

Rasmussen and Williams - Gaussian Processes for Machine Learning

C E Rasmussen, C K I Williams, Gaussian Processes, M I T Press, and Michael I Jordan. Rasmussen and Williams - Gaussian Processes for Machine Learning . 2006

work page 2006