InfoDPCCA: Information-Theoretic Dynamic Probabilistic Canonical Correlation Analysis

Shiqin Tang; Shujian Yu

REVIEW 4 major objections 6 minor 22 references

InfoDPCCA: Information-Theoretic Dynamic Probabilistic Canonical Correlation Analysis

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read InfoDPCCA extracts only mutual information between paired sequences.

desk verdict A plausible IB-based dynamic CCA with a genuine contribution, but the central 'only mutual information' guarantee rests on an invalid marginalization and is never verified. read the letter →

arxiv 2506.08884 v1 pith:4FX6WCZ2 submitted 2025-06-10 cs.LG cs.ITmath.IT

Shiqin Tang , Shujian Yu This is my paper

classification cs.LGcs.ITmath.IT MSC 62H2062M1068T0794A17

keywords probabilisticcanonicalcorrelationanalysisdynamiclatentvariablemodelsinformationbottleneckconditionalmutualrepresentationlearningsequentialdatavariationalinferencefMRI

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

InfoDPCCA is a dynamic probabilistic canonical correlation analysis model for two paired time series. It claims that a single shared latent state per time step can be forced, by an information-theoretic objective, to encode only the information the two sequences hold in common, while separate private latent states absorb sequence-specific dynamics. The paper argues that this makes the shared representation interpretable and robust, and it reports that the extracted states separate patient groups and neural states more cleanly than earlier dynamic CCA on synthetic and fMRI data. A two-step training scheme first optimizes the information-theoretic objective and then fits the full generative model with the first step fixed.

What carries the argument

The load-bearing object is the information-theoretic objective in Eq. (16): a sum over time of an information bottleneck term $\alpha I(z_t^0; x^{1:2}_{1:t}) - I(z_t^0; x^{1:2}_{t+1})$ plus a regularization term $\beta [I(z_t^0; x^1_{1:t} | x^2_{1:t}) + I(z_t^0; x^2_{1:t} | x^1_{1:t})]$. The regularizer is what forces the shared latent to discard single-sequence information, and the paper makes it tractable through the variational bound in Eq. (22), which replaces intractable posteriors with learned stochastic encoders $q^{1:2}_0$, $q^1_0$, and $q^2_0$, leading to the training objective in Eq. (26). A two-step scheme first optimizes this objective and then trains private latents and emissions with the shared encoder fixed, with a residual connection that reuses the step-I emitter. The graphical model also imposes serial independence, which lets the posterior factor as in Eq. (30) and supports the information-theoretic derivation.

What would settle it

Train InfoDPCCA on two independent random-walk sequences that share no dynamics; if the regularizer is working, the shared latent state should carry no predictive information about either stream and its dimension should effectively collapse, whereas a loose bound would let $z_t^0$ reconstruct sequence-specific structure.

Watch

Extended reading notes

Core claim

The paper's central claim is that replacing DPCCA's purely structural shared latent with an objective of the form in Eq. (16) — an information bottleneck term plus a regularization term $I(z^0; x^1 | x^2) + I(z^0; x^2 | x^1)$ — yields shared latent states $z_t^0$ that carry the mutual information between $x^1$ and $x^2$ and exclude sequence-specific information. The paper derives a variational bound in Eq. (22) and a tractable training objective in Eq. (26), and shows through a global-mean correlation metric on a two-dimensional chaotic synthetic map that the two-step procedure recovers the known shared dynamics (72% versus 65% for step II alone). On autism-spectrum, Alzheimer's, and eyes-open/eyes-closed fMRI data, the shared states learned by InfoDPCCA give higher NMI and Silhouette clustering scores than PCA or DPCCA, which the paper takes as evidence that the common dynamics are captured. The paper thus claims InfoDPCCA works both as a generative model and as a stochastic representation learning method whose shared latent space has a clear information-theoretic meaning.

Load-bearing premise

The load-bearing premise is that the variational bound in Eq. (22) is tight enough, and the marginal replacement in Eq. (27) is accurate enough, that minimizing the training objective truly drives the two conditional-mutual-information terms in Eq. (16) to zero.

Editorial extensions

If this is right

If the regularizer works as claimed, $z_t^0$ has a precise interpretation: it is a stochastic summary of the past that predicts the next observation of both streams while being purged of each stream's private dynamics.
The two-step training lets the same architecture serve both as a representation learner (step I) and as a generative model (step II) without retraining the shared encoder.
Because $z_t^0$ is designed to exclude single-sequence information, downstream classifiers or clustering methods applied to the shared states should be more robust to noise in one of the streams.
On the reported fMRI benchmarks, the shared states separate diagnostic groups and neural states better than PCA and DPCCA, suggesting the method can serve as a preprocessing step for biomarker discovery.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial extension: If the variational bound in Eq. (22) is tight, InfoDPCCA becomes a temporal analogue of multi-view information bottleneck, and a natural extension is to apply the same conditional-mutual-information regularizer to more than two streams to isolate the information common to all views.
Editorial extension: The objective suggests a direct audit method: after training, one can estimate $I(z^0; x^1 | x^2)$ on held-out data; if it remains far from zero, the 'only mutual information' claim fails and the hyperparameters $\alpha, \beta$ would need adjustment.
Editorial extension: Because step I is unsupervised, adding a label-dependent term such as $I(z^0; y)$ to the objective would likely sharpen group separation on fMRI data; the authors list a supervised extension as future work, and it is a minimal change to Eq. (16).

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Probably worth your time if you work on probabilistic CCA or view-based representation learning. The genuinely new part is the objective in Eq. (16): it adds conditional-MI terms to the standard IB objective to push the shared latent z0 away from view-specific information, plus a two-step training scheme and a residual connection. Each component is known, but the combination is new relative to DPCCA and MVIB. The paper also ships code, writes the variational bounds explicitly, and is honest twice (Sec 5.1 and 5.2) that there is no criterion to verify the central claim. That honesty is real credit.

The soft spot is load-bearing. Eq. (27) claims q1_0(z0|x1) approximates p(z0|x1), but it replaces an integral over x2 with a single evaluation at the observed x2. That is not a marginalization. For a Gaussian encoder, the marginal over x2 is strictly wider than any fixed-x2 conditional. Remark 1 is also circular: it assumes the conditional independence the regularizer is supposed to enforce. So the bound in Eq. (22) is loose, and minimizing the objective does not establish that the conditional MIs are small. The experiments never test this directly. The Hénon data has no view-private latent component, so leakage cannot be detected; the fMRI clustering proxy measures separability, not mutual information. The experiments also lack error bars, and Step I alone often matches Step II, which weakens the two-step story.

That said, this is not a fatal paper. The method is clearly specified, the code is available, and the gap is a missing theoretical justification rather than a demonstrated failure. The core idea is intact, and the failure mode is fixable: treat q1_0 as an approximate marginal, add a diagnostic that estimates conditional MI on synthetic data with private components, and add error bars. The self-citation to Tang et al. 2025b is appropriate because that is the direct predecessor.

Who gets value: people building dynamic latent variable models or multimodal representations. It deserves a serious referee, but I would push for major revision and would not let the "only mutual information" claim through as stated.

Referee Report

4 major / 6 minor

Summary. The manuscript introduces InfoDPCCA, a dynamic probabilistic CCA model for two paired time series. The model factorizes the latent state into a shared component z0_t and private components z1_t and z2_t, and is trained in two steps: Step I optimizes an information-theoretic objective (Eq. 16) that combines an information bottleneck term with a regularization term meant to suppress sequence-specific information in z0_t; Step II trains the full generative model with the Step I mappings fixed. The paper also proposes residual emitter connections and RNN reuse for training stability. Experiments on a synthetic Hénon map dataset and three resting-state fMRI datasets (ADNI, NYU, ECEO) report reconstruction/correlation scores and clustering metrics (NMI/Silhouette) comparing InfoDPCCA with PCA and DPCCA. The authors explicitly state that no known criterion exists to validate whether z0_t encodes only the mutual information, and that hyperparameters α and β have no closed-form solution. The central claim, repeated in the abstract and Section 4.3, is that InfoDPCCA 'explicitly enforces' the shared latent to contain only the mutual information between the sequences.

Significance. If the central claim were rigorously established, InfoDPCCA would be a useful tool for extracting interpretable shared dynamics from paired sequential data, and the variational bound on the conditional mutual information terms would be a meaningful theoretical contribution. The paper is clearly written, provides code, and includes several honest statements of its limitations (Remark 1; Section 5.1). The two-step training scheme and residual connection are sensible engineering contributions that appear to improve stability. However, the paper's main theoretical guarantee is not proven: the variational bound in Eq. (22) is loose, and its justification via Eq. (27) rests on an invalid marginalization that is essentially circular. The experiments do not measure the quantity that the theory is meant to control, and the synthetic benchmark has no private-information component by construction. The contribution is therefore better characterized as a heuristic info-theoretic objective plus a practical training recipe than as a method with an established interpretability guarantee.

major comments (4)

[§4.3, Eq. (27), Remark 1] The proof that minimizing Eq. (26) drives the conditional mutual information terms in Eq. (16) to zero is not valid. The replacement of p(z0|x1) by q^{1:2}_0(z0|x1:2) in Eq. (27) is an equality only when z0 is conditionally independent of x2 given x1, which is exactly the property the regularizer is meant to enforce; otherwise the right-hand side is a mixture over x2 whose covariance is generally larger than that of any single conditional. This is a circular argument, and the bound in Eq. (22) is therefore not shown to be tight. In addition, the expectations defined in Eq. (24) evaluate the second term under q^{1:2}_0 rather than under the required p(z0|x1) (or q^1_0), so the final objective does not correspond to the stated bound. The abstract's claim that InfoDPCCA 'explicitly enforces' the only-mutual-information property is not supported.
[§5.1] The Hénon synthetic experiment cannot detect leakage of view-private information because the generative process contains no private latent component: both observations are linear projections of the same two-dimensional shared latent with independent Gaussian noise. Any method that recovers the shared latent from either view alone would receive a high correlation score on the metric in Eq. (36). The experiment thus validates that InfoDPCCA can reconstruct the common signal, but it provides no evidence that z0_t excludes information specific to x1 or x2.
[§5.2, Table 1] The fMRI clustering evaluation is a proxy that does not test the information-theoretic claim. NMI and Silhouette scores measure class separability of the extracted representations, not the conditional mutual information terms I(z0;x1|x2) and I(z0;x2|x1) that the regularizer is designed to minimize. Moreover, no error bars or significance tests accompany Table 1, and the ECEO column shows a large drop from Step I (NMI 0.572) to Step II (NMI 0.228), which is inconsistent with the text's assertion that Step I and Step II produce similar representations. The clustering results, while suggestive, do not establish the interpretability guarantee.
[§4.3 (hyperparameters)] The default choice α=1, β=0.1 is used in all experiments without a sensitivity analysis or cross-validation. Because the regularization term in Eq. (16) is the mechanism claimed to enforce the only-mutual-information property, the absence of any analysis of β leaves open the possibility that the reported gains come from the IB term or from the two-step architecture rather than from the regularization. This is a load-bearing gap for the empirical support of the central claim.

minor comments (6)

[§4.3, Eq. (21)] The notation H(z0; x2) is nonstandard; the intended quantity appears to be the conditional entropy H(z0|x2), and the same applies to H(z0; x1:2). Please correct the entropy notation throughout the derivation.
[Throughout] The name 'Hénon' is garbled as 'HÃ©non' in several places (e.g., Section 5.1 and Figure 6), and Section 5.1 contains the typo 'creteria' for 'criterion'. Please fix these textual errors.
[§4.1 and Eq. (35)] In Section 4.1, 'unique latent variables z1_z and z2_z' appears to be a typo for z1_t and z2_t; in Eq. (35), the expression for σ_t uses bµ where bσ seems intended.
[Algorithm 1] Algorithm 1 uses flags reuse_RNN and residual_connection without having defined them in the text; please state the default settings and clarify whether the reported results use reuse_RNN.
[Abstract and §4.3] The language should be consistent: the objective can 'encourage' or 'promote' the desired property, but 'explicitly enforces' overstates what is currently shown, especially given the acknowledged lack of a verification criterion.
[References] The reference to Koller [2009] is for Daphne Koller; the author name is misspelled as 'Daphane Koller' in the bibliography.

Circularity Check

1 steps flagged · score 6.0 of 10

The 'only mutual information' guarantee rests on Eq. (27), which assumes away the conditional dependence the regularizer is meant to remove.

self definitional [Section 4.3, Remark 1, Eq. (27)]
"Minimizing this term drives q^1_0(z0|x1) to align with q^{1:2}_0(z0|x1:2), which serves as an approximation to p(z0|x1). To see why, note that p(z0|x1) = ∫ p(z0, x2|x1)dx2 = ∫ p(x2|x1)q^{1:2}_0(z0|x1:2)dx2 ≈ ∫ p_D(x2|x1)q^{1:2}_0(z0|x1:2)dx2 = q^{1:2}_0(z0|x1:2)."

The displayed equality treats the x2-marginal of q^{1:2}_0(z0|x1:2) as q^{1:2}_0(z0|x1:2) itself. That equality is valid only if q^{1:2}_0(z0|x1,x2) does not depend on x2, i.e. I(z0;x2|x1)=0, which is exactly one of the two conditional-mutual-information terms the regularization term in Eq. (16) is introduced to drive to zero. The paper thus invokes the target conditional-independence property to justify the tightness of the variational bound, so the conclusion that the shared latent encodes only the mutual information is assumed rather than derived. This is load-bearing: Remark 1 is the paper's only argument that minimizing Eq. (26) genuinely enforces the regularization term in Eq. (16).

full rationale

The central derivation chain has one circular step. Eq. (27) justifies the tightness of the bound used to minimize I(z0;x1|x2)+I(z0;x2|x1) by replacing the marginal p(z0|x1) with the joint-conditional q^{1:2}_0(z0|x1:2), which is only valid under the conditional independence that the regularization term is supposed to enforce. The paper's headline interpretability claim therefore reduces, at the point of proof, to the property it claims to establish. No fitted parameter is renamed as a prediction, and the self-citations to D2PCCA/Tang et al. are used as a baseline rather than as a load-bearing uniqueness or existence argument. The experiments use external ground truth (Hénon synthetic latents, held-out fMRI labels), so the empirical body has independent content, but it does not test I(z0;x1|x2) or I(z0;x2|x1) directly and cannot repair the circular derivation of the guarantee. This is a partial, not total, circularity: the model and objective still have independent algorithmic content, but the central information-theoretic claim is not established by the paper's derivation.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The method rests on standard information-theoretic identities plus three structural choices: the encoder is treated as the true posterior, the latent states are assumed to be serially independent, and Eq. (27) approximates a mixture over x2 by a point evaluation. The free parameters alpha, beta and the latent dimensionality are chosen by hand.

free parameters (3)

alpha = 1 (default)
Trade-off between compression and regularization in objective (16); no closed-form solution, authors suggest cross-validation and default to alpha=1.
beta = 0.1 (default)
Weight of the conditional-MI regularization term in (16); default 0.1.
Latent dimensionality = not reported
Dimensionalities of z0, z1, z2 are chosen by hand and not stated in the main text; they affect model capacity.

assumptions (5)

standard math Chain rule of mutual information: I(z;x1,x2)=I(z;x2)+I(z;x1|x2)
Used in Eq. (21) to rewrite the regularization term.
standard math Gibbs' inequality D_KL(p||q) >= 0
Used in Eq. (23) to establish the variational bound in Eq. (22).
domain assumption Modeling assumption p(z0|x1:2)=q^{1:2}_0(z0|x1:2)
Assumed before Eq. (22); standard in variational inference but not exact for finite-capacity encoders.
ad hoc to paper Serial independence: z^i_t is independent of z^j_{t-n} given x^{1:2}_{1:T} for all i,j,n
Eq. (14), introduced to make the posterior factorization (30) valid and to enable the two-step training; a structural modeling choice without empirical justification.
ad hoc to paper Approximation p(z0|x1) is approximately q^{1:2}_0(z0|x1:2) in Eq. (27)
Remark 1 argues training aligns q^1_0 with p(z0|x1), but the displayed derivation replaces an average over x2 by a single q^{1:2}_0, which is not a proper marginalization.

how reviews work

0 comments

Cite this review

Pith. "Pith review of InfoDPCCA: Information-Theoretic Dynamic Probabilistic Canonical Correlation Analysis." pith.science (2026). https://pith.science/paper/4FX6WCZ2

@misc{pith2026250608884,
  author       = {Pith},
  title        = {Pith review of: InfoDPCCA: Information-Theoretic Dynamic Probabilistic Canonical Correlation Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4FX6WCZ2}},
  note         = {Machine review of arXiv:2506.08884}
}

read the original abstract

Extracting meaningful latent representations from high-dimensional sequential data is a crucial challenge in machine learning, with applications spanning natural science and engineering. We introduce InfoDPCCA, a dynamic probabilistic Canonical Correlation Analysis (CCA) framework designed to model two interdependent sequences of observations. InfoDPCCA leverages a novel information-theoretic objective to extract a shared latent representation that captures the mutual structure between the data streams and balances representation compression and predictive sufficiency while also learning separate latent components that encode information specific to each sequence. Unlike prior dynamic CCA models, such as DPCCA, our approach explicitly enforces the shared latent space to encode only the mutual information between the sequences, improving interpretability and robustness. We further introduce a two-step training scheme to bridge the gap between information-theoretic representation learning and generative modeling, along with a residual connection mechanism to enhance training stability. Through experiments on synthetic and medical fMRI data, we demonstrate that InfoDPCCA excels as a tool for representation learning. Code of InfoDPCCA is available at https://github.com/marcusstang/InfoDPCCA.

Figures

Figures reproduced from arXiv: 2506.08884 by the authors.

**Figure 3.** Graphical models in the two-step training of In [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

**Figure 4.** Venn diagram illustrating the relationship between [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗

Figures from the paper (4 more)

**Figure 6.** Figure 6: Reconstruction on an HÃ©non map using InfoDPCCA. Note that metrics like RMSE would not work here because, for instance, a model that extracts the latent states exactly like the ground truth but with an opposite sign would fail the RMSE test. A visual inspection on the…

**Figure 7.** Figure 7: Visual inspection of clustering tasks on three datasets (ADNI, NYU, ECEO) using four models (PCA, DPCCA, [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

**Figure 8.** Figure 8: Venn diagram illustrating the relationship between the feature of the view [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]

**Figure 9.** Figure 9: Extracted shared latent states for the supplementary experiment on the ECEO dataset. [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

22 extracted references · 13 canonical work pages

[1]

Deep variational information bottleneck

Alexander A Alemi, Ian Fischer, Joshua V Dillon, and Kevin Murphy. Deep variational information bottleneck. arXiv preprint arXiv:1612.00410,

arXiv
[5]

URL https://www.nature

doi: 10.1038/mp.2014.54. URL https://www.nature. com/articles/mp.2014.54. Marco Federici, Anjan Dutta, Patrick Forré, Nate Kush- man, and Zeynep Akata. Learning robust representations via multi-view information bottleneck. arXiv preprint arXiv:2002.07017,

arXiv 2014
[7]

Dynamical variational autoencoders: A comprehensive review

Laurent Girin, Simon Leglaive, Xiaoyu Bie, Julien Diard, Thomas Hueber, and Xavier Alameda-Pineda. Dynamical variational autoencoders: A comprehensive review. arXiv preprint arXiv:2008.12595,

arXiv 2008
[8]

Identity mappings in deep residual networks

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In Com- puter Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Pro- ceedings, Part IV 14, pages 630–645. Springer,

work page 2016
[14]

Rep- resentation learning with contrastive predictive coding

Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Rep- resentation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748,

arXiv
[15]

Deep Dynamic Probabilistic Canonical Correlation Analysis

Amichai Painsky, Meir Feder, and Naftali Tishby. Non- linear canonical correlation analysis: A compressed rep- resentation approach. arXiv preprint arXiv:2502.05155,

work page Pith review arXiv
[18]

Accessed: 2025-02-

URL https://github.com/ zzstefan/BrainNetClass. Accessed: 2025-02-

work page 2025
[19]

A hierarchical taxonomy for deep state space models

Shiqin Tang, Pengxing Feng, Shujian Yu, Yining Dong, and S Joe Qin. A hierarchical taxonomy for deep state space models. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2025a. Shiqin Tang, Shujian Yu, Yining Dong, and S. Joe Qin. Deep dynamic probabilistic canonical correlation analys...

arXiv 2025

Show all 22 references

[21]

of Data Science , City University of Hong Kong, Hong Kong 2 Inst

Supplementary Material Shiqin Tang1,2 Shujian Yu3,4 1 Dept. of Data Science , City University of Hong Kong, Hong Kong 2 Inst. of Artificial Intelligence , Peking University, Beijing, China 3 Dept. of Computer Science, Vrije Universiteit Amsterdam, Amsterdam, Netherlands 4 Dept...

2020
[22]

This term captures the shared structure between views, potentially including information not directly relevant to a specific supervised task

In contrast to the supervised nature of VIB, which emphasizes label prediction, Multi-View IB seeks to maximize I(z1; v2). This term captures the shared structure between views, potentially including information not directly relevant to a specific supervised task. We summarize...

2025
[1997]

URL https: //papers.nips.cc/paper/1997/hash/ 5b4d9034971fd4f13f315bdb2f8ad56d-Abstract. html. Z. Stefan. A toolbox for brain network construction and classification,

1997
[2000]

Deep variational canonical correlation analysis

Weiran Wang, Xinchen Yan, Honglak Lee, and Karen Livescu. Deep variational canonical correlation analysis. arXiv preprint arXiv:1610.03454,

arXiv
[2005]

Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio

ISSN 1532-4435. Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder–decoder for statistical machine trans- lation. In Proceedings of the 2014 Conference on ...

2014
[2014]

Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron C Courville, and Yoshua Bengio

doi: 10.3115/v1/D14-1179. Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron C Courville, and Yoshua Bengio. A recurrent latent variable model for sequential data. Advances in neural information processing systems, 28,

doi
[2016]

Past-future information bottleneck for linear feedback systems

Nadav Amir, Stas Tiomkin, and Naftali Tishby. Past-future information bottleneck for linear feedback systems. In 2015 54th IEEE Conference on Decision and Control (CDC), pages 5737–5742. IEEE,

2015
[2017]

Deep generative state-space modeling of fmri images for psychiatric disorder diagnosis

Koki Kusano, Tetsuo Tashiro, Takashi Matsubara, and Kuni- aki Uehara. Deep generative state-space modeling of fmri images for psychiatric disorder diagnosis. In 2019 inter- national joint conference on neural networks (IJCNN) , pages 1–7. IEEE,

2019
[2018]

Which bits went where? past and future transfer entropy decom- position with the information bottleneck

Kieran A Murphy, Zhuowen Yin, and Dani S Bassett. Which bits went where? past and future transfer entropy decom- position with the information bottleneck. arXiv preprint arXiv:2411.04992,

arXiv
[2020]

Latent representation and simulation of markov processes via time-lagged information bottleneck

Marco Federici, Patrick Forré, Ryota Tomioka, and Basti- aan S Veeling. Latent representation and simulation of markov processes via time-lagged information bottleneck. arXiv preprint arXiv:2309.07200,

arXiv
[2021]

Auto-encoding varia- tional bayes

Diederik P Kingma and Max Welling. Auto-encoding varia- tional bayes. arXiv preprint arXiv:1312.6114,

arXiv
[2022]

Schölkopf, A

B. Schölkopf, A. Smola, and K.-R. Müller. Kernel principal component analysis. In Proceedings of the 1997 Confer- ence on Advances in Neural Information Processing Sys- tems (NIPS 1997), pages 582–588,

1997
[2023]

Com- pressed predictive information coding

Rui Meng, Tianyi Luo, and Kristofer Bouchard. Com- pressed predictive information coding. arXiv preprint arXiv:2203.02051,

arXiv
[2025]

Accessed: 2025- 02-11

URL https://adni.loni.usc.edu/. Accessed: 2025- 02-11. Damjan Kalajdzievski, Ximeng Mao, Pascal Fortier-Poisson, Guillaume Lajoie, and Blake Richards. Transfer entropy bottleneck: Learning sequence to sequence information transfer. arXiv preprint arXiv:2211.16607,

2025 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Deep variational information bottleneck

Alexander A Alemi, Ian Fischer, Joshua V Dillon, and Kevin Murphy. Deep variational information bottleneck. arXiv preprint arXiv:1612.00410,

arXiv

[2] [5]

URL https://www.nature

doi: 10.1038/mp.2014.54. URL https://www.nature. com/articles/mp.2014.54. Marco Federici, Anjan Dutta, Patrick Forré, Nate Kush- man, and Zeynep Akata. Learning robust representations via multi-view information bottleneck. arXiv preprint arXiv:2002.07017,

arXiv 2014

[3] [7]

Dynamical variational autoencoders: A comprehensive review

Laurent Girin, Simon Leglaive, Xiaoyu Bie, Julien Diard, Thomas Hueber, and Xavier Alameda-Pineda. Dynamical variational autoencoders: A comprehensive review. arXiv preprint arXiv:2008.12595,

arXiv 2008

[4] [8]

Identity mappings in deep residual networks

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In Com- puter Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Pro- ceedings, Part IV 14, pages 630–645. Springer,

work page 2016

[5] [14]

Rep- resentation learning with contrastive predictive coding

Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Rep- resentation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748,

arXiv

[6] [15]

Deep Dynamic Probabilistic Canonical Correlation Analysis

Amichai Painsky, Meir Feder, and Naftali Tishby. Non- linear canonical correlation analysis: A compressed rep- resentation approach. arXiv preprint arXiv:2502.05155,

work page Pith review arXiv

[7] [18]

Accessed: 2025-02-

URL https://github.com/ zzstefan/BrainNetClass. Accessed: 2025-02-

work page 2025

[8] [19]

A hierarchical taxonomy for deep state space models

Shiqin Tang, Pengxing Feng, Shujian Yu, Yining Dong, and S Joe Qin. A hierarchical taxonomy for deep state space models. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2025a. Shiqin Tang, Shujian Yu, Yining Dong, and S. Joe Qin. Deep dynamic probabilistic canonical correlation analys...

arXiv 2025