pith. sign in

arxiv: 2601.19963 · v3 · submitted 2026-01-27 · 💻 cs.LG · cs.AI

Cross-Session Decoding of Neural Spiking Data via Task-Conditioned Latent Alignment

Canyang Zhao , Bolin Peng , J. Patrick Mayo , Ce Ju , Bing Liu This is my paper

Pith reviewed 2026-05-16 10:22 UTC · model grok-4.3

classification 💻 cs.LG cs.AI
keywords neural decodingcross-session transferlatent alignmentspiking datamotor cortexbrain-machine interfaceautoencodertask conditioning
0
0 comments X

The pith

Task-conditioned latent alignment transfers neural representations across sessions to improve decoding with limited data.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Neural decoders often perform poorly when only limited spiking data is recorded from a new session. The paper proposes TCLA, which first learns a compact neural representation from a data-rich source session via an autoencoder. It then aligns the limited target-session latents to the source ones while conditioning on the ongoing task. This transfer supports training a decoder that generalizes better than training from the scarce target data alone. Tests on macaque motor and oculomotor center-out tasks show consistent accuracy gains, including a 0.386 rise in R-squared for y-velocity decoding.

Core claim

TCLA learns a low-dimensional neural representation from a source session with sufficient data. For target sessions with limited data, TCLA then aligns the target latent representations to the source session in a task-conditioned manner, enabling effective transfer of learned neural representations to support decoder training in the target session.

What carries the argument

Task-Conditioned Latent Alignment (TCLA) framework built on an autoencoder that maps source and target spiking data into aligned low-dimensional latents while preserving task identity.

Load-bearing premise

Aligning target latent representations to the source in a task-conditioned manner preserves the information needed for accurate decoding without introducing misalignment or loss of task-relevant features.

What would settle it

A controlled experiment on the same macaque datasets showing zero or negative change in coefficient of determination when switching from target-only training to TCLA would falsify the performance claim.

Figures

Figures reproduced from arXiv: 2601.19963 by Bing Liu, Bolin Peng, Canyang Zhao, Ce Ju, J. Patrick Mayo.

Figure 1
Figure 1. Figure 1: Architecture of the proposed model. The model is composed of a shared autoencoder module and session-specific layers. Neural spiking data from [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Experimental Paradigm. (a) Eight-direction motor center-out task. Two [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: T-SNE visualization of the latent representations from the source and target sessions. Each column corresponds to one dataset ( [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Comparison of decoding performance across methods and Datasets. Each column represents a different dataset. The top row shows [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
read the original abstract

Training a high-performing neural decoder can be difficult when only limited data are available from a recording session. To address this challenge, we propose a Task-Conditioned Latent Alignment framework (TCLA) for cross-session neural decoding with limited target-session data. Building upon an autoencoder architecture, TCLA first learns a low-dimensional neural representation from a source session with sufficient data. For target sessions with limited data, TCLA then aligns the target latent representations to the source session in a task-conditioned manner, enabling effective transfer of learned neural representations to support decoder training in the target session. We evaluate TCLA on the macaque motor and oculomotor center-out datasets. Compared to baseline methods trained solely on target-session data, TCLA consistently improves decoding performance across datasets and decoding settings, with gains in the coefficient of determination of up to 0.386 for y coordinate velocity decoding in a motor dataset. These results suggest that TCLA provides an effective strategy for transferring knowledge from source to target sessions, improving neural decoding performance under conditions with limited target-session data.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 0 minor

Summary. The paper proposes Task-Conditioned Latent Alignment (TCLA), an autoencoder-based framework that learns low-dimensional neural representations from a data-rich source session and aligns limited-data target-session latents to the source in a task-conditioned manner to improve cross-session decoding performance. On macaque motor and oculomotor center-out datasets, TCLA is reported to outperform baselines trained only on target data, with R² gains reaching 0.386 for y-coordinate velocity decoding.

Significance. If the alignment step reliably transfers usable decoding information without distorting task-relevant dimensions, the method could meaningfully reduce data requirements for neural decoders in brain-computer interface applications. The reported quantitative gains across datasets and settings indicate potential practical value, though the absence of implementation details, baseline specifications, and statistical validation leaves the magnitude and robustness of the improvement difficult to assess.

major comments (2)
  1. [Abstract] Abstract: the central claim of consistent R² improvements (up to 0.386) is presented without any description of the baseline methods, the precise alignment loss, the conditioning variables, statistical tests, or error analysis, rendering the quantitative results unverifiable from the provided text.
  2. [Abstract] Abstract (alignment description): the task-conditioned matching of target latents to the source autoencoder is asserted to enable effective transfer, yet no argument or experiment is supplied showing that the chosen conditioning variables (e.g., target direction or velocity) span the full space of session-specific spiking variability; unmodeled covariates could therefore collapse distinct target trajectories onto the same source latent region and erode the reported decoding gains.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments on our manuscript. We have revised the abstract and added clarifications in the main text to improve verifiability and address concerns about conditioning variables. Below we respond to each major comment point by point.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central claim of consistent R² improvements (up to 0.386) is presented without any description of the baseline methods, the precise alignment loss, the conditioning variables, statistical tests, or error analysis, rendering the quantitative results unverifiable from the provided text.

    Authors: We agree that the abstract as originally written does not provide enough context for the quantitative claims. In the revised manuscript we have updated the abstract to briefly specify the baseline (decoders trained only on target-session data), the alignment loss (task-conditioned MSE between aligned latents), the conditioning variables (target direction and velocity), and that gains are statistically significant via paired t-tests across sessions (p < 0.05). Full experimental details, error bars, and analysis remain in the Methods and Results sections; the abstract revision is limited by length constraints but now makes the central claim verifiable at a high level. revision: yes

  2. Referee: [Abstract] Abstract (alignment description): the task-conditioned matching of target latents to the source autoencoder is asserted to enable effective transfer, yet no argument or experiment is supplied showing that the chosen conditioning variables (e.g., target direction or velocity) span the full space of session-specific spiking variability; unmodeled covariates could therefore collapse distinct target trajectories onto the same source latent region and erode the reported decoding gains.

    Authors: We acknowledge that the abstract alone does not contain an explicit argument or experiment demonstrating that direction and velocity span all session-specific variability. The full manuscript already contains an analysis (Section 4.3) showing these variables capture the dominant task-related structure in the center-out task. We have added a dedicated paragraph in the Discussion section that directly addresses the risk of unmodeled covariates, includes a new ablation study comparing conditioning on direction+velocity versus additional covariates (e.g., speed), and notes the empirical robustness of the reported gains. We agree this limitation should be stated more explicitly and have done so. revision: partial

Circularity Check

0 steps flagged

No significant circularity; empirical gains independent of any self-referential derivation

full rationale

The paper proposes TCLA as an autoencoder-based alignment method that learns source latents then aligns target latents task-conditionally before decoder training. No equations, uniqueness theorems, or fitted-parameter predictions are shown that reduce the reported coefficient-of-determination gains to inputs defined by the result itself. Performance claims rest on direct comparisons to target-only baselines across macaque datasets, with no self-citation load-bearing steps or ansatz smuggling. The derivation chain is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review prevents identification of specific free parameters or axioms; none are explicitly stated in the provided text.

pith-pipeline@v0.9.0 · 5492 in / 985 out tokens · 36478 ms · 2026-05-16T10:22:49.871172+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

18 extracted references · 18 canonical work pages

  1. [1]

    Intracortical recording stability in human brain–computer interface users.Journal of neural engineering, 15(4):046016, 2018

    John E Downey, Nathaniel Schwed, Steven M Chase, Andrew B Schwartz, and Jennifer L Collinger. Intracortical recording stability in human brain–computer interface users.Journal of neural engineering, 15(4):046016, 2018

    work page 2018

  2. [2]

    Adaptive decoding for brain-machine interfaces through bayesian parameter updates.Neural computation, 23(12):3162–3204, 2011

    Zheng Li, Joseph E O’Doherty, Mikhail A Lebedev, and Miguel AL Nicolelis. Adaptive decoding for brain-machine interfaces through bayesian parameter updates.Neural computation, 23(12):3162–3204, 2011

    work page 2011

  3. [3]

    Unsupervised adaptation of brain- machine interface decoders.Frontiers in neuroscience, 6:164, 2012

    Tayfun G ¨urel and Carsten Mehring. Unsupervised adaptation of brain- machine interface decoders.Frontiers in neuroscience, 6:164, 2012

    work page 2012

  4. [4]

    Rapid adaptation of brain–computer interfaces to new neuronal ensembles or participants via generative modelling

    Shixian Wen, Allen Yin, Tommaso Furlanello, Matthew G Perich, Lee E Miller, and Laurent Itti. Rapid adaptation of brain–computer interfaces to new neuronal ensembles or participants via generative modelling. Nature biomedical engineering, 7(4):546–558, 2023

    work page 2023

  5. [5]

    Neural manifolds for the control of movement.Neuron, 94(5):978–984, 2017

    Juan A Gallego, Matthew G Perich, Lee E Miller, and Sara A Solla. Neural manifolds for the control of movement.Neuron, 94(5):978–984, 2017

    work page 2017

  6. [6]

    Inferring single- trial neural population dynamics using sequential auto-encoders.Nature methods, 15(10):805–815, 2018

    Chethan Pandarinath, Daniel J O’Shea, Jasmine Collins, Rafal Jozefow- icz, Sergey D Stavisky, Jonathan C Kao, Eric M Trautmann, Matthew T Kaufman, Stephen I Ryu, Leigh R Hochberg, et al. Inferring single- trial neural population dynamics using sequential auto-encoders.Nature methods, 15(10):805–815, 2018

    work page 2018

  7. [7]

    Latent diffusion for neural spiking data.Ad- vances in Neural Information Processing Systems, 37:118119–118154, 2024

    Jaivardhan Kapoor, Auguste Schulz, Julius Vetter, Felix Pei, Richard Gao, and Jakob H Macke. Latent diffusion for neural spiking data.Ad- vances in Neural Information Processing Systems, 37:118119–118154, 2024

    work page 2024

  8. [8]

    Robust alignment of cross-session recordings of neural population activity by behaviour via unsupervised domain adaptation

    Justin Jude, Matthew Perich, Lee Miller, and Matthias Hennig. Robust alignment of cross-session recordings of neural population activity by behaviour via unsupervised domain adaptation. InInternational Conference on Machine Learning, pages 10462–10475. PMLR, 2022

    work page 2022

  9. [9]

    Stabilization of a brain–computer interface via the alignment of low- dimensional spaces of neural activity.Nature biomedical engineering, 4(7):672–685, 2020

    Alan D Degenhart, William E Bishop, Emily R Oby, Elizabeth C Tyler-Kabara, Steven M Chase, Aaron P Batista, and Byron M Yu. Stabilization of a brain–computer interface via the alignment of low- dimensional spaces of neural activity.Nature biomedical engineering, 4(7):672–685, 2020

    work page 2020

  10. [10]

    Stabilizing brain- computer interfaces through alignment of latent dynamics.Nature Communications, 16(1):4662, 2025

    Brianna M Karpowicz, Yahia H Ali, Lahiru N Wimalasena, Andrew R Sedler, Mohammad Reza Keshtkaran, Kevin Bodkin, Xuan Ma, Daniel B Rubin, Ziv M Williams, Sydney S Cash, et al. Stabilizing brain- computer interfaces through alignment of latent dynamics.Nature Communications, 16(1):4662, 2025

    work page 2025

  11. [11]

    Neural constraints on learning.Nature, 512(7515):423–426, 2014

    Patrick T Sadtler, Kristin M Quick, Matthew D Golub, Steven M Chase, Stephen I Ryu, Elizabeth C Tyler-Kabara, Byron M Yu, and Aaron P Batista. Neural constraints on learning.Nature, 512(7515):423–426, 2014

    work page 2014

  12. [12]

    A kernel two-sample test.The journal of machine learning research, 13(1):723–773, 2012

    Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Sch¨olkopf, and Alexander Smola. A kernel two-sample test.The journal of machine learning research, 13(1):723–773, 2012

    work page 2012

  13. [13]

    Enabling hy- perparameter optimization in sequential autoencoders for spiking neural data.Advances in neural information processing systems, 32, 2019

    Mohammad Reza Keshtkaran and Chethan Pandarinath. Enabling hy- perparameter optimization in sequential autoencoders for spiking neural data.Advances in neural information processing systems, 32, 2019

    work page 2019

  14. [14]

    Using adversarial networks to extend brain computer interface decoding accuracy over time.elife, 12:e84296, 2023

    Xuan Ma, Fabio Rizzoglio, Kevin L Bodkin, Eric Perreault, Lee E Miller, and Ann Kennedy. Using adversarial networks to extend brain computer interface decoding accuracy over time.elife, 12:e84296, 2023

    work page 2023

  15. [15]

    Decoding continuous tracking eye movements from cortical spiking activity.International journal of neural systems, 35(1):2450070, 2024

    Kendra K Noneman and J Patrick Mayo. Decoding continuous tracking eye movements from cortical spiking activity.International journal of neural systems, 35(1):2450070, 2024

    work page 2024

  16. [16]

    A large-scale neural network training framework for generalized estimation of single-trial population dynamics.Nature Methods, 19(12):1572–1577, 2022

    Mohammad Reza Keshtkaran, Andrew R Sedler, Raeed H Chowdhury, Raghav Tandon, Diya Basrai, Sarah L Nguyen, Hansem Sohn, Mehrdad Jazayeri, Lee E Miller, and Chethan Pandarinath. A large-scale neural network training framework for generalized estimation of single-trial population dynamics.Nature Methods, 19(12):1572–1577, 2022

    work page 2022

  17. [17]

    Visualizing data using t-sne.Journal of machine learning research, 9(Nov):2579–2605, 2008

    Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne.Journal of machine learning research, 9(Nov):2579–2605, 2008

    work page 2008

  18. [18]

    A comprehensive survey on transfer learning.Proceedings of the IEEE, 109(1):43–76, 2020

    Fuzhen Zhuang, Zhiyuan Qi, Keyu Duan, Dongbo Xi, Yongchun Zhu, Hengshu Zhu, Hui Xiong, and Qing He. A comprehensive survey on transfer learning.Proceedings of the IEEE, 109(1):43–76, 2020

    work page 2020