pith. sign in

arxiv: 2403.15409 · v1 · submitted 2024-03-02 · 📡 eess.SP · cs.LG· q-bio.NC

Coupled generator decomposition for fusion of electro- and magnetoencephalography data

Anders Stevnhoved Olsen , Jesper Duemose Nielsen , Morten M{\o}rup This is my paper

Pith reviewed 2026-05-24 02:55 UTC · model grok-4.3

classification 📡 eess.SP cs.LGq-bio.NC
keywords coupled generator decompositiondata fusionEEG MEGsparse PCAface perceptionmultimodal neuroimagingmultisubject analysisstochastic optimization
0
0 comments X

The pith

Coupled generator decomposition fuses EEG and MEG to isolate shared face responses while revealing 170ms activation differences.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces coupled generator decomposition as a generalization of sparse principal component analysis for data fusion across sources like EEG and MEG recordings from multiple subjects. It applies the method to brain responses in a face perception experiment and finds that activation around 170 milliseconds in the fusiform face area differs between real and scrambled faces, with the difference clearest in the combined multimodal multisubject model. The framework accounts for modality- and subject-specific variability while extracting common stimulus-driven signals. This matters because it offers an efficient way to combine noisy neuroimaging data without forcing all responses to be identical across sources.

Core claim

Coupled generator decomposition identifies common features across diverse data sources while accommodating modality- and subject-specific variability. Leveraging multisubject multimodal EEG and MEG data from face perception stimuli, the approach reveals altered ∼170ms fusiform face area activation for scrambled faces as opposed to real faces, particularly evident in the multimodal multisubject model. Model parameters are inferred using stochastic optimization in PyTorch, which matches conventional quadratic programming performance for SPCA but runs considerably faster. The method is implemented in an accessible toolbox supporting fusion for SPCA, archetypal analysis, and directional archetyp

What carries the argument

Coupled generator decomposition, a generalization of sparse principal component analysis that couples generators across data sources to extract shared components while permitting source-specific variability.

If this is right

  • Split-half cross-validation on EEG/MEG trials identifies optimal model order and regularization strengths for models of varying complexity.
  • The multimodal multisubject model detects the activation difference more clearly than a group-level model assuming fully shared responses.
  • Stochastic optimization achieves comparable accuracy to quadratic programming but with substantially faster execution times.
  • The toolbox enables data fusion extensions to archetypal analysis and directional archetypal analysis in addition to SPCA.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The separation of shared and specific components could be tested on other stimulus categories to determine if the 170ms difference is unique to faces.
  • Extending the coupling to additional recording types such as simultaneous EEG-fMRI might improve spatial localization of the observed effects.
  • Individual subject contributions to the shared components could be examined to study variability in face processing across people.

Load-bearing premise

The coupled generator decomposition framework correctly separates shared stimulus-driven responses from modality- and subject-specific variability without the fusion process itself creating or masking the reported activation differences.

What would settle it

If split-half cross-validated models without coupling show no 170ms fusiform difference or if the difference appears equally in separate per-modality analyses, the claim that fusion specifically isolates the effect would not hold.

Figures

Figures reproduced from arXiv: 2403.15409 by Anders Stevnhoved Olsen, Jesper Duemose Nielsen, Morten M{\o}rup.

Figure 1
Figure 1. Figure 1: Variability in ERP waveform across subjects for a chosen right [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Boxplot of model convergence across stochastic optimization in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Lineplot of model performance across number of components, [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Sparse PCA and archetypal analysis results on data from a multimodal multisubject face perception neuroimaging experiment. The coupled generator [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
read the original abstract

Data fusion modeling can identify common features across diverse data sources while accounting for source-specific variability. Here we introduce the concept of a \textit{coupled generator decomposition} and demonstrate how it generalizes sparse principal component analysis (SPCA) for data fusion. Leveraging data from a multisubject, multimodal (electro- and magnetoencephalography (EEG and MEG)) neuroimaging experiment, we demonstrate the efficacy of the framework in identifying common features in response to face perception stimuli, while accommodating modality- and subject-specific variability. Through split-half cross-validation of EEG/MEG trials, we investigate the optimal model order and regularization strengths for models of varying complexity, comparing these to a group-level model assuming shared brain responses to stimuli. Our findings reveal altered $\sim170ms$ fusiform face area activation for scrambled faces, as opposed to real faces, particularly evident in the multimodal, multisubject model. Model parameters were inferred using stochastic optimization in PyTorch, demonstrating comparable performance to conventional quadratic programming inference for SPCA but with considerably faster execution. We provide an easily accessible toolbox for coupled generator decomposition that includes data fusion for SPCA, archetypal analysis and directional archetypal analysis. Overall, our approach offers a promising new avenue for data fusion.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript introduces coupled generator decomposition as a generalization of sparse principal component analysis (SPCA) for multimodal data fusion. Applied to multisubject EEG/MEG recordings from a face perception experiment, the framework uses split-half cross-validation to select model order and regularization strengths, compares against a group-level shared-response model, and reports an altered ~170 ms fusiform face area activation for scrambled versus intact faces that is particularly evident in the multimodal multisubject model. Inference is performed via stochastic optimization in PyTorch, with a toolbox released that implements data fusion variants of SPCA, archetypal analysis, and directional archetypal analysis.

Significance. If the decomposition accurately isolates shared stimulus-driven components, the work supplies a flexible framework for fusing EEG and MEG while accommodating modality- and subject-specific variability. Positive elements include the public toolbox release, the demonstration of faster PyTorch inference relative to quadratic programming, and the use of cross-validation for hyperparameter selection. The reported timing difference in face processing would strengthen the contribution if the method's fidelity to shared temporal structure is more directly established.

major comments (2)
  1. [Methods (coupled generator decomposition)] Methods section on the coupled generator model: no recovery simulation is presented in which known shared components with controlled timing and topography are injected into synthetic EEG/MEG data to test whether the coupling and regularization terms preserve or distort the ~170 ms profile. This check is load-bearing for the central empirical claim.
  2. [Results (model comparison)] Results on model comparison: the abstract states that the ~170 ms difference is 'particularly evident' in the multimodal multisubject model versus the group-level model, yet no quantitative metrics (effect sizes, cross-validated reconstruction error, or statistical contrast of the activation difference across model classes) are supplied to support the differential claim.
minor comments (2)
  1. [Abstract] Abstract: the toolbox description lists included methods but does not specify which fusion variants (e.g., for archetypal analysis) are actually demonstrated on the EEG/MEG data.
  2. [Methods] Notation: ensure that the definitions of shared versus modality-specific generators are introduced with explicit symbols before their use in the optimization objective.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive review and for highlighting positive aspects of the work such as the toolbox release and cross-validation procedure. We address each major comment below.

read point-by-point responses

  1. Referee: Methods section on the coupled generator model: no recovery simulation is presented in which known shared components with controlled timing and topography are injected into synthetic EEG/MEG data to test whether the coupling and regularization terms preserve or distort the ~170 ms profile. This check is load-bearing for the central empirical claim.

    Authors: We agree that a recovery simulation is a valuable addition to validate preservation of the temporal profile under the coupling and regularization terms. In the revised manuscript we will include a simulation study that injects known shared components with controlled timing and topography into synthetic EEG/MEG data and quantifies recovery of the ~170 ms profile. revision: yes

  2. Referee: Results on model comparison: the abstract states that the ~170 ms difference is 'particularly evident' in the multimodal multisubject model versus the group-level model, yet no quantitative metrics (effect sizes, cross-validated reconstruction error, or statistical contrast of the activation difference across model classes) are supplied to support the differential claim.

    Authors: We acknowledge that quantitative metrics would strengthen the differential claim. In the revised manuscript we will report effect sizes for the activation difference, cross-validated reconstruction errors for each model class, and a statistical contrast of the ~170 ms difference between the multimodal multisubject model and the group-level model. revision: yes

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper introduces coupled generator decomposition as a generalization of SPCA for multimodal fusion, with model parameters inferred via stochastic optimization and validated through split-half cross-validation on EEG/MEG data. The central empirical claim (altered ~170 ms fusiform face area activation) is presented as an observation from fitted models rather than a derived prediction that reduces to the inputs by construction. No equations equate a claimed result to a fitted parameter or self-citation chain; the framework is self-contained against external benchmarks like standard SPCA, with no load-bearing self-citations or ansatz smuggling. The derivation chain remains independent of the reported activation differences.

Axiom & Free-Parameter Ledger

2 free parameters · 1 axioms · 0 invented entities

The framework rests on the modeling assumption that brain responses can be decomposed into coupled generators with shared and modality/subject-specific components; model order and regularization strengths are selected via cross-validation rather than derived from first principles.

free parameters (2)
  • model order
    Optimal model order investigated via split-half cross-validation for models of varying complexity
  • regularization strengths
    Optimal regularization strengths chosen via cross-validation for the coupled decomposition
axioms (1)
  • domain assumption Brain responses to stimuli can be represented as a sum of shared generators plus modality- and subject-specific components
    This decomposition assumption underpins the entire coupled generator framework described in the abstract

pith-pipeline@v0.9.0 · 5765 in / 1166 out tokens · 20910 ms · 2026-05-24T02:55:01.694787+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

19 extracted references · 19 canonical work pages · 1 internal anchor

  1. [1]

    Magnetoencephalography for brain electrophysiology and imaging,

    S. Baillet, “Magnetoencephalography for brain electrophysiology and imaging,” Nature Neuroscience, 2017. DOI: 10.1038/nn.4504

    work page doi:10.1038/nn.4504 2017

  2. [2]

    EEG and MEG: Relevance to neuroscience,

    F. Lopes da Silva, “EEG and MEG: Relevance to neuroscience,” Neuron, 2013. DOI: 10.1016/J.NEURON.2013.10.017

    work page doi:10.1016/j.neuron.2013.10.017 2013

  3. [3]

    MEG and multimodal integration,

    S. P. Ahlfors, “MEG and multimodal integration,” in Magnetoen- cephalography: From Signals to Dynamic Cortical Networks , 2019. DOI: 10.1007/978-3-319-62657-4 7-1

    work page doi:10.1007/978-3-319-62657-4 2019

  4. [4]

    MEG and EEG data fusion: Simultaneous localisation of face-evoked responses,

    R. N. Henson, E. Mouchlianitis, and K. J. Friston, “MEG and EEG data fusion: Simultaneous localisation of face-evoked responses,” NeuroImage, 2009. DOI: 10.1016/j.neuroimage.2009.04.063

    work page doi:10.1016/j.neuroimage.2009.04.063 2009

  5. [5]

    Similarity-based fusion of MEG and fMRI reveals spatio-temporal dynamics in human cortex during visual object recognition,

    R. M. Cichy, D. Pantazis, and A. Oliva, “Similarity-based fusion of MEG and fMRI reveals spatio-temporal dynamics in human cortex during visual object recognition,” Cerebral Cortex, 2016. DOI: 10. 1093/cercor/bhw135

    work page 2016

  6. [6]

    Electromagnetic brain mapping,

    S. Baillet, J. Mosher, and R. Leahy, “Electromagnetic brain mapping,” IEEE Signal Processing Magazine , 2001. DOI: 10.1109/79.962275

    work page doi:10.1109/79.962275 2001

  7. [7]

    MEG–EEG information fusion and elec- tromagnetic source imaging: From theory to clinical application in epilepsy,

    R. A. Chowdhury et al. , “MEG–EEG information fusion and elec- tromagnetic source imaging: From theory to clinical application in epilepsy,” Brain Topography, 2015. DOI: 10.1007/s10548-015-0437- 3

    work page doi:10.1007/s10548-015-0437- 2015

  8. [8]

    Archetypal analysis,

    A. Cutler and L. Breiman, “Archetypal analysis,” Technometrics,

    work page

  9. [9]

    DOI: 10.2307/1269949

    work page doi:10.2307/1269949

  10. [10]

    Archetypal analysis for machine learning and data mining,

    M. Mørup and L. K. Hansen, “Archetypal analysis for machine learning and data mining,” Neurocomputing, 2012. DOI: 10 . 1016 / J.NEUCOM.2011.06.033

    work page 2012

  11. [11]

    Archetypal analysis for modeling multisubject fMRI data,

    J. L. Hinrich et al. , “Archetypal analysis for modeling multisubject fMRI data,” IEEE Journal on Selected Topics in Signal Processing ,

    work page

  12. [12]

    DOI: 10.1109/JSTSP.2016.2595103

    work page doi:10.1109/jstsp.2016.2595103 2016

  13. [13]

    Combining electro- and magnetoencephalography data using directional archetypal analysis,

    A. S. Olsen, R. M. T. Høegh, J. L. Hinrich, K. H. Madsen, and M. Mørup, “Combining electro- and magnetoencephalography data using directional archetypal analysis,” Frontiers in Neuroscience, 2022

    work page 2022

  14. [14]

    Sparse pca, a new method for unsupervised analyses of fmri data,

    K. Sj ¨ostrand, T. E. Lund, K. H. Madsen, and R. Larsen, “Sparse pca, a new method for unsupervised analyses of fmri data,” in Proc. International Society of Magnetic Resonance In Medicine-ISMRM , 2006

    work page 2006

  15. [15]

    Sparse principal component analysis,

    H. Zou, T. Hastie, and R. Tibshirani, “Sparse principal component analysis,” Journal of Computational and Graphical Statistics , 2006

    work page 2006

  16. [16]

    Adam: A Method for Stochastic Optimization

    D. P. Kingma and J. L. Ba, “Adam: A method for stochastic opti- mization,” 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings , 2014. DOI: 10.48550/ arxiv.1412.6980

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  17. [17]

    A multi-subject, multi-modal human neuroimaging dataset,

    D. G. Wakeman and R. N. Henson, “A multi-subject, multi-modal human neuroimaging dataset,” Scientific Data 2015 2:1 , 2015. DOI: 10.1038/sdata.2015.1

    work page doi:10.1038/sdata.2015.1 2015

  18. [18]

    MEG and EEG data analysis with MNE-python,

    A. Gramfort et al., “MEG and EEG data analysis with MNE-python,” Frontiers in Neuroscience, 2013

    work page 2013

  19. [19]

    The neural sources of n170: Understanding timing of activation in face-selective areas,

    C. Gao, S. Conte, J. E. Richards, W. Xie, and T. Hanayik, “The neural sources of n170: Understanding timing of activation in face-selective areas,” Psychophysiology, 2019. DOI: 10.1111/psyp.13336

    work page doi:10.1111/psyp.13336 2019