Pith. sign in

REVIEW 2 major objections 6 minor 12 references

Automatic EEG Independent Component Classification Using ICLabel in Python

T0 review · 2 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A validated Python port of the ICLabel classifier reproduces MATLAB likelihoods to within 0.001 percent on the tested datasets.

desk verdict A clean, credible Python port of ICLabel with solid equivalence testing on epoched data, but the continuous-data code paths are untested and the claims overreach slightly. read the letter →

arxiv 2411.17721 v1 pith:XTCKQ4ZV submitted 2024-11-20 eess.SP cs.LGq-bio.NC

Arnaud Delorme , Dung Truong , Luca Pion-Tonachini , Scott Makeig This is my paper · ORCID
classification eess.SPcs.LGq-bio.NC
keywords ICLabelEEGLABindependentcomponentanalysisEEGartifactclassificationPythonimplementationcross-platformvalidationconvolutionalneuralnetworkhigh-performancecomputing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper reports a Python implementation of the ICLabel classifier, the neural-network tool that assigns each independent component of an EEG decomposition likelihoods over seven source classes: brain, eyes, muscle, heart, line noise, single channel, and other. Its central claim is that the Python port reproduces the original MATLAB implementation almost exactly: on the EEGLAB tutorial dataset and 13 additional subjects from the tutorial study, the largest difference between any class likelihood from the two versions was below $0.001\%$, and no component changed its most-likely class. The practical point is that ICLabel, which could not run under the open-source interpreter used by EEGLAB pipelines, becomes available to Python-based and MATLAB-license-free high-performance computing workflows without giving up comparability with published MATLAB results.

What carries the argument

The load-bearing object is the ICLabel convolutional network itself: three branches process the component's scalp topography, log power spectrum, and autocorrelation, their outputs are reshaped and concatenated, and a final convolutional layer with softmax produces the seven class probabilities. Matching MATLAB required re-implementing its `v4` biharmonic spline interpolation for scalp maps, because interpolation is where scientific libraries differ most; agreement was then measured by the maximum absolute percentage difference per feature across channels, components, and frequencies. The conversion workflow combined AI-assisted translation with manual debugging whenever a candidate translation differed by more than $0.1\%$ from MATLAB.

What would settle it

Take a continuous EEG recording, split it into more than five 3-second epochs, run the complete ICLabel pipeline in both MATLAB and Python, and compute the maximum absolute percentage difference in autocorrelation and in the seven final likelihoods; a difference above the paper's reported bounds, or a component whose highest-likelihood class differs between languages, would disprove the equivalence claim as stated.

Watch

Extended reading notes

Core claim

The discovery is that a line-by-line conversion of ICLabel can be exact enough to be treated as interchangeable with the MATLAB original on the datasets tested. For the classifier's three input features, the maximum absolute percentage differences between MATLAB and Python were below $10^{-5}\%$ for interpolated scalp topographies, $10^{-3}\%$ for log power spectral densities, and $0.01\%$ for autocorrelations; when both neural networks were fed the same precomputed features the difference was below $10^{-5}\%$; and the full pipeline produced likelihood differences below $0.001\%$. In both the tutorial dataset and the 13-subject study, assigning each component to its highest-likelihood class gave no disagreement between Python and MATLAB. The paper treats these numbers as evidence that the port is faithful to the original.

Load-bearing premise

The equivalence is demonstrated only on data that already consists of extracted epochs; the Python and MATLAB code paths for continuous recordings — the Welch-based method used with more than five 3-second epochs and the time-based method used with fewer — are never executed, so their identical behavior is assumed rather than verified.

Editorial extensions

If this is right

  • ICLabel can be applied in Python-only pipelines, including high-performance computing systems where MATLAB licenses are unavailable.
  • EEG data structures from Python-based EEG environments can be converted into the expected format, so the same seven-class classifier can serve tools built around either language.
  • Class likelihoods from the two implementations can be treated as interchangeable for practical purposes, with the largest observed end-to-end difference below $0.001\%$.
  • Published MATLAB-based ICLabel results can be reproduced or extended without a MATLAB installation, provided the same ICA decomposition is used.
  • The validated port supplies a reference point that the earlier, unvalidated Python derivative lacked.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the equivalence shown here is for already-epoched data; the autocorrelation branches used for continuous recordings are untested, so a careful user should rerun the comparison once on continuous data before trusting identical behavior there.
  • Editorial extension: a sharper test is to push both implementations with many components and long continuous recordings, where the Welch-based autocorrelation path is active, and check whether the $0.001\%$ bound still holds.
  • Editorial extension: the 'no misclassification' claim concerns only the argmax class; workflows that threshold low-likelihood classes or use the full probability vector should verify that small probability differences do not change their decisions.
  • Editorial extension: the exact match depends on reproducing MATLAB's interpolation and spectral routines; any future port that substitutes default Python interpolation will likely need the same level of care.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. This paper reports a Python reimplementation of the ICLabel EEG independent-component classifier, converted from the MATLAB plugin with AI assistance and manual debugging. The authors compare the two implementations at the feature level (scalp topographies, power spectral densities, autocorrelations), at the neural-network level, and in a full-pipeline comparison on the EEGLAB tutorial dataset plus the Memory condition of the EEGLAB STERN tutorial study (13 subjects). They report maximum relative differences below 0.001% in final class likelihoods and no disagreements in the highest-likelihood class assignments.

Significance. The manuscript addresses a practical need: a validated, open-source Python implementation of ICLabel that can run outside MATLAB and integrate with MNE-based workflows. The validation strategy is sound and appropriately direct, comparing the same features and the full pipeline against the reference MATLAB implementation with no fitted parameters in the comparison itself. The reported agreement is strong on the tested data, and the availability of the code on GitHub is a useful contribution. The paper's scope is a software-validation note rather than a methodological advance, so the significance is moderate but real for the EEG preprocessing community.

major comments (2)
  1. [Results C and Methods B] The validation exercises only the epoched-data autocorrelation path. Both test sets (eeglab_data_epochs_ica.set and the STERN Memory condition) are epoched, so the eeg_autocorr_fftw branch is the only autocorrelation implementation compared end-to-end. Results C states that continuous data uses either eeg_autocorr_welch (more than five 3-sec epochs) or eeg_autocorr (five or fewer), and the paper gives no numerical comparison for either branch. Because the three functions differ in window extraction and spectral estimation, the Abstract's broad claim that the Python implementation reproduces MATLAB ICLabel is not yet established for continuous recordings. I recommend either adding continuous-data comparisons or explicitly restricting the equivalence claim to epoched data throughout the abstract and discussion.
  2. [Methods C] The 'maximum percentage absolute difference' metric is not unambiguously defined. The displayed formula is garbled, and it is unclear whether the denominator is the MATLAB feature value at each index (which would make percentages unstable near zero, e.g., for interpolated scalp maps or likelihoods close to zero), the maximum absolute value over the feature set, or some other normalization. Since every quantitative claim in Results A-D uses this metric, please state the definition explicitly with a numbered equation and the exact MATLAB/Python code used to compute it, and report whether any floor or outlier handling was applied.
minor comments (6)
  1. [Abstract] The phrase 'differences in classification percentage below 0.001%' conflates class-likelihood differences with classification outcomes; the paper separately reports that no highest-likelihood class assignments differed, and the abstract should use 'class likelihoods' for the 0.001% figure.
  2. [Results B] The text 'the maximum absolute difference ... was 10^-3 percent' should read 'was below 10^-3 percent' or 'was on the order of 10^-3 percent,' since an exact equality is unlikely and inconsistent with Figure 3.
  3. [Results A] The sentence reporting the tutorial dataset gives a mean difference while the next sentence gives a maximum difference; please clarify which summary statistic is being reported for each dataset and correct 'all datasets of the EEGLAB tutorial study' to 'all subjects of the EEGLAB tutorial study.'
  4. [Methods A and C] The 0.1% threshold used to decide when to debug is not tied to the metric defined in Methods C; explain whether this is the same percentage-difference measure or a separate acceptance threshold.
  5. [Code Availability] Please report the exact versions of MATLAB, EEGLAB, Python, PyTorch, and the commit hash of the iclabel_python repository used for the comparison, so that readers can reproduce the reported numbers.
  6. [Figure 3 caption] The caption says 'Maximum likelihood difference histograms,' but the panels show differences in topography, PSD, and autocorrelation features; adjust the wording to describe the feature-level differences actually plotted.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the Python ICLabel port is validated against the external MATLAB implementation through empirical comparisons, with no fitted parameter or definitional identity forcing the result.

full rationale

The paper's central claim is that a newly written Python implementation reproduces the output of the existing MATLAB ICLabel classifier. The load-bearing evidence is a direct numerical comparison between the two implementations on the EEGLAB tutorial dataset and the 13-subject STERN study Memory condition. This is an external-reference validation, not a derivation from inputs that already contain the conclusion: the MATLAB outputs serve as ground truth, the Python code was re-implemented (including the difficult v4 scalp-map interpolation), and agreement below 0.001% likelihood difference was measured rather than assumed. No parameter is fitted to the comparison data, no prediction is renamed from a fit, and no uniqueness theorem or author-derived ansatz is invoked to force the equivalence. The only self-references are to prior EEGLAB/ICLabel work, but these are appropriate citations to the system being ported and to the datasets' origin; they do not substitute for the empirical comparison. A genuine limitation is that the continuous-data autocorrelation branches (eeg_autocorr_welch and eeg_autocorr) are not exercised by either validation dataset, so the full generality of the equivalence claim is not demonstrated for those paths. That is a coverage gap and a correctness risk, not circularity, because the untested branches are not defined in terms of the tested output nor fitted to make the comparison pass. The manuscript itself describes these functions as 'similar' without presenting numerical evidence for them, but the absence of evidence for one branch does not make the tested results circular. Overall, the derivation chain is self-contained and empirically grounded, so the circularity score is 0.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

No free parameters are fitted. The paper relies on the MATLAB reference behavior as the standard and on the representativeness of the two tutorial datasets. No new entities are postulated.

assumptions (2)
  • domain assumption The MATLAB ICLabel implementation is treated as ground truth for equivalence.
    The whole validation compares Python against MATLAB output; if the MATLAB implementation itself were incorrect or version-dependent, equivalence would not imply correctness. This is standard for software port validation.
  • domain assumption The EEGLAB tutorial dataset and STERN study data are assumed representative of ICLabel inputs.
    The paper draws general equivalence conclusions from two epoched datasets; this assumption is not separately tested and is the basis for the weakest assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Automatic EEG Independent Component Classification Using ICLabel in Python." pith.science (2026). https://pith.science/paper/XTCKQ4ZV

@misc{pith2026241117721,
  author       = {Pith},
  title        = {Pith review of: Automatic EEG Independent Component Classification Using ICLabel in Python},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XTCKQ4ZV}},
  note         = {Machine review of arXiv:2411.17721}
}
read the original abstract

ICLabel is an important plug-in function in EEGLAB, the most widely used software for EEG data processing. A powerful approach to automated processing of EEG data involves decomposing the data by Independent Component Analysis (ICA) and then classifying the resulting independent components (ICs) using ICLabel. While EEGLAB pipelines support high-performance computing (HPC) platforms running the open-source Octave interpreter, the ICLabel plug-in is incompatible with Octave because of its specialized neural network architecture. To enhance cross-platform compatibility, we developed a Python version of ICLabel that uses standard EEGLAB data structures. We compared ICLabel MATLAB and Python implementations to data from 14 subjects. ICLabel returns the likelihood of classification in 7 classes of components for each ICA component. The returned IC classifications were virtually identical between Python and MATLAB, with differences in classification percentage below 0.001%.

Figures

Figures reproduced from arXiv: 2411.17721 by the authors.

Figure 1
Figure 1. Workflow for AI-assisted conversion of MATLAB code to Python code. Here, an example add.m MATLAB function is converted into a Python function, add.py. A script (call_add.m) to call the MATLAB function is also converted into an equivalent Python script (call_add.py). The converted Python script saves the result to a .mat file for compatibility with MATLAB. The comparison step involves running both the original MATLAB… view at source ↗
Figure 3
Figure 3. Maximum likelihood difference histograms [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 5
Figure 5. Python script to call ICLabel on an EEGLAB [PITH_FULL_IMAGE:figures/full_fig_p004_5.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Results of the ICLabel pipeline in MATLAB a [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 6 canonical work pages

  1. [1]

    and Makeig, S

    Delorme, A. and Makeig, S. (2004). Eeglab: an open source toolbox for analysis of single-trial eeg dynamics including independent component analysis. Journal of Neuroscience Methods, 134(1), 9-21. https://doi.org/10.1016/j.jneumeth.2003.10.009

    doi:10.1016/j.jneumeth.2003.10.009 2004

  2. [2]

    & Makeig, S

    Delorme, A., Mullen, T., Kothe, C., Acar, Z., Bigdely-Shamlo, N., Vankov, A. & Makeig, S. (2011). Eeglab, sift, nft, bcilab, and erica: new tools for advanced eeg processing. Computational Intelligence and Neuroscience, 2011, 1-12. https://doi.org/10.1155/2011/130714

    work page doi:10.1155/2011/130714 2011

  3. [3]

    Fayaz, M. (2023). The bibliometric analysis of eeglab software in the web of science indexed articles.. https://doi.org/10.20944/preprints202310.0940.v1

    work page arXiv 2023

  4. [4]

    The Art of Brainwaves: A Survey on Event-Related Potential Visualization Practices

    Mikheev V, Skukies R, Ehinger BV. The Art of Brainwaves: A Survey on Event-Related Potential Visualization Practices. Aperture Neuro. 2024;4. doi:10.52294/001c.116386

    work page doi:10.52294/001c.116386 2024

  5. [5]

    ICLabel: An automated electroencephalographic independent component classifier, dataset, and website

    Pion-Tonachini L, Kreutz-Delgado K, Makeig S. ICLabel: An automated electroencephalographic independent component classifier, dataset, and website. Neuroimage. 2019 Sep;198:181-197. doi: 10.1016/j.neuroimage.2019.05.026. Epub 2019 May 16. PMID: 31103785; PMCID: PMC6592775

    doi:10.1016/j.neuroimage.2019.05.026 2019

  6. [6]

    EEG is better left alone

    Delorme A. EEG is better left alone. Sci Rep. 2023 Feb 9;13(1):2372. doi: 10.1038/s41598-023-27528-0. PMID: 36759667; PMCID: PMC9911389

    doi:10.1038/s41598-023-27528-0 2023

  7. [7]

    MNE-ICALabel, Python package for labeling independent components, https://mne.tools/mne-icalabel/stable/index.html

    work page

  8. [8]

    MEG and EEG data analysis with MNE-Python

    Gramfort A, Luessi M, Larson E, Engemann DA, Strohmeier D, Brodbeck C, Goj R, Jas M, Brooks T, Parkkonen L, Hämäläinen M. MEG and EEG data analysis with MNE-Python. Front Neurosci. 2013 Dec 26;7:267. doi: 10.3389/fnins.2013.00267. PMID: 24431986; PMCID: PMC3872725

    arXiv 2013

Show all 12 references
  1. [9]

    An information-maximization approach to blind separation and blind deconvolution

    Bell AJ, Sejnowski TJ. An information-maximization approach to blind separation and blind deconvolution. Neural Comput. 1995 Nov;7(6):1129-59. doi: 10.1162/neco.1995.7.6.1129. PMID: 7584893

    1995 doi

  2. [10]

    Onton, J., Delorme, A., and Makeig, S. (2005). Frontal midline EEG dynamics during working memory. Neuroimage 27, 241-356. https://doi.org/10.1016/j.neuroimage.2005.04.014. Datasets (Sternberg task) are available at https://eeglab.org/tutorials/tutorial_data.html

    2005 doi

  3. [11]

    W., Bateman, D., Hauberg, S., & Wehbring, R

    Eaton, J. W., Bateman, D., Hauberg, S., & Wehbring, R. (2023). GNU Octave Manual. Network Theory Ltd. ISBN-13: 978-0954612061

    2023

  4. [12]

    Jones, E., Oliphant, T., Peterson, P., & Others. (2001–). SciPy: Open source scientific tools for Python. Retrieved from http://www.scipy.org/

    2001

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.