{"id":"0f4d46d6-c859-45a9-83de-29a80f1064b8","arxiv_id":"2411.17721","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"A Python implementation of the EEG IC classifier ICLabel reproduces the MATLAB version's outputs within 0.001% on two EEGLAB tutorial datasets.","lead":"This paper reports a Python version of ICLabel, a popular tool that sorts brain versus artifact signals in EEG data. The Python version matches the original MATLAB classifier almost exactly on standard test data, making it usable on high-performance computers without a MATLAB license.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The equivalence claim is validated only on epoched data; the continuous-data autocorrelation branches (eeg_autocorr_welch/eeg_autocorr) are never tested, so a bug there would invalidate the general claim.","rationale":"The paper's measured differences on the tutorial dataset and the 13 STERN subjects are direct, useful evidence for the epoched-data path, and the 0.001% likelihood agreement and zero misclassifications are genuinely supportive. My concern is not with those measurements but with the scope of the claim drawn from them. The reader's weakest assumption correctly identifies that the continuous-data autocorrelation branches are untested, and this is the most load-bearing gap because the paper's stated motivation is a general Python replacement usable on HPC platforms, where continuous EEG files are common. The relevant code paths are not identical in structure, so 'similar' does not guarantee numerical equivalence. The proposed test would settle the issue by directly comparing the untested branches on continuous data. Since the reader already assigned CONDITIONAL on exactly this basis, my read does not change the verdict. Secondary reproducibility concerns, such as the lack of a pinned commit hash, reinforce the conditional status but are not the central scientific gap.","tokens_in":5986,"tokens_out":3691,"duration_ms":36590,"concrete_test":"Run both MATLAB and Python ICLabel on continuous EEG datasets that exercise the untested branches: (i) a continuous file with more than five 3-sec epochs, forcing eeg_autocorr_welch, and (ii) a continuous file with five or fewer 3-sec epochs, forcing eeg_autocorr. Compare the raw autocorrelation feature vectors and the final 7-class likelihoods using the same maximum percentage absolute difference defined in Methods C, and count any maximum-likelihood class flips. If either branch shows differences above 0.001% or any class flip, the general equivalence claim must be narrowed to epoched data.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central equivalence claim is supported only through eeg_autocorr_fftw, the epoched-data path. Both validation datasets are epoched: the EEGLAB tutorial dataset (eeglab_data_epochs_ica.set) and the STERN study Memory condition. Results C, however, states that continuous data uses either eeg_autocorr_welch (more than five 3-sec epochs) or eeg_autocorr (five or fewer epochs), and no comparison in the paper exercises either branch. The paper asserts the three functions are 'similar', but that is an assertion, not evidence of numerical equivalence. These branches differ in window extraction and in the time-based versus Welch spectral implementation, where an FFT-scaling, window-normalization, overlap, or segment-count bug could alter the autocorrelation features and propagate through the neural network to change likelihoods. Without a continuous-data test, the abstract's broad claim that the Python implementation reproduces MATLAB ICLabel is not established for the full input space, and users processing continuous recordings could silently receive different classifications than the MATLAB version.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper reports a Python reimplementation of the ICLabel EEG independent-component classifier, converted from the MATLAB plugin with AI assistance and manual debugging. The authors compare the two implementations at the feature level (scalp topographies, power spectral densities, autocorrelations), at the neural-network level, and in a full-pipeline comparison on the EEGLAB tutorial dataset plus the Memory condition of the EEGLAB STERN tutorial study (13 subjects). They report maximum relative differences below 0.001% in final class likelihoods and no disagreements in the highest-likelihood class assignments.","tokens_in":6179,"tokens_out":5676,"duration_ms":55139,"significance":"The manuscript addresses a practical need: a validated, open-source Python implementation of ICLabel that can run outside MATLAB and integrate with MNE-based workflows. The validation strategy is sound and appropriately direct, comparing the same features and the full pipeline against the reference MATLAB implementation with no fitted parameters in the comparison itself. The reported agreement is strong on the tested data, and the availability of the code on GitHub is a useful contribution. The paper's scope is a software-validation note rather than a methodological advance, so the significance is moderate but real for the EEG preprocessing community.","major_comments":[{"comment":"The validation exercises only the epoched-data autocorrelation path. Both test sets (eeglab_data_epochs_ica.set and the STERN Memory condition) are epoched, so the eeg_autocorr_fftw branch is the only autocorrelation implementation compared end-to-end. Results C states that continuous data uses either eeg_autocorr_welch (more than five 3-sec epochs) or eeg_autocorr (five or fewer), and the paper gives no numerical comparison for either branch. Because the three functions differ in window extraction and spectral estimation, the Abstract's broad claim that the Python implementation reproduces MATLAB ICLabel is not yet established for continuous recordings. I recommend either adding continuous-data comparisons or explicitly restricting the equivalence claim to epoched data throughout the abstract and discussion.","section":"Results C and Methods B"},{"comment":"The 'maximum percentage absolute difference' metric is not unambiguously defined. The displayed formula is garbled, and it is unclear whether the denominator is the MATLAB feature value at each index (which would make percentages unstable near zero, e.g., for interpolated scalp maps or likelihoods close to zero), the maximum absolute value over the feature set, or some other normalization. Since every quantitative claim in Results A-D uses this metric, please state the definition explicitly with a numbered equation and the exact MATLAB/Python code used to compute it, and report whether any floor or outlier handling was applied.","section":"Methods C"}],"minor_comments":[{"comment":"The phrase 'differences in classification percentage below 0.001%' conflates class-likelihood differences with classification outcomes; the paper separately reports that no highest-likelihood class assignments differed, and the abstract should use 'class likelihoods' for the 0.001% figure.","section":"Abstract"},{"comment":"The text 'the maximum absolute difference ... was 10^-3 percent' should read 'was below 10^-3 percent' or 'was on the order of 10^-3 percent,' since an exact equality is unlikely and inconsistent with Figure 3.","section":"Results B"},{"comment":"The sentence reporting the tutorial dataset gives a mean difference while the next sentence gives a maximum difference; please clarify which summary statistic is being reported for each dataset and correct 'all datasets of the EEGLAB tutorial study' to 'all subjects of the EEGLAB tutorial study.'","section":"Results A"},{"comment":"The 0.1% threshold used to decide when to debug is not tied to the metric defined in Methods C; explain whether this is the same percentage-difference measure or a separate acceptance threshold.","section":"Methods A and C"},{"comment":"Please report the exact versions of MATLAB, EEGLAB, Python, PyTorch, and the commit hash of the iclabel_python repository used for the comparison, so that readers can reproduce the reported numbers.","section":"Code Availability"},{"comment":"The caption says 'Maximum likelihood difference histograms,' but the panels show differences in topography, PSD, and autocorrelation features; adjust the wording to describe the feature-level differences actually plotted.","section":"Figure 3 caption"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a straightforward software-validation note, and the core comparison design is good. The main technical gap is that the full-pipeline equivalence claim is supported only for epoched data, while the paper does not test the continuous-data autocorrelation branches; the under-specified difference metric also needs clarification. Both issues are fixable within the scope of the paper, either by adding tests or by narrowing the claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core contribution here is a validated Python implementation of ICLabel, with direct equivalence testing against the MATLAB version on public datasets. That is genuinely useful: it removes the MATLAB license requirement for automated IC classification in Python/HPC workflows, and it does so more rigorously than the existing MNE-ICALabel derivative, which has no formal validation. The re-implementation of the v4 spline interpolation is the hard part, and they got it to within 1e-5 percent. The feature-level comparisons (topography, PSD, autocorrelation) and the full-pipeline likelihood comparison are the right way to test a port, and the reported differences are tiny, with no classification disagreements on the test data. The paper is honest about using AI-assisted conversion and gives the code.\n\nThe main soft spot is the coverage of the autocorrelation code paths. Both test datasets are epoched, so only eeg_autocorr_fftw is exercised. The continuous-data branches — eeg_autocorr_welch for more than five 3-sec epochs and eeg_autocorr for fewer — are never tested. The paper says these functions are \"similar\" and that eeg_autocorr is \"implemented as eeg_autocorr_welch,\" but that is an assertion, not evidence. A bug in window extraction, Welch scaling, or segment handling could silently change features and propagate through the network. The abstract's broad \"virtually identical\" claim needs either a continuous-data test or a qualifier that only epoched data was validated. This is a moderate issue, not a disproof: the feature computations are standard and the network is likely robust, but the gap is real and easily fixable. A second, minor issue is that the GitHub link has no pinned commit hash, which weakens reproducibility.\n\nThe paper is a tooling contribution, not a new scientific result about EEG or classification. The target reader is someone building Python-based EEG pipelines, especially on HPC clusters without MATLAB licenses. It deserves a serious referee, though the revision should add at least one continuous-data comparison or explicitly scope the claims, and pin the code version. If they close that gap, I'd be comfortable recommending acceptance.","headline":"A clean, credible Python port of ICLabel with solid equivalence testing on epoched data, but the continuous-data code paths are untested and the claims overreach slightly.","tokens_in":635,"tokens_out":1432,"would_cite":true,"duration_ms":30284,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A validated Python port of the ICLabel classifier reproduces MATLAB likelihoods to within 0.001 percent on the tested datasets.","keywords":["ICLabel","EEGLAB","independent component analysis","EEG artifact classification","Python implementation","cross-platform validation","convolutional neural network","high-performance computing"],"falsifier":"Take a continuous EEG recording, split it into more than five 3-second epochs, run the complete ICLabel pipeline in both MATLAB and Python, and compute the maximum absolute percentage difference in autocorrelation and in the seven final likelihoods; a difference above the paper's reported bounds, or a component whose highest-likelihood class differs between languages, would disprove the equivalence claim as stated.","tokens_in":5801,"feed_emoji":"🧠","tokens_out":9294,"duration_ms":80375,"temperature":0.7,"pith_summary":"This paper reports a Python implementation of the ICLabel classifier, the neural-network tool that assigns each independent component of an EEG decomposition likelihoods over seven source classes: brain, eyes, muscle, heart, line noise, single channel, and other. Its central claim is that the Python port reproduces the original MATLAB implementation almost exactly: on the EEGLAB tutorial dataset and 13 additional subjects from the tutorial study, the largest difference between any class likelihood from the two versions was below $0.001\\%$, and no component changed its most-likely class. The practical point is that ICLabel, which could not run under the open-source interpreter used by EEGLAB pipelines, becomes available to Python-based and MATLAB-license-free high-performance computing workflows without giving up comparability with published MATLAB results.","feed_headline":"Python ICLabel matches MATLAB scores to below 0.001 percent","feed_subtitle":"Python port of ICLabel matches MATLAB scores below 0.001 percent on 14 datasets, so no MATLAB license is needed.","key_machinery":"The load-bearing object is the ICLabel convolutional network itself: three branches process the component's scalp topography, log power spectrum, and autocorrelation, their outputs are reshaped and concatenated, and a final convolutional layer with softmax produces the seven class probabilities. Matching MATLAB required re-implementing its `v4` biharmonic spline interpolation for scalp maps, because interpolation is where scientific libraries differ most; agreement was then measured by the maximum absolute percentage difference per feature across channels, components, and frequencies. The conversion workflow combined AI-assisted translation with manual debugging whenever a candidate translation differed by more than $0.1\\%$ from MATLAB.","core_discovery":"The discovery is that a line-by-line conversion of ICLabel can be exact enough to be treated as interchangeable with the MATLAB original on the datasets tested. For the classifier's three input features, the maximum absolute percentage differences between MATLAB and Python were below $10^{-5}\\%$ for interpolated scalp topographies, $10^{-3}\\%$ for log power spectral densities, and $0.01\\%$ for autocorrelations; when both neural networks were fed the same precomputed features the difference was below $10^{-5}\\%$; and the full pipeline produced likelihood differences below $0.001\\%$. In both the tutorial dataset and the 13-subject study, assigning each component to its highest-likelihood class gave no disagreement between Python and MATLAB. The paper treats these numbers as evidence that the port is faithful to the original.","pith_inferences":["Editorial extension: the equivalence shown here is for already-epoched data; the autocorrelation branches used for continuous recordings are untested, so a careful user should rerun the comparison once on continuous data before trusting identical behavior there.","Editorial extension: a sharper test is to push both implementations with many components and long continuous recordings, where the Welch-based autocorrelation path is active, and check whether the $0.001\\%$ bound still holds.","Editorial extension: the 'no misclassification' claim concerns only the argmax class; workflows that threshold low-likelihood classes or use the full probability vector should verify that small probability differences do not change their decisions.","Editorial extension: the exact match depends on reproducing MATLAB's interpolation and spectral routines; any future port that substitutes default Python interpolation will likely need the same level of care."],"forward_implications":["ICLabel can be applied in Python-only pipelines, including high-performance computing systems where MATLAB licenses are unavailable.","EEG data structures from Python-based EEG environments can be converted into the expected format, so the same seven-class classifier can serve tools built around either language.","Class likelihoods from the two implementations can be treated as interchangeable for practical purposes, with the largest observed end-to-end difference below $0.001\\%$.","Published MATLAB-based ICLabel results can be reproduced or extended without a MATLAB installation, provided the same ICA decomposition is used.","The validated port supplies a reference point that the earlier, unvalidated Python derivative lacked."],"supporting_citations":[{"why":"Supplies the EEGLAB data structures and the tutorial dataset used as the first validation case.","marker":"[1]"},{"why":"Defines the original ICLabel classifier, its seven classes, and the trained network that the Python port reproduces.","marker":"[5]"},{"why":"Provides the Infomax ICA algorithm whose precomputed component weights the test datasets use.","marker":"[9]"},{"why":"Provides the 13-subject STERN study dataset used for the larger validation comparison.","marker":"[10]"},{"why":"Provides the file interface used by the converted Python code to exchange MATLAB-compatible data.","marker":"[12]"}],"fun_headline_variants":["Python ICLabel matches MATLAB to <0.001%","ICLabel Python port: identical to MATLAB, no license needed","Exact ICLabel replication: Python vs MATLAB under 0.001%","Python ICLabel: same results as MATLAB, no Octave workaround"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The equivalence is demonstrated only on data that already consists of extracted epochs; the Python and MATLAB code paths for continuous recordings — the Welch-based method used with more than five 3-second epochs and the time-based method used with fewer — are never executed, so their identical behavior is assumed rather than verified.","fun_headline_variants_meta":{"raw":{"variants":["Python ICLabel matches MATLAB to <0.001%","ICLabel Python port: identical to MATLAB, no license needed","Exact ICLabel replication: Python vs MATLAB under 0.001%","Python ICLabel: same results as MATLAB, no Octave workaround"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000515,"raw_usage":{"total_tokens":2464,"prompt_tokens":870,"completion_tokens":1594,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":486,"completion_tokens_details":{"reasoning_tokens":1516}},"tokens_in":486,"tokens_out":1594,"duration_ms":12417,"temperature":1.0,"reasoning_tokens":1516,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:55:45.204806+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a continuous EEG recording, split it into more than five 3-second epochs, run the complete ICLabel pipeline in both MATLAB and Python, and compute the maximum absolute percentage difference in autocorrelation and in the seven final likelihoods; a difference above the paper's reported bounds, or a component whose highest-likelihood class differs between languages, would disprove the equivalence claim as stated.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the file interface used by the converted Python code to exchange MATLAB-compatible data."}],"review_version":1}