{"id":"48a8f2cd-dc99-44e2-930f-b5ff5d388f11","arxiv_id":"2607.21119","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A dispersion-based framework quantifies temporal, spatial, and frequency ERD/S variability at within-trial, between-trial, and between-group levels, with correlations of −0.2 to −0.4 to BCI classification performance.","lead":"BCI performance suffers from variability in users' brain signals. This paper proposes a unified set of interpretable metrics that separately quantify temporal, spatial, and frequency variability in motor-imagery EEG, and shows that lower variability correlates with better classification accuracy.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Independence of modality metrics is unverified: marginal projections in Eqs. (2)–(4) allow cross-modal leakage, so the claim to 'independently quantify' temporal, spatial, and frequency variability is not established.","rationale":"The reader's weakest assumption identifies the independence of the modality metrics as the most fragile point. My analysis agrees: the operators in Eqs. (2)–(4) are averages over all dimensions except the target, so they are marginal projections. Changes in one modality can affect the projections of another, and the paper provides no control or simulation to show this contamination is negligible. This is load-bearing because the central claim of the paper is not merely that these metrics correlate with performance, but that they independently quantify distinct types of variability, which is the proposed framework's main novelty. If independence fails, the interpretability and the modality-specific conclusions are undermined. A controlled simulation with known ground truth is the most direct way to settle this. The reader's verdict of CONDITIONAL is appropriate; my concern reinforces the need for such a test before the independence claim can be accepted. I therefore recommend keeping the verdict as CONDITIONAL, since the issue is not a proven flaw but an unverified assumption that substantially affects the interpretation of the results.","tokens_in":17823,"tokens_out":5577,"duration_ms":57780,"concrete_test":"Simulate ERD/S tensors with known, independent temporal, spatial, and frequency variability. For each trial or sub-epoch, generate X_i[c,f,t] = μ[c,f,t] + ε_t(t) + ε_c(c) + ε_f(f) + noise, where the ε terms are drawn from zero-mean distributions with controlled variances. Create conditions that increase only temporal, only spatial, or only frequency variability (e.g., multiply the spatial pattern by a sub-epoch-specific vector, shift the spectral centroid, or add a time-varying amplitude). Compute WiTrialTemp, WiTrialSpat, and WiTrialFreq for each condition. Test whether each metric changes significantly only when its own modality's variance is increased and remains constant when other modalities vary (e.g., via ANOVA or permutation tests). If increasing spatial variance changes WiTrialTemp, the independence claim fails. This directly tests the definitions in Eqs. (2)–(4).","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central methodological claim is that the operators φt, φs, φf (Eqs. 2–4) independently quantify temporal, spatial, and frequency variability. But each operator is a marginal projection. For example, φt(Z)[t] = (1/CF) Σ_c Σ_f Z[c,f,t] averages over all channels and frequency bins. A change in spatial pattern across sub-epochs will alter this average even if each channel's temporal waveform is unchanged. Conversely, temporal power fluctuations can change the spatial vector φs, since averaging over time weights each channel by its mean power. Frequency shifts can also affect the spatial vector if different frequencies have different topographies. The angular distance in Eq. (6) removes only overall amplitude scaling, not these cross-modal couplings. The paper provides no evidence that such contamination is negligible in real EEG, where volume conduction and source mixing make cross-modal dependencies the norm. Thus the 'independently quantify' claim is not supported, and attributing a high temporal variability score to a purely temporal phenomenon is unjustified. This is load-bearing because the framework's novelty over existing metrics (e.g., STDERD, class stability) and its interpretability rest on this separation. Furthermore, the correlation analyses (e.g., WiTrialTemp vs. performance) may be measuring a mixture of modalities, weakening conclusions about which specific type of variability affects BCI performance.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a unified framework for quantifying temporal, spatial, and frequency variability of event-related (de)synchronization (ERD/S) in EEG-based BCIs. Variability is defined as the average distance between extracted ERD/S slices and a reference centroid, with three operators (Eqs. 2–4) that average over all dimensions except the target modality, yielding nine metrics at within-trial, between-trial, and between-trial-group levels. Using two motor imagery datasets (Dreyer2023, Lee2019; N=133 after exclusions), the authors compute Spearman correlations between these metrics and BCI classification performance for within-user and cross-user settings, with Benjamini–Hochberg correction and bootstrap standard errors. They report modest negative correlations (−0.2 to −0.4) across most conditions, suggesting lower variability is associated with better performance, and they compare the proposed metrics against existing variability metrics (STDERD, class stability, MITT). The framework is released as an open-source Python package.","tokens_in":18228,"tokens_out":5041,"duration_ms":53699,"significance":"If the central claim holds, the framework provides a useful toolbox for characterizing EEG variability in MI-BCI, with a flexible and interpretable formulation that could support dataset comparison, classifier robustness analysis, and variability-aware user training. The use of two public datasets, explicit statistical corrections, and an open-source release are notable strengths. However, the headline claim that the metrics 'independently quantify' temporal, spatial, and frequency variability is not established by the marginal-projection definitions, and the correlational validation would benefit from a non-variability control and a more careful treatment of test-user-referenced metrics. The contribution is potentially significant for the BCI community, but the current evidence is weaker than the abstract suggests.","major_comments":[{"comment":"","section":"Section II, Eqs. (2)–(4) and Section II-4"},{"comment":"","section":"Section IV and Section V-A (correlational validation)"},{"comment":"","section":"Section IV, Fig. 3 and Section V-B (BtwTrialGrp-TR)"},{"comment":"","section":"Section II-B1a (WiTrialTemp)"}],"minor_comments":[{"comment":"","section":"Throughout"},{"comment":"","section":"Section III-B"},{"comment":"","section":"Section II-3"},{"comment":"","section":"Section IV, Fig. 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical study with a useful open-source contribution, but the central 'independence' claim is not adequately supported by the mathematical definitions. I see the independence issue as a correctness-risk concern rather than a mere wording problem, and it should be addressed with additional analyses or a revised claim before publication. The correlational results are also not as consistent as the abstract implies, and the lack of a non-variability control baseline is a substantive gap. I recommend major revision rather than rejection, as the framework and empirical effort are valuable and the issues are addressable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my read on arXiv:2607.21119. This one is worth reading if you work on EEG variability or BCI user training. The core proposal is simple: define variability as average distance of ERD/S slices to a centroid, and instantiate that with different operators, distance functions, and hierarchical levels. That yields nine metrics covering within-trial, between-trial, and between-trial-group variability across temporal, spatial, and frequency domains. Most of the building blocks are standard — variance, angular distance, DTW, Fréchet means — but the unified framework and the explicit comparison to existing metrics (STDERD, class stability, MITT) are a real contribution. The empirical work is solid: two open datasets, N=133, two classifiers, within- and cross-user, BH-corrected p-values, bootstrap SEs. The discussion is honest about MITT's confound with class separability and about the speculative causal reading of the correlations.\n\nWhere I'd push back is the claim that the metrics \"independently\" quantify the three modalities. The operators φt, φs, φf are marginal projections — each averages over the other two dimensions. That means a temporal variability score can move because spatial or spectral patterns shift over time, not because the time course actually changed at any single channel. The paper gives no evidence that this leakage is negligible in real EEG, where volume conduction makes cross-modal coupling the default. This is not a fatal flaw — the metrics still measure something interpretable — but the \"independent\" wording is stronger than the math supports. The easiest fix is to call them \"projected\" or \"marginal\" variability, or add a synthetic-data experiment where you generate known temporal-only changes and show the spatial metric doesn't move.\n\nSecond, the correlational story would be stronger with a control. The observed negative correlations (around -0.2 to -0.4) could be partly confounded with overall ERD amplitude or signal quality: subjects with stronger desynchronization often have both cleaner responses and higher accuracy. A partial correlation controlling for mean ERD amplitude, or a permutation baseline on a shuffled metric, would help.\n\nMinor: the text says a Python package is released but I couldn't find a link in this version. Please add it. And note WiTrialTemp is just the variance of sub-epoch mean amplitudes — that's fine, but it should be flagged as such rather than presented as something more exotic.\n\nNet: this paper deserves peer review. It's a useful toolkit for the BCI subfield and the evaluation is honestly done. I'd recommend major revision with a request to soften or validate the independence claim and add the control correlation. If that gets addressed, I'd use these metrics in my own work.","headline":"A coherent and useful variability-measurement toolkit for BCI research, but the 'independent modality' claim is stronger than the math supports and should be toned down or backed with a synthetic validation.","tokens_in":18584,"tokens_out":3451,"would_cite":true,"duration_ms":34025,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Lower variability in EEG responses across time, space, and frequency is associated with higher brain-computer interface performance.","keywords":["event-related desynchronization","brain-computer interface","EEG variability","motor imagery","temporal spatial frequency","variability metrics","BCI performance","interpretable metrics"],"falsifier":"Simulate ERD/S tensors with known, separately controllable jitter in time, space, and frequency; if the temporal variability score rises when only spatial jitter is injected, the independence claim at the heart of the framework is falsified.","tokens_in":17785,"feed_emoji":"🧠","tokens_out":10076,"duration_ms":85167,"temperature":0.7,"pith_summary":"This paper attempts to do something the BCI field has lacked: break the messy signal instability that ruins brain-computer interfaces into three interpretable components—temporal, spatial, and frequency variability—and measure each one at three levels: within a trial, across trials, and across groups of trials. Using two public motor-imagery datasets with 133 total users, it finds that lower variability in most of these components is significantly associated with higher classification accuracy (rank-order correlations around −0.2 to −0.4). It also finds that a Riemannian-based classifier's accuracy is more tightly linked to variability than a deep convolutional network's, and that the variability of the person actually operating the BCI matters more than the variability of the training group. If these associations hold, the metrics give researchers a common yardstick for comparing datasets and classifiers, and a target for training feedback that could make BCIs usable by more people.","feed_headline":"Lower EEG variability tracks higher BCI accuracy in 133 users","feed_subtitle":"Splits brain-signal jitter into time, space, and frequency; shows which instabilities hurt BCI control.","key_machinery":"The load-bearing object is a marginal-slice dispersion metric. For each trial, the ERD/S tensor is reduced to one modality by averaging over the other two dimensions—time series via averaging over channels and frequencies, spatial maps via averaging over frequencies and time, spectral patterns via averaging over channels and time. Variability is then v = (1/n) Σ distance(x_i, M), with per-modality distances: squared Euclidean for within-trial temporal, angular distance for spatial and spectral patterns, and dynamic time warping for between-trial and between-group time series. The centroid M is chosen as the ordinary average for Euclidean distances, or as the point on the unit sphere that min","core_discovery":"The paper's central claim is that event-related (de)synchronization (ERD/S) variability can be quantified as the mean distance between extracted slices of an ERD/S tensor and a reference centroid, and that this single dispersion formula, v = (1/n) Σ distance(x_i, M), reproduces and refines prior variability measures. When applied to 133 users across two datasets, the metrics show that within-trial and between-trial variability in time, space, and frequency carry negative rank-order correlations with BCI performance in most conditions, with the test user's own variability predicting cross-user accuracy while the training group's average does not. The paper further observes that a tangent-spac","pith_inferences":["The same slice-and-dispersion recipe should transfer to other EEG tensors—ERP latencies, connectivity matrices, or time-frequency images from other paradigms—so the framework is a generic variability meter, not just an ERD/S one.","The independence of the three modalities is untested: because each slice marginalizes away the other dimensions, cross-contamination is plausible. A synthetic experiment that injects jitter in one modality and checks whether the other metrics move would settle this cheaply.","If the negative correlation is causal, then adding a variability penalty on the temporal slice to a deep decoder's loss function is a direct, testable way to convert this paper's correlation into an intervention.","The training-group null result suggests a practical ceiling for static cross-user calibration—collecting more diverse training data will not fix an unstable current user, so online adaptation using the user's own variability score is the obvious next step."],"forward_implications":["If the observed negative correlations are real, then real-time variability scores can serve as feedback during user training, giving users a target to stabilize rather than just an accuracy number.","The test-user-only effect implies that adaptive BCIs should compute variability from the current user's live signals, not from the training population's averages.","The stronger variability sensitivity of the Riemannian classifier relative to the deep network suggests that datasets with known spatial pattern drift are better handled by convolutional architectures, or that covariance-based pipelines need spatial alignment before classification.","The positive test-referenced group-wise correlations, if causal, suggest that enriching training sets with users whose spatial and spectral patterns are unlike the target user may improve cross-user generalization."],"fun_headline_variants":["EEG variability metric predicts BCI success in 133 users","Low brain-signal jitter links to better BCI control","Three-way EEG variability score maps BCI performance","Variability in time, space, frequency flags poor BCI accuracy","How EEG signal instability predicts BCI success"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The metrics are presented as independent measures of time, space, and frequency, but each one is computed on a slice obtained by averaging over the other two modalities; the load-bearing, untested premise is that fluctuations in one modality do not bleed into another, so spatial jitter could masquerade as temporal variability.","fun_headline_variants_meta":{"raw":{"variants":["EEG variability metric predicts BCI success in 133 users","Low brain-signal jitter links to better BCI control","Three-way EEG variability score maps BCI performance","Variability in time, space, frequency flags poor BCI accuracy","How EEG signal instability predicts BCI success"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000712,"raw_usage":{"total_tokens":3078,"prompt_tokens":819,"completion_tokens":2259,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":563,"completion_tokens_details":{"reasoning_tokens":2179}},"tokens_in":563,"tokens_out":2259,"duration_ms":13245,"temperature":1.0,"reasoning_tokens":2179,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T08:22:39.251106+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Simulate ERD/S tensors with known, separately controllable jitter in time, space, and frequency; if the temporal variability score rises when only spatial jitter is injected, the independence claim at the heart of the framework is falsified.","supporting_citations":[],"review_version":1}