{"id":"bf82e2bb-6c04-4810-8c6e-c780ea935862","arxiv_id":"2504.21427","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"MPEC reports a new combination of manifold clustering and ensemble classification for EEG, claiming 78.12% accuracy on BCI Competition IV 2a, but its superiority is not established due to evaluation protocol differences and an unspecified curvature term.","lead":"This paper proposes MPEC, an EEG classification pipeline that combines covariance and RBF features, clusters them on a Riemannian manifold with a modified K-means, and classifies via a stacking ensemble. It reports 78.12% average accuracy on BCI Competition IV 2a, but the comparison to prior methods uses mismatched evaluation protocols.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 78.12% headline accuracy comes from an 80-20 random trial split, while Table III baselines were evaluated under different (typically session-transfer) protocols; the claimed superiority may be an evaluation artifact.","rationale":"The paper's central contribution is the claimed empirical superiority of MPEC on BCI-IV-2a, and that claim rests entirely on Table III. The reader's weakest-assumption pinpoints exactly the fragile link: the 80-20 random split is not shown to be comparable to the protocols used to generate the baseline accuracies. This is not a matter of consensus or taste; it is a direct threat to the validity of the headline number. A random split of trials from the same sessions leaks session-specific structure into training, which is known to inflate EEG classification accuracy compared to cross-session transfer. Since the baselines are cited from external papers, almost certainly under their own protocols, the cross-table comparison in Table III is not an apples-to-apples comparison. I also note that the paper's own Algorithm I contains an apparent typo (w1 used twice) and an undefined tangent-slope quantity, but those affect reproducibility more than the specific accuracy claim. If the concrete test shows MPEC retains its advantage under the official session-transfer protocol, the central claim would be substantially rehabilitated; otherwise the reported result is an artifact. Given the reader already reached REJECT with high correctness risk, my analysis does not move that verdict: the concern is the same and it lands.","tokens_in":10715,"tokens_out":4135,"duration_ms":46017,"concrete_test":"Rerun MPEC on BCI-IV-2a using the official evaluation protocol: train on session 1 and test on session 2 for each of the nine subjects, and report per-subject and average accuracy. Independently, re-run the five Table III baselines under the same 80-20 random split used for MPEC and report their average accuracies. If MPEC's cross-session accuracy drops materially (e.g., below the best baseline) or if the baselines' random-split accuracies rise to match MPEC's 78.12%, then Table III's claimed superiority is an evaluation artifact rather than an algorithmic gain.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is empirical: MPEC outperforms five published baselines on BCI Competition IV dataset 2a by roughly 4.5 to 29 percentage points (Table III). The load-bearing assumption is that all numbers in Table III are comparable. They are not, as the paper itself states in Section IV.B: 'The train-test split is set at 80-20 for all models.' For BCI-IV-2a, the standard benchmark protocol is train on session 1 and test on session 2, because EEG is highly non-stationary across recording sessions. An 80-20 random split draws training and test trials from the same two sessions, so within-session correlations (electrode drift, impedance changes, fatigue, recording artifacts) can be exploited, inflating accuracy relative to cross-session evaluation. The baselines in Table III are taken from their own papers, which typically use the official session-transfer protocol or other subject-specific validation; the paper supplies no evidence that any baseline was re-run under the same random 80-20 split. Thus the reported 78.12% superiority in Table III may reflect protocol mismatch rather than manifold preservation. A secondary but related reproducibility gap: Algorithm I's combined distance in step 2.1.3 uses w1 twice (should presumably be w1*d_R + w2*theta), and the 'slope of the tangent plane' theta is never defined, so the exact method cannot be re-run from the text. However, the protocol mismatch alone is sufficient to invalidate the headline comparison as presented.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes MPEC, a four-phase EEG classification pipeline evaluated on BCI Competition IV dataset 2a: correlation-based channel selection, covariance and RBF-kernel feature matrices processed as SPD matrices, a modified K-means clustering algorithm on the Riemannian manifold, per-cluster tangent-space projection, and a stacking ensemble (SVM, logistic regression, MLP, random forest) with a Ridge meta-model. The authors report an average accuracy of 78.12% for MPEC and compare this with five published baselines, claiming substantial improvements in multi-class motor-imagery classification. The central claim is empirical: that preserving Riemannian manifold structure through the proposed clustering and projection strategy yields superior accuracy.","tokens_in":11101,"tokens_out":5174,"duration_ms":57408,"significance":"If the reported results were valid, the paper would offer a useful contribution to Riemannian-geometry-based EEG classification, and the ensemble-of-clustered-classifiers architecture is a reasonable idea worth exploring. The manuscript has strengths: it uses a standard public dataset, provides pseudocode for both algorithms, and reports standard metrics. However, the evidence for the central claim is not reliable as presented. The evaluation protocol for MPEC is not comparable to that of the cited baselines, and the core clustering metric contains an undefined term and an apparent typo. The paper also does not provide code, final hyperparameter values, or a clear validation scheme, so the reported advantages cannot be independently reproduced or attributed to the method.","major_comments":[{"comment":"The headline accuracy comparison in Table III is not based on a common evaluation protocol. Section IV.B states that 'the train-test split is set at 80-20 for all models,' which applies only to the authors' own models. The baselines in Table III are cited from their original papers, and no evidence is given that any baseline was re-run under the same random 80-20 split. For BCI Competition IV dataset 2a, the standard benchmark protocol is cross-session evaluation (train on one session, test on the other), because EEG statistics shift across sessions. An 80-20 random split over the combined sessions can exploit within-session correlations and inflate accuracy relative to cross-session protocols. Since Section IV.D gives no protocol details for the five baselines, the reported improvement of MPEC over Jindal et al. [21], Hou et al. [22], Wijaya et al. [23], Rodrigues et al. [24], and Amin et al. [25] may be an artifact of this protocol mismatch rather than evidence of algorithmic superiority. This is the load-bearing issue for the paper's main claim.","section":"Section IV.B and Table III"},{"comment":"The proposed clustering distance is not fully specified. Step 2.1.2 asks to 'calculate the slope of the tangent plane at x_i in relation to each centroid, theta_{x_i},' but theta is never defined anywhere in the paper, and the phrase does not correspond to a standard quantity in Riemannian geometry. Step 2.1.3 then writes the combined distance as D(x_i,c_j) = w1*d_R(x_i,c_j) + w1*theta_{x_i}, which uses w1 twice, contradicting the initialization w1 + w2 = 1 in step 1.2; presumably this should be w1*d_R + w2*theta. Because the clustering objective is the core novelty of Phase 2, the exact algorithm cannot be re-implemented from the text.","section":"Algorithm I, steps 2.1.2-2.1.3"},{"comment":"The fusion of the covariance matrix and the RBF kernel matrix is dimensionally inconsistent. Equation (1) defines the covariance matrix C in R^{n x n}, with n indexing EEG channels, while Equation (3) defines the RBF kernel matrix K_{ij} = K(x_i, x_j) for N signal samples, so K is naturally in R^{N x N}. Algorithm II, Phase 1, step 1.4, however, instructs the method to 'combine the covariance matrix and RBF kernel matrices using a weighted sum.' Unless N equals n, which is not the case here (N is the number of trials and n is the number of channels), the weighted sum of C and K is undefined. The paper needs to clarify the actual dimensions and how the two matrices are fused.","section":"Equations (1), (3) and Algorithm II, Phase 1"},{"comment":"The empirical evaluation lacks a clean model-selection procedure. The text reports that feature subsets, sigma, K, and the fusion weights were selected by 'trial and error' and that the best performance was used for the final model, but no final values for w1, w2, or the covariance/RBF fusion weights are reported, and it is unclear whether selection was performed on the test set. The statement that 'a 5-fold cross-validation was employed to validate the model and fine-tune its performance' does not clarify whether cross-validation was nested inside the training split or used for direct model selection on the evaluation data. Without a separate validation set or nested cross-validation, the reported 78.12% accuracy may be optimistically biased.","section":"Section IV.B"}],"minor_comments":[{"comment":"Step 2.1.3 appears twice with different instructions (normalization and combination); the duplicate numbering should be fixed.","section":"Algorithm I"},{"comment":"The figure caption lists '[22] [23] [24] [25] [26] MPEC' but the reference list has no [26]; Table III labels the baselines as [21]-[25]. The caption should be corrected.","section":"Fig. 3 caption"},{"comment":"The proof of positive definiteness of the covariance matrix is not rigorous: a covariance matrix is positive semidefinite in general, and positive definiteness holds only under full-rank conditions. The claim that 'the variance of any non-trivial EEG signal is greater than zero' does not by itself establish v^T C v > 0 for every nonzero v.","section":"Section III.B.1"},{"comment":"The argument for positive definiteness of the RBF kernel matrix is incomplete: positivity of the exponential function does not imply the kernel matrix is positive definite; the standard proof requires that the Gram matrix be positive definite for distinct points. The wording 'except when v = 0' is not the correct condition.","section":"Section III.B.1, RBF kernel discussion"},{"comment":"For each baseline in Table III, the protocol used in the original papers (e.g., cross-session, subject-dependent, number of trials) should be stated, so the reader can judge comparability.","section":"Section IV.D"}],"recommendation":"reject","confidential_remarks":"The manuscript's central empirical claim is unsupported as submitted because the baseline comparison in Table III does not share a common evaluation protocol with the proposed method, and the core clustering distance is not defined precisely enough to be reproduced. These issues are fixable in principle, but they affect the validity of the main result. I would suggest the authors re-evaluate all baselines under the same protocol and provide a complete, unambiguous specification of the tangent-slope term and the feature-fusion step before resubmission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The headline number is not trustworthy as reported. MPEC is a reasonable engineering pipeline: covariance plus RBF features, clustering on the SPD manifold with a distance that supposedly mixes Riemannian distance and local curvature, tangent-space projection, and a stacking ensemble. The writing is clear, and the parameter list is fairly detailed. That is the good part.\n\nThe stress-test concern is the central issue. The paper evaluates MPEC on an 80-20 random split of the 576 trials (two sessions per subject) and then compares against baseline accuracies taken directly from the cited papers. Those papers almost certainly use the standard BCI-IV-2a protocol: train on one session, test on the other. EEG is strongly non-stationary across sessions; a random split lets the model exploit within-session correlations (drift, impedance, fatigue). So the reported 78.12% versus 73.57% for Wijaya et al. does not demonstrate algorithmic superiority; it demonstrates protocol mismatch. The authors give no evidence that any baseline was re-run under the same 80-20 split. This alone invalidates the headline comparison.\n\nThe method is also under-specified. The \"slope of the tangent plane\" (theta_xi) is never defined, so the novel clustering step cannot be re-run. Algorithm I also has a typo: step 2.1.3 writes w1*d_R + w1*theta, presumably intending w2*theta. More seriously, the feature engineering phase combines a covariance matrix (channels x channels) with an RBF kernel matrix (samples x samples) via a weighted sum; those dimensions do not match. If the authors intended something else, it is not explained.\n\nHyperparameter selection is a third concern: weights and cluster counts were chosen by trial and error on the benchmark, with sigma=0.1 and K=3 reported. There is no nested validation or clear separation between tuning and testing, so the 78.12% may partially reflect overfitting to the test set. The 5-fold cross-validation is mentioned but not clearly integrated into the protocol.\n\nMinor point: the positive-definiteness proof for the covariance matrix is hand-wavy; covariance matrices can be singular if channels are linearly dependent, though with 22 channels and enough trials it is usually full rank.\n\nCredit where due: the pipeline is a sensible combination of published ideas, and the literature review correctly situates it in Riemannian geometry for EEG. If the evaluation were redone with matched protocols, the ensemble-of-clustering idea might be worth exploring. But as presented, the empirical claim is unsupported and the method is not reproducible.\n\nRecommendation: I would not accept this in its current form. A serious referee should ask for a full re-evaluation under a matching cross-session protocol (or at least a re-run of baselines under the same split), a precise definition of the curvature term, and a fix to the dimension mismatch. If the authors can do that, there may be a modest contribution here. For now, treat the headline accuracy as unverified.","headline":"A plausible but under-specified pipeline whose headline accuracy is likely an artifact of comparing an 80-20 random split to cross-session baselines.","tokens_in":11594,"tokens_out":3817,"would_cite":false,"duration_ms":40315,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MPEC claims 78.12% average accuracy on BCI Competition IV dataset 2a by preserving the Riemannian geometry of EEG covariance features.","keywords":["brain-computer interfaces","EEG signal classification","ensemble modeling","clustering-based classification","Riemannian manifold","covariance matrix","RBF kernel","motor imagery"],"falsifier":"Run MPEC under the BCI Competition IV 2a session-to-session protocol (training on session 1, testing on session 2 for each subject) against the same five baselines; if the accuracy margin nearly vanishes, the advantage comes from the split rather than the manifold. Alternatively, ablate the manifold-aware clustering by replacing its weighted distance with Euclidean distance on vectorized covariance matrices while keeping all other phases fixed; if accuracy does not drop, manifold preservation is not the load-bearing component.","tokens_in":10546,"feed_emoji":"🧠","tokens_out":6060,"duration_ms":56090,"temperature":0.7,"pith_summary":"The paper proposes MPEC, a classification pipeline for EEG-based brain-computer interfaces that keeps the non-Euclidean geometry of EEG signals intact instead of flattening it into Euclidean space. It argues that treating covariance matrices as points on a Riemannian manifold, clustering them with a manifold-aware distance, and then projecting each cluster to the tangent space before stacking four classifiers yields more accurate decoding of four motor-imagery tasks. On the BCI Competition IV dataset 2a, MPEC reports 78.12% average accuracy, above the 49.00% to 73.57% range of five published baselines. If correct, this shows that geometric faithfulness in feature engineering and clustering, not just more powerful classifiers, is a practical lever for BCI accuracy.","feed_headline":"Manifold-aware EEG classifier beats five baselines at 78%","feed_subtitle":"MPEC clusters EEG covariance features on their curved space, then stacks four learners to beat five baselines.","key_machinery":"The load-bearing object is the weighted Riemannian distance $D(x_i, c_j) = w_1 d_R(x_i, c_j) + w_2 \\theta_{x_i}$, where $d_R$ is the geodesic distance on the manifold of SPD covariance matrices and $\\theta_{x_i}$ is the slope of the tangent plane at the point, encoding local curvature. This distance replaces Euclidean distance inside K-means so that cluster assignments honor the curved geometry of EEG covariance features. Around this, the pipeline wraps the covariance-plus-RBF feature representation, per-cluster tangent-space projection, and a stacking ensemble whose meta-model is a ridge classifier; the claim is that the combination preserves manifold information through to the final decision.","core_discovery":"MPEC claims that EEG classification accuracy improves when every stage of the pipeline respects the Riemannian geometry of symmetric positive definite (SPD) matrices. The method represents each trial by a covariance matrix and an RBF kernel matrix, both of which are SPD and hence carry a manifold structure. A modified K-means clusters these matrices using a weighted sum of the Riemannian (geodesic) distance and the tangent-plane slope, preserving local curvature; each cluster is then mapped to its tangent space for linear classification. A stacking ensemble of SVM, logistic regression, MLP, and random forest with a ridge meta-model produces the final label. On all nine subjects of BCI Competition IV dataset 2a, the ensemble reaches 78.12% accuracy, beating the five compared baselines, which the authors attribute to the manifold-preserving design rather than to any single learner.","pith_inferences":["An ablation that sets the curvature weight $w_2$ to zero would isolate how much the tangent-slope term, rather than the Riemannian distance alone, contributes to the reported gain; the paper does not include this control.","If manifold preservation is the active ingredient, the same clustering-plus-tangent-projection recipe should transfer to other SPD-feature domains such as MEG or fNIRS classification, which the paper does not test.","The paper uses a random 80-20 split across both recording sessions; under the original competition's session-to-session transfer protocol, the margin over baselines could shrink, and measuring that is a sharper test of the claim."],"forward_implications":["Independent replication on the same data with the same split should reproduce MPEC's 78.12% average accuracy if the method is stable.","The per-cluster tangent-space step implies that local linearization is enough for classification, so the method could scale to higher-density EEG without global Euclidean embeddings.","Because the ensemble's four learners share one manifold-preserved representation, any accuracy gain is attributable to that representation, not to a single classifier's strength.","The spread across subjects (55.14% to 90.74%) suggests the method's benefit depends on subject-specific signal quality; identifying that dependency is a direct next step."],"supporting_citations":[{"why":"Supplies the BCI Competition IV dataset 2a used for all experiments and comparisons.","marker":"[12]"},{"why":"Provides the Riemannian-kernel classification approach for BCI that MPEC builds on.","marker":"[6]"},{"why":"Motivates covariance matrices as representations of motor-imagery EEG signals.","marker":"[13]"},{"why":"Justifies tangent-space projection for dimensionality reduction of SPD data.","marker":"[4]"},{"why":"Supplies the stacking ensemble methodology used to combine the weak learners.","marker":"[17]"},{"why":"Is the strongest baseline in Table III, giving the comparison MPEC must beat.","marker":"[23]"}],"fun_headline_variants":["Manifold-aware EEG classifier hits 78% on BCI IV","Riemannian geometry boosts EEG accuracy to 78%","Ensemble on curved space beats five EEG baselines","EEG via manifold-preserving clustering reaches 78%","MPEC: geometry-guided EEG ensemble achieves 78%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported accuracy gap rests on the assumption that the 80-20 random split is a fair comparison to the protocols used by the five baselines, so the difference in Table III reflects algorithmic superiority rather than an easier evaluation setup.","fun_headline_variants_meta":{"raw":{"variants":["Manifold-aware EEG classifier hits 78% on BCI IV","Riemannian geometry boosts EEG accuracy to 78%","Ensemble on curved space beats five EEG baselines","EEG via manifold-preserving clustering reaches 78%","MPEC: geometry-guided EEG ensemble achieves 78%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000338,"raw_usage":{"total_tokens":1839,"prompt_tokens":890,"completion_tokens":949,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":506,"completion_tokens_details":{"reasoning_tokens":866}},"tokens_in":506,"tokens_out":949,"duration_ms":9371,"temperature":1.0,"reasoning_tokens":866,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:03:30.333267+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run MPEC under the BCI Competition IV 2a session-to-session protocol (training on session 1, testing on session 2 for each subject) against the same five baselines; if the accuracy margin nearly vanishes, the advantage comes from the split rather than the manifold. Alternatively, ablate the manifold-aware clustering by replacing its weighted distance with Euclidean distance on vectorized covariance matrices while keeping all other phases fixed; if accuracy does not drop, manifold preservation is not the load-bearing component.","supporting_citations":[{"cited_title":"Review of the BCI competition IV,","cited_arxiv_id":null,"evidence_quote":"Supplies the BCI Competition IV dataset 2a used for all experiments and comparisons."},{"cited_title":"Classification of covariance matrices using a Riemannian -based kernel for BCI applications,","cited_arxiv_id":null,"evidence_quote":"Provides the Riemannian-kernel classification approach for BCI that MPEC builds on."},{"cited_title":"Robust averaging of covariances for EEG recordings classification in motor imagery brain- computer interfaces,","cited_arxiv_id":null,"evidence_quote":"Motivates covariance matrices as representations of motor-imagery EEG signals."},{"cited_title":"Dimensionality reduction of SPD data based on riemannian manifold tangent spaces and isometry,","cited_arxiv_id":null,"evidence_quote":"Justifies tangent-space projection for dimensionality reduction of SPD data."},{"cited_title":"Automatic personality prediction: an enhanced method using ensemble modeling,","cited_arxiv_id":null,"evidence_quote":"Supplies the stacking ensemble methodology used to combine the weak learners."},{"cited_title":"Improving multi -class EEG-motor imagery classification using two -stage detection on one - versus-one approach,","cited_arxiv_id":null,"evidence_quote":"Is the strongest baseline in Table III, giving the comparison MPEC must beat."}],"review_version":1}