REVIEW 4 major objections 6 minor 31 references
Transfer Learning with Active Sampling for Rapid Training and Calibration in BCI-P300 Across Health States and Multi-centre Data
T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read Active sampling raises P300 BCI accuracy by 5.36% and cuts training time by 61%.
desk verdict The paper's headline improvement is a misread of its own tables, and the sampling factor was chosen on the test set; the multi-centre dataset and the core idea still deserve a proper evaluation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is Dense Poisson Disk Sampling (Dense PDS), a repulsive point process that generates diverse mini-batches (here, a whole sampled dataset of 1200 trials per subject) by requiring any two selected points to be at least a disk radius apart. Unlike random sampling, PDS makes the second-order product density vanish for nearby points, which drives the variance term in stochastic gradient descent negative and accelerates convergence. The 'Dense' variant draws points according to a categorical distribution over a mingling index, so the decision boundary between P300 and non-P300 classes is densely sampled. This sampled set is then used in Adaptive Transfer Learning (ATL): a Deep4Net CNN is pre-trained on source subjects' sampled data, its first convolutional layer is frozen, and the remaining layers are fine-tuned on the target subject's sampled data in 10% increments of the adaptation rate.
What would settle it
Re-run the subject-adaptive comparison with the sampling factor selected on a validation split that is never used for the final test, and check whether the 5.36% accuracy gain, the 12.22% standard-deviation reduction, and the 61% time saving persist; if they shrink or vanish, the reported advantage is an artifact of using the test set to choose the sample size.
Extended reading notes
Core claim
The central discovery is that deliberately sampling a diverse, non-redundant subset of EEG trials—rather than using all available data—improves both accuracy and stability of adaptive transfer learning for P300 detection. The authors introduce Active Sampling (AS), a Dense Poisson Disk Sampling procedure that selects 1200 samples per subject, balancing P300 and non-P300 trials near the decision boundary through a mingling index. When a Deep4Net CNN pre-trained in a subject-independent manner is fine-tuned on this sampled target data, the averaged subject-adaptive accuracy rises by 5.36% and the standard deviation drops by 12.22% on the combined original and replicated multi-centre datasets. The method also reduces CPU training time by 61.17% and is presented as a general mechanism for reducing stochastic gradient noise and overfitting during domain adaptation.
Load-bearing premise
The paper picks the 1200-sample factor by choosing the size that gives the best subject-independent test accuracy on the same subjects that later appear in the reported adaptive results, so the headline gains assume this selection does not leak test information into the method choice.
Editorial extensions
If this is right
- A new BCI user could be calibrated with roughly 1200 sampled trials per subject instead of the full session data, cutting setup time while maintaining or improving accuracy.
- Multi-centre EEG data from different countries, amplifiers, and patient groups can be pooled for transfer learning without the usual performance collapse, since sampling reduces inter-subject variance.
- The 12.22% reduction in standard deviation means subject-adaptive P300 classification becomes more predictable across individuals, which matters for clinical assistive devices.
- Training time drops by about 61%, so repeated recalibration during a session becomes feasible in real-world BCI use.
- The approach yields higher bitrates at small numbers of stimulus blocks, meaning faster communication for spelling.
Reading between the lines
- Because the sampling factor (1200) was chosen by maximizing subject-independent test accuracy on the same subjects later used in the adaptive evaluation, the reported 5.36% gain and 61% time reduction are likely optimistic; an unbiased nested validation could lower them.
- The same Dense PDS active-sampling scheme could be applied to other ERP-based BCIs (e.g., N170, error-related potentials) and to imbalanced classification beyond EEG, wherever stochastic gradient noise and redundant data slow convergence.
- Combining Active Sampling with generative data augmentation, which the authors mention as future work, might push the accuracy gain further—but this is not tested in the paper.
- The multi-centre replication design itself is a template: reproducing a published P300 protocol in a different country with different hardware is a strong stress test for transferability claims in BCI research.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes an Active Sampling (AS) method based on Dense Poisson Disk Sampling (PDS) for adaptive transfer learning in P300-based brain-computer interfaces. The method is evaluated on two EEG datasets collected in different countries, one public and one newly replicated, with both healthy and patient populations. The authors report that AS improves averaged classification accuracy by 5.36% and reduces standard deviation by 12.22% at a 40% adaptation rate, while also reducing training time by about 61%. The paper includes subject-dependent, subject-independent, and subject-adaptive classification experiments, together with computational time and bitrate analyses.
Significance. If the reported results were valid, the proposed AS method would be a valuable contribution to reducing BCI calibration time in heterogeneous, multi-centre settings. The paper's strengths include the use of two distinct datasets with patient participants, a multi-centre replication design, and a clear focus on computational cost. However, the evaluation protocol contains a test-set selection bias, the headline numbers are inconsistent with the paper's own tables, and the without-AS baseline exhibits near-chance collapses that drive the reported improvement. These issues mean the central quantitative claim is not currently supported.
major comments (4)
- [Section V-A, Table III] The sampling factor (1200 samples per subject) is selected by maximizing subject-independent test accuracy on the same subjects that are later used in the subject-adaptive comparison. Step 3 of the selection procedure states 'The best subject-independent classification accuracy test and its corresponding sample size were chosen.' This is test-set peeking: the hyperparameter is chosen using the test performance, so the reported improvements with AS are optimistically biased and cannot be interpreted as an unbiased estimate of the method's advantage.
- [Abstract and Section V-D, Tables VI and VII] The abstract claims a 5.36% accuracy improvement at a 40% adaptation rate. According to Table VI, the mean accuracy without AS at 40% is 78.13%, and according to Table VII, the mean accuracy with AS at 40% is 82.62%, a difference of 4.49 percentage points. The value 5.36 arises only from comparing w/o AS at 40% (78.13%) with w/ AS at 50% (83.49%). The reported 'standard deviation reduction of 12.22%' is also the absolute difference between 17.44% and 5.22% (in percentage points), not a relative reduction (which would be about 70%). The headline numbers need to be corrected to match the data.
- [Table VI] The subject-adaptive baseline without AS collapses to near-chance levels for several subjects (e.g., S02 at 19.63% at 40%; S08 at 60.06% at 40% and 17.65% at 80%). The mean improvement with AS is therefore driven largely by stabilizing these collapsed runs rather than by a consistent sampling advantage. The paper should report per-subject results with repeated training runs and demonstrate that AS also improves accuracy for subjects whose without-AS baseline is not collapsed.
- [Section VI] The claim that Dense PDS is stable and consistent is not supported in the main text. The discussion refers to multiple repetitions in supplementary Table S1, but the number of repetitions, the variance across runs, and the specific results are not reported in the paper. Without these details, the reader cannot assess the statistical significance or robustness of the reported differences.
minor comments (6)
- [Throughout] The term 'Poison Disk Sampling' should be 'Poisson Disk Sampling' (the authors consistently misspell 'Poisson').
- [Section V-E] The text refers to the second dataset as 'IE' in the phrase 'using 1200 samples across both datasets (OE and IE)', but the dataset is named 'ME' elsewhere; please correct this typo.
- [Fig. 11 caption] The caption says 'Boxplot of training time' but the figure is a violin plot; the caption should match the figure type.
- [Table VIII] The dataset labels 'OE' and 'ME' appear as row entries in the table, which makes the grouping of subjects unclear; consider placing the dataset in a separate column.
- [References] Reference [24] is a general review of deep learning for EEG; the original Deep4Net architecture paper (Schirrmeister et al., 2017) should be cited instead or in addition.
- [Section V-D] The sentence 'the average classification accuracy for a subject adaptive with 40% of the adaptation rate in OE+ME w/o AS outperformed OE+ME w/AS with 5.36%' has the comparison direction reversed; it should read 'w/ AS outperformed w/o AS'.
Circularity Check
Central 5.36% gain is an in-sample result because the sampling factor was chosen by maximizing test accuracy on the same subjects, and the headline figure does not match the paper's own tables.
-
fitted input called prediction
[Section V-A (Samples number selection) feeding Section V-D and the Abstract]
"3) The best subject-independent classification accuracy test and its corresponding sample size were chosen. ... After this thorough analysis, we selected the 1200 sample size. ... Our results reported that the average classification accuracy for a subject adaptive with 40% of the adaptation rate in OE+ME w/o AS outperformed OE+ME w/AS with 5.36% and a standard deviation reduction of 12.22%."
The Sampling Factor is the key hyperparameter of Active Sampling. It was selected by maximizing subject-independent test accuracy over the exact same 17 subjects who later populate the adaptive comparison in Tables VI and VII. The advertised improvement is therefore not an out-of-sample prediction: the method's main tuning knob was chosen by peeking at the evaluation subjects' test scores, biasing the comparison in favor of AS. The abstract presents the gain as a fixed result for the 40% adaptation rate, yet the paper's own tables give a 4.49 percentage-point difference at that rate (78.13% to 82.62%), while the 5.36 figure only appears when comparing different adaptation-rate columns. Thus the headline 'prediction' reduces to a test-selected operating point rather than a confirmed effect.
full rationale
This is an empirical systems paper without a mathematical derivation chain, so most circularity categories—self-definitional, imported uniqueness, ansatz via citation—do not apply. The core sampling method is attributed to external prior work [20], and the adaptive-transfer protocol cites Zhang et al. [10]; neither is load-bearing self-citation. The multi-centre replication and the training-time reduction are concrete and independent of the circularity concern: subsampling each subject to 1200 samples mechanically lowers CPU time, and the observed ~61% reduction is plausible by construction. However, the central quantitative claim is compromised. Section V-A chooses the Sampling Factor by maximizing subject-independent test accuracy on the same subjects later used for the adaptive comparison, making the reported improvement an in-sample, test-selected result rather than a held-out prediction. Additionally, the abstract's 5.36% gain does not match Tables VI and VII, which show a 4.49 percentage-point difference at 40% adaptation, and the '12.22% standard deviation reduction' is actually an absolute percentage-point drop (17.44 to 5.22). These evaluation-selection issues make the headline accuracy claim partially circular, while the mechanism and time savings remain independently credible.
Assumptions & free parameters
free parameters (4)
- Sampling factor (sample count per subject) =
1200
- Reported adaptation rate =
40% in the abstract; 80% in Fig. 10b; 10% in Fig. 10a
- Rejection radius r0 in Dense PDS =
not specified
- Mingling index distribution parameter pi0 =
not specified
assumptions (4)
- standard math Theorem 1 variance formula for repulsive point processes (from Zhang et al. 2019)
- domain assumption Gradients are aligned for nearby points and the loss is smooth in its data argument
- ad hoc to paper The two EEG datasets can be pooled with no channel harmonization, and Euclidean distance in the raw sample space is a meaningful diversity metric for PDS
- domain assumption Freezing the first convolutional layer of Deep4Net is a valid adaptive transfer strategy
Cite this review
Pith. "Pith review of Transfer Learning with Active Sampling for Rapid Training and Calibration in BCI-P300 Across Health States and Multi-centre Data." pith.science (2026). https://pith.science/paper/FMXWX6H4
@misc{pith2026241217833,
author = {Pith},
title = {Pith review of: Transfer Learning with Active Sampling for Rapid Training and Calibration in BCI-P300 Across Health States and Multi-centre Data},
year = {2026},
howpublished = {\url{https://pith.science/paper/FMXWX6H4}},
note = {Machine review of arXiv:2412.17833}
}
read the original abstract
Machine learning and deep learning advancements have boosted Brain-Computer Interface (BCI) performance, but their wide-scale applicability is limited due to factors like individual health, hardware variations, and cultural differences affecting neural data. Studies often focus on uniform single-site experiments in uniform settings, leading to high performance that may not translate well to real-world diversity. Deep learning models aim to enhance BCI classification accuracy, and transfer learning has been suggested to adapt models to individual neural patterns using a base model trained on others' data. This approach promises better generalizability and reduced overfitting, yet challenges remain in handling diverse and imbalanced datasets from different equipment, subjects, multiple centres in different countries, and both healthy and patient populations for effective model transfer and tuning. In a setting characterized by maximal heterogeneity, we proposed P300 wave detection in BCIs employing a convolutional neural network fitted with adaptive transfer learning based on Poison Sampling Disk (PDS) called Active Sampling (AS), which flexibly adjusts the transition from source data to the target domain. Our results reported for subject adaptive with 40% of adaptive fine-tuning that the averaged classification accuracy improved by 5.36% and standard deviation reduced by 12.22% using two distinct, internationally replicated datasets. These results outperformed in classification accuracy, computational time, and training efficiency, mainly due to the proposed Active Sampling (AS) method for transfer learning.
Figures
Reference graph
Works this paper leans on
-
[1]
Progress in brain computer in- terface: Challenges and opportunities,
S. Saha, K. A. Mamun, K. Ahmed, R. Mostafa, G. R. Naik, S. Darvishi, A. H. Khandoker, and M. Baumert, “Progress in brain computer in- terface: Challenges and opportunities,” Front. Syst. Neurosci. , vol. 15, p. 578875, 2021
work page 2021
-
[2]
A novel deep learning scheme for motor imagery eeg decoding based on spatial representation fusion,
J. Yang, Z. Ma, J. Wang, and Y . Fu, “A novel deep learning scheme for motor imagery eeg decoding based on spatial representation fusion,” IEEE Access, vol. 8, pp. 202100–202110, 2020
work page 2020
-
[3]
C. F. Vega, J. Quevedo, E. Escand ´on, M. Kiani, W. Ding, and J. Andreu- Perez, “Fuzzy temporal convolutional neural networks in p300-based brain–computer interface for smart home interaction,” Applied Soft Computing, vol. 117, p. 108359, 3 2022
work page 2022
-
[4]
Improving speller bci performance using a cluster-based under-sampling method,
S. A. Cortez, C. Flores, and J. Andreu-Perez, “Improving speller bci performance using a cluster-based under-sampling method,” in IEEE SSCI, pp. 576–581, 2020
work page 2020
-
[5]
S. A. Cortez, C. Flores, and J. Andreu-Perez, “Under-sampling and classification of p300 single-trials using self-organized maps and deep neural networks for a speller bci,” in IEEE SMC, pp. 2972–2978, 2020
work page 2020
-
[6]
The application of transfer learning in p300 detection,
Y . Liu, C. Yang, and Z. Li, “The application of transfer learning in p300 detection,” in IEEE CBS, pp. 412–417, 2018
work page 2018
-
[7]
Cross- domain mlp and cnn transfer learning for biological signal processing: Eeg and emg,
J. J. Bird, J. Kobylarz, D. R. Faria, A. Ekart, and E. P. Ribeiro, “Cross- domain mlp and cnn transfer learning for biological signal processing: Eeg and emg,” IEEE Access, vol. 8, pp. 54789–54801, 2020
work page 2020
-
[8]
A deep transfer convolutional neural network framework for eeg signal classification,
G. Xu, X. Shen, S. Chen, Y . Zong, C. Zhang, H. Yue, M. Liu, F. Chen, and W. Che, “A deep transfer convolutional neural network framework for eeg signal classification,” IEEE Access, vol. 7, pp. 112767–112776, 2019
work page 2019
Show all 31 references
-
[9]
Classification and transfer learning of eeg during a kinesthetic motor imagery task using deep convolutional neural networks,
A. Craik, A. Kilicarslan, and J. L. Contreras-Vidal, “Classification and transfer learning of eeg during a kinesthetic motor imagery task using deep convolutional neural networks,” in IEEE EMBC , pp. 3046–3049, 2019
2019
-
[10]
Adaptive transfer learning for eeg motor imagery classification with deep convolutional neural network,
K. Zhang, N. Robinson, S. W. Lee, and C. Guan, “Adaptive transfer learning for eeg motor imagery classification with deep convolutional neural network,” Neural Networks, vol. 136, pp. 1–10, 4 2021
2021
-
[11]
Accelerating minibatch stochastic gradient descent using stratified sampling,
P. Zhao and T. Zhang, “Accelerating minibatch stochastic gradient descent using stratified sampling,” 2014
2014
-
[12]
Posterior transfer learning with active sampling,
J. Pan and Y . Tu, “Posterior transfer learning with active sampling,” in Big Data and Security: Second International Conference, ICBDS 2020, Singapore, Singapore, December 20–22, 2020, Revised Selected Papers 2, pp. 639–651, Springer, 2021
2020
-
[13]
Imbalanced data classification: Using transfer learning and active sampling,
Y . Liu, G. Yang, S. Qiao, M. Liu, L. Qu, N. Han, T. Wu, G. Yuan, and Y . Peng, “Imbalanced data classification: Using transfer learning and active sampling,” Engineering Applications of Artificial Intelligence, vol. 117, p. 105621, 2023
2023
-
[14]
Positive-negative momen- tum: Manipulating stochastic gradient noise to improve generalization,
Z. Xie, L. Yuan, Z. Zhu, and M. Sugiyama, “Positive-negative momen- tum: Manipulating stochastic gradient noise to improve generalization,” in ICML, pp. 11448–11458, PMLR, 2021
2021
-
[15]
Kulesza and B
A. Kulesza and B. Taskar, Determinantal Point Processes for Machine Learning. Hanover, MA, USA: Now Publishers Inc., 2012
2012
-
[16]
k-dpps: Fixed-size determinantal point processes,
A. Kulesza and B. Taskar, “k-dpps: Fixed-size determinantal point processes,” in ICML, pp. 1193–1200, 2011
2011
-
[17]
Determinantal point processes for machine learning,
A. Kulesza and B. Taskar, “Determinantal point processes for machine learning,” Found. Trends Mach. Learn. , vol. 5, no. 2–3, pp. 123–286, 2012
2012
-
[18]
Diverse mini-batch active learning,
F. Zhdanov, “Diverse mini-batch active learning,” CoRR, vol. abs/1901.05954, 2019
1901 arXiv
-
[19]
Stochastic learning on im- balanced data: Determinantal point processes for mini-batch diversifica- tion,
C. Zhang, H. Kjellstr ¨om, and S. Mandt, “Stochastic learning on im- balanced data: Determinantal point processes for mini-batch diversifica- tion,” CoRR, vol. abs/1705.00607, 2017
2017 arXiv
-
[20]
Active mini-batch sampling using repulsive point processes,
C. Zhang, C. ¨Oztireli, S. Mandt, and G. Salvi, “Active mini-batch sampling using repulsive point processes,” in AAAI, vol. 33, pp. 5741– 5748, 2019
2019
-
[21]
A comparison of methods for generating poisson disk distributions,
A. Lagae and P. Dutr ´e, “A comparison of methods for generating poisson disk distributions,” Comput. Graph. Forum, vol. 27, no. 1, pp. 114–129, 2008
2008
-
[22]
An efficient p300-based brain–computer interface for disabled subjects,
U. Hoffmann, J.-M. Vesin, T. Ebrahimi, and K. Diserens, “An efficient p300-based brain–computer interface for disabled subjects,” J. Neurosci. Methods, vol. 167, no. 1, pp. 115–125, 2008. Brain-Computer Interfaces (BCIs)
2008
-
[23]
Perfor- mance evaluation of a p300 brain-computer interface using a kernel extreme learning machine classifier,
C. Flores, C. Fonseca, D. Achanccaray, and J. Andreu-Perez, “Perfor- mance evaluation of a p300 brain-computer interface using a kernel extreme learning machine classifier,” in IEEE SMC , pp. 3715–3719, IEEE, 2018
2018
-
[24]
Deep learning techniques for eeg signal applications–a review,
D. Merlin Praveena, D. Angelin Sarah, and S. Thomas George, “Deep learning techniques for eeg signal applications–a review,” IETE journal of Research, vol. 68, no. 4, pp. 3030–3037, 2022. AUTHOR et al.: PREPARATION OF PAPERS FOR IEEE TRANSACTIONS AND JOURNALS (FEBRUARY 2017) 11
2022
-
[25]
Brain–computer interfaces for communication and control,
J. R. Wolpaw, N. Birbaumer, D. J. McFarland, G. Pfurtscheller, and T. M. Vaughan, “Brain–computer interfaces for communication and control,” Clinical Neurophysiology, vol. 113, no. 6, pp. 767–791, 2002
2002
-
[26]
A gentle introduction and survey on computing with words (cww) methodologies,
P. K. Gupta and J. Andreu-Perez, “A gentle introduction and survey on computing with words (cww) methodologies,” Neurocomputing, vol. 500, pp. 921–937, 2022
2022
-
[27]
Towards understanding human functional brain development with explainable artificial intelligence: Challenges and perspectives,
M. Kiani, J. Andreu-Perez, H. Hagras, S. Rigato, and M. L. Filip- petti, “Towards understanding human functional brain development with explainable artificial intelligence: Challenges and perspectives,” IEEE Computational Intelligence Magazine , vol. 17, no. 1, pp. 16–33, 2022
2022
-
[28]
Real time recognition of human activities from wearable sensors by evolving classifiers,
J. Andreu, R. D. Baruah, and P. Angelov, “Real time recognition of human activities from wearable sensors by evolving classifiers,” in 2011 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2011) , pp. 2786–2793, IEEE, 2011
2011
-
[29]
Effective brain connectivity for fnirs with fuzzy cog- nitive maps in neuroergonomics,
M. Kiani, J. Andreu-Perez, H. Hagras, E. I. Papageorgiou, M. Prasad, and C.-T. Lin, “Effective brain connectivity for fnirs with fuzzy cog- nitive maps in neuroergonomics,” IEEE Transactions on Cognitive and Developmental Systems, vol. 14, no. 1, pp. 50–63, 2019
2019
-
[30]
Deep learning towards intelligent vehicle fault diagnosis,
M. Al-Zeyadi, J. Andreu-Perez, H. Hagras, C. Royce, D. Smith, P. Rzon- sowski, and A. Malik, “Deep learning towards intelligent vehicle fault diagnosis,” in 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1–7, IEEE, 2020
2020
-
[31]
Single-trial recognition of video gamer’s expertise from brain haemodynamic and facial emotion re- sponses,
A. R. Andreu-Perez, M. Kiani, J. Andreu-Perez, P. Reddy, J. Andreu- Abela, M. Pinto, and K. Izzetoglu, “Single-trial recognition of video gamer’s expertise from brain haemodynamic and facial emotion re- sponses,” Brain sciences, vol. 11, no. 1, p. 106, 2021
2021
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.