Pith. sign in

REVIEW 2 major objections 2 minor 50 references

EEG signals during code comprehension classify programmer skill levels at up to 92 percent accuracy.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-07-02 20:12 UTC pith:5HG6YQD2

load-bearing objection EEG classification of programmer skill reports high accuracies but the results are likely driven by age confounds rather than expertise. the 2 major comments →

arxiv 2606.30879 v2 pith:5HG6YQD2 submitted 2026-06-29 cs.HC

Neural Signatures of Programming Expertise: Classifying Programmer Skill Levels Using EEG Data

Maurice Rekrut , Mahima Mahabaleshwar Acharya , Taisiia Ulianova , Norman Peitek , Annabelle Bergum , Mariya Toneva , Sven Apel , Antonio Kr\"uger This is my paper
classification cs.HC
keywords EEGprogramming expertiseskill classificationneural correlatescode comprehensionmachine learningbrain activation patterns
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tests whether brain recordings can measure programming expertise more directly than interviews or coding tests. Researchers took an existing set of EEG data from 37 programmers spanning one to thirty years of experience and extracted features such as signal entropy and frequency-band power. They trained Random Forest models to sort the programmers into expert, intermediate, and novice groups. The models reached high accuracy in cross-validation, and certain brain patterns, especially localized centro-frontal activity, aligned with higher skill. The work argues that neural measures could therefore supplement conventional assessment methods.

Core claim

EEG entropy showed the strongest correlation with skill level; experts displayed highly localized centro-frontal activation during code comprehension while other groups showed more distributed frontal activity; Random Forest classifiers using these and related features reached 91.83 percent average accuracy on binary expert-versus-novice classification and 78.15 percent on three-class separation under stratified 10-fold cross-validation, with leave-one-subject-out results of 85.00 percent and 58.80 percent respectively.

What carries the argument

Random Forest classifiers applied to EEG entropy, frequency-band powers, and spatial activation patterns extracted from code-comprehension and resting-state recordings.

Load-bearing premise

The EEG patterns mainly track stable differences in programming skill rather than age, education, or task familiarity.

What would settle it

A follow-up experiment that collects new EEG data from programmers whose ages and years of experience are deliberately mismatched and then checks whether classification accuracy remains above chance.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Individual frequency bands outperform full-spectrum EEG for skill classification.
  • Resting-state recordings alone support strong prediction of skill level.
  • Localized centro-frontal activation marks expert performance while less experienced programmers recruit broader networks.
  • Neural features can complement traditional hiring and evaluation tools.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the same patterns appear in other technical domains, EEG could serve as a general probe for cognitive expertise.
  • Longitudinal recordings could test whether training changes these brain signatures over months or years.
  • Real-time EEG feedback during coding tasks might help identify when a programmer is operating at expert efficiency.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The manuscript claims that EEG data recorded during code comprehension from 37 programmers can be used with Random Forest classifiers to accurately classify skill levels (binary experts vs. novices at 91.83%, multi-class at 78.15% in 10-fold CV; lower in LOSO), with EEG entropy showing the strongest correlation to skill and experts exhibiting localized centro-frontal activation patterns.

Significance. Should the findings prove robust to demographic confounds, they would offer a promising neural complement to traditional skill assessment methods in software engineering. The empirical approach, including multiple validation schemes and feature analyses across frequency bands and resting state, provides a solid foundation for further investigation into cognitive signatures of expertise.

major comments (2)
  1. [Abstract and Methods] Abstract and Methods (dataset description): The skill groups are defined solely by self-reported years of experience (range 1–30 y, mean 8.1 y) with no reported age-matching, education covariate regression, or demographic balancing across strata. Because years of experience is collinear with age and age is a known modulator of EEG entropy, power spectra, and connectivity, the reported entropy–skill correlation and the 91.83 % / 78.15 % accuracies could be driven by demographic variables rather than programming expertise.
  2. [Results] Results (classification performance): Accuracies are stated without error bars, confidence intervals, or explicit feature definitions; the multi-class LOSO accuracy drops to 58.80 %, consistent with subject-specific rather than generalizable skill signatures. This directly affects the central claim that the EEG features capture stable skill-level differences.
minor comments (2)
  1. [Abstract] The source of the existing EEG dataset should be explicitly cited.
  2. [Methods] The exact thresholds or criteria used to assign participants to novice/intermediate/expert bins should be stated.

Simulated Author's Rebuttal

2 responses · 1 unresolved

We thank the referee for their constructive comments, which highlight important considerations for interpreting our results. We address each major comment below and have revised the manuscript to strengthen the presentation of limitations and statistical reporting.

read point-by-point responses

  1. Referee: [Abstract and Methods] Abstract and Methods (dataset description): The skill groups are defined solely by self-reported years of experience (range 1–30 y, mean 8.1 y) with no reported age-matching, education covariate regression, or demographic balancing across strata. Because years of experience is collinear with age and age is a known modulator of EEG entropy, power spectra, and connectivity, the reported entropy–skill correlation and the 91.83 % / 78.15 % accuracies could be driven by demographic variables rather than programming expertise.

    Authors: We acknowledge this as a valid concern. Years of experience is collinear with age, and the original dataset did not collect age or other demographic covariates, precluding age-matching or regression. Self-reported experience remains the standard proxy in programming expertise research, but we agree it limits causal attribution to expertise per se. In revision we will add an explicit Limitations subsection discussing this potential confound and its implications for the entropy correlations and classification results. revision: yes

  2. Referee: [Results] Results (classification performance): Accuracies are stated without error bars, confidence intervals, or explicit feature definitions; the multi-class LOSO accuracy drops to 58.80 %, consistent with subject-specific rather than generalizable skill signatures. This directly affects the central claim that the EEG features capture stable skill-level differences.

    Authors: We will add error bars and 95% confidence intervals to all accuracy figures in the revised Results. Feature definitions are detailed in Methods; we will add cross-references in Results for clarity. The LOSO drop (already reported) does indicate subject-specific variance, which is common in EEG. The stratified CV results still demonstrate within-sample discriminative power of the features. We will revise the discussion to temper claims of generalizability and emphasize the need for larger, demographically controlled validation studies. revision: yes

standing simulated objections not resolved
  • Controlling for age or other demographics in the analyses, because the source dataset does not contain these variables.

Circularity Check

0 steps flagged

No circularity: empirical ML classification on external dataset

full rationale

The paper performs standard supervised classification (Random Forest) on a pre-existing EEG dataset of 37 subjects. Reported accuracies (91.83% binary, 78.15% multi-class in stratified 10-fold CV; lower in LOSO) are computed directly from held-out folds of the recorded signals and labels; no equations, fitted parameters, or self-citations reduce these quantities to quantities defined inside the study. Feature extraction (entropy, band power, etc.) and brain-region observations are post-hoc descriptions of the same data, not inputs that are redefined as outputs. The derivation chain is therefore self-contained against external benchmarks and contains none of the enumerated circularity patterns.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Only the abstract is available; no explicit free parameters, axioms, or invented entities are described.

pith-pipeline@v0.9.1-grok · 5841 in / 1201 out tokens · 32284 ms · 2026-07-02T20:12:30.554685+00:00 · methodology

0 comments
read the original abstract

Accurately assessing a programmer's skill level is critical for hiring, team composition, and performance evaluation in the software industry. Conventional methods, such as coding tests or interviews, often fail to capture the full spectrum of cognitive abilities underlying programming expertise. This study explores using electroencephalography (EEG) and machine learning to investigate neural correlates of programming skill. We analyzed an existing EEG dataset recorded during code comprehension from 37 programmers with 1 to 30 years of experience (8.1 +/- 6.3 years) to examine relationships between neural activity and expertise. Additionally, we conducted classification experiments using Random Forest classifiers with diverse features for binary (experts vs. novices) and multi-class (experts, intermediates, novices) setups. We identified EEG features and brain regions associated with programming expertise. Specifically, EEG entropy showed the strongest correlation with skill level. Furthermore, experts' brains were characterized by highly localized centro-frontal activation, whereas frontal activation in other groups was part of a more distributed network. Regarding classification, our setup achieved an average accuracy of 91.83% (binary) and 78.15% (multi-class) in stratified 10-fold cross-validation, while leave-one-subject-out validation achieved 85.00% and 58.80%, respectively. Individual frequency bands outperformed full-spectrum analyses, and both program comprehension and resting-state data yielded strong results. These findings demonstrate that EEG features effectively capture neural correlates across different skill levels and highlight the potential of neural data to complement traditional methods of skill assessment.

Figures

Figures reproduced from arXiv: 2606.30879 by Annabelle Bergum, Antonio Kr\"uger, Mahima Mahabaleshwar Acharya, Mariya Toneva, Maurice Rekrut, Norman Peitek, Sven Apel, Taisiia Ulianova.

Figure 1
Figure 1. Figure 1: Visualization of the experimental design of Peitek et [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The skill score as distribution of correct answers per minute across participants, separated by the categorized expertise [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Visualization of the fixed 4-second epoching approach [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Results of the Spearman Correlation analysis for baseline data (top row) and program comprehension data (bottom row) [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Topoplots of CSP analysis for the three groups of programmers. First and second plot show baseline, third and fourth [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Comparison of different time window positions start, middle and end with stratified 10 fold evaluation for binary and [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Comparison of different time window positions start, middle and end with leave one subject out evaluation for binary [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

50 extracted references · 50 canonical work pages

  1. [1]

    Sur- gical skill level classification model development using eeg and eye-gaze data and machine learning algorithms,

    S. B. Shafiei, S. Shadpour, J. L. Mohler, F. Sasangohar, C. Gutierrez, M. Seilanian Toussi, and A. Shafqat, “Sur- gical skill level classification model development using eeg and eye-gaze data and machine learning algorithms,” Journal of Robotic Surgery, vol. 17, 2023

    work page 2023

  2. [2]

    A game player expertise level classification system using electroencephalography (eeg),

    S. M. Anwar, S. M. U. Saeed, M. Majid, S. Usman, C. A. Mehmood, and W. Liu, “A game player expertise level classification system using electroencephalography (eeg),”Applied Sciences, vol. 8, no. 1, 2018

    work page 2018

  3. [3]

    Expertise classification using functional brain networks and normalized transfer entropy of eeg in design applications,

    M. Z. Baig and M. Kavakli, “Expertise classification using functional brain networks and normalized transfer entropy of eeg in design applications,” inProceedings of the 2019 11th International Conference on Computer and Automation Engineering, ser. ICCAE 2019, 2019, p. 41–46

    work page 2019

  4. [4]

    Classification of user competency levels using eeg and convolutional neural network in 3d modelling application,

    ——, “Classification of user competency levels using eeg and convolutional neural network in 3d modelling application,”Expert Systems with Applications, vol. 146, p. 113202, 2020

    work page 2020

  5. [5]

    Toward using alpha and theta brain waves to quantify programmer expertise,

    I. Crk and T. Kluthe, “Toward using alpha and theta brain waves to quantify programmer expertise,” in2014 36th Annual International Conference of the IEEE Engineer- ing in Medicine and Biology Society, 2014, pp. 5373– 5376

    work page 2014

  6. [6]

    Comparing programming lan- guage comprehension between novice and expert pro- grammers using eeg analysis,

    S. Lee, A. Matteson, D. Hooshyar, S. Kim, J. Jung, G. Nam, and H. Lim, “Comparing programming lan- guage comprehension between novice and expert pro- grammers using eeg analysis,” in2016 IEEE 16th Inter- national Conference on Bioinformatics and Bioengineer- ing (BIBE), 2016, pp. 350–355

    work page 2016

  7. [7]

    Mining biometric data to predict programmer expertise and task difficulty,

    S. Lee, D. Hooshyar, H. Ji, K. Nam, and H. Lim, “Mining biometric data to predict programmer expertise and task difficulty,”Cluster Computing, vol. 21, 03 2018

    work page 2018

  8. [8]

    Program comprehension: Past, present, and future,

    J. Siegmund, “Program comprehension: Past, present, and future,” in2016 IEEE 23rd International Confer- ence on Software Analysis, Evolution, and Reengineering (SANER), vol. 5, 2016, pp. 13–20

    work page 2016

  9. [9]

    Correlates of programmer efficacy and their link to experience: A combined eeg and eye-tracking study,

    N. Peitek, A. Bergum, M. Rekrut, J. Mucke, M. Nadig, C. Parnin, J. Siegmund, and S. Apel, “Correlates of programmer efficacy and their link to experience: A combined eeg and eye-tracking study,” inProceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Soft- ware Engineering, 2022, p. 120–131

    work page 2022

  10. [10]

    Cog- nitive task difficulty analysis using eeg and data mining,

    A. Duraisingam, R. Palaniappan, and S. Andrews, “Cog- nitive task difficulty analysis using eeg and data mining,” in2017 Conference on Emerging Devices and Smart Systems (ICEDSS), 2017, pp. 52–57

    work page 2017

  11. [11]

    An empirical evaluation of machine learning tech- niques to classify code comprehension based on eeg data,

    L. J. Gonc ¸ales, K. Farias, L. S. Kupssinsk¨u, and M. Sega- lotto, “An empirical evaluation of machine learning tech- niques to classify code comprehension based on eeg data,”Expert Systems with Applications, vol. 203, p. 117354, 2022

    work page 2022

  12. [12]

    Understanding program- ming expertise: An empirical study of phasic brain wave changes,

    I. Crk, T. Kluthe, and A. Stefik, “Understanding program- ming expertise: An empirical study of phasic brain wave changes,”ACM Trans. Comput.-Hum. Interact., vol. 23, no. 1, Dec. 2016

    work page 2016

  13. [13]

    Hiring is broken: What do developers say about technical interviews?

    M. Behroozi, C. Parnin, and T. Barik, “Hiring is broken: What do developers say about technical interviews?” in 2019 IEEE Symposium on Visual Languages and Human- Centric Computing (VL/HCC), 2019, pp. 1–9

    work page 2019

  14. [14]

    Can eeg be adopted as a neuroscience reference for assessing software pro- grammers’ cognitive load?

    J. Medeiros, R. Couceiro, G. Duarte, J. Dur ˜aes, J. Castel- hano, C. Duarte, M. Castelo-Branco, H. Madeira, P. de Carvalho, and C. Teixeira, “Can eeg be adopted as a neuroscience reference for assessing software pro- grammers’ cognitive load?” vol. 21, no. 7, 2021

    work page 2021

  15. [15]

    Towards an affordable brain computer interface for the assessment of pro- grammers’ mental workload,

    M. V . Kosti, K. Georgiadis, D. A. Adamos, N. Laskaris, D. Spinellis, and L. Angelis, “Towards an affordable brain computer interface for the assessment of pro- grammers’ mental workload,”International Journal of Human-Computer Studies, vol. 115, pp. 52–66, 2018

    work page 2018

  16. [16]

    On the influence of the baseline in neuroimag- ing experiments on program comprehension,

    A. Bergum, N. Peitek, M. Rekrut, J. Siegmund, and S. Apel, “On the influence of the baseline in neuroimag- ing experiments on program comprehension,”ACM Trans. Softw. Eng. Methodol., Jun. 2025

    work page 2025

  17. [17]

    Predicting java computer program- ming task difficulty levels using eeg for educational environments,

    R. Palaniappan, A. Duraisingam, N. Chinnaiah, and M. Murugappan, “Predicting java computer program- ming task difficulty levels using eeg for educational environments,” inInternational Conference on Human- Computer Interaction. Springer, 2019, pp. 446–460

    work page 2019

  18. [18]

    Identifying program confusion using electroencephalo- gram measurements,

    M. K.-C. Yeh, Y . Yan, Y . Zhuang, and L. A. DeLong, “Identifying program confusion using electroencephalo- gram measurements,”Behaviour & Information Technol- ogy, vol. 41, no. 12, pp. 2528–2545, 2022

    work page 2022

  19. [19]

    Unex- pected but informative: What fixation-related potentials tell us about the processing of confusing program code,

    A. Bergum, A.-M. Maurer, N. Peitek, R. Bader, A. Meck- linger, V . Demberg, J. Siegmund, and S. Apel, “Unex- pected but informative: What fixation-related potentials tell us about the processing of confusing program code,” 2026

    work page 2026

  20. [20]

    Evaluation of machine learning techniques to classify code comprehension based on developers’ eeg data,

    L. J. Gonc ¸ales, K. Farias, L. S. Kupssinsk¨u, and M. Sega- lotto, “Evaluation of machine learning techniques to classify code comprehension based on developers’ eeg data,” inProceedings of the 19th Brazilian Symposium on Human Factors in Computing Systems, ser. IHC ’20. New York, NY , USA: Association for Computing Machinery, 2020

    work page 2020

  21. [21]

    Evaluation of brain activity while pair programming,

    A. Thapaliya, “Evaluation of brain activity while pair programming,” inProceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Com- panion Proceedings, 2020, pp. 104–106

    work page 2020

  22. [22]

    Decoding the representation of code in the brain: An fmri study of code review and expertise,

    B. Floyd, T. Santander, and W. Weimer, “Decoding the representation of code in the brain: An fmri study of code review and expertise,” in2017 IEEE/ACM 39th In- ternational Conference on Software Engineering (ICSE), 2017, pp. 175–186

    work page 2017

  23. [23]

    A look into programmers’ heads,

    N. Peitek, J. Siegmund, S. Apel, C. K ¨astner, C. Parnin, A. Bethmann, T. Leich, G. Saake, and A. Brechmann, “A look into programmers’ heads,”IEEE Transactions on Software Engineering, vol. 46, no. 4, pp. 442–462, 2020

    work page 2020

  24. [24]

    The role of the insula in intuitive expert bug detection in computer code: an fmri study,

    J. Castelhano, I. C. Duarte, C. Ferreira, J. Duraes, H. Madeira, and M. Castelo-Branco, “The role of the insula in intuitive expert bug detection in computer code: an fmri study,”Brain imaging and behavior, vol. 13, 14 no. 3, pp. 623–637, 2019

    work page 2019

  25. [25]

    Neurological divide: An fmri study of prose and code writing,

    R. Krueger, Y . Huang, X. Liu, T. Santander, W. Weimer, and K. Leach, “Neurological divide: An fmri study of prose and code writing,” inProceedings of the acm/ieee 42nd international conference on software engineering, 2020, pp. 678–690

    work page 2020

  26. [26]

    Measuring and Modeling Programming Expe- rience,

    J. Siegmund, C. K ¨astner, J. Liebig, S. Apel, and S. Ha- nenberg, “Measuring and Modeling Programming Expe- rience,”Empirical Software Engineering, vol. 19, no. 5, pp. 1299–1334, 2014

    work page 2014

  27. [27]

    Robinson,Likert Scale, 2014, pp

    J. Robinson,Likert Scale, 2014, pp. 3620–3621

    work page 2014

  28. [28]

    Program comprehension and code complexity metrics: An fmri study,

    N. Peitek, S. Apel, C. Parnin, A. Brechmann, and J. Sieg- mund, “Program comprehension and code complexity metrics: An fmri study,” inProc. Int’l Conf. Software Engineering (ICSE). ACM, 2021, pp. 524–536

    work page 2021

  29. [29]

    Report of the committee on meth- ods of clinical examination in electroencephalography,

    H. H. Jasper, “Report of the committee on meth- ods of clinical examination in electroencephalography,” Electroencephalography and Clinical Neurophysiology, vol. 10, pp. 370–375, 1958

    work page 1958

  30. [30]

    Psychopy2: Experiments in behavior made easy,

    P. J, G. JR, S. S, M. M, H. R, S. H, K. E, and L. JK, “Psychopy2: Experiments in behavior made easy,” in Behavior research methods, vol. 51(1), 2019, pp. 195– 203

    work page 2019

  31. [31]

    Decoding semantic categories from eeg activity in object-based decision tasks,

    M. Rekrut, M. Sharma, M. Nadig, J. Alexandersson, and A. Kruger, “Decoding semantic categories from eeg activity in object-based decision tasks,” 02 2020, pp. 1–7

    work page 2020

  32. [32]

    Use of common average reference and large-laplacian spatial-filters enhances eeg signal-to-noise ratios in in- trinsic sensorimotor activity,

    S. Tsuchimoto, S. Shibusawa, S. Iwama, M. Hayashi, K. Okuyama, N. Mizuguchi, K. Kato, and J. Ushiba, “Use of common average reference and large-laplacian spatial-filters enhances eeg signal-to-noise ratios in in- trinsic sensorimotor activity,”Journal of neuroscience methods, vol. 353, p. 109089, 2021

    work page 2021

  33. [33]

    An eye tracking study on camelcase and under score identifier styles,

    B. Sharif and J. I. Maletic, “An eye tracking study on camelcase and under score identifier styles,” in2010 IEEE 18th International Conference on Program Com- prehension. IEEE, 2010, pp. 196–205

    work page 2010

  34. [34]

    Eye Move- ments in Code Reading: Relaxing the Linear Order,

    T. Busjahn, R. Bednarik, A. Begel, M. Crosby, J. Pater- son, C. Schulte, B. Sharif, and S. Tamm, “Eye Move- ments in Code Reading: Relaxing the Linear Order,” 2015, pp. 255–265

    work page 2015

  35. [35]

    What Drives the Reading Order of Programmers? An Eye Tracking Study,

    N. Peitek, J. Siegmund, and S. Apel, “What Drives the Reading Order of Programmers? An Eye Tracking Study,” 2020, p. 342–353

    work page 2020

  36. [36]

    Blind separation of sources, part i: An adaptive algorithm based on neuromimetic architecture,

    C. Jutten and J. Herault, “Blind separation of sources, part i: An adaptive algorithm based on neuromimetic architecture,”Signal Processing, vol. 24, no. 1, pp. 1– 10, 1991

    work page 1991

  37. [37]

    Eeglab: an open source tool- box for analysis of single-trial eeg dynamics,

    A. Delorme and S. Makeig, “Eeglab: an open source tool- box for analysis of single-trial eeg dynamics,”Journal of Neuroscience Methods, vol. 134, pp. 9–12, 01 2004

    work page 2004

  38. [38]

    The use of fast fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms,

    P. Welch, “The use of fast fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms,”IEEE Transactions on Audio and Electroacoustics, vol. 15, no. 2, pp. 70–73, 1967

    work page 1967

  39. [39]

    Software code complexity assessment using eeg features,

    J. Medeiros, R. Couceiro, J. Castelhano, M. C. Branco, G. Duarte, C. Duarte, J. Dur˜aes, H. Madeira, P. Carvalho, and C. Teixeira, “Software code complexity assessment using eeg features,” in2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2019, pp. 1413–1416

    work page 2019

  40. [40]

    Array programming with NumPy,

    C. R. Harris, K. J. Millman, S. J. van der Walt, R. Gom- mers, P. Virtanen, D. Cournapeau, E. Wieser, J. Taylor, S. Berg, N. J. Smith, R. Kern, M. Picus, S. Hoyer, M. H. van Kerkwijk, M. Brett, A. Haldane, J. F. del R ´ıo, M. Wiebe, P. Peterson, P. G´erard-Marchant, K. Sheppard, T. Reddy, W. Weckesser, H. Abbasi, C. Gohlke, and T. E. Oliphant, “Array prog...

    work page 2020

  41. [41]

    SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python,

    P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C. J. Carey, ˙I. Polat, Y . Feng, E. W. Moore, J. VanderPlas, D. Laxalde, J. Perk- told, R. Cimrman, I. Henriks...

    work page 2020

  42. [42]

    Improving standards in brain-behavior correlation analyses,

    G. A. Rousselet and C. R. Pernet, “Improving standards in brain-behavior correlation analyses,”Frontiers in hu- man neuroscience, vol. 6, p. 119, 2012

    work page 2012

  43. [43]

    The quantitative extraction and topographic mapping of the abnormal components in the clinical eeg,

    Z. Koles, “The quantitative extraction and topographic mapping of the abnormal components in the clinical eeg,” Electroencephalography and Clinical Neurophysiology, vol. 79, no. 6, pp. 440–447, 1991

    work page 1991

  44. [44]

    Optimizing spatial filters for robust eeg single-trial analysis,

    B. Blankertz, R. Tomioka, S. Lemm, M. Kawanabe, and K.-r. Muller, “Optimizing spatial filters for robust eeg single-trial analysis,”IEEE Signal Processing Magazine, vol. 25, no. 1, pp. 41–56, 2008

    work page 2008

  45. [45]

    Eeg-based brain-computer interface using subject-specific spatial filters,

    G. Pfurtscheller, C. Guger, and H. Ramoser, “Eeg-based brain-computer interface using subject-specific spatial filters,” inEngineering Applications of Bio-Inspired Arti- ficial Neural Networks, J. Mira and J. V . S´anchez-Andr´es, Eds., 1999, pp. 248–254

    work page 1999

  46. [46]

    Simultaneous measure- ment of program comprehension with fmri and eye track- ing: A case study,

    N. Peitek, J. Siegmund, C. Parnin, S. Apel, J. C. Hofmeister, and A. Brechmann, “Simultaneous measure- ment of program comprehension with fmri and eye track- ing: A case study,” inProceedings of the 12th ACM/IEEE international symposium on empirical software engineer- ing and measurement, 2018, pp. 1–10

    work page 2018

  47. [47]

    Measuring neural efficiency of program com- prehension

    J. Siegmund, N. Peitek, C. Parnin, S. Apel, J. Hofmeister, C. K ¨astner, A. Begel, A. Bethmann, and A. Brech- mann, “Measuring neural efficiency of program com- prehension.” inProc. Europ. Software Engineering Conf./Foundations of Software Engineering (ESEC/FSE). ACM, 2017, pp. 140–150

    work page 2017

  48. [48]

    Factors contributing to the difficulties in teaching and learning of computer programming: A lit- erature review,

    C. S. Cheah, “Factors contributing to the difficulties in teaching and learning of computer programming: A lit- erature review,”Contemporary Educational Technology, vol. 12, no. 2, p. ep272, 2020

    work page 2020

  49. [49]

    Stuck and frustrated or in flow and happy: Sensing developers’ emotions and progress,

    S. C. M ¨uller and T. Fritz, “Stuck and frustrated or in flow and happy: Sensing developers’ emotions and progress,” in2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, vol. 1, 2015, pp. 688–699. 15

    work page 2015

  50. [50]

    Hiring is broken: What do developers say about technical interviews?

    M. Behroozi, C. Parnin, and T. Barik, “Hiring is broken: What do developers say about technical interviews?” in 2019 IEEE Symposium on Visual Languages and Human- Centric Computing (VL/HCC). IEEE, 2019, pp. 1–9

    work page 2019