Pre-Training Graph Contrastive Masked Autoencoders are Strong Distillers for EEG

Hua Xie; Kanhao Zhao; Lifang He; Xinxu Wei; Yong Jiao; Yu Zhang

REVIEW 4 major objections 5 minor 16 references

Pre-Training Graph Contrastive Masked Autoencoders are Strong Distillers for EEG

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that unified graph contrastive masked autoencoder pretraining plus graph topology distillation lets a lightweight low-density EEG student model match a large high-density teacher without pretraining.

desk verdict A genuinely novel combination of graph contrastive and masked-autoencoder pretraining for EEG, with plausible gains, but the headline LD-student-matches-HD-teacher claim needs a subject-disjoint evaluation before it is believed. read the letter →

arxiv 2411.19230 v2 pith:VSTU3DOO submitted 2024-11-28 cs.LG cs.AI

Xinxu Wei , Kanhao Zhao , Yong Jiao , Hua Xie , Lifang He , Yu Zhang This is my paper · ORCID

classification cs.LGcs.AI

keywords EEGclassificationgraphneuralnetworksself-supervisedpre-trainingcontrastivelearningmaskedautoencoderknowledgedistillationhigh-densitytolow-densityresting-state

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that abundant unlabeled high-density EEG can be turned into a working low-density diagnostic model: a lightweight student trained on 16- or 32-electrode data, after a unified graph self-supervised pretraining and topology distillation, reaches accuracy comparable to a large teacher trained on 64- or 128-electrode data. The method, EEG-DisGCMAE, couples graph contrastive pretraining with a graph masked autoencoder and then distills the teacher's learned adjacency structure into the student through positive and negative electrode-pair contrasts. If true, this matters because high-density EEG hardware is expensive and cumbersome, while low-density caps are cheap and portable; the paper reports gains of roughly three to five percentage points over prior graph pretraining baselines on four clinical classification tasks. The authors frame the contribution as a transfer-learning bridge over both the labeled/unlabeled gap and the density gap.

What carries the argument

The load-bearing objects are EEG graphs with alpha-band PSD node features and Pearson-correlation adjacency, a teacher-student pair of graph encoders, and a shared momentum key queue. The pretraining objective combines a contrastive loss over reconstructed query-key pairs with a masked reconstruction loss over both node features and adjacency. The distillation mechanism is the graph topology distillation loss, which computes linear-kernel node similarities and uses KL divergence over selected positive and negative pairs: positive pairs are low-density node pairs that are directly connected or connected through a deleted electrode in the teacher's high-density graph; negative pairs are edges present only in the student's low-density graph. This carries the density-transfer argument: deleted electrodes contribute relational structure the student must recover.

What would settle it

Run Algorithm 1 verbatim on a 64-electrode high-density graph and its 16-electrode low-density subgraph; step 8's A^h[V^d, :V^l] returns a 48 by 16 matrix while A^h_{1-hop} is 16 by 16, so the OR in steps 9 and 10 cannot be evaluated. Implementing the prose description of 2-hop positives instead of the printed algorithm, or removing the 2-hop term and keeping only direct edges, will change the GTD loss and downstream accuracy; comparing these variants settles whether the reported low-density gains come from the claimed topology distillation or from the pretraining objective alone.

Watch

Extended reading notes

Core claim

EEG-DisGCMAE's central claim is that its unified graph self-supervised pretraining (GCMAE-PT), combined with a graph topology distillation loss (GTD), makes a tiny student model on low-density EEG perform comparably to a large teacher on high-density EEG. During pretraining, the teacher and student encoders simultaneously reconstruct masked graph views and contrast those reconstructed samples against a shared queue of high- and low-density key embeddings; the two objectives supervise each other. During fine-tuning, GTD defines positive pairs from the high-density graph's direct and indirect (through removed electrodes) connections and negative pairs for edges the student sees but the teacher does not, then aligns the student's node-similarity kernel toward the teacher's. Combined with cross-entropy and logit KL-divergence distillation, this pipeline reports substantial gains over graph contrastive and masked autoencoder baselines on four clinical classification tasks from the EMBARC and HBN datasets.

Load-bearing premise

The paper's practical gains rest on the assumption that missing-electrode information in the high-density graph can be recovered as indirect two-hop connections between low-density electrodes, but the printed Algorithm 1 cannot actually compute those pairs: the submatrix indexed by deleted electrodes has different dimensions from the mask it is combined with, so the published procedure is incomplete and the reported gains depend on an unstated corrected implementation.

Editorial extensions

If this is right

A lightweight student with 1.3M parameters on low-density data, after GCMAE-PT and GTD, reaches accuracy comparable to a large 5.7M-parameter teacher on high-density data without pretraining, as reported in the main comparison table.
The unified pretraining objective outperforms graph contrastive pretraining alone, graph masked autoencoder pretraining alone, and their sequential combination, so the two objectives are claimed to supervise each other rather than merely stack.
Combining logit distillation with the graph topology distillation loss gives the best distillation performance, beating either loss alone and beating the G-CRD contrastive distillation baseline on the HBN MDD task.
The model degrades less than baselines under Gaussian noise and random electrode dropping, which the paper attributes to the robustness-oriented masked reconstruction in pretraining.
Pretraining on a heterogeneous dataset held out from the target site still transfers: HBN-only pretraining improves EMBARC severity classification by up to 4.4 percentage points with GTD, and including both datasets raises the gain to 5.6 percentage points.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the central claim holds, clinical EEG diagnostics could plausibly run on 8 to 16 electrode caps and still approach 64 to 128 channel accuracy, which is the cost and portability argument the paper's impact statement gestures toward.
The 2-hop positive-pair rule is a separable hypothesis: replacing it with 1-hop pairs only, or with random pairs, would isolate whether the teacher's deleted electrodes are the actual channel through which low-density performance improves.
The unified pretraining may extend to other sensor-array graphs with missing-channel asymmetry, such as MEG, fNIRS, or wearable multi-sensor data, though the paper only tests EEG under the 10-20 system.
The reported robustness to noise and electrode dropping suggests the pretraining acts partly as a data augmentation regularizer; an explicit test would be comparing GCMAE-PT against a compute-matched purely augmentative contrastive baseline.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper attacks a real problem: using abundant unlabeled high-density EEG to make low-density EEG diagnostics usable. The core novelty is real — unifying graph contrastive pretraining with graph masked-autoencoder pretraining in a single objective, plus a graph topology distillation loss that transfers connectivity knowledge from an HD teacher to an LD student. No prior work cited combines these two graph SSL paradigms, and the application to EEG is sensible. The empirical story is also consistent: their method beats GCL-only, GMAE-only, and sequential combinations across multiple tasks, and the ablations show both GCMAE-PT and GTD contribute. Credit where due: the held-out validation in Appendix N (pretrain on HBN, fine-tune on EMBARC severity) is a good check, the code is released, and the loss curves and robustness experiments are reasonable.

That said, there are soft spots, and one is load-bearing. Section 5.3's central claim — a tiny LD student matches a large HD teacher without pretraining — rests on numbers from HBN MDD and ASD, but the paper never states that pretraining is subject-disjoint from the downstream test folds. Appendix D.3 builds the pretraining corpus from both EMBARC and HBN subjects; Appendix D.4 then evaluates the same datasets with 10-fold cross-validation. Since self-supervised pretraining on resting-state EEG can encode subject identity, this is not a nit. The held-out check only covers EMBARC severity, not the HBN tasks where the headline gains appear. I can't tell from the text whether the actual implementation separated subjects, and the authors should say so explicitly.

The other issues are secondary but real. Algorithm 1's indirect-connectivity line uses Ah[V_d, :V_l], a |V_d| x |V_l| matrix, to mask a |V_l| x |V_l| KL matrix — the printed pseudocode cannot execute. The intended 2-hop logic is clear from Section 3.3, so this is likely a typo, but it needs fixing. Also, Appendix H says alpha band was selected as best, yet Table 10 shows All Bands outperforming alpha on all four tasks. That is a direct internal contradiction and should be corrected or explained. And there are no error bars anywhere, which makes it hard to judge whether the 1-2% gains are meaningful.

Who is this for? Researchers working on EEG graph representation learning, SSL for physiological signals, and HD-to-LD transfer. It is a serious paper with a correctable empirical gap. I would send it to review, but with the clear expectation that the authors provide a subject-disjoint evaluation for all four tasks, fix the pseudocode, and report variance. If the subject-leakage concern is resolved, this becomes a solid contribution.

Referee Report

4 major / 5 minor

Summary. The paper proposes EEG-DisGCMAE, a framework for pre-training graph neural networks on high-density (HD) EEG data and distilling the knowledge to a lightweight student model operating on low-density (LD) EEG. The method combines graph contrastive learning and masked autoencoder pre-training into a unified objective (GCMAE-PT), and introduces a graph topology distillation (GTD) loss that transfers pairwise topological similarities from an HD teacher to an LD student. The authors evaluate on four binary classification tasks across the EMBARC and HBN datasets, reporting that the tiny student model with LD input reaches or exceeds the performance of a large HD teacher without pre-training. The paper includes ablations over pre-training objectives, distillation losses, density levels, model sizes, and frequency bands, with code publicly available.

Significance. If the reported results hold under a properly controlled evaluation protocol, the paper addresses a practically important problem: exploiting abundant unlabeled HD EEG to improve diagnosis with cheap, portable LD EEG. The unified contrastive-generative pre-training objective is a reasonable and novel combination, and the GTD loss is an interesting mechanism for distilling topological structure across electrode densities. The paper provides broad ablations and makes its code available, which are strengths. However, the significance is currently conditional because of a likely data-leakage issue in the HBN experiments and because the GTD algorithm as printed is not executable. These issues must be resolved before the central claims can be accepted.

major comments (4)

[§5.3/Table 1, §D.3–D.4, Appendix N] The pretraining corpus is constructed from all EMBARC and HBN subjects (Appendix D.3), and the HBN downstream tasks are evaluated with 10-fold cross-validation on the same HBN data (Appendix D.4). The paper nowhere states that pretraining is subject-disjoint or fold-disjoint with respect to the downstream test subjects. If HBN subjects used in pretraining also appear in the fine-tuning/test folds, self-supervised pretraining can encode subject-identifying patterns, which would inflate the reported LD student performance and invalidate the headline claim that the tiny LD student approaches the HD teacher. Appendix N provides a held-out check only for the EMBARC severity task (pretrain on HBN, fine-tune on EMBARC); no subject-disjoint check is reported for the HBN MDD and ASD tasks, where the largest gains appear. The authors should either provide evidence that pretraining and downstream evaluation are subject-disjoint for all four tasks or rerun the HBN experiments under a subject-disjoint protocol.
[Algorithm 1, lines 8–10] The indirect-connectivity term Ah[V_d, :V_l] has shape |V_d| × |V_l|, whereas the KL matrices Z_l and Z_h in lines 9–10 are |V_l| × |V_l|. The printed algorithm therefore cannot compute the positive-pair mask described in Section 3.3, where two LD nodes are positive if they are connected through a removed node in V_d. The intended mask should be a |V_l| × |V_l| matrix, e.g., (Ah[V_l, V_d] @ Ah[V_d, V_l]) > 0 or an equivalent Boolean reduction. As written, Algorithm 1 is not executable, and the GTD loss used in the experiments depends on an unstated corrected computation. Please correct the algorithm and ensure it matches the released code.
[Appendix H, Table 10] The text states that the alpha band 'consistently yielded the best performance across all tasks', but Table 10 shows that the 'All Bands' configuration outperforms the alpha band on all four tasks (e.g., HBN MDD: 86.3/87.6 vs 84.8/85.4; EMBARC Sex: 74.6/79.0 vs 73.4/76.7). This is an internal contradiction. If the main experiments used only the alpha band, then a better-performing configuration was known but not adopted; if they used all bands, the description is inaccurate. Please clarify which band configuration was used in the main results and reconcile the text with Table 10.
[§D.4 and all result tables] The paper reports that downstream fine-tuning uses '10-fold cross-validation with 10 runs' (Appendix D.4), but no standard deviations, confidence intervals, or significance tests are reported anywhere. Many comparisons in Table 1 differ by only 1–2% in AUROC/ACC (e.g., Ours-Tiny (DGCNN) vs GraphMAE2 on HBN MDD LD: 81.6/82.4 vs 81.3/83.0). Without variance information, the reader cannot assess whether the claimed improvements are statistically meaningful. Please report mean ± standard deviation over the 10 runs and, where feasible, significance tests for the key comparisons.

minor comments (5)

[§3.2, sentence after Eq. (7)] There is a typo: 'teahcer' should be 'teacher'.
[§5.3, paragraph 1] The word 'Transofrmer' should be 'Transformer'.
[Eq. (9) and surrounding text] The notation softmax(Z_l_ij) and softmax(Z_h_ij) is ambiguous because Z_ij is a scalar kernel value; please specify the dimension over which the softmax is taken (e.g., over all j for fixed i, or over the full matrix).
[Table 8 and Appendix F] The paper claims the model 'generalizes remarkably well to the SEED dataset', but the SEED results in Table 8 appear to be for 'Ours' with pre-training on medical data only; however, it is unclear whether the SEED downstream evaluation uses any data from SEED during pretraining. Please clarify the pretraining protocol for the SEED experiment.
[References] The citation for LaBraM (Jiang et al.) lacks a year and venue; please provide the complete reference.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the claimed gains are empirical comparisons against external baselines, with no equation-level reduction of predictions to fitted inputs or self-citation chains.

full rationale

The paper's central claims are validated empirically: GCMAE-PT pretraining on unlabeled EEG graphs and GTD distillation are evaluated on downstream classification tasks via 10-fold cross-validation with 10 runs, against external baselines such as GCN, GFormer, EEGNet, GraphCL, GraphMAE, and LaBraM. The core mechanism—using the teacher's HD-graph topology to define positive/negative pairs for the LD student—is a knowledge-distillation loss, not circular: the student's LD performance is not forced to equal the teacher's by construction, since the student has fewer electrodes, fewer parameters, and must still solve the downstream classification task from its own inputs. No load-bearing claim is justified solely by a self-citation; the cited components (MoCo, GCC, GraphMAE, G-CRD, etc.) are external method components. The only notable issues are non-circular correctness and validity concerns: Algorithm 1 line 8 forms Ah[V_d, :V_l] with shape |V_d| x |V_l| and combines it with a |V_l| x |V_l| KL mask, so the printed pseudocode is not executable as written, although the Section 3.3 text describes an intended 2-hop mask; and Appendices D.3/D.4 do not explicitly state that pretraining is subject-disjoint from the downstream 10-fold test subjects for the HBN tasks, which is a potential data-leakage concern rather than a circular-reasoning defect. Neither issue makes a prediction reduce to its inputs by definition.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

The framework introduces no new physical or conceptual entities. Its free parameters are conventional deep learning hyperparameters, several unreported, plus a post hoc band choice that the paper's own table contradicts. The axioms are mostly stated design assumptions, two of them explicitly labeled as assumptions in Section 3.2.

free parameters (7)

EEG frequency band = alpha (8-14 Hz)
Appendix H selects alpha after ablation; Table 10 shows All Bands outperforms alpha on all four tasks, so the choice is post hoc and not supported by the paper's own table.
Node and edge masking ratio = 50%
Chosen from ablation in Figure 8; central to GMAE-PT and reconstruction quality.
Temperature tau in contrastive loss = not reported
Controls contrastive loss in Eq. 5; no value is given.
Adjacency binarization threshold theta = not reported
Used in Algorithm 1 and Eq. 14 to define graph edges; no value is given.
Loss weighting coefficients = implicit unit weights
Equations 7 and 11 sum losses without reported coefficients or tuning details.
Key queue size = not reported
The momentum key queue in Eq. 4 is central to contrastive pretraining; its capacity is unspecified.
Pretraining sliding window length and overlap = 50s and 20s
Appendix D.3 sets these to expand the pretraining corpus; affects dataset statistics and model exposure.

assumptions (6)

domain assumption An LD EEG graph is a node-subgraph of the HD EEG graph, with V_l subset V_h and removed nodes V_d.
Appendix B.1 Observation and Section 3.2; fundamental to defining GTD positive and negative pairs and HD-to-LD distillation.
ad hoc to paper Assumption 1: jointly reconstructing contrastive samples and contrasting reconstructed samples yields a more robust distiller.
Section 3.2 states this as an assumption with no independent evidence; it motivates GCMAE-PT.
ad hoc to paper Assumption 2: joint pretraining of teacher and student against a shared key queue improves downstream distillation.
Section 3.2 states this as an assumption; it motivates the shared Key Pool.
domain assumption Alpha-band PSD features and Pearson-correlation adjacency matrices form a graph that captures discriminative EEG topology.
Section 3.1 and Appendix H; all models are evaluated on this graph construction, with alpha band selection itself post hoc.
domain assumption MSE reconstruction of node features and of A via X_tilde X_tilde^T is a valid generative pretraining objective.
Eq. 2; used for GMAE-PT and follows GraphMAE-style practice.
ad hoc to paper KL divergence between softmax kernel similarities of teacher and student is a valid distillation metric for positive and negative pairs.
Eq. 9 and Algorithm 1; no theoretical justification is given beyond empirical ablation.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Pre-Training Graph Contrastive Masked Autoencoders are Strong Distillers for EEG." pith.science (2026). https://pith.science/paper/VSTU3DOO

@misc{pith2026241119230,
  author       = {Pith},
  title        = {Pith review of: Pre-Training Graph Contrastive Masked Autoencoders are Strong Distillers for EEG},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VSTU3DOO}},
  note         = {Machine review of arXiv:2411.19230}
}

read the original abstract

Effectively utilizing extensive unlabeled high-density EEG data to improve performance in scenarios with limited labeled low-density EEG data presents a significant challenge. In this paper, we address this challenge by formulating it as a graph transfer learning and knowledge distillation problem. We propose a Unified Pre-trained Graph Contrastive Masked Autoencoder Distiller, named EEG-DisGCMAE, to bridge the gap between unlabeled and labeled as well as high- and low-density EEG data. Our approach introduces a novel unified graph self-supervised pre-training paradigm, which seamlessly integrates the graph contrastive pre-training with the graph masked autoencoder pre-training. Furthermore, we propose a graph topology distillation loss function, allowing a lightweight student model trained on low-density data to learn from a teacher model trained on high-density data during pre-training and fine-tuning. This method effectively handles missing electrodes through contrastive distillation. We validate the effectiveness of EEG-DisGCMAE across four classification tasks using two clinical EEG datasets with abundant data. The source code is available at https://github.com/weixinxu666/EEG_DisGCMAE.

Figures

Figures reproduced from arXiv: 2411.19230 by the authors.

**Figure 2.** The intuitive diagram for the selection of positive [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** (a) compares model sizes and performance where [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (6 more)

**Figure 4.** Figure 4: Ablation studies of EEG patterns on the EMBARC datasets for MDD severity classification task. (a) is the pattern [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

**Figure 5.** Figure 5: Illustrations of loss curves in both the pre-training [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

**Figure 6.** Figure 6: Illustration of our proposed pipeline and other previous contrastive pre-training and masked autoencoder pre [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

**Figure 7.** Figure 7: The schematic diagram of two assumption in Section [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

**Figure 8.** Figure 8: Ablation studies on different node and edge dropping (for GCL-PT) and masking (for GMAE-PT) ratios. A 50% [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]

**Figure 9.** Figure 9: The ablation studies of distillation across different density settings (a) and kernels (b). [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 8 canonical work pages

[2]

As shown in Fig

(21) (a) (b) Figure 9: The ablation studies of distillation across different density settings (a) and kernels (b). As shown in Fig. 9, we conducted ablation experiments on the GTD loss. Figure 9(a) illustrates the results of distillation in three scenarios: high-to-low (H2L), high-to-medium (H2M), and high-to-high (H2H). Note that H2H is a special case. A...

work page 2002
[6]

B., Foged, M

Justesen, A. B., Foged, M. T., Fabricius, M., Skaarup, C., Hamrouni, N., Martens, T., Paulson, O. B., Pinborg, L. H., and Beniczky, S. Diagnostic yield of high-density versus low-density eeg: The effect of spatial sampling, timing and duration of recording. Clinical Neurophysiology, 130 (11):2060–2064,

work page
[11]

Multi-scale dis- tillation from multiple graph neural networks

Zhang, C., Liu, J., Dang, K., and Zhang, W. Multi-scale dis- tillation from multiple graph neural networks. InProceed- ings of the AAAI Conference on Artificial Intelligence , volume 36, pp. 4337–4344, 2022a. Zhang, C., Zhang, C., Song, J., Yi, J. S. K., Zhang, K., and Kweon, I. S. A survey on masked autoencoder for self- supervised learning in vision and...

arXiv
[12]

Deep graph contrastive representation learning

Zhu, Y ., Xu, Y ., Yu, F., Liu, Q., Wu, S., and Wang, L. Deep graph contrastive representation learning. arXiv preprint arXiv:2006.04131,

arXiv 2006
[13]

Preliminaries of Dynamic GNNs In traditional GNNs, the adjacency matrix A is static

11 Graph Contrastive Masked Autoencoders for EEG Distillers A. Preliminaries of Dynamic GNNs In traditional GNNs, the adjacency matrix A is static. However, in this paper, we adopt dynamic GNNs, where the adjacency matrix can be dynamically adjusted during training to suit the specific task better. This approach allows the model to adapt the graph structu...

work page 2020
[14]

with Graph Masked Autoencoders Pre-training (Hou et al., 2022), allowing us to model and capture the relationships among the three entities described in Eq

work page 2022
[17]

(Boessen et al., 2013), and 130 subjects are diagnosed with mild depression (HAMD17 score ≤ 17). The HBN dataset, which includes a range of diseases, has significantly fewer labeled samples compared to the total data volume due to the high number of samples without explicit MDD and ASD diagnostic labels. Additionally, the number of labeled subjects differ...

work page 2013
[2002]

Bert: Pre-training of deep bidirectional transformers for lan- guage understanding

Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for lan- guage understanding. arXiv preprint arXiv:1810.04805,

arXiv

Show all 16 references

[2007]

Cross-subject and cross-device wearable eeg emotion recognition using frontal eeg under virtual reality scenes

Kuang, F., Shu, L., Hua, H., Wu, S., Zhang, L., Xu, X., Liu, Y ., and Jiang, M. Cross-subject and cross-device wearable eeg emotion recognition using frontal eeg under virtual reality scenes. In 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 3...

2021
[2014]

Kipf, T. N. and Welling, M. Semi-supervised classifica- tion with graph convolutional networks. arXiv preprint arXiv:1609.02907,

arXiv
[2018]

Differential entropy feature for eeg-based emotion classification

Duan, R.-N., Zhu, J.-Y ., and Lu, B.-L. Differential entropy feature for eeg-based emotion classification. In 2013 6th international IEEE/EMBS conference on neural engineer- ing (NER), pp. 81–84. IEEE,

2013
[2019]

Kingma, D. P. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980,

arXiv
[2020]

Distilling the knowledge in a neural network

Hinton, G., Vinyals, O., and Dean, J. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531,

arXiv
[2021]

A., Saab, K., Zhang, X., Huang, Q., Dubost, F., Rubin, D

Tang, S., Dunnmon, J. A., Saab, K., Zhang, X., Huang, Q., Dubost, F., Rubin, D. L., and Lee-Messer, C. Self- supervised graph neural networks for improved elec- troencephalographic seizure analysis. arXiv preprint arXiv:2104.08336,

arXiv
[2022]

Graphmae2: A decoding-enhanced masked self-supervised graph learner

Hou, Z., He, Y ., Cen, Y ., Liu, X., Dong, Y ., Kharlamov, E., and Tang, J. Graphmae2: A decoding-enhanced masked self-supervised graph learner. In Proceedings of the ACM web conference 2023, pp. 737–746,

2023
[2023]

Graph-mlp: Node classification without message passing in graph

Hu, Y ., You, H., Wang, Z., Wang, Z., Zhou, E., and Gao, Y . Graph-mlp: Node classification without message passing in graph. arXiv preprint arXiv:2106.04051,

arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [2]

As shown in Fig

(21) (a) (b) Figure 9: The ablation studies of distillation across different density settings (a) and kernels (b). As shown in Fig. 9, we conducted ablation experiments on the GTD loss. Figure 9(a) illustrates the results of distillation in three scenarios: high-to-low (H2L), high-to-medium (H2M), and high-to-high (H2H). Note that H2H is a special case. A...

work page 2002

[2] [6]

B., Foged, M

Justesen, A. B., Foged, M. T., Fabricius, M., Skaarup, C., Hamrouni, N., Martens, T., Paulson, O. B., Pinborg, L. H., and Beniczky, S. Diagnostic yield of high-density versus low-density eeg: The effect of spatial sampling, timing and duration of recording. Clinical Neurophysiology, 130 (11):2060–2064,

work page

[3] [11]

Multi-scale dis- tillation from multiple graph neural networks

Zhang, C., Liu, J., Dang, K., and Zhang, W. Multi-scale dis- tillation from multiple graph neural networks. InProceed- ings of the AAAI Conference on Artificial Intelligence , volume 36, pp. 4337–4344, 2022a. Zhang, C., Zhang, C., Song, J., Yi, J. S. K., Zhang, K., and Kweon, I. S. A survey on masked autoencoder for self- supervised learning in vision and...

arXiv

[4] [12]

Deep graph contrastive representation learning

Zhu, Y ., Xu, Y ., Yu, F., Liu, Q., Wu, S., and Wang, L. Deep graph contrastive representation learning. arXiv preprint arXiv:2006.04131,

arXiv 2006

[5] [13]

Preliminaries of Dynamic GNNs In traditional GNNs, the adjacency matrix A is static

11 Graph Contrastive Masked Autoencoders for EEG Distillers A. Preliminaries of Dynamic GNNs In traditional GNNs, the adjacency matrix A is static. However, in this paper, we adopt dynamic GNNs, where the adjacency matrix can be dynamically adjusted during training to suit the specific task better. This approach allows the model to adapt the graph structu...

work page 2020

[6] [14]

with Graph Masked Autoencoders Pre-training (Hou et al., 2022), allowing us to model and capture the relationships among the three entities described in Eq

work page 2022

[7] [17]

(Boessen et al., 2013), and 130 subjects are diagnosed with mild depression (HAMD17 score ≤ 17). The HBN dataset, which includes a range of diseases, has significantly fewer labeled samples compared to the total data volume due to the high number of samples without explicit MDD and ASD diagnostic labels. Additionally, the number of labeled subjects differ...

work page 2013

[8] [2002]

Bert: Pre-training of deep bidirectional transformers for lan- guage understanding

Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for lan- guage understanding. arXiv preprint arXiv:1810.04805,

arXiv