REVIEW 4 major objections 5 minor 23 references
M2oE: Multimodal Collaborative Expert Peptide Model
T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read M2oE fuses peptide sequence and graph structure with sparse mixture-of-experts and cross-attention, reporting 86.2% accuracy on antimicrobial peptide classification and an R-squared of 0.951 on aggregation propensity.
desk verdict A plausible architecture whose own results table undercuts its main claim: no AP gain over the best baseline, and a single-seed 1.5-point AMP edge. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The key object is the Sparse Cross Mixture of Experts (SCMoE) fusion module, which routes tokens from the sequence and graph encoders to shared experts using a learnable gating matrix with noisy top-$k$ selection, and then aligns the two modalities through cross-attention that swaps query vectors between the sequence and graph feature spaces. The gating noise gives low-ranked tokens a chance to be assigned to experts, and two auxiliary losses (load and importance) prevent expert collapse. A learnable scalar $\alpha$ in the final MLP weights the sequence and graph contributions, so the fusion adapts to the data distribution rather than using fixed weights.
What would settle it
Re-run M2oE and the best baselines on the same datasets with many random seeds and multiple 8:1:1 splits and compare the distributions; if the AMP accuracy advantage over GraphSAGE is smaller than the run-to-run spread, the central claim of superiority is not supported.
Extended reading notes
Core claim
M2oE jointly encodes a peptide's primary sequence with a Transformer and its bond structure with a graph encoder, then fuses both through a sparse cross mixture-of-experts (SCMoE) module that includes cross-attention between the two modalities and a learnable weight $\alpha$ that decides how much each modality contributes. The central claim is that this collaboration outperforms any single modality: on aggregation propensity it reports $R^2 = 0.951$ with MAE $3.68\times10^{-2}$ and MSE $2.21\times10^{-3}$, and on antimicrobial peptide classification it reports 86.2% accuracy, compared with 84.7% for the best graph-only baseline and 0.951 for the best sequence-only baseline. The paper also claims, based on ablations, that both the cross-attention and the mixture-of-experts routing are necessary for the gain.
Load-bearing premise
The reported advantage relies on the assumption that a single fixed 8:1:1 data split and one random seed produce stable evaluation numbers, so the 1.5-point accuracy gain is real rather than a fluctuation.
Editorial extensions
If this is right
- Single-modality models underperform on peptide datasets where their modality is weak; M2oE's routed fusion gives more consistent results across classification and regression.
- Removing the cross-attention or the mixture-of-experts routing degrades performance on the AP task, so both components are credited with the improvement.
- The learnable $\alpha$ lets the model adapt the sequence/graph balance to the dataset, which should transfer to other molecular property tasks.
- The reported 86.2% AMP accuracy is 1.5 percentage points above the best graph baseline, while AP $R^2$ matches the best sequence baseline at 0.951.
- The paper suggests the multimodal expert design can be extended to peptide generation tasks.
Reading between the lines
- If the single-split results are stable, we would expect M2oE's AMP margin to persist across random seeds; otherwise the 1.5-point gain may not be reproducible.
- The cross-attention weights could be inspected to see which amino acids and graph beads are aligned, offering a possible interpretability tool for peptide design.
- The same SCMoE design could be applied to other molecular systems where sequence and 3D structure both matter, such as protein-protein interfaces or RNA property prediction.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes M2oE, a multimodal peptide prediction model that combines a Transformer-based sequence encoder and a GCN/GraphSAGE-style graph encoder with a sparse cross mixture-of-experts (SCMoE) fusion module, cross-attention between modalities, a learnable fusion weight alpha, and auxiliary MoE load-balancing losses. The model is evaluated on two tasks from the Liu et al. benchmark: AMP classification (accuracy) and AP regression (MAE, MSE, R2), using an 8:1:1 split. The authors claim that M2oE outperforms single-modality and mixed baselines on both datasets, and that ablation experiments confirm the contribution of each module.
Significance. The paper has a clear and explicit architecture, provides a public code repository, and evaluates on an external benchmark with held-out test data, which are strengths. If the reported gains are reproducible, the work offers a modest but useful contribution by demonstrating that combining sequence and structure via an MoE router with cross-attention can improve peptide prediction. However, the current empirical support is thin: the key claim of supremacy on AP is contradicted by the paper's own results, and the AMP advantage rests on a single split with no error bars. The architecture is largely a composition of known components (MoE, cross-attention, learnable fusion), so the significance hinges on the evaluation being reliable.
major comments (4)
- [Section III, Table III] The headline claim that M2oE "surpasses baseline results" is not supported by Table III on the AP regression task. M2oE(Parallel) reports R2 = 0.951, which exactly matches SwitchTransformer's rounded R2 = 0.951, and M2oE's MAE (3.68E-2) and MSE (2.21E-3) are worse than SwitchTransformer's (3.65E-2 and 2.15E-3). Thus, the proposed model does not improve over the best sequence baseline on AP; it ties on R2 and is inferior on the other two metrics. The central claim as written is false on this task and must be revised or substantiated with further results.
- [Section III, Table III] On AMP, the only reported advantage of M2oE(Parallel) over GraphSAGE is 86.2% vs 84.7%, a 1.5 percentage-point gain from a single 8:1:1 split with no repeated runs, standard deviations, or significance tests (Section II-A). The fragility of this result is underscored by the fact that the other two fusion variants, M2oE(WS) and M2oE(Concat), perform worse than GraphSAGE on the same task. The authors should report multiple random seeds and appropriate significance tests before claiming superiority on AMP.
- [Section III, Table II and Table III] The ablation study in Table II is also based on single runs. The incremental R2 improvements from adding Cross-Attention and MoE are small (0.942 to 0.949 to 0.951, with the without-CRA variant at 0.949) and are likely within run-to-run variation. Without repeated experiments or confidence intervals, the conclusion that both components are effective is not statistically supported.
- [Section I and Section III] The comparison set in Table III is limited to generic sequence and graph models. No peptide-specific state-of-the-art predictors are included, notably the co-modeling method of Liu et al. [21] that provided the benchmark, or other peptide prediction methods mentioned in the introduction such as AMPEP [22]. Since the paper's contribution is specifically to peptide prediction, the absence of these baselines makes the significance claim difficult to assess.
minor comments (5)
- [Section III, first paragraph] The sentence "sequence and graph models are single-modality frameworks evaluated on classification (AP) and regression (AMP) datasets" is reversed: per Table I, AMP is the classification task and AP is the regression task. This should be corrected.
- [Section II-C, Equation (2)] Equation (2) is self-referential: the router formula uses alpha_j before it is defined on the following line, and the notation contains artifacts such as "T opk" and "Sof tplus". The formula needs to be rewritten so that all quantities are defined before use.
- [Section II-C, Cross-Attention paragraph] The text says "it can be represented as follows:" and is immediately followed by "where Fseq, Fgra denote features ... and dk is the scaling factor respectively," but no displayed equation appears before that sentence; the actual cross-attention expression is only implied in Equation (3). This formatting issue should be fixed.
- [Section III, Table III] The abbreviation "M2oE(WS)" is not defined; the authors should spell out that it refers to weighted-sum fusion.
- [Section II and Section III] The manuscript does not report training hyperparameters (number of experts C, top-k value, learning rate, batch size, epochs, hidden dimensions, random seed, or alpha initialization). The code link is appreciated, but the core experimental setup should be stated in the paper to allow independent reproduction.
Circularity Check
No significant circularity: the model is trained and evaluated on an external benchmark, and all claimed components are standard trainable mechanisms rather than post-hoc fitted inputs.
full rationale
The paper's central claims are empirical results from training M2oE on the Liu et al. benchmark datasets with an 8:1:1 split, and the reported metrics are evaluated on held-out test sets. No derivation step reduces to its own input: the learnable fusion weight α in Equation 4 is a standard trained parameter, not a constant fitted to the test set and then reported as a prediction. The SCMoE routing, cross-attention, and auxiliary losses are all defined as trainable components with stated formulas, and their contributions are assessed through ablation experiments on the same external data. The paper cites prior work for benchmark construction and baseline architectures, but these citations are not self-citations and do not function as the sole justification for the reported performance. Even where the paper's prose overstates the comparison results, such as claiming to surpass baselines when Table III shows an R2 tie on AP and a single-seed 1.5-point gain on AMP, that is a correctness or statistical-validity concern, not a circularity concern. Consequently, no circular step can be exhibited from the paper's own equations or citation chain.
Assumptions & free parameters
free parameters (4)
- alpha fusion weight =
unknown
- expert count C =
unknown
- top-k routing value =
unknown
- importance loss weight omega_imp =
unknown
assumptions (4)
- standard math GCN update rule with normalized adjacency matrix (Eq. 1) correctly propagates node features.
- domain assumption The graph representation of peptides, with nodes as beads and edges as chemical bonds sourced from Liu et al. [21], adequately captures spatial structure.
- domain assumption Dataset splits (8:1:1) from Liu et al. [21] are unbiased and reproduceable for comparison.
- ad hoc to paper Adding Gaussian noise to the top-k router allocation improves expert utilization.
Cite this review
Pith. "Pith review of M2oE: Multimodal Collaborative Expert Peptide Model." pith.science (2026). https://pith.science/paper/EAIR3L5H
@misc{pith2026241115208,
author = {Pith},
title = {Pith review of: M2oE: Multimodal Collaborative Expert Peptide Model},
year = {2026},
howpublished = {\url{https://pith.science/paper/EAIR3L5H}},
note = {Machine review of arXiv:2411.15208}
}
read the original abstract
Peptides are biomolecules comprised of amino acids that play an important role in our body. In recent years, peptides have received extensive attention in drug design and synthesis, and peptide prediction tasks help us better search for functional peptides. Typically, we use the primary sequence and structural information of peptides for model encoding. However, recent studies have focused more on single-modal information (structure or sequence) for prediction without multi-modal approaches. We found that single-modal models are not good at handling datasets with less information in that particular modality. Therefore, this paper proposes the M2oE multi-modal collaborative expert peptide model. Based on previous work, by integrating sequence and spatial structural information, employing expert model and Cross-Attention Mechanism, the model's capabilities are balanced and improved. Experimental results indicate that the M2oE model performs excellently in complex task predictions.
Figures
Reference graph
Works this paper leans on
-
[21]
Co-modeling the sequential and graphical routes for peptide representation learning,
Z. Liu, G. Wang, J. Wang, J. Zheng, and S. Z. Li, “Co-modeling the sequential and graphical routes for peptide representation learning,” arXiv e-prints, pp. arXiv–2310, 2023
work page 2023
-
[22]
P. Bhadra, J. Yan, J. Li, S. Fong, and S. W. Siu, “Ampep: Sequence-based prediction of antimicrobial peptides using distribution patterns of amino acid properties and random forest,” Scientific reports, vol. 8, no. 1, p. 1697, 2018
work page 2018
-
[1]
I. W. Hamley, Introduction to peptide science . John Wiley & Sons, 2020
work page 2020
-
[2]
Deepmspeptide: peptide detectability prediction using deep learning,
G. Serrano, E. Guruceaga, and V . Segura, “Deepmspeptide: peptide detectability prediction using deep learning,” Bioinformatics, vol. 36, no. 4, pp. 1279–1280, 2020
work page 2020
-
[3]
X. Chen, C. Li, M. T. Bernards, Y . Shi, Q. Shao, and Y . He, “Sequence- based peptide identification, generation, and property prediction with deep learning: a review,” Molecular Systems Design & Engineering , vol. 6, no. 6, pp. 406–428, 2021
work page 2021
-
[4]
Explainable deep hypergraph learning modeling the peptide secondary structure prediction,
Y . Jiang, R. Wang, J. Feng, J. Jin, S. Liang, Z. Li, Y . Yu, A. Ma, R. Su, Q. Zou et al., “Explainable deep hypergraph learning modeling the peptide secondary structure prediction,” Advanced Science, vol. 10, no. 11, p. 2206151, 2023
work page 2023
-
[5]
Deep learning for novel antimicro- bial peptide design,
C. Wang, S. Garlick, and M. Zloh, “Deep learning for novel antimicro- bial peptide design,” Biomolecules, vol. 11, no. 3, p. 471, 2021
work page 2021
-
[6]
Models and data of amplify: a deep learning tool for antimicrobial peptide prediction,
C. Li, R. L. Warren, and I. Birol, “Models and data of amplify: a deep learning tool for antimicrobial peptide prediction,” BMC Research Notes, vol. 16, no. 1, p. 11, 2023
work page 2023
Show all 23 references
-
[7]
xdeep-acpep: deep learning method for anticancer peptide activity prediction based on convolutional neural network and multitask learning,
J. Chen, H. H. Cheong, and S. W. Siu, “xdeep-acpep: deep learning method for anticancer peptide activity prediction based on convolutional neural network and multitask learning,” Journal of chemical information and modeling, vol. 61, no. 8, pp. 3789–3803, 2021
2021
-
[8]
Finding structure in time,
J. L. Elman, “Finding structure in time,” Cognitive science , vol. 14, no. 2, pp. 179–211, 1990
1990
-
[9]
Long short-term memory,
S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997
1997
-
[10]
Bidirectional recurrent neural net- works,
M. Schuster and K. K. Paliwal, “Bidirectional recurrent neural net- works,” IEEE transactions on Signal Processing , vol. 45, no. 11, pp. 2673–2681, 1997
1997
-
[11]
Attention is all you need,
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017
2017
-
[12]
Semi-supervised classification with graph convolutional networks,
T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” arXiv preprint arXiv:1609.02907 , 2016
2016 arXiv
-
[13]
Efficient prediction of peptide self-assembly through sequential and graphical encoding,
Z. Liu, J. Wang, Y . Luo, S. Zhao, W. Li, and S. Z. Li, “Efficient prediction of peptide self-assembly through sequential and graphical encoding,” Briefings in Bioinformatics, vol. 24, no. 6, p. bbad409, 2023
2023
-
[14]
Git-mol: A multi-modal large language model for molecular science with graph, image, and text,
P. Liu, Y . Ren, J. Tao, and Z. Ren, “Git-mol: A multi-modal large language model for molecular science with graph, image, and text,” Computers in Biology and Medicine , vol. 171, p. 108073, 2024
2024
-
[15]
Graph mixture of experts: Learning on large-scale graphs with explicit diversity modeling,
H. Wang, Z. Jiang, Y . You, Y . Han, G. Liu, J. Srinivasa, R. Kompella, Z. Wang et al. , “Graph mixture of experts: Learning on large-scale graphs with explicit diversity modeling,” Advances in Neural Informa- tion Processing Systems , vol. 36, 2024
2024
-
[16]
Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity,
W. Fedus, B. Zoph, and N. Shazeer, “Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity,” Journal of Machine Learning Research , vol. 23, no. 120, pp. 1–39, 2022
2022
-
[17]
Using hierarchi- cal mixture of experts model for fusion of outbreak detection methods,
N. Jafarpour, D. Precup, M. Izadi, and D. Buckeridge, “Using hierarchi- cal mixture of experts model for fusion of outbreak detection methods,” in AMIA Annual Symposium Proceedings, vol. 2013. American Medical Informatics Association, 2013, p. 663
2013
-
[18]
A multimodal mixture-of-experts model for dynamic emotion prediction in movies,
A. Goyal, N. Kumar, T. Guha, and S. S. Narayanan, “A multimodal mixture-of-experts model for dynamic emotion prediction in movies,” in 2016 ieee international conference on acoustics, speech and signal processing (icassp). IEEE, 2016, pp. 2822–2826
2016
-
[19]
Mixture- of-experts for biomedical question answering,
D. Dai, W. Jiang, J. Zhang, Y . Lyu, Z. Sui, and B. Chang, “Mixture- of-experts for biomedical question answering,” in CCF International Conference on Natural Language Processing and Chinese Computing . Springer, 2023, pp. 604–615
2023
-
[20]
Task-customized mixture of adapters for general image fusion,
P. Zhu, Y . Sun, B. Cao, and Q. Hu, “Task-customized mixture of adapters for general image fusion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 7099–7108
2024
-
[23]
Outrageously large neural networks: The sparsely-gated mixture-of-experts layer,
N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean, “Outrageously large neural networks: The sparsely-gated mixture-of-experts layer,” arXiv preprint arXiv:1701.06538 , 2017. 4
2017 arXiv
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.