REVIEW 4 major objections 5 minor 20 references
Attending To Syntactic Information In Biomedical Event Extraction Via Graph Neural Networks
T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read Using the full dependency parse graph, not just the shortest path between trigger and argument, improves biomedical event extraction when combined with BioBERT and a graph convolutional network.
desk verdict Plausible full-dependency-graph idea and a clean motivating problem, but the ablation doesn't isolate the graph and the SOTA claim is contradicted on Genia13; worth one serious referee round, not more. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the normalized full adjacency matrix of the dependency parse tree, obtained from an off-the-shelf parser and used as the graph the GCN convolves over. Node features are BioBERT token encodings, so the GCN re-embeds each word with its syntactic neighborhood. For argument role classification, the model concatenates a head representation computed by one MLP for the trigger token with a dependent representation computed by a second MLP for the argument token, then passes the concatenation through a linear layer with softmax. The two separate MLPs are the mechanism intended to counteract oversmoothing and oversquashing, by letting distant nodes interact in the classifier without adding more GCN layers.
What would settle it
Re-running BioBert-BEE and BioBert-GNN-BEE on the same splits with a shared evaluation script and identical preprocessing would settle the claim. If a properly tuned no-graph baseline matches the graph model's F1 on Genia11 and Genia13, then the dependency graph is not the source of the gain; likewise, if published systems re-implemented under the same conditions beat the graph model, the SOTA comparison fails.
Extended reading notes
Core claim
The paper aims to establish that feeding the full adjacency matrix of the dependency parsing graph through a two-layer graph convolutional network initialized with BioBERT's contextual token representations gives a practically useful token embedding for biomedical event extraction. Trigger and argument classification are performed by separate MLP networks over head and dependent tokens, introduced to reduce oversmoothing, oversquashing, and missing positional encoding. The ablation between the no-graph BioBert-BEE and the full-graph BioBert-GNN-BEE is the main evidence: total F1 rises from 60.97 to 69.81 on Genia11 and from 69.76 to 77.48 on Genia13. Against published systems, the model reports total F1 values of 69.81 on Genia11, 77.48 on Genia13, 70.29 on cancer genetics, and 75.70 on pathway curation, characterized as a slight improvement over prior state of the art.
Load-bearing premise
The reported advantage over published systems assumes the numbers being compared were produced under the same data splits, preprocessing, tokenization, and metric computation; if they were not, the slight state-of-the-art edge could be an artifact of evaluation differences rather than the graph.
Editorial extensions
If this is right
- Shortest-path and pruned-path syntax encodings should be replaced with full dependency graph encodings for event extraction, if the reported gains are real.
- The ablation numbers imply syntactic structure contributes substantially over plain contextual embeddings, with Genia11 total F1 rising from 60.97 to 69.81.
- A shallow GCN with separate head and dependent MLPs can handle long-range syntactic relations without the depth that usually causes oversmoothing.
- The design is practical: a pretrained biomedical language model plus an off-the-shelf dependency parser and a two-layer GCN yields several F1 points with no additional supervision.
Reading between the lines
- A natural extension would be to test whether the same full-graph GCN layer helps other structured prediction tasks with nested outputs, such as general relation extraction or abstract meaning representation parsing, where shortest-path heuristics are common.
- The comparisons are against published numbers rather than re-implemented baselines; a shared-benchmark study could show whether the slight gains persist under identical evaluation.
- Because the dependency graph comes from an off-the-shelf parser, corrupting the parse graph with random edge deletions could directly test whether full-graph completeness or something else drives the improvement.
- The approach injects syntax without extra training data, so it could cheaply be grafted onto other BioBERT-based pipelines, potentially benefiting tasks where negation and long-range arguments matter.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes two biomedical event extraction (BEE) models, BioBert-BEE and BioBert-GNN-BEE. The first uses a BioBERT encoder followed by sequence-labeling and argument-role classifiers. The second augments this architecture with a two-layer graph convolutional network over the full normalized dependency-parsing adjacency matrix of each sentence, with separate MLP 'head' and 'dependent' networks for trigger-argument scoring. The authors evaluate on Genia11, Genia13, pathway curation (pc), and cancer genetics (cg), reporting trigger identification/classification and argument identification/classification F1 scores. Their ablation study compares the two models to claim that dependency-graph information improves BEE, and they state that the proposed model slightly outperforms state-of-the-art models across datasets.
Significance. If the empirical claims were fully supported, the contribution would be practically useful: it replaces shortest-dependency-path pruning with the full dependency graph via a GCN on top of contextual BioBERT representations, and it provides a simple two-MLP mechanism to counter GCN oversmoothing. The authors are to be credited for testing across four datasets and for making the no-graph vs. graph comparison the central experiment. However, the current evidence is not yet convincing: the ablation does not isolate the syntactic edge information from added GCN capacity, the state-of-the-art claim is contradicted by the paper's own Table 1 on Genia13, and there are no statistical tests or shared evaluation details. The central empirical conclusion therefore needs substantial additional work before the paper can be accepted.
major comments (4)
- [§3.2, Tables 3–4] The ablation does not isolate the effect of dependency-graph information. BioBert-BEE has no GCN layers, while BioBert-GNN-BEE adds a two-layer GCN plus the MLP head/dependent blocks, so the large improvements (for example Genia11 total F1 from 60.97 to 69.81) could stem from additional parameters, feature propagation, or randomness rather than from syntactic edges. To support the claim that 'dependency parsing graph improves the performance,' the comparison must hold model capacity and propagation fixed, for example by comparing a GCN with the true adjacency matrix against a GCN with identity/self-loop adjacency or a randomly rewired adjacency. Without such a control, the causal attribution to syntactic information is not established.
- [§3.1, Table 1; Abstract] The abstract's claim that the model 'slightly outperforms state-of-the-art models on BEE over different datasets' is not supported by the paper's own numbers. On Genia13, OneEE's total F1 is 78.05 while the proposed model's total is 77.48, so the proposed model is worse on that dataset. The claim should be restricted to the datasets where the improvement actually holds (e.g., Genia11, pc, cg), or the discrepancy must be explained. In addition, the word 'significant' in the abstract is used without any statistical significance testing; the paper reports no standard deviations, no multiple runs, and no paired hypothesis tests, so apparent differences may not be reliable.
- [§3.1, Tables 1–2] The comparison against published state-of-the-art numbers is not controlled. Published baselines use their own preprocessing, tokenization, data splits, and metric definitions, so the 'total' scores may not be computed in the same way as the authors' macro-averaged total (which appears to average the four subtask F1 scores). For a credible SOTA comparison, the authors should either rerun the strongest baselines under their own evaluation script or clearly state that the comparison is against published numbers with potentially different metrics, and they should justify why such a comparison is valid. At minimum, the exact formula for 'total' and the metric-computation details must be specified.
- [§2.2 and §3.1] The manuscript omits essential implementation details needed to reproduce the central experiments. There is no specification of hyperparameters (learning rate, batch size, number of epochs, dropout, GCN hidden dimensions), no definition of how the adjacency matrix is normalized beyond the phrase 'normalise the full adjacency matrix,' and no description of the initialization of the GCN feature vectors or the random seed. The paper also does not provide a code repository or data-access details beyond saying the datasets are on Hugging Face. These omissions are load-bearing for an empirical paper whose main evidence is a new training configuration.
minor comments (5)
- [Throughout] The model name is written inconsistently as 'BioBert', 'BioBERT', 'bioBert', and 'BioBertGNNBEE'; please standardize the notation.
- [§2.2, Equation (3)] There are formatting errors around Equation (3): 'where|inEquation3 istheconcatenation' is missing spaces and the upright '|' is unclear; please rewrite this sentence and display the concatenation operator clearly.
- [Tables 1–2] The tables contain incomplete rows and inconsistent dashes (for example, GEANet and Extended TEES have only total scores), and the label 'ours-RO1-withGraph' is redundant. Please make the table headers and row labels consistent and add a note explaining how missing values are handled.
- [Figures 3–4] Figures 3 and 4 appear to be reproduced from another paper; if so, permission or explicit acknowledgment is required, and the source should be clearly credited in the caption.
- [References] Reference [13] lacks a venue and page numbers, and reference [20] lacks a publisher and location; some entries also have inconsistent author-name formatting. Please unify the bibliography style.
Circularity Check
No circularity: the claims are empirical comparisons against external baselines, with no fitted parameter renamed as a prediction and no load-bearing self-citation.
full rationale
The paper's central claim is that applying a GCN over the full dependency adjacency matrix on top of BioBERT improves biomedical event extraction. This is evaluated by training two systems and comparing them to published state-of-the-art numbers. There is no derivation in which an output quantity is defined in terms of the quantity it is supposed to predict, no fitted parameter is relabeled as a predicted result, and no load-bearing appeal to the authors' prior work. The ablation in Section 3.2 compares BioBert-BEE without a GCN to BioBert-GNN-BEE with a two-layer GCN; while this may confound the effect of syntactic edges with added model capacity, that is an experimental-design threat to internal validity, not circularity. The abstract's claim of 'slightly outperforming' state of the art is also not always supported by Table 1 (OneEE beats the proposed model on Genia13 total F1), but a mismatch between a claim and reported numbers is a correctness issue, not a circular-reasoning issue. No equation in the paper reduces to an input by construction, and the method is self-contained against external benchmark datasets.
Assumptions & free parameters
free parameters (2)
- Hidden dimensions and training hyperparameters
- Macro-averaged total metric =
mean of TI/TC/AI/AC F1
assumptions (4)
- domain assumption SciSpacy dependency parses are treated as error-free inputs; no learning or correction is applied to them.
- ad hoc to paper Full dependency graph adjacency provides better or equal information than pruned paths for event extraction.
- domain assumption The macro-averaged total F1 is a meaningful aggregate for comparing event extraction systems.
- standard math Standard GCN and MLP operations are assumed from the literature (e.g., [17]).
Cite this review
Pith. "Pith review of Attending To Syntactic Information In Biomedical Event Extraction Via Graph Neural Networks." pith.science (2026). https://pith.science/paper/32OT6G5M
@misc{pith2026250101158,
author = {Pith},
title = {Pith review of: Attending To Syntactic Information In Biomedical Event Extraction Via Graph Neural Networks},
year = {2026},
howpublished = {\url{https://pith.science/paper/32OT6G5M}},
note = {Machine review of arXiv:2501.01158}
}
read the original abstract
Many models are proposed in the literature on biomedical event extraction(BEE). Some of them use the shortest dependency path(SDP) information to represent the argument classification task. There is an issue with this representation since even missing one word from the dependency parsing graph may totally change the final prediction. To this end, the full adjacency matrix of the dependency graph is used to embed individual tokens using a graph convolutional network(GCN). An ablation study is also done to show the effect of the dependency graph on the overall performance. The results show a significant improvement when dependency graph information is used. The proposed model slightly outperforms state-of-the-art models on BEE over different datasets.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[16]
Neurocomputing422, 118–128 (01 2021)
Zhang, J., He, Q., Zhang, Y.: Syntax grounded graph convolutional network for joint entity and event extraction. Neurocomputing422, 118–128 (01 2021)
work page 2021
-
[1]
In: International Joint Conference on Artificial Intelligence (2020)
Abboud, R., Ceylan, I.I., Grohe, M., Lukasiewicz, T.: The surprising power of graph neural networks with random node initialization. In: International Joint Conference on Artificial Intelligence (2020)
work page 2020
-
[2]
In: Demner-Fushman, D., Cohen, K.B., Anani- adou, S., Tsujii, J
Bjorne, J., Salakoski, T.: Biomedical event extraction using convolutional neural networks and dependency parsing. In: Demner-Fushman, D., Cohen, K.B., Anani- adou, S., Tsujii, J. (eds.) Proceedings of the BioNLP 2018 workshop. pp. 98–108. Association for Computational Linguistics, Melbourne, Australia (Jul 2018)
work page 2018
-
[3]
In: Pro- ceedings of the 29th International Conference on Computational Linguistics
Cao, H., Li, J., Su, F., Li, F., Fei, H., Wu, S., Li, B., Zhao, L., Ji, D.: OneEE: A one-stage framework for fast overlapping and nested event extraction. In: Pro- ceedings of the 29th International Conference on Computational Linguistics. pp. 1953–1964. International Committee on Computational Linguistics, Gyeongju, Re- public of Korea (Oct 2022)
work page 1953
-
[4]
Dozat, T., Manning, C.D.: Deep biaffine attention for neural dependency parsing (2017)
work page 2017
-
[5]
In: Inui, K., Jiang, J., Ng, V., Wan, X
Espinosa, K.J., Miwa, M., Ananiadou, S.: A search-based neural model for biomed- ical nested and overlapping event detection. In: Inui, K., Jiang, J., Ng, V., Wan, X. (eds.) Proceedings of the 2019 Conference on Empirical Methods in Natural Lan- guage Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). pp. ...
work page 2019
-
[6]
MLPInit: Embarrassingly Simple GNN Training Acceleration with MLP Initialization
Han, X., Zhao, T., Liu, Y., Hu, X., Shah, N.: Mlpinit: Embarrassingly simple gnn training acceleration with mlp initialization. ArXivabs/2210.00102 (2022)
work page Pith review arXiv 2022
-
[7]
Huang, K.H., Yang, M., Peng, N.: Biomedical event extraction with hierarchi- cal knowledge graphs. pp. 1277–1285 (01 2020).https://doi.org/10.18653/v1/ 2020.findings-emnlp.114
doi:10.18653/v1/ 2020
Show all 20 references
-
[8]
In: Proceedings of the 57th Annual Meeting of the Association for Com- putational Linguistics
Ji, T., Wu, Y., Lan, M.: Graph-based dependency parsing with graph neural net- works. In: Proceedings of the 57th Annual Meeting of the Association for Com- putational Linguistics. pp. 2475–2485. Association for Computational Linguistics, Florence, Italy (Jul 2019)
2019
-
[9]
In: Burstein, J., Doran, C., Solorio, T
Li, D., Huang, L., Ji, H., Han, J.: Biomedical event extraction based on knowledge- driven tree-LSTM. In: Burstein, J., Doran, C., Solorio, T. (eds.) Proceedings of the 2019 Conference of the North American Chapter of the Association for Compu- tational Linguistics: Human Lang...
2019
-
[10]
In: Proceedings of the 58th Annual Meeting of the Associa- tion for Computational Linguistics
Lin, Y., Ji, H., Huang, F., Wu, L.: A joint neural model for information extraction with global features. In: Proceedings of the 58th Annual Meeting of the Associa- tion for Computational Linguistics. pp. 7999–8009. Association for Computational Linguistics, Online (Jul 2020)
2020
-
[11]
In: Erk, K., Smith, N.A
Miwa, M., Bansal, M.: End-to-end relation extraction using LSTMs on sequences and tree structures. In: Erk, K., Smith, N.A. (eds.) Proceedings of the 54th Annual MeetingoftheAssociationforComputationalLinguistics(Volume1:LongPapers). pp. 1105–1116. Association for Computationa...
2016
-
[12]
BMC Bioinformatics
Miwa M, A.S.: Adaptable, high recall, event extraction system with minimal con- figuration. BMC Bioinformatics. 2015
2015
-
[13]
Sheng, J., Guo, S., Yu, B., Li, Q., Hei, Y., Wang, L., Liu, T., Xu, H.: Casee: A joint learning framework with cascade decoding for overlapping event extraction (07 2021) 12 Authors Suppressed Due to Excessive Length
2021
-
[14]
Su, F., Qian, T., Zhou, J., Li, B., Li, F., Teng, C., Ji, D.: A tree-like structured perceptronfortransition-basedbiomedicaleventextraction.Knowledge-BasedSys- tems 283, 111180 (2024)
2024
-
[15]
(eds.) The 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks
Zanella, L., Toussaint, Y.: How much do knowledge graphs impact transformer models for extracting biomedical events? In: Demner-fushman, D., Ananiadou, S., Cohen, K. (eds.) The 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks. pp. 145–155. Associ...
2023
-
[17]
In: Riloff, E., Chiang, D., Hockenmaier, J., Tsujii, J
Zhang,Y., Qi, P.,Manning,C.D.:Graph convolution over pruned dependencytrees improves relation extraction. In: Riloff, E., Chiang, D., Hockenmaier, J., Tsujii, J. (eds.) Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. pp. 2205–2215. Assoc...
2018
-
[18]
In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Zhang, Z., Ji, H.: Abstract Meaning Representation guided graph encoding and decoding for joint information extraction. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. pp. 39–49...
2021
-
[19]
Information Sciences 550, 27–40 (2021)
Zhao, W., Zhang, J., Yang, J., He, T., Ma, H., Li, Z.: A novel joint biomedical event extraction framework via two-level modeling of documents. Information Sciences 550, 27–40 (2021)
2021
-
[20]
Zhu, L., Zheng, H.: Biomedical event extraction with a novel combination strategy based on hybrid deep neural networks (05 2020)
2020
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.