Pith. sign in

REVIEW 4 major objections 5 minor 20 references

Attending To Syntactic Information In Biomedical Event Extraction Via Graph Neural Networks

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Pith's one-line read Using the full dependency parse graph, not just the shortest path between trigger and argument, improves biomedical event extraction when combined with BioBERT and a graph convolutional network.

desk verdict Plausible full-dependency-graph idea and a clean motivating problem, but the ablation doesn't isolate the graph and the SOTA claim is contradicted on Genia13; worth one serious referee round, not more. read the letter →

arxiv 2501.01158 v2 pith:32OT6G5M submitted 2025-01-02 cs.CL

classification cs.CL
keywords biomedicaleventextractiongraphconvolutionalnetworkdependencyparsingBioBERTsyntacticinformationargumentclassificationnestedeventsNLPsharedtasks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's central claim is that biomedical event extraction improves when token representations are enriched by the entire dependency parse of a sentence rather than by the shortest path between a trigger and its argument. The authors build two models with identical structure, one with and one without a graph convolutional network over the full dependency adjacency matrix on top of BioBERT, and an ablation shows the graph version wins on all four datasets. The paper also reports that the graph model slightly outperforms published state-of-the-art systems on the Genia11, Genia13, cancer genetics, and pathway curation datasets. The authors argue the full graph avoids the fragility of shortest-path representations, where dropping a single word such as 'not' can flip a prediction.

What carries the argument

The load-bearing object is the normalized full adjacency matrix of the dependency parse tree, obtained from an off-the-shelf parser and used as the graph the GCN convolves over. Node features are BioBERT token encodings, so the GCN re-embeds each word with its syntactic neighborhood. For argument role classification, the model concatenates a head representation computed by one MLP for the trigger token with a dependent representation computed by a second MLP for the argument token, then passes the concatenation through a linear layer with softmax. The two separate MLPs are the mechanism intended to counteract oversmoothing and oversquashing, by letting distant nodes interact in the classifier without adding more GCN layers.

What would settle it

Re-running BioBert-BEE and BioBert-GNN-BEE on the same splits with a shared evaluation script and identical preprocessing would settle the claim. If a properly tuned no-graph baseline matches the graph model's F1 on Genia11 and Genia13, then the dependency graph is not the source of the gain; likewise, if published systems re-implemented under the same conditions beat the graph model, the SOTA comparison fails.

Watch

Extended reading notes

Core claim

The paper aims to establish that feeding the full adjacency matrix of the dependency parsing graph through a two-layer graph convolutional network initialized with BioBERT's contextual token representations gives a practically useful token embedding for biomedical event extraction. Trigger and argument classification are performed by separate MLP networks over head and dependent tokens, introduced to reduce oversmoothing, oversquashing, and missing positional encoding. The ablation between the no-graph BioBert-BEE and the full-graph BioBert-GNN-BEE is the main evidence: total F1 rises from 60.97 to 69.81 on Genia11 and from 69.76 to 77.48 on Genia13. Against published systems, the model reports total F1 values of 69.81 on Genia11, 77.48 on Genia13, 70.29 on cancer genetics, and 75.70 on pathway curation, characterized as a slight improvement over prior state of the art.

Load-bearing premise

The reported advantage over published systems assumes the numbers being compared were produced under the same data splits, preprocessing, tokenization, and metric computation; if they were not, the slight state-of-the-art edge could be an artifact of evaluation differences rather than the graph.

Editorial extensions

If this is right

  • Shortest-path and pruned-path syntax encodings should be replaced with full dependency graph encodings for event extraction, if the reported gains are real.
  • The ablation numbers imply syntactic structure contributes substantially over plain contextual embeddings, with Genia11 total F1 rising from 60.97 to 69.81.
  • A shallow GCN with separate head and dependent MLPs can handle long-range syntactic relations without the depth that usually causes oversmoothing.
  • The design is practical: a pretrained biomedical language model plus an off-the-shelf dependency parser and a two-layer GCN yields several F1 points with no additional supervision.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension would be to test whether the same full-graph GCN layer helps other structured prediction tasks with nested outputs, such as general relation extraction or abstract meaning representation parsing, where shortest-path heuristics are common.
  • The comparisons are against published numbers rather than re-implemented baselines; a shared-benchmark study could show whether the slight gains persist under identical evaluation.
  • Because the dependency graph comes from an off-the-shelf parser, corrupting the parse graph with random edge deletions could directly test whether full-graph completeness or something else drives the improvement.
  • The approach injects syntax without extra training data, so it could cheaply be grafted onto other BioBERT-based pipelines, potentially benefiting tasks where negation and long-range arguments matter.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes two biomedical event extraction (BEE) models, BioBert-BEE and BioBert-GNN-BEE. The first uses a BioBERT encoder followed by sequence-labeling and argument-role classifiers. The second augments this architecture with a two-layer graph convolutional network over the full normalized dependency-parsing adjacency matrix of each sentence, with separate MLP 'head' and 'dependent' networks for trigger-argument scoring. The authors evaluate on Genia11, Genia13, pathway curation (pc), and cancer genetics (cg), reporting trigger identification/classification and argument identification/classification F1 scores. Their ablation study compares the two models to claim that dependency-graph information improves BEE, and they state that the proposed model slightly outperforms state-of-the-art models across datasets.

Significance. If the empirical claims were fully supported, the contribution would be practically useful: it replaces shortest-dependency-path pruning with the full dependency graph via a GCN on top of contextual BioBERT representations, and it provides a simple two-MLP mechanism to counter GCN oversmoothing. The authors are to be credited for testing across four datasets and for making the no-graph vs. graph comparison the central experiment. However, the current evidence is not yet convincing: the ablation does not isolate the syntactic edge information from added GCN capacity, the state-of-the-art claim is contradicted by the paper's own Table 1 on Genia13, and there are no statistical tests or shared evaluation details. The central empirical conclusion therefore needs substantial additional work before the paper can be accepted.

major comments (4)
  1. [§3.2, Tables 3–4] The ablation does not isolate the effect of dependency-graph information. BioBert-BEE has no GCN layers, while BioBert-GNN-BEE adds a two-layer GCN plus the MLP head/dependent blocks, so the large improvements (for example Genia11 total F1 from 60.97 to 69.81) could stem from additional parameters, feature propagation, or randomness rather than from syntactic edges. To support the claim that 'dependency parsing graph improves the performance,' the comparison must hold model capacity and propagation fixed, for example by comparing a GCN with the true adjacency matrix against a GCN with identity/self-loop adjacency or a randomly rewired adjacency. Without such a control, the causal attribution to syntactic information is not established.
  2. [§3.1, Table 1; Abstract] The abstract's claim that the model 'slightly outperforms state-of-the-art models on BEE over different datasets' is not supported by the paper's own numbers. On Genia13, OneEE's total F1 is 78.05 while the proposed model's total is 77.48, so the proposed model is worse on that dataset. The claim should be restricted to the datasets where the improvement actually holds (e.g., Genia11, pc, cg), or the discrepancy must be explained. In addition, the word 'significant' in the abstract is used without any statistical significance testing; the paper reports no standard deviations, no multiple runs, and no paired hypothesis tests, so apparent differences may not be reliable.
  3. [§3.1, Tables 1–2] The comparison against published state-of-the-art numbers is not controlled. Published baselines use their own preprocessing, tokenization, data splits, and metric definitions, so the 'total' scores may not be computed in the same way as the authors' macro-averaged total (which appears to average the four subtask F1 scores). For a credible SOTA comparison, the authors should either rerun the strongest baselines under their own evaluation script or clearly state that the comparison is against published numbers with potentially different metrics, and they should justify why such a comparison is valid. At minimum, the exact formula for 'total' and the metric-computation details must be specified.
  4. [§2.2 and §3.1] The manuscript omits essential implementation details needed to reproduce the central experiments. There is no specification of hyperparameters (learning rate, batch size, number of epochs, dropout, GCN hidden dimensions), no definition of how the adjacency matrix is normalized beyond the phrase 'normalise the full adjacency matrix,' and no description of the initialization of the GCN feature vectors or the random seed. The paper also does not provide a code repository or data-access details beyond saying the datasets are on Hugging Face. These omissions are load-bearing for an empirical paper whose main evidence is a new training configuration.
minor comments (5)
  1. [Throughout] The model name is written inconsistently as 'BioBert', 'BioBERT', 'bioBert', and 'BioBertGNNBEE'; please standardize the notation.
  2. [§2.2, Equation (3)] There are formatting errors around Equation (3): 'where|inEquation3 istheconcatenation' is missing spaces and the upright '|' is unclear; please rewrite this sentence and display the concatenation operator clearly.
  3. [Tables 1–2] The tables contain incomplete rows and inconsistent dashes (for example, GEANet and Extended TEES have only total scores), and the label 'ours-RO1-withGraph' is redundant. Please make the table headers and row labels consistent and add a note explaining how missing values are handled.
  4. [Figures 3–4] Figures 3 and 4 appear to be reproduced from another paper; if so, permission or explicit acknowledgment is required, and the source should be clearly credited in the caption.
  5. [References] Reference [13] lacks a venue and page numbers, and reference [20] lacks a publisher and location; some entries also have inconsistent author-name formatting. Please unify the bibliography style.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the claims are empirical comparisons against external baselines, with no fitted parameter renamed as a prediction and no load-bearing self-citation.

full rationale

The paper's central claim is that applying a GCN over the full dependency adjacency matrix on top of BioBERT improves biomedical event extraction. This is evaluated by training two systems and comparing them to published state-of-the-art numbers. There is no derivation in which an output quantity is defined in terms of the quantity it is supposed to predict, no fitted parameter is relabeled as a predicted result, and no load-bearing appeal to the authors' prior work. The ablation in Section 3.2 compares BioBert-BEE without a GCN to BioBert-GNN-BEE with a two-layer GCN; while this may confound the effect of syntactic edges with added model capacity, that is an experimental-design threat to internal validity, not circularity. The abstract's claim of 'slightly outperforming' state of the art is also not always supported by Table 1 (OneEE beats the proposed model on Genia13 total F1), but a mismatch between a claim and reported numbers is a correctness issue, not a circular-reasoning issue. No equation in the paper reduces to an input by construction, and the method is self-contained against external benchmark datasets.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central result depends on the choice to use SciSpacy parses as ground truth, the architectural decision to encode the full graph with a two-layer GCN, and the metric aggregation. No new physical or mathematical entities are introduced. The main untracked degrees of freedom are undisclosed hyperparameters.

free parameters (2)
  • Hidden dimensions and training hyperparameters
    The architecture's behavior depends on undisclosed choices such as hidden size, dropout, learning rate, and number of epochs. The paper only states a two-layer GCN.
  • Macro-averaged total metric = mean of TI/TC/AI/AC F1
    The 'total' F1 used to rank systems is the unweighted mean of the four subtask F1s; baseline papers may not use the same aggregation, which can change the ordering.
assumptions (4)
  • domain assumption SciSpacy dependency parses are treated as error-free inputs; no learning or correction is applied to them.
    Section 2 states SciSpacy is used and 'no learning is required to obtain such syntactic information', so parse errors are assumed to be negligible.
  • ad hoc to paper Full dependency graph adjacency provides better or equal information than pruned paths for event extraction.
    The paper motivates full-graph embedding with the 'missing word' argument from [17], but the claim that the full graph is superior is not derived, only supported by the ablation.
  • domain assumption The macro-averaged total F1 is a meaningful aggregate for comparing event extraction systems.
    In Table 1, systems are ranked by 'total', defined as the mean of four subtask F1s, but this weighting may not match the original papers.
  • standard math Standard GCN and MLP operations are assumed from the literature (e.g., [17]).
    The equations in Section 2 rely on standard softmax, ReLU, and graph convolution definitions without derivation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Attending To Syntactic Information In Biomedical Event Extraction Via Graph Neural Networks." pith.science (2026). https://pith.science/paper/32OT6G5M

@misc{pith2026250101158,
  author       = {Pith},
  title        = {Pith review of: Attending To Syntactic Information In Biomedical Event Extraction Via Graph Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/32OT6G5M}},
  note         = {Machine review of arXiv:2501.01158}
}
read the original abstract

Many models are proposed in the literature on biomedical event extraction(BEE). Some of them use the shortest dependency path(SDP) information to represent the argument classification task. There is an issue with this representation since even missing one word from the dependency parsing graph may totally change the final prediction. To this end, the full adjacency matrix of the dependency graph is used to embed individual tokens using a graph convolutional network(GCN). An ablation study is also done to show the effect of the dependency graph on the overall performance. The results show a significant improvement when dependency graph information is used. The proposed model slightly outperforms state-of-the-art models on BEE over different datasets.

Figures

Figures reproduced from arXiv: 2501.01158 by the authors.

Figure 1
Figure 1. An example of pathway curation sample from BioNLP13-pc indicating the subject is dependent on the verb. There exists many open source dependency parsing libraries such as Scispacy and StanfordNLP which provide very accurate dependency parsing graph. Nonetheless, some researchers argue that these models have some errors that propagates downstream task and they prefer to use large language models(LLM) to circumvent th… view at source ↗
Figure 2
Figure 2. an example of a dependency parse from BioNLP2011 [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The architecture of tree LSTMRNN model proposed in [11] [17] uses GNN for better representations of entities for the task of relation extraction but the present paper aims to extend the use of GNN to BEE. [17] introduced a contextualized GCN (C-GCN) model, where the input word vectors are first fed into a BiLSTM network to generate contextualized representations, which are then used as to initialise the GNN. Pruning… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: taken from [17] that shows SDP between two entities in the bold. the tree down to the path, K = 1 keeps all nodes that are directly attached to the path, and K = ∞ retains the entire LCA subtree. They combine this pruning strategy with GCN model, by directly feeding th…
Figure 5
Figure 5. Figure 5: architecture of bioBert based biomedical event extraction model [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: architecture of GNN based biomedical event extraction model which is ini￾tialised by the encoder of bioBertBEE [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 19 canonical work pages

  1. [16]

    Neurocomputing422, 118–128 (01 2021)

    Zhang, J., He, Q., Zhang, Y.: Syntax grounded graph convolutional network for joint entity and event extraction. Neurocomputing422, 118–128 (01 2021)

  2. [1]

    In: International Joint Conference on Artificial Intelligence (2020)

    Abboud, R., Ceylan, I.I., Grohe, M., Lukasiewicz, T.: The surprising power of graph neural networks with random node initialization. In: International Joint Conference on Artificial Intelligence (2020)

  3. [2]

    In: Demner-Fushman, D., Cohen, K.B., Anani- adou, S., Tsujii, J

    Bjorne, J., Salakoski, T.: Biomedical event extraction using convolutional neural networks and dependency parsing. In: Demner-Fushman, D., Cohen, K.B., Anani- adou, S., Tsujii, J. (eds.) Proceedings of the BioNLP 2018 workshop. pp. 98–108. Association for Computational Linguistics, Melbourne, Australia (Jul 2018)

  4. [3]

    In: Pro- ceedings of the 29th International Conference on Computational Linguistics

    Cao, H., Li, J., Su, F., Li, F., Fei, H., Wu, S., Li, B., Zhao, L., Ji, D.: OneEE: A one-stage framework for fast overlapping and nested event extraction. In: Pro- ceedings of the 29th International Conference on Computational Linguistics. pp. 1953–1964. International Committee on Computational Linguistics, Gyeongju, Re- public of Korea (Oct 2022)

  5. [4]

    Dozat, T., Manning, C.D.: Deep biaffine attention for neural dependency parsing (2017)

  6. [5]

    In: Inui, K., Jiang, J., Ng, V., Wan, X

    Espinosa, K.J., Miwa, M., Ananiadou, S.: A search-based neural model for biomed- ical nested and overlapping event detection. In: Inui, K., Jiang, J., Ng, V., Wan, X. (eds.) Proceedings of the 2019 Conference on Empirical Methods in Natural Lan- guage Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). pp. ...

  7. [6]

    MLPInit: Embarrassingly Simple GNN Training Acceleration with MLP Initialization

    Han, X., Zhao, T., Liu, Y., Hu, X., Shah, N.: Mlpinit: Embarrassingly simple gnn training acceleration with mlp initialization. ArXivabs/2210.00102 (2022)

  8. [7]

    Huang, K.H., Yang, M., Peng, N.: Biomedical event extraction with hierarchi- cal knowledge graphs. pp. 1277–1285 (01 2020).https://doi.org/10.18653/v1/ 2020.findings-emnlp.114

Show all 20 references
  1. [8]

    In: Proceedings of the 57th Annual Meeting of the Association for Com- putational Linguistics

    Ji, T., Wu, Y., Lan, M.: Graph-based dependency parsing with graph neural net- works. In: Proceedings of the 57th Annual Meeting of the Association for Com- putational Linguistics. pp. 2475–2485. Association for Computational Linguistics, Florence, Italy (Jul 2019)

  2. [9]

    In: Burstein, J., Doran, C., Solorio, T

    Li, D., Huang, L., Ji, H., Han, J.: Biomedical event extraction based on knowledge- driven tree-LSTM. In: Burstein, J., Doran, C., Solorio, T. (eds.) Proceedings of the 2019 Conference of the North American Chapter of the Association for Compu- tational Linguistics: Human Lang...

  3. [10]

    In: Proceedings of the 58th Annual Meeting of the Associa- tion for Computational Linguistics

    Lin, Y., Ji, H., Huang, F., Wu, L.: A joint neural model for information extraction with global features. In: Proceedings of the 58th Annual Meeting of the Associa- tion for Computational Linguistics. pp. 7999–8009. Association for Computational Linguistics, Online (Jul 2020)

  4. [11]

    In: Erk, K., Smith, N.A

    Miwa, M., Bansal, M.: End-to-end relation extraction using LSTMs on sequences and tree structures. In: Erk, K., Smith, N.A. (eds.) Proceedings of the 54th Annual MeetingoftheAssociationforComputationalLinguistics(Volume1:LongPapers). pp. 1105–1116. Association for Computationa...

  5. [12]

    BMC Bioinformatics

    Miwa M, A.S.: Adaptable, high recall, event extraction system with minimal con- figuration. BMC Bioinformatics. 2015

  6. [13]

    Sheng, J., Guo, S., Yu, B., Li, Q., Hei, Y., Wang, L., Liu, T., Xu, H.: Casee: A joint learning framework with cascade decoding for overlapping event extraction (07 2021) 12 Authors Suppressed Due to Excessive Length

  7. [14]

    Su, F., Qian, T., Zhou, J., Li, B., Li, F., Teng, C., Ji, D.: A tree-like structured perceptronfortransition-basedbiomedicaleventextraction.Knowledge-BasedSys- tems 283, 111180 (2024)

  8. [15]

    (eds.) The 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks

    Zanella, L., Toussaint, Y.: How much do knowledge graphs impact transformer models for extracting biomedical events? In: Demner-fushman, D., Ananiadou, S., Cohen, K. (eds.) The 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks. pp. 145–155. Associ...

  9. [17]

    In: Riloff, E., Chiang, D., Hockenmaier, J., Tsujii, J

    Zhang,Y., Qi, P.,Manning,C.D.:Graph convolution over pruned dependencytrees improves relation extraction. In: Riloff, E., Chiang, D., Hockenmaier, J., Tsujii, J. (eds.) Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. pp. 2205–2215. Assoc...

  10. [18]

    In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

    Zhang, Z., Ji, H.: Abstract Meaning Representation guided graph encoding and decoding for joint information extraction. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. pp. 39–49...

  11. [19]

    Information Sciences 550, 27–40 (2021)

    Zhao, W., Zhang, J., Yang, J., He, T., Ma, H., Li, Z.: A novel joint biomedical event extraction framework via two-level modeling of documents. Information Sciences 550, 27–40 (2021)

  12. [20]

    Zhu, L., Zheng, H.: Biomedical event extraction with a novel combination strategy based on hybrid deep neural networks (05 2020)

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.