Pith. sign in

REVIEW 2 major objections 5 minor 50 references

An Improved Neural Baseline for Temporal Relation Extraction

T0 review · 2 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a fairly standard LSTM, given contextualized word embeddings and a Siamese encoder of a temporal common-sense knowledge base, beats the previous best feature-based system on temporal relation extraction by about 10…

desk verdict A solid neural baseline paper whose headline result survives scrutiny; the CSE component's 2-point gain is real but not cleanly attributable to external common sense because it was trained on CogCompTime's own output. read the letter →

arxiv 1909.00429 v1 pith:JS634CLP submitted 2019-09-01 cs.CL

Qiang Ning , Sanjay Subramanian , Dan Roth This is my paper · ORCID
classification cs.CL
keywords temporalrelationextractionTempRelMATREScommonsenseencoderSiamesenetworkcontextualizedembeddingsintegerlinearprogrammingLSTMbaseline
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Temporal relation extraction — deciding whether one event happens before, after, or simultaneously with another — has resisted neural methods, partly because training data has been small and noisy. This paper argues that a standard LSTM, combined with contextualized word embeddings (ELMo or BERT) and a Siamese encoder trained on a corpus-derived temporal knowledge base, outperforms the previous best feature-based system by about 10 percent absolute accuracy and F1, a 25 percent error reduction. The gains hold on two benchmark datasets, MATRES and TCR. The paper's larger point is that neural approaches were previously limited less by architecture or data size than by annotation quality, and the new system can serve as a strong baseline.

What carries the argument

The system combines four components: (1) an LSTM that encodes each event by the hidden state at its position and concatenates the two event hidden states to represent an event pair; (2) a Siamese network trained to fit TEMPROB, an automatically induced table of typical temporal relations between verb pairs, which generalizes to unseen or rare verb pairs through learned verb embeddings; (3) a feedforward network that turns the concatenated LSTM and CSE representations into confidence scores over before, after, equal, and vague; and (4) integer linear programming that enforces global transitivity constraints on the predicted temporal graph.

What would settle it

Rebuild TEMPROB with a different base extractor, or with no extractor at all, then retrain the Siamese encoder and evaluate Concat+CSE on MATRES and TCR; if the margin over plain Concat disappears or becomes insignificant, the CSE gain is an artifact of the baseline's own supervision rather than generalizable temporal knowledge.

Watch

Extended reading notes

Core claim

On MATRES, the Concat+CSE system reaches about 71.7 accuracy, 76.7 F1, 66.0 awareness, and 71.5 three-metric average, versus CogCompTime's 61.6, 66.6, 60.8, and 63.0, with p < 0.001 under McNemar's test. The gains are roughly composed of 2 points from the LSTM formulation itself, 4 points from contextualized embeddings, and 2 points from the common sense encoder. On TCR, the same system outperforms CogCompTime by roughly 8 percent on the three-metric average. The paper also finds that simply concatenating the LSTM hidden states at the two event positions works as well as inserting XML position indicators, and that ELMo and BERT give statistically significant improvements over static embeddings while not differing significantly from each other.

Load-bearing premise

The common sense encoder is trained on statistics produced by the very system it is measured against, so if those statistics carry CogCompTime's systematic errors or label biases, the reported 2-point CSE gain is not a clean measurement of external temporal common sense.

Editorial extensions

If this is right

  • A standard LSTM with contextualized embeddings can serve as a strong baseline for temporal relation extraction, replacing feature-based systems without architectural innovation.
  • The finding that simple concatenation of event-position hidden states matches XML position indicators simplifies future neural designs for this task.
  • The sizable gains from ELMo and BERT over static embeddings suggest contextualized representations carry information essential for temporal ordering.
  • The roughly 2-point contribution from the common sense encoder indicates that external temporal knowledge, when injected compactly, adds measurable value beyond context.
  • If the bottleneck was annotation quality rather than architecture, further improvements should come from more high-quality multi-axis temporal annotations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because TEMPROB was reconstructed with CogCompTime as the base method, the CSE's contribution may partly encode the baseline's own biases; a cleaner test would train CSE on a knowledge base built with an independent or randomized extractor and check whether the 2-point gain persists.
  • The paper does not directly probe CSE's generalization to unseen verb pairs; a focused evaluation comparing CSE outputs on rare tuples against human temporal judgments would test whether the Siamese encoder truly supplies common sense or merely interpolates familiar statistics.
  • If the data-quality interpretation is correct, then collecting larger MATRES-style same-axis annotations should yield further neural gains without changing the architecture, a prediction that future dataset releases could settle.
  • The reported decomposition into LSTM, contextualized embeddings, and CSE contributions is specific to these benchmarks; transferring the same system to clinical or multilingual temporal corpora would reveal which components travel.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. This paper proposes a neural baseline for temporal relation extraction based on a bidirectional LSTM over word embeddings, with event positions handled either by XML position indicators or by concatenating the hidden states at the event positions (Concat). The model is evaluated on MATRES and TCR with five embedding types (word2vec, GloVe, FastText, ELMo, BERT), and is further augmented with a Siamese 'common sense encoder' (CSE) trained on TEMPROB, plus ILP-based global inference. The authors report that Concat with ELMo or BERT plus CSE outperforms the feature-based CogCompTime by about 10% absolute accuracy and F1 on MATRES and by similar margins on TCR, and they argue that the results support a strong neural baseline for future work.

Significance. If the results hold, the paper makes a useful contribution by showing that a relatively standard LSTM with contextualized embeddings can substantially beat a feature-based system on the MATRES benchmark, and by providing a systematic comparison of position-indicator versus concatenation handling, five embedding choices, and a common-sense injection method. The paper also reports statistical significance tests (McNemar, paired t-test) and states that the system is public, which supports reproducibility. The main caveat is that the reported contribution of the CSE component is entangled with the baseline used to reconstruct TEMPROB; this does not undermine the core neural-versus-feature-based finding, but it needs to be addressed before the common-sense attribution can be accepted.

major comments (2)
  1. [Section 3.2] The CSE is trained on TEMPROB 'reconstructed using the same method described in Ning et al. (2018b) with the base method changed to CogCompTime.' Since CogCompTime is also the comparison baseline in Tables 2 and 3, the approximately 2-point gain attributed to CSE could partly reflect the model learning to reproduce CogCompTime's own output distribution rather than acquiring independent common-sense knowledge. The authors should compare against the original TEMPROB (Ning et al., 2018b) or a TEMPROB built with an independent base extractor, and/or analyze which errors CSE fixes to show that they are not simply CogCompTime's systematic errors. This is load-bearing for the paper's claim that CSE contributes real temporal common sense.
  2. [Section 4.2, Table 2] The sentence 'Roughly speaking, the 8% gain is contributed by LSTMs for 2%, contextualized embeddings for 4%, and CSE for 2%' is not supported by the reported ablations; Table 2 shows only Concat versus Concat+CSE, not a controlled decomposition into LSTM, embedding, and CSE contributions. Moreover, the Concat-only rows (e.g., BERT accuracy 69.1 versus CogCompTime 61.6) already show a substantial gain, so the 'about 10%' headline depends on the CSE increment that is subject to the concern raised above. The authors should either present the decomposition as a rough estimate with a supporting ablation, or reframe the main claim to rely on the Concat-only result.
minor comments (5)
  1. [Section 1] The sentence 'We think it is important for to understand' contains a typo: 'for to' should be 'for us to'.
  2. [Section 3.2] The compound 'T EMPROB' is rendered with an inconsistent space in several places (e.g., 'T EMPROB' and 'T EMPROB'); it should be 'TEMPROB' consistently.
  3. [Table 3] The column header 'Emb.' lists ELMo and BERT as entries, but the system column only says 'Concat+CSE'; it would be clearer to label the rows explicitly as 'Concat+CSE (ELMo)' and 'Concat+CSE (BERT)'.
  4. [Appendix A] The sentence 'The third metric came into use since the TempEval3 workshop (UzZaman et al., 2013), which involves graph closure and reduction on top of the second metric, hoping to better capture how useful a TempRel system is..' contains a doubled period at the end.
  5. [Section 4.2 and Appendix B] The paper reports only p<0.001 for the main McNemar tests; reporting exact p-values or effect sizes, and ideally confidence intervals for the metric differences, would strengthen the significance claims, especially given the small test set (837 TempRels in MATRES PT).

Circularity Check

1 steps flagged · score 3.0 of 10

CSE gain is not separable from CogCompTime's own outputs because TEMPROB was reconstructed with CogCompTime as the base extractor.

  1. other [Section 3.2 (Common Sense Encoder), Section 4.2 (Results and Discussion), Table 2 caption]
    "Note that the TEMPROB we use is reconstructed using the same method described in Ning et al. (2018b) with the base method changed to CogCompTime. ... CogCompTime (Ning et al., 2018d) is the previous state-of-the-art feature-based system. ... Roughly speaking, the 8% gain is contributed by LSTMs for 2%, contextualized embeddings for 4%, and CSE for 2%."

    CSE is a Siamese network trained to fit TEMPROB, but the TEMPROB used here was produced by the same CogCompTime system that serves as the comparison baseline. Thus the auxiliary supervision fed to CSE is a smoothed or embedded version of the baseline's own frequency judgments. When the paper attributes 2 of the 8 percentage points to CSE (Section 4.2), part of that gain may simply be the model re-absorbing CogCompTime's output distribution and errors, rather than external common sense. The paper provides no control with an independently reconstructed TEMPROB and no analysis of whether CogCompTime-specific biases propagate into CSE.

full rationale

The overall architecture and headline result are self-contained with respect to the comparison: Table 2's Concat+BERT (69.1 accuracy, 74.4 F1) and Concat+ELMo (67.7, 74.0) already exceed CogCompTime (61.6, 66.6) by a large margin, so the 'neural baseline beats feature-based CogCompTime' claim does not depend on the CSE component. The evaluation uses fixed public splits of MATRES and TCR; citing the papers that introduced these datasets is standard practice and not circular. The only substantive circularity is localized to the CSE contribution. The paper explicitly states that TEMPROB was reconstructed with CogCompTime as the base method, and CogCompTime is the baseline used for the comparison. A Siamese encoder fitted to that resource can therefore learn the baseline's biases; attributing the CSE-added gain to 'common sense' is not cleanly supported. Because this is a confound rather than an identity-by-construction, and because the main claim has independent support, score 3 rather than higher.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests mainly on benchmark data (MATRES and TCR), pretrained embeddings, and a reconstructed TEMPROB. No new physical or mathematical entities are introduced. The main design choices are listed above.

free parameters (6)
  • LSTM hidden size = 128 or 256 (Figure 1)
    Chosen architecture hyperparameter; not reported whether tuned per embedding, and no sensitivity analysis is given.
  • Dropout rate = 0.3
    Fixed in all experiments; not tuned systematically.
  • CSE training epochs = 20
    Reported in Section 4.2; chosen by validation on the TEMPROB split.
  • CSE learning rate = 1e-4
    Adam optimizer setting for the Siamese network.
  • StepLR schedule = gamma=0.5 every 10 epochs
    Learning rate decay for the LSTM; authors state performance is not sensitive to it.
  • Development split fraction = 20% of TB+AQ training data
    Used to tune learning rates and epochs for model selection.
assumptions (4)
  • domain assumption Event positions are known at test time.
    Section 3 states the system 'only takes a sequence of word embeddings as input (assuming that the position of events are known)'. The benchmark evaluations provide gold events, so end-to-end performance with automatic event extraction is not measured.
  • domain assumption MATRES annotations are reliable and the four-way label set is complete.
    The paper relies on MATRES's high IAA (kappa around 0.84) and on the four labels before, after, equal, and vague; annotation noise bounds achievable accuracy.
  • domain assumption TEMPROB statistics contain useful temporal common sense.
    The CSE is trained on TEMPROB, automatically extracted from a corpus using CogCompTime; the paper assumes this resource transfers to the MATRES and TCR test domains.
  • standard math ILP transitivity constraints are valid for the temporal graph.
    Global inference enforces transitivity, a standard constraint in temporal reasoning, used in many prior systems.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An Improved Neural Baseline for Temporal Relation Extraction." pith.science (2026). https://pith.science/paper/JS634CLP

@misc{pith2026190900429,
  author       = {Pith},
  title        = {Pith review of: An Improved Neural Baseline for Temporal Relation Extraction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JS634CLP}},
  note         = {Machine review of arXiv:1909.00429}
}
read the original abstract

Determining temporal relations (e.g., before or after) between events has been a challenging natural language understanding task, partly due to the difficulty to generate large amounts of high-quality training data. Consequently, neural approaches have not been widely used on it, or showed only moderate improvements. This paper proposes a new neural system that achieves about 10% absolute improvement in accuracy over the previous best system (25% error reduction) on two benchmark datasets. The proposed system is trained on the state-of-the-art MATRES dataset and applies contextualized word embeddings, a Siamese encoder of a temporal common sense knowledge base, and global inference via integer linear programming (ILP). We suggest that the new approach could serve as a strong baseline for future research in this area.

Figures

Figures reproduced from arXiv: 1909.00429 by the authors.

Figure 1
Figure 1. Overview of the neural network structures studied in this paper. Networks (a) and (b) are two ways to [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. An example confusion matrix, where the four [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 43 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Steven Bethard, Leon Derczynski, Guergana Savova, James Pustejovsky, and Marc Verhagen. 2015. SemEval -2015 Task 6: Clinical TempEval . In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 806--814

    work page 2015

  4. [4]

    Steven Bethard, James H Martin, and Sara Klingenstein. 2007. Timelines from text: Identification of syntactic temporal relations. In Semantic Computing, 2007. ICSC 2007. International Conference on, pages 11--18. IEEE

    work page 2007

  5. [5]

    Steven Bethard, Guergana Savova, Wei-Te Chen, Leon Derczynski, James Pustejovsky, and Marc Verhagen. 2016. SemEval -2016 Task 12: Clinical TempEval . In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 1052--1062

    work page 2016

  6. [6]

    Steven Bethard, Guergana Savova, Martha Palmer, and James Pustejovsky. 2017. Semeval-2017 task 12: Clinical tempeval. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 565--572. Association for Computational Linguistics

    work page 2017

  7. [7]

    Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2016. Enriching word vectors with subword information. arXiv preprint arXiv:1607.04606

    arXiv 2016

  8. [8]

    Jane Bromley, Isabelle Guyon, Yann LeCun, Eduard S \"a ckinger, and Roopak Shah. 1994. Signature verification using a ``siamese'' time delay neural network. In The Conference on Advances in Neural Information Processing Systems (NIPS), pages 737--744

    work page 1994

Show all 50 references
  1. [9]

    Taylor Cassidy, Bill McDowell, Nathanel Chambers, and Steven Bethard. 2014. An annotation framework for dense event ordering. In Proc. of the Annual Meeting of the Association of Computational Linguistics (ACL), pages 501--506

    2014

  2. [10]

    Nathanael Chambers, Taylor Cassidy, Bill McDowell, and Steven Bethard. 2014. Dense event ordering with a multi-pass architecture. Transactions of the Association for Computational Linguistics, 2:273--284

    2014

  3. [11]

    Nathanael Chambers and Dan Jurafsky. 2008 a . Jointly combining implicit constraints improves temporal ordering. In Proc. of the Conference on Empirical Methods for Natural Language Processing (EMNLP)

    2008

  4. [12]

    Nathanael Chambers and Daniel Jurafsky. 2008 b . Unsupervised Learning of Narrative Event Chains . In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics, ACL 2008 , pages 789--797

    2008

  5. [13]

    Nathanael Chambers, Shan Wang, and Dan Jurafsky. 2007. Classifying temporal relations between events. In Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions, pages 173--176. Association for Computational Linguistics

    2007

  6. [14]

    Fei Cheng and Yusuke Miyao. 2017. Classifying temporal relations by bidirectional LSTM over dependency paths. In Proc. of the Annual Meeting of the Association of Computational Linguistics (ACL), volume 2, pages 1--6

    2017

  7. [15]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT : Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

    2018 arXiv

  8. [16]

    Dietterich

    Thomas G. Dietterich. 1998. Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation

    1998

  9. [17]

    Dmitriy Dligach, Timothy Miller, Chen Lin, Steven Bethard, and Guergana Savova. 2017. Neural temporal relation extraction. volume 2, pages 746--751

    2017

  10. [18]

    Quang Do, Wei Lu, and Dan Roth. 2012. Joint inference for event timeline construction. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)

    2012

  11. [19]

    Brian S Everitt. 1992. The analysis of contingency tables

    1992

  12. [20]

    Sepp Hochreiter and J \"u rgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735--1780

    1997

  13. [21]

    Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

    2014 arXiv

  14. [22]

    Artuur Leeuwenberg and Marie-Francine Moens. 2017. Structured learning for temporal relation extraction from clinical records. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics

    2017

  15. [23]

    Artuur Leeuwenberg and Marie-Francine Moens. 2018. Temporal information extraction by predicting relative time-lines. Proc. of the Conference on Empirical Methods for Natural Language Processing (EMNLP)

    2018

  16. [24]

    Chen Lin, Timothy Miller, Dmitriy Dligach, Steven Bethard, and Guergana Savova. 2017. Representations of time expressions for temporal relation extraction with convolutional neural networks. BioNLP 2017, pages 322--327

    2017

  17. [25]

    Hector Llorens, Nathanael Chambers, Naushad UzZaman, Nasrin Mostafazadeh, James Allen, and James Pustejovsky. 2015. SemEval -2015 Task 5: QA TEMPEVAL - evaluating temporal information understanding with question answering. In Proceedings of the 9th International Workshop on Se...

    2015

  18. [26]

    Inderjeet Mani, Marc Verhagen, Ben Wellner, Chong Min Lee, and James Pustejovsky. 2006. Machine learning of temporal relations. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Lin...

    2006

  19. [27]

    Yuanliang Meng and Anna Rumshisky. 2018. Context-aware neural model for temporal information extraction. In Proc. of the Annual Meeting of the Association of Computational Linguistics (ACL), volume 1, pages 527--536

    2018

  20. [28]

    Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Distributed Representations of Words and Phrases and their Compositionality . In Proceedings of NIPS 2013, pages 1--9

    2013

  21. [29]

    Anne-Lyse Minard, Manuela Speranza, Eneko Agirre, Itziar Aldabe, Marieke van Erp, Bernardo Magnini, German Rigau, Ruben Urizar, and Fondazione Bruno Kessler. 2015. SemEval -2015 Task 4: TimeLine : Cross-document event ordering. In Proceedings of the 9th International Workshop ...

    2015

  22. [30]

    Nasrin Mostafazadeh, Alyson Grealish, Nathanael Chambers, James Allen, and Lucy Vanderwende. 2016. CaTeRS : Causal and temporal relation scheme for semantic annotation of event structures. In Proceedings of the 4th Workshop on Events: Definition, Detection, Coreference, and Re...

    2016

  23. [31]

    Qiang Ning, Zhili Feng, and Dan Roth. 2017. A structured learning approach to temporal relation extraction. In Proceedings of the Conference on Empirical Methods for Natural Language Processing (EMNLP), pages 1038--1048, Copenhagen, Denmark

    2017

  24. [32]

    Qiang Ning, Zhili Feng, Hao Wu, and Dan Roth. 2018 a . Joint reasoning for temporal and causal relations. In Proceedings of the Annual Meeting of the Association of Computational Linguistics (ACL), pages 2278--2288

    2018

  25. [33]

    Qiang Ning, Hao Wu, Haoruo Peng, and Dan Roth. 2018 b . Improving temporal relation extraction with a globally acquired statistical resource. In Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL), pages 841--851

    2018

  26. [34]

    Qiang Ning, Hao Wu, and Dan Roth. 2018 c . A multi-axis annotation scheme for event temporal relations. In Proceedings of the Annual Meeting of the Association of Computational Linguistics (ACL), pages 1318--1328

    2018

  27. [35]

    Qiang Ning, Ben Zhou, Zhili Feng, Haoruo Peng, and Dan Roth. 2018 d . Cogcomptime: A tool for understanding time in natural language. In Proceedings of the Conference on Empirical Methods for Natural Language Processing (EMNLP)

    2018

  28. [36]

    Tim O'Gorman, Kristin Wright-Bettner, and Martha Palmer. 2016. Richer event description: Integrating event coreference with temporal, causal and bridging annotation. In Proceedings of the 2nd Workshop on Computing News Storylines (CNS 2016), pages 47--56, Austin, Texas. Associ...

    2016

  29. [37]

    Ajay Patel, Alexander Sands, Chris Callison-Burch, and Marianna Apidianaki. 2018. Magnitude: A fast, efficient universal vector embedding utility package. Proc. of the Conference on Empirical Methods for Natural Language Processing (EMNLP)

    2018

  30. [38]

    Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In EMNLP, pages 1532--1543

    2014

  31. [39]

    Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer

    Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proc. of the Annual Meeting of the North American Association of Computational Linguistics (NAACL)

    2018

  32. [40]

    William F Styler IV, Steven Bethard, Sean Finan, Martha Palmer, Sameer Pradhan, Piet C de Groen, Brad Erickson, Timothy Miller, Chen Lin, Guergana Savova, et al. 2014. Temporal annotation in the clinical domain. Transactions of the Association for Computational Linguistics, 2:143

    2014

  33. [41]

    Julien Tourille, Olivier Ferret, Aurelie Neveol, and Xavier Tannier. 2017. Neural architecture for temporal relation extraction: A bi-lstm approach for detecting narrative containers. In Proc. of the Annual Meeting of the Association of Computational Linguistics (ACL), volume ...

    2017

  34. [42]

    Naushad UzZaman, Hector Llorens, James Allen, Leon Derczynski, Marc Verhagen, and James Pustejovsky. 2013. SemEval -2013 Task 1: TEMPEVAL -3: Evaluating time expressions, events, and temporal relations. *SEM, 2:1--9

    2013

  35. [43]

    Marc Verhagen, Robert Gaizauskas, Frank Schilder, Mark Hepple, Graham Katz, and James Pustejovsky. 2007. SemEval -2007 Task 15: TempEval temporal relation identification. In Proceedings of the 4th International Workshop on Semantic Evaluations, pages 75--80. Association for Co...

    2007

  36. [44]

    Marc Verhagen and James Pustejovsky. 2008. Temporal processing with the TARSQI toolkit. In 22nd International Conference on on Computational Linguistics: Demonstration Papers, pages 189--192. Association for Computational Linguistics

    2008

  37. [45]

    Marc Verhagen, Roser Sauri, Tommaso Caselli, and James Pustejovsky. 2010. SemEval -2010 Task 13: TempEval -2. In Proceedings of the 5th international workshop on semantic evaluation, pages 57--62. Association for Computational Linguistics

    2010

  38. [46]

    Zhenqi Xu, Jiani Hu, and Weihong Deng. 2016. Recurrent convolutional neural network for video classification. In Multimedia and Expo (ICME), 2016 IEEE International Conference on, pages 1--6. IEEE

    2016

  39. [47]

    Katsumasa Yoshikawa, Sebastian Riedel, Masayuki Asahara, and Yuji Matsumoto. 2009. Jointly identifying temporal relations with markov logic. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Langu...

    2009

  40. [48]

    Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, and Jun Zhao. 2014. Relation classification via convolutional deep neural network. In Proc. the International Conference on Computational Linguistics (COLING), pages 2335--2344

    2014

  41. [49]

    Dongxu Zhang and Dong Wang. 2015. Relation classification via recurrent neural network. arXiv preprint arXiv:1508.01006

    2015 arXiv

  42. [50]

    Shu Zhang, Dequan Zheng, Xinchen Hu, and Ming Yang. 2015. Bidirectional long short-term memory networks for relation classification. In Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation, pages 73--78

    2015

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.