pith. sign in

arxiv: 1907.11769 · v4 · pith:4UJCOBCHnew · submitted 2019-07-26 · 💻 cs.CL

Automatically Learning Construction Injury Precursors from Text

Henrietta Baker , Matthew R. Hallowell , Antoine J.-P. Tixier This is my paper

Pith reviewed 2026-05-24 15:26 UTC · model grok-4.3

classification 💻 cs.CL
keywords construction safetyinjury precursorsnatural language processingdeep learningtext classificationaccident reportsCNNSVM
0
0 comments X

The pith

NLP models trained on accident reports can surface valid injury precursors from the text.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tests three approaches—CNNs, hierarchical attention networks, and TF-IDF plus SVM—to extract predictive textual patterns from raw construction safety reports. After training, each model is paired with a post-hoc method that ranks the phrases most strongly associated with specific injury outcomes. The central finding is that some of the highest-ranked phrases are genuine injury precursors rather than noise. This matters because the construction industry now generates large volumes of digitized reports that could be turned into preventive knowledge without exhaustive manual review.

Core claim

Among the textual patterns flagged by the trained models as most predictive of each safety outcome, valid injury precursors can be found. The same methods let users visualize which parts of a report drive a given prediction.

What carries the argument

Post-training identification of the textual patterns that are, on average, the most predictive of each safety outcome, applied to CNN, HAN, and TF-IDF+SVM classifiers.

If this is right

  • Safety teams can inspect the surfaced phrases to understand why particular incidents occur.
  • Large report collections can be screened automatically for recurring risk patterns.
  • Users can see which sentences or terms drive each model's output for any individual report.
  • The three modeling approaches can be compared on how many of their top patterns are actionable precursors.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the patterns hold across new datasets, they could be turned into checklists or training modules that target the same language.
  • The approach might transfer to other text-heavy domains such as aviation or healthcare incident logs.
  • Repeated application could reveal whether certain precursors cluster by project type or region.

Load-bearing premise

The phrases the models rank highest truly reflect causal precursors rather than reporting habits, data biases, or spurious correlations.

What would settle it

A field trial that intervenes on the extracted precursors at some sites and records whether injury rates drop relative to matched control sites.

Figures

Figures reproduced from arXiv: 1907.11769 by Antoine J.-P. Tixier, Henrietta Baker, Matthew R. Hallowell.

Figure 1
Figure 1. Figure 1: Toy example of CNN architecture for document classification, with an input document [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Hierarchical Attention Network (HAN), reproduced with permission from [42]. [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Linear SVM decision boundary in the two-dimensional case. [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Training history for incident type. the nearest-neighbors. Such a search engine could be used to answer questions such as “What incident reports in the database are the most similar to a given incident report?”, which could be very useful in practice. 2.5 2.0 1.5 1.0 0.5 PC1 0.15 0.20 0.25 0.30 0.35 0.40 0.45 PC2 initial doc vecs han injury_bodypart_harmed HD HD/WST FG/THB EYE LWEXT UPEXT 2 1 0 1 2 3 4 PC1… view at source ↗
Figure 5
Figure 5. Figure 5: HAN bodypart report embeddings before training (left) and after training (right). 6. Automatically learning incident precursors from text In this section, we propose, for each model, a method to learn valid injury precursors from the raw dataset by using the trained models. These precursors are identified among the parts of the raw input reports that we find the models rely on the most, on average, in pred… view at source ↗
Figure 6
Figure 6. Figure 6: CNN probabilistic forecast and word saliency visualization for [PITH_FULL_IMAGE:figures/full_fig_p024_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: HAN probabilistic forecast, sentence and word attention, for the example of Fig. 6. [PITH_FULL_IMAGE:figures/full_fig_p026_7.png] view at source ↗
read the original abstract

In light of the increasing availability of digitally recorded safety reports in the construction industry, it is important to develop methods to exploit these data to improve our understanding of safety incidents and ability to learn from them. In this study, we compare several approaches to automatically learn injury precursors from raw construction accident reports. More precisely, we experiment with two state-of-the-art deep learning architectures for Natural Language Processing (NLP), Convolutional Neural Networks (CNN) and Hierarchical Attention Networks (HAN), and with the established Term Frequency - Inverse Document Frequency representation (TF-IDF) + Support Vector Machine (SVM) approach. For each model, we provide a method to identify (after training) the textual patterns that are, on average, the most predictive of each safety outcome. We show that among those pieces of text, valid injury precursors can be found. The proposed methods can also be used by the user to visualize and understand the models' predictions.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper compares CNN, HAN, and TF-IDF+SVM models for classifying construction safety reports by outcome and extracting post-training textual patterns (n-grams or attention spans) that are most predictive of each outcome. It asserts that valid injury precursors can be identified among these patterns and provides visualization methods for model predictions.

Significance. If the extracted patterns can be shown to correspond to genuine causal precursors rather than artifacts, the multi-model comparison and post-hoc extraction approach would offer a scalable way to mine growing corpora of digital safety reports for prevention insights. The explicit provision of pattern-identification methods after training on held-out data is a methodological strength.

major comments (2)
  1. [Abstract] Abstract, final paragraph: the central empirical claim that 'valid injury precursors can be found' among the extracted patterns supplies no quantitative validation, inter-rater reliability, error bars, or protocol for distinguishing causal precursors from reporting biases or spurious correlations; this verification step is load-bearing for the contribution.
  2. [Methods/Results] Methods/Results sections (pattern extraction procedure): no ground-truth precursor annotations, temporal ordering checks, or intervention-style tests are described to confirm that high-weight features reflect precursors rather than post-hoc phrasing common in safety corpora.
minor comments (1)
  1. [Methods] The description of hyperparameter choices (SVM kernel/regularization, CNN/HAN filters and attention heads) could be expanded with sensitivity analysis to strengthen reproducibility.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback emphasizing the importance of validation for the extracted patterns. We respond to each major comment below.

read point-by-point responses

  1. Referee: [Abstract] Abstract, final paragraph: the central empirical claim that 'valid injury precursors can be found' among the extracted patterns supplies no quantitative validation, inter-rater reliability, error bars, or protocol for distinguishing causal precursors from reporting biases or spurious correlations; this verification step is load-bearing for the contribution.

    Authors: The claim rests on qualitative review by construction safety experts who confirmed alignment of top patterns with established precursors from safety literature. We will revise the abstract to explicitly state this is a qualitative demonstration rather than a validated causal analysis, and add a brief protocol description (expert inspection of top-weighted n-grams/attention spans) in the Results section. Quantitative elements such as inter-rater reliability or error bars on precursor validity are not feasible without new annotations; we will instead report model performance with confidence intervals where applicable and tone the language to 'plausible precursors'. revision: partial

  2. Referee: [Methods/Results] Methods/Results sections (pattern extraction procedure): no ground-truth precursor annotations, temporal ordering checks, or intervention-style tests are described to confirm that high-weight features reflect precursors rather than post-hoc phrasing common in safety corpora.

    Authors: We will expand the Methods section to fully detail the post-training pattern extraction (top n-grams by weight for SVM/TF-IDF; attention spans for HAN; filter activations for CNN) and add a dedicated Limitations paragraph. This will acknowledge the lack of ground-truth precursor labels in the dataset, the retrospective nature of reports precluding temporal ordering or intervention tests, and the risk of reporting biases. The multi-model consistency is presented as supporting evidence against pure artifacts, but we agree this does not constitute causal confirmation. revision: yes

Circularity Check

0 steps flagged

No circularity detected in the derivation chain

full rationale

The paper trains standard supervised NLP models (CNN, HAN, TF-IDF+SVM) on labeled construction safety reports to predict discrete safety outcomes, then applies post-training feature extraction (n-gram weights or attention) to surface the most predictive text spans. The claim that valid injury precursors appear among these spans is presented as an empirical observation from manual review rather than a mathematical identity, a fitted parameter renamed as a prediction, or a result forced by self-citation. No equations or uniqueness theorems are invoked, no self-citations are load-bearing, and evaluation uses held-out data, keeping the finding independent of the training inputs.

Axiom & Free-Parameter Ledger

2 free parameters · 2 axioms · 0 invented entities

The central claim rests on the representativeness of the (unspecified) report corpus and on the assumption that model attention weights identify causal rather than merely correlational text. No new physical entities are introduced.

free parameters (2)
  • SVM regularization parameter and kernel choice
    Standard hyperparameter fitted during training of the baseline model.
  • CNN and HAN architecture hyperparameters (filters, attention heads, etc.)
    Learned or chosen during model training on the safety-report corpus.
axioms (2)
  • domain assumption The training corpus is representative of the broader population of construction incidents.
    Invoked implicitly when claiming that discovered patterns are general injury precursors.
  • domain assumption Model attention or feature importance maps to semantically meaningful precursors.
    Required for the claim that the extracted text pieces are 'valid injury precursors'.

pith-pipeline@v0.9.0 · 5689 in / 1154 out tokens · 23665 ms · 2026-05-24T15:26:16.191589+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. AI-based Prediction of Independent Construction Safety Outcomes from Universal Attributes

    cs.LG 2019-08 unverdicted novelty 4.0

    NLP-derived attributes from construction incident reports remain strongly predictive of independently labeled safety outcomes even after removing potential label leakage, with injury severity now well predicted on a d...

Reference graph

Works this paper leans on

61 extracted references · 61 canonical work pages · cited by 1 Pith paper · 17 internal anchors

  1. [1]

    Reinsel, J

    D. Reinsel, J. Gantz, J. Rydning, Data age 2025. The digitization of the world: from edge to core, Tech. rep., Accessed 21st January 2020 (2018)

    work page 2025

  2. [2]

    Grimes, Unstructured data and the 80 percent rule, Accessed 21st January 2020 (2008)

    S. Grimes, Unstructured data and the 80 percent rule, Accessed 21st January 2020 (2008). URL http://breakthroughanalysis.com/2008/08/01/ unstructured-data-and-the-80-percent-rule/

    work page 2020

  3. [3]

    D. D. Woods, E. S. Patterson, E. M. Roth, Can we ever escape from data overload? A cognitive systems diagnosis, Cognition, Technology & Work 4 (1) (2002) 22–36. doi:10.1007/ s101110200002

    work page 2002

  4. [4]

    Henke, J

    N. Henke, J. Bughin, M. Chui, J. Manyika, T. Saleh, B. Wiseman, G. Sethupathy, The age of analytics: competing in a data-driven world, Tech. rep., Accessed 21st January 2020 (2016). 30

    work page 2020

  5. [5]

    Lukic, A

    D. Lukic, A. Littlejohn, A. Margaryan, A framework for learning from incidents in the work- place, Safety Science 50 (4) (2012) 950–957. doi:10.1016/j.ssci.2011.12.032

    work page doi:10.1016/j.ssci.2011.12.032 2012

  6. [6]

    J. M. Sanne, Incident reporting or storytelling? Competing schemes in a safety-critical and hazardous work setting, Safety Science 46 (8) (2008) 1205–1222.doi:10.1016/j.ssci. 2007.06.024

    work page doi:10.1016/j.ssci 2008

  7. [7]

    W. J. Wiatrowski, J. A. Janocha, Comparing fatal work injuries in the united states and the european union, Tech. rep., Accessed 21st January 2020 (June 2014). URL https://www.bls.gov/opub/mlr/2014/article/ comparing-fatal-work-injuries-us-eu.htm

    work page 2020

  8. [8]

    Goodfellow, Y

    I. Goodfellow, Y . Bengio, A. Courville, Deep learning, MIT Press, 2016. URL http://www.deeplearningbook.org

    work page 2016

  9. [9]

    Schmidhuber, Deep learning in neural networks: An overview, Neural Networks 61 (2015) 85 – 117

    J. Schmidhuber, Deep learning in neural networks: An overview, Neural Networks 61 (2015) 85–117. doi:10.1016/j.neunet.2014.09.003

    work page doi:10.1016/j.neunet.2014.09.003 2015

  10. [10]

    LeCun, L

    Y . LeCun, L. Bottou, Y . Bengio, P. Haffner, Gradient-based learning applied to document recognition, Proceedings of the IEEE 86 (11) (1998) 2278–2324. doi:10.1109/5. 726791

    work page doi:10.1109/5 1998

  11. [11]

    Bengio, R

    Y . Bengio, R. Ducharme, P. Vincent, C. Jauvin, A neural probabilistic language model, Journal of Machine Learning Research 3 (Feb) (2003) 1137–1155. doi:10.1007/ 3-540-33486-6_6

    work page 2003

  12. [12]

    Mikolov, I

    T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, J. Dean, Distributed representations of words and phrases and their compositionality, in: Advances in Neural Information Processing Sys- tems, 2013, pp. 3111–3119

    work page 2013

  13. [13]

    Efficient Estimation of Word Representations in Vector Space

    T. Mikolov, K. Chen, G. Corrado, J. Dean, Efficient estimation of word representations in vector space, arXiv preprint, arXiv:1301.3781

    work page internal anchor Pith review Pith/arXiv arXiv

  14. [14]

    Convolutional Neural Networks for Sentence Classification

    Y . Kim, Convolutional neural networks for sentence classification, arXiv preprint, arXiv:1408.5882

    work page internal anchor Pith review Pith/arXiv arXiv

  15. [15]

    Effective Approaches to Attention-based Neural Machine Translation

    M.-T. Luong, H. Pham, C. D. Manning, Effective approaches to attention-based neural machine translation, arXiv preprint, arXiv:1508.04025

    work page internal anchor Pith review Pith/arXiv arXiv

  16. [16]

    Hirschberg, C

    J. Hirschberg, C. D. Manning, Advances in natural language processing, Science 349 (6245) (2015) 261–266. doi:10.1126/science.aaa8685

    work page doi:10.1126/science.aaa8685 2015

  17. [17]

    S. Katz, Estimation of probabilities from sparse data for the language model component of a speech recognizer, IEEE Transactions on Acoustics, Speech, and Signal Processing 35 (3) (1987) 400–401. doi:10.1109/TASSP.1987.1165125

    work page doi:10.1109/tassp.1987.1165125 1987

  18. [18]

    Zhang, J

    X. Zhang, J. Zhao, Y . LeCun, Character-level convolutional networks for text classification, in: Advances in Neural Information Processing Systems, 2015, pp. 649–657

    work page 2015

  19. [19]

    Bojanowski, E

    P. Bojanowski, E. Grave, A. Joulin, T. Mikolov, Enriching word vectors with subword in- formation, Transactions of the Association for Computational Linguistics 5 (2017) 135–146. doi:10.1162/tacl_a_00051. 31

    work page doi:10.1162/tacl_a_00051 2017

  20. [20]

    A. J.-P. Tixier, M. Vazirgiannis, M. R. Hallowell, Word embeddings for the construction do- main, arXiv preprint, arXiv:1610.09333

    work page internal anchor Pith review Pith/arXiv arXiv

  21. [21]

    Shang et al, Energy-based Self-attentive Learning of Abstractive Communities for Spoken Language Understanding, arXiv:1904.09491 [cs.CL]

    G. Shang, A. J.-P. Tixier, M. Vazirgiannis, J.-P. Lorr ´e, Energy-based self-attentive learning of abstractive communities for spoken language understanding, arXiv preprint, arXiv:1904.09491

    work page arXiv 1904

  22. [22]

    Pennington, R

    J. Pennington, R. Socher, C. Manning, Glove: global vectors for word representation, in: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014, pp. 1532–1543. doi:10.3115/v1/D14-1

    work page doi:10.3115/v1/d14-1 2014

  23. [23]

    M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, L. Zettlemoyer, Deep contextualized word representations, arXiv preprint, arXiv:1802.05365

    work page internal anchor Pith review Pith/arXiv arXiv

  24. [24]

    BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

    J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: pre-training of deep bidirectional trans- formers for language understanding, arXiv preprint, arXiv:1810.04805

    work page internal anchor Pith review Pith/arXiv arXiv

  25. [25]

    C. H. Caldas, L. Soibelman, Automating hierarchical document classification for construction management information systems, Automation in Construction 12 (4) (2003) 395–406. doi: 10.1016/S0926-5805(03)00004-9

    work page doi:10.1016/s0926-5805(03)00004-9 2003

  26. [26]

    Y . M. Goh, C. Ubeynarayana, Construction accident narrative classification: an evaluation of text mining techniques, Accident Analysis & Prevention 108 (2017) 122–130. doi:10. 1016/j.aap.2017.08.026

    work page 2017

  27. [27]

    Zhang, H

    F. Zhang, H. Fleyeh, X. Wang, M. Lu, Construction site accident analysis using text mining and natural language processing techniques, Automation in Construction 99 (2019) 238–248. doi:10.1016/j.autcon.2018.12.016

    work page doi:10.1016/j.autcon.2018.12.016 2019

  28. [28]

    Chokor, H

    A. Chokor, H. Naganathan, W. K. Chong, M. El Asmar, Analyzing arizona osha injury reports using unsupervised machine learning, Procedia Engineering 145 (2016) 1588–1593. doi: 10.1016/j.proeng.2016.04.200

    work page doi:10.1016/j.proeng.2016.04.200 2016

  29. [30]

    Yu, J.-y

    W.-d. Yu, J.-y. Hsu, Content-based text mining technique for retrieval of cad documents, Au- tomation in Construction 31 (2013) 65–74. doi:10.1016/j.autcon.2012.11.037

    work page doi:10.1016/j.autcon.2012.11.037 2013

  30. [31]

    Y . Zou, A. Kiviniemi, S. W. Jones, Retrieving similar cases for construction project risk man- agement using natural language processing techniques, Automation in Construction 80 (2017) 66–76. doi:10.1016/j.autcon.2017.04.003

    work page doi:10.1016/j.autcon.2017.04.003 2017

  31. [32]

    T. Kim, S. Chi, Accident case retrieval and analyses: using natural language processing in the construction industry, Journal of Construction Engineering and Management 145 (3) (2019) 04019004. doi:10.1061/(ASCE)CO.1943-7862.0001625

    work page doi:10.1061/(asce)co.1943-7862.0001625 2019

  32. [33]

    T. P. Williams, J. Gong, Predicting construction cost overruns using text mining, numerical data and ensemble classifiers, Automation in Construction 43 (2014) 23–29. doi:10.1016/j. autcon.2014.02.014. 32

    work page doi:10.1016/j 2014

  33. [34]

    A. J.-P. Tixier, M. R. Hallowell, B. Rajagopalan, D. Bowman, Application of machine learning to construction injury prediction, Automation in Construction 69 (2016) 102–114. doi:10. 1016/j.autcon.2016.05.016

    work page 2016

  34. [35]

    A. J.-P. Tixier, M. R. Hallowell, B. Rajagopalan, Construction safety risk modeling and simu- lation, Risk Analysis 37 (10) (2017) 1917–1935. doi:10.1111/risa.12772

    work page doi:10.1111/risa.12772 2017

  35. [36]

    A. J.-P. Tixier, M. R. Hallowell, B. Rajagopalan, D. Bowman, Automated content analysis for construction safety: a natural language processing system to extract precursors and outcomes from unstructured injury reports, Automation in Construction 62 (2016) 45–56. doi:10. 1016/j.autcon.2015.11.001

    work page 2016

  36. [37]

    S. Moon, T. Kim, B.-G. Hwang, S. Chi, Analysis of construction accidents based on semantic search and natural language processing, in: ISARC. Proceedings of the International Sympo- sium on Automation and Robotics in Construction, V ol. 35, IAARC Publications, 2018, pp. 1–6. doi:10.22260/ISARC2018/0109

    work page doi:10.22260/isarc2018/0109 2018

  37. [38]

    Chung, Bridge damage factor recognition from inspection reports using active recurrent neural network, [MEng Thesis] (2018)

    S. Chung, Bridge damage factor recognition from inspection reports using active recurrent neural network, [MEng Thesis] (2018). URL http://s-space.snu.ac.kr/handle/10371/143675

    work page 2018

  38. [39]

    Effective Use of Word Order for Text Categorization with Convolutional Neural Networks

    R. Johnson, T. Zhang, Effective use of word order for text categorization with convolutional neural networks, arXiv preprint, arXiv:1412.1058

    work page internal anchor Pith review Pith/arXiv arXiv

  39. [40]

    A Sensitivity Analysis of (and Practitioners' Guide to) Convolutional Neural Networks for Sentence Classification

    Y . Zhang, B. Wallace, A sensitivity analysis of (and practitioners’ guide to) convolutional neu- ral networks for sentence classification, arXiv preprint, arXiv:1510.03820

    work page internal anchor Pith review Pith/arXiv arXiv

  40. [41]

    Z. Yang, D. Yang, C. Dyer, X. He, A. Smola, E. Hovy, Hierarchical attention networks for doc- ument classification, in: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2016, pp. 1480–1489. doi:10.18653/v1/N16-1174

    work page doi:10.18653/v1/n16-1174 2016

  41. [42]

    A. J.-P. Tixier, Notes on deep learning for nlp, arXiv preprint, arXiv:1808.09772

    work page internal anchor Pith review Pith/arXiv arXiv

  42. [43]

    Goldberg, A primer on neural network models for natural language processing, Journal of Artificial Intelligence Research 57 (2016) 345–420.doi:10.5555/3176748.3176757

    Y . Goldberg, A primer on neural network models for natural language processing, Journal of Artificial Intelligence Research 57 (2016) 345–420.doi:10.5555/3176748.3176757

    work page doi:10.5555/3176748.3176757 2016

  43. [44]

    J. L. Elman, Finding structure in time, Cognitive Science 14 (2) (1990) 179–211. doi:10. 1016/0364-0213(90)90002-E

    work page 1990

  44. [45]

    Long short-term memory

    S. Hochreiter, J. Schmidhuber, Long short-term memory, Neural Computation 9 (8) (1997) 1735–1780. doi:10.1162/neco.1997.9.8.1735

    work page doi:10.1162/neco.1997.9.8.1735 1997

  45. [46]

    K. Cho, B. Van Merri ¨enboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, Y . Bengio, Learning phrase representations using rnn encoder-decoder for statistical machine translation, arXiv preprint, arXiv:1406.1078

    work page internal anchor Pith review Pith/arXiv arXiv

  46. [47]

    Greff, R

    K. Greff, R. K. Srivastava, J. Koutn ´ık, B. R. Steunebrink, J. Schmidhuber, Lstm: A search space odyssey, IEEE Transactions on Neural Networks and Learning Systems 28 (10) (2017) 2222–2232. doi:10.1109/TNNLS.2016.2582924. 33

    work page doi:10.1109/tnnls.2016.2582924 2017

  47. [48]

    Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling

    J. Chung, C. Gulcehre, K. Cho, Y . Bengio, Empirical evaluation of gated recurrent neural networks on sequence modeling, arXiv preprint, arXiv:1412.3555

    work page internal anchor Pith review Pith/arXiv arXiv

  48. [49]

    Sutskever, O

    I. Sutskever, O. Vinyals, Q. V . Le, Sequence to sequence learning with neural networks, in: Advances in Neural Information Processing Systems, 2014, pp. 3104–3112

    work page 2014

  49. [50]

    Neural Machine Translation by Jointly Learning to Align and Translate

    D. Bahdanau, K. Cho, Y . Bengio, Neural machine translation by jointly learning to align and translate, arXiv preprint, arXiv:1409.0473

    work page internal anchor Pith review Pith/arXiv arXiv

  50. [51]

    Z. Lin, M. Feng, C. N. d. Santos, M. Yu, B. Xiang, B. Zhou, Y . Bengio, A structured self- attentive sentence embedding, arXiv preprint, arXiv:1703.03130

    work page internal anchor Pith review Pith/arXiv arXiv

  51. [52]

    B. E. Boser, I. M. Guyon, V . N. Vapnik, A training algorithm for optimal margin classifiers, in: ACM Proceedings of the Fifth Annual Workshop on Computational Learning Theory, 1992, pp. 144–152. doi:10.1145/130385.130401

    work page doi:10.1145/130385.130401 1992

  52. [53]

    ˇReh˚uˇrek, P

    R. ˇReh˚uˇrek, P. Sojka, Software framework for topic modelling with large corpora, in: Proceed- ings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, ELRA, Valletta, Malta, 2010, pp. 45–50. doi:10.13140/2.1.2393.1847

    work page doi:10.13140/2.1.2393.1847 2010

  53. [54]

    L. N. Smith, Cyclical learning rates for training neural networks, in: 2017 IEEE Win- ter Conference on Applications of Computer Vision (W ACV), IEEE, 2017, pp. 464–472. doi:10.1109/WACV.2017.58

    work page doi:10.1109/wacv.2017.58 2017

  54. [55]

    L. N. Smith, A disciplined approach to neural network hyper-parameters: Part 1–learning rate, batch size, momentum, and weight decay, arXiv preprint, arXiv:1803.09820

    work page internal anchor Pith review Pith/arXiv arXiv

  55. [56]

    A. C. Wilson, R. Roelofs, M. Stern, N. Srebro, B. Recht, The marginal value of adaptive gradient methods in machine learning, in: Advances in Neural Information Processing Systems, 2017, pp. 4148–4158

    work page 2017

  56. [57]

    Srivastava, G

    N. Srivastava, G. E. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov, Dropout: a simple way to prevent neural networks from overfitting, Journal of Machine Learning Research 15 (1) (2014) 1929–1958

    work page 2014

  57. [58]

    TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems

    M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, et al., Tensorflow: Large-scale machine learning on heterogeneous dis- tributed systems, arXiv preprint, arXiv:1603.04467

    work page internal anchor Pith review Pith/arXiv arXiv

  58. [59]

    Chollet, et al., Keras, https://github.com/fchollet/keras (2015)

    F. Chollet, et al., Keras, https://github.com/fchollet/keras (2015)

    work page 2015

  59. [60]

    Pedregosa, G

    F. Pedregosa, G. Varoquaux, A. Gramfort, V . Michel, B. Thirion, O. Grisel, M. Blondel, P. Pret- tenhofer, R. Weiss, V . Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Per- rot, E. Duchesnay, Scikit-learn: machine learning in python, Journal of Machine Learning Research 12 (2011) 2825–2830. doi:10.5555/1953048.2078195

    work page doi:10.5555/1953048.2078195 2011

  60. [61]

    Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps

    K. Simonyan, A. Vedaldi, A. Zisserman, Deep inside convolutional networks: Visualising im- age classification models and saliency maps, arXiv preprint, arXiv:1312.6034

    work page internal anchor Pith review Pith/arXiv arXiv

  61. [62]

    J. Li, X. Chen, E. Hovy, D. Jurafsky, Visualizing and understanding neural models in nlp, arXiv preprint, arXiv:1506.01066. 34

    work page internal anchor Pith review Pith/arXiv arXiv