Eliciting Knowledge from Experts:Automatic Transcript Parsing for Cognitive Task Analysis

He Jiang; Jiaming Shen; Junyi Du; Xiang Ren

arxiv: 1906.11384 · v1 · pith:P7YJLMZ4new · submitted 2019-06-26 · 💻 cs.CL

Eliciting Knowledge from Experts:Automatic Transcript Parsing for Cognitive Task Analysis

Junyi Du , He Jiang , Jiaming Shen , Xiang Ren This is my paper

Pith reviewed 2026-05-25 15:21 UTC · model grok-4.3

classification 💻 cs.CL

keywords cognitive task analysistranscript parsingweakly-supervised learningdistant supervisioninformation extractionsequence labelingrelation extractionconversational text

0 comments

The pith

A weakly-supervised framework parses CTA interview transcripts into structured knowledge by splitting the task into sequence labeling and span-pair relation extraction.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Cognitive task analysis turns expert interviews into structured representations such as flowcharts, but this requires heavy manual parsing of transcripts. The paper proposes a weakly-supervised information extraction method that divides the work into sequence labeling for identifying elements and relation extraction for linking them across sentences. Distant supervision is drawn from existing human-curated protocol files instead of new labeled data. Neighbor sentences are added as context to handle the conversational nature of the input. If the approach holds, it reduces the labor needed to scale expert knowledge elicitation.

Core claim

The authors claim that automated CTA transcript parsing can be achieved by partitioning the process into a sequence labeling task and a text span-pair relation extraction task, trained via distant supervision signals extracted from human-curated protocol files. To capture long-range dependencies in conversational text, models receive neighbor sentences as additional input, and various context-modeling architectures are tested. Real-world CTA transcripts are manually annotated to evaluate the resulting structured outputs.

What carries the argument

Weakly-supervised information extraction framework that partitions parsing into sequence labeling and span-pair relation extraction, using distant supervision from protocol files and neighbor sentences for context modeling.

If this is right

Transcript parsing can proceed with far less new manual annotation than before.
Protocol files already created by experts become reusable training resources.
Context from neighboring sentences improves relation extraction accuracy in dialogue.
The same split into labeling and relation tasks applies to other low-resource conversational extraction settings.
Evaluation on real annotated transcripts provides a direct test of generalization from protocols to interviews.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same distant-supervision split could be tried on transcripts from other knowledge-elicitation interviews outside psychology.
If the models succeed, they could feed directly into tools that build interactive flowcharts or decision trees from raw recordings.
Performance on very long transcripts might reveal limits of the neighbor-sentence context window.
Combining the output structures with existing CTA software could create end-to-end pipelines with minimal human review.

Load-bearing premise

Signals taken from human-curated protocol files are accurate and relevant enough to train models that still perform well on noisy, context-dependent interview transcripts.

What would settle it

Run the trained models on a held-out set of manually annotated CTA transcripts and check whether precision and recall for the extracted labels and relations remain above a no-distant-supervision baseline.

Figures

Figures reproduced from arXiv: 1906.11384 by He Jiang, Jiaming Shen, Junyi Du, Xiang Ren.

**Figure 1.** Figure 1: An example of CTA interview transcript and the human parsed structured text (protocol). In the protocol, splitting by the highlighted line numbers indicating the sources in transcript, phrases in protocol (called protocol phrases) are abstractive description of actions in the transcript. In the transcript, the highlighted numbers are line numbers, and the bolded are text spans matched by protocol phrases… view at source ↗

**Figure 2.** Figure 2: The framework of Automated CTA Transcripts Parsing. Text spans are extracted via the sequence labeling model, then the relations between text spans are extracted by the text span-pair relation extraction model (span-pair RE model). In the end we assemble the results into structured knowledge (flowchart) for CTA. dens the RE model to process global information of the text. One previous work (Park and Motah… view at source ↗

**Figure 3.** Figure 3: The construction of text span with context tc. The example shows two text spans with context using K = 2. Neighbours of text span t are denoted by t+i and t−i , 0 < i <= K (Fig.3). But in the normal sentence pair classification setting, they are concatenated into a single sequence while its segmentation is ignored. Here, we explored some variants of the neural model, to incorporate the context segmentatio… view at source ↗

**Figure 4.** Figure 4: Visualization of the hidden state masking. Hidden states for the context sentences are masked before pooling. Hidden states masking. In this model variant we inject the context segmentation into models by masking out the final layer hidden states for the context sentences and aggregate the remaining hidden states using a pooling function. This structure enables us to incorporate context segmentation inf… view at source ↗

**Figure 5.** Figure 5: The dataset generation pipeline. The protocol is first parsed into a graph with relations between protocol phrases (shown as phrase), then match the protocol phrases with the text spans in transcripts (shown as span). Finally, sequence labeling dataset and span-pair RE dataset are created according to the matches and the relations. We consider three types of procedural relations during the parsing: hnone… view at source ↗

**Figure 6.** Figure 6: The micro F1 score of models on different context level K, evaluated on generated test set [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗

**Figure 7.** Figure 7: The micro F1 score of models on different context level K, evaluated on manual matching test set. the input of models. The level of context is controlled by a hyperparameter K ( [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗

read the original abstract

Cognitive task analysis (CTA) is a type of analysis in applied psychology aimed at eliciting and representing the knowledge and thought processes of domain experts. In CTA, often heavy human labor is involved to parse the interview transcript into structured knowledge (e.g., flowchart for different actions). To reduce human efforts and scale the process, automated CTA transcript parsing is desirable. However, this task has unique challenges as (1) it requires the understanding of long-range context information in conversational text; and (2) the amount of labeled data is limited and indirect---i.e., context-aware, noisy, and low-resource. In this paper, we propose a weakly-supervised information extraction framework for automated CTA transcript parsing. We partition the parsing process into a sequence labeling task and a text span-pair relation extraction task, with distant supervision from human-curated protocol files. To model long-range context information for extracting sentence relations, neighbor sentences are involved as a part of input. Different types of models for capturing context dependency are then applied. We manually annotate real-world CTA transcripts to facilitate the evaluation of the parsing tasks

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

This paper defines a new CTA transcript parsing task and sketches a weak-supervision split into labeling plus span relations, but supplies no results to show whether the approach works.

read the letter

The core contribution is framing automated CTA transcript parsing as a new task and breaking it into sequence labeling followed by span-pair relation extraction, with distant supervision pulled from human-curated protocol files plus neighbor-sentence context. That partitioning is a clean way to handle the conversational structure, and the choice to bring in external protocol signals is a practical response to the low-resource constraint. The paper also correctly flags the two main difficulties: long-range dependencies in interview text and the indirect nature of the available labels. Those observations are accurate and useful for anyone who has tried to apply standard IE pipelines to this kind of data. The manual annotation of real transcripts for evaluation is a necessary step and shows they are thinking about how to measure progress. The main limitation is the complete absence of any quantitative results, ablation, or even a simple check on how well the distant labels match the newly annotated data. Without those numbers it is impossible to tell whether the protocol-derived signals are accurate enough or whether the neighbor-sentence trick actually helps with the noise and context issues. The stress-test concern about alignment between protocol files and interview variability is therefore on target; the paper states the assumption but does not test it. This work is aimed at the small intersection of NLP researchers who do domain-specific extraction and applied psychologists who run CTA studies. A reader in either group could get value from the task definition and the proposed split, but the lack of evidence means it is not yet ready for a serious referee process. I would recommend desk rejection unless the authors add the missing experiments and error analysis.

Referee Report

2 major / 0 minor

Summary. The paper proposes a weakly-supervised information extraction framework for automated parsing of Cognitive Task Analysis (CTA) interview transcripts into structured knowledge. It partitions the parsing process into a sequence labeling task and a text span-pair relation extraction task, using distant supervision from human-curated protocol files. Neighbor sentences are incorporated to model long-range context dependencies in conversational text, and different models are applied for capturing these dependencies. Real-world CTA transcripts are manually annotated to support evaluation of the tasks in a low-resource setting.

Significance. If the framework is shown to be effective, it would address a practical bottleneck in applied psychology by reducing the heavy manual labor required to parse expert interview transcripts. The approach targets real challenges in information extraction from noisy, context-dependent conversational data under limited supervision, which could enable scaling of CTA methods if the distant supervision proves reliable.

major comments (2)

[Abstract] Abstract: the description of the framework, partitioning into sequence labeling plus span-pair relation extraction, and use of distant supervision supplies no quantitative results, error analysis, ablation studies, or performance metrics. This is load-bearing for the central claim, as the effectiveness of the overall approach cannot be evaluated without empirical evidence on how well the models perform on the annotated transcripts.
[Abstract] Abstract / Evaluation plan: no direct audit or precision/recall figures are reported comparing the distant supervision labels derived from protocol files against the newly collected manual annotations. This directly impacts the weakest assumption that protocol-derived signals are sufficiently accurate and relevant to train generalizable models despite conversational noise and long-range dependencies.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback on the abstract and evaluation aspects of our work. We address the two major comments point-by-point below and will incorporate revisions to strengthen the presentation of results and validation of the distant supervision signals.

read point-by-point responses

Referee: [Abstract] Abstract: the description of the framework, partitioning into sequence labeling plus span-pair relation extraction, and use of distant supervision supplies no quantitative results, error analysis, ablation studies, or performance metrics. This is load-bearing for the central claim, as the effectiveness of the overall approach cannot be evaluated without empirical evidence on how well the models perform on the annotated transcripts.

Authors: We agree that the abstract should include key quantitative results, error analysis highlights, and ablation findings to support the central claims. The body of the manuscript reports these details (including model performance on the manually annotated transcripts), but the abstract does not summarize them. We will revise the abstract to incorporate the main performance metrics and a brief reference to the evaluation setup. revision: yes
Referee: [Abstract] Abstract / Evaluation plan: no direct audit or precision/recall figures are reported comparing the distant supervision labels derived from protocol files against the newly collected manual annotations. This directly impacts the weakest assumption that protocol-derived signals are sufficiently accurate and relevant to train generalizable models despite conversational noise and long-range dependencies.

Authors: The current evaluation measures end-task performance of models trained on distant supervision against the manual annotations, which provides an indirect assessment of the distant labels' utility. However, we acknowledge that a direct audit (precision/recall between protocol-derived labels and manual annotations) is not included. This is a valid point for strengthening the validation of the distant supervision. We will add this analysis, reporting precision and recall figures on the overlap between the two label sources. revision: yes

Circularity Check

0 steps flagged

No circularity; external protocol files supply independent distant supervision

full rationale

The paper describes a design choice to partition CTA parsing into sequence labeling plus span-pair relation extraction and to obtain training signals via distant supervision from separately curated protocol files. No equations, fitted parameters, or self-citations appear in the provided text. The supervision source is external to the model and the transcripts, so the central claim does not reduce to a self-definition or a fitted-input prediction. The approach is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The framework rests on standard assumptions of information extraction models and the utility of distant supervision; no new free parameters, axioms beyond domain standards, or invented entities are introduced in the abstract.

axioms (1)

domain assumption Neighbor sentences provide sufficient long-range context for relation extraction in conversational transcripts.
Invoked to justify including neighbor sentences as input for modeling context dependency.

pith-pipeline@v0.9.0 · 5723 in / 1127 out tokens · 27378 ms · 2026-05-25T15:21:39.273180+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Foundation/RealityFromDistinction.lean reality_from_one_distinction unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

We manually annotate real-world CTA transcripts to facilitate the evaluation of the parsing tasks

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

29 extracted references · 29 canonical work pages · 5 internal anchors

[1]

Tim Salimans Alec Radford, Karthik Narasimhan and Ilya Sutskever. 2018. Improving language understanding with unsupervised learning. Technical report, OpenAI

work page 2018
[2]

Rie Kubota Ando and Tong Zhang. 2005. A framework for learning predictive structures from multiple tasks and unlabeled data. Journal of Machine Learning Research, 6(Nov):1817--1853

work page 2005
[3]

Learning Cognitive Models using Neural Networks

Devendra Singh Chaplot, Christopher MacLellan, Ruslan Salakhutdinov, and Kenneth R. Koedinger. 2018. http://arxiv.org/abs/1806.08065 Learning cognitive models using neural networks . CoRR, abs/1806.08065

work page internal anchor Pith review Pith/arXiv arXiv 2018
[4]

Richard E Clark and Fred Estes. 1996. Cognitive task analysis for training. International Journal of Educational Research, 25(5):403--417

work page 1996
[5]

Andrew M Dai and Quoc V Le. 2015. Semi-supervised sequence learning. In Advances in neural information processing systems, pages 3079--3087

work page 2015
[6]

Jacob Devlin, Ming - Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. http://arxiv.org/abs/1810.04805 BERT: pre-training of deep bidirectional transformers for language understanding . CoRR, abs/1810.04805

work page internal anchor Pith review Pith/arXiv arXiv 2018
[7]

Jenny Rose Finkel, Trond Grenager, and Christopher Manning. 2005. Incorporating non-local information into information extraction systems by gibbs sampling. In Proceedings of the 43rd annual meeting on association for computational linguistics, pages 363--370. Association for Computational Linguistics

work page 2005
[8]

Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional lstm-crf models for sequence tagging. arXiv preprint arXiv:1508.01991

work page internal anchor Pith review Pith/arXiv arXiv 2015
[9]

Nan Li, Eliane Stampfer, William Cohen, and Kenneth Koedinger. 2013. General and efficient cognitive model discovery using a simulated student. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 35

work page 2013
[10]

Linqing Liu, Yao Lu, Min Yang, Qiang Qu, Jia Zhu, and Hongyan Li. 2018. https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16238/16492 Generative adversarial network for abstractive text summarization

work page 2018
[11]

Empower Sequence Labeling with Task-Aware Neural Language Model

Liyuan Liu, Jingbo Shang, Frank F. Xu, Xiang Ren, Huan Gui, Jian Peng, and Jiawei Han. 2017. http://arxiv.org/abs/1709.04109 Empower sequence labeling with task-aware neural language model . CoRR, abs/1709.04109

work page internal anchor Pith review Pith/arXiv arXiv 2017
[12]

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111--3119

work page 2013
[13]

Ramesh Nallapati, Bowen Zhou, Caglar Gulcehre, Bing Xiang, et al. 2016. Abstractive text summarization using sequence-to-sequence rnns and beyond. arXiv preprint arXiv:1602.06023

work page internal anchor Pith review Pith/arXiv arXiv 2016
[14]

Hogun Park and Hamid Reza Motahari Nezhad. 2018. https://doi.org/10.1145/3184558.3186347 Learning procedures from text: Codifying how-to procedures in deep neural networks . In Companion Proceedings of the The Web Conference 2018, WWW '18, pages 351--358, Republic and Canton of Geneva, Switzerland. International World Wide Web Conferences Steering Committee

work page doi:10.1145/3184558.3186347 2018
[15]

Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. http://www.aclweb.org/anthology/D14-1162 Glove: Global vectors for word representation . In Empirical Methods in Natural Language Processing (EMNLP), pages 1532--1543

work page 2014
[16]

Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer

Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proc. of NAACL

work page 2018
[17]

Lance A Ramshaw and Mitchell P Marcus. 1999. Text chunking using transformation-based learning. In Natural language processing using very large corpora, pages 157--176. Springer

work page 1999
[18]

Kaitlyn Roose, Elizabeth Veinott, and Shane Mueller. 2018. https://doi.org/10.1145/3270316.3271522 The tracer method: The dynamic duo combining cognitive task analysis and eye tracking

work page doi:10.1145/3270316.3271522 2018
[19]

Jan Maarten Schraagen, Susan F Chipman, and Valerie L Shalin. 2000. Cognitive task analysis. Psychology Press

work page 2000
[20]

Thomas L Seamster and Richard E Redding. 2017. Applied cognitive task analysis in aviation. Routledge

work page 2017
[21]

Dou Shen, Jian-Tao Sun, Hua Li, Qiang Yang, and Zheng Chen. 2007. Document summarization using conditional random fields. In IJCAI, volume 7, pages 2862--2867

work page 2007
[22]

Aarne Talman, Anssi Yli-Jyrä, and Jörg Tiedemann. 2018. http://arxiv.org/abs/1808.08762 Natural language inference with hierarchical bilstm max pooling architecture

work page arXiv 2018
[23]

David D Woods et al. 1989. Cognitive task analysis: An approach to knowledge acquisition for intelligent system design. In Studies in Computer Science and Artificial Intelligence, volume 5, pages 233--264. Elsevier

work page 1989
[24]

Daojian Zeng, Kang Liu, Yubo Chen, and Jun Zhao. 2015. Distant supervision for relation extraction via piecewise convolutional neural networks. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1753--1762

work page 2015
[25]

Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, Jun Zhao, et al. 2014. Relation classification via convolutional deep neural network

work page 2014
[26]

Yuhao Zhang, Victor Zhong, Danqi Chen, Gabor Angeli, and Christopher D Manning. 2017. Position-aware attention and supervised data improve slot filling. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 35--45

work page 2017
[27]

Chen Zhong, John Yen, Peng Liu, Rob Erbacher, Renee Etoty, and Christopher Garneau. 2015. https://doi.org/10.1145/2746194.2746203 An integrated computer-aided cognitive task analysis method for tracing cyber-attack analysis processes . In Proceedings of the 2015 Symposium and Bootcamp on the Science of Security, HotSoS '15, pages 9:1--9:11, New York, NY, USA. ACM

work page doi:10.1145/2746194.2746203 2015
[28]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

work page
[29]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

work page

[1] [1]

Tim Salimans Alec Radford, Karthik Narasimhan and Ilya Sutskever. 2018. Improving language understanding with unsupervised learning. Technical report, OpenAI

work page 2018

[2] [2]

Rie Kubota Ando and Tong Zhang. 2005. A framework for learning predictive structures from multiple tasks and unlabeled data. Journal of Machine Learning Research, 6(Nov):1817--1853

work page 2005

[3] [3]

Learning Cognitive Models using Neural Networks

Devendra Singh Chaplot, Christopher MacLellan, Ruslan Salakhutdinov, and Kenneth R. Koedinger. 2018. http://arxiv.org/abs/1806.08065 Learning cognitive models using neural networks . CoRR, abs/1806.08065

work page internal anchor Pith review Pith/arXiv arXiv 2018

[4] [4]

Richard E Clark and Fred Estes. 1996. Cognitive task analysis for training. International Journal of Educational Research, 25(5):403--417

work page 1996

[5] [5]

Andrew M Dai and Quoc V Le. 2015. Semi-supervised sequence learning. In Advances in neural information processing systems, pages 3079--3087

work page 2015

[6] [6]

Jacob Devlin, Ming - Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. http://arxiv.org/abs/1810.04805 BERT: pre-training of deep bidirectional transformers for language understanding . CoRR, abs/1810.04805

work page internal anchor Pith review Pith/arXiv arXiv 2018

[7] [7]

Jenny Rose Finkel, Trond Grenager, and Christopher Manning. 2005. Incorporating non-local information into information extraction systems by gibbs sampling. In Proceedings of the 43rd annual meeting on association for computational linguistics, pages 363--370. Association for Computational Linguistics

work page 2005

[8] [8]

Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional lstm-crf models for sequence tagging. arXiv preprint arXiv:1508.01991

work page internal anchor Pith review Pith/arXiv arXiv 2015

[9] [9]

Nan Li, Eliane Stampfer, William Cohen, and Kenneth Koedinger. 2013. General and efficient cognitive model discovery using a simulated student. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 35

work page 2013

[10] [10]

Linqing Liu, Yao Lu, Min Yang, Qiang Qu, Jia Zhu, and Hongyan Li. 2018. https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16238/16492 Generative adversarial network for abstractive text summarization

work page 2018

[11] [11]

Empower Sequence Labeling with Task-Aware Neural Language Model

Liyuan Liu, Jingbo Shang, Frank F. Xu, Xiang Ren, Huan Gui, Jian Peng, and Jiawei Han. 2017. http://arxiv.org/abs/1709.04109 Empower sequence labeling with task-aware neural language model . CoRR, abs/1709.04109

work page internal anchor Pith review Pith/arXiv arXiv 2017

[12] [12]

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111--3119

work page 2013

[13] [13]

Ramesh Nallapati, Bowen Zhou, Caglar Gulcehre, Bing Xiang, et al. 2016. Abstractive text summarization using sequence-to-sequence rnns and beyond. arXiv preprint arXiv:1602.06023

work page internal anchor Pith review Pith/arXiv arXiv 2016

[14] [14]

Hogun Park and Hamid Reza Motahari Nezhad. 2018. https://doi.org/10.1145/3184558.3186347 Learning procedures from text: Codifying how-to procedures in deep neural networks . In Companion Proceedings of the The Web Conference 2018, WWW '18, pages 351--358, Republic and Canton of Geneva, Switzerland. International World Wide Web Conferences Steering Committee

work page doi:10.1145/3184558.3186347 2018

[15] [15]

Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. http://www.aclweb.org/anthology/D14-1162 Glove: Global vectors for word representation . In Empirical Methods in Natural Language Processing (EMNLP), pages 1532--1543

work page 2014

[16] [16]

Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer

Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proc. of NAACL

work page 2018

[17] [17]

Lance A Ramshaw and Mitchell P Marcus. 1999. Text chunking using transformation-based learning. In Natural language processing using very large corpora, pages 157--176. Springer

work page 1999

[18] [18]

Kaitlyn Roose, Elizabeth Veinott, and Shane Mueller. 2018. https://doi.org/10.1145/3270316.3271522 The tracer method: The dynamic duo combining cognitive task analysis and eye tracking

work page doi:10.1145/3270316.3271522 2018

[19] [19]

Jan Maarten Schraagen, Susan F Chipman, and Valerie L Shalin. 2000. Cognitive task analysis. Psychology Press

work page 2000

[20] [20]

Thomas L Seamster and Richard E Redding. 2017. Applied cognitive task analysis in aviation. Routledge

work page 2017

[21] [21]

Dou Shen, Jian-Tao Sun, Hua Li, Qiang Yang, and Zheng Chen. 2007. Document summarization using conditional random fields. In IJCAI, volume 7, pages 2862--2867

work page 2007

[22] [22]

Aarne Talman, Anssi Yli-Jyrä, and Jörg Tiedemann. 2018. http://arxiv.org/abs/1808.08762 Natural language inference with hierarchical bilstm max pooling architecture

work page arXiv 2018

[23] [23]

David D Woods et al. 1989. Cognitive task analysis: An approach to knowledge acquisition for intelligent system design. In Studies in Computer Science and Artificial Intelligence, volume 5, pages 233--264. Elsevier

work page 1989

[24] [24]

Daojian Zeng, Kang Liu, Yubo Chen, and Jun Zhao. 2015. Distant supervision for relation extraction via piecewise convolutional neural networks. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1753--1762

work page 2015

[25] [25]

Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, Jun Zhao, et al. 2014. Relation classification via convolutional deep neural network

work page 2014

[26] [26]

Yuhao Zhang, Victor Zhong, Danqi Chen, Gabor Angeli, and Christopher D Manning. 2017. Position-aware attention and supervised data improve slot filling. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 35--45

work page 2017

[27] [27]

Chen Zhong, John Yen, Peng Liu, Rob Erbacher, Renee Etoty, and Christopher Garneau. 2015. https://doi.org/10.1145/2746194.2746203 An integrated computer-aided cognitive task analysis method for tracing cyber-attack analysis processes . In Proceedings of the 2015 Symposium and Bootcamp on the Science of Security, HotSoS '15, pages 9:1--9:11, New York, NY, USA. ACM

work page doi:10.1145/2746194.2746203 2015

[28] [28]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

work page

[29] [29]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

work page