BERT-DST: Scalable End-to-End Dialogue State Tracking with Bidirectional Encoder Representations from Transformer

Guan-Lin Chao; Ian Lane

arxiv: 1907.03040 · v1 · pith:WST2HX3Xnew · submitted 2019-07-05 · 💻 cs.CL

BERT-DST: Scalable End-to-End Dialogue State Tracking with Bidirectional Encoder Representations from Transformer

Guan-Lin Chao , Ian Lane This is my paper

Pith reviewed 2026-05-25 01:55 UTC · model grok-4.3

classification 💻 cs.CL

keywords dialogue state trackingBERTend-to-end DSTscalable DSTparameter sharingslot value extractioncontextual representations

0 comments

The pith

BERT-DST directly extracts slot values from dialogue context with a shared BERT encoder for scalable tracking.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper targets the scalability problem in dialogue state tracking when ontologies are dynamic and many slot values are unseen at training time. It replaces candidate generation or tagging steps with direct extraction of values as word segments from the context. BERT supplies contextual representations that identify values by semantic surroundings, while parameter sharing across slots keeps the model size fixed and moves language knowledge between slots. The result is stronger performance on the scalable Sim-M and Sim-R benchmarks and competitive scores on DSTC2 and WOZ 2.0. A reader cares because this removes the need for exhaustive candidate lists that grow with real-world ontologies.

Core claim

BERT-DST is an end-to-end dialogue state tracker that directly extracts slot values from the dialogue context using BERT as the encoder. Encoder parameters are shared across all slots so that their count does not grow linearly with ontology size and language representation knowledge transfers among slots. On the benchmark scalable DST datasets Sim-M and Sim-R the model with cross-slot sharing outperforms prior work, while on the standard DSTC2 and WOZ 2.0 datasets it reaches competitive performance.

What carries the argument

BERT encoder with cross-slot parameter sharing that performs direct value extraction from context

If this is right

Parameter count stays constant as the ontology grows.
Language knowledge learned for one slot improves extraction for other slots.
Direct extraction removes the error propagation that occurs in separate candidate-generation or tagging stages.
Unseen values are handled without retraining as long as they occur in the context.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same direct-extraction pattern could be tested on related sequence-labeling tasks such as coreference resolution inside conversations.
Sharing might lower sample complexity for rare slots, which could be checked by training curves on progressively smaller data splits.
If values must be generated rather than extracted, a hybrid model that first decides whether extraction is possible would be a natural next step.

Load-bearing premise

The correct slot value, other than none or dontcare, always appears as a contiguous word segment inside the dialogue context.

What would settle it

A set of test dialogues in which the true slot value is a paraphrase or implication never stated verbatim as a context segment; the tracker should then fail to recover the correct state.

Figures

Figures reproduced from arXiv: 1907.03040 by Guan-Lin Chao, Ian Lane.

**Figure 1.** Figure 1: Architecture of the proposed BERT-DST framework. The diagram is color-coded such that modules with the same color share the same parameters. For each user turn, BERT-DST takes as input the recent dialogue context (system utterance in previous turn and the user utterance), and outputs turn-level dialogue state. BERT dialogue context encoding module ΦBERT (blue) produces contextualized sentence-level and tok… view at source ↗

**Figure 3.** Figure 3: While a proper selection of slot value dropout rate can result in slight improvement on Sim-R, the effect of slot value DST Models DSTC2 WOZ 2.0 DST + LU Candidates [7] 67.0% - DST + n-gram Candidates [8] 68.2±1.8% - DST + Oracle Candidates [5] 70.3% - Pointer Network [6] 72.1% - Delex.-Based Model [2] 69.1% 70.8% Delex. + Semantic Dict. [2] 72.9% 83.7% Neural Belief Tracker [2] 73.4% 84.2% GLAD [3] 74.5±0… view at source ↗

**Figure 4.** Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗

read the original abstract

An important yet rarely tackled problem in dialogue state tracking (DST) is scalability for dynamic ontology (e.g., movie, restaurant) and unseen slot values. We focus on a specific condition, where the ontology is unknown to the state tracker, but the target slot value (except for none and dontcare), possibly unseen during training, can be found as word segment in the dialogue context. Prior approaches often rely on candidate generation from n-gram enumeration or slot tagger outputs, which can be inefficient or suffer from error propagation. We propose BERT-DST, an end-to-end dialogue state tracker which directly extracts slot values from the dialogue context. We use BERT as dialogue context encoder whose contextualized language representations are suitable for scalable DST to identify slot values from their semantic context. Furthermore, we employ encoder parameter sharing across all slots with two advantages: (1) Number of parameters does not grow linearly with the ontology. (2) Language representation knowledge can be transferred among slots. Empirical evaluation shows BERT-DST with cross-slot parameter sharing outperforms prior work on the benchmark scalable DST datasets Sim-M and Sim-R, and achieves competitive performance on the standard DSTC2 and WOZ 2.0 datasets.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

BERT-DST shows that cross-slot parameter sharing lets BERT do direct value extraction for scalable DST, with clear gains on Sim-M/Sim-R under a narrow scope.

read the letter

The core contribution is a straightforward application of BERT for end-to-end DST that skips candidate generation. By sharing the encoder across slots, the model keeps a fixed parameter count even as the ontology grows and lets representations transfer between slots. This matches the practical need for dynamic ontologies where values can be unseen at training time but still appear as contiguous spans in the dialogue context. The abstract reports outperformance on the scalable benchmarks Sim-M and Sim-R plus competitive numbers on DSTC2 and WOZ 2.0, which is the main empirical result to evaluate. The design is clean on its own terms and directly targets the inefficiency of n-gram or tagger-based candidates. The main limitation is the explicit scope: the approach only applies when the target value (other than none or dontcare) is literally present as a word segment. That rules out cases needing generation or inference beyond the text, so the claim is scoped rather than general. Results on the standard datasets are described as competitive rather than dominant, and the abstract gives no error analysis or ablation on how much the sharing itself drives the gains versus BERT alone. The citation pattern in the abstract looks standard for the area. This paper is aimed at researchers working on task-oriented dialogue systems who care about scaling to large or changing slot sets. It is solid enough on the empirical side and the assumption is stated up front, so it deserves a serious referee even if revisions will be needed on the experimental details and broader applicability.

Referee Report

0 major / 2 minor

Summary. The manuscript proposes BERT-DST, an end-to-end dialogue state tracker that encodes dialogue context with BERT and directly extracts slot values as contiguous word segments from the context (except none/dontcare). It introduces cross-slot parameter sharing to keep the parameter count independent of ontology size and to enable knowledge transfer across slots. The work explicitly scopes its claims to the word-segment condition and reports that BERT-DST with sharing outperforms prior methods on the scalable DST benchmarks Sim-M and Sim-R while remaining competitive on DSTC2 and WOZ 2.0.

Significance. If the empirical results hold under the stated scoping, the paper supplies a practical, scalable DST approach that sidesteps explicit candidate generation and n-gram enumeration. Strengths include the clear statement of the operating condition, the use of a pre-trained contextual encoder, and the parameter-sharing design that prevents linear growth with ontology size. These elements directly address the dynamic-ontology problem highlighted in the abstract.

minor comments (2)

[Abstract] The abstract states that BERT-DST 'outperforms prior work' on Sim-M and Sim-R but does not name the exact prior systems or report the absolute joint-goal accuracy numbers; adding these in §4 or Table 1 would allow readers to assess the magnitude of the improvement.
[§3] The description of the direct-extraction head (presumably in §3) should include the precise span-prediction loss and how 'none' and 'dontcare' are handled separately, as these choices are load-bearing for reproducibility on the reported datasets.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the positive summary, significance assessment, and recommendation of minor revision. No major comments were listed in the report, so we have no specific points to address point-by-point. We are happy to incorporate any minor suggestions that may arise.

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper is an empirical ML contribution: it defines BERT-DST as a fine-tuned BERT encoder with cross-slot parameter sharing for direct span extraction under an explicitly scoped assumption (slot values appear as contiguous segments). Performance claims rest on reported numbers from standard benchmarks (Sim-M, Sim-R, DSTC2, WOZ 2.0) rather than any derivation, fitted parameter renamed as prediction, or self-citation chain. No equations or load-bearing steps reduce to author-defined inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Review based on abstract only; the central claim rests on the pre-trained BERT model (parameters fitted on external corpora) and the assumption that slot values appear as contiguous word segments. No explicit free parameters, axioms, or invented entities are stated in the provided text.

pith-pipeline@v0.9.0 · 5743 in / 1137 out tokens · 24519 ms · 2026-05-25T01:55:35.642464+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

29 extracted references · 29 canonical work pages · 4 internal anchors

[1]

The dialogue states predicted by DST are used by the downstream dialogue man- agement component to produce API calls to a backend database and generate responses to the user [1]

Introduction Dialogue state tracking (DST), a core component in today’s task-oriented dialogue systems, maintains user’s intentional states through the course of a dialogue. The dialogue states predicted by DST are used by the downstream dialogue man- agement component to produce API calls to a backend database and generate responses to the user [1]. A di...

work page
[2]

BERT-DST: Scalable End-to-End Dialogue State Tracking with Bidirectional Encoder Representations from Transformer

BERT In this section, we brieﬂy describe BERT [9] and how its archi- tecture can be applied to scalable DST in our framework. BERT is a multi-layer bidirectional Transformer en- coder [14], which is a stack of multiple identical layers each containing a multi-head self-attention and a fully-connected sub-layer with residual connections [15]. The input to ...

work page internal anchor Pith review Pith/arXiv arXiv 1907
[3]

The procedures of lan- guage model pre-training are detailed in [9]

and the English Wikipedia corpora. The procedures of lan- guage model pre-training are detailed in [9]. With extra projec- tion layers and ﬁne-tuning the deep structure, BERT has been successfully applied to various tasks such as reading compre- hension, named entity recognition, sentiment analysis, etc. Our proposed application of BERT to scalable DST is...

work page
[4]

For each user turn, BERT- DST takes the recent dialogue context as input and outputs the turn-level dialogue state

BERT-DST In this section, we describe in detail the proposed BERT-DST framework, as shown in Figure 1. For each user turn, BERT- DST takes the recent dialogue context as input and outputs the turn-level dialogue state. First, the dialogue context input is encoded by the BERT-based encoding module to produce con- textualized sentence-level and token-level ...

work page
[5]

The model’s prediction has to jointly match all the informable slot labels to be considered correct

Experiments We evaluate our models using joint goal accuracy [12], a stan- dard metric for DST. The model’s prediction has to jointly match all the informable slot labels to be considered correct. 4.1. Datasets We evaluate our models on four benchmark datasets: Sim-M, Sim-R [11], DSTC2 [12] and WOZ 2.0 [13]. The statistics of the datasets are shown in Tab...

work page
[6]

Results and Discussion Table 1 presents the performance of the proposed BERT-DST models compared to prior work on the scalable DST datasets Sim-M and Sim-R. In [7, 5], the DST component scores slot values from a candidate list, which is slot tagging predictions of a jointly-trained language understanding component (DST + LU Candidates), or the ground trut...

work page
[7]

Not requiring candidate value generation, BERT-DST directly pre- dicts slot values from the dialogue context

Conclusions We introduce BERT-DST, a scalable end-to-end dialogue state tracker to handle unknown ontology and unseen slot values. Not requiring candidate value generation, BERT-DST directly pre- dicts slot values from the dialogue context. The key component is the BERT dialogue context encoding module which produces contextualized representations effecti...

work page
[8]

The hidden information state model: A practical framework for pomdp-based spoken dialogue manage- ment,

S. Young, M. Ga ˇsi´c, S. Keizer, F. Mairesse, J. Schatzmann, B. Thomson, and K. Yu, “The hidden information state model: A practical framework for pomdp-based spoken dialogue manage- ment,” Computer Speech & Language, 2010

work page 2010
[9]

Neural belief tracker: Data-driven dialogue state track- ing,

N. Mrk ˇsi´c, D. ´O. S ´eaghdha, T.-H. Wen, B. Thomson, and S. Young, “Neural belief tracker: Data-driven dialogue state track- ing,” inAnnual Meeting of the Association for Computational Lin- guistics (ACL), 2017

work page 2017
[10]

Global-locally self-attentive encoder for dialogue state tracking,

V . Zhong, C. Xiong, and R. Socher, “Global-locally self-attentive encoder for dialogue state tracking,” inAnnual Meeting of the As- sociation for Computational Linguistics (ACL), 2018

work page 2018
[11]

Towards universal dialogue state tracking,

L. Ren, K. Xie, L. Chen, and K. Yu, “Towards universal dialogue state tracking,” in Conference on Empirical Methods in Natural Language Processing (EMNLP), 2018

work page 2018
[12]

Scalable multi-domain dialogue state tracking,

A. Rastogi, D. Hakkani-T ¨ur, and L. Heck, “Scalable multi-domain dialogue state tracking,” in Automatic Speech Recognition and Understanding Workshop (ASRU), 2017

work page 2017
[13]

An end-to-end approach for handling unknown slot values in dialogue state tracking,

P. Xu and Q. Hu, “An end-to-end approach for handling unknown slot values in dialogue state tracking,” in Annual Meeting of the Association for Computational Linguistics (ACL), 2018

work page 2018
[14]

Multi-task learning for joint language understanding and dialogue state tracking,

A. Rastogi, R. Gupta, and D. Hakkani-Tur, “Multi-task learning for joint language understanding and dialogue state tracking,” in Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), 2018

work page 2018
[15]

Flexible and scalable state tracking framework for goal-oriented dialogue systems,

R. Goel, S. Paul, T. Chung, J. Lecomte, A. Mandal, D. Hakkani- Tur, and A. A. AI, “Flexible and scalable state tracking framework for goal-oriented dialogue systems,” in NeurIPS Conversational AI workshop, 2018

work page 2018
[16]

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” Computing Research Repos- itory, vol. arXiv:1810.04805, 2018. [Online]. Available: http://arxiv.org/abs/1810.04805

work page internal anchor Pith review Pith/arXiv arXiv 2018
[17]

Targeted feature dropout for robust slot ﬁlling in natural language understanding,

P. Xu and R. Sarikaya, “Targeted feature dropout for robust slot ﬁlling in natural language understanding,” in Annual Conference of the International Speech Communication Association, 2014

work page 2014
[18]

Building a Conversational Agent Overnight with Dialogue Self-Play

P. Shah, D. Hakkani-T ¨ur, G. T¨ur, A. Rastogi, A. Bapna, N. Nayak, and L. Heck, “Building a conversational agent overnight with dialogue self-play,” Computing Research Repository , vol. arXiv:1801.04871, 2018. [Online]. Available: http://arxiv.org/ abs/1801.04871

work page internal anchor Pith review Pith/arXiv arXiv 2018
[19]

The second di- alog state tracking challenge,

M. Henderson, B. Thomson, and J. D. Williams, “The second di- alog state tracking challenge,” in Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), 2014

work page 2014
[20]

A network-based end-to-end trainable task-oriented dialogue system,

T.-H. Wen, D. Vandyke, N. Mrkˇsi´c, M. Gasic, L. M. R. Barahona, P.-H. Su, S. Ultes, and S. Young, “A network-based end-to-end trainable task-oriented dialogue system,” inConference of the Eu- ropean Chapter of the Association for Computational Linguistics (EACL), 2017

work page 2017
[21]

Attention is all you need,

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems, 2017

work page 2017
[22]

Deep residual learning for image recognition,

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Computer Vision and Pattern Recognition (CVPR), 2016

work page 2016
[23]

cloze procedure: A new tool for measuring read- ability,

W. L. Taylor, “cloze procedure: A new tool for measuring read- ability,”Journalism Bulletin, 1953

work page 1953
[24]

Aligning books and movies: Towards story- like visual explanations by watching movies and reading books,

Y . Zhu, R. Kiros, R. Zemel, R. Salakhutdinov, R. Urtasun, A. Tor- ralba, and S. Fidler, “Aligning books and movies: Towards story- like visual explanations by watching movies and reading books,” in International Conference on Computer Vision (ICCV), 2015

work page 2015
[25]

Squad: 100,000+ questions for machine comprehension of text,

P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang, “Squad: 100,000+ questions for machine comprehension of text,” in Con- ference on Empirical Methods in Natural Language Processing (EMNLP), 2016

work page 2016
[26]

Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

Y . Wu, M. Schuster, Z. Chen, Q. V . Le, M. Norouzi, W. Macherey, M. Krikun, Y . Cao, Q. Gao, K. Macherey et al., “Google’s neural machine translation system: Bridging the gap between human and machine translation,” Computing Research Repository, vol. arXiv:1609.08144, 2016. [Online]. Available: http://arxiv.org/abs/1609.08144

work page internal anchor Pith review Pith/arXiv arXiv 2016
[27]

Tensor2Tensor for neural machine translation,

A. Vaswani, S. Bengio, E. Brevdo, F. Chollet, A. Gomez, S. Gouws, L. Jones, Ł. Kaiser, N. Kalchbrenner, N. Parmar et al., “Tensor2Tensor for neural machine translation,” in Conference of the Association for Machine Translation in the Americas, 2018

work page 2018
[28]

Adam: A method for stochastic opti- mization,

D. P. Kingma and J. Ba, “Adam: A method for stochastic opti- mization,” in International Conference on Learning Representa- tions (ICLR), 2014

work page 2014
[29]

Dropout: a simple way to prevent neural net- works from overﬁtting,

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: a simple way to prevent neural net- works from overﬁtting,”Journal of Machine Learning Research, 2014. A. Appendix Datasets # Dialogues Slots(train, dev, test) Sim-M 384, 120, 264 date, time, numtickets, theatre name, movie(5/5; 26/26) Sim-R 1116, 349, 775 date, time, c...

work page 2014

[1] [1]

The dialogue states predicted by DST are used by the downstream dialogue man- agement component to produce API calls to a backend database and generate responses to the user [1]

Introduction Dialogue state tracking (DST), a core component in today’s task-oriented dialogue systems, maintains user’s intentional states through the course of a dialogue. The dialogue states predicted by DST are used by the downstream dialogue man- agement component to produce API calls to a backend database and generate responses to the user [1]. A di...

work page

[2] [2]

BERT-DST: Scalable End-to-End Dialogue State Tracking with Bidirectional Encoder Representations from Transformer

BERT In this section, we brieﬂy describe BERT [9] and how its archi- tecture can be applied to scalable DST in our framework. BERT is a multi-layer bidirectional Transformer en- coder [14], which is a stack of multiple identical layers each containing a multi-head self-attention and a fully-connected sub-layer with residual connections [15]. The input to ...

work page internal anchor Pith review Pith/arXiv arXiv 1907

[3] [3]

The procedures of lan- guage model pre-training are detailed in [9]

and the English Wikipedia corpora. The procedures of lan- guage model pre-training are detailed in [9]. With extra projec- tion layers and ﬁne-tuning the deep structure, BERT has been successfully applied to various tasks such as reading compre- hension, named entity recognition, sentiment analysis, etc. Our proposed application of BERT to scalable DST is...

work page

[4] [4]

For each user turn, BERT- DST takes the recent dialogue context as input and outputs the turn-level dialogue state

BERT-DST In this section, we describe in detail the proposed BERT-DST framework, as shown in Figure 1. For each user turn, BERT- DST takes the recent dialogue context as input and outputs the turn-level dialogue state. First, the dialogue context input is encoded by the BERT-based encoding module to produce con- textualized sentence-level and token-level ...

work page

[5] [5]

The model’s prediction has to jointly match all the informable slot labels to be considered correct

Experiments We evaluate our models using joint goal accuracy [12], a stan- dard metric for DST. The model’s prediction has to jointly match all the informable slot labels to be considered correct. 4.1. Datasets We evaluate our models on four benchmark datasets: Sim-M, Sim-R [11], DSTC2 [12] and WOZ 2.0 [13]. The statistics of the datasets are shown in Tab...

work page

[6] [6]

Results and Discussion Table 1 presents the performance of the proposed BERT-DST models compared to prior work on the scalable DST datasets Sim-M and Sim-R. In [7, 5], the DST component scores slot values from a candidate list, which is slot tagging predictions of a jointly-trained language understanding component (DST + LU Candidates), or the ground trut...

work page

[7] [7]

Not requiring candidate value generation, BERT-DST directly pre- dicts slot values from the dialogue context

Conclusions We introduce BERT-DST, a scalable end-to-end dialogue state tracker to handle unknown ontology and unseen slot values. Not requiring candidate value generation, BERT-DST directly pre- dicts slot values from the dialogue context. The key component is the BERT dialogue context encoding module which produces contextualized representations effecti...

work page

[8] [8]

The hidden information state model: A practical framework for pomdp-based spoken dialogue manage- ment,

S. Young, M. Ga ˇsi´c, S. Keizer, F. Mairesse, J. Schatzmann, B. Thomson, and K. Yu, “The hidden information state model: A practical framework for pomdp-based spoken dialogue manage- ment,” Computer Speech & Language, 2010

work page 2010

[9] [9]

Neural belief tracker: Data-driven dialogue state track- ing,

N. Mrk ˇsi´c, D. ´O. S ´eaghdha, T.-H. Wen, B. Thomson, and S. Young, “Neural belief tracker: Data-driven dialogue state track- ing,” inAnnual Meeting of the Association for Computational Lin- guistics (ACL), 2017

work page 2017

[10] [10]

Global-locally self-attentive encoder for dialogue state tracking,

V . Zhong, C. Xiong, and R. Socher, “Global-locally self-attentive encoder for dialogue state tracking,” inAnnual Meeting of the As- sociation for Computational Linguistics (ACL), 2018

work page 2018

[11] [11]

Towards universal dialogue state tracking,

L. Ren, K. Xie, L. Chen, and K. Yu, “Towards universal dialogue state tracking,” in Conference on Empirical Methods in Natural Language Processing (EMNLP), 2018

work page 2018

[12] [12]

Scalable multi-domain dialogue state tracking,

A. Rastogi, D. Hakkani-T ¨ur, and L. Heck, “Scalable multi-domain dialogue state tracking,” in Automatic Speech Recognition and Understanding Workshop (ASRU), 2017

work page 2017

[13] [13]

An end-to-end approach for handling unknown slot values in dialogue state tracking,

P. Xu and Q. Hu, “An end-to-end approach for handling unknown slot values in dialogue state tracking,” in Annual Meeting of the Association for Computational Linguistics (ACL), 2018

work page 2018

[14] [14]

Multi-task learning for joint language understanding and dialogue state tracking,

A. Rastogi, R. Gupta, and D. Hakkani-Tur, “Multi-task learning for joint language understanding and dialogue state tracking,” in Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), 2018

work page 2018

[15] [15]

Flexible and scalable state tracking framework for goal-oriented dialogue systems,

R. Goel, S. Paul, T. Chung, J. Lecomte, A. Mandal, D. Hakkani- Tur, and A. A. AI, “Flexible and scalable state tracking framework for goal-oriented dialogue systems,” in NeurIPS Conversational AI workshop, 2018

work page 2018

[16] [16]

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” Computing Research Repos- itory, vol. arXiv:1810.04805, 2018. [Online]. Available: http://arxiv.org/abs/1810.04805

work page internal anchor Pith review Pith/arXiv arXiv 2018

[17] [17]

Targeted feature dropout for robust slot ﬁlling in natural language understanding,

P. Xu and R. Sarikaya, “Targeted feature dropout for robust slot ﬁlling in natural language understanding,” in Annual Conference of the International Speech Communication Association, 2014

work page 2014

[18] [18]

Building a Conversational Agent Overnight with Dialogue Self-Play

P. Shah, D. Hakkani-T ¨ur, G. T¨ur, A. Rastogi, A. Bapna, N. Nayak, and L. Heck, “Building a conversational agent overnight with dialogue self-play,” Computing Research Repository , vol. arXiv:1801.04871, 2018. [Online]. Available: http://arxiv.org/ abs/1801.04871

work page internal anchor Pith review Pith/arXiv arXiv 2018

[19] [19]

The second di- alog state tracking challenge,

M. Henderson, B. Thomson, and J. D. Williams, “The second di- alog state tracking challenge,” in Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), 2014

work page 2014

[20] [20]

A network-based end-to-end trainable task-oriented dialogue system,

T.-H. Wen, D. Vandyke, N. Mrkˇsi´c, M. Gasic, L. M. R. Barahona, P.-H. Su, S. Ultes, and S. Young, “A network-based end-to-end trainable task-oriented dialogue system,” inConference of the Eu- ropean Chapter of the Association for Computational Linguistics (EACL), 2017

work page 2017

[21] [21]

Attention is all you need,

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems, 2017

work page 2017

[22] [22]

Deep residual learning for image recognition,

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Computer Vision and Pattern Recognition (CVPR), 2016

work page 2016

[23] [23]

cloze procedure: A new tool for measuring read- ability,

W. L. Taylor, “cloze procedure: A new tool for measuring read- ability,”Journalism Bulletin, 1953

work page 1953

[24] [24]

Aligning books and movies: Towards story- like visual explanations by watching movies and reading books,

Y . Zhu, R. Kiros, R. Zemel, R. Salakhutdinov, R. Urtasun, A. Tor- ralba, and S. Fidler, “Aligning books and movies: Towards story- like visual explanations by watching movies and reading books,” in International Conference on Computer Vision (ICCV), 2015

work page 2015

[25] [25]

Squad: 100,000+ questions for machine comprehension of text,

P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang, “Squad: 100,000+ questions for machine comprehension of text,” in Con- ference on Empirical Methods in Natural Language Processing (EMNLP), 2016

work page 2016

[26] [26]

Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

Y . Wu, M. Schuster, Z. Chen, Q. V . Le, M. Norouzi, W. Macherey, M. Krikun, Y . Cao, Q. Gao, K. Macherey et al., “Google’s neural machine translation system: Bridging the gap between human and machine translation,” Computing Research Repository, vol. arXiv:1609.08144, 2016. [Online]. Available: http://arxiv.org/abs/1609.08144

work page internal anchor Pith review Pith/arXiv arXiv 2016

[27] [27]

Tensor2Tensor for neural machine translation,

A. Vaswani, S. Bengio, E. Brevdo, F. Chollet, A. Gomez, S. Gouws, L. Jones, Ł. Kaiser, N. Kalchbrenner, N. Parmar et al., “Tensor2Tensor for neural machine translation,” in Conference of the Association for Machine Translation in the Americas, 2018

work page 2018

[28] [28]

Adam: A method for stochastic opti- mization,

D. P. Kingma and J. Ba, “Adam: A method for stochastic opti- mization,” in International Conference on Learning Representa- tions (ICLR), 2014

work page 2014

[29] [29]

Dropout: a simple way to prevent neural net- works from overﬁtting,

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: a simple way to prevent neural net- works from overﬁtting,”Journal of Machine Learning Research, 2014. A. Appendix Datasets # Dialogues Slots(train, dev, test) Sim-M 384, 120, 264 date, time, numtickets, theatre name, movie(5/5; 26/26) Sim-R 1116, 349, 775 date, time, c...

work page 2014