Dialogue Coherence Assessment Without Explicit Dialogue Act Labels

Iryna Gurevych; Mohsen Mesgar; Sebastian B\"ucker

REVIEW 2 major objections 6 minor 27 references

Dialogue Coherence Assessment Without Explicit Dialogue Act Labels

T0 review · 2 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Dialogue coherence can be ranked without dialogue act labels at evaluation time, using act prediction only as a training-side auxiliary task.

desk verdict MTL with dialogue act prediction as an auxiliary task is a real improvement for dialogue coherence assessment, but the claims need calibration and the synthetic-only evaluation limits the practical reach. read the letter →

arxiv 1908.08486 v2 pith:H4HYB6H2 submitted 2019-08-22 cs.CL

Mohsen Mesgar , Sebastian Bücker , Iryna Gurevych This is my paper · ORCID

classification cs.CL

keywords dialoguecoherenceactpredictionmulti-tasklearningutterancerepresentationrankingperturbation-basedevaluationDailyDialogSwitchBoard

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Dialogue coherence assessment usually needs to know each utterance's speech-act type at scoring time, either from annotators or a separate classifier. This paper argues that those labels are only needed during training: it couples the coherence ranker with a dialogue-act predictor in a multi-task setup, sharing an utterance encoder, so the act labels shape the representations without being inputs at evaluation. On DailyDialog the resulting M-DiCoh model beats the entity-grid-plus-act baseline EAGrid by more than 20 accuracy points on most of the tested perturbation tasks, and on SwitchBoard it is broadly on par, though weaker on utterance replacement. If this holds, coherence evaluation of dialogue systems can drop expensive act annotation and avoid propagating act-classifier errors into the score.

What carries the argument

The machinery is a multi-task learner with two heads on one utterance encoder. An utterance is encoded by word embeddings, a BiLSTM, and self-attention; the coherence head (DiCoh) runs another BiLSTM and attention over the utterance vectors and maps the dialogue to a scalar score, while the dialogue-act head (DAP) is a softmax over act types. The two are trained jointly with the weighted loss from Equation (11), pairing a margin ranking loss on dialogue pairs with average cross-entropy over act labels, where the trade-off weights $\gamma_1, \gamma_2$ are learned following the uncertainty-weighting idea. Dialogue act labels therefore act as a regularizer that pushes the shared encoder to attend to act-salient words during training, and are not needed at inference.

What would settle it

Collect human coherence rankings for unperturbed dialogues from DailyDialog and SwitchBoard and compare them with M-DiCoh's scores: if the model's ranking correlates with human judgments no better than a model trained without the dialogue-act auxiliary task, the central claim fails. A simpler version is to score real dialogues that differ only by a single swapped turn and check whether human-preferred dialogues receive higher scores.

Watch

Extended reading notes

Core claim

Dialogue coherence ranking can be done without supplying dialogue act (DA) labels at evaluation time, provided DA prediction is used as an auxiliary training signal. The paper's M-DiCoh model trains a pairwise coherence scorer (DiCoh) together with a softmax DA predictor (DAP) over a shared BiLSTM-plus-attention utterance encoder; DA labels appear only in the auxiliary cross-entropy loss. In the perturbation-based ranking tests, M-DiCoh reaches 95.92% accuracy on utterance ordering for DailyDialog against 71.72% for EAGrid and 68.21% for ASeq, with analogous margins on insertion and even-utterance ordering. On SwitchBoard, where DA labels are far more fine-grained and imbalanced, M-DiCoh scores 99.41% on utterance ordering and 85.04% on insertion, close to or above EAGrid, but trails on utterance replacement (58.67% vs. 75.61%).

Load-bearing premise

The evaluation assumes that every synthetically perturbed dialogue is always less coherent than the original, so accuracy in telling original from perturbed is a valid measure of coherence; if the model is instead exploiting surface artifacts of the perturbations, the rankings would not transfer to real dialogue systems.

Editorial extensions

If this is right

Coherence scoring no longer requires gold or predicted dialogue act labels at evaluation time, removing an annotation bottleneck.
The model can compare open-domain dialogue agents for coherence without an upstream dialogue-act tagger, so act-classifier errors do not propagate into the coherence score.
The two new perturbation types, utterance replacement and even-utterance ordering, add harder, dialogue-specific tests to the evaluation protocol.
On DailyDialog the multi-task formulation beats the entity-grid baseline by large margins; on SwitchBoard the gap closes, indicating robustness depends on DA granularity and utterance length.
Because act labels are only a training signal, the same architecture can be used in domains where acts are annotated for a small training set but not for evaluation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension of the paper's logic is that the auxiliary DA task helps mainly by focusing attention on question verbs, adjectives, and content nouns; ablating attention or replacing the DA head with a turn-boundary prediction head would isolate whether the gain comes from act semantics or from any discourse-level regularizer.
The paper's observation that coherence supervision does not improve DA prediction (M-DAP is on par with S-DAP) suggests the transfer is one-directional; one could test pretraining the shared encoder on a large DA corpus before coherence training as a cheaper substitute for joint multi-task learning.
If the reason for the SwitchBoard gap is fine-grained, imbalanced DA labels, then mapping SwitchBoard acts onto a coarse taxonomy before multi-task training should recover most of the margin; this is directly checkable.
The same multi-task recipe could be applied to other dialogue quality dimensions, such as using sentiment or topic labels as auxiliary tasks for coherence, since the method only requires a cheap auxiliary label that correlates with the target quality.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

The one to know: this paper shows you can train a dialogue coherence ranker with dialogue act prediction as an auxiliary task, so at evaluation you only need the raw text. That's a genuinely useful move—prior work like Cervone et al. fed gold DA labels into the model at test time, which is expensive and brittle. The architecture itself is a standard BiLSTM with self-attention, but the MTL framing is new.

What's good: the experiments are reasonably thorough. Four perturbation domains, five seeds, code released. On DailyDialog, M-DiCoh beats EAGrid by wide margins on every perturbation. Cross-domain transfer is also consistently better than EAGrid, which is a nice robustness point.

Now the soft spots. First, the abstract overclaims. The 'more than 20 accuracy points' doesn't hold for UR on DailyDialog (83.02 vs 68.49, about 14.5 points), and 'on par' on SwitchBoard ignores UR, where EAGrid gets 75.61 and M-DiCoh gets 58.67. If you cite these numbers, cite the table, not the abstract.

Second, the evaluation is entirely on synthetic perturbations. All positive/negative pairs come from UO, UI, UR, EUO, and the model is just asked to tell which dialogue is the original. That could be solved by shallow artifacts rather than true coherence. The authors acknowledge this and list human correlation as future work, which is honest, but it means the practical claim—'suitable for evaluating dialogue agents in real scenarios'—is not yet backed. This is the load-bearing limitation if you care about real-world use.

Minor: hyperparameters are tuned only on DailyDialog UO and applied to SwitchBoard unchanged, so SwitchBoard performance is probably not fully optimized. Not a fatal flaw, but worth noting.

Bottom line: it's a solid method paper that will be useful to anyone working on dialogue evaluation or coherence modeling. It deserves a serious referee and publication after claim calibration and clearer limitation framing. I'd cite it for the MTL trick.

Recommendation: send to peer review; ask for a corrected abstract and a short human-correlation experiment if feasible.

Referee Report

2 major / 6 minor

Summary. The paper addresses dialogue coherence assessment by training a coherence ranker (DiCoh) jointly with a dialogue act prediction (DAP) auxiliary task in a multi-task learning framework. The utterance encoder is shared, and at evaluation only utterance text is needed, so explicit DA labels are not required. Experiments on DailyDialog and SwitchBoard under four synthetic perturbation domains (UO, UI, UR, EUO) compare M-DiCoh with baselines including EAGrid, ASeq, CoSim, and random; additionally, cross problem-domain evaluation on DailyDialog and DA prediction accuracy are reported. The paper claims substantial gains on DailyDialog and on-par performance on SwitchBoard, and releases source code.

Significance. The idea of using DA prediction as an auxiliary task to enrich utterance representations for coherence is sensible and novel relative to prior entity-grid and DA-transition approaches. The paper ships machine-checked experiments with five random seeds, multiple perturbation protocols, and cross-domain evaluation, and the code is released. If the synthetic evaluation reflects coherence, the method would be practically useful because it removes the need for gold DA labels at evaluation time. However, the absence of any human correlation study and the abstract's overstatement of results temper the significance.

major comments (2)

[§4.2 and §5] All experiments train and test on pairs of original dialogues and their synthetic perturbations (UO, UI, UR, EUO), under the assumption that the original dialogue is always more coherent than the perturbed one. This assumption is load-bearing for the paper's central claim that the model assesses dialogue coherence without DA labels, yet no human correlation or direct validation that the perturbations systematically degrade coherence is reported; the paper explicitly defers this to future work in Section 5. Without such evidence, the reported gains could reflect shallow artifacts of the perturbation functions rather than coherence per se. Please add a human-ranking correlation study or otherwise validate the perturbation protocol, or temper the claims of practical dialogue coherence assessment.
[Abstract and Table 3] The abstract states the model 'substantially (more than 20 accuracy points) outperforms its strong competitors on the DailyDialogue corpus, and performs on par with them on the SwitchBoard corpus.' In Table 3, M-DiCoh's margin over EAGrid on DailyDialog UR is only 83.02 - 68.49 = 14.53 points, below 20. On SwitchBoard UR, M-DiCoh (58.67) is far below EAGrid (75.61) and ASeq (63.48), so 'on par' does not hold for UR. The empirical summary should be corrected to match the reported numbers.

minor comments (6)

[Table 1] There are typographical errors: 'utt2 He looks strong.' is missing a colon after 'utt2', and 'utt1::' has a double colon.
[Section C] Hyperparameter tuning was performed only on the UO problem-domain of DailyDialog, and the same hyperparameters were used for SwitchBoard without tuning. This should be stated more prominently in the main text, as it may affect the SwitchBoard comparisons.
[Table 3] Standard deviations are reported only for S-DiCoh and M-DiCoh; baselines are reported without variance. Consider reporting variance for all models if multiple runs were performed, or justify the single-run reporting for baselines.
[Figure 2] The labels 'diali' and 'dialj' in the figure are not defined in the caption; please define them as the two dialogues in the input pair.
[Section 4.6] The result that M-DAP performs on par with S-DAP is interpreted as showing coherence information does not help dialogue act prediction. This point could be stated more clearly, since the same section also emphasizes the benefits of the auxiliary task in the reverse direction.
[Table 4] The attention-weight analysis is qualitative and based on a single example. A quantitative analysis over many examples would strengthen the claim that the auxiliary task focuses attention on salient words.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: auxiliary dialogue-act prediction is independent supervision, and the pairwise coherence labels come from an external perturbation protocol rather than from the model's own outputs.

full rationale

The paper's central claim is that dialogue coherence can be assessed without DA labels at evaluation by using DA prediction as an auxiliary MTL task. This is not circular: DA labels are used only to define the auxiliary cross-entropy loss (Eq. 8), while the coherence ranking loss (Eq. 10) uses pairwise preference labels generated by the four perturbation methods in Section 4.2. Neither signal is derived from the model's predicted coherence scores; the target evaluation metric (accuracy of ranking original vs perturbed dialogues, Eq. 12) is an external protocol inherited from prior coherence work. The Section 5 statement that human-ranking correlation is future work is a validity limitation of the synthetic perturbation setup, not an indication that the prediction is fitted from the target data. The only overlapping-author citation used in the method (Gao et al. 2019 for the pairwise margin loss, Eq. 10) is not load-bearing: the loss is stated explicitly in the paper and is a standard ranking objective, and no result of that cited paper is imported as evidence for the coherence claim. Consequently, no step in the derivation chain reduces by construction to its inputs.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim depends on the validity of the synthetic perturbation protocol, the quality of DA annotations, and the chosen neural architecture. No new physical or conceptual entities are introduced.

free parameters (5)

learning rate = 0.0005
Tuned via grid search on DailyDialog UO validation set and used for all experiments.
utterance LSTM hidden size = 128
Chosen by grid search; affects the utterance representation capacity.
dialogue LSTM hidden size = 256
Chosen by grid search; affects the dialogue-level contextualization.
DA dropout rate = 0.1
Applied to utterance vectors; chosen by grid search.
initial gamma1 and gamma2 = 2.0
Trainable parameters in the weighted loss of Equation 11, initialized to 2.0 and updated during training.

assumptions (4)

domain assumption The original dialogue is always more coherent than its perturbed version in all four problem-domains.
The entire evaluation rests on this assumption, defined in Section 4.2 and used to create all training and test pairs.
domain assumption The DA labels in DailyDialog and SwitchBoard are correct and useful for learning utterance representations.
The MTL loss in Equation 8 uses these gold labels as auxiliary supervision.
domain assumption The BiLSTM plus self-attention architecture can represent utterance and dialogue coherence.
The model is built on this architecture without proof of sufficiency.
domain assumption Pre-trained GloVe embeddings provide suitable lexical semantics for the coherence task.
The utterance encoder initializes with GloVe embeddings, assuming they capture word meaning well enough for coherence.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Dialogue Coherence Assessment Without Explicit Dialogue Act Labels." pith.science (2026). https://pith.science/paper/H4HYB6H2

@misc{pith2026190808486,
  author       = {Pith},
  title        = {Pith review of: Dialogue Coherence Assessment Without Explicit Dialogue Act Labels},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H4HYB6H2}},
  note         = {Machine review of arXiv:1908.08486}
}

read the original abstract

Recent dialogue coherence models use the coherence features designed for monologue texts, e.g. nominal entities, to represent utterances and then explicitly augment them with dialogue-relevant features, e.g., dialogue act labels. It indicates two drawbacks, (a) semantics of utterances is limited to entity mentions, and (b) the performance of coherence models strongly relies on the quality of the input dialogue act labels. We address these issues by introducing a novel approach to dialogue coherence assessment. We use dialogue act prediction as an auxiliary task in a multi-task learning scenario to obtain informative utterance representations for coherence assessment. Our approach alleviates the need for explicit dialogue act labels during evaluation. The results of our experiments show that our model substantially (more than 20 accuracy points) outperforms its strong competitors on the DailyDialogue corpus, and performs on par with them on the SwitchBoard corpus for ranking dialogues concerning their coherence.

Figures

Figures reproduced from arXiv: 1908.08486 by the authors.

**Figure 2.** A low-level illustration of our MTL-based approac [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Comparing EAGrid (black bars) and M-DiCoh (white b [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: The EAGrid representations of Dialogue pre [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 25 canonical work pages

[6]

Evaluating coherence in dia- logue systems using entailment. In Proceedings of the 2019 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language T echnologies, Minneapolis, Min- nesota., 2–7 June 2019, pages 3806–3812. Y oumna Farag and Helen Y annakoudakis

work page 2019
[7]

In Proceed- ings of the 57th Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Pa- pers), Florence, Italy, 28 July – 2 August, 2019, pages 629–639

Multi- task learning for coherence modeling. In Proceed- ings of the 57th Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Pa- pers), Florence, Italy, 28 July – 2 August, 2019, pages 629–639. Sudeep Gandhe and David Traum

work page 2019
[10]

In Proceedings of the 28th International Joint Con- ference on Artiﬁcial Intelligence, Macao, China, 10–16 August 2019, pages 2350–2356

Reward learning for efﬁcient reinforcement learning in ext ractive document summarisation. In Proceedings of the 28th International Joint Con- ference on Artiﬁcial Intelligence, Macao, China, 10–16 August 2019, pages 2350–2356. Marjan Ghazvininejad, Chris Brockett, Ming Wei Chang, Bill Dolan, Jianfeng Gao, Wen Tau Yih, and Michel Galley

work page 2019
[11]

In Proceedings of the 32ed Conference on the Advancement of Artiﬁcial Intelli- gence, New Orleans, Louisiana, 2–7 February 2018, pages 5110–5117

A knowledge-grounded neu- ral conversation model. In Proceedings of the 32ed Conference on the Advancement of Artiﬁcial Intelli- gence, New Orleans, Louisiana, 2–7 February 2018, pages 5110–5117. Barbara J. Grosz and Candace L. Sidner

work page 2018
[13]

In Pro- ceedings of the International Conference on Com- puter Vision and Pattern Recognition,Salt Lake City, UT, 18–22 June 2018, pages 7482–7491

Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In Pro- ceedings of the International Conference on Com- puter Vision and Pattern Recognition,Salt Lake City, UT, 18–22 June 2018, pages 7482–7491. Y anran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu

work page 2018
[15]

In Proceedings of the 33rd Conference on the Advancement of Artiﬁcial In- telligence, Honolulu, Hawaii, 21 January –1 Febru- ary 2019, pages 6722–6729

Dialogue generation: From imitation learning to in- verse reinforcement learning. In Proceedings of the 33rd Conference on the Advancement of Artiﬁcial In- telligence, Honolulu, Hawaii, 21 January –1 Febru- ary 2019, pages 6722–6729. Chia-Wei Liu, Ryan Lowe, Iulian Serban, Mike Nose- worthy, Laurent Charlin, and Joelle Pineau

work page 2019
[16]

In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas, 1–5 November 2016, pages 2122–2132

How NOT to evaluate your dialogue system: An em- pirical study of unsupervised evaluation metrics for dialogue response generation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas, 1–5 November 2016, pages 2122–2132. Mohsen Mesgar and Michael Strube

work page 2016
[18]

In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, 31 October – 4 November 2018, pages 4328–4339

A neural local coherence model for text quality assessment. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, 31 October – 4 November 2018, pages 4328–4339. Jeffrey Pennington, Richard Socher, and Christopher Manning

work page 2018

Show all 27 references

[19]

In Proceedings of the 2014 Conference on Empiri- cal Methods in Natural Language Processing, Doha, Qatar, 25–29 October 2014, pages 1532–1543

Glove: Global vectors for word representation . In Proceedings of the 2014 Conference on Empiri- cal Methods in Natural Language Processing, Doha, Qatar, 25–29 October 2014, pages 1532–1543. C. Raymond Perrault and James F. Allen

2014
[20]

In Proceedings of the 21st International Florida Artiﬁcial Intelligence Research Society Conference, Coconut Grove, Florida, 15–17 May 2008, pages 195–200

Analyzing dialog coherence using transition patterns in le xical and semantic features . In Proceedings of the 21st International Florida Artiﬁcial Intelligence Research Society Conference, Coconut Grove, Florida, 15–17 May 2008, pages 195–200. Vipul Raheja and Joel Tetreault

2008
[21]

Dialogue act classiﬁcation with context-aware self-attention. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language T echnologies, Minneapolis, Minnesota., 2–7 June 2019, pages 3727–3733. Nils Rei...

2019
[22]

CoRR, abs/1803.09578

Why comparing single performance scores does not allow to dr aw conclusions about machine learning approaches . CoRR, abs/1803.09578. Gerard Salton

arXiv
[23]

In Proceedings of the 30th Conference on the Advancement of Artiﬁcial In- telligence, Phoenix, Arizona, 12–17 February 2016, pages 3776–3783

Building end-to-end dialogue systems using generative hierar- chical neural network models. In Proceedings of the 30th Conference on the Advancement of Artiﬁcial In- telligence, Phoenix, Arizona, 12–17 February 2016, pages 3776–3783. Dat Tien Nguyen and Shaﬁq Joty

2016
[24]

In Proceed- ings of the 55th Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers), V ancouver, Canada, 30 July – 4 August 2017, pages 1320–1330

A neural local coherence model . In Proceed- ings of the 55th Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers), V ancouver, Canada, 30 July – 4 August 2017, pages 1320–1330. Svitlana V akulenko, Maarten de Rijke, Michael Cochez, V adim S...

2017
[25]

In Proceedings of the 17th International Semantic W eb Conference,Monterey, Ca., 8-12 October 2018, pages 634–651

Measuring semantic coherence of a conversation. In Proceedings of the 17th International Semantic W eb Conference,Monterey, Ca., 8-12 October 2018, pages 634–651. Xinnuo Xu, Ondˇ rej Duˇ sek, Ioannis Konstas, and V er- ena Rieser

2018
[26]

In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, 31 October – 4 November 2018, pages 3981–3991

Better conversations by model- ing, ﬁltering, and optimizing for coherence and di- versity. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, 31 October – 4 November 2018, pages 3981–3991. Hainan Zhang, Y anyan Lan, J...

2018
[27]

S”), object (“ O

Reinforcing coherence for se- quence to sequence model in dialogue generation. In Proceedings of the 27th International Joint Confer- ence on Artiﬁcial Intelligence, Stockholm, Sweden, 13–19 July 2018, pages 4567–4573. A More Details on EAGrid EAGrid is a recent model for dial...

2018
[1998]

A pre- liminary model of centering in dialog. In Pro- ceedings of the 17th International Conference on Computational Linguistics and 36th Annual Meet- ing of the Association for Computational Linguis- tics, Montr´ eal, Qu´ ebec, Canada, 10–14 August 1998, pages 1475–1477. Ales...

1998
[2005]

In Pro- ceedings of the 43rd Annual Meeting of the Asso- ciation for Computational Linguistics, Ann Arbor, Mich., 25–30 June 2005, pages 141–148

Modeling local coherence: An entity-based approach. In Pro- ceedings of the 43rd Annual Meeting of the Asso- ciation for Computational Linguistics, Ann Arbor, Mich., 25–30 June 2005, pages 141–148. Regina Barzilay and Mirella Lapata

2005
[2008]

In Pro- ceedings of the 9th SIGdial W orkshop on Discourse and Dialogue, Columbus, Ohio, 19–20 June 2008, pages 172–181

Evaluation understudy for dialogue coherence models. In Pro- ceedings of the 9th SIGdial W orkshop on Discourse and Dialogue, Columbus, Ohio, 19–20 June 2008, pages 172–181. Sudeep Gandhe and David Traum

2008
[2010]

Using entity-based features to model coherence in student essays. In Proceedings of Human Language T echnologies 2010: The Conference of the North American Chapter of the Association for Computa- tional Linguistics, Los Angeles, Cal., 2–4 June 2010, pages 681–684. Donna K. Byr...

2010
[2013]

In Proceed- ings of the 51st Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Pa- pers), Soﬁa, Bulgaria, 4–9 August 2013, pages 93–

Graph-based local coherence modeling . In Proceed- ings of the 51st Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Pa- pers), Soﬁa, Bulgaria, 4–9 August 2013, pages 93–

2013
[2014]

In Proceedings of T extGraphs-9: Graph-based Methods for Natural Language Processing, W ork- shop at EMNLP 2014, Doha, Qatar, 29 October 2014, pages 1–5

Normalized entity graph for computing local coherence . In Proceedings of T extGraphs-9: Graph-based Methods for Natural Language Processing, W ork- shop at EMNLP 2014, Doha, Qatar, 29 October 2014, pages 1–5. Mohsen Mesgar and Michael Strube

2014
[2016]

In 7th International W orkshop on Spoken Dialogue Systems, Saariselk”a, Finland, 13–16 Jan- uary 2016, pages 141–150

A semi- automated evaluation metric for dialogue model co- herence. In 7th International W orkshop on Spoken Dialogue Systems, Saariselk”a, Finland, 13–16 Jan- uary 2016, pages 141–150. Y ang Gao, Christian M. Meyer, Mohsen Mesgar, and Iryna Gurevych

2016
[2017]

In Proceedings of the Eighth International Joint Conference on Nat- ural Language Processing (V olume 1: Long Papers), Taipei, Taiwan, 27 November – 1 December, 2017, pages 986–995

Dailydialog: A manually labelled multi-turn dialogue dataset. In Proceedings of the Eighth International Joint Conference on Nat- ural Language Processing (V olume 1: Long Papers), Taipei, Taiwan, 27 November – 1 December, 2017, pages 986–995. Ziming Li, Julia Kiseleva, and Ma...

2017
[2018]

In Proceedings of the 19th Annual Conference of the In- ternational Speech Communication Association, Hy- derabad, 2–6 September 2018, pages 1011–1015

Coherence models for dialogue . In Proceedings of the 19th Annual Conference of the In- ternational Speech Communication Association, Hy- derabad, 2–6 September 2018, pages 1011–1015. Emily Dinan, V arvara Logacheva, V alentin Malykh, Alexander H. Miller, Kurt Shuster, Jack Ur...

2018
[2019]

CoRR, abs/1902.00098

The second conversational intelligence challenge (convai 2). CoRR, abs/1902.00098. Nouha Dziri, Ehsan Kamalloo, Kory Mathewson, and Osmar Zaiane

1902 arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [6]

Evaluating coherence in dia- logue systems using entailment. In Proceedings of the 2019 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language T echnologies, Minneapolis, Min- nesota., 2–7 June 2019, pages 3806–3812. Y oumna Farag and Helen Y annakoudakis

work page 2019

[2] [7]

In Proceed- ings of the 57th Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Pa- pers), Florence, Italy, 28 July – 2 August, 2019, pages 629–639

Multi- task learning for coherence modeling. In Proceed- ings of the 57th Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Pa- pers), Florence, Italy, 28 July – 2 August, 2019, pages 629–639. Sudeep Gandhe and David Traum

work page 2019

[3] [10]

In Proceedings of the 28th International Joint Con- ference on Artiﬁcial Intelligence, Macao, China, 10–16 August 2019, pages 2350–2356

Reward learning for efﬁcient reinforcement learning in ext ractive document summarisation. In Proceedings of the 28th International Joint Con- ference on Artiﬁcial Intelligence, Macao, China, 10–16 August 2019, pages 2350–2356. Marjan Ghazvininejad, Chris Brockett, Ming Wei Chang, Bill Dolan, Jianfeng Gao, Wen Tau Yih, and Michel Galley

work page 2019

[4] [11]

In Proceedings of the 32ed Conference on the Advancement of Artiﬁcial Intelli- gence, New Orleans, Louisiana, 2–7 February 2018, pages 5110–5117

A knowledge-grounded neu- ral conversation model. In Proceedings of the 32ed Conference on the Advancement of Artiﬁcial Intelli- gence, New Orleans, Louisiana, 2–7 February 2018, pages 5110–5117. Barbara J. Grosz and Candace L. Sidner

work page 2018

[5] [13]

In Pro- ceedings of the International Conference on Com- puter Vision and Pattern Recognition,Salt Lake City, UT, 18–22 June 2018, pages 7482–7491

Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In Pro- ceedings of the International Conference on Com- puter Vision and Pattern Recognition,Salt Lake City, UT, 18–22 June 2018, pages 7482–7491. Y anran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu

work page 2018

[6] [15]

In Proceedings of the 33rd Conference on the Advancement of Artiﬁcial In- telligence, Honolulu, Hawaii, 21 January –1 Febru- ary 2019, pages 6722–6729

Dialogue generation: From imitation learning to in- verse reinforcement learning. In Proceedings of the 33rd Conference on the Advancement of Artiﬁcial In- telligence, Honolulu, Hawaii, 21 January –1 Febru- ary 2019, pages 6722–6729. Chia-Wei Liu, Ryan Lowe, Iulian Serban, Mike Nose- worthy, Laurent Charlin, and Joelle Pineau

work page 2019

[7] [16]

In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas, 1–5 November 2016, pages 2122–2132

How NOT to evaluate your dialogue system: An em- pirical study of unsupervised evaluation metrics for dialogue response generation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas, 1–5 November 2016, pages 2122–2132. Mohsen Mesgar and Michael Strube

work page 2016

[8] [18]

In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, 31 October – 4 November 2018, pages 4328–4339

A neural local coherence model for text quality assessment. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, 31 October – 4 November 2018, pages 4328–4339. Jeffrey Pennington, Richard Socher, and Christopher Manning

work page 2018