Pith. sign in

REVIEW 6 cited by

DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2009.13570 v2 pith:ABVP7BBP submitted 2020-09-28 cs.CL cs.AI

Shikib Mehri , Mihail Eric , Dilek Hakkani-Tur This is my paper · ORCID
classification cs.CLcs.AI
keywords dialoguetask-orientedbenchmarkdialogluelanguagemodelsresearchunderstanding
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

A long-standing goal of task-oriented dialogue research is the ability to flexibly adapt dialogue models to new domains. To progress research in this direction, we introduce DialoGLUE (Dialogue Language Understanding Evaluation), a public benchmark consisting of 7 task-oriented dialogue datasets covering 4 distinct natural language understanding tasks, designed to encourage dialogue research in representation-based transfer, domain adaptation, and sample-efficient task learning. We release several strong baseline models, demonstrating performance improvements over a vanilla BERT architecture and state-of-the-art results on 5 out of 7 tasks, by pre-training on a large open-domain dialogue corpus and task-adaptive self-supervised training. Through the DialoGLUE benchmark, the baseline methods, and our evaluation scripts, we hope to facilitate progress towards the goal of developing more general task-oriented dialogue models.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Few-Shot Query Intent Detection via Relation-Aware Prompt Learning

    cs.CL 2025-09 conditional novelty 6.0 of 10

    SAID pretrains language models with query-query and query-answer relation-aware soft prompts, then transfers them via intent-specific prompts for few-shot intent detection, reporting up to 27% relative accuracy gains.

  2. Dynamic Label Name Refinement for Few-Shot Dialogue Intent Classification

    cs.CL 2024-12 conditional novelty 6.0 of 10

    Dynamic, per-query rewriting of intent labels by an LLM improves few-shot dialogue intent classification accuracy compared to retrieval ICL with original labels.

  3. MORTAR: Multi-turn Metamorphic Testing for LLM-based Dialogue Systems

    cs.SE 2024-12 conditional novelty 6.0 of 10

    MORTAR uses dialogue-level perturbations and metamorphic relations to reveal context-dependent bugs in multi-turn LLM dialogue systems without an LLM judge.

  4. Chain-of-Thought Training for Open E2E Spoken Dialogue Systems

    cs.CL 2025-05 conditional novelty 5.0 of 10

    Staged ASR-to-text-response-to-TTS training makes open end-to-end spoken dialogue systems trainable on 300 hours of public human-human data and more coherent than one-step speech-to-speech models.

  5. Know Your Mistakes: Towards Preventing Overreliance on Task-Oriented Conversational AI Through Accountability Modeling

    cs.CL 2025-01 conditional novelty 5.0 of 10

    Adding a slot-presence classifier head to generative LLM dialogue state trackers improves joint goal accuracy by about 3% absolute and allows detected errors to be self-corrected or confirmed by the user.

  6. Building Task Bots with Self-learning for Enhanced Adaptability, Extensibility, and Factuality

    cs.CL 2025-08 conditional novelty 2.0 of 10

    A thesis that combines self-learning from dialog logs, schema-guided prompting, and self-aligned factuality to build task bots with minimal human intervention.

Pith tools