Pith. sign in

REVIEW 4 cited by

Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2107.03451 v3 pith:ZXKE7XTB submitted 2021-07-07 cs.CL cs.AI

Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling

Emily Dinan , Gavin Abercrombie , A. Stevie Bergman , Shannon Spruit , Dirk Hovy , Y-Lan Boureau , Verena Rieser This is my paper
classification cs.CL cs.AI
keywords conversationalmodelsend-to-enddecisionsframeworkpotentialreleaseresearchers
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
read the original abstract

Over the last several years, end-to-end neural conversational agents have vastly improved in their ability to carry a chit-chat conversation with humans. However, these models are often trained on large datasets from the internet, and as a result, may learn undesirable behaviors from this data, such as toxic or otherwise harmful language. Researchers must thus wrestle with the issue of how and when to release these models. In this paper, we survey the problem landscape for safety for end-to-end conversational AI and discuss recent and related work. We highlight tensions between values, potential positive impact and potential harms, and provide a framework for making decisions about whether and how to release these models, following the tenets of value-sensitive design. We additionally provide a suite of tools to enable researchers to make better-informed decisions about training and releasing end-to-end conversational AI models.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned

    cs.CL 2022-08 accept novelty 6.0

    RLHF-aligned language models show increasing resistance to red teaming with scale up to 52B parameters, unlike prompted or rejection-sampled models, supported by a released dataset of 38,961 attacks.

  2. LaMDA: Language Models for Dialog Applications

    cs.CL 2022-01 unverdicted novelty 6.0

    LaMDA shows that fine-tuning on human-value annotations and consulting external knowledge sources significantly improves safety and factual grounding in large dialog models beyond what scaling alone achieves.

  3. Ethical and social risks of harm from Language Models

    cs.CL 2021-12 accept novelty 6.0

    The authors provide a detailed taxonomy of 21 risks associated with language models, covering discrimination, information leaks, misinformation, malicious applications, interaction harms, and societal impacts like job...

  4. Safe Inference-Time Alignment via Lagrangian Reward Augmentation

    cs.LG 2026-07 conditional novelty 5.5

    Dualizing Safe RLHF yields a one-dimensional convex calibration of λ that defines a drop-in safety-aware reward for Best-of-N and token-level inference-time decoders.