Faster Transformer Decoding: N-gram Masked Self-Attention

Ankur Bapna; Ciprian Chelba; Mia Chen; Noam Shazeer

Faster Transformer Decoding: N-gram Masked Self-Attention

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2001.04589 v2 pith:YH4RGUPK submitted 2020-01-14 cs.LG cs.CLstat.ML

Ciprian Chelba , Mia Chen , Ankur Bapna , Noam Shazeer This is my paper

classification cs.LGcs.CLstat.ML

keywords self-attentiongramldotsmaskedassumptionbleucomputingdata

verification ladder T0 review T1 audit T2 compute T3 formal

0 comments

abstract

Motivated by the fact that most of the information relevant to the prediction of target tokens is drawn from the source sentence $S=s_1, \ldots, s_S$, we propose truncating the target-side window used for computing self-attention by making an $N$-gram assumption. Experiments on WMT EnDe and EnFr data sets show that the $N$-gram masked self-attention model loses very little in BLEU score for $N$ values in the range $4, \ldots, 8$, depending on the task.

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Chat-Ghosting: A Comparative Study of Methods for Auto-Completion in Dialog Systems
cs.CL 2025-07 conditional novelty 5.0 of 10

Simple tries and n-gram models beat large neural models for chat autocompletion on seen prefixes, while fine-tuned transformers and conversational context lead on unseen ones.

Pith tools