RECORD
Substance over Style: Document-Level Targeted Content Transfer
Computation and Language · Submitted 16 Oct 2020
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Both automatic and human evaluations show that our model out-performs existing methods by generating coherent and diverse rewrites that obey the constraint while remaining close to the original document.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Existing language models excel at writing from scratch, but many real-world scenarios require rewriting an existing document to fit a set of constraints. Although sentence-level rewriting has been fairly well-studied, little work has addressed the challenge of rewriting an entire document coherently. In this work, we introduce the task of document-level targeted content transfer and address it in the recipe domain, with a recipe as the document and a dietary restriction (such as vegan or dairy-free) as the targeted constraint. We propose a novel model for this task based on the generative pre-trained language model (GPT-2) and train on a large number of roughly-aligned recipe pairs (https://github.com/microsoft/document-level-targeted-content-transfer). Both automatic and human evaluations show that our model out-performs existing methods by generating coherent and diverse rewrites that obey the constraint while remaining close to the original document. Finally, we analyze our model's rewrites to assess progress toward the goal of making language generation more attuned to constraints that are substantive rather than stylistic.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Existing language models excel at writing from scratch, but many real-world scenarios require rewriting an existing document to fit a set of constraints. | background 0.99 | 0.04 | 0.02 | asserted 1.00 |
| s1 | Although sentence-level rewriting has been fairly well-studied, little work has addressed the challenge of rewriting an entire document coherently. | gap 0.99 | 0.05 | 0.02 | asserted 0.99 |
| s2 | In this work, we introduce the task of document-level targeted content transfer and address it in the recipe domain, with a recipe as the document and a dietary restriction (such as vegan or dairy-free) as the targeted constraint. | contribution 1.00 | 0.88 | 0.04 | asserted 0.97 |
| s3 | We propose a novel model for this task based on the generative pre-trained language model (GPT-2) and train on a large number of roughly-aligned recipe pairs (https://github.com/microsoft/document-level-targeted-content-transfer). | method 0.96 | 0.96 | 0.04 | asserted 0.99 |
| s4 | Both automatic and human evaluations show that our model out-performs existing methods by generating coherent and diverse rewrites that obey the constraint while remaining close to the original document. | result 1.00 | 0.96 | 0.81 | asserted 1.00 |
| s5 | Finally, we analyze our model's rewrites to assess progress toward the goal of making language generation more attuned to constraints that are substantive rather than stylistic. | contribution 0.66 | 0.90 | 0.15 | unclear 0.48 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Existing language models excel at writing from scratch, but many real-world scenarios require rewriting an existing document to fit a set of constraints. Although sentence-level rewriting has been fairly well-studied, little work has addressed the challenge of rewriting an entire document coherently. In this work, we introduce the task of document-level targeted content transfer and address it in the recipe domain, with a recipe as the document and a dietary restriction (such as vegan or dairy-free) as the targeted constraint. We propose a novel model for this task based on the generative pre-trained language model (GPT-2) and train on a large number of roughly-aligned recipe pairs (https://github.com/microsoft/document-level-targeted-content-transfer). Both automatic and human evaluations show that our model out-performs existing methods by generating coherent and diverse rewrites that obey the constraint while remaining close to the original document. Finally, we analyze our model's rewrites to assess progress toward the goal of making language generation more attuned to constraints that are substantive rather than stylistic.
Human review
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites · 10 sentences without a confident label
Contests 1 paper
-
Prior work on controlled generation guides the output of a model using attribute classifiers [ref] or control codes [ref] , but we find that these models do not perform well on our transfer task (<ref>).
CTRL: A Conditional Transformer Language Model for Controllable Generation · Introduction · contest 0.87
Uses the method or tool of 3 papers
-
[ref] propose a method for automatically aligning two recipes of the same dish.
A Recipe for Creating Multimodal Aligned Datasets for Sequential Tasks · Create Recipe and Step Pairs · 0.91
-
We build a PPLM model for our task using a GPT-2 model fine-tuned on $∼$1.2 million recipes (<ref>) as the pre-trained language model and using separate bag-of-words classifiers for each of our dietary constraints.[See appendix for PPLM implementation details.] CTRL: The conditional transformer language model [ref] uses a `control' code to govern the style and content of the generated text.
CTRL: A Conditional Transformer Language Model for Controllable Generation · Document-level Controllable Baselines · 0.81
-
Diversity: Since generation models can produce results that are bland and repetitive, we measure the diversity of the generated recipes in terms of the proportion of unique trigrams [ref] .
A Diversity-Promoting Objective Function for Neural Conversation Models · Automatic Metrics · 0.92
Cites as support 1 paper
-
This strategy has proven effective in other domains, including automatic storytelling, where prompting a model with a rough “storyline" helps models stay on-topic [ref] .
Plan-And-Write: Towards Better Automatic Storytelling · Contextual Rewriter + Ingredient Prompt · 0.97
Cites as background 4 papers
-
Recipe generation: Recipe generation has been a research focus for decades, using methods ranging from rule-based planning systems [ref] to more recent neural network models that use targeted information such as entity types [ref] , cooking actions [ref] , ingredients [ref] , or order information [ref] to guide the generations.
Simulating Action Dynamics with Neural Process Networks · Related Work · 0.97
-
Text attribute transfer: Most work in text attribute transfer has been at the sentence level, including sentiment [ref] , formality [ref] , tense [ref] , and authorship [ref] .
Toward Controlled Generation of Text · Related Work · 0.99
-
Text attribute transfer: Most work in text attribute transfer has been at the sentence level, including sentiment [ref] , formality [ref] , tense [ref] , and authorship [ref] .
Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer · Related Work · 0.99
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.