Byte-Level Recursive Convolutional Auto-Encoder for Text

Xiang Zhang; Yann LeCun

RECORD

Byte-Level Recursive Convolutional Auto-Encoder for Text

Xiang Zhang, Yann LeCun

Computation and Language · Submitted 6 Feb 2018

abstract compiled · 2026-09-23 · no language-model review

The authors' main claim We show that non-sequential text generation from a fixed-length representation is not only possible, but also achieved much better auto-encoding results than recurrent networks.

how it is stated · asserted 1.00 read the record →

Comment (0) Read on arXiv ↗

Cite ▾

Claim paper

The compiled record

The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.

Open the compiled record

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 5,524 input tokens.

What the abstract itself states, as probabilities: claims first 0.05 · states limitation 0.09 · human participants 0.02 · controlled comparison 0.71 · randomized assignment 0.01 · states quantitative result 0.59 · compares with prior results 0.90 · mentions code or data availability 0.02.

The abstract, sentence by sentence

arXiv listing abstract v1. Results and the main claim are underlined; show every role.

This article proposes to auto-encode text at byte-level using convolutional networks with a recursive architecture. The motivation is to explore whether it is possible to have scalable and homogeneous text generation at byte-level in a non-sequential fashion through the simple task of auto-encoding. We show that non-sequential text generation from a fixed-length representation is not only possible, but also achieved much better auto-encoding results than recurrent networks. The proposed model is a multi-stage deep convolutional encoder-decoder framework using residual connections, containing up to 160 parameterized layers. Each encoder or decoder contains a shared group of modules that consists of either pooling or upsampling layers, making the network recursive in terms of abstraction levels in representation. Results for 6 large-scale paragraph datasets are reported, in 3 languages including Arabic, Chinese and English. Analyses are conducted to study several properties of the proposed model.

background contribution method result interpretation main claim

What the classifier reports for each sentence. The number is the probability it gave the label.
#	sentence	role	the authors' own work	states the main conclusion	strength
s0	This article proposes to auto-encode text at byte-level using convolutional networks with a recursive architecture.	contribution 0.89	0.53	0.06	asserted 0.88
s1	The motivation is to explore whether it is possible to have scalable and homogeneous text generation at byte-level in a non-sequential fashion through the simple task of auto-encoding.	background 0.54	0.08	0.04	hedged possibility 0.89
s2	We show that non-sequential text generation from a fixed-length representation is not only possible, but also achieved much better auto-encoding results than recurrent networks.	result 0.99	0.95	0.77	asserted 1.00
s3	The proposed model is a multi-stage deep convolutional encoder-decoder framework using residual connections, containing up to 160 parameterized layers.	method 0.99	0.74	0.05	asserted 1.00
s4	Each encoder or decoder contains a shared group of modules that consists of either pooling or upsampling layers, making the network recursive in terms of abstraction levels in representation.	method 1.00	0.68	0.03	asserted 1.00
s5	Results for 6 large-scale paragraph datasets are reported, in 3 languages including Arabic, Chinese and English.	method 0.61	0.92	0.03	asserted 0.95
s6	Analyses are conducted to study several properties of the proposed model.	contribution 0.80	0.84	0.02	unclear 0.53

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.88.

What was examined, and what was not

sources read

arXiv listing abstract v1

not examined

full text
figures and tables
references and citing sentences
relations to other papers

levels run

P abstract

not run

S sections
T body sentences
C citations
R cross-paper relations

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. Every number in the measurements table is the authors', checked by code. Record as JSON

The paper

Original abstract

This article proposes to auto-encode text at byte-level using convolutional networks with a recursive architecture. The motivation is to explore whether it is possible to have scalable and homogeneous text generation at byte-level in a non-sequential fashion through the simple task of auto-encoding. We show that non-sequential text generation from a fixed-length representation is not only possible, but also achieved much better auto-encoding results than recurrent networks. The proposed model is a multi-stage deep convolutional encoder-decoder framework using residual connections, containing up to 160 parameterized layers. Each encoder or decoder contains a shared group of modules that consists of either pooling or upsampling layers, making the network recursive in terms of abstraction levels in representation. Results for 6 large-scale paragraph datasets are reported, in 3 languages including Arabic, Chinese and English. Analyses are conducted to study several properties of the proposed model.

Human review

How Pith checks papers → · Full audit of this paper →

Discussion

Know something about this paper? Leave a comment, or highlight any passage above to comment on it.