A Hierarchical Neural Autoencoder for Paragraphs and Documents
read the original abstract
Natural language generation of coherent long texts like paragraphs or longer documents is a challenging problem for recurrent networks models. In this paper, we explore an important step toward this generation task: training an LSTM (Long-short term memory) auto-encoder to preserve and reconstruct multi-sentence paragraphs. We introduce an LSTM model that hierarchically builds an embedding for a paragraph from embeddings for sentences and words, then decodes this embedding to reconstruct the original paragraph. We evaluate the reconstructed paragraph using standard metrics like ROUGE and Entity Grid, showing that neural models are able to encode texts in a way that preserve syntactic, semantic, and discourse coherence. While only a first step toward generating coherent text units from neural models, our work has the potential to significantly impact natural language generation and summarization\footnote{Code for the three models described in this paper can be found at www.stanford.edu/~jiweil/ .
This paper has not been read by Pith yet.
Forward citations
Cited by 4 Pith papers
-
Interests Burn-down Diffusion Process for Personalized Collaborative Filtering
A new interests burn-down diffusion process models decaying user interests for personalized collaborative filtering and outperforms prior generative methods in the StageCF implementation.
-
Recommender Systems with Heterogeneous Side Information
A novel framework jointly captures flat and hierarchical side information in recommender systems and shows significant performance gains over state-of-the-art methods on real-world datasets.
-
Ranking sentences from product description & bullets for better search
Two RL-based extractive summarization models rank sentences from product fields by leveraging titles and click-through logs to improve search relevance.
-
On the notion of number in humans and machines
Experiments indicate deep learning models achieve higher accuracy on numerosity tasks for counts below human subitizing capacity.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.