pith. machine review for the scientific record. sign in

arxiv: 1705.00557 · v1 · submitted 2017-04-23 · 💻 cs.CL · cs.LG· cs.NE· stat.ML

Recognition: unknown

Discourse-Based Objectives for Fast Unsupervised Sentence Representation Learning

Yacine Jernite , Samuel R. Bowman , David Sontag
Authors on Pith no claims yet
classification 💻 cs.CL cs.LGcs.NEstat.ML
keywords modelsobjectivesentencetrainunsupervisedallowingcoherencediscourse
0
0 comments X
read the original abstract

This work presents a novel objective function for the unsupervised training of neural network sentence encoders. It exploits signals from paragraph-level discourse coherence to train these models to understand text. Our objective is purely discriminative, allowing us to train models many times faster than was possible under prior methods, and it yields models which perform well in extrinsic evaluations.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

    cs.CL 2019-09 accept novelty 7.0

    ALBERT reduces BERT parameters via embedding factorization and layer sharing, adds inter-sentence coherence pretraining, and reaches SOTA on GLUE, RACE, and SQuAD with fewer parameters than BERT-large.

  2. Unsupervised Dense Information Retrieval with Contrastive Learning

    cs.IR 2021-12 unverdicted novelty 6.0

    Contrastive learning trains unsupervised dense retrievers that beat BM25 on most BEIR datasets and support cross-lingual retrieval across scripts.