Autoregressive Text Generation Beyond Feedback Loops

Florian Schmidt; Stephan Mandt; Thomas Hofmann

REVIEW 3 major objections 8 minor 46 references

Autoregressive Text Generation Beyond Feedback Loops

T0 review · 3 major / 8 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Autoregressive text generation can move out of the hidden state and into a globally normalized CRF observation layer.

desk verdict Interesting latent-CRF idea with a real correctness gap in the low-rank parameterization and experiments too thin to support the abstract's claim. read the letter →

arxiv 1908.11658 v1 pith:BHSGPVCX submitted 2019-08-30 cs.LG cs.CLstat.ML

Florian Schmidt , Stephan Mandt , Thomas Hofmann This is my paper · ORCID

classification cs.LGcs.CLstat.ML

keywords exposurebiasteacherforcingconditionalrandomfieldsstatespacemodelsunconditionaltextgenerationvariationalinferenceglobalnormalizationwordembeddings

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the autoregressive feedback loop, in which the previously predicted token is fed into the next hidden state, is not necessary for coherent text generation and is the source of exposure bias. It proposes a latent state space model whose hidden-state transitions stay Markovian and non-autoregressive, paired with a conditional random field (CRF) observation model that scores whole word sequences with pairwise potentials between neighboring words. Because the observation model is globally normalized, local word correlations are expressed without corrupting the state trajectory with training-time ground-truth tokens. On unconditional sentence generation from a simplified SNLI text corpus, the model reports lower external n-gram perplexity than a deterministic RNN baseline, much lower bigram perplexity than a unary state space model, and repetition rates closer to held-out data than either, while avoiding the short, repetitive failure modes observed in the GAN baseline.

What carries the argument

The load-bearing object is the CRF observation model with energy S(w; h) = Σ_t ψ(w_t; h_t) + ψ(w_{t−1}, w_t; h_{t−1:t}), globally normalized as p(w|h) = exp S(w,h) / Σ_{w'} exp S(w',h). Its pairwise potential matrix T_{ij} = exp ψ(i,j) is factorized into X^T S(h_{t−1}, h_t) Y, where X, Y are context-independent word-embedding matrices and S is a small context-dependent matrix produced by a neural network (diagonal in the reported best configuration). This factorization does two jobs: it keeps memory at O(d|V|) and the forward/backward computation of Z at O(d|V|T), and it injects pre-trained embedding geometry into the pairwise score. The same backward messages used to compute the normalizer also supply the ancestral-sampling conditionals p(w_t | w_{1:t−1}, h), so a future-looking factor β_{t+1}(w_t) implicitly marginalizes all future words when judging the current word. That future-looking factor is what distinguishes the globally normalized model from locally normalized softmax chains: each local decision is assessed against the full set of possible completions.

What would settle it

Train the same model on a corpus where adjacent-word statistics are deliberately made uninformative (for example, by shuffling words inside small windows while preserving vocabulary and length), and measure external n-gram perplexity and repetition against a matched-size RNN; if the CRF model still clearly wins, the gains come from something other than its pairwise observation potentials, and if it loses, the reported advantage is driven by exactly the bigram correlations the model was designed to express.

Watch

Extended reading notes

Core claim

The central discovery is that autoregressive prediction and autoregressive hidden-state evolution can be separated: a model can generate word sequences autoregressively through a globally normalized CRF observation model while keeping the latent state trajectory free of feedback from previous outputs. Writing S(w; h) = sum_t ψ(w_t; h_t) + ψ(w_{t−1}, w_t; h_{t−1:t}) and defining p(w|h) = exp S / Z makes the unary softmax model a special case and adds explicit bigram correlations whose partition function Z can still be computed exactly by dynamic programming. The paper derives ancestral sampling for p(w|h) from backward probabilities β_t(w_{t−1}) = sum_{w_t} exp(ψ(w_{t−1},w_t)+ψ(w_t)) β_{t+1}(w_t), and shows that the pairwise matrix can be factorized as T = X^T S(h_{t−1}, h_t) Y with context-independent word embeddings and a learned d×d interaction matrix—diagonal in the best experiments—making training cost O(d|V|T), comparable to softmax logits. Using variational inference with a backward-RNN inference model, the resulting model outperforms a matched-size deterministic RNN on bigram perplexity, matches it on trigram perplexity, and achieves token-repetition statistics much closer to the held-out oracle, supporting the paper's claim that the feedback loop is not the essential ingredient of good unconditional generation.

Load-bearing premise

The approach hinges on the assumption that words' local coherence can be captured by a low-rank, effectively diagonal pairwise potential between adjacent words; if real corpora require longer-range or denser interactions, the model collapses toward a unary state space model and its advantage over the RNN baseline disappears.

Editorial extensions

If this is right

Exposure bias in the hidden-state trajectory disappears by construction: during training, ground-truth tokens enter only the globally normalized observation likelihood, never the transition distribution p(h_t | h_{t−1}).
Training the CRF normalizer implicitly scores all |V|^T possible sequences at every step, so the model sees negative evidence from plausible but wrong continuations rather than only from sampled or gold tokens.
Generation becomes a two-stage ancestral procedure: draw the latent trajectory from the Markovian prior, then decode words one by one with conditionals built from the CRF's backward messages, with no need for scheduled sampling or adversarial corrections.
At matched hidden-state size, the model beats a deterministic RNN on bigram perplexity and matches it on trigram perplexity while producing repetition rates much closer to held-out text, and it does not exhibit the length collapse and diversity loss seen in the GAN baseline after longer training.
Because the pairwise matrix is factorized through word embeddings, the same CRF observation layer can be attached to any latent sequential model or used as a drop-in replacement for a softmax output layer, inheriting its global-normalization benefits.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the central claim generalizes, exposure-bias remedies such as scheduled sampling or professor forcing may be treating a symptom: the bias lives in the hidden-state feedback loop, and a globally normalized observation layer removes the loop rather than patching the discrepancy; the natural next test is conditional generation, where the paper's unconditional-only evidence leaves a gap.
The diagonal-S result hints that the observed gains come mostly from suppressing length-2 repetition and smoothing bigram transitions rather than from long-range syntax; a direct ablation varying the rank of S and the order of the CRF would separate these effects.
The β_{t+1}(w_t) term in the sampling conditionals is a cheap form of look-ahead: each next-word choice is scored against all possible futures through the backward message, a mechanism that could be transferred to ordinary softmax decoders as a test-time correction or used as a tractable proposal distribution for importance sampling.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Dear colleague,

The punchline: this paper has a genuinely interesting idea—put word-level correlations in a globally normalized CRF observation model instead of in the autoregressive state feedback—but as written, Section 3.3 has an internal inconsistency that undermines the model's validity, and the experiments don't support the abstract's 'performance improvements' claim.

What's new and good: the combination of a latent state space model with a pairwise CRF observation model, the ancestral sampling derivation from backward probabilities (Eqs. 6–7), and the low-rank embedding factorization of the pairwise potentials is not in the cited work. The exposition of why global normalization avoids exposure bias in the hidden state trajectory is clear and thoughtful. The authors make code and 100K generated sentences available, which is reproducible evidence that should be credited.

The main soft spot is the one from the stress-test. Eq. (8) defines T as the element-wise exponential of the score matrix, so T is strictly positive. Then Section 3.3 states T = X^T S(h_{t-1}, h_t) Y. With X and Y initialized from GloVe, negative entries are common, so X^T S Y can have negative entries even if S is positive. The sentence about 'strictly positive activation functions around the layers in (8)' doesn't specify which layers; positive S alone doesn't guarantee positivity of the product. If the intended construction is T = exp(X^T S Y), then the O(d|V|T) complexity claim is unjustified because exponentiating a low-rank matrix generally yields a dense matrix, so the beta recursion would cost O(|V|^2 T). Either way, the section doesn't define a valid tractable CRF observation model. This is load-bearing, not a typo.

The second issue is the experiments. Table 2 reports a single run, no error bars, no significance tests. The RNN baseline uses a 16-dimensional state; the proposed model is on par with it on trigram perplexity and only modestly better on bigram perplexity. The GAN comparison is mixed: SeqGAN-20E achieves substantially lower perplexity (22.4 vs 40.1) even though it scores worse on diversity and length. The abstract's statement that the model shows 'performance improvements compared to RNN and GAN baselines' is therefore overstated. A few seeds and confidence intervals would clarify whether the differences are real.

The core idea is worth preserving. A serious referee should engage with this paper, not desk-reject it. But the authors need to fix the parameterization (e.g., explicitly use exp on the product and show how to keep it low-rank, or constrain X and Y to be positive) and re-run the experiments with proper statistics.

I'd bring it to reading group, because the flaw is instructive. I would not cite it as a valid method until the correctness issue is resolved.

Best,

Referee Report

3 major / 8 minor

Summary. The paper proposes a latent state space model for unconditional text generation in which the observation model is a conditional random field (CRF) with pairwise potentials between consecutive words, instead of an autoregressive softmax. This removes feedback of predicted words into the hidden state, thereby addressing exposure bias. The authors derive an efficient backward recursion for the CRF normalizer and an ancestral sampling scheme, factorize the pairwise transition matrix as T = X^T S Y using pretrained GloVe embeddings, and train the model via variational inference. Experiments on an SNLI-derived corpus compare against RNN, GAN, and non-autoregressive baselines, reporting perplexity, repetition, length, and diversity metrics.

Significance. The conceptual contribution is interesting: it offers a middle ground between locally normalized autoregressive models and fully non-autoregressive models, with tractable exact global normalization for pairwise potentials. If the parameterization issue is fixed, the approach could be a useful alternative for low-complexity text generation. The paper also provides a derivation of sampling from the CRF and releases code and generated data, which aids reproducibility. However, the current technical inconsistency in the factorization and the thin experimental evidence prevent acceptance in the present form.

major comments (3)

[Section 3.3, Eq. (9)] The paper defines T = exp A element-wise (Eq. 8) and then factorizes T as T = X^T S(h_{t-1}, h_t) Y (Eq. 9). This is internally inconsistent: for a valid CRF, T must be strictly positive element-wise, but X and Y are initialized with GloVe embeddings, which contain negative entries, and a strictly positive S does not guarantee positivity of X^T S Y. The sentence 'Since A does not have more structure than being strictly positive element-wise, it is sufficient to use strictly positive activation functions around the layers in (8) to obtain a valid factorization' is also unclear, since A = log T can take any real value. If the intended construction is T = exp(X^T S Y), then the complexity claim O(d|V|T) is unjustified, because exponentiating a low-rank matrix generally yields a dense |V|×|V| matrix, and the matrix-vector product in Eq. (8) cannot be computed by associativity alone. The authors must specify the parameterization precisely, prove positivity, and either justify the complexity or revise it.
[Section 4.2, Table 2] All quantitative results are from a single run, with no error bars or significance tests. The claimed improvement over DRNN is based on bigram perplexity (40.1 vs. 47.1), while trigram perplexity is essentially tied (41.9 vs. 43.5). Since the generative models are stochastic and training involves variational inference with random initializations, the observed differences could be within run-to-run variation. The repetition and diversity metrics likewise lack uncertainty quantification. The authors should report multiple seeds with standard errors and, if possible, a significance test (e.g., bootstrap over generated samples), or temper the claims in Section 6 accordingly.
[Section 6 (Conclusion) and Section 5 (Discussion)] The conclusion states 'we have shown an alternative methodology...' and 'we obtain better results than a deterministic RNN in a low-dimensional setting,' but the empirical support is limited to a single corpus (SNLI) of relatively simple sentences, and the comparison with DRNN is not statistically validated. This weakens the central claim that the proposed model offers a generally better methodology. The authors should either provide additional datasets/experiments or frame the conclusions as preliminary findings.

minor comments (8)

[Section 3.3] 'Sine' should be 'Since' in the sentence about strictly positive activation functions.
[Section 6] 'parameteriation' should be 'parameterization'.
[Section 3.2] 'requried' should be 'required', and 'amendable' should be 'amenable'.
[Section 4.2] 'adress' should be 'address'.
[Equations (5) and (7)] The notation for potentials is inconsistent: Eq. (5) uses ψ(w_t; h_t) and ψ(w_{t-1}, w_t; h_{t-1:t}), while Eq. (7) omits the dependence on h. Please make the notation uniform.
[References] The citation 'M. Ziegler and M. Rush' in the introduction and related work should be consistent with the bibliography entry (Zachary M. Ziegler and Alexander M. Rush).
[Appendix A] In the training excerpt, 'a cat unk at a sheep' renders the unknown-word token as 'unk'; consider using <unk> for clarity.
[Appendix (Training details)] The paper does not state the number of training epochs, learning rate, or other hyperparameters for the proposed model; please add these to the appendix.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the CRF observation model is trained on the ELBO and evaluated with external n-gram perplexity; the only self-citations are non-load-bearing.

full rationale

I found no circularity in this paper. The central derivation is self-contained: the generative model is trained by maximizing the ELBO in Eq. (11), where the reconstruction term log p(w|h) is the globally normalized CRF likelihood with exact normalizer Z computed via the backward recursion of Eqs. (6)-(8); none of the evaluation statistics (n-gram perplexity, repetition, length, diversity) enter the training objective or the parameterization. The sampling beta factors are algebraic quantities derived from the model, not fitted constants, and the low-rank transition factorization of Eq. (9) is a modeling choice rather than a prediction equivalent to its inputs. The external Kneser-Ney language models used for evaluation are estimated on the training corpus, but they are not used to train or select the generative model, so this is an external benchmark. The only self-citations, Schmidt and Hofmann (2018) and M. Ziegler and M. Rush (2019), are used to position the unary-only SSM ablation and to remark on a known issue with latent variable models; they do not supply a load-bearing theorem, a uniqueness argument, or a fitted parameter. Concerns about the positivity or expressiveness of the low-rank factorization are correctness or modeling risks, not circular reductions.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The central claims rest on the CRF observation model and the low-rank factorization. No invented entities appear. The main caveats are the modeling assumptions listed above, especially the diagonal restriction on S and the fixed variational posterior.

free parameters (4)

latent state dimension = 16
Chosen by hand; the paper reports that larger latent dimensions do not help, unlike RNN hidden sizes.
word embedding dimension = 100
Pretrained GloVe embeddings of dimension 100 are used for X and Y; larger embeddings improved performance somewhat.
pairwise interaction matrix S structure = diagonal
The paper found a diagonal matrix to perform best, which further restricts the low-rank factorization.
maximum sentence length = 15 tokens
Data preprocessing limits sentences to 15 tokens, excluding 9% of the SNLI-derived corpus and simplifying the generation task.

assumptions (6)

domain assumption The observation model is a linear-chain CRF over words: p(w|h) proportional to exp S(w;h) with S containing unary and pairwise potentials over consecutive words (Eq. 5).
This structure is chosen to make the global normalizer computable by dynamic programming; it limits explicit correlations to adjacent words.
domain assumption Latent states follow a first-order Markov chain p(h) = product over t of p(h_t | h_{t-1}) with no dependence on previous observations (Eq. 2).
This is the core design decision that removes autoregressive feedback from the state evolution.
domain assumption The variational posterior factorizes as q(h|w) = product over t of q(h_t | h_{t-1}, w_{t:T}) driven by a backward RNN (Eq. 12).
The paper does not assess the tightness of this approximation, so a poor posterior could weaken the ELBO training signal.
ad hoc to paper The pairwise potential matrix T is factorized as X^T S(h_{t-1}, h_t) Y with strictly positive activations and pretrained GloVe embeddings (Eq. 9).
This factorization is introduced to reduce complexity from O(|V|^2) to O(d|V|), but it restricts the family of pairwise potentials; in practice S is set to a diagonal matrix.
standard math The CRF normalizer and sampling probabilities are computed with the backward recursion beta_t (Eq. 7), a standard chain-CRF algorithm.
The recursion is exact for linear-chain structures and is used both for training and for ancestral sampling.
standard math Stochastic gradients are obtained via the reparameterization trick for Gaussian latent variables.
This is a standard technique for training deep latent Gaussian models.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Autoregressive Text Generation Beyond Feedback Loops." pith.science (2026). https://pith.science/paper/BHSGPVCX

@misc{pith2026190811658,
  author       = {Pith},
  title        = {Pith review of: Autoregressive Text Generation Beyond Feedback Loops},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BHSGPVCX}},
  note         = {Machine review of arXiv:1908.11658}
}

read the original abstract

Autoregressive state transitions, where predictions are conditioned on past predictions, are the predominant choice for both deterministic and stochastic sequential models. However, autoregressive feedback exposes the evolution of the hidden state trajectory to potential biases from well-known train-test discrepancies. In this paper, we combine a latent state space model with a CRF observation model. We argue that such autoregressive observation models form an interesting middle ground that expresses local correlations on the word level but keeps the state evolution non-autoregressive. On unconditional sentence generation we show performance improvements compared to RNN and GAN baselines while avoiding some prototypical failure modes of autoregressive models.

Figures

Figures reproduced from arXiv: 1908.11658 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 17 canonical work pages

[1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Daniel Andor, Chris Alberti, David Weiss, Aliaksei Severyn, Alessandro Presta, Kuzman Ganchev, Slav Petrov, and Michael Collins. 2016. http://arxiv.org/abs/1603.06042 Globally normalized transition-based neural networks . CoRR, abs/1603.06042

arXiv 2016
[4]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. http://arxiv.org/abs/1409.0473 Neural machine translation by jointly learning to align and translate . In ICLR

arXiv 2015
[5]

David Belanger, Bishan Yang, and Andrew McCallum. 2017. http://proceedings.mlr.press/v70/belanger17a.html End-to-end learning for structured prediction energy networks . In ICML

work page 2017
[6]

Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. 2015. http://arxiv.org/abs/1506.03099 Scheduled sampling for sequence prediction with recurrent neural networks . In NIPS

arXiv 2015
[7]

David M Blei, Alp Kucukelbir, and Jon D McAuliffe. 2017. https://arxiv.org/abs/1601.00670 Variational inference: A review for statisticians . Journal of the American Statistical Association

arXiv 2017
[8]

Samuel R.\ Bowman, Gabor Angeli, Christopher Potts, and Christopher D.\ Manning. 2015. http://nlp.stanford.edu/pubs/snli_paper.pdf A large annotated corpus for learning natural language inference . In EMNLP

work page 2015

Show all 46 references

[9]

Bowman, Luke Vilnis, Oriol Vinyals, Andrew M

Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew M. Dai, Rafal J \' o zefowicz, and Samy Bengio. 2016. http://arxiv.org/abs/1511.06349 Generating sentences from a continuous space . In ACL

2016 arXiv
[10]

Massimo Caccia, Lucas Caccia, William Fedus, Hugo Larochelle, Joelle Pineau, and Laurent Charlin. 2018. http://arxiv.org/abs/1811.02549 Language gans falling short . CoRR, abs/1811.02549

2018 arXiv
[11]

Justin Domke. 2013. http://arxiv.org/abs/1301.3193 Learning graphical model parameters with approximate marginal inference . In IEEE Transactions on Pattern Analysis and Machine Intelligence

2013 arXiv
[12]

Goodfellow, and Andrew M

William Fedus, Ian J. Goodfellow, and Andrew M. Dai. 2018. http://arxiv.org/abs/1801.07736 Maskgan: Better text generation via filling in the \_ \_ \_ \_ \_ \_ . In ICLR

2018 arXiv
[13]

Marco Fraccaro, S ren Kaae S nderby, Ulrich Paquet, and Ole Winther. 2016. http://papers.nips.cc/paper/6039-sequential-neural-models-with-stochastic-layers.pdf Sequential neural models with stochastic layers . In NIPS

2016
[14]

Eran Goldman and Jacob Goldberger. 2017. http://arxiv.org/abs/1705.07420 Structured image classification from conditional random field with deep class embedding . arXiv preprint arXiv:1705.07420

2017 arXiv
[15]

Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. http://www.deeplearningbook.org Deep Learning . MIT Press

2016
[16]

Courville, and Yoshua Bengio

Anirudh Goyal, Alex Lamb, Ying Zhang, Saizheng Zhang, Aaron C. Courville, and Yoshua Bengio. 2016. http://papers.nips.cc/paper/6099-professor-forcing-a-new-algorithm-for-training-recurrent-networks Professor forcing: A new algorithm for training recurrent networks . In NIPS

2016
[17]

Anirudh Goyal, Alessandro Sordoni, Marc - Alexandre C \^ o t \' e , Nan Rosemary Ke, and Yoshua Bengio. 2017. http://papers.nips.cc/paper/7248-z-forcing-training-stochastic-recurrent-networks Z-forcing: Training stochastic recurrent networks . In NIPS

2017
[18]

Alex Graves. 2013. https://arxiv.org/pdf/1308.0850 Generating sequences with recurrent neural networks . arXiv preprint arXiv:1308.0850

2013 arXiv
[19]

Zhiheng Huang, Wei Xu, and Kai Yu. 2017. http://arxiv.org/abs/1508.01991 Bidirectional LSTM-CRF models for sequence tagging . In First Workshop on Subword and Character Level Models in NLP

2017 arXiv
[20]

Yoon Kim, Carl Denton, Luong Hoang, and Alexander M. Rush. 2017. http://arxiv.org/abs/1702.00887 Structured attention networks . In ICLR

2017 arXiv
[21]

Kingma and Jimmy Ba

Diederik P. Kingma and Jimmy Ba. 2014. http://arxiv.org/abs/1412.6980 Adam: A method for stochastic optimization . In ICLR

2014 arXiv
[22]

Kingma, Tim Salimans, and Max Welling

Diederik P. Kingma, Tim Salimans, and Max Welling. 2016. http://arxiv.org/abs/1606.04934 Improving variational inference with inverse autoregressive flow . In NIPS

2016 arXiv
[23]

a henb \

Philipp Kr \" a henb \" u hl and Vladlen Koltun. 2012. http://arxiv.org/abs/1210.5644 Efficient inference in fully connected crfs with gaussian edge potentials . In NIPS

2012 arXiv
[24]

R \' e mi Leblond, Jean - Baptiste Alayrac, Anton Osokin, and Simon Lacoste - Julien. 2018. http://arxiv.org/abs/1706.04499 SEARNN: training rnns with global-local losses . In ICLR

2018 arXiv
[25]

Jason Lee, Elman Mansimov, and Kyunghyun Cho. 2018. http://arxiv.org/abs/1802.06901 Deterministic non-autoregressive neural sequence modeling by iterative refinement . In EMNLP

2018 arXiv
[26]

Ziegler and Alexander M

Zachary M. Ziegler and Alexander M. Rush. 2019. https://arxiv.org/pdf/1901.10548 Latent normalizing flows for discrete sequences . arXiv preprint arXiv:1901.10548

2019 arXiv
[27]

Xuezhe Ma and Eduard H. Hovy. 2016. http://arxiv.org/abs/1603.01354 End-to-end sequence labeling via bi-directional lstm-cnns-crf . In ACL

2016 arXiv
[28]

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. http://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf Distributed representations of words and phrases and their compositionality . In NIPS

2013
[29]

Thanapon Noraset, David Demeter, and Doug Downey. 2018. https://aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16961 Controlling global statistics in recurrent neural network text generation

2018
[30]

Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. https://nlp.stanford.edu/pubs/glove.pdf Glove: Global vectors for word representation . In EMNLP

2014
[31]

Marc'Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. 2016. https://arxiv.org/pdf/1511.06732 Sequence level training with recurrent neural networks . In ICLR

2016 arXiv
[32]

Rennie, Etienne Marcheret, Youssef Mroueh, Jerret Ross, and Vaibhava Goel

Steven J. Rennie, Etienne Marcheret, Youssef Mroueh, Jerret Ross, and Vaibhava Goel. 2016. http://arxiv.org/abs/1612.00563 Self-critical sequence training for image captioning . CoRR, abs/1612.00563

2016 arXiv
[33]

Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. 2014. http://arxiv.org/abs/1401.4082 Stochastic back-propagation and variational inference in deep latent gaussian models . In ICML

2014 arXiv
[34]

Rush, Sumit Chopra, and Jason Weston

Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015. http://arxiv.org/abs/1509.00685 A neural attention model for abstractive sentence summarization . In EMNLP

2015 arXiv
[35]

Florian Schmidt and Thomas Hofmann. 2018. http://papers.nips.cc/paper/7854-deep-state-space-models-for-unconditional-word-generation.pdf Deep state space models for unconditional word generation . In NeurIPS

2018
[36]

Stanislau Semeniuta, Aliaksei Severyn, and Sylvain Gelly. 2018. http://arxiv.org/abs/1806.04936 On accurate evaluation of gans for language generation . In ICML workshop on Theoretical Foundations and Applications of Deep Generative Models

2018 arXiv
[37]

Iulian Vlad Serban, Alessandro Sordoni, Yoshua Bengio, Aaron C Courville, and Joelle Pineau. 2016. https://arxiv.org/pdf/1507.04808 Building end-to-end dialogue systems using generative hierarchical neural network models. In AAAI

2016 arXiv
[38]

Fei Sha and Fernando Pereira. 2003. http://aclweb.org/anthology/N03-1028 Shallow parsing with conditional random fields . In NAACL

2003
[39]

Andreas Stolcke. 2002. https://www.sri.com/work/publications/srilm-extensible-language-modeling-toolkit Srilm -- an extensible language modeling toolkit . In ICSLP

2002
[40]

Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. http://dl.acm.org/citation.cfm?id=2969033.2969173 Sequence to sequence learning with neural networks . In NIPS

2014
[41]

Zhaopeng Tu, Yang Liu, Lifeng Shang, Xiaohua Liu, and Hang Li. 2016. http://arxiv.org/abs/1611.01874 Neural machine translation with reconstruction . CoRR, abs/1611.01874

2016 arXiv
[42]

Gomez, Lukasz Kaiser, and Illia Polosukhin

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. http://arxiv.org/abs/1706.03762 Attention is all you need . In NIPS

2017 arXiv
[43]

Sam Wiseman and Alexander M. Rush. 2016. http://arxiv.org/abs/1606.02960 Sequence-to-sequence learning as beam-search optimization . In EMNLP

2016 arXiv
[44]

Jingjing Xu, Xu Sun, Xuancheng Ren, Junyang Lin, Bingzhen Wei, and Wei Li. 2018. http://arxiv.org/abs/1802.01345 DP-GAN: diversity-promoting generative adversarial network for generating informative and diversified text . In EMNLP

2018 arXiv
[45]

Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. 2017. http://arxiv.org/abs/1609.05473 Seqgan: Sequence generative adversarial nets with policy gradient . In AAAI

2017 arXiv
[46]

Cheng Zhang, Judith Butepage, Hedvig Kjellstrom, and Stephan Mandt. 2018. https://arxiv.org/abs/1711.05597 Advances in variational inference . IEEE transactions on pattern analysis and machine intelligence

2018 arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Daniel Andor, Chris Alberti, David Weiss, Aliaksei Severyn, Alessandro Presta, Kuzman Ganchev, Slav Petrov, and Michael Collins. 2016. http://arxiv.org/abs/1603.06042 Globally normalized transition-based neural networks . CoRR, abs/1603.06042

arXiv 2016

[4] [4]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. http://arxiv.org/abs/1409.0473 Neural machine translation by jointly learning to align and translate . In ICLR

arXiv 2015

[5] [5]

David Belanger, Bishan Yang, and Andrew McCallum. 2017. http://proceedings.mlr.press/v70/belanger17a.html End-to-end learning for structured prediction energy networks . In ICML

work page 2017

[6] [6]

Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. 2015. http://arxiv.org/abs/1506.03099 Scheduled sampling for sequence prediction with recurrent neural networks . In NIPS

arXiv 2015

[7] [7]

David M Blei, Alp Kucukelbir, and Jon D McAuliffe. 2017. https://arxiv.org/abs/1601.00670 Variational inference: A review for statisticians . Journal of the American Statistical Association

arXiv 2017

[8] [8]

Samuel R.\ Bowman, Gabor Angeli, Christopher Potts, and Christopher D.\ Manning. 2015. http://nlp.stanford.edu/pubs/snli_paper.pdf A large annotated corpus for learning natural language inference . In EMNLP

work page 2015