Pith. sign in

REVIEW 3 major objections 4 minor 64 references

Latent Relation Language Models

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A language model that copies entity spans from knowledge graphs through latent relations beats word-only baselines.

desk verdict A clean and tractable latent-variable LM over KG relations with real but uneven gains; the posterior entity-link claim is not yet supported, and the NKLM comparison has a soft spot. read the letter →

arxiv 1908.07690 v1 pith:OS4ZQSE3 submitted 2019-08-21 cs.CL

Hiroaki Hayashi , Zecong Hu , Chenyan Xiong , Graham Neubig This is my paper · ORCID
classification cs.CL
keywords latentrelationlanguagemodelsknowledgegraphconditionedmodelingopen-vocabularyperplexityvariablemarginalizationentitylinkingspan-basedgenerationforward-backwardalgorithmdistantsupervision
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Latent Relation Language Models (LRLMs) are conditional language models that generate a document as a sequence of spans, each produced either from a fixed word vocabulary or by copying a surface form of an entity connected to the topic by a knowledge-graph relation. Because the text does not reveal which spans are entities or which relations are being expressed, the model treats the source choice, span boundaries, and relation as latent variables and marginalizes over all valid segmentations with the forward-backward algorithm. The paper reports that LRLM achieves lower open-vocabulary perplexity—the standard prediction-error score for language models, lower being better—than both a vanilla word-level language model and a previous hard-decision knowledge-graph language model on all three datasets tested, with either LSTM or Transformer-XL backbones. This matters because rare, topically important words and factual phrases are exactly where ordinary language models struggle, and the same machinery yields posterior probabilities over relations that can act as entity-link annotations.

What carries the argument

The central object is the latent variable sequence $Z = \{(\pi_t, \sigma_t, \rho_t)\}$, where $\pi_t$ is the source choice (relation or word), $\sigma_t=(\ell_t,r_t)$ is the span of tokens generated at step $t$, and $\rho_t=(e_t,a_t)$ names the relation and surface form when the source is relation. A valid sequence is exactly a segmentation of the text in which every relation span equals a surface form of an object connected to the topic entity $s$ in the knowledge-graph subgraph. The model defines $P(X\mid G',s)=\sum_{Z} P(X,Z\mid G',s)$, factorizes each step as source probability times span probability, and sums the lattice of segmentations with the forward-backward algorithm. This machinery is what lets multi-word entity copying, overlapping candidate spans, and posterior entity-link scores all emerge from a single training objective.

What would settle it

Take a test set with hand-annotated entity mentions and compare LRLM's perplexity and posterior span accuracy when the candidate span set comes from the gold annotations versus from pure string matching; if the two perform identically, the distant-supervision linker's recall is not the driver of the reported gains, and if gold spans improve results, the matching noise is the limiting factor.

Watch

Extended reading notes

Core claim

The central claim is that marginalizing over latent relation spans, rather than making hard decisions about which phrases are entities, is what lets a language model actually exploit a knowledge graph. The model factorizes each step as a choice between generating a word and generating a relation span; a relation span is valid only if it matches one of the surface forms of an object connected to the topic entity in the graph, and overlapping candidate spans are allowed because they are resolved by marginalization. Training maximizes the total probability of the text summed over every valid latent segmentation, which the forward-backward algorithm makes tractable. The experiments show LRLM reaching lower open-vocabulary perplexity than the vanilla LM and the hard-copy NKLM baseline on WikiFacts, WikiText-S, and WikiText-F with LSTM and Transformer-XL backbones (with one LSTM comparison not reaching statistical significance), and qualitative examples show posterior relation probabilities that track the content of the surrounding text.

Load-bearing premise

The load-bearing premise is that string-matching the knowledge graph's surface forms against the text yields a candidate span set that covers the entity mentions the model needs, while the false positives from this distant supervision are rare or noisy enough for training to learn to ignore.

Editorial extensions

If this is right

  • Conditioning on a knowledge graph through latent relation spans lowers open-vocabulary perplexity compared with a word-only language model and with a hard-decision copy model, and the gain grows as the subgraph has more relations.
  • The trained model can annotate an existing text with posterior probabilities for relation-generated spans, providing entity-link-style information without separate supervision.
  • Because the span machinery sits on top of any sequence model, the same method works with LSTM and Transformer-XL backbones.
  • Sampling from LRLM produces full entity mentions rather than the partial or broken mentions that word-by-word copy models produce.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The posterior span probabilities are shown qualitatively but not measured against gold entity-link labels; a precision/recall evaluation on an annotated set would tell whether the by-product is ready for use in information extraction.
  • The same latent-span formulation should transfer to other structured contexts—tables, databases, program schemas—wherever candidate surface forms can be enumerated and matched against text.
  • Since the distant-supervision linker is the entry point for all knowledge, replacing string matching with learned mention detection or fuzzy matching is the most direct lever for improving both coverage and robustness to false positives.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Latent Relation Language Models (LRLMs), which augment a word-level language model with span-level generation from a knowledge graph. A latent variable sequence selects, at each step, whether the next token is a vocabulary word or a surface form of an object related to a topic entity; training marginalizes over all valid segmentations with a forward-backward algorithm. The authors construct WikiText-S and WikiText-F by matching Wikidata surface forms against WikiText-103 articles, and report open-vocabulary perplexity on WikiFacts and these datasets with LSTM and Transformer-XL backbones. They further present posterior span probabilities as entity-link annotations and show qualitative examples.

Significance. The model formulation is clean: the factorization in Section 3.2 and the forward recursion in Section 3.3 make latent-variable marginalization tractable, and the framework is agnostic to the underlying sequence encoder. If the reported gains are due to KG-aware relation modeling, this would be a useful way to condition language models on structured knowledge and to obtain soft entity-link annotations. The open-vocabulary evaluation and the inclusion of two backbone architectures strengthen the empirical study. However, the paper does not yet provide quantitative evidence that the model distinguishes true from false-positive relation mentions, and the comparison to NKLM involves a span-pruning preprocessing step whose effect is not quantified; these gaps must be closed before the semantic interpretation of the perplexity gains is justified.

major comments (3)
  1. [Section 4.2 and Eq. (1)] The paper relies on the model's ability to ignore false-positive distant-supervision spans, but this is never measured. Because Eq. (1) marginalizes over all valid segmentations, any string-match false positive ('United States' as an <origin> mention in a non-entity context) contributes an additional term to the training objective; nothing in the objective penalizes assigning high probability to a relation path for such a span. The perplexity gains in Table 2 could therefore come from a generic multi-word copy mechanism rather than from knowledge-graph relation semantics, and the posterior entity-link claim in Section 6.3 remains unsupported: Table 4 shows two favorable examples, and Table 3 counts mention validity in generated samples rather than posterior precision/recall over real text. I ask for (i) posterior precision/recall against gold entity annotations on a held-out sample, and (ii) a control experiment that replaces the relation embeddings or surface forms with random or shuffled versions; if perplexity gains persist, the mechanism is copying, not relation modeling.
  2. [Appendix D] The NKLM comparison on WikiText is conducted after a greedy span-pruning pass that removes all overlaps, preferring longer spans, earlier starts, and canonical surface forms. Because LRLM is specifically designed to marginalize over overlapping spans, this preprocessing may remove information that NKLM could have used, while LRLM still sees all matches. The paper does not report how many gold mentions are pruned or how NKLM performs under alternative pruning orders. Please provide that analysis or an ablation; otherwise the claim that LRLM outperforms NKLM on WikiText is not fully controlled.
  3. [Section 6.3] The displayed formula for the posterior is not well-defined as written. In Section 3.3, alpha_i is a scalar forward marginal probability summing over all latent paths ending at token i; the term P(Z | x_{<l_i}) used in the posterior formula is therefore not a quantity that has been defined, and the decomposition P(X,Z) = alpha_i * P(Z | x_{<l_i}) * beta_i does not correspond to the forward-backward recursion described earlier. Please define the forward and backward variables with their precise conditioning and derive the posterior of a single span tuple (pi, sigma, rho) by summing over all compatible previous and future latent variables; otherwise the posterior examples in Table 4 are not reproducible.
minor comments (4)
  1. [Section 6.1 and Table 2] The blanket statement that LRLM 'out-performs the baselines on all datasets' should be qualified, because the LSTM+WikiText-S row shows a small improvement over NKLM that the authors themselves report as not statistically significant; please add confidence intervals or exact p-values for all rows.
  2. [Table 1] The column 'Ment/Doc' is not defined; it should state whether these are gold entity mentions or surface-form matches produced by the linking procedure, since the two are very different quantities.
  3. [Section 5.1 and Appendix B] The paper would benefit from a statement on whether code and the constructed WikiText-S/WikiText-F datasets will be released, because the matching and pruning pipelines contain many choices that are difficult to reproduce exactly from the text alone.
  4. [Section 6.3] The notation Z is overloaded: it denotes the full latent variable sequence in Eq. (1), but in the posterior formula it appears to denote a single span tuple; please use distinct notation for these two objects.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: LRLM predictions are held-out perplexities; latent spans are marginalized, not fitted to targets.

full rationale

The paper's central claim is that LRLM improves open-vocabulary perplexity over Vanilla LM and NKLM on WikiFacts, WikiText-S, and WikiText-F (Table 2). These numbers are computed on held-out articles after training, so the evaluation target is not used to fit the relation, source, or surface-form parameters. The latent span set is constructed by string matching KG surface forms (Section 4.2), but the model objective in Equation (1) and the forward probability alpha_N in Section 3.3 marginalize over all valid segmentations rather than being defined in terms of the reported perplexity or the posterior annotations. The posterior P(Z|X) in Section 6.3 is a by-product of the trained model, not a quantity inserted into training; its qualitative examples illustrate model behavior but do not constitute a circular derivation. The paper borrows latent predictor machinery from Ling et al. (2016), which is an external, non-overlapping citation, and no load-bearing claim rests on a self-citation or a uniqueness theorem. The unsupported assumption that the model learns to ignore distant-supervision false positives is a correctness or evidence concern, not a circularity one, because the loss does not fit the model to gold mention labels and the resulting spans are not renamed predictions. No equation reduces to its own input, so the appropriate finding is no significant circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 1 invented entities

The central empirical claim rests on standard algorithmic machinery (forward-backward), external resources (OpenKE, fastText, pre-trained char LM), and a distant-supervision construction of the latent space. The main hand-chosen hyperparameters are listed; no new physical or mathematical entities are postulated beyond the latent variable sequence Z.

free parameters (5)
  • Per-dataset LSTM hyperparameters = embedding 400/400/512; hidden 1000/1000/1024; layers 2/2/4; dropout 0.5/0.5/0.1
    Chosen per dataset in Appendix B; affect both baseline and LRLM, so reported gains could shift if these were retuned.
  • Relation linear hidden unit size = 1000/1000/800
    Dimension of the linear layer for relation selection (Table 5); selected by hand for each dataset.
  • Low-rank softmax bottleneck K = 1000/1000/500 (LSTM); corresponding Transformer-XL value not separately listed
    Bottleneck dimension for the word softmax, stated in Section 3.4.2; impacts word generation capacity.
  • Entity embedding dimension = 50 for WikiFacts, 100 for WikiText
    Taken from OpenKE pre-trained embeddings; external dimension choice.
  • fastText embedding size = 300
    Used for surface form embeddings in Section 3.4.3; fixed external dimension.
assumptions (4)
  • standard math The forward-backward algorithm computes the marginal likelihood exactly because the latent segmentation variables form a linear-chain structure.
    Baum et al. (1970); used in Section 3.3 to justify tractable marginalization over Z.
  • domain assumption The character-level LM, pre-trained on the training set and fixed, provides a valid probability for OOV token spell-outs, so open-vocabulary perplexity is well-defined.
    Section 3.4.2; if the char LM is uncalibrated or domain-shifted, perplexity comparisons could be biased, though all models use the same char LM.
  • domain assumption Wikidata surface forms and aliases have sufficient coverage and correctness for entity mentions in Wikipedia text.
    Section 4.2; the latent span space is built from these surface forms; if coverage is poor, relation-based generation cannot help.
  • ad hoc to paper Distant supervision noise (false positive spans) can be learned to be ignored by the model.
    Section 4.2: 'We rely on our model's ability to ignore such mentions'; no proof is given; empirical results suggest it holds on these datasets.
invented entities (1)
  • Latent variable sequence Z = {(pi_t, sigma_t, rho_t)}
    purpose: Models the hidden choice of generating from the word vocabulary or copying a relation span; enables tractable marginalization and posterior span probabilities.
    A modeling construct, not a physically observable entity; no falsifiable prediction outside the model's outputs.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Latent Relation Language Models." pith.science (2026). https://pith.science/paper/OS4ZQSE3

@misc{pith2026190807690,
  author       = {Pith},
  title        = {Pith review of: Latent Relation Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OS4ZQSE3}},
  note         = {Machine review of arXiv:1908.07690}
}
read the original abstract

In this paper, we propose Latent Relation Language Models (LRLMs), a class of language models that parameterizes the joint distribution over the words in a document and the entities that occur therein via knowledge graph relations. This model has a number of attractive properties: it not only improves language modeling performance, but is also able to annotate the posterior probability of entity spans for a given text through relations. Experiments demonstrate empirical improvements over both a word-based baseline language model and a previous approach that incorporates knowledge graph information. Qualitative analysis further demonstrates the proposed model's ability to learn to predict appropriate relations in context.

Figures

Figures reproduced from arXiv: 1908.07690 by the authors.

Figure 1
Figure 1. Overview of our task of language model￾ing conditioned on structured knowledge. For a given topic, we want to learn an LM that leverages the knowl￾edge graph through relations when modeling the text. frequency). Previous work has noted that while neural LMs greatly out-perform alternatives such as n-gram models on frequent words, they often under-perform on these rare words due to their limited parameter budget, whi… view at source ↗
Figure 2
Figure 2. While generating, our model switches between the two sources, namely “Relation” and “Word”. Nodes [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Samples from the three models for a topic entity “ [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Word-average log-probabilities on develop [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

64 extracted references · 42 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Sungjin Ahn, Heeyoul Choi, Tanel P \"a rnamaa, and Yoshua Bengio. 2016. http://arxiv.org/abs/1608.00318 A neural knowledge language model . CoRR, arXiv:1608.00318

    arXiv 2016

  4. [4]

    Gabor Angeli, Percy Liang, and Dan Klein. 2010. http://aclweb.org/anthology/D10-1049 A simple domain-independent probabilistic approach to generation . In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 502--512. Association for Computational Linguistics

    work page 2010

  5. [5]

    Philip Arthur, Graham Neubig, and Satoshi Nakamura. 2016. https://doi.org/10.18653/v1/D16-1162 Incorporating discrete translation lexicons into neural machine translation . In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1557--1567. Association for Computational Linguistics

    doi:10.18653/v1/d16-1162 2016

  6. [6]

    Alexei Baevski and Michael Auli. 2019. https://openreview.net/forum?id=ByxZX20qFQ Adaptive input representations for neural language modeling . In International Conference on Learning Representations

    2019

  7. [7]

    Dzmitry Bahdanau, Tom Bosc, Stanis aw Jastrz e bski, Edward Grefenstette, Pascal Vincent, and Yoshua Bengio. 2017. https://arxiv.org/abs/1706.00286 Learning to compute word embeddings on the fly . CoRR, arXiv:1706.00286

    work page Pith review arXiv 2017

  8. [8]

    Regina Barzilay and Mirella Lapata. 2005. http://aclweb.org/anthology/H05-1042 Collective content selection for concept-to-text generation . In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 331--338. Association for Computational Linguistics

    work page 2005

Show all 64 references
  1. [9]

    Baum, Ted Petrie, George Soules, and Norman Weiss

    Leonard E. Baum, Ted Petrie, George Soules, and Norman Weiss. 1970. http://www.jstor.org/stable/2239727 A maximization technique occurring in the statistical analysis of probabilistic functions of markov chains . The Annals of Mathematical Statistics, 41(1):164--171

    1970

  2. [10]

    Yoshua Bengio, R \'e jean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. http://www.jmlr.org/papers/v3/bengio03a.html A neural probabilistic language model . Journal of Machine Learning Research, 3(Feb):1137--1155

    2003

  3. [11]

    Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. https://www.aclweb.org/anthology/Q17-1010 Enriching word vectors with subword information . Transactions of the Association for Computational Linguistics, 5:135--146

    2017

  4. [12]

    Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. https://doi.org/10.1145/1376616.1376746 Freebase : A collaboratively created graph database for structuring human knowledge . In Proceedings of the 2008 ACM SIGMOD International Conference on Ma...

    2008

  5. [13]

    Bowman, Luke Vilnis, Oriol Vinyals, Andrew Dai, Rafal Jozefowicz, and Samy Bengio

    Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew Dai, Rafal Jozefowicz, and Samy Bengio. 2016. http://aclweb.org/anthology/K16-1002 Generating sentences from a continuous space . In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pages...

    2016

  6. [14]

    Jacob Buckman and Graham Neubig. 2018. https://transacl.org/ojs/index.php/tacl/article/view/1261 Neural lattice language models . Transactions of the Association for Computational Linguistics, 6:529--541

    2018

  7. [15]

    Diego Ceccarelli, Claudio Lucchese, Salvatore Orlando, Raffaele Perego, and Salvatore Trani. 2013. http://doi.acm.org/10.1145/2505515.2505711 Learning relatedness measures for entity linking . In Proceedings of the 22nd ACM International conference on Information & Knowledge M...

    2013

  8. [16]

    Chen and Raymond J

    David L. Chen and Raymond J. Mooney. 2008. http://doi.acm.org/10.1145/1390156.1390173 Learning to sportscast: A test of grounded language acquisition . In Proceedings of the 25th International Conference on Machine Learning, pages 128--135. Association for Computing Machinery

    2008

  9. [17]

    Junyoung Chung, Sungjin Ahn, and Yoshua Bengio. 2017. https://openreview.net/forum?id=S1di0sfgl Hierarchical multiscale recurrent neural networks . In International Conference on Learning Representations

    2017

  10. [18]

    Elizabeth Clark, Yangfeng Ji, and Noah A. Smith. 2018. https://doi.org/10.18653/v1/N18-1204 Neural text generation in stories using entity representations as context . In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Ling...

    2018 doi

  11. [19]

    Dai and Quoc V

    Andrew M. Dai and Quoc V. Le. 2015. http://papers.nips.cc/paper/5949-semi-supervised-sequence-learning Semi-supervised sequence learning . In Advances in Neural Information Processing Systems 28, pages 3079--3087

    2015

  12. [20]

    Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc Le, and Ruslan Salakhutdinov. 2019. https://www.aclweb.org/anthology/P19-1285 Transformer- XL : Attentive language models beyond a fixed-length context . In Proceedings of the 57th Annual Meeting of the Association fo...

    2019

  13. [21]

    Rotem Dror, Gili Baumer, Segev Shlomov, and Roi Reichart. 2018. https://www.aclweb.org/anthology/P18-1128 The hitchhiker ' s guide to testing statistical significance in natural language processing . In Proceedings of the 56th Annual Meeting of the Association for Computationa...

    2018

  14. [22]

    Andersen, Helen Yannakoudakis, and Ekaterina Kochmar

    Mariano Felice, Zheng Yuan, istein E. Andersen, Helen Yannakoudakis, and Ekaterina Kochmar. 2014. http://aclweb.org/anthology/W14-1702 Grammatical error correction using hybrid systems and type filtering . In Proceedings of the Eighteenth Conference on Computational Natural La...

    2014

  15. [23]

    Octavian-Eugen Ganea and Thomas Hofmann. 2017. https://doi.org/10.18653/v1/D17-1277 Deep joint entity disambiguation with local neural attention . In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2619--2629, Copenhagen, Denmark. ...

    2017 doi

  16. [24]

    Eli Goldberg, Norbert Driedger, and Richard I Kittredge. 1994. https://ieeexplore.ieee.org/document/294135 Using natural-language processing to produce weather forecasts . IEEE Expert, 9(2):45--53

    1994

  17. [25]

    \'E douard Grave, Armand Joulin, Moustapha Ciss \'e , David Grangier, and Herv \'e J \'e gou. 2017. http://proceedings.mlr.press/v70/grave17a.html Efficient softmax approximation for GPU s . In Proceedings of the 34th International Conference on Machine Learning, volume 70 of ...

    2017

  18. [26]

    Jiatao Gu, Zhengdong Lu, Hang Li, and Victor O.K. Li. 2016. http://aclweb.org/anthology/P16-1154 Incorporating copying mechanism in sequence-to-sequence learning . In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers...

    2016

  19. [27]

    Xu Han, Shulin Cao, Xin Lv, Yankai Lin, Zhiyuan Liu, Maosong Sun, and Juanzi Li. 2018. http://aclweb.org/anthology/D18-2024 OpenKE : An open toolkit for knowledge embedding . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demo...

    2018

  20. [28]

    Sepp Hochreiter and J\" u rgen Schmidhuber. 1997. https://doi.org/10.1162/neco.1997.9.8.1735 Long short-term memory . Neural Computation, 9(8):1735--1780

    1997 doi

  21. [29]

    Yangfeng Ji, Chenhao Tan, Sebastian Martschat, Yejin Choi, and Noah A. Smith. 2017. http://aclweb.org/anthology/D17-1195 Dynamic entity representations in neural language models . In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages ...

    2017

  22. [30]

    Chlo \'e Kiddon, Luke Zettlemoyer, and Yejin Choi. 2016. https://doi.org/10.18653/v1/D16-1032 Globally coherent text generation with neural checklist models . In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 329--339. Association...

    2016 doi

  23. [31]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. 2015. https://arxiv.org/abs/1412.6980 Adam : A method for stochastic optimization . In International Conference on Learning Representations

    2015 arXiv

  24. [32]

    Ioannis Konstas and Mirella Lapata. 2013. https://www.jair.org/index.php/jair/article/view/10841 A global model for concept-to-text generation . Journal of Artificial Intelligence Research, 48:305--346

    2013

  25. [33]

    R \'e mi Lebret, David Grangier, and Michael Auli. 2016. https://doi.org/10.18653/v1/D16-1128 Neural text generation from structured data with application to the biography domain . In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages...

    2016 doi

  26. [34]

    Wang Ling, Phil Blunsom, Edward Grefenstette, Karl Moritz Hermann, Tom \'a s Ko c isk \'y , Fumin Wang, and Andrew Senior. 2016. https://doi.org/10.18653/v1/P16-1057 Latent predictor networks for code generation . In Proceedings of the 54th Annual Meeting of the Association fo...

    2016 doi

  27. [35]

    Tianyu Liu, Kexiang Wang, Lei Sha, Baobao Chang, and Zhifang Sui. 2018. https://aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16599 Table-to-text generation by structure-aware seq2seq learning . Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence

    2018

  28. [36]

    Liu, Matthew E

    Robert Logan, Nelson F. Liu, Matthew E. Peters, Matt Gardner, and Sameer Singh. 2019. https://www.aclweb.org/anthology/P19-1598 B arack ' s wife H illary: Using knowledge graphs for fact-aware language modeling . In Proceedings of the 57th Annual Meeting of the Association for...

    2019

  29. [37]

    Minh-Thang Luong and Christopher D. Manning. 2016. https://doi.org/10.18653/v1/P16-1100 Achieving open vocabulary neural machine translation with hybrid word-character models . In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1...

    2016 doi

  30. [38]

    Stephen Merity, Nitish Shirish Keskar, and Richard Socher. 2017 a . https://arxiv.org/abs/1708.02182 Regularizing and optimizing LSTM language models . CoRR, arXiv:1708.02182

    2017 arXiv

  31. [39]

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2017 b . https://openreview.net/forum?id=Byj72udxe Pointer sentinel mixture models . In International Conference on Learning Representations

    2017

  32. [40]

    Yishu Miao and Phil Blunsom. 2016. http://aclweb.org/anthology/D16-1031 Language as a latent variable: Discrete generative models for sentence compression . In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 319--328, Austin, Texas...

    2016

  33. [41]

    Tom \'a s Mikolov, Martin Karafi \'a t, Luk \'a s Burget, Jan C ernock \`y , and Sanjeev Khudanpur. 2010. https://www.isca-speech.org/archive/interspeech\_2010/i10\_1045.html Recurrent neural network based language model . In Eleventh Annual Conference of the International Spe...

    2010

  34. [42]

    Mike Mintz, Steven Bills, Rion Snow, and Daniel Jurafsky. 2009. https://www.aclweb.org/anthology/P09-1113 Distant supervision for relation extraction without labeled data . In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International J...

    2009

  35. [43]

    Graham Neubig and Chris Dyer. 2016. http://aclweb.org/anthology/D16-1124 Generalizing and hybridizing count-based and neural language models . In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1163--1172, Austin, Texas. Associatio...

    2016

  36. [44]

    Md Rizwan Parvez, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Chang. 2018. http://aclweb.org/anthology/P18-1221 Building language models for text with named entities . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long...

    2018

  37. [45]

    Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. https://openreview.net/forum?id=BJJsrmfCZ Automatic differentiation in PyTorch

    2017

  38. [46]

    Laura Perez-Beltrachini and Mirella Lapata. 2018. https://www.aclweb.org/anthology/N18-1137 Bootstrapping generators from noisy data . In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog...

    2018

  39. [47]

    Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. https://doi.org/10.18653/v1/N18-1202 Deep contextualized word representations . In Proceedings of the 2018 Conference of the North American Chapter of the Associ...

    2018 doi

  40. [48]

    Francesco Piccinno and Paolo Ferragina. 2014. http://doi.acm.org/10.1145/2633211.2634350 From TagME to WAT : a new entity annotator . In Proceedings of the First International Workshop on Entity Recognition & Disambiguation, pages 55--62. Association for Computing Machinery

    2014

  41. [49]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. https://blog.openai.com/better-language-models/ Language models are unsupervised multitask learners . Preprint

    2019

  42. [50]

    Ehud Reiter, Somayajulu Sripada, Jim Hunter, Jin Yu, and Ian Davy. 2005. https://doi.org/10.1016/j.artint.2005.06.006 Choosing words in computer-generated weather forecasts . Artificial Intelligence, 167(1-2):137--169

    2005 doi

  43. [51]

    Gerard Salton and Michael J. McGill. 1986. Introduction to Modern Information Retrieval. McGraw-Hill, Inc., New York, NY, USA

    1986

  44. [52]

    Martin Sundermeyer, Ralf Schl \"u ter, and Hermann Ney. 2012. https://www.isca-speech.org/archive/interspeech\_2012/i12\_0194.html LSTM neural networks for language modeling . In Thirteenth Annual Conference of the International Speech Communication Association

    2012

  45. [53]

    Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. http://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks Sequence to sequence learning with neural networks . In Advances in Neural Information Processing Systems 27, pages 3104--3112

    2014

  46. [54]

    Yaohua Tang, Fandong Meng, Zhengdong Lu, Hang Li, and Philip LH Yu. 2016. https://arxiv.org/abs/1606.01792 Neural machine translation with external phrase memory . CoRR, arXiv:1606.01792

    2016 arXiv

  47. [55]

    Joerg Ueberla. 1994. https://doi.org/10.1006/csla.1994.1007 Analysing a simple language model some general conclusions for language models for speech recognition . Computer Speech & Language, 8(2):153--176

    1994

  48. [56]

    Denny Vrande c i\' c and Markus Kr\" o tzsch. 2014. https://doi.org/10.1145/2629489 Wikidata : A free collaborative knowledgebase . Communications of the ACM, 57(10):78--85

    2014 doi

  49. [57]

    Qingyun Wang, Xiaoman Pan, Lifu Huang, Boliang Zhang, Zhiying Jiang, Heng Ji, and Kevin Knight. 2018. http://aclweb.org/anthology/W18-6502 Describing a knowledge base . In Proceedings of the 11th International Conference on Natural Language Generation, pages 10--21. Associatio...

    2018

  50. [58]

    Williams and Jing Peng

    Ronald J. Williams and Jing Peng. 1990. https://doi.org/10.1162/neco.1990.2.4.490 An efficient gradient-based algorithm for on-line training of recurrent network trajectories . Neural Computation, 2(4):490--501

    1990 doi

  51. [59]

    Sam Wiseman, Stuart Shieber, and Alexander Rush. 2018. http://aclweb.org/anthology/D18-1356 Learning neural templates for text generation . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3174--3187, Brussels, Belgium. Associati...

    2018

  52. [60]

    Zhilin Yang, Zihang Dai, Ruslan Salakhutdinov, and William W. Cohen. 2018. https://openreview.net/forum?id=HkwZSG-CZ Breaking the softmax bottleneck: A high-rank RNN language model . In International Conference on Learning Representations

    2018

  53. [61]

    Zichao Yang, Phil Blunsom, Chris Dyer, and Wang Ling. 2017. https://doi.org/10.18653/v1/D17-1197 Reference-aware language models . In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1850--1859. Association for Computational Linguistics

    2017 doi

  54. [62]

    Pengcheng Yin, Chunting Zhou, Junxian He, and Graham Neubig. 2018. http://aclweb.org/anthology/P18-1070 StructVAE : Tree-structured latent variable models for semi-supervised semantic parsing . In Proceedings of the 56th Annual Meeting of the Association for Computational Ling...

    2018

  55. [63]

    Dani Yogatama, Phil Blunsom, Chris Dyer, Edward Grefenstette, and Wang Ling. 2016. https://openreview.net/forum?id=Skvgqgqxe Learning to compose words into sentences with reinforcement learning . In International Conference on Learning Representations

    2016

  56. [64]

    George Kingsley Zipf. 1949. Human behavior and the principle of least effort: An introduction to human eoclogy. Addison-Wesley Press

    1949

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.