Pith. sign in

REVIEW 4 major objections 4 minor 61 references

Leveraging Sentence Similarity in Natural Language Generation: Improving Beam Search using Range Voting

T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper argues that for language generation the right objective is not the most likely sequence but the most representative one — the winner of a range-voting election in which every beam hypothesis votes for every other, weighted by…

desk verdict A training-free decoding reranker that extends MBR with asymmetric n-gram similarities; the core experiments are solid, but the human-eval claim leans on a sign test that hides a reversed 'much worse' tail. read the letter →

arxiv 1908.06288 v2 pith:GP2BILJL submitted 2019-08-17 cs.CL

Sebastian Borgeaud , Guy Emerson This is my paper · ORCID
classification cs.CL
keywords rangevotingbeamsearchnaturallanguagegenerationrepresentativenesssimilaritymeasureminimumBayesriskdecodingimagecaptioningmachinetranslation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Beam search for language generation traditionally returns the single most probable sequence, but this paper argues that the mode of a discrete distribution over sentences is often the wrong answer: probability mass split across many similar long sentences can make a short, boring sentence the winner. The proposed fix is to make the beam hold an election. Each candidate sentence receives votes from every other sentence in the beam, weighted by the voter's model probability and by a similarity score between voter and candidate, and the sentence with the highest total is the "most representative" output. On image captioning and English-German translation, this range-voting re-ranking beats plain beam search, length normalisation, and diverse decoding on BLEU, produces longer and more diverse outputs, and wins head-to-head human evaluations. The method is model-agnostic and needs no retraining.

What carries the argument

The central object is a range-voting election over a beam. Each sequence $v$ in the voter set $V$ casts a vote of strength $\mathrm{sim}(v,c)$ for each candidate sequence $c$, weighted by its model probability $P(v)$, and the winner maximizes $\mathrm{score}(c)=\sum_{v\in V} P(v)\,\mathrm{sim}(v,c)$. Range voting is independent of clones, so splitting probability mass across near-identical sequences does not dilute their collective strength. The paper pairs this with asymmetric n-gram similarities (precision and overlap) that let longer, more informative candidates collect votes from shorter voters, plus a cosine similarity between averaged decoder hidden states.

What would settle it

On the two tasks, decode with range voting using a similarity measure that has nothing to do with content—for example, how many function words two sentences share, or random scores drawn once per pair. If BLEU and human preference still match or beat standard beam search as beam size grows, the reported gains are produced by probability-weighted voting itself, not by similarity-based representativeness. A second check: sample many sequences from the model and test whether the range-voting winner is closer to the probability-weighted average of hidden states than the beam-search argmax; if it is not, the "representative" label does not hold.

Watch

Extended reading notes

Core claim

The central discovery is that representativeness, not likelihood, should drive decoding when a language model puts comparable probability on a cluster of similar outputs. The paper formalizes representativeness as the winner of a range-voting election over beam hypotheses: $\mathrm{score}(c)=\sum_{v\in V} P(v)\,\mathrm{sim}(v,c)$. This recovers Minimum Bayes Risk decoding when the similarity measure is BLEU, but with an independent motivation and with freedom to choose asymmetric similarity measures that reward informative candidates. Empirically, the method removes the usual BLEU drop as beam size grows in image captioning, shrinks it in translation, and improves human preference ratings on both tasks.

Load-bearing premise

The load-bearing premise is that the similarity measure actually captures which sentences are alike in the way that matters; if the measure is a poor proxy for representativeness, the "representative" winner is only representative under that flawed metric.

Editorial extensions

If this is right

  • On image captioning, BLEU no longer falls as beam size grows from 10 to 100, so the method removes one source of sensitivity to the beam-size hyperparameter.
  • Because the asymmetric n-gram similarities reward candidates that contain the voter plus more information, outputs become longer and lexically more diverse, moving caption and translation length closer to that of references.
  • Range voting is a superset of MBR decoding: swapping in any similarity measure, not just the evaluation metric, lets practitioners bias decoding toward informativeness or faithfulness without retraining the model.
  • Decoupling voters from candidates pays off: with candidates fixed at beam size 4, increasing the voter set to 1000 via sampling, stochastic beam search, or filtered beam search raises BLEU, implying search quality and selection quality can be improved separately.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the similarity measure is the true engine of the gains, replacing n-gram overlap with a calibrated semantic similarity should widen the gap over beam search; if gains persist under a semantically meaningless similarity, the mechanism is probability-weighted averaging rather than representativeness.
  • The candidate/voter separation suggests an inference-time recipe the paper does not fully exploit: keep candidates small and high-precision while drawing a large, diverse voter set by sampling, a direction Figure 3 indicates should keep improving BLEU.
  • The human-evaluation tails show representative decoding can also amplify model errors, so a natural extension is to gate voting with a faithfulness or quality filter to avoid rewarding confident wrong information.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a decoding strategy based on range voting: given a set of candidate sequences and voter sequences from beam search, it scores each candidate as the probability-weighted average similarity to all voters. Several similarity measures are tested (n-gram overlap, bag precision, averaged hidden-state cosine). Experiments on MSCOCO image captioning and WMT14 En-De translation report higher BLEU, longer and more diverse outputs, and a human evaluation that the authors claim shows the method is preferred. The method is positioned as an alternative to MBR decoding.

Significance. The proposed method is simple, model-agnostic, and addresses the known issue of beam-search degradation at large beam sizes. The empirical gains in BLEU at large k (e.g., 28.23 vs 26.31 BLEU-4 on captioning at k=100; 24.62 vs 22.59 on translation at k=100) are substantial. The paper also provides a useful discussion of vote splitting and its connection to MBR. However, the human-evaluation evidence is weaker than claimed, and the theoretical claim about not needing a model-distribution approximation is inaccurate. The contribution is a solid empirical study of a simple re-ranking objective, but the paper needs revisions to validate the headline claims.

major comments (4)
  1. [§4.1.4, §4.2.4, Conclusion] The human evaluations compare different beam sizes between conditions: in captioning, standard beam search at k=10 is compared with precision2 at k=100, and in translation, length normalisation at k=4 is compared with precision2 at k=10. This confounds the effect of range voting with the effect of beam size and output length. Additionally, the 'much better' counts in captioning show the voted caption is judged much worse 40 times versus much better 27 times (p=0.142), so the strong-preference direction is reversed. The conclusion that the method is 'rated higher' is therefore overstated. Please run matched-beam-size comparisons and report the strong-preference statistics in the conclusion.
  2. [§3.3] The claim that range voting 'does not require the assumption that we can approximate the true distribution by the model's distribution' is incorrect. Eq. (1) weights each voter by P(v), which is exactly the model distribution. Score(c) = sum_v P(v) sim(v,c) is the expected utility under the model posterior, i.e., MBR with a different utility function. The difference from classic MBR is in the choice and asymmetry of sim, not in the absence of the approximation.
  3. [§4.1.1, §4.2.1, §4.2.5] The best-performing similarity metric is selected on the test set (lstm states for captioning, overlap2 for translation, precision2 for the human evaluations). For example, in §4.2.5 'we fix the similarity metric to overlap2, which was the best performing metric for large k≥4 (§4.2.1)'. This test-set peeking can inflate the reported gains. Please report metric selection on a validation set or provide a multiple-comparison analysis, and acknowledge this limitation.
  4. [Tables 1 and 4] The BLEU tables report point estimates without confidence intervals or significance tests, except for a single paired bootstrap test in §4.1.1. Several conclusions rely on small differences (e.g., captioning at k=10). Please provide bootstrap confidence intervals or p-values for the key comparisons to establish that the improvements are not within noise.
minor comments (4)
  1. [Eq. (2)] The measure precisionn(v,c) divides by |bag_n(v)|, so it is a recall score relative to v, not a precision score. This naming is confusing, particularly in the comparison with BLEU in §3.3. Consider renaming it to 'recalln' or clarifying the definition.
  2. [Title and §2.2] The title contains 'Range V oting' with a space, and §2.2 contains 'doesnt' (missing apostrophe). Please proofread.
  3. [§4.1.4] The description of the rating scale reads 'one caption much better, slightly better, or no difference'; it should clarify that each caption can be rated on the five-point scale (e.g., 'one caption much better, one slightly better, or no difference').
  4. [Table 4] The copy-filtering heuristic is applied only to the range-voting rows and to the 'beam search (no copy)' baseline, while the other baselines (length normalisation, diverse decoding, diverse beam search) are not filtered. This asymmetry should be stated explicitly or the baselines should be re-run with filtering.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the voting score is a definition and the reported gains are external empirical results.

full rationale

The paper's core object, score(c) = sum_v P(v) * sim(v,c) (Eq. 1), is presented as a definition of representativeness, not as a prediction derived from the data. The n-gram similarity measures (Eqs. 2-3) are explicitly designed to favor informative and longer candidates, so the observed length increases are a designed effect confirmed empirically, not a circular inference. The claimed BLEU improvements are evaluated against external references (MSCOCO captions and newstest2014) and are not encoded in the scoring rule, so they are not forced by construction. The paper also transparently acknowledges the relation to MBR decoding, noting that choosing sim = BLEU recovers MBR; this is a connection, not a renamed result. No load-bearing self-citation or imported uniqueness theorem appears. The only mild concern is that some similarity measures are selected by their BLEU score on the same test set (e.g., overlap2 in Sections 4.2.1 and 4.2.5), which is test-set peeking rather than circularity, and the human-evaluation 'much better/worse' reversal (27 vs 40 for captioning; 36 vs 28 for translation) is disclosed by the paper and is a robustness limitation, not a circular step. The derivation chain is therefore self-contained with respect to circularity.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central claim does not depend on fitted parameters in the derivation, but the empirical evaluation selects the best similarity metric on the test set. The main assumptions are about the quality of beam search coverage and the chosen similarity proxies.

free parameters (1)
  • similarity measure selection = overlap2 for MT (test-set BLEU)
    In Section 4.2.5, the authors fix overlap2 as the best metric based on BLEU on newstest2014, i.e., selected on the test set rather than a held-out set.
assumptions (3)
  • domain assumption The model's probability distribution over sequences is a valid distribution, and beam search provides a representative sample of it.
    The method uses beam search to define both candidates and voters; this assumes the beam covers the relevant probability mass.
  • domain assumption N-gram overlap and averaged hidden-state cosine similarity are adequate proxies for semantic representativeness.
    The method relies on these similarity measures to define representativeness; if they are poor, the selected output may not be representative.
  • standard math Range voting satisfies independence of clones, which justifies using it to avoid vote splitting.
    The paper invokes Tideman's independence of clones and range voting properties as motivation for the approach.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Leveraging Sentence Similarity in Natural Language Generation: Improving Beam Search using Range Voting." pith.science (2026). https://pith.science/paper/GP2BILJL

@misc{pith2026190806288,
  author       = {Pith},
  title        = {Pith review of: Leveraging Sentence Similarity in Natural Language Generation: Improving Beam Search using Range Voting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GP2BILJL}},
  note         = {Machine review of arXiv:1908.06288}
}
read the original abstract

We propose a method for natural language generation, choosing the most representative output rather than the most likely output. By viewing the language generation process from the voting theory perspective, we define representativeness using range voting and a similarity measure. The proposed method can be applied when generating from any probabilistic language model, including n-gram models and neural network models. We evaluate different similarity measures on an image captioning task and a machine translation task, and show that our method generates longer and more diverse sentences, providing a solution to the common problem of short outputs being preferred over longer and more informative ones. The generated sentences obtain higher BLEU scores, particularly when the beam size is large. We also perform a human evaluation on both tasks and find that the outputs generated using our method are rated higher.

Figures

Figures reproduced from arXiv: 1908.06288 by the authors.

Figure 1
Figure 1. Image from the MSCOCO validation dataset [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The lengths (left) and the number of unique bigrams in the generated translations (right). Baseline [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Performance of overlap2 range voting, vary￾ing the number of voters for a fixed set of candidates. constraining the set of candidates to avoid model errors. This was similarly done by Tromble et al. (2008), who refer to the sets of voters and candi￾dates as the “evidence” and “hypothesis” spaces. For the voters, we increase k from 4 to 1000 and apply 3 different search methods: sampling k times, stochastic beam sear… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 52 canonical work pages

  1. [1]

    Palakorn Achananuparp, Xiaohua Hu, and Xiajiong Shen. 2008. The evaluation of sentence similarity measures. In Proceedings of the 10th International Conference on Data Warehousing and Knowledge Discovery , pages 305--316. Springer

    work page 2008

  2. [2]

    Steven Bird, Ewan Klein, and Edward Loper. 2009. https://www.nltk.org/ Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit . O'Reilly Media Inc

    work page 2009

  3. [3]

    Fr \'e d \'e ric Blain, Lucia Specia, and Pranava Madhyastha. 2017. http://aamt.info/app-def/S-102/mtsummit/2017/wp-content/uploads/sites/2/2017/09/MTSummitXVI_ResearchTrack.pdf Exploring hypotheses spaces in neural machine translation . In Proceedings of the 16th Machine Translation Summit (MT Summit XVI). Asia-Pacific Association for Machine Translation (AAMT)

    work page 2017

  4. [4]

    Ond r ej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Leveling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint-Amand, et al. 2014. https://www.aclweb.org/anthology/W14-3302 Findings of the 2014 Workshop on Statistical Machine Translation . In Proceedings of the Ninth Workshop on Statistical Machine Translation, pa...

    work page 2014

  5. [5]

    Peter F Brown, John Cocke, Stephen A Della Pietra, Vincent J Della Pietra, Frederick Jelinek, Jennifer C Lai, and Robert L Mercer. 1995. Method and system for natural language translation. US Patent 5,477,451

    work page 1995

  6. [6]

    Chris Callison-Burch, Miles Osborne, and Philipp Koehn. 2006. http://aclweb.org/anthology/E06-1032 Re-evaluation the role of BLEU in machine translation research . In Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics (EACL)

    work page 2006

  7. [7]

    Kris Cao and Stephen Clark. 2017. http://aclweb.org/anthology/E17-2029 Latent variable dialogue models and their diversity . In Proceedings of the 15 th Conference of the European Chapter of the Association for Computational Linguistics (EACL)

    work page 2017

  8. [8]

    Daniel Cer, Mona Diab, Eneko Agirre, I \ n igo Lopez-Gazpio, and Lucia Specia. 2017. http://aclweb.org/anthology/S17-2001 SemEval -2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation . In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017) , pages 1--14

    work page 2017

Show all 61 references
  1. [9]

    Yining Chen, Sorcha Gilroy, Kevin Knight, and Jonathan May. 2018. http://aclweb.org/anthology/N18-1205 Recurrent neural networks as weighted language recognizers . In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguist...

    2018

  2. [10]

    Eldan Cohen and Christopher Beck. 2019. http://proceedings.mlr.press/v97/cohen19a.html Empirical analysis of beam search performance degradation in neural sequence models . In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Mac...

    2019

  3. [11]

    Bhuwan Dhingra, Lihong Li, Xiujun Li, Jianfeng Gao, Yun-Nung Chen, Faisal Ahmed, and Li Deng. 2017. http://aclweb.org/anthology/P17-1045 Towards end-to-end reinforcement learning of dialogue agents for information access . In Proceedings of the 55th Annual Meeting of the Assoc...

    2017

  4. [12]

    Bryan Eikema and Wilker Aziz. 2020. https://arxiv.org/abs/2005.10283v1 Is MAP decoding all you need? the inadequacy of the mode in neural machine translation . Unpublished manuscript, arXiv preprint 2005.10283v1

    2020 arXiv

  5. [13]

    John D Emerson and Gary A Simon. 1979. https://www.jstor.org/stable/pdf/2683818.pdf Another look at the sign test when ties are present: The problem of confidence intervals . The American Statistician, 33(3):140--142

    1979

  6. [14]

    Angela Fan, Mike Lewis, and Yann Dauphin. 2018. https://doi.org/10.18653/v1/P18-1082 Hierarchical neural story generation . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889--898. Association for Compu...

    2018 doi

  7. [15]

    Markus Freitag and Yaser Al-Onaizan. 2017. http://aclweb.org/anthology/W17-3207 Beam search strategies for neural machine translation . In Proceedings of the First Workshop on Neural Machine Translation, pages 56--60

    2017

  8. [16]

    Markus Freitag, Isaac Caswell, and Scott Roy. 2019. https://www.aclweb.org/anthology/W19-5204 APE at scale and its implications on MT evaluation biases . In Proceedings of the Fourth Conference on Machine Translation (Volume 1: Research Papers), pages 34--44

    2019

  9. [17]

    Markus Freitag, David Grangier, and Isaac Caswell. 2020. https://arxiv.org/abs/2004.06063v1 BLEU might be guilty but references are not innocent . Unpublished manuscript, arXiv preprint 2004.06063v1

    2020 arXiv

  10. [18]

    Albert Gottlieb Heckscher. 1892. Bidrag til grundl ggelse af en afstemningsl re. Om methoderne ved udfindelse af stemmerflerhed i parlamenter (afsteming over ndringforslag m.v.) ved valg og domstole . Ph.D. thesis, University of Copenhagen

  11. [19]

    Sepp Hochreiter and J \"u rgen Schmidhuber. 1997. https://www.mitpressjournals.org/doi/abs/10.1162/neco.1997.9.8.1735 Long Short-Term Memory . Neural Computation, 9(8):1735--1780

    1997 doi

  12. [20]

    Maria Holmqvist, Sara Stymne, Jody Foo, and Lars Ahrenberg. 2009. https://www.aclweb.org/anthology/W09-0421 Improving alignment for smt by reordering and augmenting the training corpus . In Proceedings of the Fourth Workshop on Statistical Machine Translation, pages 120--124. ...

    2009

  13. [21]

    Ari Holtzman, Jan Buys, Maxwell Forbes, and Yejin Choi. 2020. https://openreview.net/forum?id=rygGQyrFvH The curious case of neural text degeneration . In Proceedings of the 8th International Conference on Learning Representations (ICLR)

    2020

  14. [22]

    Sergey Ioffe and Christian Szegedy. 2015. http://proceedings.mlr.press/v37/ioffe15.html Batch normalization: Accelerating deep network training by reducing internal covariate shift . In Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedi...

    2015

  15. [23]

    Philipp Koehn. 2004 a . Pharaoh: a beam search decoder for phrase-based statistical machine translation models. In Conference of the Association for Machine Translation in the Americas, pages 115--124. Springer

    2004

  16. [24]

    Philipp Koehn. 2004 b . https://www.aclweb.org/anthology/W04-3250 Statistical significance tests for machine translation evaluation . In Proceedings of the 2004 conference on empirical methods in natural language processing (EMNLP), pages 388--395

    2004

  17. [25]

    Philipp Koehn and Rebecca Knowles. 2017. http://aclweb.org/anthology/W17-3204 Six challenges for neural machine translation . In First Workshop on Neural Machine Translation, pages 28--39. Association for Computational Linguistics

    2017

  18. [26]

    Wouter Kool, Herke Van Hoof, and Max Welling. 2019. http://proceedings.mlr.press/v97/kool19a.html Stochastic beams and where to find them: The G umbel-top-k trick for sampling sequences without replacement . In Proceedings of the International Conference on Machine Learning, p...

    2019

  19. [27]

    Shankar Kumar and William Byrne. 2004. https://www.aclweb.org/anthology/N04-1022 Minimum B ayes- R isk decoding for statistical machine translation . In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational L...

    2004

  20. [28]

    Kuncheva

    Ludmila I. Kuncheva. 2004. Combining Pattern Classifiers: Methods and Algorithms. John Wiley & Sons

    2004

  21. [29]

    Kuncheva and Juan J Rodr \' guez

    Ludmila I. Kuncheva and Juan J Rodr \' guez. 2014. A weighted voting framework for classifiers ensembles. Knowledge and Information Systems, 38(2):259--275

    2014

  22. [30]

    David Kurokawa, Cyril Goutte, and Pierre Isabelle. 2009. http://www.mt-archive.info/MTS-2009-Kurokawa.pdf Automatic detection of translated text and its impact on machine translation . In Proceedings of the 12th Machine Translation Summit (MT Summit XII). Association for Machi...

    2009

  23. [31]

    Eerik Lagerspetz. 2016. Social Choice and Democratic Values. Springer

    2016

  24. [32]

    Gennadi Lembersky, Noam Ordan, and Shuly Wintner. 2012. https://www.mitpressjournals.org/doi/full/10.1162/COLI_a_00111 Language models for machine translation: Original vs. translated texts . Computational Linguistics, 38(4):799--825

    2012 doi

  25. [33]

    Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016 a . http://aclweb.org/anthology/N16-1014 A diversity-promoting objective function for neural conversation models . In Proceedings of the 2016 Conference of the North American Chapter of the Association...

    2016

  26. [34]

    Jiwei Li and Dan Jurafsky. 2016. https://arxiv.org/abs/1601.00372v2 Mutual information and diverse decoding improve neural machine translation . Unpublished manuscript, arXiv preprint 1601.00372v2

    2016 arXiv

  27. [35]

    Jiwei Li, Will Monroe, and Dan Jurafsky. 2016 b . https://arxiv.org/abs/1611.08562v2 A simple, fast diverse decoding algorithm for neural generation . Unpublished manuscript, arXiv preprint 1611.08562v2

    2016 arXiv

  28. [36]

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll \'a r, and C Lawrence Zitnick. 2014. Microsoft COCO : Common objects in context. In European Conference on Computer Vision, pages 740--755. Springer

    2014

  29. [37]

    Jonas Mueller and Aditya Thyagarajan. 2016. https://www.aaai.org/ocs/index.php/AAAI/AAAI16/paper/viewFile/12195/12023 Siamese recurrent architectures for learning sentence similarity . In Proceedings of the 30th AAAI Conference on Artificial Intelligence

    2016

  30. [38]

    Myle Ott, Michael Auli, David Grangier, and Marc'Aurelio Ranzato. 2018. http://proceedings.mlr.press/v80/ott18a.html Analyzing uncertainty in neural machine translation . In Proceedings of the 35th International Conference on Machine Learning (ICML)

    2018

  31. [39]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. http://aclweb.org/anthology/P02-1040 BLEU : a method for automatic evaluation of machine translation . In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics , pages 311--318

    2002

  32. [40]

    Atish Pawar and Vijay Mago. 2019. https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8630924 Challenging the boundaries of unsupervised learning for semantic similarity . IEEE Access, 7

    2019

  33. [41]

    Matt Post. 2018. https://www.aclweb.org/anthology/W18-6319 A call for clarity in reporting BLEU scores . In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186--191. Association for Computational Linguistics

    2018

  34. [42]

    Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. https://www.aclweb.org/anthology/P16-1162 Neural machine translation of rare words with subword units . In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p...

    2016

  35. [43]

    Hiroaki Shimizu, Masao Utiyama, Eiichiro Sumita, and Satoshi Nakamura. 2012. https://www.isca-speech.org/archive/iwslt_12/papers/sltc_117.pdf Minimum B ayes- R isk decoding extended with similar examples: NAIST-NICT at IWSLT 2012 . In International Workshop on Spoken Language ...

    2012

  36. [44]

    Warren D. Smith. 2000. https://rangevoting.org/WarrenSmithPages/homepage/rangevote.pdf Range voting . Unpublished manuscript

    2000

  37. [45]

    Warren D. Smith. 2005. https://rangevoting.org/WarrenSmithPages/homepage/rerange.pdf Reweighted range voting -- new multiwinner voting method . Unpublished manuscript

    2005

  38. [46]

    Felix Stahlberg and Bill Byrne. 2019. https://www.aclweb.org/anthology/D19-1331/ On NMT search errors and model errors: Cat got your tongue? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on ...

    2019

  39. [47]

    Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016. https://www.cv-foundation.org/openaccess/content_cvpr_2016/html/Szegedy_Rethinking_the_Inception_CVPR_2016_paper.html Rethinking the inception architecture for computer vision . In Procee...

    2016

  40. [48]

    Nicolaus Tideman. 1987. Independence of clones as a criterion for voting rules. Social Choice and Welfare, 4(3):185--206

    1987

  41. [49]

    Nicolaus Tideman. 2006. Collective Decisions and Voting: The Potential for Public Choice. Routledge

    2006

  42. [50]

    Roy Tromble, Shankar Kumar, Franz Och, and Wolfgang Macherey. 2008. https://www.aclweb.org/anthology/D08-1065 Lattice M inimum B ayes- R isk decoding for statistical machine translation . In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing...

    2008

  43. [51]

    Zhaopeng Tu, Yang Liu, Lifeng Shang, Xiaohua Liu, and Hang Li. 2017. https://www.aaai.org/ocs/index.php/AAAI/AAAI17/paper/viewPDFInterstitial/14161/14083 Neural machine translation with reconstruction . In Proceedings of the 31st AAAI Conference on Artificial Intelligence

    2017

  44. [52]

    Ashish Vaswani, Samy Bengio, Eugene Brevdo, Francois Chollet, Aidan Gomez, Stephan Gouws, Llion Jones, ukasz Kaiser, Nal Kalchbrenner, Niki Parmar, Ryan Sepassi, Noam Shazeer, and Jakob Uszkoreit. 2018. https://www.aclweb.org/anthology/W18-1819/ Tensor2Tensor for neural machin...

    2018

  45. [53]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. https://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf Attention is all you need . In Advances in neural information processing systems, p...

    2017

  46. [54]

    Ashwin Vijayakumar, Michael Cogswell, Ramprasaath Selvaraju, Qing Sun, Stefan Lee, David Crandall, and Dhruv Batra. 2018. https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/17329 Diverse beam search for improved description of complex scenes

    2018

  47. [55]

    Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2015. Show and tell: A neural image caption generator. In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 3156--3164. Institute of Electrical and Electronics Engineers

    2015

  48. [56]

    Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2017. https://doi.org/10.1109/TPAMI.2016.2587640 Show and tell: Lessons learned from the 2015 MSCOCO image captioning challenge . IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4):652--663

    2017

  49. [57]

    Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Klingner Jeff, Shah Apurva, Johnson Melvin, Liu Xiaobing, Kaiser ukasz, Gouws Stephan, Kato Yoshikiyo, Kudo Taku, Kazawa Hideto, Stevens Ke...

    2016 arXiv

  50. [58]

    Xinnuo Xu, Ond r ej Du s ek, Ioannis Konstas, and Verena Rieser. 2018. http://aclweb.org/anthology/D18-1432 Better conversations by modeling, filtering, and optimizing for coherence and diversity . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language ...

    2018

  51. [59]

    Zhen Xu, Bingquan Liu, Baoxun Wang, SUN Chengjie, Xiaolong Wang, Zhuoran Wang, and Chao Qi. 2017. http://aclweb.org/anthology/D17-1065 Neural response generation via GAN with an approximate embedding layer . In Proceedings of the 2017 Conference on Empirical Methods in Natural...

    2017

  52. [60]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  53. [61]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.