Pith. sign in

REVIEW 5 major objections 5 minor 51 references

LLMs expose most of a short sentence's keywords in the first two decoding steps; this paper claims that property, the Holographic Characteristic, can be read off and the rest of the sentence generated in parallel with comparable quality.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 06:31 UTC pith:SZAMV3AR

load-bearing objection Interesting observation and a plausible plug-in, but the key Markov assumption is unvalidated and the evidence is more suggestive than conclusive. the 5 major comments →

arxiv 2601.22546 v2 pith:SZAMV3AR submitted 2026-01-30 cs.CL cs.AI

Towards the Holographic Characteristic of LLMs for Efficient Short-text Generation

Shun Qian , Bingquan Liu , Chengjie Sun , Zhen Xu , Baoxun Wang This is my paper
classification cs.CL cs.AI
keywords Large language modelsHolographic Characteristickeyword extractionlexically constrained generationparallel decodinginference accelerationshort-text generationdialogue generation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to establish that language models carry most of a short sentence's semantic content—specifically its keywords—in the probability distribution of the very first generation steps, a tendency it names the Holographic Characteristic. It then argues that this trait can be used directly: read the first two decoding steps, extract the likely keywords, and fill the rest of the sentence in parallel with a separate constrained generator, no retraining or draft model required. On three Chinese dialogue datasets and three open LLMs of different sizes, the resulting HOLO plugin produces sentences comparable to the base models on dialogue quality, while cutting inference time by up to 92.6% and GPU memory by up to 61.7% for the larger models. The paper reads these results as evidence both that the Holographic Characteristic is real and that it is practically exploitable.

Core claim

The central discovery is the Holographic Characteristic: across three Chinese dialogue datasets and three models from 2.8B to 13B parameters, 40–65% of the tokens that end up in the generated response are already assigned to the top 1% of the vocabulary distribution at the first decoding step. The paper argues this means the beginning of generation is not a narrow prefix but a compressed sketch of the whole target sentence. On that basis, HOLO estimates each keyword's marginal probability from the first two steps, orders the keywords into chains using the same transition probabilities, and hands the chains to a lexically constrained insertion-based generator (POINTER with mask-predict) that

What carries the argument

The Holographic Characteristic is the enabling observation, defined as the tendency of LLMs to assign comparatively high probability to target-side keywords in the first inference step. The machine that turns it into text is a two-step estimator: the first-step probability distribution is treated as the initial state of a first-order Markov chain, and the second-step conditional distributions serve as the transition matrix (Eqs. 5–8), so the marginal probability of any keyword at any position is approximated by repeated multiplication of the transition matrix against the initial state. A nucleus-style top-p set (p=0.9) approximates the keyword vocabulary, a beam search over the transition pr

Load-bearing premise

All of HOLO's keyword estimates stand on the first-order Markov approximation that P(y_i | y_{i-1}, X) equals the first-to-second-step transition P(y_2 | y_1, X) for every position i, and on the claim that the top-p nucleus of the first step contains the true keyword set; if natural language generation departs from that stationarity, the extracted keywords are not the ones the model would actually use.

What would settle it

On a held-out set, run the base LLM to completion, extract the actual keyword set from the finished sentences, and check whether HOLO's first-two-step estimate recovers it substantially better than a random top-1% token set; if precision and recall are at chance, the Holographic Characteristic is not doing the work.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Any autoregressive LLM can be turned into a parallel short-text generator without retraining, distillation, or a draft model—just by reading the first two decoding steps.
  • Inference time and GPU memory for large models drop sharply: the paper reports 56.9%–92.6% time reduction and 55.5%–61.7% memory reduction for the two larger base models; the speed advantage grows as the base model gets slower.
  • The quality ceiling shifts from the LLM to the constrained generation layer: HOLO beats its base models on most automatic metrics but trails on BLEU-4 and perplexity, which the paper attributes to the lexically constrained generator.
  • Because only the first two steps are base-model-dependent, the remaining parallel generation steps can run on a much smaller, cheaper component, making the approach complementary to speculative sampling and KV-cache compression.
  • The Holographic Characteristic, if it holds, gives a direct way to probe what an LLM 'intends' to say: the opening distribution is a compressible semantic prior for the whole output.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Inference: The paper's two-step estimator is only as good as the first-order Markov approximation; a natural stress test is to track how estimated keyword probabilities drift as the target position grows, since a small auxiliary model could correct the drift without giving up the Holographic Characteristic.
  • Inference: If the Holographic Characteristic is a general property of next-token-prediction training, it should be measurable in other constrained generation settings—translation, summarization, and keyword-guided story writing—where the target side has clear keyword anchors; a probe there would tell whether the phenomenon is specific to dialogue or universal.
  • Inference: The paper's own bottleneck statement (the lexically constrained generator limits quality) suggests an obvious follow-up: swap in a stronger insertion-based or diffusion-style parallel generator while keeping the two-step keyword extractor fixed; if quality improves, the Holographic Characteristic is confirmed as the real carrier of the signal.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper claims that LLMs exhibit a 'Holographic Characteristic': target-side keywords are assigned disproportionately high probability during the first one or two decoding steps. Based on this, it proposes HOLO, a plugin that estimates target keyword probabilities from the first two step distributions, builds ordered keyword chains, and completes short text with a modified POINTER insertion model. Experiments on three Chinese dialogue datasets and three base LLMs report quality roughly comparable to the base models, while reducing inference latency and memory for two of the three base models.

Significance. If the Holographic Characteristic were rigorously established, the paper would offer an interesting, training-free route to parallel generation that does not require a draft model or an NAR-trained model. The paper has several strengths: it covers three different architectures/scales and three datasets, reports automatic and LLM-based human-like metrics, includes a mask-predict ablation, and transparently reports the efficiency trade-off for EVA2.0 in Table 5. However, the core estimation procedure and the main empirical evidence currently have load-bearing gaps, so the contribution is not yet established.

major comments (5)
  1. [§3.1, Eqs. (3)–(8) and (11)] The entire keyword estimator rests on the first-order stationary Markov approximation P(y_i|y_<i,X) ≈ P(y_i|y_{i-1},X) and P(y_i|y_{i-1},X) = P(y_2|y_1=y_{i-1},X). Natural language is not a first-order time-homogeneous Markov process, and the paper provides no empirical validation beyond the assertion that V_p(1) 'dominates the most probability'. The paper even acknowledges that 'with the increase of position i, the estimation bias will also inevitably increase' but never quantifies it. I ask for a direct validation: on a held-out set, compare the estimated P(y_i|X) from the two-step Markov chain with the model's actual next-token distributions at positions i=2,3,..., and report divergence or keyword overlap. Without this, the P(w|X) used to extract keywords may be unfaithful.
  2. [§3.1, Eq. (10)] The displayed inequality is incorrect. For a fixed y2, the quantity ∑_{y1∉V_p(1)} P(y2|y1,X) is not bounded by 1; it can be as large as |V \ V_p(1)|. Thus the step '≤ ∑_{V_p(1)}… + (1−p) ∑_{V−V_p(1)} P(y2|y1,X)' is not a valid bound. The subsequent claim that this sum is close to (|V|−|V_p(1)|)/|V| is also unsubstantiated. A valid bound would need something like ∑_{V−V_p(1)} P(y1|X)P(y2|y1,X) ≤ (1−p) max_{y1∉V_p(1)} P(y2|y1,X) ≤ (1−p), or a different decomposition. This should be corrected before the estimator is used.
  3. [Table 1] The empirical evidence for the central claim is partly circular. The 'ground-truth keywords' are extracted from sentences generated by the very same LLM whose first-step distribution is being tested. A high overlap between those keywords and the model's own top-1% tokens may reflect self-consistency of the model's sampling distribution rather than a property of genuinely target-side semantic content. There is also no chance baseline: with a 50k–100k vocabulary, a random top 1% set would already give a small non-zero overlap. Please (a) describe the 'simple statistical method' used for keyword extraction, (b) add a chance baseline, and (c) re-run the experiment with human-annotated keywords or with reference responses, or at least with keywords extracted from a different model.
  4. [§4.2.1, Table 3] The sentence 'the HOLO plugin outperforms the corresponding base models on 5 out of 7 metrics' is not supported by Table 3. In several groups HOLO is worse on F1 (Weibo/EVA, LCCC/EVA), on BLEU-4 in five of the nine groups, on PPL in five of the nine groups, and on Distinct-2 in three groups. Aggregating across all groups into a single '5/7' claim is misleading. Please report per-dataset results honestly, add significance tests if possible, and state which of the 9 groups improve/worsen on each metric.
  5. [§3.2.1, Eq. (12)] The keyword-chain probability formula multiplies the transition probability by an additional P_F(w=c_i|X) for every keyword after the first. As written, P(C|X) is not a proper probability of the chain; it double-counts the marginal keyword probability and may distort the ranking of candidate chains. Either define P_F precisely and justify the product, or use the standard Markov chain probability P(y1=c1|X)∏_{i=2}^m P(y2=c_i|y1=c_{i-1},X).
minor comments (5)
  1. [§4.2.2] The text refers to 'human evaluation' in some places, but the results are produced by G-Eval using GPT-4. Please consistently call this 'LLM-based evaluation' or 'human-like evaluation', and note the limitations of using the same type of model family for evaluation.
  2. [§4.1.3 and Appendix A] The Chinese POINTER pre-training description lacks concrete hyperparameters (learning rate, batch size, number of steps, mask-predict iterations, importance-score weights). Please provide these for reproducibility.
  3. [§4.2.3] The complexity expression O(|V_y| + log N / |C|) is unclear: please define all variables and explain why the number of one-step inferences is independent of |V|, especially since the method uses full vocabulary distributions in the first two steps.
  4. [Table 1] The caption says 'top 1% probability' but does not state the vocabulary size or the number of candidate tokens per model. Please include this information, and ideally report standard deviations or results over multiple sampling runs.
  5. [Eq. (11)] The notation T is introduced as 'the step of the generation procedure'; it should be defined as the maximum target length N used in Eq. (2), otherwise the average over positions is ambiguous.

Circularity Check

0 steps flagged

No significant circularity: the paper's derivation is an explicit approximation chain with independent downstream evaluation; the unvalidated first-order Markov assumption is a correctness risk, not circularity.

full rationale

The central pipeline estimates P_M(w|X) via Eqs. 2–11 using an explicit first-order Markov assumption (Eq. 3: P(y_i|y_<i,X) ≈ P(y_i|y_{i-1},X)) and time-homogeneity (Eq. 4). These are stated approximations, not results derived from the target claim; no equation reduces to itself. Table 1 measures whether tokens appearing in the model's own generated sentences fall in the top 1% of the first-step distribution; this is an internal consistency check, not a fitted parameter renamed as a prediction, and the overlap is not forced by construction. The HOLO plugin's output quality (Tables 3–6) is evaluated against independent text-quality metrics and human-like G-Eval, not against the extracted keyword set, so the evaluation is not circular. The paper contains no load-bearing self-citations; references [13,51] for keyword-guided generation and [50] for POINTER are external. The main weaknesses—unvalidated Markov assumption and the internal nature of the Table 1 ground-truth—are correctness/validity concerns rather than circularity, so the score is 0.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The paper contributes a new empirical label and a plug-in, but the central claim rests on an unvalidated Markov approximation, several hand-set thresholds tuned on the test data, and a separately trained POINTER model that is not released.

free parameters (5)
  • nucleus cutoff p = 0.9
    Eq. 9; heuristic choice for V1^(p); no sensitivity analysis is reported.
  • max keyword-chain length L = 7 (EVA2.0-2.8B), 12 (ChatGLM-6B), 10 (Belle-13B)
    Chosen 'based on the average keywords number of generated sentences on the Douban dataset' (§4.1.3), i.e., tuned on the evaluation set.
  • number of keyword chains Z = 5
    Set in §4.1.3 without ablation.
  • top-k for final keyword selection = not specified
    §3.1 says 'top-k probable words' but k is never reported.
  • importance-score weights for Chinese POINTER pre-training = not specified
    Appendix A defines alpha_t as sum of TF-IDF, POS, and YAKE scores but gives no weights or normalization details.
axioms (5)
  • ad hoc to paper Autoregressive LLM generation is a stationary first-order Markov process; P(y_i|y_<i,X) ≈ P(y_2|y_1=y_{i-1},X) for all i (Eqs. 3-4).
    Load-bearing for keyword estimation; no empirical validation is provided.
  • domain assumption The keyword vocabulary V_y is small (<100) and occupies most of the probability mass of the first-step distribution (Section 3.1).
    Needed to justify replacing the full-vocabulary sum with V1^(p); not measured for the test data.
  • domain assumption The POINTER insertion model (and the authors' Chinese POINTER) can produce fluent sentences from arbitrary keyword chains.
    HOLO output quality depends on this; no independent evaluation of Chinese POINTER is given.
  • domain assumption GPT-4-based G-Eval scores approximate human judgments of informativeness, coherence, and humanness.
    Cited [28]; used as 'human-like' evaluation without human validation on this data.
  • domain assumption The BERT ranker fine-tuned following [19] reliably selects the best candidate sentence.
    Only P@1/10=0.86 is reported; no analysis of ranking failures is given.

pith-pipeline@v1.3.0-alltime-deepseek · 14105 in / 16883 out tokens · 167086 ms · 2026-08-03T06:31:04.846542+00:00 · methodology

0 comments
read the original abstract

The recent advancements in Large Language Models (LLMs) have attracted interest in exploring their in-context learning abilities and chain-of-thought capabilities. However, there are few studies investigating the specific traits related to the powerful generation capacity of LLMs. This paper aims to delve into the generation characteristics exhibited by LLMs. Through our investigation, we have discovered that language models tend to capture target-side keywords at the beginning of the generation process. We name this phenomenon the Holographic Characteristic of language models. For the purpose of exploring this characteristic and further improving the inference efficiency of language models, we propose a plugin called HOLO, which leverages the Holographic Characteristic to extract target-side keywords from language models within a limited number of generation steps and complements the sentence with a parallel lexically constrained text generation method. To verify the effectiveness of HOLO, we conduct massive experiments on language models of varying architectures and scales in the short-text generation scenario. The results demonstrate that HOLO achieves comparable performance to the baselines in terms of both automatic and human-like evaluation metrics and highlight the potential of the Holographic Characteristic.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

51 extracted references · 1 canonical work pages

  1. [1]

    Gqa: Training generalized multi-query transformer models from multi-head checkpoints, in: The 2023 Conference on Empirical Methods in Natural Language Processing

    Ainslie, J., Lee-Thorp, J., de Jong, M., Zemlyanskiy, Y., Lebron, F., Sanghai, S., 2023. Gqa: Training generalized multi-query transformer models from multi-head checkpoints, in: The 2023 Conference on Empirical Methods in Natural Language Processing

    2023

  2. [2]

    Palm2technical report

    Anil,R.,Dai,A.M.,Firat,O.,Johnson,M.,Lepikhin,D.,Passos,A.,Shakeri,S.,Taropa,E.,Bailey,P.,Chen,Z.,etal.,2023. Palm2technical report. arXiv preprint arXiv:2305.10403

    Pith/arXiv arXiv 2023

  3. [3]

    Plato-2:Towardsbuildinganopen-domainchatbotvia curriculum learning, in: Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp

    Bao,S.,He,H.,Wang,F.,Wu,H.,Wang,H.,Wu,W.,Guo,Z.,Liu,Z.,Xu,X.,2021a. Plato-2:Towardsbuildinganopen-domainchatbotvia curriculum learning, in: Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 2513–2525

    2021

  4. [4]

    Bao, Y., Huang, S., Xiao, T., Wang, D., Dai, X., Chen, J., 2021b. Non-autoregressive translation by learning target categorical codes, in: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 5749–5759

    2021

  5. [5]

    Glat:Glancingatlatentvariablesforparalleltextgeneration, in: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp

    Bao,Y.,Zhou,H.,Huang,S.,Wang,D.,Qian,L.,Dai,X.,Chen,J.,Li,L.,2022. Glat:Glancingatlatentvariablesforparalleltextgeneration, in: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 8398–8409

    2022

  6. [6]

    Sparks of artificial general intelligence: Early experiments with gpt-4

    Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y.T., Li, Y., Lundberg, S., et al., 2023. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712

    Pith/arXiv arXiv 2023

  7. [7]

    Medusa: Simple llm inference acceleration framework with multiple decoding heads

    Cai, T., Li, Y., Geng, Z., Peng, H., Lee, J.D., Chen, D., Dao, T., 2024. Medusa: Simple llm inference acceleration framework with multiple decoding heads. arXiv preprint arXiv:2401.10774

    Pith/arXiv arXiv 2024

  8. [8]

    YAKE! Collection-Independent Automatic Keyword Extractor

    Campos, R., Mangaravite, V., Pasquali, A., Jorge, A.M., Nunes, C., Jatowt, A., 2018. YAKE! Collection-Independent Automatic Keyword Extractor. p. 806–810. URL:http://dx.doi.org/10.1007/978-3-319-76941-7_80, doi:10.1007/978-3-319-76941-7_80

    work page doi:10.1007/978-3-319-76941-7_80 2018

  9. [9]

    Accelerating large language model decoding with speculative sampling

    Chen, C., Borgeaud, S., Irving, G., Lespiau, J.B., Sifre, L., Jumper, J., 2023a. Accelerating large language model decoding with speculative sampling. arXiv preprint arXiv:2302.01318

    Pith/arXiv arXiv

  10. [10]

    Learning to memorize entailment and discourse relations for persona-consistent dialogues, in: Proceedings of the AAAI conference on artificial intelligence, pp

    Chen, R., Wang, J., Yu, L.C., Zhang, X., 2023b. Learning to memorize entailment and discourse relations for persona-consistent dialogues, in: Proceedings of the AAAI conference on artificial intelligence, pp. 12653–12661

  11. [11]

    Palm: Scaling language modeling with pathways

    Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H.W., Sutton, C., Gehrmann, S., et al., 2022. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311

    Pith/arXiv arXiv 2022

  12. [12]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Devlin, J., Chang, M.W., Lee, K., Toutanova, K., 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

    Pith/arXiv arXiv 2018

  13. [13]

    Wizard of wikipedia: Knowledge-powered conversational agents, in: International Conference on Learning Representations

    Dinan, E., Roller, S., Shuster, K., Fan, A., Auli, M., Weston, J., 2019. Wizard of wikipedia: Knowledge-powered conversational agents, in: International Conference on Learning Representations

    2019

  14. [14]

    Understandingandimprovinglexicalchoiceinnon-autoregressivetranslation, in: International Conference on Learning Representations

    Ding,L.,Wang,L.,Liu,X.,Wong,D.F.,Tao,D.,Tu,Z.,2020. Understandingandimprovinglexicalchoiceinnon-autoregressivetranslation, in: International Conference on Learning Representations

    2020

  15. [15]

    Gptq: Accurate post-training quantization for generative pre-trained transformers

    Frantar, E., Ashkboos, S., Hoefler, T., Alistarh, D., 2022. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323

    Pith/arXiv arXiv 2022

  16. [16]

    How does gpt obtain its ability? tracing emergent abilities of language models to their sources

    Fu, Y., Peng, H., Khot, T., 2022. How does gpt obtain its ability? tracing emergent abilities of language models to their sources. Yao Fu’s Notion

    2022

  17. [17]

    Ghazvininejad, M., Levy, O., Liu, Y., Zettlemoyer, L., 2019. Mask-predict: Parallel decoding of conditional masked language models, in: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 6112–6121

    2019

  18. [18]

    Non-autoregressive neural machine translation, in: International Conference on Learning Representations (ICLR)

    Gu, J., Bradbury, J., Xiong, C., Li, V., Socher, R., 2018. Non-autoregressive neural machine translation, in: International Conference on Learning Representations (ICLR)

    2018

  19. [19]

    Gu, Y., Wen, J., Sun, H., Song, Y., Ke, P., Zheng, C., Zhang, Z., Yao, J., Liu, L., Zhu, X., et al., 2023. Eva2. 0: Investigating open-domain chinese dialogue systems with large-scale pre-training. Machine Intelligence Research 20, 207–219

    2023

  20. [20]

    He, H., Choi, J.D., 2021. The stem cell hypothesis: Dilemma behind multi-task learning with transformer encoders, in: Proceedings of the 2021ConferenceonEmpiricalMethodsinNaturalLanguageProcessing,AssociationforComputationalLinguistics,OnlineandPuntaCana, Dominican Republic. pp. 5555–5577. URL:https://aclanthology.org/2021.emnlp-main.451

    2021

  21. [21]

    Improvingnon-autoregressivetranslationmodelswithoutdistillation,in:InternationalConference on Learning Representations

    Huang,X.S.,Perez,F.,Volkovs,M.,2022. Improvingnon-autoregressivetranslationmodelswithoutdistillation,in:InternationalConference on Learning Representations

    2022

  22. [22]

    Belle: Be everyone’s large language model engine.https://github

    Ji, Y., Deng, Y., Gong, Y., Peng, Y., Niu, Q., Ma, B., Li, X., 2023. Belle: Be everyone’s large language model engine.https://github. com/LianjiaTech/BELLE. Shun Qian et al.:Preprint submitted to ElsevierPage 10 of 11 Towards the Holographic Characteristic of LLMs for Efficient Short-text Generation

    2023

  23. [23]

    Squeezellm:Dense-and-sparsequantization

    Kim,S.,Hooper,C.,Gholami,A.,Dong,Z.,Li,X.,Shen,S.,Mahoney,M.W.,Keutzer,K.,2023. Squeezellm:Dense-and-sparsequantization. arXiv preprint arXiv:2306.07629

    Pith/arXiv arXiv 2023

  24. [24]

    Efficient memory management for large language model serving with pagedattention, in: Proceedings of the 29th Symposium on Operating Systems Principles, pp

    Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C.H., Gonzalez, J., Zhang, H., Stoica, I., 2023. Efficient memory management for large language model serving with pagedattention, in: Proceedings of the 29th Symposium on Operating Systems Principles, pp. 611–626

    2023

  25. [25]

    Fast inference from transformers via speculative decoding, in: International Conference on Machine Learning, PMLR

    Leviathan, Y., Kalman, M., Matias, Y., 2023. Fast inference from transformers via speculative decoding, in: International Conference on Machine Learning, PMLR. pp. 19274–19286

    2023

  26. [26]

    A diversity-promoting objective function for neural conversation models

    Li, J., Galley, M., Brockett, C., Gao, J., Dolan, B., 2015. A diversity-promoting objective function for neural conversation models. arXiv preprint arXiv:1510.03055

    Pith/arXiv arXiv 2015

  27. [27]

    Lin,C.Y.,Och,F.J.,2004. Automaticevaluationofmachinetranslationqualityusinglongestcommonsubsequenceandskip-bigramstatistics, in: Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04), pp. 605–612

    2004

  28. [28]

    G-eval: Nlg evaluation using gpt-4 with better human alignment, in: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp

    Liu, Y., Iter, D., Xu, Y., Wang, S., Xu, R., Zhu, C., 2023. G-eval: Nlg evaluation using gpt-4 with better human alignment, in: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 2511–2522

    2023

  29. [29]

    Are emergent abilities in large language models just in-context learning? arXiv preprint arXiv:2309.01809

    Lu, S., Bigoulaeva, I., Sachdeva, R., Madabushi, H.T., Gurevych, I., 2023. Are emergent abilities in large language models just in-context learning? arXiv preprint arXiv:2309.01809

    Pith/arXiv arXiv 2023

  30. [30]

    arXiv preprint arXiv:2305.09781

    Miao,X.,Oliaro,G.,Zhang,Z.,Cheng,X.,Wang,Z.,Wong,R.Y.Y.,Chen,Z.,Arfeen,D.,Abhyankar,R.,Jia,Z.,2023.Specinfer:Accelerating generative llm serving with speculative inference and token tree verification. arXiv preprint arXiv:2305.09781

    Pith/arXiv arXiv 2023

  31. [31]

    Bleu: a method for automatic evaluation of machine translation, in: Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp

    Papineni, K., Roukos, S., Ward, T., Zhu, W.J., 2002. Bleu: a method for automatic evaluation of machine translation, in: Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311–318

    2002

  32. [32]

    Guiding non-autoregressive neural machine translation decoding with reordering information, in: Proceedings of the AAAI Conference on Artificial Intelligence, pp

    Ran, Q., Lin, Y., Li, P., Zhou, J., 2021. Guiding non-autoregressive neural machine translation decoding with reordering information, in: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 13727–13735

    2021

  33. [33]

    Non-autoregressive machine translation with latent alignments, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp

    Saharia, C., Chan, W., Saxena, S., Norouzi, M., 2020. Non-autoregressive machine translation with latent alignments, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1098–1108

    2020

  34. [34]

    Are emergent abilities of large language models a mirage? Advances in Neural Information Processing Systems 36

    Schaeffer, R., Miranda, B., Koyejo, S., 2024. Are emergent abilities of large language models a mirage? Advances in Neural Information Processing Systems 36

    2024

  35. [35]

    Shang, L., Lu, Z., Li, H., 2015. Neural responding machine for short-text conversation, in: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 1577–1586

    2015

  36. [36]

    Accelerating llm inference with staged speculative decoding, in: Workshop on Efficient Systems for Foundation Models@ ICML2023

    Spector, B.F., Re, C., 2023. Accelerating llm inference with staged speculative decoding, in: Workshop on Efficient Systems for Foundation Models@ ICML2023

    2023

  37. [37]

    Llama: Open and efficient foundation language models

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al., 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    Pith/arXiv arXiv 2023

  38. [38]

    Alarge-scalechineseshort-textconversationdataset,in:NLPCC

    Wang,Y.,Ke,P.,Zheng,Y.,Huang,K.,Jiang,Y.,Zhu,X.,Huang,M.,2020. Alarge-scalechineseshort-textconversationdataset,in:NLPCC. URL:https://arxiv.org/abs/2008.03946

    Pith/arXiv arXiv 2020

  39. [39]

    Wang, Y., Tian, F., He, D., Qin, T., Zhai, C.X., Liu, T.Y., 2019. Non-autoregressive machine translation with auxiliary regularization, in: ProceedingsoftheThirty-ThirdAAAIConferenceonArtificialIntelligenceandThirty-FirstInnovativeApplicationsofArtificialIntelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, p...

    2019

  40. [40]

    Emergent abilities of large language models

    Wei,J.,Tay,Y.,Bommasani,R.,Raffel,C.,Zoph,B.,Borgeaud,S.,Yogatama,D.,Bosma,M.,Zhou,D.,Metzler,D.,etal.,2022a. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682

    Pith/arXiv arXiv

  41. [41]

    Chain of thought prompting elicits reasoning in large language models

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., Chi, E., Le, Q., Zhou, D., 2022b. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903

    Pith/arXiv arXiv

  42. [42]

    Wu, Y., Wu, W., Xing, C., Zhou, M., Li, Z., 2017. Sequential matching network: A new architecture for multi-turn response selection in retrieval-based chatbots, in: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 496–505

    2017

  43. [43]

    Smoothquant: Accurate and efficient post-training quantization for large language models, in: International Conference on Machine Learning, PMLR

    Xiao, G., Lin, J., Seznec, M., Wu, H., Demouth, J., Han, S., 2023a. Smoothquant: Accurate and efficient post-training quantization for large language models, in: International Conference on Machine Learning, PMLR. pp. 38087–38099

  44. [44]

    Asurveyonnon-autoregressivegenerationforneuralmachinetranslation and beyond

    Xiao,Y.,Wu,L.,Guo,J.,Li,J.,Zhang,M.,Qin,T.,Liu,T.y.,2023b. Asurveyonnon-autoregressivegenerationforneuralmachinetranslation and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence

  45. [45]

    An explanation of in-context learning as implicit bayesian inference, in: International Conference on Learning Representations

    Xie, S.M., Raghunathan, A., Liang, P., Ma, T., 2021. An explanation of in-context learning as implicit bayesian inference, in: International Conference on Learning Representations

    2021

  46. [46]

    GLM-130b: An open bilingual pre-trained model, in: The Eleventh International Conference on Learning Representations (ICLR)

    Zeng, A., Liu, X., Du, Z., Wang, Z., Lai, H., Ding, M., Yang, Z., Xu, Y., Zheng, W., Xia, X., Tam, W.L., Ma, Z., Xue, Y., Zhai, J., Chen, W., Liu, Z., Zhang, P., Dong, Y., Tang, J., 2023. GLM-130b: An open bilingual pre-trained model, in: The Eleventh International Conference on Learning Representations (ICLR). URL:https://openreview.net/forum?id=-Aw0rrrPUF

    2023

  47. [47]

    Glm-130b: An open bilingual pre-trained model

    Zeng, A., Liu, X., Du, Z., Wang, Z., Lai, H., Ding, M., Yang, Z., Xu, Y., Zheng, W., Xia, X., et al., 2022. Glm-130b: An open bilingual pre-trained model. arXiv preprint arXiv:2210.02414

    Pith/arXiv arXiv 2022

  48. [48]

    Opt: Open pre-trained transformer language models

    Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X.V., et al., 2022. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068

    Pith/arXiv arXiv 2022

  49. [49]

    Zhang,Y.,Sun,S.,Galley,M.,Chen,Y.C.,Brockett,C.,Gao,X.,Gao,J.,Liu,J.,Dolan,B.,2020a.Dialogpt:Large-scalegenerativepre-training for conversational response generation, in: ACL, system demonstration

  50. [50]

    Zhang, Y., Wang, G., Li, C., Gan, Z., Brockett, C., Dolan, W.B., 2020b. Pointer: Constrained progressive text generation via insertion-based generative pre-training, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 8649– 8670

    2020

  51. [51]

    Kpt: Keyword-guided pre-training for grounded dialog generation

    Zhu, Q., Mi, F., Zhang, Z., Wang, Y., Li, Y., Jiang, X., Liu, Q., Zhu, X., Huang, M., 2022. Kpt: Keyword-guided pre-training for grounded dialog generation. arXiv preprint arXiv:2212.01739 . Shun Qian et al.:Preprint submitted to ElsevierPage 11 of 11

    Pith/arXiv arXiv 2022