Pith. sign in

REVIEW 4 major objections 5 minor 3 cited by

An Engorgio Prompt Makes Large Language Model Babble on

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A short optimized prompt can push autoregressive LLMs to generate until their maximum length, multiplying per-query inference cost by 2–13×.

desk verdict A useful and technically solid attack paper whose headline numbers are real but temperature-dependent; the abstract overclaims without the temperature caveat. read the letter →

arxiv 2412.19394 v2 pith:MKMMPNKK submitted 2024-12-27 cs.CR cs.AI

Jianshuo Dong , Ziyuan Zhang , Qingjie Zhang , Tianwei Zhang , Hao Wang , Hewu Li , Qi Li , Chao Zhang
show 2 more authors
Ke Xu Han Qiu
This is my paper · ORCID
classification cs.CRcs.AI
keywords inferencecostattackadversarialpromptsend-of-sequencetokenautoregressivelanguagemodelsGumbel-SoftmaxoptimizationserviceavailabilitypromptLLMdenialof
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a short, carefully optimized prompt can make modern decoder-only large language models generate text until they hit the pre-set maximum output length, turning a single query into a much larger compute bill. The attack, called Engorgio, works by suppressing the probability of the end-of-sequence () token along the generation path, so the model does not stop talking. On six base models the optimized prompts reach roughly 90–100% of the maximum length, while normal queries reach only 0–40%; across all 13 tested models the claimed effect is 2–13× higher inference cost per query than normal inputs. The authors argue this is a practical availability threat because many services that deploy open-weight models rate-limit requests rather than tokens.

What carries the argument

The core machinery is a parameterized proxy distribution $\theta \in \mathbb{R}^{S \times V}$ over the whole context, where $S$ is the maximum allowable length and $V$ is the vocabulary size. In the generation stage, each row is normalized with Gumbel-Softmax — a differentiable relaxation of sampling from a categorical distribution — and used as weights to aggregate token embeddings into a soft embedding $E(\theta)$ that is fed to the model, so gradients flow back into $\theta$. Two losses drive the update: the <EOS> escape loss $\mathcal{L}_{\mathrm{esc}}(\theta) = \sum_i \operatorname{Softmax}(f_\Theta(E(\theta)_{1:i}))_\kappa$, which penalizes the softmax probability of the <EOS> token at every position, and the self-mentor loss $\mathcal{L}_{\mathrm{sm}}(\theta)$, which cross-entropy-matches the proxy distribution's predictions against the model's own predictions to keep the optimized sequence self-consistent. After optimization, the distribution typically converges to a single high-probability token sequence, and that sequence is sampled and sent as the Engorgio prompt.

What would settle it

Decode the released Engorgio prompts on LLaMA-7B (max length 1024) at temperatures 0.1, 0.3, 0.5, and 0.7 and compare Avg-len and Avg-rate with the paper's Table 3; a decisive negative result would be near-normal output lengths at default API sampling temperature, showing the attack does not generalize beyond carefully chosen decoding settings.

Watch

Extended reading notes

Core claim

The central claim is that an untargeted objective — penalizing the <EOS> token everywhere it might appear — is sufficient to control the length of autoregressive generation. The authors show that optimizing a Gumbel-Softmax parameterized proxy distribution over a short prompt yields a discrete prompt that lowers <EOS> probability across all positions, making LLaMA, LLaMA-2, OPT, GPT-2, and SFT variants such as Alpaca, Vicuna, Koala, StableLM, Orca, Samantha, and ChatGLM emit outputs near the maximum allowable length: 90%+ of the maximum length on base models, and near-maximum on SFT models when a semantic prefix is added. A real-world deployment on a Hugging Face inference endpoint shows that a small fraction of attack queries sharply increases queueing latency for normal users and nearly cuts throughput. The paper interprets the length increase as a lower bound on cost increase, since Transformer inference cost grows super-linearly with response length.

Load-bearing premise

The load-bearing premise is that suppressing the <EOS> token on an optimized soft sequence transfers to the model's actual token-by-token sampling; if the sampled tokens deviate from that trajectory — for example at high sampling temperature — the suppression can fail.

Editorial extensions

If this is right

  • If the central claim is right, any service that deploys an open-weight LLM and rate-limits by request rather than by tokens can be forced to spend near-maximum compute per query, making request-level limits the main attack surface.
  • Base models (LLaMA-30B, LLaMA-7B, LLaMA-2-7B, OPT-125M, OPT-1.3B, GPT2-large) can be driven to roughly 90–100% of maximum output length, versus 0–40% for normal inputs, so the effect holds across model families and scales from 125M to 30B parameters.
  • SFT models, which tend to give short answers, need a semantic prefix fused with the Engorgio prompt to reach near-maximum lengths; without it some models respond with gibberish or 'not understand'.
  • Engorgio prompts transfer between related models (e.g., Vicuna-crafted prompts achieve 96% Avg-rate on Koala), which extends the attack to black-box settings where the attacker only controls a proxy model.
  • Because each additional generated token costs a new forward pass over a longer context, the reported 2–13× output-length increase corresponds to a super-linear inference-cost increase, so the cost ratios are a lower bound.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An unstated consequence is that the same <EOS>-suppression recipe could be applied to coherent-looking prompts: if Engorgio is woven into natural instructions, the already high false-positive rate of perplexity filtering would make detection even harder.
  • The paper's temperature results point to a cheap mitigation it does not pursue: raising sampling temperature or using diverse decoding for suspiciously long generations would break the optimized EOS-suppression trajectory, at some cost in response quality.
  • The mechanism is broader than text completion: any autoregressive model with a stop token and request-level rate limits, such as code completion or multimodal generation, could be stretched by the same untargeted escape-loss optimization.
  • A production-scheduler replication would clarify whether prompt caching and continuous batching absorb the extra tokens, which the paper's simplified queuing model does not address.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Engorgio, a white-box inference-cost attack against autoregressive large language models. The attacker optimizes a Gumbel-Softmax parameterized distribution over a prompt and a planned future context, using an <EOS>-escape loss (Eq. 6) and a self-mentor loss (Eq. 7), then samples a discrete prompt and sends it to the victim. Experiments on 13 open-source models report that Engorgio prompts drive outputs to roughly 84-100% of the configured maximum length at the temperature used in the main tables, compared with much shorter outputs for normal inputs and prior baselines. The paper also reports transferability across related models, an ablation study, a defense analysis, and a Hugging Face inference-endpoint case study showing reduced throughput and increased queueing for normal clients.

Significance. If the stated effectiveness holds across realistic deployment conditions, Engorgio is a meaningful advance in the inference-cost-attack literature: it adapts sponge-style attacks from encoder-decoder models to decoder-only autoregressive LLMs, contributes a reusable optimization procedure, and evaluates on a broad and relevant model zoo. The code release and the real-world endpoint study are valuable assets for reproducing and stress-testing the attack. However, the significance is currently qualified by the paper's own temperature study: the 90-100% claims are established for low-temperature decoding, while SFT models degrade sharply at temperature 0.7. The central methodological idea is sound, but the unqualified abstract and contribution statements overstate the attack's robustness for typical chat-service decoding settings.

major comments (4)
  1. [Section 4.1 (Table 1) and Section 4.4 (Table 3)] The main results in Table 1 are reported without a stated decoding temperature, but the StableLM, Samantha, and ChatGLM rows match the temperature-0.1 rows of Table 3 almost exactly (StableLM 1021.6/98%, Samantha 944.0/89%, ChatGLM 979.6/95%). This indicates that the headline SFT results were produced at temperature 0.1, yet the abstract and Section 4.2 claim 'roughly 90-100%' without that qualification. At temperature 0.7, Table 3 shows Avg-rate collapsing to 33%, 24%, and 71% for those same models, and Table 4 shows that adding a semantic prefix restores StableLM to 83% but leaves Alpaca at only 43%. Because many production chat services sample at temperatures around 0.6-1.0, the abstract's unqualified central claim is not supported for SFT models under typical decoding settings. This also exposes the fragility of the core transfer assumption: <EOS> suppression is optimized on the fixed soft trajectory E(θ) in Eq. (6), but it does not reliably transfer to stochastic autoregressive decoding at higher temperatures. Please state the default temperature in Section 4.1, qualify the abstract and conclusion accordingly, or add experiments at the temperatures used by the services discussed in Appendix A.2.
  2. [Section 4.1 and Appendix A.5] The evaluation reports Avg-len and Avg-rate without any measure of dispersion. Appendix A.5 mentions that Avg-len is computed by sampling 100 times, but no standard deviation, confidence interval, or number of repetitions is provided for any table, and the sample size underlying Avg-rate is not specified. Given that Table 3 shows large swings across neighboring temperatures (e.g., StableLM falls from 62% at temperature 0.3 to 28% at 0.5), the paper's claims of 'stable' suppression need error bars, per-run statistics, or at least a statement of the number of independent runs before the reader can distinguish systematic degradation from sampling noise.
  3. [Section 4.4] The sentence 'For talkative base models, they are tempted to respond endlessly when a high temperature of 0.7 is used' asserts temperature robustness for base models, but Table 3 reports data only for SFT models. No base-model experiments at temperature 0.7 are presented in the main text or appendices. Either provide those results, or remove or qualify the sentence so that the reader does not infer base-model high-temperature behavior that the paper does not demonstrate.
  4. [Section 4.5 and Figure 3] The real-world case study does not report the decoding temperature, sampling parameters, or the number of requests contributing to each bar in Figure 3, and the figure has no error bars. Since Table 3 establishes that Engorgio's effectiveness is strongly temperature-dependent for SFT models such as StableLM, the case study is not interpretable without stating the endpoint's temperature and sampling configuration. Please add this information and, ideally, compare at least two temperatures that match the threat scenarios described in Appendix A.2.
minor comments (5)
  1. [Abstract] The abstract contains the typo 'Engergio' in the sentence 'our real-world experiment demonstrates Engergio's threat'; it should be 'Engorgio'.
  2. [Eq. (6)] The notation 'Softmax(fTheta(E(theta)1:i))kappa' should explicitly state that the softmax is taken over the vocabulary dimension, and the text should clarify whether the sum over i includes the prompt positions within E(theta) or only the generated positions after the prompt; the current notation is easy to misread.
  3. [Eq. (4)] The indexing of theta as 'theta_{t+1:s-m}' is confusing because t is the Engorgio prompt length and m is the template length; a small worked example or a consistent subscript convention would help the reader map the shape (s-m) x V to the sequence composition in Figure 6.
  4. [Section 4.2] The phrase 'the silent nature of SFT models' is not defined; consider replacing it with a concrete statement such as 'the tendency of these SFT models to produce short responses on normal and special inputs.'
  5. [Appendix A.5] The latency model in Eqs. (8) and (9) assumes constant batch processing time and does not account for variable-length batching or preemption; this is acceptable as a first-order model, but the assumption should be stated more prominently where the real-world results are interpreted.

Circularity Check

1 steps flagged · score 2.0 of 10

No significant circularity: the attack's central result is an empirical white-box optimization, with only a minor definitional self-check in the Section 4.6 effectiveness analysis.

  1. self definitional [Section 4.6, Q2; Eq. (6) in Section 3.3]
    "To verify that <EOS> escape loss reduces the probability of <EOS> appearance, we calculate the highest probability of the <EOS> token at all the S positions. We formulate this as µ = max({Softmax(fΘ(E(θ)1:i))κ}S i=1) which signals the highest probability of the interruption of the generation process. ... We find that the decrease of <EOS> escape loss can lead the maximum probability of the occurrence of <EOS> token to a low level (close to 0). This substantiates the effectiveness of <EOS> escape loss in stopping <EOS> token from appearing."

    Eq. (6) defines the <EOS> escape loss as the sum over positions of Softmax(fΘ(E(θ)1:i))κ, which is exactly the per-position softmax <EOS> probability that Q2's µ takes the maximum of on the same optimized soft sequence. Therefore the observed drop in µ when the loss decreases is entailed by the definition of the loss (a sum of non-negative terms cannot fall without its terms, including the maximal term, being driven down on the optimized trajectory). This 'verification' does not independently demonstrate that <EOS> suppression transfers to the discrete, stochastic decoding path; that transfer is only established by the separate generation-length measurements in Tables 1-4. Hence this is a minor definitional self-check, not a load-bearing prediction.

full rationale

The paper's central chain is: parameterize a proxy distribution θ over token sequences, aggregate embeddings via Gumbel-Softmax, optimize θ with an <EOS> escape loss plus a self-mentor loss, sample a discrete prompt, and then measure generated output length under the target model. None of these steps reduces to its inputs by construction. The escape loss directly minimizes softmax <EOS> probability on the soft sequence, while the reported Avg-len and Avg-rate are measured on actual stochastic generation with the sampled discrete prompt; Tables 1, 3, and 4 empirically test that transfer. Optimizing against the same white-box model is the declared threat model of the paper, not circular reasoning. The self-mentor loss makes the prompt distribution resemble the model's own continuation distribution, but that is a design choice rather than a hidden import of the conclusion. There are no load-bearing self-citations, no imported uniqueness theorems, and no ansatz smuggled in via citation. The only definitional shortcut is the Section 4.6 Q2 'verification,' which checks EOS suppression by re-measuring the same softmax EOS quantity that the loss minimizes; this is true by construction and adds no independent evidence, but it does not carry the paper's main result. The temperature sensitivity of SFT models at higher decoding temperatures (Table 3) is a legitimate external-validity concern about the strength of the claim, not a circularity. Overall, the derivation is empirically self-contained, so the circularity score is low.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The method introduces no new physical or architectural entities. The proxy distribution is an optimization construct. The analysis relies on two empirical transfer assumptions: soft-to-discrete prompt and fixed-trajectory-to-stochastic-generation.

free parameters (3)
  • Loss coefficient λ = 1 (ablated over 0.1-10)
    Balances <EOS> escape loss and self-mentor loss in Eq. 5; chosen empirically; Appendix B.5 shows weak sensitivity.
  • Engorgio prompt length t = 32 (ablated over 32, 64, 128)
    Number of tokens in the generated prompt; default 32; Section 4.3 shows longer prompts improve SFT models but are not required.
  • Gumbel-Softmax temperature τ = 1 (tested 0.1-0.7)
    Controls stochasticity of soft token samples in Eq. 3; default 1; lower temperatures improve SFT model results (Section 4.4).
assumptions (3)
  • domain assumption The target model fΘ is differentiable and gradients can be backpropagated through input embeddings to update θ.
    Required for the gradient-based optimization in Section 3.2; standard for transformer-based LLMs.
  • domain assumption A discrete token sequence sampled from the optimized distribution w closely approximates the aggregated soft embedding E(θ) in terms of model behavior.
    Gradients are computed on soft embeddings, but the final Engorgio prompt is discrete; Figure 4 provides partial empirical support.
  • domain assumption Reducing the <EOS> probability on the fixed optimized sequence E(θ) generalizes to the actual random autoregressive generation process.
    The escape loss is evaluated on the fixed S-length sequence (Eq. 6); Table 3 shows this generalization weakens at high temperature for SFT models.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An Engorgio Prompt Makes Large Language Model Babble on." pith.science (2026). https://pith.science/paper/MKMMPNKK

@misc{pith2026241219394,
  author       = {Pith},
  title        = {Pith review of: An Engorgio Prompt Makes Large Language Model Babble on},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MKMMPNKK}},
  note         = {Machine review of arXiv:2412.19394}
}
abstract

Auto-regressive large language models (LLMs) have yielded impressive performance in many real-world tasks. However, the new paradigm of these LLMs also exposes novel threats. In this paper, we explore their vulnerability to inference cost attacks, where a malicious user crafts Engorgio prompts to intentionally increase the computation cost and latency of the inference process. We design Engorgio, a novel methodology, to efficiently generate adversarial Engorgio prompts to affect the target LLM's service availability. Engorgio has the following two technical contributions. (1) We employ a parameterized distribution to track LLMs' prediction trajectory. (2) Targeting the auto-regressive nature of LLMs' inference process, we propose novel loss functions to stably suppress the appearance of the <EOS> token, whose occurrence will interrupt the LLM's generation process. We conduct extensive experiments on 13 open-sourced LLMs with parameters ranging from 125M to 30B. The results show that Engorgio prompts can successfully induce LLMs to generate abnormally long outputs (i.e., roughly 2-13$\times$ longer to reach 90%+ of the output length limit) in a white-box scenario and our real-world experiment demonstrates Engergio's threat to LLM service with limited computing resources. The code is released at: https://github.com/jianshuod/Engorgio-prompt.

Figures

Figures reproduced from arXiv: 2412.19394 by the authors.

Figure 1
Figure 1. Distributions of the total lengths (input plus [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The pipeline of Engorgio. The whole pipeline is divided into two stages. During the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Results of attacking real-world LLM services (“MU”: malicious user, “NU”: normal user). [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Loss curves on OPT-125M (base model) and [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 6
Figure 6. Figure 6: Sequence composition, with a token sequence in the testing stage and a distribution matrix [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: The correlation of output length and the total FLOPs, on LLaMA-7B and ChatGLM-6B. [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: Distribution of sample length. B.5 ADDITIONAL ABLATION STUDY Impact of loss coefficient. We study the configuration of the loss coefficient λ, which is used to balance the scale of <EOS> escape loss and self-mentor loss. We take OPT-125M and LLaMA-7B 22 [PITH_FULL_IMA…
Figure 9
Figure 9. Figure 9: The examples of Engorgio prompts. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_9.png]
Figure 10
Figure 10. Figure 10: The examples of sponge examples. 27 [PITH_FULL_IMAGE:figures/full_fig_p027_10.png]
Figure 11
Figure 11. Figure 11: The examples of normal inputs. 28 [PITH_FULL_IMAGE:figures/full_fig_p028_11.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Exploring Multimodal Challenges in Toxic Chinese Detection: Taxonomy, Benchmark, and Findings

    cs.CL 2025-05 conditional novelty 6.0 of 10

    A new taxonomy and dataset of 8 types of perturbed toxic Chinese show nine top LLMs often miss these obfuscated insults, and small-sample ICL or fine-tuning causes overcorrection.

  2. $PD^3F$: A Pluggable and Dynamic DoS-Defense Framework Against Resource Consumption Attacks Targeting Large Language Models

    cs.CR 2025-05 conditional novelty 6.0 of 10

    PD3F combines a resource-based reputation scheduler with an early-termination logit adjustment to mitigate long-generation DoS attacks on LLMs.

  3. Crabs: Consuming Resource via Auto-generation for LLM-DoS Attack under Black-box Settings

    cs.CL 2024-12 conditional novelty 6.0 of 10

    AutoDoS automatically builds a prompt that makes black-box LLMs generate maximum-length responses, consuming disproportionate compute and degrading service performance.

Reference graph

Works this paper leans on

12 extracted references · 11 canonical work pages · cited by 3 Pith papers

  1. [1]

    It’s about how hard you can get hit and keep moving forward.” - George Chuvalo

    ”It’s not about how hard you hit. It’s about how hard you can get hit and keep moving forward.” - George Chuvalo

    work page

  2. [3]

    ”The will to win is not nearly as important as the will to prepare to win.” - Bobby Knight

    work page

  3. [6]

    Rather, seek it

    ”I hated every minute of training, but I said to myself, Don’t shy away from pain. Rather, seek it. For you are hurt now and again, but you aren’t hurt all the time. The more pain you embrace, the less you will hurt all around.” - Paula Radcliffe

    work page

  4. [9]

    ”The more pain you embrace, the less you will hurt all around.” - Paula Radcliffe

    work page

  5. [10]

    ”You can’t go back and change the beginning, but you can start where you are and change the ending.” - C.S. Lewis

    work page

  6. [11]

    ”The only way you can truly fail is if you give up.” - Rich Gannon

    work page

  7. [15]

    LLM-readable semantics

    IEEE, 2022. Yoshua Bengio, R ´ejean Ducharme, Pascal Vincent, and Christian Jauvin. A neural probabilistic language model. Journal of Machine Learning Research, 3:1137–1155, 2003. Aydar Bulatov, Yuri Kuratov, and Mikhail S Burtsev. Scaling transformer to 1M tokens and beyond with RMT. arXiv preprint arXiv:2304.11062, 2023. Nicholas Carlini, Florian Tramer...

    arXiv 2022

  8. [22]

    ”Champions keep playing until they get it right.” - Bill Russell

    work page

Show all 12 references
  1. [23]

    It is hard work, perseverance, learning, studying, sacrifice and most of all, love of what you are doing or learning to do.” - Pele

    ”Success is no accident. It is hard work, perseverance, learning, studying, sacrifice and most of all, love of what you are doing or learning to do.” - Pele

  2. [24]

    I’ve lost almost 300 games

    ”I’ve missed more than 9000 shots in my career. I’ve lost almost 300 games. 26 times, I’ve been trusted to take the game-winning shot and missed. I’ve failed over and over and over again in my life. And that is why I succeed.” - Michael Jordan

  3. [25]

    ”The only way to prove that you’re a good sport is to lose.” - Ernie Banks

  4. [26]

    This is a new {object} at {place}\

    ”Champions keep playing until they get it right.” - Bill Russell 26 25 Published as a conference paper at ICLR 2025 LLaMA-30B Alpaca Autres compr ending bushity fun thousand stories icefund thousands questions thousand games afterwards date believe cried left won held been old...

    2025

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.