{"id":"030a960e-0769-4a03-ba01-b83a5dbec4c3","arxiv_id":"2412.19394","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Engorgio generates adversarial prompts that suppress <EOS> token probability and make auto-regressive LLMs generate near-maximum-length outputs, multiplying inference cost by 2-13x.","lead":"This paper presents Engorgio, an attack that crafts prompts making large language models generate extremely long responses, driving up computation cost and latency. It matters because a few such prompts could overwhelm shared LLM services and make other users wait.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline 90-100% Avg-rate is achieved only at temperature 0.1; at realistic temperatures (0.7) SFT models drop to 23-71%, so the unqualified central claim is not established for SFT models under typical decoding settings.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern: the transfer of optimized <EOS> suppression from the soft sequence to stochastic generation, with the paper's own Table 3 showing degradation at higher temperatures for SFT models. My stress-test sharpens this by pinpointing that the headline Table 1 results were generated at temperature 0.1, a setting that is not disclosed in the main experimental setup and is unrepresentative of many real LLM services. This is a correctness risk rather than a mere robustness footnote: it changes the scope of the central claim from 'can induce 90-100% max-length outputs' to 'can do so at near-greedy decoding for SFT models.' The prefix mitigation partially addresses high-temperature cases but does not restore the headline numbers, so the conditional verdict is appropriate. I do not see a more fundamental flaw: the method is technically novel, the loss design is principled, the experiments are extensive, and the code release aids verification. The missing temperature disclosure and the unqualified abstract claim are the key issues, and they align with the reader's conditional assessment. Hence no verdict change.","tokens_in":28476,"tokens_out":6449,"duration_ms":57705,"concrete_test":"Run the released Engorgio code to reproduce Table 1 for all 13 models first at temperature 0.1, then at temperature 0.7 without any semantic prefix. Record Avg-len and Avg-rate for each model. If the average Avg-rate across the 7 SFT models at 0.7 falls below 50%, the abstract's unqualified '90-100%' claim must be qualified to the low-temperature regime; also check the code's default temperature setting to confirm the Table 1 conditions.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that Engorgio prompts 'stably suppress the appearance of <EOS>' and induce 90-100% max-length outputs rests on the transfer of <EOS> suppression from the optimized soft trajectory E(θ) (Eq. 6) to the actual stochastic autoregressive sampling path. The paper's own Table 3 shows this transfer is fragile: for SFT models, Avg-rate collapses from 98%/90%/95% at temperature 0.1 to 33%/24%/71% at temperature 0.7 for StableLM, Samantha, and ChatGLM respectively. Cross-referencing Table 1 with Table 3 reveals that the headline Table 1 results were generated at temperature 0.1 (e.g., StableLM 1021.6/98%, Samantha 944.0/89%, ChatGLM 979.6/95%), yet Section 4.1 omits the default temperature. Most commercial chat services operate at higher temperatures (0.6-1.0), so the attack's effectiveness in realistic deployments is not supported by the unqualified abstract claim. The prefix mitigation restores some performance (StableLM 83% at 0.7) but not to the 90-100% headline, and Alpaca remains at only 43%. Thus the load-bearing assumption—that EOS suppression optimized on a fixed soft sequence transfers to the stochastic generation path—is empirically violated for SFT models at standard temperatures, making the central claim overgeneralized.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Engorgio, a white-box inference-cost attack against autoregressive large language models. The attacker optimizes a Gumbel-Softmax parameterized distribution over a prompt and a planned future context, using an <EOS>-escape loss (Eq. 6) and a self-mentor loss (Eq. 7), then samples a discrete prompt and sends it to the victim. Experiments on 13 open-source models report that Engorgio prompts drive outputs to roughly 84-100% of the configured maximum length at the temperature used in the main tables, compared with much shorter outputs for normal inputs and prior baselines. The paper also reports transferability across related models, an ablation study, a defense analysis, and a Hugging Face inference-endpoint case study showing reduced throughput and increased queueing for normal clients.","tokens_in":28701,"tokens_out":7952,"duration_ms":72456,"significance":"If the stated effectiveness holds across realistic deployment conditions, Engorgio is a meaningful advance in the inference-cost-attack literature: it adapts sponge-style attacks from encoder-decoder models to decoder-only autoregressive LLMs, contributes a reusable optimization procedure, and evaluates on a broad and relevant model zoo. The code release and the real-world endpoint study are valuable assets for reproducing and stress-testing the attack. However, the significance is currently qualified by the paper's own temperature study: the 90-100% claims are established for low-temperature decoding, while SFT models degrade sharply at temperature 0.7. The central methodological idea is sound, but the unqualified abstract and contribution statements overstate the attack's robustness for typical chat-service decoding settings.","major_comments":[{"comment":"The main results in Table 1 are reported without a stated decoding temperature, but the StableLM, Samantha, and ChatGLM rows match the temperature-0.1 rows of Table 3 almost exactly (StableLM 1021.6/98%, Samantha 944.0/89%, ChatGLM 979.6/95%). This indicates that the headline SFT results were produced at temperature 0.1, yet the abstract and Section 4.2 claim 'roughly 90-100%' without that qualification. At temperature 0.7, Table 3 shows Avg-rate collapsing to 33%, 24%, and 71% for those same models, and Table 4 shows that adding a semantic prefix restores StableLM to 83% but leaves Alpaca at only 43%. Because many production chat services sample at temperatures around 0.6-1.0, the abstract's unqualified central claim is not supported for SFT models under typical decoding settings. This also exposes the fragility of the core transfer assumption: <EOS> suppression is optimized on the fixed soft trajectory E(θ) in Eq. (6), but it does not reliably transfer to stochastic autoregressive decoding at higher temperatures. Please state the default temperature in Section 4.1, qualify the abstract and conclusion accordingly, or add experiments at the temperatures used by the services discussed in Appendix A.2.","section":"Section 4.1 (Table 1) and Section 4.4 (Table 3)"},{"comment":"The evaluation reports Avg-len and Avg-rate without any measure of dispersion. Appendix A.5 mentions that Avg-len is computed by sampling 100 times, but no standard deviation, confidence interval, or number of repetitions is provided for any table, and the sample size underlying Avg-rate is not specified. Given that Table 3 shows large swings across neighboring temperatures (e.g., StableLM falls from 62% at temperature 0.3 to 28% at 0.5), the paper's claims of 'stable' suppression need error bars, per-run statistics, or at least a statement of the number of independent runs before the reader can distinguish systematic degradation from sampling noise.","section":"Section 4.1 and Appendix A.5"},{"comment":"The sentence 'For talkative base models, they are tempted to respond endlessly when a high temperature of 0.7 is used' asserts temperature robustness for base models, but Table 3 reports data only for SFT models. No base-model experiments at temperature 0.7 are presented in the main text or appendices. Either provide those results, or remove or qualify the sentence so that the reader does not infer base-model high-temperature behavior that the paper does not demonstrate.","section":"Section 4.4"},{"comment":"The real-world case study does not report the decoding temperature, sampling parameters, or the number of requests contributing to each bar in Figure 3, and the figure has no error bars. Since Table 3 establishes that Engorgio's effectiveness is strongly temperature-dependent for SFT models such as StableLM, the case study is not interpretable without stating the endpoint's temperature and sampling configuration. Please add this information and, ideally, compare at least two temperatures that match the threat scenarios described in Appendix A.2.","section":"Section 4.5 and Figure 3"}],"minor_comments":[{"comment":"The abstract contains the typo 'Engergio' in the sentence 'our real-world experiment demonstrates Engergio's threat'; it should be 'Engorgio'.","section":"Abstract"},{"comment":"The notation 'Softmax(fTheta(E(theta)1:i))kappa' should explicitly state that the softmax is taken over the vocabulary dimension, and the text should clarify whether the sum over i includes the prompt positions within E(theta) or only the generated positions after the prompt; the current notation is easy to misread.","section":"Eq. (6)"},{"comment":"The indexing of theta as 'theta_{t+1:s-m}' is confusing because t is the Engorgio prompt length and m is the template length; a small worked example or a consistent subscript convention would help the reader map the shape (s-m) x V to the sequence composition in Figure 6.","section":"Eq. (4)"},{"comment":"The phrase 'the silent nature of SFT models' is not defined; consider replacing it with a concrete statement such as 'the tendency of these SFT models to produce short responses on normal and special inputs.'","section":"Section 4.2"},{"comment":"The latency model in Eqs. (8) and (9) assumes constant batch processing time and does not account for variable-length batching or preemption; this is acceptable as a first-order model, but the assumption should be stated more prominently where the real-world results are interpreted.","section":"Appendix A.5"}],"recommendation":"major_revision","confidential_remarks":"The methodology is technically sound and the model coverage is broad, but the gap between the abstract's unqualified 90-100% claim and the temperature-limited results in Section 4.4 is the main barrier to acceptance. The authors' own limitation paragraph in Section 5 concedes that high-temperature cases are only partially addressed, so the fix is primarily a matter of recalibrating claims and supplying missing configuration details rather than redoing the core experiments. I would not reject the paper, but the revision needs to be substantive."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core result is real: optimizing a Gumbel-Softmax-parameterized proxy distribution over the whole context with an untargeted <EOS> escape loss plus a self-mentor loss reliably makes several open-weight LLMs generate near-max-length outputs. That is genuinely new relative to sponge examples and LLMEffiChecker, and the evaluation is broad: thirteen models, 125M to 30B, base and SFT, with code released. The transferability experiments and the real-world endpoint case study are a plus, and the output-length metric is not circular with the training objective. This paper deserves a serious referee.\n\nThe soft spot is the temperature dependence, and it is more than cosmetic. The headline 90-100% Avg-rate comes from Table 1, which is generated at temperature 0.1, but Section 4.1 never states the default temperature. Table 3 shows what happens at temperature 0.7: StableLM drops to 33%, Samantha to 24%, ChatGLM to 71%. The paper does report this honestly and proposes a semantic-prefix mitigation, but the abstract and conclusion still claim the prompts 'stably suppress' <EOS> without qualification. For realistic chat services that run at higher temperatures, the attack is substantially weaker against SFT models. That is a qualification, not a refutation: base models still respond well at 0.7, and the prefix restoration gets StableLM back to 83%.\n\nTwo smaller issues: there are no error bars or significance tests on the aggregated metrics, and the transfer from the optimized soft trajectory to the actual stochastic generation path is only partially validated. The fixed-trajectory optimization can fail when sampling deviates, as the high-temperature results show. These are fixable in revision. The real-world experiment is a single model on one serving stack, fine as a case study but not a general claim.\n\nBottom line: a credible, useful attack paper with a clear technical contribution and honest reporting of its own temperature sensitivity. The main fix is to qualify the central claim and report the default decoding settings. I would cite it and would bring it to a reading group. Send it to a serious referee; just ask for the missing caveats and a bit more statistical care.","headline":"A useful and technically solid attack paper whose headline numbers are real but temperature-dependent; the abstract overclaims without the temperature caveat.","tokens_in":613,"tokens_out":751,"would_cite":true,"duration_ms":19184,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A short optimized prompt can push autoregressive LLMs to generate until their maximum length, multiplying per-query inference cost by 2–13×.","keywords":["inference cost attack","adversarial prompts","end-of-sequence token","autoregressive language models","Gumbel-Softmax optimization","service availability","prompt optimization","LLM denial of service"],"falsifier":"Decode the released Engorgio prompts on LLaMA-7B (max length 1024) at temperatures 0.1, 0.3, 0.5, and 0.7 and compare Avg-len and Avg-rate with the paper's Table 3; a decisive negative result would be near-normal output lengths at default API sampling temperature, showing the attack does not generalize beyond carefully chosen decoding settings.","tokens_in":28215,"feed_emoji":"💸","tokens_out":8394,"duration_ms":66492,"temperature":0.7,"pith_summary":"This paper tries to establish that a short, carefully optimized prompt can make modern decoder-only large language models generate text until they hit the pre-set maximum output length, turning a single query into a much larger compute bill. The attack, called Engorgio, works by suppressing the probability of the end-of-sequence (<EOS>) token along the generation path, so the model does not stop talking. On six base models the optimized prompts reach roughly 90–100% of the maximum length, while normal queries reach only 0–40%; across all 13 tested models the claimed effect is 2–13× higher inference cost per query than normal inputs. The authors argue this is a practical availability threat because many services that deploy open-weight models rate-limit requests rather than tokens.","feed_headline":"Adversarial prompts make LLMs babble to 90–100% of max length","feed_subtitle":"Crafted prompts suppress the end-of-generation signal, stretching responses and multiplying inference cost per query 2–13×.","key_machinery":"The core machinery is a parameterized proxy distribution $\\theta \\in \\mathbb{R}^{S \\times V}$ over the whole context, where $S$ is the maximum allowable length and $V$ is the vocabulary size. In the generation stage, each row is normalized with Gumbel-Softmax — a differentiable relaxation of sampling from a categorical distribution — and used as weights to aggregate token embeddings into a soft embedding $E(\\theta)$ that is fed to the model, so gradients flow back into $\\theta$. Two losses drive the update: the <EOS> escape loss $\\mathcal{L}_{\\mathrm{esc}}(\\theta) = \\sum_i \\operatorname{Softmax}(f_\\Theta(E(\\theta)_{1:i}))_\\kappa$, which penalizes the softmax probability of the <EOS> token at every position, and the self-mentor loss $\\mathcal{L}_{\\mathrm{sm}}(\\theta)$, which cross-entropy-matches the proxy distribution's predictions against the model's own predictions to keep the optimized sequence self-consistent. After optimization, the distribution typically converges to a single high-probability token sequence, and that sequence is sampled and sent as the Engorgio prompt.","core_discovery":"The central claim is that an untargeted objective — penalizing the <EOS> token everywhere it might appear — is sufficient to control the length of autoregressive generation. The authors show that optimizing a Gumbel-Softmax parameterized proxy distribution over a short prompt yields a discrete prompt that lowers <EOS> probability across all positions, making LLaMA, LLaMA-2, OPT, GPT-2, and SFT variants such as Alpaca, Vicuna, Koala, StableLM, Orca, Samantha, and ChatGLM emit outputs near the maximum allowable length: 90%+ of the maximum length on base models, and near-maximum on SFT models when a semantic prefix is added. A real-world deployment on a Hugging Face inference endpoint shows that a small fraction of attack queries sharply increases queueing latency for normal users and nearly cuts throughput. The paper interprets the length increase as a lower bound on cost increase, since Transformer inference cost grows super-linearly with response length.","pith_inferences":["An unstated consequence is that the same <EOS>-suppression recipe could be applied to coherent-looking prompts: if Engorgio is woven into natural instructions, the already high false-positive rate of perplexity filtering would make detection even harder.","The paper's temperature results point to a cheap mitigation it does not pursue: raising sampling temperature or using diverse decoding for suspiciously long generations would break the optimized EOS-suppression trajectory, at some cost in response quality.","The mechanism is broader than text completion: any autoregressive model with a stop token and request-level rate limits, such as code completion or multimodal generation, could be stretched by the same untargeted escape-loss optimization.","A production-scheduler replication would clarify whether prompt caching and continuous batching absorb the extra tokens, which the paper's simplified queuing model does not address."],"forward_implications":["If the central claim is right, any service that deploys an open-weight LLM and rate-limits by request rather than by tokens can be forced to spend near-maximum compute per query, making request-level limits the main attack surface.","Base models (LLaMA-30B, LLaMA-7B, LLaMA-2-7B, OPT-125M, OPT-1.3B, GPT2-large) can be driven to roughly 90–100% of maximum output length, versus 0–40% for normal inputs, so the effect holds across model families and scales from 125M to 30B parameters.","SFT models, which tend to give short answers, need a semantic prefix fused with the Engorgio prompt to reach near-maximum lengths; without it some models respond with gibberish or 'not understand'.","Engorgio prompts transfer between related models (e.g., Vicuna-crafted prompts achieve 96% Avg-rate on Koala), which extends the attack to black-box settings where the attacker only controls a proxy model.","Because each additional generated token costs a new forward pass over a longer context, the reported 2–13× output-length increase corresponds to a super-linear inference-cost increase, so the cost ratios are a lower bound."],"supporting_citations":[{"why":"Introduces sponge examples, the inference-cost attack baseline that Engorgio compares against and extends to decoder-only LLMs.","marker":"Shumailov et al. (2021)"},{"why":"Proposes LLMEffiChecker, the perturbation-based efficiency attack whose failure on modern LLMs motivates a new method.","marker":"Feng et al. (2024)"},{"why":"Provides the optimization-based targeted attack that forces repeated content, the closest adversarial-prompt baseline.","marker":"Geiping et al. (2024)"},{"why":"Supplies the Gumbel-Softmax reparameterization used to make the discrete prompt distribution differentiable.","marker":"Jang et al. (2017)"},{"why":"Establishes prefix-tuning and continuous soft prompts, the embedding-aggregation idea behind the proxy distribution.","marker":"Li & Liang (2021)"},{"why":"Shows gradient-based search over text transformers, supporting the choice of a parameterized distribution over tokens.","marker":"Guo et al. (2021)"},{"why":"Defines the Transformer self-attention architecture whose per-forward-pass cost grows with context, making longer outputs super-linearly expensive.","marker":"Vaswani et al. (2017)"},{"why":"Releases the LLaMA model family that serves as the main target and proxy models in the experiments.","marker":"Touvron et al. (2023)"},{"why":"Formulates autoregressive sequence generation, the inference mechanism that the <EOS>-suppression attack exploits.","marker":"Graves (2013)"}],"fun_headline_variants":["Engorgio prompts force LLMs to babble to max length, 2–13x cost","Crafty prompts suppress end-token, making LLMs blabber on 2–13x longer","Inference-cost attack: prompts stretch LLM outputs to 90–100% max length","Adversarial prompts make LLMs ramble, boosting compute 2–13x per query","Engorgio: a prompt trick to inflate LLM generation cost up to 13x"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that suppressing the <EOS> token on an optimized soft sequence transfers to the model's actual token-by-token sampling; if the sampled tokens deviate from that trajectory — for example at high sampling temperature — the suppression can fail.","fun_headline_variants_meta":{"raw":{"variants":["Engorgio prompts force LLMs to babble to max length, 2–13x cost","Crafty prompts suppress end-token, making LLMs blabber on 2–13x longer","Inference-cost attack: prompts stretch LLM outputs to 90–100% max length","Adversarial prompts make LLMs ramble, boosting compute 2–13x per query","Engorgio: a prompt trick to inflate LLM generation cost up to 13x"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000389,"raw_usage":{"total_tokens":2082,"prompt_tokens":1010,"completion_tokens":1072,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":626,"completion_tokens_details":{"reasoning_tokens":958}},"tokens_in":626,"tokens_out":1072,"duration_ms":8309,"temperature":1.0,"reasoning_tokens":958,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T00:37:53.317775+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Decode the released Engorgio prompts on LLaMA-7B (max length 1024) at temperatures 0.1, 0.3, 0.5, and 0.7 and compare Avg-len and Avg-rate with the paper's Table 3; a decisive negative result would be near-normal output lengths at default API sampling temperature, showing the attack does not generalize beyond carefully chosen decoding settings.","supporting_citations":[],"review_version":1}