Pith. sign in

REVIEW 5 major objections 3 minor 1 cited by

Prompt-Based One-Shot Exact Length-Controlled Generation with LLMs

T0 review · 5 major / 3 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A countdown-marker prompt suffix lets an off-the-shelf LLM hit an exact token count in a single generation, with no fine-tuning or iterative sampling.

desk verdict A real prompt-engineering effect, but the abstract's MT-Bench-LI headline numbers are contradicted by the paper's own Table 4, and the quality-preservation claim does not survive contact with the tables. read the letter →

arxiv 2508.13805 v1 pith:CKFRWL3V submitted 2025-08-19 cs.CL cs.AI

Juncheng Xie , Hung-yi Lee This is my paper · ORCID
classification cs.CLcs.AI
keywords length-controlledgenerationcountdownpromptingexactlengthcontrolpromptengineeringMT-Bench-LILIFEBENCHchain-of-thoughtLLMtokencounting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's central claim is that exact length control can be achieved through prompt engineering alone: appending a countdown-marker suffix that asks the model to write word, word, ... <1>word, <0> turns the output stream into a visible counter, so an off-the-shelf LLM can hit a target token count in one pass. Across open-ended generation, XSUM summarization, MT-Bench-LI instruction following, and LIFEBENCH, the authors report large jumps in exact-match compliance compared with a naive length instruction. The strongest reported gains are on random text generation (from under 2% to 94.2% exact match for GPT-4.1, English) and XSUM (to 96.4% at reference length); on MT-Bench-LI the paper's Table 4 shows GPT-4.1 exact match rising from 9.7% to 74.9%, while the abstract's "above 95%" figure does not match that table. If these results hold, strict length control becomes a lightweight prompt technique applicable to closed and open models alike.

What carries the argument

Countdown-marker scaffold with decrement one: a suffix that demands the output be an explicitly numbered sequence of the form <N>w_N, <N-1>w_{N-1}, ..., <1>w_1, <0>, with the marker value decreasing by exactly one after every emitted word, character, or code line. By externalizing the count into the visible output, generation becomes a serial scratchpad computation, which the paper argues lifts the required computation from constant-depth $\mathsf{NC}^0$ to $\mathsf{NC}^1$ and lets iterative counting happen in the open. The single-token decrement is load-bearing: any larger step would require the model to count several unseen tokens before the next marker update and would reintroduce the internal-counting failure regime.

What would settle it

Re-run the MT-Bench-LI evaluation with GPT-4.1 under the exact CAPEL suffix, count post-processed output tokens, and compare the exact-match rate with the abstract's "above 95%" claim and Table 4's 74.9%; a replication that lands near 75% rather than above 95% would show the headline figure is not reproducible as stated. A second check: request a 1000-word neutral passage and parse whether every marker from <1000> to <1> appears exactly once with exactly one word between consecutive markers; the paper's own taxonomy predicts most failures will be fused marker-word pairs, so if the model instead fails by wholesale refusal or random drift, the proposed failure mechanism is wrong.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that transformer LLMs fail at length control not because they lack the words but because they cannot maintain an internal counter, and that this internal arithmetic can be replaced by an external, one-step-at-a-time countdown printed in the output. The CAPEL suffix instructs the model to emit exactly the markers <N> down to <1>, each immediately followed by one token and then <0>, with explicit rules that markers are not counted, skips and repeats are forbidden, and nothing follows <0>. Under this prompt the model "writes while counting," reducing each length decision to a local pattern-completion step rather than a global estimate. The paper reports exact-match rates above 90% on several tasks and models, and a diagnostics study in which the same models can count two tokens reliably but fall below 20% accuracy at ten tokens, supporting the claim that the scaffold, not the model's latent counting ability, does the work. On MT-Bench-LI, the paper reports a quality drop for GPT-4.1 (single-answer score 4.839 to 4.063) alongside the compliance gain, which the authors address with a single-pass Draft-to-CAPEL variant that recovers much of the judged quality while keeping one-shot cost.

Load-bearing premise

The method assumes the model will keep printing the countdown markers in strict descending order, exactly one marker per token, all the way to <0>; if it fuses a marker with a word, repeats or skips a marker, or stops early, the exact-length promise breaks, and the paper's error analysis shows those are precisely the dominant failure modes.

Editorial extensions

If this is right

  • Exact length control becomes a drop-in prompt: the same CAPEL suffix transfers across proprietary and open-weight models, so systems with hard word or character limits can be built without retraining or multi-pass decoding.
  • On XSUM and random-text generation, CAPEL pushes exact-match rates above 90% for several models, meaning common summarization and constrained-writing tasks could rely on a single pass rather than revise-until-fit loops.
  • The scaffold works for both English word counts and Chinese character counts, which suggests the mechanism is about visible enumeration rather than a language-specific tokenizer.
  • Because the marker sequence is visible, length failures become locally diagnosable: a fused marker, skipped number, or missing <0> tells you exactly where the count went wrong.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the mechanism is robust, length control may not be a separate capability but a byproduct of serial bookkeeping; other counting-like constraints, such as exact paragraph counts, exact list-item counts, or "exactly k examples," may yield to the same externalized-countdown pattern.
  • The paper's own numbers indicate the method is strongest at moderate lengths and decays toward thousands of tokens; a testable extension would be adaptive marker granularity, such as chunk-level markers for long outputs, to push the exact-match boundary past 1K tokens.
  • The MT-Bench-LI quality drop for GPT-4.1 under CAPEL suggests pure exactness and judged quality can pull in opposite directions; if quality matters more than perfect compliance, the single-pass Draft-to-CAPEL variant may be the practically useful operating point.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 3 minor

Summary. The paper proposes CAPEL, a prompt-only method that appends descending countdown markers (<N> ... <0>) to a length instruction so that an off-the-shelf LLM writes while counting, aiming for exact token/word/character counts in one pass without fine-tuning or iterative sampling. The method is evaluated on an in-house random-text task, XSUM, MT-Bench-LI, and the LIFEBENCH Equal-To track across eleven API and local models, with metrics for exact-match rate, MAE/MALD, and judged quality, plus a counting diagnostic and an error taxonomy. The central claim is that countdown markers sharply improve exact length compliance across many model-task pairs and that on MT-Bench-LI the flagship GPT-4.1 result exceeds 95% exact match while preserving answer quality.

Significance. A genuinely prompt-only mechanism for near-exact length control would be practically valuable, especially for black-box APIs, and the paper contributes a simple, parameter-free template and an honest failure analysis. The method is not fitted to evaluation benchmarks: the CAPEL suffix appears fixed across tasks, and the headline gains are measured on external benchmarks (XSUM, MT-Bench-LI, LIFEBENCH). The evidence does show large compliance improvements for many model-task pairs, and the error taxonomy (marker-word fusion, safety-filter early stops) is a useful diagnostic. However, the paper's own tables contradict several headline claims, particularly on MT-Bench-LI and on summarization quality preservation, so the significance as stated is not yet established.

major comments (5)
  1. [Abstract and §4.5, Table 4] The abstract and §4.5 claim that GPT-4.1 strict compliance on MT-Bench-LI exceeds 95%, and §4.5 states EM rises from 9.7% to 96.3%, but Table 4 reports CAPEL EM of 74.9% for GPT-4.1 (baseline 9.7%). The 96.3% figure does not appear in any table for this setting; it matches neither Table 4 nor any other MT-Bench-LI result. The flagship quantitative claim is therefore unsupported by the reported data.
  2. [§4.4, Table 3] The text claims CAPEL 'preserves ROUGE-L within ±0.02 of the baseline,' but Table 3 shows larger drops for most models, e.g., GPT-4.1-nano 0.162→0.133 (−0.029), Qwen3-4B 0.174→0.122 (−0.052), and Qwen3-8B 0.177→0.129 (−0.048). The quality-cost claim is thus not supported by the table, and the same issue affects the abstract's assertion that judged answer quality is preserved.
  3. [§4.5, Table 5] The statement that Draft→CAPEL 'outperforms every iterative baseline at the same (one-shot) cost' is contradicted by Table 5: the iterative memory baselines AccMem and Mem achieve single-answer scores of 5.78, higher than Draft→CAPEL's 5.33, and CAPEL's 4.06 is the lowest among the listed GPT-4.1 variants. The claim is only valid for EM, not for quality.
  4. [§4.6, Figure 3] The text asserts that 'CAPEL sustains EM > 60% and LS > 90 up to 256 tokens for four of the seven models,' but Figure 3 reports EM values at 256 tokens of 44% (GPT-4.1), 30% (Qwen3-32B-AWQ), 52% (Qwen3-8B), and 0% (GPT-4o-mini), with no model above 60% at that budget. The claimed per-length robustness is not what the figure shows.
  5. [§6 Conclusion] The conclusion that CAPEL 'raises exact-match rates from under 30% to above 95%' is only true for the XSUM reference-length condition (e.g., 96.4% for GPT-4.1 in Table 3). On LIFEBENCH the maximum CAPEL EM is 42.6% (Table 6), and on MT-Bench-LI the maximum CAPEL EM is 80.6% (Table 4). The conclusion over-generalizes a single favorable condition to all settings.
minor comments (3)
  1. [Reproducibility Checklist] The checklist states that the random-text corpus will be attached only after acceptance and that the code is behind a private GitHub link to be made public later; for a prompt-engineering paper, releasing the exact prompt templates and evaluation scripts with the submission would materially aid reproducibility.
  2. [References] The reference 'Zhang, X.; and ... 2025' is incomplete and cannot be resolved; the three Qwen3 technical report references should be distinguished or merged properly.
  3. [§4.6 and Figure 2] Figure 2's x-axis is labeled 'Target length (words/characters),' but LIFEBENCH budgets are specified in tokens; the discrepancy should be clarified in the caption or main text.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: CAPEL is a fixed prompt evaluated on external benchmarks; no fitted parameter is renamed as a prediction.

full rationale

This paper's derivation is self-contained with respect to circularity. CAPEL is a fixed prompt suffix defined in Section 3 and applied unchanged to four external benchmarks; no parameter is fitted to any evaluation set. The exact-match metric compares post-processed output token/character counts to external target lengths, so the headline result is measured, not encoded by construction. The only design choice that could invite a circularity concern, the countdown decrement d=1, is motivated in Section 5.1 by an independent counting diagnostic (Table 7) and is presented as an explanation of an already-specified prompt, not as a fitted parameter derived from the benchmark outcomes. The paper contains no self-citations; all cited prior work (Hansel, LIFEBench, MT-Bench-LI, BB-MH) is external and independently defined. The abstract's above-95% MT-Bench-LI claim conflicts with Table 4's 74.9% EM and is a reporting/consistency problem, not a circularity problem. No step reduces to its own input, so the circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper contributes a prompt design rather than a fitted model, so the ledger contains no fitted numbers and no invented entities. The main entries are the hand-chosen decrement d=1 and marker format; the load-bearing dependencies are behavioral assumptions about model compliance with the marker protocol, which the paper's own error taxonomy shows is where failures occur.

free parameters (3)
  • Countdown decrement d=1 = 1
    Hand-chosen design parameter; Section 5.1 justifies d=1 post hoc via a counting diagnostic, arguing that d>1 would force multi-token internal counting.
  • Marker syntax <k> = Angle-bracket integer markers
    Format chosen by the authors; no search or comparison is reported, but the central claim depends on the model treating markers as countable separators.
  • Code-aware token rule = One code line equals one token
    Ad hoc extension added for MT-Bench code items (Section 3.3); it changes the counting unit and therefore the definition of exact length for those items.
assumptions (4)
  • domain assumption LLMs can follow complex multi-rule format instructions
    CAPEL depends on the model obeying marker order, spacing rules, and the stop-after-<0> rule; invoked throughout Section 3.
  • domain assumption Word or character count is the correct ground-truth length metric
    Section 4.2 defines compliance by word count (English) and character count (Chinese); model tokenizer behavior is assumed to align with these units.
  • standard math Transformers cannot count unboundedly and scratchpad output lifts expressivity from NC0 to NC1
    Borrowed from Feng et al. 2023 and Li et al. 2024c in Section 3.1; used to explain why the countdown scaffold should work.
  • domain assumption o4-mini is a reliable quality judge under length constraints
    Section 4.2 adopts o4-mini as the LLM-as-judge; no validation of judge agreement against humans is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Prompt-Based One-Shot Exact Length-Controlled Generation with LLMs." pith.science (2026). https://pith.science/paper/CKFRWL3V

@misc{pith2026250813805,
  author       = {Pith},
  title        = {Pith review of: Prompt-Based One-Shot Exact Length-Controlled Generation with LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CKFRWL3V}},
  note         = {Machine review of arXiv:2508.13805}
}
read the original abstract

Controlling the length of text produced by large language models (LLMs) remains challenging: models frequently overshoot or undershoot explicit length instructions because they cannot reliably keep an internal token count. We present a prompt-based, one-shot strategy that compels an off-the-shelf LLM to generate exactly a desired number of tokens - words (English) or characters (Chinese) - without any fine-tuning or iterative sampling. The prompt appends countdown markers and explicit counting rules so that the model "writes while counting." We evaluate on four settings: open-ended generation (1-1000 tokens), XSUM summarization, MT-Bench-LI instruction following, and the LIFEBENCH equal-length track. On MT-Bench-LI, strict length compliance with GPT-4.1 leaps from below 30% under naive prompts to above 95% with our countdown prompt, surpassing the popular draft-then-revise baseline, while judged answer quality is preserved. These results show that precise length control can be achieved through prompt engineering alone, offering a lightweight alternative to training- or decoding-based methods.

Figures

Figures reproduced from arXiv: 2508.13805 by the authors.

Figure 1
Figure 1. MALD versus target length (N=1−1000). Lines show GPT-4.1 (EN/CN), and DeepSeek-V3 (EN/CN) under both prompts. Model-specific anomalies. Two smaller models, o4-mini and gpt-4o-mini, deviate from the trend: CAPEL increases their error, roughly by an order of mag￾nitude in Chinese and, for o4-mini, on English as well. This is unlikely to be a context window issue: the worst case prompt (1000 requested words+1000 countd… view at source ↗
Figure 2
Figure 2. Length Score on LIFEBENCH. Left = BASELINE, right = CAPEL. X-axis = target length (words/characters); Y-axis = model. • GPT-4o-mini outlier. Despite strong short-length com￾pliance, the model’s LS plunges at 4096-8192 tokens (40.5→0.3). Manual inspection reveals premature termi￾nation after printing the final marker but omitting trailing content-a behavior visible as a dark band in [PITH_FULL_IMAGE:figures/full_fig… view at source ↗
Figure 3
Figure 3. Exact-match rate (%) on LIFEBENCH. Left = BASELINE, right = CAPEL. X-axis = target length (words/characters); Y-axis = model. F Additional Figures for the Counting Diagnostic [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Exact-match accuracy heatmap for the token-counting diagnostic on English sentences (word level). Rows correspond [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]
Figure 5
Figure 5. Figure 5: Exact-match accuracy heatmap for the token-counting diagnostic on Chinese sentences (character level). Rows cor [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Length Value Model: Scalable Value Pretraining for Token-Level Length Modeling

    cs.CL 2026-04 unverdicted novelty 6.0 of 10

    LenVM trains a token-level value head to predict discounted remaining length, enabling length control and efficiency steering on LLMs and VLMs.

Reference graph

Works this paper leans on

46 extracted references · 21 canonical work pages · cited by 1 Pith paper

  1. [1]

    Abadji, J.; Su \'a rez, P. J. O.; Romary, L.; and Sagot, B. 2022. Towards a Cleaner Document-Oriented Multilingual Crawled Corpus. CoRR, abs/2201.06642. Contains the OSCAR-2201 dataset

    arXiv 2022

  2. [2]

    Aggarwal, P.; and Welleck, S. 2025. L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning. arXiv preprint arXiv:2503.04697

    arXiv 2025

  3. [3]

    AI, M. 2024. Introducing Mistral Large. https://mistral.ai/news/mistral-large. Accessed: 2025-07-29

    work page 2024

  4. [4]

    Anthropic. 2024. Claude 3 Family: Opus, Sonnet, Haiku. https://www.anthropic.com/news/claude-3. Accessed: 2025-07-29

    work page 2024

  5. [5]

    P.; Wang, L.; and King, I

    Chan, H. P.; Wang, L.; and King, I. 2021. Controllable Summarization with Constrained Markov Decision Process. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), 1447--1463

    work page 2021

  6. [6]

    Chang, Y.; and Bisk, Y. 2024. Language Models Need Inductive Biases to Count Inductively. arXiv preprint arXiv:2405.20131

    arXiv 2024

  7. [7]

    DeepSeek-AI . 2024. DeepSeek-V3 Technical Report . arXiv:2412.19437

    arXiv 2024

  8. [8]

    DeepSeek-AI . 2025. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning . arXiv:2501.12948

    arXiv 2025

Show all 46 references
  1. [9]

    Dubois, Y.; Galambosi, B.; Liang, P.; and Hashimoto, T. B. 2024. Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators. arXiv preprint arXiv:2404.04475

    2024 arXiv

  2. [10]

    Feng, G.; Zhang, B.; Gu, Y.; Ye, H.; He, D.; and Wang, L. 2023. Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective. In Proceedings of the 37th Conference on Neural Information Processing Systems (NeurIPS 2023)

    2023

  3. [11]

    Gokaslan, A.; Cohen, V.; Pavlick, E.; and Tellex, S. 2019. OpenWebText Corpus. http://skylion007.github.io/OpenWebTextCorpus

    2019

  4. [12]

    Gu, Y.; Wang, W.; Feng, X.; Zhong, W.; Zhu, K.; Chua, T.; and Qin, B. 2024. Length Controlled Generation for Black-Box LLMs. arXiv preprint arXiv:2412.14656

    2024 arXiv

  5. [13]

    Javorsk\' y , D.; Bojar, O.; and Yvon, F. 2025. Prompting LLMs: Length Control for Isometric Machine Translation. arXiv preprint arXiv:2506.04855

    2025 arXiv

  6. [14]

    Jie, R.; Meng, X.; Shang, L.; Jiang, X.; and Liu, Q. 2024. Prompt-based Length Controlled Generation with Multiple Control Types. In Findings of the Association for Computational Linguistics: ACL 2024, 1067--1085

    2024

  7. [15]

    Li, G.; Xia, T.; Chang, Y.; and Wu, Y. 2024 a . Length-Controlled Margin-Based Preference Optimization without Reference Model. arXiv preprint arXiv:2502.14643

    2024 arXiv

  8. [16]

    E.; and Stoica, I

    Li, T.; Chiang, W.-L.; Frick, E.; Dunlap, L.; Wu, T.; Zhu, B.; Gonzalez, J. E.; and Stoica, I. 2024 b . From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline. arXiv preprint arXiv:2406.11939

    2024 arXiv

  9. [17]

    Li, Z.; Liu, H.; Zhou, D.; and Ma, T. 2024 c . Chain of Thought Empowers Transformers to Solve Inherently Serial Problems. arXiv preprint arXiv:2402.12875

    2024 arXiv

  10. [18]

    Miculicich, L.; Xie, Y.; Wang, S.; and He, P. 2023. Summarization with Precise Length Control. arXiv preprint arXiv:2305.05171

    2023 arXiv

  11. [19]

    B.; and Lapata, M

    Narayan, S.; Cohen, S. B.; and Lapata, M. 2018. Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), 1797--1807. Brus...

    2018

  12. [20]

    OpenAI. 2023. GPT-4 Turbo with 128K Context (OpenAI DevDay Announcement). OpenAI News, Nov 2023

    2023

  13. [21]

    OpenAI. 2024. GPT-4o System Card . arXiv:2410.21276

    2024 arXiv

  14. [22]

    OpenAI. 2025 a . GPT-4o mini: advancing cost-efficient intelligence

    2025

  15. [23]

    OpenAI. 2025 b . Introducing GPT-4.1 in the API

    2025

  16. [24]

    OpenAI. 2025 c . Introducing GPT-4.1 Mini in the API

    2025

  17. [25]

    OpenAI. 2025 d . Introducing GPT-4.1 Nano in the API

    2025

  18. [26]

    OpenAI. 2025 e . OpenAI o3 and o4-mini System Card

    2025

  19. [27]

    Qin, Y.; Song, K.; Hu, Y.; Yao, W.; Cho, S.; Wang, X.; Wu, X.; Liu, F.; Liu, P.; and Yu, D. 2024. InFoBench : Evaluating Instruction Following Ability in Large Language Models. arXiv preprint arXiv:2401.03601

    2024 arXiv

  20. [28]

    Shin, A.; and Kaneko, K. 2024. Large Language Models Lack Understanding of Character Composition of Words . arXiv preprint arXiv:2405.11357

    2024 arXiv

  21. [29]

    Song, S.; Lee, J.; and Ko, H. 2025. Hansel: Output Length Controlling Framework for Large Language Models. In Proceedings of the AAAI Conference on Artificial Intelligence

    2025

  22. [30]

    Stergiadis, E.; Belligoli, Z.; Fainman, E.; and Gusev, I. 2025. Controlling Summarization Length Through EOS Token Weighting. arXiv preprint arXiv:2506.05017

    2025 arXiv

  23. [31]

    H.; Le, Q

    Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Ichter, B.; Xia, F.; Chi, E. H.; Le, Q. V.; and Zhou, D. 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In Advances in Neural Information Processing Systems (NeurIPS 2022), volume 35, 24824--24837

    2022

  24. [32]

    Xu, N.; and Ma, X. 2025. LLM The Genius Paradox: A Linguistic and Math Expert ' s Struggle with Simple Word-based Counting Problems. In Chiruzzo, L.; Ritter, A.; and Wang, L., eds., Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association fo...

    2025

  25. [35]

    Yang, A.; Li, A.; Yang, F.; et al. 2025 c . Qwen3 Technical Report. arXiv:2505.09388

    2025 arXiv

  26. [36]

    X.; Fong, J.; and Su, W

    Yu, Z.; Wu, Z.; Zheng, H.; Yuan, Z. X.; Fong, J.; and Su, W. 2021. LenAtten: An Effective Length Controlling Unit For Text Summarization. arXiv preprint arXiv:2106.00316

    2021 arXiv

  27. [37]

    Yuan, F.; Li, Y.; Li, J.; and et al. 2024. Following Length Constraints in Instructions. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Section 3.1.1 describes the MT‑Bench‑LI setup

    2024

  28. [38]

    Zhang, G.; Li, X.; Sun, L.; Lyu, L.; Liu, Y.; and Su, S. 2025. LIFEBENCH : Length Instruction Following Evaluation Benchmark. arXiv preprint arXiv:2505.16234

    2025 arXiv

  29. [39]

    Zhang, T.; Zhu, C.; Shen, Y.; Luo, W.; Zhang, Y.; Liang, H.; Yang, F.; Lin, M.; Qiao, Y.; Chen, W.; Cui, B.; Zhang, W.; and Zhou, Z. 2024. CFBench : A Comprehensive Constraints-Following Benchmark for LLMs. arXiv preprint arXiv:2408.01122

    2024

  30. [40]

    Zhang, X.; and ... 2025. Zero-Shot Strategies for Length-Controllable Summarization. arXiv preprint arXiv:2501.00233

    2025 arXiv

  31. [41]

    Zhang, X.; Abdul-Mageed, M.; and Lakshmanan, L. V. S. 2024. Autoregressive + Chain of Thought Recurrent: Recurrence’s Role in Language Models’ Computability and a Revisit of Recurrent Transformer. arXiv preprint arXiv:2409.09239

    2024 arXiv

  32. [42]

    Zhang, X.; Cao, J.; and You, C. 2024. Counting Ability of Large Language Models and Impact of Tokenization . arXiv preprint arXiv:2410.19730

    2024 arXiv

  33. [43]

    Zhang, Y.; and He, Z. 2024. Large Language Models Can Not Perform Well in Understanding and Manipulating Natural Language at Both Character and Word Levels? In Al-Onaizan, Y.; Bansal, M.; and Chen, Y.-N., eds., Findings of the Association for Computational Linguistics: EMNLP 2...

    2024

  34. [44]

    P.; Gonzalez, J

    Zheng, L.; Chiang, W.-L.; Sheng, Y.; Zhuang, S.; Wu, Z.; Zhuang, Y.; Lin, Z.; Li, Z.; Li, D.; Xing, E. P.; Gonzalez, J. E.; and Stoica, I. 2023. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. In Advances in Neural Information Processing Systems 36, Datasets and Benchm...

    2023

  35. [45]

    Zhou, J.; Lu, T.; Mishra, S.; Brahma, S.; Basu, S.; Luan, Y.; Zhou, D.; and Hou, L. 2023 a . Instruction-Following Evaluation for Large Language Models. arXiv:2311.07911

    2023 arXiv

  36. [46]

    Zhou, W.; Jiang, Y.; Wilcox, E.; Cotterell, R.; and Sachan, M. 2023 b . Controlled Text Generation with Natural Language Instructions. In Proceedings of the 40th International Conference on Machine Learning (ICML), 11154--11174

    2023

  37. [47]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...

  38. [48]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.