{"id":"032cf433-2f84-46aa-9660-7b746a9968a4","arxiv_id":"2412.14033","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Periodic hidden special tokens that count remaining words during finetuning give LLMs accurate and extrapolatable output length control without hurting output quality.","lead":"This paper introduces Hansel, a finetuning framework that inserts periodic hidden tokens encoding remaining word count into training data, teaching LLMs to hit target output lengths. It reports large reductions in length error across four model architectures and datasets, including lengths never seen in training.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central mechanism is underdetermined: Hansel changes two variables relative to Gretel (a special-token length condition and periodic countdown tokens), and the paper gives no evidence that the model emits the periodic tokens at inference or that they cause the MAE gain.","rationale":"Good-faith reading: the empirical pattern is consistent and the MAE improvements are large, but the paper's mechanistic story is not directly tested. The reader's weakest_assumption focused on emission fidelity; my concern is stronger because even perfect emission would not establish causation, since the design lacks a control with only the initial special token. The proposed suppression test is decisive: it removes exactly the periodic contribution at inference while holding the initial token constant. If suppressing the tokens leaves MAE low, the central claim would reduce to 'a special-token length condition helps finetuning,' which is much weaker than the periodic-countdown claim. If MAE degrades, the periodic protocol is validated. Secondary issues, such as missing error bars, exclusion of infinite generations from reported statistics, and absent code/data, also affect confidence but are less central to the mechanism. The appropriate verdict remains conditional on this mechanism check, matching the reader's CONDITIONAL verdict.","tokens_in":16985,"tokens_out":15869,"duration_ms":142538,"concrete_test":"Reproduce the Phi-2 XSum Hansel model (Delta=20, delta=1). For target lengths 5, 20, 50, 80, 130, decode each test item twice: (A) standard inference; (B) identical inference but mask the logits of all periodic countdown special tokens (|k> and |x>|y> forms) after the first generated position, so the model cannot emit any countdown token. Compare MAE and ROUGE-L between (A) and (B). Also record, in (A), the word offsets at which each countdown token is emitted and the distance from |0> to the final token. If (B)'s MAE stays close to (A)'s, the periodic tokens are not causally necessary; if it degrades sharply, the mechanism is real.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that periodically inserted countdown tokens, not just the target-length signal, drive Hansel's length control. The experiments compare Hansel against Gretel, but this comparison conflates two factors: the length condition is given as a special token (|x>|y>) rather than text, and the reference outputs contain additional periodic tokens. The initial token alone already conveys the exact target length; if a finetuned model can attend to that token and produce the right number of words, the periodic countdown could be epiphenomenal. The paper states 'the special tokens will also appear while inference' (Method) but never reports whether the model actually generates |x-1>, |x-2>, ..., |0> at the expected word offsets, nor how termination relates to |0>. MAE alone cannot distinguish the countdown mechanism from a model that uses only the initial token and learned length estimation. This is load-bearing because the extrapolation claim and the 'general means of length control' conclusion are attributed specifically to the periodic protocol.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Hansel, a finetuning framework that aims to give decoder-only LLMs accurate control over output length. During finetuning, the reference outputs are augmented with special tokens that encode the remaining word count: an initial token of the form |x>|y> gives the full remaining length, and further tokens |x-1>, |x-2>, ..., |0> are inserted at intervals of Delta words. The same tokens are expected to appear at inference. The authors compare against a prompt-based length-control finetuning baseline (Gretel) and vanilla finetuning across four datasets (CNN/DM, XSum, DailyDialog, MultiWOZ) and four base models (Phi-2, BLOOM, OPT, T5), reporting large reductions in mean absolute error (MAE) of output length, robust extrapolation to target lengths outside the training distribution, no degradation in ROUGE-L or G-Eval scores, and elimination of infinite generation. The paper also reports a small-sample transfer experiment in which 50 Hansel-style instruction examples transfer length control to a new task.","tokens_in":17280,"tokens_out":7051,"duration_ms":64354,"significance":"If the empirical claims hold, Hansel is a simple, architecture-agnostic finetuning recipe for length control that could be useful in applied settings such as summarization and dialogue systems, and the extrapolation result is genuinely interesting. The breadth of the evaluation—four datasets, four positional-encoding families, and multiple target lengths—is a notable strength, and the method does not require architecture changes or pretraining modifications. However, the paper's central explanatory claim, that the periodically emitted countdown tokens are what makes the method work, is not directly tested: the comparison to Gretel changes both the representation of the target length (special token vs. text) and the presence of periodic tokens in the reference. In addition, the reported results are single runs without error bars or significance tests, and the hyperparameters Delta and delta are selected using the same test sets on which the headline results are reported. These issues leave the mechanism and the statistical robustness of the central claim underdetermined.","major_comments":[{"comment":"The central comparison, Hansel vs. Gretel (Table 1), conflates two variables: the target length is given as a special token (|x>|y>) rather than as text, and the reference outputs contain periodic countdown tokens. The paper never ablates the periodic tokens. To support the claim that 'the model learns a general means of length control' through periodic tokens, the authors should run a control in which the reference contains the initial |x>|y> token but no periodic tokens (or in which the periodic tokens are not used at inference), and compare its MAE to full Hansel. In addition, the paper states that 'the special tokens will also appear while inference' but never reports whether the model actually emits |x-1>, |x-2>, ..., |0> at the expected word offsets, nor how termination relates to |0>. This is directly measurable from generation logs and is the load-bearing link between the training protocol and the extrapolation results. Without such evidence, the MAE gains could be produced by a model that attends to the initial token alone and learns a length-estimation mapping.","section":"Method: The Hansel dataset / The Hansel framework"},{"comment":"The hyperparameters Delta and delta are selected using the same test sets that are later used for the headline results in Tables 1, 2, and 5. The text reads 'we conclude that Delta = 20 and delta = 1 is the best combination of hyperparameters' after comparing test-set MAE in Table 4, and then those values are used for all other reported experiments on the same test sets. This test-set-based selection can inflate the reported improvements. The authors should hold out a validation split for hyperparameter selection, or at minimum report the results for several hyperparameter settings on a validation set and disclose that the test set was used only once.","section":"Results: The effect of hyperparameters (Table 4)"},{"comment":"No experiment is repeated and no confidence intervals or significance tests are provided. In Table 1, for example, the DailyDialog MAE improves from 0.24 (Gretel) to 0.09 (Hansel); with no variance estimate, this difference may be within run-to-run noise. Given the very small absolute MAE values in the dialogue datasets, the authors should report multiple seeds (at least 3) with mean and standard deviation, or a paired significance test across the test set, to establish that the observed improvements are not artifacts of a single run.","section":"Experimental Setup: Metric and all result tables"},{"comment":"The paper excludes infinite generations from the MAE and quality statistics, noting in 'Results' that such samples are 'counted separately.' However, the number of excluded samples per cell is not reported for Table 2, where the extrapolation comparisons are the strongest claim. The aggregate statement ('8.4 and 15.3 times per 10,000 generations' for vanilla and Gretel, none for Hansel) is insufficient to assess whether the MAE differences at extreme target lengths, e.g., 130 words in DailyDialog (Hansel 2.07 vs. Gretel 18.51), are driven partly by the removal of divergent generations. Please report the per-cell infinite-generation counts for all three methods.","section":"Results: Target length extrapolation (Table 2)"}],"minor_comments":[{"comment":"The term 'hidden special tokens' is never defined precisely. The tokens are visible in the training text and are expected to be emitted at inference; clarify whether 'hidden' means that they are excluded from the natural-language context, or that they are masked in some way during training.","section":"Abstract and Introduction"},{"comment":"The notation |x> (equivalently |x>|0>) and the formula for the initial token |floor(l/Delta)>|l mod Delta> is clear in the example, but the general reader would benefit from an explicit statement that the first token encodes the full remaining length and subsequent tokens count down by Delta, with the residual serving as an offset.","section":"Method: The Hansel dataset"},{"comment":"The claim that 'length control has yet to be achieved even for the larger LLMs' is an overstatement in light of the cited work on length-following in instruction-tuned models (e.g., Yuan et al. 2024) and should be qualified.","section":"Introduction / Related Work"},{"comment":"The aggregation of the four G-Eval category scores into a single number in Tables 1 and 5 is not described. Please state whether the average is over the raw category scores (which have different scales, e.g., 1-3 for fluency/engagingness) or over normalized scores.","section":"Experimental Setup: Metric"},{"comment":"The phrase 'special token's affect on the output distribution' should be 'effect'; the same typo appears elsewhere in the appendix.","section":"Appendix: Ablation Studies"},{"comment":"The appendix states that Hansel training uses 20% vanilla examples and 20% Gretel examples in addition to Hansel examples, and that Gretel training also includes 20% vanilla examples. This mixing ratio is an additional free parameter that is not varied or compared with a purely Hansel dataset; please clarify whether the reported Gretel baseline uses exactly the same vanilla mixture, and discuss any effect of this composition choice.","section":"Appendix: Training and Inference Examples"}],"recommendation":"major_revision","confidential_remarks":"The paper presents a simple, practical idea with broad empirical evaluation, but the central mechanism claim is not yet supported. The most important missing evidence is a direct ablation isolating the periodic countdown tokens from the initial target-length token, plus an analysis of the model's emitted special tokens during inference. The test-set-based hyperparameter selection and lack of significance testing are also fixable within a revision. I see no fundamental obstacle to the method being useful, but the current form is premature for acceptance. I recommend major revision and would be willing to re-review."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Periodic hidden special tokens carrying the remaining word count are a genuinely new idea for length control, and the finetuning-only recipe works across four models and datasets with large MAE reductions and no quality loss. The extrapolation numbers are striking. Worth taking seriously.\n\nWhat's new: the periodic insertion of countdown tokens that decrement every Delta words, plus the delta trick to avoid abrupt termination at |0>. The recipe is simple and architecture-agnostic; the experiments cover four positional encodings, four datasets, and include quality metrics (ROUGE, G-Eval). The hyperparameter tables and the few-shot transfer study in the appendix are useful extras.\n\nThe soft spot is the mechanism claim. The paper says the periodic tokens are the crucial difference from Gretel, but the Hansel-vs-Gretel comparison changes two things at once: the length signal is a special token instead of text, and the reference outputs contain periodic tokens. There is no ablation that keeps the initial token and drops the periodic ones, and no direct measurement of whether the model emits |x-1>, |x-2>, ... at the expected offsets during inference. End-to-end MAE alone cannot rule out the model learning a strong length estimator from the initial token. The extrapolation results are suggestive but not proof of the countdown mechanism.\n\nMinor issues: no error bars or significance tests, Delta and delta are chosen on test-set MAE plus qualitative inspection, infinite generations are excluded from the statistics, and no code/data are released. These are fixable in revision.\n\nWho benefits: people working on summarization, dialogue, and on-device interfaces with strict length constraints. The method is easy to try and likely to be useful. It deserves a serious referee; the mechanism question can be settled with an ablation and a token-emission analysis.","headline":"Periodic countdown tokens are a genuinely new finetuning-only length-control trick with strong empirical results, but the mechanism claim is undercut by the absence of an ablation that isolates the periodic tokens from the initial length token.","tokens_in":17739,"tokens_out":4351,"would_cite":true,"duration_ms":36353,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Periodic hidden tokens give LLMs precise word-count control","keywords":["length control","special tokens","finetuning","decoder language models","extrapolation","mean absolute error","summarization","dialogue generation"],"falsifier":"Decode a Hansel-finetuned model with a target length beyond the training range and record the generated token stream. If the periodic tokens $|x-1\\rangle, |x-2\\rangle, \\ldots, |0\\rangle$ do not appear at the expected stride, or are emitted only in the first few steps, then the low mean absolute error cannot be attributed to the periodic counting mechanism, and the extrapolation result would need a different explanation.","tokens_in":16834,"feed_emoji":"🎯","tokens_out":6639,"duration_ms":52169,"temperature":0.7,"pith_summary":"The paper claims that finetuning a decoder LLM on data augmented with hidden special tokens that record the remaining word count every fixed number of words gives the model a far more accurate and general sense of output length than prompting it to hit a word count. On four summarization and dialogue datasets, this method (Hansel) reduces the mean absolute error between generated and target length to a fraction of what prompt-based finetuning achieves, and it keeps errors small even for target lengths never seen during finetuning, such as 130-word summaries or 5-word replies. The practical payoff is that length control becomes a general capability acquired through finetuning alone, without architectural changes, making it applicable to existing large decoder models.","feed_headline":"Hidden tokens make LLM output lengths precise to under one word","feed_subtitle":"Periodic length-marking tokens beat prompt-based control and extrapolate to unseen target lengths.","key_machinery":"The central object is the periodic hidden-special-token protocol. During finetuning, each reference output is augmented so a token $|x\\rangle\\langle y|$ appears at the start, then tokens $|x-1\\rangle, |x-2\\rangle, \\ldots, |0\\rangle$ are inserted every $\\Delta$ words (excluding the first token), where $x = \\lfloor l/\\Delta\\rfloor$ and $y = l \\bmod \\Delta$ for reference length $l$. A residual parameter $\\delta$ makes some training examples treat the total length as $l-1$ through $l-\\delta$, so the model learns not to stop abruptly at $|0\\rangle$, and the $N=10$ tokens before $|0\\rangle$ are label-masked to avoid teaching incomplete sentences. The protocol is what carries the argument: it gives the model a hard-coded chain-of-thought for counting, and the paper's extrapolation results are its signature.","core_discovery":"Hansel's central claim is that periodically injected hidden special tokens during finetuning teach a decoder LLM to track its own progress toward a target length, turning length control into a general counting skill rather than a memorized range of lengths. The tokens encode the remaining word count: the first token $|x\\rangle\\langle y|$ announces that $x$ full strides of $\\Delta$ words plus $y$ words are left, and after every $\\Delta$ words the next token $|x-1\\rangle, |x-2\\rangle, \\ldots, |0\\rangle$ appears. At inference the model is given the initial token corresponding to the requested length and, according to the claim, reproduces the periodic token stream, which keeps it on track and lets it stop near $|0\\rangle$. The paper's experiments report that this yields mean absolute errors of 0.05 to 0.98 words across four datasets and four model architectures, versus 0.11 to 3.62 for prompt-based finetuning, and that the advantage grows sharply for target lengths far from the training distribution.","pith_inferences":["A direct ablation would be to decode with the initial token but strip the periodic tokens as they appear, or vice versa, to isolate how much of the length control comes from the initial announcement versus the periodic checkpoints.","Because the initial token acts as a length switch, the same finetuned model could serve both length-controlled and unconstrained generation by simply omitting or including that token, which the paper's blended training suggests.","The periodic-token idea could transfer to preference-optimization methods that suffer from overly long outputs, giving direct preference optimization an explicit length channel rather than a length penalty.","Testing with character-level or token-level units might reveal different optimal strides, since the paper notes counting difficulty increases from sentences to tokens to words to characters."],"forward_implications":["Length control becomes a general capability: MAE stays below about one word even when targets are far outside the finetuning distribution, whereas prompt-finetuned models diverge rapidly.","The method is architecture-agnostic: it works with rotary, ALiBi, learned, and T5-bias positional encodings, so it applies to most pretrained decoder LLMs through finetuning only.","Output quality is not sacrificed: ROUGE-L and G-Eval scores for Hansel match or slightly exceed vanilla finetuning across datasets.","Hansel eliminates infinite generation in the tested settings, with no infinite loops in 10,000 generations, while vanilla and prompt-finetuned models produced 8.4 and 15.3 per 10,000 respectively.","The same protocol can control multiple units at once, such as sentences and words, by using separate token families."],"supporting_citations":[{"why":"Introduced feeding remaining-length information into sequence-to-sequence models, the idea Hansel adapts to decoder LLMs.","marker":"Kikuchi et al. 2016"},{"why":"Used a special length marker in training; Hansel's periodic re-marking is the key difference.","marker":"Fan, Grangier, and Auli 2018"},{"why":"Supplies the CNN/DailyMail dataset used for one summarization evaluation.","marker":"Hermann et al. 2015"},{"why":"Supplies the XSum dataset for a second summarization evaluation.","marker":"Narayan, Cohen, and Lapata 2018"},{"why":"Supplies the DailyDialog dataset for dialogue response evaluation.","marker":"Li et al. 2017"},{"why":"Supplies the MultiWOZ dataset for task-oriented dialogue evaluation.","marker":"Zang et al. 2020"},{"why":"Provides the Phi-2 2.7-billion-parameter model used as the primary baseline for the main experiments.","marker":"Li et al. 2023"}],"fun_headline_variants":["Hidden tokens give LLMs exact word-count control","Counting tokens let LLMs hit any target output length","Periodic tokens teach LLMs to extrapolate length control","New trick: hidden tokens keep LLM text at set length"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's success depends on the finetuned model spontaneously re-emitting the periodic special tokens during inference at the same stride used in training; the paper infers this from end-to-end length error rather than directly checking the generated token stream.","fun_headline_variants_meta":{"raw":{"variants":["Hidden tokens give LLMs exact word-count control","Counting tokens let LLMs hit any target output length","Periodic tokens teach LLMs to extrapolate length control","New trick: hidden tokens keep LLM text at set length"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000315,"raw_usage":{"total_tokens":1795,"prompt_tokens":965,"completion_tokens":830,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":581,"completion_tokens_details":{"reasoning_tokens":765}},"tokens_in":581,"tokens_out":830,"duration_ms":6930,"temperature":1.0,"reasoning_tokens":765,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T12:32:11.718611+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Decode a Hansel-finetuned model with a target length beyond the training range and record the generated token stream. If the periodic tokens $|x-1\\rangle, |x-2\\rangle, \\ldots, |0\\rangle$ do not appear at the expected stride, or are emitted only in the first few steps, then the low mean absolute error cannot be attributed to the periodic counting mechanism, and the extrapolation result would need a different explanation.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduced feeding remaining-length information into sequence-to-sequence models, the idea Hansel adapts to decoder LLMs."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Used a special length marker in training; Hansel's periodic re-marking is the key difference."},{"cited_title":"M.; Kocisky, T.; Grefenstette, E.; Espeholt, L.; Kay, W.; Suleyman, M.; and Blunsom, P","cited_arxiv_id":null,"evidence_quote":"Supplies the CNN/DailyMail dataset used for one summarization evaluation."},{"cited_title":"B.; and Lapata, M","cited_arxiv_id":null,"evidence_quote":"Supplies the XSum dataset for a second summarization evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the MultiWOZ dataset for task-oriented dialogue evaluation."}],"review_version":1}