{"id":"a94d8e12-5263-4fc8-a1ae-5d527061d75e","arxiv_id":"2505.06297","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Using an LLM as a next-token predictor with arithmetic coding compresses LLM-generated text about 20x, roughly 4 to 7 times better than Gzip, LZMA, or neural compressors in the paper's benchmarks.","lead":"This paper shows that large language models can act as lossless text compressors for text generated by other large language models, reaching about 20x compression instead of the 3x to 10x of conventional tools. It matters because AI-generated text is growing faster than human text, so storage systems for it may benefit from compressors that understand the text generator.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Lossless claim rests on byte-identical decoder-side LLM probabilities, but Sec. 4.4 concedes nondeterminism and no encode/decode round-trip is reported; without a determinism check or code, the 20x ratios may not be realized by an actual arithmetic coder.","rationale":"The reader's weakest_assumption identifies precisely the point on which the paper's central claim depends: lossless decoding requires the exact same probability model at encode and decode time, while Section 4.4 concedes that LLM behavior is not guaranteed to be deterministic. This is not a disagreement with the community's view of LLM nondeterminism; it is a correctness risk internal to the proposed compression scheme. The paper provides no experimental check that the arithmetic coder actually decodes to the original bytes, and the evaluation does not specify the exact code path used to produce Table 5. Because the abstract's headline comparison to Gzip and the 'lossless' qualifier both rest on this unverified assumption, the concern is load-bearing. The issue is addressable: a round-trip test on the same and different hardware, plus a reported realized bitstream size, would settle it. The rest of the paper's evidence, including the consistent scaling trends and the baseline comparisons, is plausible and points in a sensible direction, so the appropriate verdict remains CONDITIONAL rather than REJECT. My check would not invalidate the paper's contribution if the round-trip succeeds and the realized ratios are close to the reported values.","tokens_in":19398,"tokens_out":1911,"duration_ms":20250,"concrete_test":"Run the full pipeline on a held-out sample from each of the 8 datasets: encode with Llama-3.1-8B using fixed FP16 weights and a fixed batch size, then decode with the same script on the same GPU and on a different GPU or CPU, verifying byte-identical reconstruction each time. Also report the actual compressed bitstream size, including arithmetic coder flush and per-chunk overhead, and compare it against the sum of -log2 P used to derive Table 5. If any round-trip fails, or if the realized ratio drops materially below the reported value, the lossless and >20x claims need qualification.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that arithmetic coding driven by LLM next-token probabilities achieves lossless compression above 20x. For arithmetic coding to be lossless, the decoder must reproduce exactly the same probability distribution over tokens at every step. The framework in Secs. 4.2 and 4.3 silently assumes this, yet Sec. 4.4 explicitly states that LLM outputs are not perfectly deterministic due to floating-point precision, MoE routing, and hardware differences, saying 'even a near-zero temperature may not always yield perfectly consistent results.' The paper never verifies that the same checkpoint, tokenizer, batch size, and inference settings yield bit-identical logits, and it reports no encode-decode round trip confirming byte identity. Moreover, the numbers in Table 5 are presented as compression ratios, but no decoder, archive format, or chunk size for that table is specified; if the ratios are computed from the sum of -log2 P (Eqs. 4-5) rather than from an actual arithmetic coder output, they omit coder flush overhead and any penalty from probability mismatches. Without either a demonstration of deterministic probabilities or a specified mechanism for shipping identical probability tables with the archive, the 'lossless' qualifier is unestablished, and the headline 20x figure may be an ideal code-length estimate rather than a realized compressed file size. This is fixable, but it is the load-bearing assumption of the paper's central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes using LLMs as probability models for arithmetic coding to losslessly compress LLM-generated text. It analyzes the redundancy of LLM-generated data through n-gram statistics, entropy, and mutual information, then compares conventional and neural compressors with an LLM-driven arithmetic coder. Across 8 datasets and 14 LLMs, the authors report compression ratios up to 23.80x, far exceeding Gzip's roughly 3x performance on the same data. The framework is standard: factorize the sequence probability with the chain rule, use an LLM to estimate conditional token probabilities, and feed those probabilities to an arithmetic coder. The paper also studies the effects of chunk size, model scale, dataset scale, instruction tuning, and domain-specific fine-tuning on compression ratio.","tokens_in":19580,"tokens_out":5232,"duration_ms":53571,"significance":"If the lossless round trip is actually realized, this would be a strong empirical demonstration that open-weight LLMs' next-token probabilities match the statistics of LLM-generated text far better than conventional compressors, with clear implications for storing the growing volume of synthetic text. The paper gives a clean, parameter-free derivation of the code-length estimator in Eqs. (3)-(5), evaluates a broad set of models and datasets, and provides a natural explanation for why domain-tuned and instruction-tuned models behave differently. The main weakness is that the headline ratios are currently conditional: arithmetic coding is lossless only when the decoder reproduces exactly the same probability tables as the encoder, and the paper does not demonstrate that this holds. As a measurement of LLM predictability, the results are interesting; as a compression claim, they require additional verification.","major_comments":[{"comment":"The losslessness guarantee of arithmetic coding requires the decoder to reproduce byte-identical conditional probabilities P(xt|x<t) at every step. Section 4.4 explicitly concedes that LLM inference is not fully deterministic across runs due to floating-point precision, MoE routing, and hardware differences, stating that 'even a near-zero temperature may not always yield perfectly consistent results.' The paper reports no encode-decode round trip, no bit-exactness check, and no specification of how a decompressor would reproduce the same logits (checkpoint, tokenizer, precision, batch size, attention implementation). This is load-bearing: if the probabilities differ at any step, the arithmetic coder diverges and the 'lossless >20x' claim fails. The authors should either demonstrate deterministic probability reproduction on their hardware or describe an archive format that ships the probability tables or model configuration, and then verify a complete round trip.","section":"§4.3, §4.4"},{"comment":"Table 5 reports 'Ours' ratios up to 23.80x and the text says arithmetic coding was implemented, but the paper does not state whether S_compressed is the length of an actual arithmetic-coded bitstream or the ideal code length from Eq. (4). Because Eq. (4) defines length as -log2 P, the reported ratio is, by construction, a cross-entropy measurement; a real coder adds flushing, byte-alignment, and tokenization overhead, and Table 5 omits these. The table also does not state the chunk size, context length, or dataset subsample size used; §5.2.4 only says 'unless otherwise specified Llama-3.1-8B.' Since compression ratios depend strongly on chunk size (Fig. 5), the headline numbers are not reproducible as reported. Specify the exact codec, overhead, and settings, and report measured file sizes for at least the main configuration.","section":"§5.3, Table 5, §5.1.3"},{"comment":"The comparison of 'Ours' to the baselines is not fully fair as presented because the authors do not state the hardware, runtime, or model inference settings used for the LLM-based method, while the baseline numbers come from standard compressor executables. Compression ratio alone is not enough: the paper should report compressed size as actually produced by the coder and, ideally, provide code or a detailed artifact description so that the numbers in Table 5 can be reproduced. Without this, the empirical claim that the method 'achieves' these ratios remains an estimate rather than a demonstrated file-based result.","section":"§5.2.4, §5.3"}],"minor_comments":[{"comment":"The abstract labels Gzip the 'state-of-the-art lossless compressor,' but the paper's own Table 3 shows that LZMA and Zstd outperform Gzip on all three pilot datasets and NNCP outperforms all dictionary methods. The comparison should be reframed as 'widely used' or should compare against the strongest baselines, including LZMA and NNCP.","section":"Abstract, §3.3"},{"comment":"The paper claims to be the 'first systematic investigation' of LLM-generated data compression and the 'first to propose leveraging LLMs to compress LLM-generated synthetic data,' but related work [14, 23, 40] already applies LLMs and foundation models to lossless text compression. The novelty claim should be narrowed to the specific setting of LLM-generated data and qualified against these prior works.","section":"Introduction, Related Work"},{"comment":"Table 5 contains several typos and inconsistencies, including the column header 'Artical' instead of 'Article' and inconsistent naming of 'Artical' in the table versus 'Article' in the text; also, §5.4 and §5.5 contain 'Peformance' for 'Performance.'","section":"§5.3, Table 5"},{"comment":"The quadratic extrapolation predicting '16,000 TB within the next decade' is presented without confidence intervals, data-point details, or model diagnostics; it is motivational and should be softened or moved to the introduction with appropriate caveats.","section":"Fig. 1, §1"},{"comment":"The statement that DeepSeek-Coder-1.3B performs comparably to Llama1B 'may be due to insufficient training' is speculative; either provide supporting evidence such as training corpus size or remove the speculation.","section":"§5.7.2"},{"comment":"The claim that the proposed method is 'stable' with dataset scale is based on a narrow range (file sizes 10 to 80 units) and no error analysis; the units are unspecified and should be stated.","section":"§5.6, Fig. 7"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical study of LLM predictability on LLM-generated text, but it is not yet a demonstrated lossless compression system. The central issue is that the reported ratios appear to be ideal code lengths rather than measured arithmetic-coded file sizes, and the determinism requirement for the decoder is acknowledged but not verified. I would ask the authors to provide a working round-trip implementation, or at least a precise archive format with probability-table serialization, before accepting the 'lossless compression' framing. The novelty claims also need tightening in light of prior LLM-based compression work."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper measures something real: open-weight LLMs assign low cross-entropy to text sampled from other LLMs, and arithmetic coding converts that into compression ratios in the 14–24x range. But the headline 'lossless' claim is not demonstrated. Arithmetic decoding requires the exact same probability tables at encode and decode; Section 4.4 of the paper itself argues that LLM outputs are not bit-stable across runs (floating-point precision, MoE routing, hardware). The paper never reports an encode/decode round-trip, never specifies a decoder, and offers no code. As written, the 20x ratios are an ideal code-length estimate from summed -log2 P, not a verified compressed file size. That is the load-bearing flaw.\n\nWhat is actually new is the systematic benchmark: 14 models, 8 LLM-generated datasets, and findings like domain fine-tuning restoring parity (Rho-Math near Qwen14B on math, Qwen2.5-Coder matching Llama8B on code) and larger chunk sizes helping. The mechanism itself is the standard next-token predictor plus arithmetic coder from NNCP and Delétang et al. — the paper cites these — so the contribution is the evaluation, not the architecture. The direction of the results is internally consistent and plausible, and the dataset is timely.\n\nSoft spots, in proportion. First, the determinism issue above is critical and fixable: verify bit-identical logits for a fixed checkpoint and inference config, or ship the probability tables with the archive. Second, the abstract calls Gzip 'state-of-the-art' and contrasts against '3x', but Table 5 shows LZMA and Zstd at roughly 6–10x and NNCP at up to 12.6x. That overstates the baseline. Third, Section 5.4 says instruction-tuned models 'systematically outperform' base models while Section 5.7.1 says they are generally worse; the caption of Fig. 5 even stakes out a middle position. They need to pick one and report numbers consistently. Fourth, no chunk size for the main results, no subsample sizes, no code or data links. All addressable.\n\nThis paper is for compression and data-management readers who care about the fastest-growing text category. It deserves a serious referee — the question is timely and the evaluation, once fixed, would be useful. But I would condition acceptance on a demonstrable lossless round-trip and on reconciling the instruction-tuning results.\n\nRecommendation: engage with it, but require the determinism evidence and code release.","headline":"The benchmark is useful, but the paper's lossless claim is unverified because arithmetic coding requires byte-identical model behavior that the paper itself says is not guaranteed.","tokens_in":20253,"tokens_out":2991,"would_cite":false,"duration_ms":27791,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Encoding LLM-generated text with an LLM's next-token probabilities drives arithmetic coding to lossless compression ratios above 20x, far above Gzip's roughly 3x.","keywords":["lossless compression","LLM-generated text","arithmetic coding","next-token prediction","entropy coding","compression ratio","large language models","neural compression"],"falsifier":"Encode one text on machine A and decode on machine B (or with a different software build) and compare the result to the original text; any mismatch would invalidate the lossless claim. A simpler necessary check: encode the same file twice with identical weights but different floating-point settings and see whether the two bitstreams are byte-identical, which they must be for the method to work.","tokens_in":19056,"feed_emoji":"🗜️","tokens_out":9572,"duration_ms":86740,"temperature":0.7,"pith_summary":"This paper claims that LLM-generated text is so predictable to LLMs because the models themselves are trained to guess the next token, and that this predictability can be turned into a lossless compressor. Driving arithmetic coding with an LLM's next-token probabilities yields compression ratios from 14.62 to 23.80 across eight LLM-generated datasets, far above Gzip's roughly 3x and the roughly 3x to 12.6x range of earlier neural compressors. The central identity is the chain-rule factorization $P(X)=\\prod_t P(x_t \\mid x_{<t})$: the compressed length equals the sum of $\\sum_t -\\log_2 P(x_t \\mid x_{<t})$, so a model that predicts well compresses well. The paper argues this makes LLM-based entropy coding a practical storage method for the fast-growing volume of machine-written text, and shows the advantage holds across model sizes, improves with chunk size up to a saturation point, and is largest when the compressor's training domain matches the text's domain.","feed_headline":"LLMs compress their own text 20x, beating Gzip's 3x","feed_subtitle":"Using next-token probabilities as an entropy code makes machine-written text up to 23x smaller.","key_machinery":"The central object is an autoregressive language model acting as a probability oracle for an arithmetic coder. The model computes $P(x_t \\mid x_{<t})$ for each token via a transformer; the arithmetic coder maps the sequence of conditional probabilities to an interval, emitting a binary fraction whose length is about $\\sum_t -\\log_2 P(x_t \\mid x_{<t})$ (Eqs. 3-5). This works because the next-token objective used in pretraining is the same objective that minimizes code length: a better predictor means smaller KL divergence between predicted and true distributions, hence shorter codes. The paper replaces the small recurrent or transformer predictors used in prior neural compressors with large pretrained LLMs, which is what pushes the ratios into the 20x range.","core_discovery":"The paper establishes that next-token prediction is itself a compression engine. An LLM assigns a conditional probability $P(x_t \\mid x_{<t})$ to each token; arithmetic coding uses these probabilities to narrow an interval and produces a bitstream whose length is close to $\\sum_t -\\log_2 P(x_t \\mid x_{<t})$. On eight datasets of LLM-written text, this scheme achieves lossless compression ratios from 14.62 to 23.80, versus 2.73 to 5.70 for Gzip, 4.24 to 9.76 for LZMA, and 2.95 to 12.56 for the neural baselines. The advantage persists across general-purpose LLMs from 1B to 14B parameters, grows with chunk size up to a saturation point, and is largest when the compressor's training domain matches the text's domain; a 1B math-specialized model compresses math text about as well as a 14B general model. The paper further shows the effect is specific to machine-written text: the same model compresses LLM-generated movie reviews far better than human-written reviews, and the gap widens with context length. The paper explicitly notes in Section 4.4 that LLM inference is not guaranteed deterministic across runs, a necessary condition for the arithmetic decoder to reconstruct the original text.","pith_inferences":["A practical standardized LLM codec is one logical next step: if the model weights and the inference routine are fixed as part of the format, the compression scheme becomes interoperable and its ratios reproducible, much as a codec specification fixes a decoder.","Because the compressed length is the LLM's own surprise about the text, the same pipeline yields a distribution-based signal for whether a text is machine-written; this is an inference not made in the paper but directly testable with the reported setup.","The reported dependence on domain alignment suggests that specialized open-weight compressors could approach or exceed closed-model compression on their own domains, making the 20x result achievable in practice without access to the original generating model."],"forward_implications":["LLM-generated corpora can be stored losslessly at more than 20x compression with an open-weight LLM as the codebook, a large margin over the roughly 3x of Gzip.","The practical recipe for maximum compression is to use the largest available base (non-instruction-tuned) model, the longest context chunk that still shows gains (around 128 to 256 tokens), and, when the text is domain-specific, a model fine-tuned on that domain.","The method's compression ratio stays stable as file size grows, so it fits small and streaming workloads where dictionary compressors cannot accumulate repetition.","The gap between LLM text and human text compression means the value of this approach will increase as the proportion of machine-generated text in stored data grows."],"supporting_citations":[{"why":"It supplies the arithmetic coding routine that turns token probabilities into a bitstream.","marker":"[51]"},{"why":"It provides the entropy lower bound that justifies using minus log probability as the code length.","marker":"[45]"},{"why":"It is the Gzip baseline whose roughly 3x ratio the paper's 20x claim is contrasted against.","marker":"[15]"},{"why":"It is the strongest dictionary baseline (LZMA), with ratios from 4.24 to 9.76 that the method surpasses.","marker":"[58]"},{"why":"It supplies the NNCP baseline and exemplifies the context-prediction approach that the paper scales up with LLMs.","marker":"[3]"},{"why":"It shows language models can act as compressors on other modalities, an idea the paper extends specifically to LLM-generated text.","marker":"[14]"},{"why":"It provides the human-versus-machine Wikipedia and article datasets used in the central evaluation.","marker":"[41]"}],"fun_headline_variants":["LLM next-token coding shrinks its own text 20x, Gzip 3x","Next-token predictability makes LLM text compress 20x vs Gzip's 3x","Self-prediction arithmetic coding: LLM text 20x smaller than Gzip","LLM's own text compresses to 20x better than Gzip's 3x","Next-token lossless coding: LLM text 20x, Gzip 3x"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the same LLM can reproduce exactly the same token probabilities when decoding as when encoding; the paper's own deterministic-inference discussion (Section 4.4) says this may not hold because of floating-point rounding, routing choices, and hardware differences, and the paper never tests it.","fun_headline_variants_meta":{"raw":{"variants":["LLM next-token coding shrinks its own text 20x, Gzip 3x","Next-token predictability makes LLM text compress 20x vs Gzip's 3x","Self-prediction arithmetic coding: LLM text 20x smaller than Gzip","LLM's own text compresses to 20x better than Gzip's 3x","Next-token lossless coding: LLM text 20x, Gzip 3x"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000754,"raw_usage":{"total_tokens":3426,"prompt_tokens":1089,"completion_tokens":2337,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":705,"completion_tokens_details":{"reasoning_tokens":2231}},"tokens_in":705,"tokens_out":2337,"duration_ms":14869,"temperature":1.0,"reasoning_tokens":2231,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:25:18.968181+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Encode one text on machine A and decode on machine B (or with a different software build) and compare the result to the original text; any mismatch would invalidate the lossless claim. A simpler necessary check: encode the same file twice with identical weights but different floating-point settings and see whether the two bitstreams are byte-identical, which they must be for the method to work.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It provides the entropy lower bound that justifies using minus log probability as the code length."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It is the Gzip baseline whose roughly 3x ratio the paper's 20x claim is contrasted against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It is the strongest dictionary baseline (LZMA), with ratios from 4.24 to 9.76 that the method surpasses."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It supplies the NNCP baseline and exemplifies the context-prediction approach that the paper scales up with LLMs."},{"cited_title":"Delétang, A","cited_arxiv_id":null,"evidence_quote":"It shows language models can act as compressors on other modalities, an idea the paper extends specifically to LLM-generated text."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It provides the human-versus-machine Wikipedia and article datasets used in the central evaluation."}],"review_version":1}