LLM-based Source Code Compression via Thresholded Symbol Ranking

Angelo Nardone; Paolo Ferragina

REVIEW 4 major objections 6 minor 37 references

LLM-based Source Code Compression via Thresholded Symbol Ranking

T0 review · 4 major / 6 minor · reviewed 2026-07-31 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Bounding LLM next-token ranks to the top 63, with exceptions packed by ordinary compressors, beats both unbounded LLM rankers and general-purpose tools on source code.

desk verdict Solid empirical systems paper: thresholded LLM ranking is a real, usable win over unbounded LLMZip-style pipelines, and the 30-model code sweep is new; the big gap vs zstd is the claim that needs a held-out check. read the letter →

arxiv 2607.24192 v2 pith:FWXMNBXM submitted 2026-07-27 cs.IT cs.CLcs.LGmath.IT

Angelo Nardone , Paolo Ferragina This is my paper

classification cs.ITcs.CLcs.LGmath.IT

keywords losslesscompressionsourcecodelargelanguagemodelssymbolrankingthresholdeddataexceptionscoding

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that lossless source-code compression improves when large language models rank likely next symbols but only the top few ranks are kept. Earlier LLM compressors wrote every rank without a cap, which hurt speed and left ratio on the table. By stopping at T=1 or T=63, writing out-of-threshold symbols as a short exception list, and then compressing the rank stream and exceptions together with tools such as zstd or bzip, the authors obtain up to 37 percent better ratios and 40 percent higher throughput than the unbounded approach, and up to 82 percent better ratios than strong general-purpose compressors on individual files. The same pipelines produce larger gains on source code than on ordinary English text, which the authors read as evidence that code carries regularities LLMs capture but exact-match dictionary and block-sorting compressors miss. The intended setting is cold storage of large software collections, where ratio outweighs speed and files can be handled independently.

What carries the argument

Thresholded symbol ranking with exceptions: the LLM orders symbols by predicted probability; if the true next symbol lies in the top T it is replaced by its rank, otherwise by an escape while the raw symbol goes to a side list; both streams are then compressed by a general-purpose codec. T=63 is the operating point because more than 95 percent of tokens fall inside it, each rank needs only six bits, and the exception list stays small.

What would settle it

Run the identical T=63 and unbounded pipelines on a held-out source corpus that post-dates the models’ training cutoffs and uses languages or projects absent from the evaluation set; if the relative gains over zstd/bzip and over T=∞ shrink sharply or disappear, the central claim does not hold for realistic archives.

Watch

Extended reading notes

Core claim

Shannon-style thresholded symbol ranking driven by LLM predictions—especially with T=63, escape symbols for the rest, and a general-purpose backend on the two resulting streams—outperforms the unbounded (T=∞) LLM pipelines of prior work by up to 37 percent relative compression and 40 percent throughput, and outperforms strong general-purpose compressors by up to 82 percent relative ratio on single source files (68 percent on reordered archives), with the advantage larger on code than on natural language.

Load-bearing premise

The reported rank skew and the code-versus-text gap assume that the sampled multi-language files, with their size mix and likely overlap with model training data, stand for the real blobs one would archive at scale.

Editorial extensions

If this is right

Cold-storage pipelines can trade GPU time for substantially smaller footprints by applying T-bounded LLM rankers file-by-file.
Lossless compression ratio becomes a practical, task-agnostic yardstick for comparing code-specialized, general, and quantized models.
Per-file processing removes the global reordering step that general-purpose compressors need to reach their best ratios.
Models around one billion parameters, including quantized ones, already sit on a usable speed–ratio frontier.
Source code is a more favorable domain than natural language for LLM-based compressors, so domain-specific archives stand to benefit most.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same top-T-plus-exceptions pattern may transfer to other structured token streams such as bytecode, ASTs, or protocol traces where predictors are strong but unbounded ranks waste bits.
Because rank computation dominates runtime, hybrids that skip LLM calls on flat distributions or after parsing-based deduplication could close more of the throughput gap without giving up the ratio edge.
The code-versus-text gap suggests that evaluation suites for foundation models should treat lossless code compression as a standard probe of structural understanding.
A tiny distilled ranker trained only for reliable top-T prediction could move the method from cold storage toward warmer use.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The useful core here is simple. They take Shannon’s finite-T ranking (the one everyone skipped), run it with modern LLMs on source code, and show that T=63 plus a side exception stream beats the unbounded T=∞ pipelines from LLMZip/FineZip/AlphaZip on both ratio (up to ~37% relative) and throughput (~40%). That internal comparison is clean: same model, same data, mostly an encoding win (6-bit ranks vs 32-bit plus a small escape list). The 30-model Phase-One screen and the code-vs-text numbers in Table 2 are also new; nobody had done the large code study before.

What they do well: two-phase protocol, strong baselines (zstd-22, bzip3, tar-reordered archives per Boffa), rank CDFs that actually justify T=63 (≥95% mass in the top ranks for the code models), and an honest speed story—they never pretend this is production-fast. Parallelism across files is a fair practical point for cold archival. Definitions are standard and the math is just the classical transform; no overclaim there.

Soft spots, in proportion. The stress-test on Stack-Edu / The Stack v2 is real and asymmetric. The T=63 vs T=∞ gain survives contamination. The headline 68–82% gap over general-purpose compressors and the “code has more LLM-friendly regularities” reading of Table 2 do not; those rest on rank skew that memorization would inflate, and the paper never reports overlap stats, date-held-out blobs, or decontamination. They even say the corpus is “aligned with the training corpora of modern LLMs.” That is the main caveat a referee should force them to address. Secondary: no released code/artifacts, throughput still 1–2 orders below zstd, decompression slower than compression. None of that sinks the paper; it just bounds how far the significance claim can travel today.

This is for people who care about lossless code archives, cold storage trade-offs, or using compression as a model probe. Not field-reshaping, but a concrete new operating point and a usable evaluation lens. I would send it to peer review; the measurements are real and the thresholded idea is worth having on the record. I’d cite the T-bounded result and the model sweep; I’d flag the leakage issue if I used the absolute gaps.

Referee Report

4 major / 6 minor

Summary. The paper studies lossless compression of source code using LLMs as predictors in Shannon's symbol-ranking framework. Its main technical contribution is a thresholded variant: instead of recording the exact (unbounded, T=∞) rank of each token as prior LLM-based compressors do, predictions are capped at the top-T ranks (T=1 or T=63), with out-of-threshold tokens written as exceptions in a side stream; both streams are then compressed with general-purpose backends (zstd, bzip2, bzip3). Across 30 models (120M–8B, including code-specialized and quantized variants) and six programming languages (~600 MB total), the T=63 pipeline improves over T=∞ by up to 37% in compression ratio and 40% in throughput, and beats general-purpose compressors by up to 82% on individual files (68% on reordered tar archives), at one-to-two orders of magnitude lower throughput. A final comparison against Delétang et al.'s enwik9 results suggests LLM gains are larger on code than on natural language.

Significance. If the results hold, the paper makes a solid contribution at the intersection of compression and LLM evaluation: (i) a simple, well-motivated thresholded symbol-ranking transform with a concrete encoding-efficiency argument (6-bit vs 32-bit ranks plus a small exception stream) that yields up to 37% ratio and 40% throughput gains over the prior unbounded LLM pipelines; (ii) the first broad empirical study of LLM-based compression on source code, with 30 models spanning code-specialized, general-domain, and quantized variants—considerably more systematic than prior work; (iii) strong general-purpose baselines including zstd-22, bzip3, and the reordered-tar setting of Boffa et al.; and (iv) a clean per-file pipeline that parallelizes trivially, which is practically relevant to large archives such as Software Heritage. The thresholding idea is not deep, but it is effective, honestly benchmarked, and the internal T=63 vs T=∞ comparison is non-circular and convincing.

major comments (4)

[§3 Datasets; §4.2 (82%/68% claims)] Training-data contamination is a load-bearing threat to the two headline external-baseline claims. §3 states the Stack-Edu choice 'guarantees high-quality code aligned with the training corpora of modern LLMs'; Stack-Edu derives from The Stack v2, the stated training corpus of StarCoder2 and a known component of DeepSeekCoder/CodeGemma/Granite-Code training mixes. Memorized files yield systematically lower ranks, inflating the ≥95% top-63 mass and hence the 82%/68% gains over zstd/bzip3, which enjoy no such advantage. Note the internal T=63 vs T=∞ claim is robust to this (same model, same data on both sides). A concrete, cheap fix: rerun the best pipeline on a held-out set of SWH blobs created after the models' training cutoffs (or filtered by SWHID against The Stack v2), and report the delta. At minimum this threat must be discussed explicitly.
[§4.2 'Results on General-Purpose Compression Baselines'; Fig. 2] The 68% figure rests on the strength of the tar-based baseline, but the baseline configuration is under-specified. On a 100 MB reordered archive, zstd's default window (~8 MB) captures little cross-file redundancy unless --long is set; bzip3's block size likewise determines reach. Neither is stated. Moreover, for many small similar files the natural strong baseline is zstd with a trained dictionary (--train) or a long-range matcher (e.g., lrzip/xz with large dictionary); if the baselines were run with default windows, the LLM gain over 'reordered archives' is overstated. Please state exact flags and add a dictionary-trained or long-window baseline.
[Table 2 and 'Text Compression vs. Source Code Compression'] The code-vs-text conclusion is drawn from a cross-paper comparison that confounds pipeline, model, and corpus simultaneously: Delétang et al.'s numbers use arithmetic coding with Chinchilla/LLaMA2 on enwik9, while this work uses symbol ranking with code-specialized models on (likely memorized) code; zstd-22 normalization does not control for these differences, and contamination acts asymmetrically across the two rows. The claim would be far better supported by running the same general-domain models (e.g., Llama-3.2-1B, which you already benchmark) on enwik9 within your own pipeline, giving a controlled text-vs-code contrast.
[§4.2; compression-ratio accounting] Compressed sizes exclude the predictor itself. At the Phase-Two scale (~100 MB per language), the 1–3B parameter models (1–6 GB in bf16, ~0.3–1.5 GB quantized) exceed the data being compressed, so the reported ratios are not self-contained; Delétang et al. [12] report adjusted sizes for this reason. At SWH scale (petabytes) the model amortizes to nothing, so this does not threaten the conclusions, but the amortization argument should be made explicitly, ideally with the break-even corpus size computed for the recommended configurations.

minor comments (6)

[§2 Def. 1–2; §4.2 Methodology] Off-by-one concern in the bit-width formula. Definition 1 indexes ranks from 0 (σ0), so ranks 0..63 are 64 values; adding ESC gives 65 codewords, requiring 7 bits, not ⌈log2(63+1)⌉=6. If ESC is encoded by overloading a rank value or ranks are 1-indexed, say so; otherwise the T=63 stream costs 7 bits/token and some numbers shift.
[§4.2] The sweep T∈{3,7,15,31,63,127,255} is described but only the conclusion (T=63 best) is reported. A small table or curve of ratio/throughput vs T would let readers see how flat the optimum is, which matters for robustness of the central design choice.
[§4.2 (decompression remark)] Decompression throughput is stated to be slower than compression but never quantified. Since decompression is autoregressive (no teacher-forced batching within a file), the gap could be large; concrete numbers are needed for the cold-backup deployment argument.
[Figures 1–2] Fig. 1(b) legend lists ~29 models and is unreadable; consider plotting only the nine selected models or using markers/facets. Axes of Fig. 2 subpanels differ; a shared axis range would ease cross-language comparison.
[§3] No statement on code/data availability. Given the reproducibility motivation in §3 (SWHIDs), releasing the pipeline and the exact SWHID lists would materially strengthen the paper.
[various] Typos/notation: 'We revisitsymbol ranking' (§1); 'per sé' → per se (§2); 'fuzzily and weightly matched' (§2) is informal—rephrase; Table 2 'Llama-3.2-4bit 1B' naming is inconsistent with Table 1; 'DeepSeekCoder-AWQ' vs 'DeepSeekCoder-1.3B A WQ'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: empirical compression measurements against external baselines; T is a hyperparameter, not a fitted quantity renamed as prediction.

full rationale

The paper’s load-bearing claims are measured compression ratios and throughputs of T-bounded LLM symbol-ranking pipelines versus (i) the unbounded T=∞ LLM baseline and (ii) independently specified general-purpose compressors (zstd, bzip2, bzip3). Definitions 1–2 restate Shannon’s classical ranking schemes; ranks and exception streams are produced by frozen pretrained LLMs and then encoded by those external compressors. Choosing T=63 (and backend levels) on the evaluation data is ordinary hyperparameter selection: the reported ratios remain external bit-length measurements, not quantities forced by a fit. There is no self-definitional loop, no parameter fitted on a subset and re-presented as a prediction, no uniqueness theorem imported from overlapping authors, and no ansatz smuggled in via self-citation. Concerns about training-data overlap with Stack-Edu/StarCoder2 affect external validity of the gains, not circularity of the derivation. The evaluation chain is self-contained against external metrics.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The work is empirical systems research. It rests on standard information-theoretic definitions (Shannon ranking), public LLM next-token distributions, and ordinary backend compressors. No new physical entities; the only ‘invented’ pieces are the engineering pipelines. Free parameters are the usual hyperparameters (T, context length, batch size, backend level) selected on the evaluation data.

free parameters (3)

rank threshold T = 63 (primary); also T=1
Discrete grid {1,3,7,15,31,63,127,255} tried; T=63 reported as best. Directly affects rank bit-width and exception rate.
context window M and batch size B = M=512, B=32
Fixed at M=512, B=32 after preliminary GPU-efficiency runs; controls both accuracy and throughput.
backend compressor and level = bzip3-default or zstd-22
zstd/bzip2/bzip3 levels chosen for best reported ratio; not derived.

assumptions (4)

standard math Shannon symbol-ranking transform (Defs. 1–2) plus a deterministic predictor yields a lossless, reversible representation when ranks/exceptions are stored.
Classical; invoked throughout §2.
domain assumption An LLM’s next-token distribution, under fixed tokenization and deterministic decoding, is a valid stationary predictor for ranking.
Standard in the LLM-compression literature the paper builds on; required for reproducibility of ranks.
domain assumption Stack-Edu / SWH samples in six languages with ~3 KB average file size are sufficiently representative of archival source-code blobs.
Underpins external validity of all reported ratios and the code-vs-text comparison (§3).
domain assumption Arithmetic coding is omitted because prior work showed its runtime cost outweighs modest ratio gains.
Stated in §2; shapes the design space the paper explores.

invented entities (1)

T-bounded LLM symbol-ranking pipeline with side exception list
purpose: Bound rank alphabet, cut bit width, and raise throughput while preserving lossless recovery.
Engineering realization of Shannon’s finite-T regime; no independent existence outside this compression system.

how reviews work

0 comments

Cite this review

Pith. "Pith review of LLM-based Source Code Compression via Thresholded Symbol Ranking." pith.science (2026). https://pith.science/paper/FWXMNBXM

@misc{pith2026260724192,
  author       = {Pith},
  title        = {Pith review of: LLM-based Source Code Compression via Thresholded Symbol Ranking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FWXMNBXM}},
  note         = {Machine review of arXiv:2607.24192}
}

abstract

We study the problem of lossless compression of source code, motivated by the storage demands of large-scale software archives, such as Software Heritage (https://www.softwareheritage.org/). General-purpose compressors (e.g., zstd, bzip2) offer a good trade-off between compression ratio and speed, but fail to exploit all special regularities inherent in source code. Recent approaches leverage Large Language Models (LLMs) within Shannon's symbol-ranking framework, relying on a scheme in which the predicted rank can grow arbitrarily. While effective at reducing space, this setting incurs significant throughput degradation, and leaves open the question whether it is necessary to explicitly encode all ranks. In this work, we introduce LLM-based compressors deploying two novel symbol-ranking variants that bound predictions to the top-$T$ ranks ($T=1$ or $63$), with out-of-threshold symbols stored as exceptions and compressed jointly with the rank stream via general-purpose compressors. We conduct the first large-scale evaluation of LLM-based source code compression across 30 LLMs, including general-domain, code-specialized, and quantized models. Our $T$-bounded approach outperforms prior LLM-based compressors both in compression ratio (up to 37% relative improvement) and compression throughput (40% faster). Compared to general-purpose compressors (e.g., zstd, bzip2), we obtain up to 82% relative compression gain but at a lower speed, thus offering a new trade-off point in the compression-speed spectrum. We also show that these gains are stronger on source code than on natural language, suggesting an interesting indication, namely that source code exposes regularities captured by LLMs but missed by general-purpose exact-match-based compressors. We conclude by commenting on open problems that offer theoretical and practical avenues of research.

Figures

Figures reproduced from arXiv: 2607.24192 by the authors.

**Figure 2.** Throughput (log scale) versus compressed percentage for the best LLM [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

37 extracted references · 7 linked inside Pith

[12]

In: International Conference on Learning Representations

Delétang, G., Ruoss, A., Duquenne, P.A., Catt, E., Genewein, T., Mattern, C., Grau-Moya, J., Wenliang, L.K., Aitchison, M., Orseau, L., et al.: Language mod- eling is compression. In: International Conference on Learning Representations. vol. 2024, pp. 14165–14181 (2024)

2024
[1]

ACM Transactions on Information Systems (TOIS)37(1), 1–30 (2018)

Alakuijala, J., Farruggia, A., Ferragina, P., Kliuchnikov, E., Obryk, R., Szabadka, Z., Vandevenne, L.: Brotli: A general-purpose data compressor. ACM Transactions on Information Systems (TOIS)37(1), 1–30 (2018)

2018
[2]

arXiv preprint arXiv:2502.02737 (2025)

Allal, L.B., Lozhkov, A., Bakouch, E., Blázquez, G.M., Penedo, G., et al.: Smol- LLM2: When smol goes big – data-centric training of a small language model. arXiv preprint arXiv:2502.02737 (2025)

arXiv 2025
[3]

Bellard, F.: Lossless data compression with neural networks (2019), available at https://bellard.org/nncp/nncp.pdf

2019
[4]

Communications of the ACM29(4), 320–330 (1986)

Bentley, J.L., Sleator, D.D., Tarjan, R.E., Wei, V.K.: A locally adaptive data com- pression scheme. Communications of the ACM29(4), 320–330 (1986)

1986
[5]

BigCode: The stack v2 (2024), software repository at Hugging Face: https://huggingface.co/datasets/bigcode/the-stack-v2

2024
[6]

Journal of Systems and Software227, 112429 (2025)

Boffa, A., et al.: On the compressibility of large-scale source code datasets. Journal of Systems and Software227, 112429 (2025)

2025
[7]

Algorithms for Molecular Biology14(1), 13 (2019)

Boucher, C., Gagie, T., Kuhnle, A., Langmead, B., Manzini, G., Mun, T.: Prefix- free parsing for building big bwts. Algorithms for Molecular Biology14(1), 13 (2019)

2019

Show all 37 references

[8]

Burrows, M., Wheeler, D.J.: A block-sorting lossless data compression algorithm. Tech. rep., Digital Equipment Corporation, Systems Research Center (1994)

1994
[9]

The Computer Journal40(2–3), 67–75 (1997)

Cleary, J.G., Teahan, W.J.: Unbounded length contexts for PPM. The Computer Journal40(2–3), 67–75 (1997)

1997
[10]

IEEE Transactions on Communications32(4), 396–402 (1984)

Cleary, J.G., Witten, I.H.: Data compression using adaptive coding and partial string matching. IEEE Transactions on Communications32(4), 396–402 (1984)

1984
[11]

Collet, Y.: Zstandard compression and the application/zstd media type (2021), rFC 8878, Internet Engineering Task Force

2021
[13]

Deutsch, P., Gailly, J.L.: Rfc1950: Zlib compressed data format specification ver- sion 3.3 (1996)

1996
[14]

Facebook: Zstandard (Zstd): Fast real-time compression algorithm (2025), software repository

2025
[15]

Fenwick, P.: Block sorting text compression – final report. Tech. rep., University of Auckland (1996)

1996
[16]

Cambridge University Press (2023)

Ferragina, P.: Pearls of Algorithm Engineering. Cambridge University Press (2023)

2023
[17]

In: Companion Proceedings of the ACM Web Conference

Fontana, T., Vigna, S., Zacchiroli, S.: Webgraph: The next generation (is in rust). In: Companion Proceedings of the ACM Web Conference. pp. 686–689 (2024)

2024
[18]

Accessed: 2025-10-28

GitHub: News and insights (2025), available at https://github.blog/news-insights/. Accessed: 2025-10-28

2025
[19]

arXiv preprint arXiv:1811.08162 (2018)

Goyal, M., Tatwawadi, K., Chandak, S., Ochoa, I.: Deepzip: Lossless data com- pression using recurrent neural networks. arXiv preprint arXiv:1811.08162 (2018)

2018 arXiv
[20]

HuggingFaceTB: Stack-edu dataset (2025), dataset available via Hugging Face

2025
[21]

arXiv preprint arXiv:2001.08361 (2020)

Kaplan, J., McCandlish, S., Henighan, T., Brown, T.B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., Amodei, D.: Scaling laws for neural language models. arXiv preprint arXiv:2001.08361 (2020)

2001 arXiv
[22]

In: Proceedings of the Data Compression Conference

Lelewer, D.A., Hirschberg, D.S.: Streamlining context models for data compression. In: Proceedings of the Data Compression Conference. p. 91 (1991) 14 A. Nardone and P. Ferragina

1991
[23]

arXiv preprint arXiv:2402.19173 (2024)

Lozhkov, A., Li, R., Allal, L.B., Cassano, F., Lamy-Poirier, J., Tazi, N., Tang, A., Pykhtar, D., Liu, J., Wei, Y., et al.: StarCoder 2 and The Stack v2: The next generation. arXiv preprint arXiv:2402.19173 (2024)

2024 arXiv
[24]

In: Proceedings of the 13th International Florida Artificial Intelligence Research Society Conference (FLAIRS)

Mahoney, M.V.: Fast text compression with neural networks. In: Proceedings of the 13th International Florida Artificial Intelligence Research Society Conference (FLAIRS). pp. 230–234 (2000)

2000
[25]

arXiv preprint arXiv:2409.17141 (2024)

Mittu, F., Bu, Y., Gupta, A., Devireddy, A., Ozdarendeli, A.E., Singh, A., Anu- manchipalli, G.: Finezip: Pushing the limits of large language models for practical lossless text compression. arXiv preprint arXiv:2409.17141 (2024)

2024 arXiv
[26]

IEEE Transactions on Communications38(11), 1917–1921 (1990)

Moffat, A.: Implementing the PPM data compression scheme. IEEE Transactions on Communications38(11), 1917–1921 (1990)

1917
[27]

Probability Theory and Related Fields125(2), 259–265 (2003)

Morris, B.: The components of the wired spanning forest are recurrent. Probability Theory and Related Fields125(2), 259–265 (2003)

2003
[28]

arXiv preprint arXiv:2409.15046 (2024)

Narashiman, S.S., Chandrachoodan, N.: Alphazip: Neural network-enhanced loss- less text compression. arXiv preprint arXiv:2409.15046 (2024)

2024 arXiv
[29]

Cambridge University Press (2016)

Navarro, G.: Compact data structures: a practical approach. Cambridge University Press (2016)

2016
[30]

Request for Comments: 1952 (1996)

Peter, D.: Gzip file format specification version 4.3. Request for Comments: 1952 (1996)

1952
[31]

IEEE Transactions on Neural Networks7(1), 142–146 (1996)

Schmidhuber, J., Heil, S.: Sequential neural text compression. IEEE Transactions on Neural Networks7(1), 142–146 (1996)

1996
[32]

Accessed: 2025- 10-28

Seward, J.: bzip2 and libbzip2: A program and library for data compression (2000), available at https://sourceware.org/pub/bzip2/docs/manual.pdf. Accessed: 2025- 10-28

2000
[33]

Bell System Technical Journal30(1), 50–64 (1951)

Shannon, C.E.: Prediction and entropy of printed english. Bell System Technical Journal30(1), 50–64 (1951)

1951
[34]

Accessed: 2025-10-28

Szewczyk, K.: bzip3: A better, faster and stronger spiritual successor to BZip2 (2022), available at https://github.com/kspalaiologos/bzip3. Accessed: 2025-10-28

2022
[35]

arXiv preprint arXiv:2306.04050 (2023)

Valmeekam, C.S.K., Narayanan, K., Kalathil, D., Chamberland, J.F., Shakkottai, S.: LLMzip: Lossless text compression using large language models. arXiv preprint arXiv:2306.04050 (2023)

2023 arXiv
[36]

In: Advances in Neural Information Processing Systems

Vaswani,A.,Shazeer,N.,Parmar,N.,Uszkoreit,J.,Jones,L.,Gomez,A.N.,Kaiser, Ł., Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems. vol. 30 (2017)

2017
[37]

Communications of the ACM30(6), 520–540 (1987)

Witten, I.H., Neal, R.M., Cleary, J.G.: Arithmetic coding for data compression. Communications of the ACM30(6), 520–540 (1987)

1987

Pith tools

Reviewed July 31, 2026 · model on record in the stance chip above.

[1] [12]

In: International Conference on Learning Representations

Delétang, G., Ruoss, A., Duquenne, P.A., Catt, E., Genewein, T., Mattern, C., Grau-Moya, J., Wenliang, L.K., Aitchison, M., Orseau, L., et al.: Language mod- eling is compression. In: International Conference on Learning Representations. vol. 2024, pp. 14165–14181 (2024)

2024

[2] [1]

ACM Transactions on Information Systems (TOIS)37(1), 1–30 (2018)

Alakuijala, J., Farruggia, A., Ferragina, P., Kliuchnikov, E., Obryk, R., Szabadka, Z., Vandevenne, L.: Brotli: A general-purpose data compressor. ACM Transactions on Information Systems (TOIS)37(1), 1–30 (2018)

2018

[3] [2]

arXiv preprint arXiv:2502.02737 (2025)

Allal, L.B., Lozhkov, A., Bakouch, E., Blázquez, G.M., Penedo, G., et al.: Smol- LLM2: When smol goes big – data-centric training of a small language model. arXiv preprint arXiv:2502.02737 (2025)

arXiv 2025

[4] [3]

Bellard, F.: Lossless data compression with neural networks (2019), available at https://bellard.org/nncp/nncp.pdf

2019

[5] [4]

Communications of the ACM29(4), 320–330 (1986)

Bentley, J.L., Sleator, D.D., Tarjan, R.E., Wei, V.K.: A locally adaptive data com- pression scheme. Communications of the ACM29(4), 320–330 (1986)

1986

[6] [5]

BigCode: The stack v2 (2024), software repository at Hugging Face: https://huggingface.co/datasets/bigcode/the-stack-v2

2024

[7] [6]

Journal of Systems and Software227, 112429 (2025)

Boffa, A., et al.: On the compressibility of large-scale source code datasets. Journal of Systems and Software227, 112429 (2025)

2025

[8] [7]

Algorithms for Molecular Biology14(1), 13 (2019)

Boucher, C., Gagie, T., Kuhnle, A., Langmead, B., Manzini, G., Mun, T.: Prefix- free parsing for building big bwts. Algorithms for Molecular Biology14(1), 13 (2019)

2019