pith. sign in

arxiv: 2306.04050 · v2 · pith:JOPDBC2Unew · submitted 2023-06-06 · 💻 cs.IT · cs.CL· cs.LG· math.IT

LLMZip: Lossless Text Compression using Large Language Models

Chandra Shekhara Kaushik Valmeekam , Krishna Narayanan , Dileep Kalathil , Jean-Francois Chamberland , Srinivas Shakkottai This is my paper
classification 💻 cs.IT cs.CLcs.LGmath.IT
keywords compressionlanguagelargelosslesstextciteenglishestimates
0
0 comments X
read the original abstract

We provide new estimates of an asymptotic upper bound on the entropy of English using the large language model LLaMA-7B as a predictor for the next token given a window of past tokens. This estimate is significantly smaller than currently available estimates in \cite{cover1978convergent}, \cite{lutati2023focus}. A natural byproduct is an algorithm for lossless compression of English text which combines the prediction from the large language model with a lossless compression scheme. Preliminary results from limited experiments suggest that our scheme outperforms state-of-the-art text compression schemes such as BSC, ZPAQ, and paq8h.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Adapting Diffusion Language Models for Lossless Pixel-Level Image Transmission

    cs.IT 2026-06 unverdicted novelty 6.0

    DDM-SSCC adapts diffusion language models for separate source-channel coding to enable lossless pixel-level image transmission with improved exact recovery on standard datasets.

  2. Language Modeling Is Compression

    cs.LG 2023-09 accept novelty 6.0

    Large language models serve as strong general-purpose lossless compressors for text, images, and audio, outperforming domain-specific methods and revealing insights into scaling, tokenization, and in-context learning.

  3. SingGuard: A Policy-Adaptive Multimodal LLM Guardrail with Dynamic Reasoning

    cs.CV 2026-06 unverdicted novelty 5.0

    SingGuard presents a policy-adaptive multimodal LLM guardrail family with hybrid reasoning regimes and a new benchmark of 56,340 examples, claiming SOTA F1 across 35 datasets and improved policy adherence under runtim...

  4. SingGuard: A Policy-Adaptive Multimodal LLM Guardrail with Dynamic Reasoning

    cs.CV 2026-06 unverdicted novelty 5.0

    SingGuard introduces a policy-adaptive multimodal LLM guardrail with dynamic reasoning regimes and SingGuard-Bench, reporting SOTA F1 scores across 35 datasets and improved policy-following accuracy under runtime shifts.

  5. Text-Preserving Lossy Text Compression: A Study of Strategic Deletion and LLM Reconstruction

    cs.CL 2026-05 unverdicted novelty 5.0

    Strategic deletion strategies paired with LLM reconstruction enable lossy semantic text compression, where word-frequency deletion serves as a competitive low-cost baseline across retention rates on news data.

  6. Real-Time Text Transmission via LLM-Based Entropy Coding over Fixed-Rate Channels

    cs.IT 2026-05 unverdicted novelty 5.0

    LLM-based entropy coding over fixed-rate channels improves real-time text transmission by reducing bits per character with scale, shifting optimal coders from complex to simpler ones like Huffman when channels are ove...

  7. TextEconomizer: Enhancing Lossy Text Compression with Denoising Transformers and Entropy Coding

    cs.CL 2026-06 unverdicted novelty 3.0

    Presents TextEconomizer, a transformer-based encoder-decoder for lossy text compression claiming 5.39x ratio, near-perfect semantic quality via standard metrics, and 153x fewer parameters than comparables.