Pith. sign in

REVIEW 3 major objections 6 minor 34 references

Lossless image compression can be done by adapting frozen language models in pixel space, without using their text tokenizers.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · grok-4.5

2026-07-10 11:04 UTC pith:U6UNPY72

load-bearing objection Clean tokenizer-free interface that makes frozen multi-family LLMs usable as pixel entropy models; rates are competitive mainly against JPEG-XL/DLPR and P2-LLM under independent 16x16 patches. the 3 major comments →

arxiv 2607.08221 v1 pith:U6UNPY72 submitted 2026-07-09 cs.CV

LUMI: Tokenizer-Agnostic LLM-Based Lossless Image Compression

classification cs.CV
keywords lossless image compressionlarge language modelsentropy modelingfrozen foundation modelstokenizer-agnosticpixel embeddingarithmetic coding
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that the usual way of squeezing images through language models—turning pixel numbers into text and reading probabilities from the vocabulary—is the wrong interface. Tokenizers split the same number differently across model families, so the coding events themselves become family-specific. LUMI instead maps raw intensity, color channel, and local position straight into the continuous embedding space of a frozen decoder-only model, then predicts a clean 256-way distribution over pixel values for arithmetic coding. Only a small external adapter (pixel embedding, position tables, soft prefix, and output head) is trained. Across natural, medical, and remote-sensing images, and with LLaMA, Qwen, and Gemma backbones, the method matches or beats tokenizer-based LLM baselines and classical codecs while remaining portable across tokenizers. The broader claim is that foundation models can serve as reusable entropy engines once image symbols are presented in their native alphabet rather than as language tokens.

Core claim

LLM-based lossless RGB compression is best formulated as pixel-space adaptation of frozen foundation models: map intensity, channel, and intra-patch position into continuous embeddings, keep the backbone fixed, and decode with a dedicated 256-way head, rather than representing pixels as tokenizer-dependent text.

What carries the argument

The tokenizer-free pixel interface (PixEmb + intra-patch position encoding + 256-way head): a 7-D intensity/channel descriptor projected into LLM embedding space, plus row-column position codes, feeding a frozen decoder that emits exact pixel probabilities for arithmetic coding.

Load-bearing premise

That modeling each 16-by-16 patch independently with a frozen language model already captures enough local statistics for competitive compression rates.

What would settle it

Train and evaluate the same external modules on identical data and splits, but with full inter-patch context or larger overlapping tiles; if rates do not improve over the independent-patch LUMI baseline, the sufficiency claim holds; if they drop sharply below classical codecs, the assumption fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. LUMI proposes a tokenizer-agnostic interface for lossless RGB image compression that attaches lightweight external modules (PixEmb intensity–channel embedding, gated row–column intra-patch position encoding, soft prefix, and a 256-way pixel head) to frozen decoder-only LLMs (LLaMA, Qwen, Gemma). Pixel values are mapped into continuous embedding space rather than textual numeric tokens, so the same pipeline avoids family-specific tokenizer fragmentation and predicts a native 256-symbol alphabet for arithmetic coding. Only the external adapters are trained. Experiments on Kodak, BRACS, and BED4RS report in-domain BPP competitive with JPEG-XL and DLPR and with a LoRA-adapted P2-LLM baseline, leave-one-domain-out generalization, ablations of PixEmb/INP/prompting, and model/data scaling on Qwen3.

Significance. The main contribution is a clean formulation of LLM-based lossless image coding as pixel-space adaptation of frozen foundation models rather than tokenizer-bound language-symbol modeling. Demonstrating a single external interface that works across three tokenizer families, with only small trainable modules and held-out BPP measured under arithmetic coding, is a useful engineering and conceptual result for the compression and foundation-model communities. Tokenizer-fragmentation evidence (Table II), embedding visualizations, and consistent multi-backbone behavior strengthen the portability claim. The work is appropriately scoped as an interface study; it does not claim to redefine the state of specialized neural lossless codecs.

major comments (3)
  1. Tables III–IV and §IV-C frame LUMI as achieving “competitive” rates, but the non-LLM comparison set is limited to JPEG-XL and DLPR. Section II-A cites stronger modern learned lossless methods (hierarchical residual, invertible flows, bit-plane models) that are not re-run under the same patch protocol. Without those anchors, the headline performance claim is only weakly supported relative to the broader learned-codec literature, even though the gain over tokenizer-based P2-LLM baselines is clear.
  2. Section III-G and the Limitations section state that non-overlapping 16×16 patches are compressed independently (T=768, no inter-patch context). This design choice is load-bearing for the claim that frozen LLMs act as effective image entropy models: classical and specialized neural codecs exploit longer-range structure that LUMI never models. The paper should either quantify the cost of this restriction (e.g., vs. larger patches or a simple inter-patch conditioner) or more carefully qualify “competitive” as holding under a deliberately restricted context regime.
  3. Table I reports only 4 training images for BRACS (9,767 patches). In-domain and leave-one-domain-out BRACS numbers in Tables III–IV and the scaling tables therefore rest on a very small image-level sample. Without image-level resampling, error bars, or multi-split results, the medical-domain and cross-domain robustness claims are statistically under-supported relative to Kodak and BED4RS.
minor comments (6)
  1. No standard errors or run-to-run variance are reported for any BPP table; even a few seeds on the adapter training would help assess stability of the small gains over JPEG-XL/DLPR.
  2. Figure 2’s embedding visualization is qualitative; a simple neighborhood or channel-separation metric would make the PixEmb vs. tokenizer comparison more rigorous.
  3. Equation (11) introduces a 7-D descriptor with polynomial and Fourier features; a short justification or ablation of the sinusoidal terms (beyond 4-D vs. 7-D channel identity) would clarify design choices.
  4. Decoding latency is noted as a limitation but never quantified; a brief wall-clock comparison against JPEG-XL/DLPR on the same hardware would contextualize practicality.
  5. Minor notation: β in Eq. (14) is called a “learnable scalar gate” but its learned range or initialization is not reported; likewise soft-prefix length P=16 is fixed without sensitivity analysis.
  6. Typographical consistency: “P 2-LLM” / “P2-LLM” spacing and “F . Arithmetic Coding” section heading spacing should be cleaned for camera-ready.

Circularity Check

0 steps flagged

No circularity: standard supervised NLL training of external adapters evaluated by held-out arithmetic-coding BPP against external codecs.

full rationale

LUMI is an empirical systems paper, not a first-principles derivation. The probabilistic objective (Eqs. 3–4, 21–22) is ordinary autoregressive cross-entropy over the 256-symbol pixel alphabet; BPP is the same quantity measured on held-out patches under arithmetic coding, not a fitted constant renamed as a prediction. Trainable modules (PixEmb, INP, soft prefix, 256-way head) are optimized on training splits and reported on image-disjoint test splits and leave-one-domain-out settings (Tables III–IV, VIII–X). Classical codecs (JPEG-XL, DLPR) and the P2-LLM baseline supply external anchors; P2-LLM is used as a comparator, not as a uniqueness theorem or load-bearing premise that forces LUMI’s rates. Fourier features, prefix tuning, and LoRA are cited from independent prior work as standard tools, not smuggled ansätze that define the result. Independent non-overlapping 16×16 patches and frozen-backbone limits are stated as design choices/limitations, not hidden circular reductions. No equation reduces by construction to its own fitted input; score 0 with empty steps is the correct finding.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 2 invented entities

The work is empirical systems research. Load-bearing modeling choices are the 16×16 independent-patch factorization, the particular 7-D intensity/channel descriptor, the additive row-column position tables, and the decision to freeze the LLM. No new physical entities are postulated; free parameters are ordinary training hyper-parameters and architectural sizes.

free parameters (4)
  • patch size Hp×Wp = 16×16
    Fixed at 16×16; determines sequence length T=768 and the spatial tables; chosen for context-length manageability rather than derived.
  • soft-prefix length P = 16
    Set to 16; a free architectural hyper-parameter of the reprogramming interface.
  • PixEmb / head / INP / prefix learning rates = 5e-4 / 5e-4 / 1e-4 / 3e-4
    Hand-chosen AdamW rates (5e-4, 5e-4, 1e-4, 3e-4); affect final BPP.
  • INP scalar gate β and row/column tables
    Learned parameters that control spatial bias strength; fitted on training patches.
axioms (3)
  • standard math Autoregressive negative log-likelihood under arithmetic coding equals ideal codelength (standard Shannon–arithmetic coding identity).
    Used throughout Section III-A to equate model quality with BPP.
  • domain assumption A frozen decoder-only LLM’s hidden states remain useful contextual features for non-text symbols once continuous embeddings are supplied.
    Central modeling premise of the frozen-backbone design (Sections III-D, III-G).
  • ad hoc to paper Independent non-overlapping patches are an acceptable approximation for the evaluated benchmarks.
    Explicitly adopted for context length and parallelism; inter-patch context deferred (III-G, Limitations).
invented entities (2)
  • PixEmb 7-D intensity-channel descriptor + MLP no independent evidence
    purpose: Map raw sub-pixel intensity and channel identity into LLM embedding space without text tokenization.
    New interface module; no independent evidence outside the compression experiments.
  • Intra-patch position encoding (INP) with gated row+column tables no independent evidence
    purpose: Restore 2-D spatial coordinates after flattening.
    Paper-specific axial embedding design; evaluated only via BPP ablations.

pith-pipeline@v1.1.0-grok45 · 20533 in / 2601 out tokens · 25308 ms · 2026-07-10T11:04:40.572204+00:00 · methodology

0 comments
read the original abstract

Large language model (LLM)-based lossless image compression methods typically represent pixel data through the native text interface of a pretrained model, converting pixel values into token sequences that the LLM processes through its vocabulary head. This design shows that pretrained language models can provide probability estimates for image coding, but it also couples compression to tokenizer behavior, vocabulary-specific numeric tokens, and model-family-specific adaptation. In this paper, we present LUMI (LLM-based Unified Model-agnostic lossless Image compression), a tokenizer-agnostic framework for lossless RGB image compression with frozen LLM backbones. LUMI replaces pixel-as-text tokenization with a pixel embedding module that maps raw intensity and channel information into the continuous embedding space of the LLM. It further introduces intra-patch position encoding to retain two-dimensional spatial structure after flattening, and uses a 256-way prediction head to produce probabilities over the native pixel alphabet. Only the pixel embedding, position encoding, soft-prefix parameters, and prediction head are trained, while the LLM backbone remains fixed. Experiments on natural, medical, and remote-sensing image benchmarks with LLaMA, Qwen, and Gemma backbones show that LUMI provides a unified interface across tokenizer families, achieves competitive compression rates, and improves cross-domain robustness over tokenizer-based LLM compression baselines. These results formulate LLM-based lossless image compression as pixel-space adaptation of frozen foundation models rather than tokenizer-specific language-symbol modeling.

Figures

Figures reproduced from arXiv: 2607.08221 by Chengkai Wu, Chris Xing Tian, Haoliang Li, Kecheng Chen, Rongqun Lin, Shiqi Wang, Siwei Ma, Xiandong Meng, Ziyu Wang.

Figure 1
Figure 1. Figure 1: Overview of the proposed LUMI framework. A rasterized RGB patch is modeled as a sequence of sub-pixel symbols, while the pixel-as-text route [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Embedding-space visualization of LLaMA tokenizer embeddings (left) and 7-D PixEmb representations (right). We sample 40 RGB pixels in HSV [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Representative sub-pixel prediction distributions on BED4RS. The three panels show consecutive sub-pixel predictions corresponding to one complete [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 34 canonical work pages · 9 internal anchors

  1. [1]

    Evaluating Large Language Models Trained on Code

    M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y . Burda, N. Joseph, G. Brockmanet al., “Evaluating large language models trained on code,”arXiv preprint arXiv:2107.03374, 2021

  2. [2]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V . Le, and D. Zhou, “Chain-of-thought prompting elicits reasoning in large language models,” inAdvances in Neural Information Processing Systems, vol. 35, 2022, pp. 24 824–24 837

  3. [3]

    Retrieval- augmented generation for knowledge-intensive nlp tasks,

    P. Lewis, E. Perez, A. Piktus, F. Petroni, V . Karpukhin, N. Goyal, H. K ¨uttler, M. Lewis, W.-t. Yih, T. Rockt ¨aschelet al., “Retrieval- augmented generation for knowledge-intensive nlp tasks,” inAdvances in Neural Information Processing Systems, vol. 33, 2020, pp. 9459–9474

  4. [4]

    Flamingo: A visual language model for few-shot learning,

    J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y . Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynoldset al., “Flamingo: A visual language model for few-shot learning,” inAdvances in Neural Information Processing Systems, vol. 35, 2022, pp. 23 716–23 736

  5. [5]

    Language modeling is compression,

    G. Del ´etang, A. Ruoss, P.-A. Duquenne, E. Catt, T. Genewein, C. Mattern, J. Grau-Moya, L. K. Wenliang, M. Aitchison, L. Orseau, M. Hutter, and J. Veness, “Language modeling is compression,” inThe Twelfth International Conference on Learning Representations, 2024. [Online]. Available: https://openreview.net/forum?id=jznbgiynus

  6. [6]

    T. M. Cover and J. A. Thomas,Elements of Information Theory, 2nd ed. Wiley-Interscience, 2006

  7. [7]

    Arithmetic coding for data compression,

    I. H. Witten, R. M. Neal, and J. G. Cleary, “Arithmetic coding for data compression,”Communications of the ACM, vol. 30, no. 6, pp. 520–540, 1987

  8. [8]

    Large Language Model for Lossless Image Compression with Visual Prompts

    J. Du, C. Zhou, N. Cao, G. Chen, Y . Chen, Z. Cheng, L. Song, G. Lu, and W. Zhang, “Large language model for lossless image compression with visual prompts,”arXiv preprint arXiv:2502.16163, 2025. [Online]. Available: https://arxiv.org/abs/2502.16163

  9. [9]

    Deep lossy plus residual coding for lossless and near-lossless image compression,

    Y . Bai, X. Liu, K. Wang, X. Ji, X. Wu, and W. Gao, “Deep lossy plus residual coding for lossless and near-lossless image compression,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 5, pp. 3577–3594, 2024

  10. [10]

    Large language models for lossless image compression: Next- pixel prediction in language space is all you need,

    K. Chen, P. Zhang, H. Liu, J. Liu, Y . Liu, J. Huang, S. Wang, H. Yan, and H. Li, “Large language models for lossless image compression: Next- pixel prediction in language space is all you need,” inThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. [Online]. Available: https://openreview.net/forum?id=FXBBy1caOX

  11. [11]

    Png (portable network graphics) specification version 1.0,

    T. Boutell, “Png (portable network graphics) specification version 1.0,” World Wide Web Consortium, Tech. Rep., 1997

  12. [12]

    The loco-i lossless image compression algorithm: Principles and standardization into jpeg- ls,

    M. J. Weinberger, G. Seroussi, and G. Sapiro, “The loco-i lossless image compression algorithm: Principles and standardization into jpeg- ls,”IEEE Transactions on Image processing, vol. 9, no. 8, pp. 1309– 1324, 2000

  13. [13]

    Jpeg xl next-generation image compression architecture and coding tools,

    J. Alakuijala, R. V . Asseldonk, S. Boukortt, M. Bruse, I.-M. Coms ¸a, M. Firsching, T. Fischbacher, E. Kliuchnikov, S. Gomez, R. Obryket al., “Jpeg xl next-generation image compression architecture and coding tools,” inApplications of digital image processing XLII. SPIE, 2019, pp. 112–124

  14. [14]

    Pixel recurrent neural networks,

    A. V . D. Oord, N. Kalchbrenner, and K. Kavukcuoglu, “Pixel recurrent neural networks,” inInternational conference on machine learning. PMLR, 2016, pp. 1747–1756

  15. [15]

    PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications

    T. Salimans, A. Karpathy, X. Chen, and D. P. Kingma, “Pixelcnn++: Improving the pixelcnn with discretized logistic mixture likelihood and other modifications,”arXiv preprint arXiv:1701.05517, 2017

  16. [16]

    Practical full resolution learned lossless image compression,

    F. Mentzer, E. Agustsson, M. Tschannen, R. Timofte, and L. V . Gool, “Practical full resolution learned lossless image compression,” inPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 10 629–10 638

  17. [17]

    Learning better lossless compression using lossy compression,

    F. Mentzer, L. V . Gool, and M. Tschannen, “Learning better lossless compression using lossy compression,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 6638– 6647

  18. [18]

    ivpf: Numerical invertible volume preserving flow for efficient lossless compression,

    S. Zhang, C. Zhang, N. Kang, and Z. Li, “ivpf: Numerical invertible volume preserving flow for efficient lossless compression,” inProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 620–629

  19. [19]

    iflow: Numerically invertible flows for efficient lossless compression via a uniform coder,

    S. Zhang, N. Kang, T. Ryder, and Z. Li, “iflow: Numerically invertible flows for efficient lossless compression via a uniform coder,”Advances in Neural Information Processing Systems, vol. 34, pp. 5822–5833, 2021

  20. [20]

    Learned lossless image com- pression based on bit plane slicing,

    Z. Zhang, H. Wang, Z. Chen, and S. Liu, “Learned lossless image com- pression based on bit plane slicing,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 27 579–27 588

  21. [21]

    Compression via Pre-trained Transformers: A Study on Byte-Level Multimodal Data

    D. Heurtel-Depeiges, A. Ruoss, J. Veness, and T. Genewein, “Com- pression via pre-trained transformers: A study on byte-level multimodal data,”arXiv preprint arXiv:2410.05078, 2024

  22. [22]

    Lossless data compression by large models,

    Z. Li, C. Huang, X. Wang, H. Hu, C. Wyeth, D. Bu, Q. Yu, W. Gao, X. Liu, and M. Li, “Lossless data compression by large models,” Nature Machine Intelligence, vol. 7, no. 5, pp. 794–799, 2025. [Online]. Available: https://doi.org/10.1038/s42256-025-01033-7

  23. [23]

    LoRA: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” in International Conference on Learning Representations, 2022. [Online]. Available: https://openreview.net/forum?id=nZeVKeeFYf9

  24. [24]

    Tokenizer Choice For LLM Training: Negligible or Crucial?

    M. Ali, M. Fromm, K. Thellmann, R. Rutmann, M. L ¨ubbering, J. Leveling, K. Klug, J. Ebert, N. Doll, J. S. Buschhoffet al., “Tokenizer choice for LLM training: Negligible or crucial?”arXiv preprint arXiv:2310.08754, 2024. [Online]. Available: https://arxiv.org/ abs/2310.08754

  25. [25]

    Prefix-tuning: Optimizing continuous prompts for generation,

    X. L. Li and P. Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” inProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 2021, pp. 4582–

  26. [26]

    Available: https://aclanthology.org/2021.acl-long.353

    [Online]. Available: https://aclanthology.org/2021.acl-long.353

  27. [27]

    The power of scale for parameter-efficient prompt tuning,

    B. Lester, R. Al-Rfou, and N. Constant, “The power of scale for parameter-efficient prompt tuning,” inProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 3045–3059. [Online]. Available: https://aclanthology.org/ 2021.emnlp-main.243

  28. [28]

    Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains

    M. Tancik, P. P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. T. Barron, and R. Ng, “Fourier features let networks learn high frequency functions in low dimensional domains,” inAdvances in Neural Information Processing Systems (NeurIPS), 2020. [Online]. Available: https: //arxiv.org/abs/2006.10739

  29. [29]

    Bracs: A dataset for breast carcinoma subtyping in h&e histology images,

    N. Brancati, A. M. Anniciello, P. Pati, D. Riccio, G. Scognamiglio, G. Jaume, G. D. Pietro, M. D. Bonito, A. Foncubierta-Rodr ´ıguez, G. Botti, M. Gabrani, F. Feroce, and M. Frucci, “Bracs: A dataset for breast carcinoma subtyping in h&e histology images,”Database, vol. 2022, p. baac093, 2022

  30. [30]

    Bed4rs: Benchmark datasets for remote sensing im- age understanding,

    CAPTAIN-WHU, “Bed4rs: Benchmark datasets for remote sensing im- age understanding,” https://captain-whu.github.io/BED4RS/, 2020, ac- cessed: 2026-07-05

  31. [31]

    The Llama 3 Herd of Models

    Llama Team, AI at Meta, “The Llama 3 herd of models,” arXiv preprint arXiv:2407.21783, 2024. [Online]. Available: https: //arxiv.org/abs/2407.21783

  32. [32]

    Qwen3 Technical Report

    A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lvet al., “Qwen3 technical report,”arXiv preprint arXiv:2505.09388, 2025. [Online]. Available: https://arxiv.org/abs/2505.09388

  33. [33]

    Gemma 3 Technical Report

    Gemma Team, “Gemma 3 technical report,”arXiv preprint arXiv:2503.19786, 2025. [Online]. Available: https://arxiv.org/abs/ 2503.19786

  34. [34]

    Neural network reprogrammability: A unified theme on model reprogramming, prompt tuning, and prompt instruction,

    Z. Ye, C. Cai, R. Dong, J. Qi, L. Feng, P.-Y . Chen, and F. Liu, “Neural network reprogrammability: A unified theme on model reprogramming, prompt tuning, and prompt instruction,” 2025. [Online]. Available: https://arxiv.org/abs/2506.04650