REVIEW 3 major objections 6 minor 34 references
Lossless image compression can be done by adapting frozen language models in pixel space, without using their text tokenizers.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-10 11:04 UTC pith:U6UNPY72
load-bearing objection Clean tokenizer-free interface that makes frozen multi-family LLMs usable as pixel entropy models; rates are competitive mainly against JPEG-XL/DLPR and P2-LLM under independent 16x16 patches. the 3 major comments →
LUMI: Tokenizer-Agnostic LLM-Based Lossless Image Compression
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
LLM-based lossless RGB compression is best formulated as pixel-space adaptation of frozen foundation models: map intensity, channel, and intra-patch position into continuous embeddings, keep the backbone fixed, and decode with a dedicated 256-way head, rather than representing pixels as tokenizer-dependent text.
What carries the argument
The tokenizer-free pixel interface (PixEmb + intra-patch position encoding + 256-way head): a 7-D intensity/channel descriptor projected into LLM embedding space, plus row-column position codes, feeding a frozen decoder that emits exact pixel probabilities for arithmetic coding.
Load-bearing premise
That modeling each 16-by-16 patch independently with a frozen language model already captures enough local statistics for competitive compression rates.
What would settle it
Train and evaluate the same external modules on identical data and splits, but with full inter-patch context or larger overlapping tiles; if rates do not improve over the independent-patch LUMI baseline, the sufficiency claim holds; if they drop sharply below classical codecs, the assumption fails.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. LUMI proposes a tokenizer-agnostic interface for lossless RGB image compression that attaches lightweight external modules (PixEmb intensity–channel embedding, gated row–column intra-patch position encoding, soft prefix, and a 256-way pixel head) to frozen decoder-only LLMs (LLaMA, Qwen, Gemma). Pixel values are mapped into continuous embedding space rather than textual numeric tokens, so the same pipeline avoids family-specific tokenizer fragmentation and predicts a native 256-symbol alphabet for arithmetic coding. Only the external adapters are trained. Experiments on Kodak, BRACS, and BED4RS report in-domain BPP competitive with JPEG-XL and DLPR and with a LoRA-adapted P2-LLM baseline, leave-one-domain-out generalization, ablations of PixEmb/INP/prompting, and model/data scaling on Qwen3.
Significance. The main contribution is a clean formulation of LLM-based lossless image coding as pixel-space adaptation of frozen foundation models rather than tokenizer-bound language-symbol modeling. Demonstrating a single external interface that works across three tokenizer families, with only small trainable modules and held-out BPP measured under arithmetic coding, is a useful engineering and conceptual result for the compression and foundation-model communities. Tokenizer-fragmentation evidence (Table II), embedding visualizations, and consistent multi-backbone behavior strengthen the portability claim. The work is appropriately scoped as an interface study; it does not claim to redefine the state of specialized neural lossless codecs.
major comments (3)
- Tables III–IV and §IV-C frame LUMI as achieving “competitive” rates, but the non-LLM comparison set is limited to JPEG-XL and DLPR. Section II-A cites stronger modern learned lossless methods (hierarchical residual, invertible flows, bit-plane models) that are not re-run under the same patch protocol. Without those anchors, the headline performance claim is only weakly supported relative to the broader learned-codec literature, even though the gain over tokenizer-based P2-LLM baselines is clear.
- Section III-G and the Limitations section state that non-overlapping 16×16 patches are compressed independently (T=768, no inter-patch context). This design choice is load-bearing for the claim that frozen LLMs act as effective image entropy models: classical and specialized neural codecs exploit longer-range structure that LUMI never models. The paper should either quantify the cost of this restriction (e.g., vs. larger patches or a simple inter-patch conditioner) or more carefully qualify “competitive” as holding under a deliberately restricted context regime.
- Table I reports only 4 training images for BRACS (9,767 patches). In-domain and leave-one-domain-out BRACS numbers in Tables III–IV and the scaling tables therefore rest on a very small image-level sample. Without image-level resampling, error bars, or multi-split results, the medical-domain and cross-domain robustness claims are statistically under-supported relative to Kodak and BED4RS.
minor comments (6)
- No standard errors or run-to-run variance are reported for any BPP table; even a few seeds on the adapter training would help assess stability of the small gains over JPEG-XL/DLPR.
- Figure 2’s embedding visualization is qualitative; a simple neighborhood or channel-separation metric would make the PixEmb vs. tokenizer comparison more rigorous.
- Equation (11) introduces a 7-D descriptor with polynomial and Fourier features; a short justification or ablation of the sinusoidal terms (beyond 4-D vs. 7-D channel identity) would clarify design choices.
- Decoding latency is noted as a limitation but never quantified; a brief wall-clock comparison against JPEG-XL/DLPR on the same hardware would contextualize practicality.
- Minor notation: β in Eq. (14) is called a “learnable scalar gate” but its learned range or initialization is not reported; likewise soft-prefix length P=16 is fixed without sensitivity analysis.
- Typographical consistency: “P 2-LLM” / “P2-LLM” spacing and “F . Arithmetic Coding” section heading spacing should be cleaned for camera-ready.
Circularity Check
No circularity: standard supervised NLL training of external adapters evaluated by held-out arithmetic-coding BPP against external codecs.
full rationale
LUMI is an empirical systems paper, not a first-principles derivation. The probabilistic objective (Eqs. 3–4, 21–22) is ordinary autoregressive cross-entropy over the 256-symbol pixel alphabet; BPP is the same quantity measured on held-out patches under arithmetic coding, not a fitted constant renamed as a prediction. Trainable modules (PixEmb, INP, soft prefix, 256-way head) are optimized on training splits and reported on image-disjoint test splits and leave-one-domain-out settings (Tables III–IV, VIII–X). Classical codecs (JPEG-XL, DLPR) and the P2-LLM baseline supply external anchors; P2-LLM is used as a comparator, not as a uniqueness theorem or load-bearing premise that forces LUMI’s rates. Fourier features, prefix tuning, and LoRA are cited from independent prior work as standard tools, not smuggled ansätze that define the result. Independent non-overlapping 16×16 patches and frozen-backbone limits are stated as design choices/limitations, not hidden circular reductions. No equation reduces by construction to its own fitted input; score 0 with empty steps is the correct finding.
Axiom & Free-Parameter Ledger
free parameters (4)
- patch size Hp×Wp =
16×16
- soft-prefix length P =
16
- PixEmb / head / INP / prefix learning rates =
5e-4 / 5e-4 / 1e-4 / 3e-4
- INP scalar gate β and row/column tables
axioms (3)
- standard math Autoregressive negative log-likelihood under arithmetic coding equals ideal codelength (standard Shannon–arithmetic coding identity).
- domain assumption A frozen decoder-only LLM’s hidden states remain useful contextual features for non-text symbols once continuous embeddings are supplied.
- ad hoc to paper Independent non-overlapping patches are an acceptable approximation for the evaluated benchmarks.
invented entities (2)
-
PixEmb 7-D intensity-channel descriptor + MLP
no independent evidence
-
Intra-patch position encoding (INP) with gated row+column tables
no independent evidence
read the original abstract
Large language model (LLM)-based lossless image compression methods typically represent pixel data through the native text interface of a pretrained model, converting pixel values into token sequences that the LLM processes through its vocabulary head. This design shows that pretrained language models can provide probability estimates for image coding, but it also couples compression to tokenizer behavior, vocabulary-specific numeric tokens, and model-family-specific adaptation. In this paper, we present LUMI (LLM-based Unified Model-agnostic lossless Image compression), a tokenizer-agnostic framework for lossless RGB image compression with frozen LLM backbones. LUMI replaces pixel-as-text tokenization with a pixel embedding module that maps raw intensity and channel information into the continuous embedding space of the LLM. It further introduces intra-patch position encoding to retain two-dimensional spatial structure after flattening, and uses a 256-way prediction head to produce probabilities over the native pixel alphabet. Only the pixel embedding, position encoding, soft-prefix parameters, and prediction head are trained, while the LLM backbone remains fixed. Experiments on natural, medical, and remote-sensing image benchmarks with LLaMA, Qwen, and Gemma backbones show that LUMI provides a unified interface across tokenizer families, achieves competitive compression rates, and improves cross-domain robustness over tokenizer-based LLM compression baselines. These results formulate LLM-based lossless image compression as pixel-space adaptation of frozen foundation models rather than tokenizer-specific language-symbol modeling.
Figures
Reference graph
Works this paper leans on
-
[1]
Evaluating Large Language Models Trained on Code
M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y . Burda, N. Joseph, G. Brockmanet al., “Evaluating large language models trained on code,”arXiv preprint arXiv:2107.03374, 2021
work page internal anchor Pith review Pith/arXiv arXiv 2021
-
[2]
Chain-of-thought prompting elicits reasoning in large language models,
J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V . Le, and D. Zhou, “Chain-of-thought prompting elicits reasoning in large language models,” inAdvances in Neural Information Processing Systems, vol. 35, 2022, pp. 24 824–24 837
work page 2022
-
[3]
Retrieval- augmented generation for knowledge-intensive nlp tasks,
P. Lewis, E. Perez, A. Piktus, F. Petroni, V . Karpukhin, N. Goyal, H. K ¨uttler, M. Lewis, W.-t. Yih, T. Rockt ¨aschelet al., “Retrieval- augmented generation for knowledge-intensive nlp tasks,” inAdvances in Neural Information Processing Systems, vol. 33, 2020, pp. 9459–9474
work page 2020
-
[4]
Flamingo: A visual language model for few-shot learning,
J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y . Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynoldset al., “Flamingo: A visual language model for few-shot learning,” inAdvances in Neural Information Processing Systems, vol. 35, 2022, pp. 23 716–23 736
work page 2022
-
[5]
Language modeling is compression,
G. Del ´etang, A. Ruoss, P.-A. Duquenne, E. Catt, T. Genewein, C. Mattern, J. Grau-Moya, L. K. Wenliang, M. Aitchison, L. Orseau, M. Hutter, and J. Veness, “Language modeling is compression,” inThe Twelfth International Conference on Learning Representations, 2024. [Online]. Available: https://openreview.net/forum?id=jznbgiynus
work page 2024
-
[6]
T. M. Cover and J. A. Thomas,Elements of Information Theory, 2nd ed. Wiley-Interscience, 2006
work page 2006
-
[7]
Arithmetic coding for data compression,
I. H. Witten, R. M. Neal, and J. G. Cleary, “Arithmetic coding for data compression,”Communications of the ACM, vol. 30, no. 6, pp. 520–540, 1987
work page 1987
-
[8]
Large Language Model for Lossless Image Compression with Visual Prompts
J. Du, C. Zhou, N. Cao, G. Chen, Y . Chen, Z. Cheng, L. Song, G. Lu, and W. Zhang, “Large language model for lossless image compression with visual prompts,”arXiv preprint arXiv:2502.16163, 2025. [Online]. Available: https://arxiv.org/abs/2502.16163
work page internal anchor Pith review Pith/arXiv arXiv 2025
-
[9]
Deep lossy plus residual coding for lossless and near-lossless image compression,
Y . Bai, X. Liu, K. Wang, X. Ji, X. Wu, and W. Gao, “Deep lossy plus residual coding for lossless and near-lossless image compression,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 5, pp. 3577–3594, 2024
work page 2024
-
[10]
K. Chen, P. Zhang, H. Liu, J. Liu, Y . Liu, J. Huang, S. Wang, H. Yan, and H. Li, “Large language models for lossless image compression: Next- pixel prediction in language space is all you need,” inThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. [Online]. Available: https://openreview.net/forum?id=FXBBy1caOX
work page 2025
-
[11]
Png (portable network graphics) specification version 1.0,
T. Boutell, “Png (portable network graphics) specification version 1.0,” World Wide Web Consortium, Tech. Rep., 1997
work page 1997
-
[12]
The loco-i lossless image compression algorithm: Principles and standardization into jpeg- ls,
M. J. Weinberger, G. Seroussi, and G. Sapiro, “The loco-i lossless image compression algorithm: Principles and standardization into jpeg- ls,”IEEE Transactions on Image processing, vol. 9, no. 8, pp. 1309– 1324, 2000
work page 2000
-
[13]
Jpeg xl next-generation image compression architecture and coding tools,
J. Alakuijala, R. V . Asseldonk, S. Boukortt, M. Bruse, I.-M. Coms ¸a, M. Firsching, T. Fischbacher, E. Kliuchnikov, S. Gomez, R. Obryket al., “Jpeg xl next-generation image compression architecture and coding tools,” inApplications of digital image processing XLII. SPIE, 2019, pp. 112–124
work page 2019
-
[14]
Pixel recurrent neural networks,
A. V . D. Oord, N. Kalchbrenner, and K. Kavukcuoglu, “Pixel recurrent neural networks,” inInternational conference on machine learning. PMLR, 2016, pp. 1747–1756
work page 2016
-
[15]
T. Salimans, A. Karpathy, X. Chen, and D. P. Kingma, “Pixelcnn++: Improving the pixelcnn with discretized logistic mixture likelihood and other modifications,”arXiv preprint arXiv:1701.05517, 2017
work page internal anchor Pith review Pith/arXiv arXiv 2017
-
[16]
Practical full resolution learned lossless image compression,
F. Mentzer, E. Agustsson, M. Tschannen, R. Timofte, and L. V . Gool, “Practical full resolution learned lossless image compression,” inPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 10 629–10 638
work page 2019
-
[17]
Learning better lossless compression using lossy compression,
F. Mentzer, L. V . Gool, and M. Tschannen, “Learning better lossless compression using lossy compression,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 6638– 6647
work page 2020
-
[18]
ivpf: Numerical invertible volume preserving flow for efficient lossless compression,
S. Zhang, C. Zhang, N. Kang, and Z. Li, “ivpf: Numerical invertible volume preserving flow for efficient lossless compression,” inProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 620–629
work page 2021
-
[19]
iflow: Numerically invertible flows for efficient lossless compression via a uniform coder,
S. Zhang, N. Kang, T. Ryder, and Z. Li, “iflow: Numerically invertible flows for efficient lossless compression via a uniform coder,”Advances in Neural Information Processing Systems, vol. 34, pp. 5822–5833, 2021
work page 2021
-
[20]
Learned lossless image com- pression based on bit plane slicing,
Z. Zhang, H. Wang, Z. Chen, and S. Liu, “Learned lossless image com- pression based on bit plane slicing,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 27 579–27 588
work page 2024
-
[21]
Compression via Pre-trained Transformers: A Study on Byte-Level Multimodal Data
D. Heurtel-Depeiges, A. Ruoss, J. Veness, and T. Genewein, “Com- pression via pre-trained transformers: A study on byte-level multimodal data,”arXiv preprint arXiv:2410.05078, 2024
work page internal anchor Pith review Pith/arXiv arXiv 2024
-
[22]
Lossless data compression by large models,
Z. Li, C. Huang, X. Wang, H. Hu, C. Wyeth, D. Bu, Q. Yu, W. Gao, X. Liu, and M. Li, “Lossless data compression by large models,” Nature Machine Intelligence, vol. 7, no. 5, pp. 794–799, 2025. [Online]. Available: https://doi.org/10.1038/s42256-025-01033-7
-
[23]
LoRA: Low-rank adaptation of large language models,
E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” in International Conference on Learning Representations, 2022. [Online]. Available: https://openreview.net/forum?id=nZeVKeeFYf9
work page 2022
-
[24]
Tokenizer Choice For LLM Training: Negligible or Crucial?
M. Ali, M. Fromm, K. Thellmann, R. Rutmann, M. L ¨ubbering, J. Leveling, K. Klug, J. Ebert, N. Doll, J. S. Buschhoffet al., “Tokenizer choice for LLM training: Negligible or crucial?”arXiv preprint arXiv:2310.08754, 2024. [Online]. Available: https://arxiv.org/ abs/2310.08754
work page internal anchor Pith review Pith/arXiv arXiv 2024
-
[25]
Prefix-tuning: Optimizing continuous prompts for generation,
X. L. Li and P. Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” inProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 2021, pp. 4582–
work page 2021
-
[26]
Available: https://aclanthology.org/2021.acl-long.353
[Online]. Available: https://aclanthology.org/2021.acl-long.353
work page 2021
-
[27]
The power of scale for parameter-efficient prompt tuning,
B. Lester, R. Al-Rfou, and N. Constant, “The power of scale for parameter-efficient prompt tuning,” inProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 3045–3059. [Online]. Available: https://aclanthology.org/ 2021.emnlp-main.243
work page 2021
-
[28]
Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
M. Tancik, P. P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. T. Barron, and R. Ng, “Fourier features let networks learn high frequency functions in low dimensional domains,” inAdvances in Neural Information Processing Systems (NeurIPS), 2020. [Online]. Available: https: //arxiv.org/abs/2006.10739
work page internal anchor Pith review Pith/arXiv arXiv 2020
-
[29]
Bracs: A dataset for breast carcinoma subtyping in h&e histology images,
N. Brancati, A. M. Anniciello, P. Pati, D. Riccio, G. Scognamiglio, G. Jaume, G. D. Pietro, M. D. Bonito, A. Foncubierta-Rodr ´ıguez, G. Botti, M. Gabrani, F. Feroce, and M. Frucci, “Bracs: A dataset for breast carcinoma subtyping in h&e histology images,”Database, vol. 2022, p. baac093, 2022
work page 2022
-
[30]
Bed4rs: Benchmark datasets for remote sensing im- age understanding,
CAPTAIN-WHU, “Bed4rs: Benchmark datasets for remote sensing im- age understanding,” https://captain-whu.github.io/BED4RS/, 2020, ac- cessed: 2026-07-05
work page 2020
-
[31]
Llama Team, AI at Meta, “The Llama 3 herd of models,” arXiv preprint arXiv:2407.21783, 2024. [Online]. Available: https: //arxiv.org/abs/2407.21783
work page internal anchor Pith review Pith/arXiv arXiv 2024
-
[32]
A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lvet al., “Qwen3 technical report,”arXiv preprint arXiv:2505.09388, 2025. [Online]. Available: https://arxiv.org/abs/2505.09388
work page internal anchor Pith review Pith/arXiv arXiv 2025
-
[33]
Gemma Team, “Gemma 3 technical report,”arXiv preprint arXiv:2503.19786, 2025. [Online]. Available: https://arxiv.org/abs/ 2503.19786
work page internal anchor Pith review Pith/arXiv arXiv 2025
-
[34]
Z. Ye, C. Cai, R. Dong, J. Qi, L. Feng, P.-Y . Chen, and F. Liu, “Neural network reprogrammability: A unified theme on model reprogramming, prompt tuning, and prompt instruction,” 2025. [Online]. Available: https://arxiv.org/abs/2506.04650
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.