Token Communication for Multimodal Large Language Model

Jingkai Ying; Khaled B. Letaief; Yuan Shen; Zhijin Qin

REVIEW 4 major objections 6 minor 42 references

Token Communication for Multimodal Large Language Model

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A neural codec inside the vision tokenizer plus a receiver-side adapter delivers MLLM-grade visual tokens at far fewer transmitted bits, beating reconstruction-based coding at the same bitrate.

desk verdict The receiver-adapter idea is promising and the ablation story is mostly clean, but the claimed edge over existing MLLM-oriented coding is not proven because the Bridge baseline was rebuilt with the proposed architecture. read the letter →

arxiv 2608.07279 v1 pith:CK3UFTS3 submitted 2026-08-07 eess.SP

Jingkai Ying , Zhijin Qin , Yuan Shen , Khaled B. Letaief This is my paper · ORCID

classification eess.SP

keywords tokencommunicationmultimodallargelanguagemodelneuralimagecompressionvision-languagesemanticalignmentadapter-basedcodingvisiontokenizerrate-adaptivecodecfeature-wiselinearmodulation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Multimodal large language models (MLLMs) consume enormous numbers of visual tokens, and transmitting images to them is expensive. This paper claims that the right place to compress is inside the vision tokenizer: encode the image into a compact latent, transmit only that bitstream, and at the receiver rebuild both a pixel image and a set of visual tokens that are injected into an intermediate layer of the frozen tokenizer. Under the same number of transmitted bits, this adapter-based token communication scheme outperforms reconstruction-based coding and an existing MLLM-oriented baseline on MME, POPE, SeedBench, and COCO Caption, with the largest gains at low bitrates. If the claim holds, a small, trainable adapter can cut transmitted data substantially without retraining the billion-parameter MLLM.

What carries the argument

The load-bearing mechanism is the injection operation $T_{\mathrm{vis}} = V^{n:N_V}\left(T_{\mathrm{vis}}^{\mathrm{rec},n-1} + \alpha T_{\mathrm{vis}}^{\mathrm{apt}}\right)$, where adapter-produced visual tokens are added into the intermediate features of the frozen vision tokenizer after $n-1$ blocks. The adapter is a one-block Vision Transformer with patchification of the decoded latent, optionally modulated by FiLM with a QP embedding; training combines a distillation objective that matches fused tokens to original-image features and a SigLIP (sigmoid language-image pre-training) pairwise alignment loss between attention-pooled visual semantics and a precomputed text embedding table. This two-stage alignment is what makes compression-oriented latents usable as native MLLM tokens.

What would settle it

Run the same evaluation on text-dense images such as documents or OCR scenes, where the paper argues the reconstruction prior matters most; if A-ImageTC falls below the reconstruction baseline there, the claimed advantage of adapter token injection does not generalize.

Watch

Extended reading notes

Core claim

The central discovery is that a neural codec and a lightweight Transformer-block adapter can together be viewed as an enlarged vision tokenizer split across a channel. The sender outputs compression-oriented latents, and the receiver uses a decoder to produce a reconstruction prior plus an adapter to produce visual tokens, which are added into the tokenizer's intermediate features at the third block with weight $\alpha = 0.1$. A two-stage training scheme first distills the injected tokens toward the original-image intermediate features, then aligns the final visual semantics with text semantics through a SigLIP loss, so the injected tokens remain informative to the frozen MLLM. The result is that for equal bits per pixel, A-ImageTC improves task performance over reconstruction and the Bridge baseline, especially in the low-rate regime, and a QP-adaptive FiLM adapter matches per-QP fixed adapters with one set of weights.

Load-bearing premise

The design assumes that simply adding the adapter's visual tokens to the frozen tokenizer's intermediate features keeps the feature manifold intact, and that semantic alignment learned on ImageNet class labels transfers to the diverse tasks in MME, POPE, SeedBench, and COCO Caption.

Editorial extensions

If this is right

Under equal bits per pixel, MLLM task performance moves closer to the uncompressed-image upper bound, with the largest gains in the low-bitrate regime.
A single QP-adaptive FiLM adapter can serve multiple codec rates, removing the need to train and store one adapter per rate point.
The framework is codec-agnostic: the same adapter works with DCVC-RT, ELIC, and VQ-based codecs, so future codec improvements can be dropped in without retraining the MLLM.
Receiver-side complexity stays nearly constant: the adapter adds about two million parameters and negligible processing time to the full pipeline.
The semantic alignment loss appears essential: ablations show training fails to converge without the SigLIP term, and replacing it with cross-entropy hurts all benchmarks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the alignment-transfer assumption holds beyond ImageNet, the same adapter recipe could be applied to other frozen visual backbones, turning any codec into an MLLM-aware compressor without joint LLM training.
The paper's separated digital transmission pipeline still shows a cliff effect at low SNR; a natural extension is to make the adapter robust to channel-induced bit errors, or to replace the separated pipeline with joint source-channel coding that feeds soft latents into the adapter.
The FiLM conditioning could be extended to condition on task metadata or prompt embeddings, not just the quantization parameter, potentially steering the injected tokens toward task-relevant semantics.
Since the paper observes the counter-intuitive CIDEr result where compression beats uncompressed input on COCO Caption for the 32B model, a testable implication is that adapter-injected tokens act as a regularization that suppresses over-detailed captioning; measuring factual consistency rather than n-gram overlap would clarify whether this is a real gain.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

I read the token communication paper from Ying et al. The core idea is neat: instead of sending pixels or task-specific coded representations, you train the transmitter's vision tokenizer to output compression latents, then at the server you reconstruct both a pixel image and adapter-generated visual tokens that are added into an intermediate transformer block of the frozen MLLM. That is a genuinely different integration of neural codec and MLLM, and the QP-adaptive FiLM module plus the two-stage distillation/SigLIP training is a sensible way to make it work. The experiments show consistent gains over plain reconstruction across MME, POPE, SeedBench, and COCO Caption with two Qwen3-VL sizes and LLaMA-Adapter, which is a decent chunk of evidence. The ablation table, apart from the missing w/o SigLIP entry, supports the roles of the reconstruction prior, the adapter, and the SigLIP loss. The wireless simulation is a plus.

The stress-test note about Bridge is accurate. Section V-A tells you that the Bridge baseline for Qwen3-VL was not the published Bridge. The authors say the original Bridge training did not converge, so they kept its loss but adopted the proposed architecture with the reconstruction prior and attention pooling head. That means the Bridge curves in Figs. 5 and 10 compare against a hybrid that contains the very components the paper argues are beneficial. The margin over "Bridge" could be from the reconstruction prior and pooling head, not from the adapter-injection idea. On LLaMA-Adapter, where Bridge appears original, A-ImageTC is roughly comparable to Bridge (d3), not clearly better. So the claim of superiority over MLLM-oriented coding is inflated. This is a load-bearing caveat, not a minor footnote.

Also, no error bars, no code or checkpoints. For rate-distortion comparisons where curves often cross within a few points, single runs are hard to judge. The authors are honest about the non-convergence, so I don't think it's a deliberate distortion, but it does mean the evidence cannot support the abstract's sweeping sentence.

My take: the framework is worth a serious referee and likely worth publishing after revision, with the baseline clearly labeled or replaced, error bars added, and artifacts released. The reader for this is someone working on source-channel coding for MLLM serving, or edge-cloud inference. I wouldn't cite it in its current form, but I'd want to see a revised version.

Referee Report

4 major / 6 minor

Summary. The paper proposes A-ImageTC, a token communication framework for multimodal large language models. A neural codec is integrated into the vision tokenizer so that only compressed latents are transmitted. At the receiver, a decoder reconstructs a pixel-domain image as a reconstruction prior, while a lightweight adapter converts the same latents into visual tokens that are injected into an intermediate Transformer block of the frozen vision tokenizer by addition. The adapter is trained in two stages: first with a distillation loss that matches the intermediate features of the uncompressed image branch, then with a SigLIP-based semantic alignment loss, with a FiLM-based QP-adaptive variant so that one adapter supports multiple codec rates. Experiments compare the method against reconstruction-based coding and the Bridge baseline on MME, POPE, SeedBench, and COCO Caption, including a Rayleigh fading wireless setting, and ablations study the roles of the adapter, the reconstruction prior, and the SigLIP loss.

Significance. If the reported results are reproducible, the contribution is practically relevant: the method avoids end-to-end LLM training, adds only about two million adapter parameters, works across two neural codecs and two MLLM scales, and shows consistent gains over reconstruction at low bit rates on several external benchmarks. The use of external benchmarks that are not part of the training objective is a real strength, as is the QP-adaptive design that avoids training one adapter per rate point. However, the main empirical claim is currently weakened by the modified Bridge baseline for Qwen3-VL, the absence of error bars, and the lack of released code or checkpoints, so the significance can only be assessed after these points are addressed.

major comments (4)

[V-A and V-B] The Qwen3-VL Bridge baseline is not the published Bridge: Section V-A states that the original training method cannot converge and that the authors therefore keep Bridge's loss function but adopt the proposed architecture with a reconstruction prior and attention pooling head. As a result, Figs. 5 and 10 compare A-ImageTC against a hybrid that already contains components the paper argues are beneficial, so the reported margin over Bridge may be due to those components rather than to adapter-based token injection. The abstract and conclusion claim superiority over existing MLLM-oriented coding schemes, but this is not established for the Qwen3-VL experiments. Please either run the original Bridge with documented convergence fixes, or relabel the baseline (e.g., 'Bridge loss plus proposed receiver') and restrict the claims accordingly.
[V-C and Figs. 5-10] The ablation evidence is incomplete in two ways. First, Table I reports no values for A-ImageTC (w/o SigLIP) and Section V-C states only that the model cannot converge properly; this does not quantify the role of the SigLIP loss and leaves open whether any semantic-alignment loss with a different schedule would work. Second, none of the performance curves in Figs. 5, 7, 8, 9, and 10 carry error bars or multiple-seed results, and some reported gaps are small relative to typical benchmark variance, for example POPE in Fig. 5a at several rate points. Please provide a converged no-SigLIP variant or state the convergence difficulty as a limitation, and add error bars or seed-level results at least for the main comparisons.
[IV-C, Eq. (44), V-B] The semantic alignment in Eq. (44) is trained with ImageNet class labels as positives, yet the evaluation benchmarks (MME, POPE, SeedBench, COCO Caption) are not class-label tasks. The paper should justify empirically that class-level SigLIP alignment transfers to general multimodal understanding and captioning, for example by showing the effect of the text embedding table source or by comparing with an alignment objective built on captions. This is not a circularity issue because the evaluation benchmarks are external, but it is a load-bearing assumption about transfer that currently has no direct evidence.
[V-A] The manuscript does not release code, checkpoints, or evaluation scripts, and the modified Bridge baseline is not fully specified: training hyperparameters, convergence criterion, and which components are frozen are only described textually. Given that the headline comparison relies on a nonstandard baseline modification, this makes the central empirical claim difficult to verify. Please release the artifacts or provide a complete recipe for reproducing every baseline.

minor comments (6)

[Table II] In Table II, 'evalutation' is a typo for 'evaluation'.
[Fig. 9] The Bridge variants d1, d2, and d3 in Fig. 9 are not defined in the text or caption.
[Notations] The notation section introduces complex-valued quantities and Hermitian/conjugate operators that do not appear in the main derivations; consider trimming them to avoid confusion.
[Figs. 5-10] The 'Uncompressed' reference is shown without markers or confidence bounds in several figures; please state the exact score values or add them to the captions.
[V-C] The BD-rate values in Table I are computed relative to Recon (ELIC), while several earlier figures use DCVC-RT as the default codec; please clarify this choice in the text.
[V-A] The initialization QP for the QP-adaptive adapter is described only as a 'middle-rate point'; please give the numerical QP index used.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claim rests on external benchmark evaluations and standard losses, not on its own outputs.

full rationale

The paper's load-bearing claim is empirical: A-ImageTC is evaluated on MME, POPE, SeedBench, and COCO Caption, none of which appear in the adapter training objective (ImageNet class labels with distillation and SigLIP losses, Eq. 35 and Eq. 44). No equation inserts the target benchmark score as an input; the training losses are standard supervised/contrastive objectives, and the codec is trained independently with a rate–distortion loss (Eq. 25). The only fairness caveat is the Qwen3-VL Bridge baseline: Section V-A states 'we keep its loss function, but adopt the proposed architecture with a reconstruction prior and use the attention pooling head to extract visual features,' because the original Bridge 'cannot converge.' This is a baseline-comparison limitation, not a circular derivation: the modified Bridge still uses the cross-entropy loss, and the reported A-ImageTC numbers are not defined in terms of Bridge's outputs. Also, the self-citations ([10], [29]) are used as prior codec/token-communication components, not as the justification for the central performance claim. Hence no circular step rises above 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

This is an empirical machine learning paper. The central claims depend on hand-chosen hyperparameters (alpha, lambda_align, injection block, SigLIP initialization, QP grid) and on transfer assumptions about frozen vision tokenizers and rate-distortion codec latents. No new physical or conceptual entities are introduced.

free parameters (5)

Injection strength alpha = 0.1
Controls the magnitude of adapter tokens added to reconstruction tokens (Eq. 8 and Eq. 33, Section V-A). Chosen by hand; all benchmark results depend on it.
SigLIP scale and bias initialization = tau=10, b=-10
Initial values of the learnable alignment logit scale and bias (Section V-A, following [17]); they shape the alignment loss that adapts the adapter.
Alignment loss weight lambda_align = ramped from 0 to 1e-3
Schedule for the SigLIP term in Eq. 45 (Section V-A); chosen by hand and affects how strongly semantic alignment shapes adapter features.
Injection block index n = 3
Number of vision tokenizer blocks before token injection (Section V-A); chosen by hand, not swept in the paper.
Codec rate-distortion tradeoff lambda_rd = multiple values implied by QP set
Neural codecs are trained with Eq. 25 for several QPs; the QP grid determines the bitrate operating points in Figures 5 through 10.

assumptions (5)

domain assumption The frozen MLLM vision tokenizer remains a valid feature extractor when adapter tokens are added at an intermediate layer.
Eq. 8 assumes addition preserves the tokenizer's learned feature semantics; no fine-tuning of the tokenizer is performed.
domain assumption Class-label SigLIP alignment on ImageNet transfers to MLLM benchmarks such as MME, POPE, SeedBench, and COCO Caption.
Section IV-C trains the adapter on ImageNet class names and templates; Section V-B evaluates on different tasks with no additional task-specific training.
domain assumption Latents produced by rate-distortion trained codecs (DCVC-RT, ELIC) are informative enough for the adapter to recover task-relevant visual tokens.
Section IV-B trains codecs only for reconstruction; the adapter's success depends on this information retention.
standard math Source coding and channel coding are separable and zero-forcing equalization gives valid received symbols.
Section III-B assumes perfect transmission for bpp-only results and uses ZF with perfect CSI for wireless simulations.
standard math Arithmetic encoding and decoding are lossless, so the quantized latents are recovered exactly at the receiver.
Eq. 2 treats the bitstream as error-free in the compression-only setting.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Token Communication for Multimodal Large Language Model." pith.science (2026). https://pith.science/paper/CK3UFTS3

@misc{pith2026260807279,
  author       = {Pith},
  title        = {Pith review of: Token Communication for Multimodal Large Language Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CK3UFTS3}},
  note         = {Machine review of arXiv:2608.07279}
}

read the original abstract

With the broad success of the Transformer architecture, token is becoming a new basic information processing unit. This trend is especially evident in multimodal large language models (MLLMs), where both visual and textual information are represented and processed as tokens. With the rapid deployment of MLLMs, the efficient transmission of tokens has become increasingly important. This paper investigates how to reduce the amount of transmitted data during interactions with MLLMs while preserving their multimodal understanding performance. To address this problem, we propose a token communication framework tailored to MLLMs. In the proposed framework, a neural codec is integrated into the vision tokenizer to control the number of transmitted bits. At the receiver, the decoded latents are processed through two paths. The decoder reconstructs image as a reconstruction prior, while the adapter converts latents into visual tokens and injects them into an intermediate layer of the vision tokenizer. To make the injected tokens suitable for MLLMs, we further design a two-stage visual-language semantic alignment training scheme. The adapter is first warmed up by a distillation loss and then aligned with textual semantics through an alignment loss. An adaptive adapter is also introduced through feature-wise linear modulation, allowing one adapter to support multiple codec rates. Extensive simulations on various MLLM benchmarks show that, under the same amount of transmitted data, the proposed scheme achieves better task performance than other image processing schemes for MLLMs.

Figures

Figures reproduced from arXiv: 2608.07279 by the authors.

**Figure 2.** The proposed token communication framework for MLLMs. After compression at the transmitter, the image is processed [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** The model architecture of proposed QP-adaptive adapter. It mainly consists of the patchification module, the FiLM [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (6 more)

**Figure 4.** Figure 4: The illustration of proposed training scheme for adapter. Before adapter training, a text embedding table is first prepared. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: Performance of different image processing schemes versus BPP on various MLLM benchmarks. (a) The results with [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

**Figure 6.** Figure 6: Qualitative example explaining the COCO Caption [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

**Figure 9.** Figure 9: Performance on the COCO Caption task when LLaMA [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]

**Figure 10.** Figure 10: MME score of different image processing schemes [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]

**Figure 11.** Figure 11: Qualitative comparison of different image processing schemes on COCO Caption, MME, SeedBench, and POPE. Both [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 31 canonical work pages

[1]

GPT-4 technical report,

J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkatet al., “GPT-4 technical report,”arXiv preprint arXiv:2303.08774, Mar. 2023

arXiv 2023
[2]

Deepseek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning,

D. Guo, D. Yang, H. Zhang, J. Song, P. Wang, Q. Zhu, R. Xu, R. Zhang, S. Ma, X. Biet al., “Deepseek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning,”arXiv preprint arXiv:2501.12948, Jan. 2025

arXiv 2025
[3]

Gemini: A family of highly capable multimodal models,

G. Team, R. Anil, S. Borgeaud, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, K. Millicanet al., “Gemini: A family of highly capable multimodal models,”arXiv preprint arXiv:2312.11805, Dec. 2023

arXiv 2023
[4]

Qwen3-VL technical report,

S. Bai, Y . Cai, R. Chen, K. Chen, X. Chen, Z. Cheng, L. Deng, W. Ding, C. Gao, C. Geet al., “Qwen3-VL technical report,”arXiv preprint arXiv:2511.21631, Nov. 2025

arXiv 2025
[5]

Attention is all you need,

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” inProc. Adv. Neural Inf. Process. Syst. (NeurIPS), Long Beach, CA, USA, Dec. 2017, pp. 5998–6008

work page 2017
[6]

State of AI: An empirical 100 trillion token study with openrouter,

M. Aubakirova, A. Atallah, C. Clark, J. Summerville, and A. Midha, “State of AI: An empirical 100 trillion token study with openrouter,” arXiv preprint arXiv:2601.10088, Jan. 2026

arXiv 2026
[7]

Token communications: A large model-driven framework for cross-modal context-aware semantic communications,

L. Qiao, M. B. Mashhadi, Z. Gao, R. Tafazolli, M. Bennis, and D. Niyato, “Token communications: A large model-driven framework for cross-modal context-aware semantic communications,”IEEE Wireless Commun. Mag., vol. 32, no. 5, pp. 80–88, Oct. 2025

work page 2025
[8]

Adaptive semantic token communication for Transformer-based edge inference,

A. Devoto, J. Pomponi, M. Merluzzi, P. Di Lorenzo, and S. Scardapane, “Adaptive semantic token communication for Transformer-based edge inference,”IEEE Trans. Mach. Learn. Commun. Netw., vol. 4, pp. 422– 437, Jan. 2026

work page 2026

Show all 42 references

[9]

ResiTok: A resilient tokenization- enabled framework for ultra-low-rate and robust image transmission,

Z. Liu, Y . Ma, and R. Tafazolli, “ResiTok: A resilient tokenization- enabled framework for ultra-low-rate and robust image transmission,” inProc. IEEE Int. Workshop Signal Process. Artif. Intell. Wireless Commun. (SPA WC), Surrey, UK, Jul. 2025, pp. 1–5

2025
[10]

Joint semantic-channel coding and modulation for token communications,

J. Ying, Z. Qin, Y . Feng, L. Wang, and X. Tao, “Joint semantic-channel coding and modulation for token communications,”IEEE Trans. Wireless Commun., vol. 25, pp. 8179–8193, 2026

2026
[11]

Towards practical real-time neural video compression,

Z. Jia, B. Li, J. Li, W. Xie, L. Qi, H. Li, and Y . Lu, “Towards practical real-time neural video compression,” inProc. IEEE Conf. Comput. Vis. Pattern Recog. (CVPR), Nashville, TN, USA, Jun. 2025, pp. 12 543– 12 552

2025
[12]

ELIC: Efficient learned image compression with unevenly grouped space- channel contextual adaptive coding,

D. He, Z. Yang, W. Peng, R. Ma, H. Qin, and Y . Wang, “ELIC: Efficient learned image compression with unevenly grouped space- channel contextual adaptive coding,” inProc. IEEE Conf. Comput. Vis. Pattern Recog. (CVPR), New Orleans, LA, USA, Jun. 2022, pp. 5718– 5727

2022
[13]

Cache-to-cache: Direct semantic communication between large lan- guage models,

T. Fu, Z. Min, H. Zhang, J. Yan, G. Dai, W. Ouyang, and Y . Wang, “Cache-to-cache: Direct semantic communication between large lan- guage models,” inProc. Int. Conf. Learn. Represent. (ICLR), Rio de Janeiro, Brazil, Apr. 2026

2026
[14]

Transmission with machine language tokens: A paradigm for task-oriented agent communication,

Z. Xiao, C. Ye, Y . Feng, Y . Hu, T. Jiao, L. Cai, and G. Liu, “Transmission with machine language tokens: A paradigm for task-oriented agent communication,”arXiv preprint arXiv:2507.21454, 2025

2025 arXiv
[15]

Video coding for machines: Compact visual representation compression for intelligent collaborative analytics,

W. Yang, H. Huang, Y . Hu, L.-Y . Duan, and J. Liu, “Video coding for machines: Compact visual representation compression for intelligent collaborative analytics,”IEEE Trans. Pattern Anal. Mach. Intell., vol. 46, no. 7, pp. 5174–5191, Jul. 2024

2024
[16]

Learning transferable visual models from natural language supervision,

A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clarket al., “Learning transferable visual models from natural language supervision,” inProc. Int. Conf. Mach. Learn. (ICML), Jul. 2021, pp. 8748–8763

2021
[17]

Sigmoid loss for language image pre-training,

X. Zhai, B. Mustafa, A. Kolesnikov, and L. Beyer, “Sigmoid loss for language image pre-training,” inProc. IEEE Int. Conf. Comput. Vis. (ICCV), Paris, France, Oct. 2023, pp. 11 975–11 986

2023
[18]

SigLIP 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features,

M. Tschannen, A. Gritsenko, X. Wang, M. F. Naeem, I. Alabdul- mohsin, N. Parthasarathy, T. Evans, L. Beyer, Y . Xia, B. Mustafa et al., “SigLIP 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features,”arXiv preprint arXiv...

2025 arXiv
[19]

FiLM: Visual reasoning with a general conditioning layer,

E. Perez, F. Strub, H. De Vries, V . Dumoulin, and A. Courville, “FiLM: Visual reasoning with a general conditioning layer,” inProc. AAAI Conf. Artif. Intell. (AAAI), New Orleans, LA, USA, Feb. 2018, pp. 3942–3951

2018
[20]

Video tokencom: Textual intent-guided multi-rate video token com- munications with UEP-based adaptive source-channel coding,

J. Men, M. B. Mashhadi, N. Wang, Y . Ma, M. Nilsson, and R. Tafazolli, “Video tokencom: Textual intent-guided multi-rate video token com- munications with UEP-based adaptive source-channel coding,”arXiv preprint arXiv:2603.02470, Mar. 2026

2026
[21]

Tokencom-UEP: Semantic importance-matched unequal error protec- tion for resilient image transmission,

K. Zhang, Z. Jin, Y . Zhang, Z. Cheng, M. Zeng, L. Qiao, and Z. Fei, “Tokencom-UEP: Semantic importance-matched unequal error protec- tion for resilient image transmission,”IEEE Wireless Commun. Lett., vol. 15, pp. 2809–2813, 2026

2026
[22]

Semantic Packet Aggregation for Token Communication via genetic beam search,

S. Lee, J. Park, J. Choi, and H. Park, “Semantic Packet Aggregation for Token Communication via genetic beam search,” inProc. IEEE Int. Workshop Signal Process. Artif. Intell. Wireless Commun. (SPA WC), Surrey, UK, Jul. 2025, pp. 1–5

2025
[23]

Vector quantized se- mantic communication system,

Q. Fu, H. Xie, Z. Qin, G. Slabaugh, and X. Tao, “Vector quantized se- mantic communication system,”IEEE Wireless Commun. Lett., vol. 12, no. 6, pp. 982–986, Jun. 2023

2023
[24]

TokenCom: Vision-Language Model for Multimodal and Multitask Token Communications,

F. Jiang, S. Tu, L. Dong, X. Li, K. Wang, C. Pan, Z. Han, and J. Wang, “TokenCom: Vision-Language Model for Multimodal and Multitask Token Communications,”arXiv preprint arXiv:2603.00482, Feb. 2026

2026
[25]

VILA-U: A unified foundation model integrating visual understanding and generation,

Y . Wu, Z. Zhang, J. Chen, H. Tang, D. Li, Y . Fang, L. Zhu, E. Xie, H. Yin, L. Yi, S. Han, and Y . Lu, “VILA-U: A unified foundation model integrating visual understanding and generation,” inProc. Int. Conf. Learn. Represent. (ICLR), Singapore, Apr. 2025

2025
[26]

BPG Image Format,

F. Bellard, “BPG Image Format,” [Online]. Available: http://bellard.org/ bpg/, accessed: May 7, 2026

2026
[27]

VVC Test Model,

VTM, “VVC Test Model,” [Online]. Available: https://vcgit.hhi. fraunhofer.de/jvet/VVCSoftware VTM, accessed: May 7, 2026

2026
[28]

Generative latent coding for ultra-low bitrate image compression,

Z. Jia, J. Li, B. Li, H. Li, and Y . Lu, “Generative latent coding for ultra-low bitrate image compression,” inProc. IEEE Conf. Comput. Vis. Pattern Recog. (CVPR), Seattle, W A, USA, Jun. 2024, pp. 26 088– 26 098

2024
[29]

ProGIC: Progressive and lightweight generative image compression with residual vector quantization,

H. Cao, C. Liang, W. Guo, Z. Qin, and J. Han, “ProGIC: Progressive and lightweight generative image compression with residual vector quantization,”arXiv preprint arXiv:2603.02897, Mar. 2026

2026 arXiv
[30]

TransTIC: Transferring Transformer-based image compression from human perception to machine perception,

Y .-H. Chen, Y .-C. Weng, C.-H. Kao, C. Chien, W.-C. Chiu, and W.-H. Peng, “TransTIC: Transferring Transformer-based image compression from human perception to machine perception,” inProc. IEEE Int. Conf. Comput. Vis. (ICCV), Paris, France, Oct. 2023, pp. 23 297–23 307

2023
[31]

High efficiency image compression for large visual-language models,

B. Li, S. Wang, S. Wang, and Y . Ye, “High efficiency image compression for large visual-language models,”IEEE Trans. Circuits Syst. Video Technol., vol. 35, no. 3, pp. 2870–2880, Mar. 2025

2025
[32]

Bridging compressed image latents and multimodal large language models,

C.-H. Kao, C. Chien, Y .-J. Tseng, Y .-H. Chen, A. Gnutti, S.-Y . Lo, W.-H. Peng, and R. Leonardi, “Bridging compressed image latents and multimodal large language models,” inProc. Int. Conf. Learn. Represent. (ICLR), Singapore, Apr. 2025

2025
[33]

When MLLMs meet compression distortion: A coding paradigm tailored to MLLMs,

J. Liu, Z. Jia, J. Li, B. Li, X. Jin, W. Zeng, and Y . Lu, “When MLLMs meet compression distortion: A coding paradigm tailored to MLLMs,” inProc. Int. Conf. Learn. Represent. (ICLR), Rio de Janeiro, Brazil, Apr. 2026

2026
[34]

Variational image compression with a Scale Hyperprior,

J. Ball ´e, D. Minnen, S. Singh, S. J. Hwang, and N. Johnston, “Variational image compression with a Scale Hyperprior,” inProc. Int. Conf. Learn. Represent. (ICLR), Vancouver, BC, Canada, Apr. 2018

2018
[35]

Deep joint source- channel coding for wireless image transmission,

E. Bourtsoulatze, D. Burth Kurka, and D. G ¨und¨uz, “Deep joint source- channel coding for wireless image transmission,”IEEE Trans. Cogn. Commun. Netw., vol. 5, no. 3, pp. 567–579, Sep. 2019

2019
[36]

Qwen-Image technical report,

C. Wu, J. Li, J. Zhou, J. Lin, K. Gao, K. Yan, S.-m. Yin, S. Bai, X. Xu, Y . Chenet al., “Qwen-Image technical report,”arXiv preprint arXiv:2508.02324, Aug. 2025

2025 arXiv
[37]

RoFormer: En- hanced transformer with rotary position embedding,

J. Su, A. Murtadha, Y . Lu, S. Pan, W. Bo, and Y . Liu, “RoFormer: En- hanced transformer with rotary position embedding,”Neurocomputing, vol. 568, p. 127063, Feb. 2024

2024
[38]

LLaMA-Adapter: Efficient fine-tuning of large language models with zero-initialized attention,

R. Zhang, J. Han, C. Liu, A. Zhou, P. Lu, Y . Qiao, H. Li, and P. Gao, “LLaMA-Adapter: Efficient fine-tuning of large language models with zero-initialized attention,” inProc. Int. Conf. Learn. Represent. (ICLR), Vienna, Austria, May 2024

2024
[39]

MME: A comprehen- sive evaluation benchmark for multimodal large language models,

C. Fu, P. Chen, Y . Shen, Y . Qin, M. Zhang, X. Lin, J. Yang, X. Zheng, K. Li, X. Sun, Y . Wu, R. Ji, C. Shan, and R. He, “MME: A comprehen- sive evaluation benchmark for multimodal large language models,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), San Diego, CA, USA,...

2025
[40]

Evaluating object hallucination in large vision-language models,

Y . Li, Y . Du, K. Zhou, J. Wang, X. Zhao, and J.-R. Wen, “Evaluating object hallucination in large vision-language models,” inProc. Conf. Empirical Methods Natural Lang. Process. (EMNLP), Singapore, Dec. 2023, pp. 292–305

2023
[41]

SEED-Bench: Benchmarking multimodal large language models,

B. Li, Y . Ge, Y . Ge, G. Wang, R. Wang, R. Zhang, and Y . Shan, “SEED-Bench: Benchmarking multimodal large language models,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog. (CVPR), Seattle, W A, USA, Jun. 2024, pp. 13 299–13 308

2024
[42]

Deep visual-semantic alignments for gen- erating image descriptions,

A. Karpathy and F.-F. Li, “Deep visual-semantic alignments for gen- erating image descriptions,” inProc. IEEE Conf. Comput. Vis. Pattern Recog. (CVPR), Boston, MA, USA, Jun. 2015, pp. 3128–3137

2015

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

GPT-4 technical report,

J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkatet al., “GPT-4 technical report,”arXiv preprint arXiv:2303.08774, Mar. 2023

arXiv 2023

[2] [2]

Deepseek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning,

D. Guo, D. Yang, H. Zhang, J. Song, P. Wang, Q. Zhu, R. Xu, R. Zhang, S. Ma, X. Biet al., “Deepseek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning,”arXiv preprint arXiv:2501.12948, Jan. 2025

arXiv 2025

[3] [3]

Gemini: A family of highly capable multimodal models,

G. Team, R. Anil, S. Borgeaud, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, K. Millicanet al., “Gemini: A family of highly capable multimodal models,”arXiv preprint arXiv:2312.11805, Dec. 2023

arXiv 2023

[4] [4]

Qwen3-VL technical report,

S. Bai, Y . Cai, R. Chen, K. Chen, X. Chen, Z. Cheng, L. Deng, W. Ding, C. Gao, C. Geet al., “Qwen3-VL technical report,”arXiv preprint arXiv:2511.21631, Nov. 2025

arXiv 2025

[5] [5]

Attention is all you need,

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” inProc. Adv. Neural Inf. Process. Syst. (NeurIPS), Long Beach, CA, USA, Dec. 2017, pp. 5998–6008

work page 2017

[6] [6]

State of AI: An empirical 100 trillion token study with openrouter,

M. Aubakirova, A. Atallah, C. Clark, J. Summerville, and A. Midha, “State of AI: An empirical 100 trillion token study with openrouter,” arXiv preprint arXiv:2601.10088, Jan. 2026

arXiv 2026

[7] [7]

Token communications: A large model-driven framework for cross-modal context-aware semantic communications,

L. Qiao, M. B. Mashhadi, Z. Gao, R. Tafazolli, M. Bennis, and D. Niyato, “Token communications: A large model-driven framework for cross-modal context-aware semantic communications,”IEEE Wireless Commun. Mag., vol. 32, no. 5, pp. 80–88, Oct. 2025

work page 2025

[8] [8]

Adaptive semantic token communication for Transformer-based edge inference,

A. Devoto, J. Pomponi, M. Merluzzi, P. Di Lorenzo, and S. Scardapane, “Adaptive semantic token communication for Transformer-based edge inference,”IEEE Trans. Mach. Learn. Commun. Netw., vol. 4, pp. 422– 437, Jan. 2026

work page 2026