Pith. sign in

REVIEW 2 major objections 4 minor 2 cited by

The paper claims that a two-stage edge-to-server inference scheme, where the server decides whether to request a higher-detail local crop based on the min-entropy of its generated tokens, substantially cuts communication cost for vision-lan

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 15:33 UTC pith:QJWS37DJ

load-bearing objection A competent engineering combination that shows min-entropy gating buys most of the full-retransmission accuracy at 60–90% less retransmission traffic, but the gate's transferability is asserted more than demonstrated. the 2 major comments →

arxiv 2512.16349 v2 pith:QJWS37DJ submitted 2025-12-18 cs.CV cs.AI

Collaborative Edge-to-Server Inference for Vision-Language Models

Soochang Song , Yongjune Kim This is my paper
classification cs.CV cs.AI
keywords edge-to-server inferencevision-language modelsmin-entropyuncertainty-aware retransmissionattention-guided croppingcommunication costvisual question answering
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes a collaborative edge-to-server inference framework for vision-language models (VLMs) where the server first runs inference on a low-resolution global image. If the average min-entropy of the generated token distribution exceeds a threshold, the server uses the model's internal attention to locate a region of interest and asks the edge device to retransmit only that cropped area at higher detail. The central claim is that this selective retransmission matches the accuracy of retransmitting everything while using a fraction of the communication cost, for example reaching accuracy comparable to full retransmission with only 60% of the additional cost on TextVQA. If true, this makes high-accuracy VLM inference more practical in bandwidth-constrained edge-to-server settings.

Core claim

The central claim is that the min-entropy of the output token distribution — not Shannon entropy or probability margin — is a reliable, label-free signal of whether a VLM's first-pass answer on a downscaled image would improve from seeing a locally cropped, higher-detail region. Averaged over the full generated sequence, min-entropy separates correct from incorrect first-pass answers well enough on TextVQA with LLaVA-1.5 (lowest distribution overlap 0.47, highest Bhattacharyya distance 0.33) that thresholding it produces a better accuracy-versus-communication tradeoff than random retransmission or unconditional high-resolution transmission.

What carries the argument

The mechanism is a two-stage protocol: stage 1 performs global-image inference and computes Gavg, the mean over generated tokens of per-token min-entropy −log2 max_y pθ(y|x, s, y<t). If Gavg ≥ η, stage 2 triggers: the server computes a relative attention map A_rel by dividing LLM attention to image tokens under the actual question by attention under a generic prompt, finds the bounding box with the highest internal-external contrast, and requests that crop from the edge device; then the LLM runs again on both global and local visual tokens. The min-entropy threshold acts as the adaptive policy that decides when extra transmission is worthwhile.

Load-bearing premise

The load-bearing assumption is that the min-entropy of first-stage output tokens reliably signals when a local crop would change the answer to the correct one, and that this separation holds strongly enough across datasets and model architectures for thresholding to beat random retransmission.

What would settle it

Compute the distribution of average min-entropy for correct versus incorrect first-pass answers on a new dataset/model pair; if the overlap is high (say above 0.7) or the Bhattacharyya distance drops below about 0.2, thresholding degenerates toward random retransmission, and the reported accuracy-communication advantage should disappear when compared directly against the random-retransmission curve at the same cost.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • On TextVQA, the proposed method achieves accuracy comparable to full retransmission with only 60% of the additional communication cost.
  • On TextVQA+OCR with LLaVA-1.5, the method reaches higher accuracy than the LLaVA-1.5-HD high-resolution baseline while requiring 0.28 additional communication cost versus 0.78.
  • Min-entropy outperforms Shannon entropy and probability margin as a retransmission criterion, both in tradeoff curves and in distribution-separation statistics.
  • Averaging min-entropy over the full output sequence — or even the first five tokens — is a more reliable uncertainty estimate than using only the starting token.
  • The approach remains effective across LLaVA-1.5, InstructBLIP, and Qwen2.5-VL on five VQA benchmarks and composes with JPEG compression for further savings.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the min-entropy signal generalizes, the gating principle could apply to other modalities (audio, video frames) where a 'local zoom' equivalent exists, and to other confidence signals beyond softmax entropy.
  • A label-free procedure for setting the threshold η — for instance, a fixed percentile of the Gavg distribution rather than per-dataset tuning — is an implied extension the paper does not provide.
  • The scheme implies a broader protocol-design principle: transmit only the pixels the model's attention says matter, and only when the model's own probabilities say it is unsure, decoupling communication cost from input resolution.
  • The same uncertainty gate could also decide when to escalate to a larger or more expensive model in a VLM cascade, treating model capacity rather than image detail as the retransmitted resource.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes a two-stage collaborative edge-to-server inference framework for vision-language models (VLMs). The edge device first transmits a downscaled global image and the question; the server performs initial inference and computes the average min-entropy of output tokens. If this entropy exceeds a threshold, the server uses the VLM's internal attention to identify a region of interest, requests a detail-preserved local crop from the edge, and performs a second inference using both global and local visual tokens. The central claim is that this selective retransmission substantially reduces communication and computation cost while maintaining inference accuracy. The paper evaluates the method with LLaVA-1.5-7B, InstructBLIP, and Qwen2.5-VL on TextVQA, POPE, A-OKVQA, GQA, and VQAv2, and compares min-entropy with Shannon entropy and probability margin as uncertainty metrics.

Significance. If the entropy-gating premise holds, the framework is a practical, training-free contribution to edge-to-server VLM inference: it reuses an existing cropping method (ViCrop), is complementary to image compression, and the reported trends are consistent across five datasets and three architectures. The mathematical formulation is sound: the min-entropy definition, the Bhattacharyya-distance comparison, and the FLOPs scaling are all appropriate. The main open risk is the transferability and calibration of the min-entropy threshold, which currently rests on one dataset/model pair; this is addressable with additional experiments. The paper does not provide code, but the method is described precisely enough to be reproduced.

major comments (2)
  1. [§VII-D, Tables II–III, Fig. 7, Algorithm 1] The load-bearing premise is that the average min-entropy of Eq. (13) is a reliable proxy for whether a local crop improves the answer. The only statistical evidence for this is on TextVQA with LLaVA-1.5-7B, where the correct/incorrect overlap is 0.47 and DB=0.33 (Table II)—a moderate separation. The favorable operating points in Figs. 4–6 are obtained by sweeping η in-sample. No separability statistics or η ranges are given for the other four datasets and two architectures in Figs. 9–10, and no label-free validation protocol for η is provided. If the entropy ranking weakens on a new dataset/model, the scheme degrades toward random retransmission, which Fig. 6 shows is far worse. Please add per-dataset/model separation measures, report how η was selected for each curve, and validate with a threshold chosen without test-set labels.
  2. [Figs. 4–10] All accuracy–cost tradeoff curves appear to be based on a single run. Several panels have very small y-axis ranges (e.g., Fig. 9(b): 75.1–76.1; Fig. 9(c): 60.3–61.0), so differences between the proposed method and random retransmission or initial inference can be a few tenths of a percentage point. Without standard errors, bootstrap intervals, or multiple seeds, the reported advantage may not be statistically significant. Please provide error bars or confidence intervals for the main tradeoff curves.
minor comments (4)
  1. [§V-B (end)] The sentence “The numerical results in Section V-B confirm...” is incorrect: Section V-B contains no numerical results. It should refer to Section VII-B or VII-C.
  2. [§VI-B] The attention layer l is said to be “empirically chosen” and the sliding-window candidate schedule is given generically. Please report the chosen l and window schedule for each architecture (LLaVA-1.5, InstructBLIP, Qwen2.5-VL), or state explicitly that the values from [11] are reused without modification; otherwise the RoI component is not fully reproducible.
  3. [§VII-B, Fig. 4] The LLaVA-1.5-HD comparison point appears to be the single reported number from [3] (purple diamond). State explicitly whether this point was re-run under the same evaluation code and OCR pipeline; if taken from [3], note that the comparison is indirect.
  4. [§V-B, Eq. (17)] The FLOPs formula is introduced before the text clarifies that it applies to the prefill stage. Add this clarification at the equation, and define n_v, d_LLM, and L immediately after Eq. (17).

Circularity Check

0 steps flagged

No significant circularity: accuracy/cost tradeoffs are measured threshold sweeps; self-citations are contextual.

full rationale

The paper's central accuracy/communication claims are empirical, not derived. Figs. 4–10 report measured accuracy vs. additional communication cost as the threshold η is swept (Sec. VII-B), and the favorable points are in-sample operating points on those sweeps. The key separator, min-entropy of output tokens, is directly validated against ground-truth correctness on TextVQA (Tables II–III, Fig. 7), not assumed from any equation; Eqs. (13) and (15) define the statistic, and the separation is checked statistically. The only externally imported mechanism, ViCrop relative attention (Eq. (22)), comes from [11] (external authors), so no self-citation chain forces the result. Self-citations [21] and [34] are contextual prior work (uncertainty-aware collaborative inference for image classification; min-entropy estimation) and are not used to prove the VLM retransmission claim. The absence of a label-free η-selection protocol is a transferability limitation, not an identity between the fitted threshold and the reported accuracy; thus no step reduces by construction to its inputs.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The system introduces no new entities and derives nothing from first principles: it measures empirical tradeoffs. The entry price is four practical free parameters (threshold, attention layer, window schedule, aggregation length) plus two behavioral assumptions about VLM internals (entropy-as-confidence, attention-as-RoI-locator), the latter inherited from [11]. The threshold and layer choices are the most consequential because the headline savings are quoted at post-hoc operating points.

free parameters (4)
  • uncertainty threshold η (ηm for min-entropy) = swept over full range; quoted operating points: additional cost 0.28 (TextVQA+OCR), 0.1 (POPE), and 60% of full-retransm
    The retransmission rule Gavg ≥ η (§V-A, Eq. 13; Algorithm 1 line 10) needs a per-task operating point. The paper sweeps η to trace curves and quotes favorable points post hoc (§VII-B); no label-free procedure for setting η is given.
  • attention layer l for relative attention map = not reported
    Section VI-B states a layer is 'empirically chosen to maximize the contrast' between high- and low-attention regions; neither the chosen layer nor its sensitivity is reported, so the RoI selection depends on an unexamined tuning choice.
  • sliding-window candidate sizes and bounding-box schedule = candidate sizes 1.0, 1.2, …, 2.0 × encoder input resolution
    The bounding box search is inherited from ViCrop [11] (§VI-B, Eq. 22) and adopted without re-validation in the new edge-to-server setting.
  • entropy aggregation length = full-sequence average (T ≤ 20); 5-token average already saturates
    Section VII-D (Fig. 8) shows performance saturates at the first 5 tokens; the aggregation choice is selected post hoc from the same data used to evaluate the framework.
axioms (4)
  • domain assumption LLM softmax predictive probabilities reflect answer reliability (low min-entropy ⇔ high confidence ⇔ correct)
    Load-bearing premise of the gating rule (§V-A). Empirically supported only on TextVQA with LLaVA-1.5 (Fig. 7, Tables II–III) and assumed to transfer to other datasets and architectures.
  • domain assumption LLM attention maps localize task-relevant image regions even when the generated answer is wrong
    Borrowed from ViCrop [11] and relied on for all RoI requests (§VI-B, Eq. 22). The paper does not re-validate this on the models or benchmarks it uses.
  • standard math Prefill-stage FLOPs scale approximately linearly with the number of visual tokens, and the prefill dominates total inference cost
    Used for the computation-cost claims in §V-B, Eqs. (17)–(20), sourced from [12], [36], [37]; supports the claim that gating halves or thirds server FLOPs.
  • domain assumption A resized global image plus one resized crop adequately represents the original image for these VQA tasks
    Both global and local images are resized to encoder input resolution (§IV, Eq. 21), so the framework never sees native resolution. The accuracy claims presuppose that content selection, rather than added resolution, is sufficient.

pith-pipeline@v1.3.0-alltime-deepseek · 16710 in / 16899 out tokens · 165411 ms · 2026-08-03T15:33:43.477790+00:00 · methodology

0 comments
read the original abstract

We propose a collaborative edge-to-server inference framework for vision-language models (VLMs) that reduces communication cost while maintaining inference accuracy. In typical deployments, visual data captured at edge devices (clients) is transmitted to the server for VLM inference. However, transmitting full-resolution images incurs high communication cost. Conversely, aggressive downsizing or excessive compression to mitigate communication overhead can discard fine-grained details, leading to accuracy degradation. To overcome this limitation, we design a communication-efficient two-stage framework. In the first stage, the server performs inference on the downsized thumbnail (global image) and quantifies the min-entropy of the output tokens. If the min-entropy exceeds a predefined threshold, the server identifies a region of interest (RoI) using the VLM's internal attention and requests the edge device to send a detail-preserved local image of the RoI. The server then refines its inference by jointly leveraging the global and local images. This selective retransmission strategy ensures that only essential visual content is additionally transmitted. Experimental results consistently confirm that the proposed framework substantially reduces communication overhead while maintaining inference accuracy across diverse VQA benchmarks.

Figures

Figures reproduced from arXiv: 2512.16349 by Soochang Song, Yongjune Kim.

Figure 1
Figure 1. Figure 1: The model takes as input an image–text pair (I, S), where I ∈ R h×w×3 denotes an original RGB image in the pixel domain and S denotes a natural language sentence (e.g., a question referring to the image in a VQA task). Here, h and w represent the height and width of the original image, respectively. The image I is first resized to a fixed resolution ˆI ∈ R hˆ×wˆ×3 to align with the input resolution of the … view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the proposed two-stage collaborative ed [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Visualization of the ViCrop method. The left image sh [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Tradeoff between computation cost and inference acc [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Comparison of uncertainty metrics for LLaVA-1.5-7B [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Analysis of min-entropy aggregation on TextVQA benc [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Comparison of tradeoff curves for different token ag [PITH_FULL_IMAGE:figures/full_fig_p010_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Tradeoff between additional communication cost and [PITH_FULL_IMAGE:figures/full_fig_p010_9.png] view at source ↗
Figure 11
Figure 11. Figure 11: Visualization of global and local images combined w [PITH_FULL_IMAGE:figures/full_fig_p011_11.png] view at source ↗
Figure 10
Figure 10. Figure 10: Tradeoff between additional communication cost an [PITH_FULL_IMAGE:figures/full_fig_p011_10.png] view at source ↗
Figure 12
Figure 12. Figure 12: Additional gain achieved by combining the proposed [PITH_FULL_IMAGE:figures/full_fig_p011_12.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. LAST: The Last Query Token Guides Visual Token Pruning for Edge-Cloud Collaborative MLLM Inference

    cs.CV 2026-07 conditional novelty 5.0

    Attention from the last query token of a compact edge VLM yields query-aware visual-token pruning that retains 95.4% full-token accuracy at 12.5% tokens across 11 benchmarks.

  2. Progressive Semantic Communication for Efficient Edge-Cloud Vision-Language Models

    cs.LG 2026-04 unverdicted novelty 5.0

    A Meta AutoEncoder framework enables adaptive, progressive compression of visual features for low-latency edge-cloud VLM inference without model fine-tuning.

Reference graph

Works this paper leans on

43 extracted references · 4 linked inside Pith · cited by 2 Pith papers

  1. [1]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Aga rwal, G. Sastry, A. Askell, P . Mishkin, J. Clark et al. , “Learning transferable visual models from natural language supervision,” in Proc. Int. Conf. Mach. Learn. (ICML) , Jul. 2021, pp. 8748–8763

    2021

  2. [2]

    Vision-language mo dels for vision tasks: a survey,

    J. Zhang, J. Huang, S. Jin, and S. Lu, “Vision-language mo dels for vision tasks: a survey,” IEEE Trans. Pattern Anal. Mach. Intell. , vol. 46, no. 8, pp. 5625–5644, Feb. 2024

    2024

  3. [3]

    Improved baselines wit h visual in- struction tuning,

    H. Liu, C. Li, Y . Li, and Y . J. Lee, “Improved baselines wit h visual in- struction tuning,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognition (CVPR), Jun. 2024, pp. 26 296–26 306

    2024

  4. [4]

    InstructBLIP: Towards general-purpose vision -language models with instruction tuning,

    W. Dai, J. Li, D. Li, A. M. H. Tiong, J. Zhao, W. Wang, B. Li, P . Fung, and S. Hoi, “InstructBLIP: Towards general-purpose vision -language models with instruction tuning,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), Dec. 2023, pp. 49 250–49 267

    2023

  5. [5]

    Qwen2.5-VL technical report,

    S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P . W ang, S. Wang, J. Tang et al. , “Qwen2.5-VL technical report,” arXiv preprint arXiv:2502.13923, Mar. 2025

    Pith/arXiv arXiv 2025

  6. [6]

    Vision -language models for edge networks: A comprehensive survey,

    A. Sharshar, L. U. Khan, W. Ullah, and M. Guizani, “Vision -language models for edge networks: A comprehensive survey,” IEEE Internet Things J. , vol. 12, Jun. 2025

    2025

  7. [7]

    Task- oriented feature compression for multimodal understandin g via device- edge co-inference,

    C. Y uan, Z. Liu, J. Lv, J. Shao, Y . Jiang, J. Zhang, and X. Li , “Task- oriented feature compression for multimodal understandin g via device- edge co-inference,” arXiv preprint arXiv:2503.12926 , Mar. 2025

    arXiv 2025

  8. [8]

    V aVLM: Toward efficient edge-cloud video analytics with vision- language models,

    Y . Zhang, H. Wang, Q. Bai, H. Liang, P . Zhu, G.-M. Muntean, and Q. Li, “V aVLM: Toward efficient edge-cloud video analytics with vision- language models,” IEEE Trans. Broadcast., vol. 71, no. 2, pp. 529–541, Apr. 2025

    2025

  9. [9]

    An image is worth 16x16 words: Transformers for image recog nition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn , X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gel ly et al. , “An image is worth 16x16 words: Transformers for image recog nition at scale,” in Proc. Int. Conf. Learn. Representations (ICLR) , Jun. 2021

    2021

  10. [10]

    Sigmo id loss for language image pre-training,

    X. Zhai, B. Mustafa, A. Kolesnikov, and L. Beyer, “Sigmo id loss for language image pre-training,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Sep. 2023, pp. 11 975–11 986

    2023

  11. [11]

    MLLMs know where to look: Training-free perception of small visual det ails with multimodal LLMs,

    J. Zhang, M. Khayatkhoei, P . Chhikara, and F. Ilievski, “MLLMs know where to look: Training-free perception of small visual det ails with multimodal LLMs,” in Proc. Int. Conf. Learn. Representations (ICLR) , Apr. 2025

    2025

  12. [12]

    Reducing activation recompu tation in large transformer models,

    V . A. Korthikanti, J. Casper, S. Lym, L. McAfee, M. Ander sch, M. Shoeybi, and B. Catanzaro, “Reducing activation recompu tation in large transformer models,” in MLSys, Jan. 2023

    2023

  13. [13]

    The operational meaning of min- and max-entropy,

    R. Konig, R. Renner, and C. Schaffner, “The operational meaning of min- and max-entropy,” IEEE Trans. Inf. Theory , vol. 55, no. 9, pp. 4337–4347, Sep. 2009

    2009

  14. [14]

    A mathematical theory of communication ,

    C. E. Shannon, “A mathematical theory of communication ,” Bell Syst. Tech. J., vol. 27, no. 3, pp. 379–423, Jul. 1948

    1948

  15. [15]

    Settles, Active Learning

    B. Settles, Active Learning. San Rafael, CA, USA: Morgan & Claypool Publishers, 2012

    2012

  16. [16]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P . Albert, A. Almahairi , Y . Babaei, N. Bashlykov, S. Batra, P . Bhargava, S. Bhosale et al. , “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288, 2023

    Pith/arXiv arXiv 2023

  17. [17]

    Gemma: Open models based on Gemini research and technology,

    T. Mesnard, C. Hardin, R. Dadashi, S. Bhupatiraju, S. Pa thak, L. Sifre, M. Rivi` ere, M. S. Kale, J. Love et al. , “Gemma: Open models based on Gemini research and technology,” arXiv preprint arXiv:2403.08295 , Mar. 2024

    Pith/arXiv arXiv 2024

  18. [18]

    Vicuna: An open-source chatbot impressing GPT- 4 with 90%* chatGPT quality,

    W.-L. Chiang, Z. Li, Z. Lin, Y . Sheng, Z. Wu, H. Zhang, L. Zheng, S. Zhuang, Y . Zhuang, J. E. Gonzalez, I. Stoica, and E. P . Xing, “Vicuna: An open-source chatbot impressing GPT- 4 with 90%* chatGPT quality,” Mar. 2023. [Online]. Availabl e: https://lmsys.org/blog/2023-03-30-vicuna/

    2023

  19. [19]

    Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural t ext process- ing,

    T. Kudo and J. Richardson, “Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural t ext process- ing,” in Proc. Conf. Empir . Methods Nat. Lang. Process. (EMNLP), Nov. 2018, pp. 66–71

    2018

  20. [20]

    Attention is all you need,

    A. V aswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jone s, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS) , Dec. 2017, pp. 5998–6008

    2017

  21. [21]

    Attent ion-aware semantic communications for collaborative inference,

    J. Im, N. Kwon, T. Park, J. Woo, J. Lee, and Y . Kim, “Attent ion-aware semantic communications for collaborative inference,” IEEE Internet Things J. , vol. 11, no. 22, pp. 37 008–37 020, Nov. 2024

    2024

  22. [22]

    Vision transform er-based semantic communications with importance-aware quantizat ion,

    J. Park, Y . Oh, Y . Kim, and Y .-S. Jeon, “Vision transform er-based semantic communications with importance-aware quantizat ion,” IEEE Internet Things J. , vol. 12, pp. 35 662–35 677, Jun. 2025

    2025

  23. [23]

    An image is worth 1/2 tokens after layer 2: Plug-and-play in ference acceleration for large vision-language models,

    L. Chen, H. Zhao, T. Liu, S. Bai, J. Lin, C. Zhou, and B. Cha ng, “An image is worth 1/2 tokens after layer 2: Plug-and-play in ference acceleration for large vision-language models,” in Proc. European Conf. Comput. Vis. (ECCV) , Oct. 2024, pp. 19–35

    2024

  24. [24]

    Beyond text-visual attention: Exploiting vi sual cues for effective token pruning in VLMs,

    Q. Zhang, A. Cheng, M. Lu, R. Zhang, Z. Zhuo, J. Cao, S. Guo , Q. She, and S. Zhang, “Beyond text-visual attention: Exploiting vi sual cues for effective token pruning in VLMs,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Oct. 2025

    2025

  25. [25]

    Beyond transmitting bits: Conte xt, seman- tics, and task-oriented communications,

    D. G¨ und¨ uz, Z. Qin, I. E. Aguerri, H. S. Dhillon, Z. Y ang , A. Y ener, K. K. Wong, and C.-B. Chae, “Beyond transmitting bits: Conte xt, seman- tics, and task-oriented communications,” IEEE J. Sel. Areas Commun. , vol. 41, no. 1, pp. 5–41, Jan. 2023

    2023

  26. [26]

    From semantic communi cation to semantic-aware networking: Model, architecture, and open problems,

    G. Shi, Y . Xiao, Y . Li, and X. Xie, “From semantic communi cation to semantic-aware networking: Model, architecture, and open problems,” IEEE Commun. Mag. , vol. 59, no. 8, pp. 44–50, Aug. 2021

    2021

  27. [27]

    What is semantic communication? A view on conveyi ng meaning in the era of machine intelligence,

    Q. Lan, D. Wen, Z. Zhang, Q. Zeng, X. Chen, P . Popovski, an d K. Huang, “What is semantic communication? A view on conveyi ng meaning in the era of machine intelligence,” J. Commun. Inf. Netw. , vol. 6, no. 4, pp. 336–371, Dec. 2021

    2021

  28. [28]

    Toward semanti c communications: Deep learning-based image semantic codin g,

    D. Huang, F. Gao, X. Tao, Q. Du, and J. Lu, “Toward semanti c communications: Deep learning-based image semantic codin g,” IEEE J. Sel. Areas Commun. , vol. 41, no. 1, pp. 55–71, 2022

    2022

  29. [29]

    A lite distributed semantic communic ation system for internet of things,

    H. Xie and Z. Qin, “A lite distributed semantic communic ation system for internet of things,” IEEE J. Sel. Areas Commun. , vol. 39, no. 1, pp. 142–153, Jan. 2020

    2020

  30. [30]

    A unified multi- task semantic communication system for multimodal data,

    G. Zhang, Q. Hu, Z. Qin, Y . Cai, G. Y u, and X. Tao, “A unified multi- task semantic communication system for multimodal data,” IEEE Trans. Commun., vol. 72, no. 7, pp. 4101–4116, Feb. 2024

    2024

  31. [31]

    A survey on hallucination in large language models: Principles, taxonomy, challenges, and op en questions,

    L. Huang, W. Y u, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qin et al. , “A survey on hallucination in large language models: Principles, taxonomy, challenges, and op en questions,” ACM Trans. Inf. Syst. , vol. 43, no. 2, pp. 1–55, Jan. 2025

    2025

  32. [32]

    Fact-checking the output of large la nguage models via token-level uncertainty quantification,

    E. Fadeeva, A. Rubashevskii, A. Shelmanov, S. Petrakov , H. Li, H. Mubarak, E. Tsymbalov, G. Kuzmin, A. Panchenko, T. Baldwi n, P . Nakov, and M. Panov, “Fact-checking the output of large la nguage models via token-level uncertainty quantification,” in Proc. Annu. Meet- ing Assoc. Comput. Linguistics (ACL) , Aug. 2024, pp. 9367–9385

    2024

  33. [33]

    Language model cascades: Token-level uncert ainty and beyond,

    N. Gupta, H. Narasimhan, A. S. Rawat, W. Jitkrittum, A. M enon, and S. Kumar, “Language model cascades: Token-level uncert ainty and beyond,” in Proc. Int. Conf. Learn. Representations (ICLR) , May 2024. 13

    2024

  34. [34]

    On the efficient estimat ion of min- entropy,

    Y . Kim, C. Guyot, and Y .-S. Kim, “On the efficient estimat ion of min- entropy,” IEEE Trans. Inf. F orensics Security , vol. 16, pp. 3013–3025, Apr. 2021

    2021

  35. [35]

    Towards VQA models that can read,

    A. Singh, V . Natarajan, M. Shah, Y . Jiang, X. Chen, D. Bat ra, D. Parikh, and M. Rohrbach, “Towards VQA models that can read,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognition (CVPR) , Jun. 2019, pp. 8317– 8326

    2019

  36. [36]

    Or ca: A distributed serving system for Transformer-based generat ive models,

    G.-I. Y u, S. J. Joo, G.-W. Kim, S. Kim, and B.-G. Chun, “Or ca: A distributed serving system for Transformer-based generat ive models,” in Proc. USENIX Symp. Operating Syst. Design Implementation ( OSDI), Jul. 2022, pp. 521–538

    2022

  37. [37]

    Taming throughput-latency trad eoff in LLM inference with Sarathi-serve,

    A. Agrawal, N. Kedia, A. Panwar, J. Mohan, N. Kwatra, B. G ulavani, A. Tumanov, and R. Ramjee, “Taming throughput-latency trad eoff in LLM inference with Sarathi-serve,” in Proc. 18th USENIX Conf. Operating Syst. Design Implement. (OSDI) , Jul. 2024, pp. 117–134

    2024

  38. [38]

    Paligemma: A versatile 3B VLM for transfer,

    L. Beyer, A. Steiner, A. S. Pinto, A. Kolesnikov, X. Wang , D. Salz, M. Neumann, I. Alabdulmohsin, M. Tschannen, E. Bugliarello et al. , “Paligemma: A versatile 3B VLM for transfer,” arXiv preprint arXiv:2407.07726, Jul. 2024

    Pith/arXiv arXiv 2024

  39. [39]

    E valuating object hallucination in large vision-language models,

    Y . Li, Y . Du, K. Zhou, J. Wang, W. X. Zhao, and J.-R. Wen, “E valuating object hallucination in large vision-language models,” in Proc. Conf. Empir . Methods Nat. Lang. Process. (EMNLP) , Dec. 2023

    2023

  40. [40]

    A-OKVQA: A benchmark for visual question answering using w orld knowledge,

    D. Schwenk, A. Khandelwal, C. Clark, K. Marino, and R. Mo ttaghi, “A-OKVQA: A benchmark for visual question answering using w orld knowledge,” in Proc. European Conf. Comput. Vis. (ECCV) . Springer, Oct. 2022, pp. 146–162

    2022

  41. [41]

    GQA: A new dataset for rea l-world visual reasoning and compositional question answering,

    D. A. Hudson and C. D. Manning, “GQA: A new dataset for rea l-world visual reasoning and compositional question answering,” i n Proc. IEEE Conf. Comput. Vis. Pattern Recognition (CVPR) , Jun. 2019, pp. 6700– 6709

    2019

  42. [42]

    Making the V in VQA matter: Elevating the role of image understandin g in visual question answering,

    Y . Goyal, T. Khot, D. Summers-Stay, D. Batra, and D. Pari kh, “Making the V in VQA matter: Elevating the role of image understandin g in visual question answering,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognition (CVPR) , Jul. 2017, pp. 6904–6913

    2017

  43. [43]

    On a measure of divergence between t wo multino- mial populations,

    A. Bhattacharyya, “On a measure of divergence between t wo multino- mial populations,” Sankhya, vol. 7, no. 4, pp. 401–406, Jul. 1946

    1946