Pith. sign in

REVIEW 3 major objections 32 references

TIGER speeds multimodal speculative decoding by giving the draft model only the visual tokens its current text needs and training it to maximize how many draft tokens the full verifier accepts.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Text-conditioned sparse visual routing plus verifier-accepted-prefix rewards lengthens accepted draft prefixes and improves exact multimodal speculative decoding speedup over strong baselines.

T0 review reviewed 2026-07-14 challenge →

load-bearing objection Solid systems paper: drafter-only text-conditioned Top-k routing plus accepted-prefix GRPO is a real, well-motivated combo; Table 1 gains are often tiny and unaudited, so the “consistent” claim is the soft spot, not the idea. the 3 major comments →

arxiv 2607.11131 v1 pith:QRIFZN6E submitted 2026-07-13 cs.CL

TIGER: Text-Conditioned Visual Gated Routing with Acceptance Alignment for Multimodal Speculative Decoding

Quynh Vo , Cong-Duy Nguyen , Ponhvoan Srey , Luu Anh Tuan , Thong Nguyen This is my paper
classification cs.CL
keywords speculative decodingvision-language modelsvisual token routingaccepted prefix lengthdraft-verifier alignmentmultimodal accelerationgroup policy optimization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Vision-language models generate answers token by token, which is slow. Speculative decoding tries to fix that by letting a small draft model propose several tokens at once for a large model to accept or reject, but in multimodal settings the draft model often fails early on vision-critical content such as OCR strings, counts, or grounded entities. TIGER argues that two mismatches cause this: the draft model is shown either all visual tokens or a fixed compressed view, including irrelevant evidence, and it is usually trained only to imitate the large model rather than to maximize the accepted prefix length that actually determines speedup. The method therefore routes a sparse, text-conditioned subset of visual tokens to the draft model each speculative block, then trains that draft model with group policy updates whose reward is the verifier-accepted prefix length after a distillation warm start. The paper reports longer accepted prefixes and higher end-to-end speculative speedup under exact full-vision verification, with comparable task accuracy and favorable quality-latency trade-offs in routing analyses.

Core claim

Under exact verifier-side speculative decoding with an unchanged full-vision verifier, text-conditioned sparse visual routing plus acceptance-aligned training of the draft model consistently lengthens verifier-accepted prefixes and raises speculative speedup over strong multimodal speculative baselines, while keeping downstream accuracy comparable.

What carries the argument

TIGER: text-conditioned visual gated routing (top-k visual tokens scored from the draft model’s current text hidden state) paired with acceptance-aligned group policy training that rewards verifier-accepted prefix length after KD warm start with KL anchoring.

Load-bearing premise

A small fixed set of visual tokens chosen from the draft model’s current text state is enough for the draft model to stay aligned with a verifier that always sees the full image, especially on vision-critical tokens.

What would settle it

On the same exact full-vision verifier protocol, measure accepted prefix length and speedup when TIGER’s router is replaced by random or static top-k selection, or when k is forced far from 8 on OCR/counting and long-video tasks; if AccLen and speedup do not fall relative to TIGER’s default routing and acceptance objective, the central claim fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 0 minor

Summary. TIGER addresses limited speculative-decoding speedups in VLMs by combining (i) text-conditioned visual gated routing that exposes the drafter only to a sparse top-k subset of visual tokens scored from the current prefix hidden state, and (ii) a two-stage training recipe: KD warm-start followed by GRPO-style acceptance alignment whose reward is the verifier-accepted contiguous prefix length under standard speculative decoding. The verifier always uses the full visual bank and is frozen. The paper evaluates exact verifier-side AccLen/speedup on LLaVA-v1.6 (7B/13B) and Qwen3-VL (4B/8B) across MMBench, ScienceQA, TextVQA, MM-Vet, and COCO Caption (Table 1), with ablations (Table 3), routing sparsity (Table 4), capability checks, context-heavy extensions (Video-MME/NLVR2/MuirBench), and extensive appendix analyses of objectives, selectors, and overhead.

Significance. If the reported AccLen and speedup edges hold under fair, reproducible comparison, the work is a useful systems contribution: it reframes multimodal speculative decoding around the runtime quantity that actually governs speedup (accepted prefix length) rather than pure imitation, and couples that objective with an adaptive drafter-side visual interface complementary to global visual-token pruning. Strengths include multi-family backbones, a clean separation of exact full-vision verification from auxiliary routing analyses, component ablations, objective variants (App. E.7), adapter-style pruning comparisons (App. E.6), and promised code plus training mixtures. The practical impact is incremental rather than transformative—the paper itself describes gains as generally modest—but the problem framing and training–inference alignment are of clear interest to efficient multimodal serving.

major comments (3)
  1. Table 1 is the load-bearing evidence for the abstract’s claim of “consistent gains” in AccLen and speedup, yet many deltas versus strong adapted baselines (ViSpec, EAGLE-3, RACER, TokenRecycling) are tiny (e.g., LLaVA-7B TextVQA AccLen 2.98 vs 2.97/2.96; COCO 3.37 vs 3.34/3.31; several Spd. ties or +0.01–0.05×), and §4.2 itself calls gains “generally modest.” No multi-seed means, standard errors, or significance tests are reported. Without variance estimates, it is not possible to judge whether the ranking is a stable method effect or adaptation/tuning noise. Please add multi-seed AccLen/Spd. with error bars (or bootstrap CIs) on at least one backbone×benchmark suite, and temper “consistent” language where differences fall within noise.
  2. Appendix B.4 documents extensive re-instantiation and retraining of Medusa, EAGLE-2/3, ViSpec, and SpecFLASH on the paper’s exact LLaVA/Qwen backbones, plus custom multimodal instrumentation of training-free RACER and TokenRecycling under an adapted acceptance protocol. This is necessary for fairness but also makes Table 1 sensitive to adaptation quality. The manuscript should state, for each trainable baseline, which components were retrained, data budget relative to TIGER’s 92.5K KD + 80K GRPO pools (Tables 7–8), and whether hyperparameter search matched TIGER’s effort. Ideally include a short “adaptation audit” (e.g., AccLen of official vs re-adapted checkpoints where available) so readers can separate method gains from reimplementation variance.
  3. The central design assumption (§3.2, Table 4, Limitations)—that a fixed small top-k (default k=8) text-conditioned subset supplies enough vision-critical evidence for the drafter to match a full-context verifier—is only partially stress-tested. Long-video Video-MME results (Table 12) remain weak and nearly tied with TokenRecycling/RACER, consistent with the risk of omitting temporally sparse global cues. Please either (i) quantify failure modes where routing drops OCR/count/global evidence the verifier uses (e.g., acceptance breakdown by token type), or (ii) report a dynamic-k / multi-query routing variant on long-video and multi-image settings so the claim’s scope is clear.

Circularity Check

0 steps flagged

No circularity: AccLen is an empirical runtime outcome under a frozen full-vision verifier, not a quantity forced by definition or self-citation.

full rationale

TIGER’s chain is: (i) route a sparse text-conditioned visual subset to the drafter (§3.2, Eqs. 6–10); (ii) warm-start with KD then optimize a GRPO-style objective whose reward is the verifier-accepted prefix length under the standard speculative acceptance test (§3.3, Eqs. 11–15); (iii) report AccLen/speedup on held-out multimodal benchmarks with an unchanged full-vision verifier (Table 1). That training targets the same runtime quantity later reported is intentional train–test alignment for speculative decoding, not a tautology: acceptance is still decided by a frozen independent verifier on the full visual bank, routing does not redefine the acceptance test, and ablations (Table 3) show AccLen falls when routing or acceptance alignment is removed—so the reported gains are not forced by construction. There is no uniqueness theorem, no load-bearing self-citation chain, no fitted scalar renamed as a prediction, and no renaming of a known closed-form result. Comparisons are empirical against external baselines under a fixed protocol. Score 0 with empty steps is the correct finding.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 2 invented entities

The central efficiency claim rests on standard speculative-decoding theory, a design choice that sparse text-conditioned visual evidence is enough for drafting, and several hand-chosen training/inference knobs (k, γ, K, β, data filters). No new physical entities; invented pieces are algorithmic modules. Free parameters are hyperparameters and sparsity, not fits that redefine the metric.

free parameters (5)
  • routing sparsity k = 8
    Number of visual tokens exposed to the drafter; default k=8 for train and infer, chosen via sensitivity on COCO Caption and capability subsets (Table 4, App. E.5).
  • draft block length γ = 32
    Speculative block size used in acceptance-aligned training (Table 6).
  • GRPO group size K = 4
    Number of candidate draft blocks sampled per example for relative reward normalization (Table 6).
  • KL anchor coefficient β = 0.02
    Strength of KL(qθ || qref) in L_accept to limit policy drift (Eq. 15, Table 6).
  • acceptance-stage learning rate / effective batch = 1e-5 / batch 8
    Optimization knobs for the second training stage (1e-5, effective batch 8; Table 6).
axioms (4)
  • domain assumption Standard speculative decoding acceptance/rejection sampling with a full-context verifier preserves the target model’s output distribution (lossless acceleration).
    Invoked throughout §1 and §3.1; verifier always uses full visual bank V.
  • ad hoc to paper A sparse, text-conditioned subset of visual tokens can substitute for the full visual bank on the drafter without destroying draft–verify agreement on vision-critical tokens.
    Core design premise of §3.2 routing; Limitations admits possible omission of global evidence.
  • ad hoc to paper Verifier-accepted prefix length is the right primary training signal for speculative efficiency (beyond token-level KD).
    Stated in §1 and formalized as R(k)=A(k) in §3.3; compared to binary/token-agreement rewards in App. E.7.
  • ad hoc to paper Medium-difficulty heuristic filtering of GRPO prompts yields useful variance in acceptance rewards.
    App. C.1 filters short/long/yes-no prompts for the 80K acceptance pool.
invented entities (2)
  • TIGER text-conditioned visual gated router (Wq, Wv, Top-k over visual bank + optional global token) no independent evidence
    purpose: Build a sparse drafter-side visual interface recomputed each speculative round from the current text hidden state.
    New module relative to fixed full vision or static compression; independent_evidence false beyond this paper’s ablations.
  • Acceptance-aligned GRPO objective with reward equal to contiguous accepted prefix length no independent evidence
    purpose: Directly optimize the runtime quantity that governs speculative speedup after KD warm-start.
    Applies group relative policy optimization to verifier acceptance outcomes; not a new physical entity but a paper-specific training construct.

reviewed 2026-07-14 · how reviews work

0 comments
Cite this review

Pith. "Pith review of TIGER: Text-Conditioned Visual Gated Routing with Acceptance Alignment for Multimodal Speculative Decoding." pith.science (2026). https://pith.science/paper/QRIFZN6E

@misc{pith2026260711131,
  author       = {Pith},
  title        = {Pith review of: TIGER: Text-Conditioned Visual Gated Routing with Acceptance Alignment for Multimodal Speculative Decoding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QRIFZN6E}},
  note         = {Machine review of arXiv:2607.11131}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Speculative decoding accelerates autoregressive generation by letting a lightweight drafter propose multiple tokens that are verified by a larger target model. Although effective for text-only LLMs, speculative decoding yields limited gains in VLMs because drafters often diverge on vision-critical content, while existing multimodal acceleration methods do not directly address irrelevant visual evidence or optimize the verifier-accepted prefix length that governs speedup. We propose TIGER, a Text-conditioned vIsual GatEd Routing framework for multimodal speculative decoding. TIGER dynamically selects a sparse set of context-relevant visual tokens based on the drafter's current textual state, rather than expose the full visual token set or a fixed compressed interface. To better align training with inference-time efficiency, we optimize the drafter with acceptance-aligned group-based policy training using verifier-derived rewards based on accepted prefix length, built on top of distillation warm start with KL anchoring. This encourages the drafter not only to imitate the target model, but also to produce speculative continuations that survive verification for longer prefixes. Experiments show that TIGER yields consistent gains in accepted prefix length and speculative speedup under exact verifier-side speculative decoding, while achieving favorable quality-latency trade-offs with comparable downstream accuracy in visual-routing analyses.

Figures

Figures reproduced from arXiv: 2607.11131 by Cong-Duy Nguyen, Luu Anh Tuan, Ponhvoan Srey, Quynh Vo, Thong Nguyen.

Figure 1
Figure 1. Figure 1: Overview of TIGER for multimodal speculative decoding. Given an image and a textual prefix, TIGER first performs text-conditioned visual routing to select a sparse subset of relevant visual tokens for the drafter. Conditioned on this routed visual context, the drafter proposes a speculative block of γ tokens. The verifier, which always operates on the full visual context, then accepts a prefix of the propo… view at source ↗
Figure 2
Figure 2. Figure 2: TIGER visual gated routing. A routing query derived from the current prefix hidden state scores each token in the visual token bank, and the top-k most relevant tokens are selected as the routed visual context for the next speculative block. so that drafts that survive verification for longer prefixes receive larger rewards. To optimize the drafter, we adopt a group-based relative policy update over the K … view at source ↗
Figure 3
Figure 3. Figure 3: Capability analysis on LLaVA-v1.6-7B across [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Execution trace of TIGER on a qualitative example. Each row shows the routed visual tokens selected at [PITH_FULL_IMAGE:figures/full_fig_p025_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Execution trace of TIGER across speculative rounds on a captioning example. Each row shows the routed [PITH_FULL_IMAGE:figures/full_fig_p026_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Execution trace of TIGER on a genus-classification example involving the Goliath heron. Each row shows [PITH_FULL_IMAGE:figures/full_fig_p027_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Execution trace of TIGER on a two-image visual reasoning example. Each row shows the routed visual [PITH_FULL_IMAGE:figures/full_fig_p028_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Execution trace of TIGER on a snow-camouflage reasoning example involving an Arctic wolf. Each [PITH_FULL_IMAGE:figures/full_fig_p029_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Execution trace of TIGER on a video reasoning example about exhaled breath while pulling apart. Each [PITH_FULL_IMAGE:figures/full_fig_p030_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Execution trace of TIGER on a visual reasoning example about identifying the person with a leg on the [PITH_FULL_IMAGE:figures/full_fig_p031_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

32 extracted references · 1 linked inside Pith

  1. [1]

    Forty-second International Conference on Machine Learning , year=

    Accelerating Large Language Model Reasoning via Speculative Search , author=. Forty-second International Conference on Machine Learning , year=

  2. [2]

    EAGLE -2: Faster Inference of Language Models with Dynamic Draft Trees

    Li, Yuhui and Wei, Fangyun and Zhang, Chao and Zhang, Hongyang. EAGLE -2: Faster Inference of Language Models with Dynamic Draft Trees. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024. doi:10.18653/v1/2024.emnlp-main.422

    doi:10.18653/v1/2024.emnlp-main.422 2024

  3. [3]

    Towards VQA Models That Can Read , year=

    Singh, Amanpreet and Natarajan, Vivek and Shah, Meet and Jiang, Yu and Chen, Xinlei and Batra, Dhruv and Parikh, Devi and Rohrbach, Marcus , booktitle=. Towards VQA Models That Can Read , year=

  4. [4]

    Proceedings of the 36th International Conference on Neural Information Processing Systems , articleno =

    Lu, Pan and Mishra, Swaroop and Xia, Tony and Qiu, Liang and Chang, Kai-Wei and Zhu, Song-Chun and Tafjord, Oyvind and Clark, Peter and Kalyan, Ashwin , title =. Proceedings of the 36th International Conference on Neural Information Processing Systems , articleno =. 2022 , isbn =

    2022

  5. [5]

    Evaluating Object Hallucination in Large Vision-Language Models

    Li, Yifan and Du, Yifan and Zhou, Kun and Wang, Jinpeng and Zhao, Xin and Wen, Ji-Rong. Evaluating Object Hallucination in Large Vision-Language Models. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023. doi:10.18653/v1/2023.emnlp-main.20

    doi:10.18653/v1/2023.emnlp-main.20 2023

  6. [6]

    2025 , url=

    Chaoyou Fu and Peixian Chen and Yunhang Shen and Yulei Qin and Mengdan Zhang and Xu Lin and Jinrui Yang and Xiawu Zheng and Ke Li and Xing Sun and Yunsheng Wu and Rongrong Ji and Caifeng Shan and Ran He , booktitle=. 2025 , url=

    2025

  7. [7]

    Proceedings of the 41st International Conference on Machine Learning , articleno =

    Yu, Weihao and Yang, Zhengyuan and Li, Linjie and Wang, Jianfeng and Lin, Kevin and Liu, Zicheng and Wang, Xinchao and Wang, Lijuan , title =. Proceedings of the 41st International Conference on Machine Learning , articleno =. 2024 , publisher =

    2024

  8. [8]

    Computer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29–October 4, 2024, Proceedings, Part VI , pages =

    Liu, Yuan and Duan, Haodong and Zhang, Yuanhan and Li, Bo and Zhang, Songyang and Zhao, Wangbo and Yuan, Yike and Wang, Jiaqi and He, Conghui and Liu, Ziwei and Chen, Kai and Lin, Dahua , title =. Computer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29–October 4, 2024, Proceedings, Part VI , pages =. 2024 , isbn =. doi:10.1007/97...

    doi:10.1007/978-3-031-72658-3_13 2024

  9. [9]

    Goyal, Yash and Khot, Tejas and Agrawal, Aishwarya and Summers-Stay, Douglas and Batra, Dhruv and Parikh, Devi , title =. Int. J. Comput. Vision , month = apr, pages =. 2019 , issue_date =. doi:10.1007/s11263-018-1116-0 , abstract =

    doi:10.1007/s11263-018-1116-0 2019

  10. [10]

    and Manning, Christopher D

    Hudson, Drew A. and Manning, Christopher D. , booktitle=. GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering , year=

  11. [11]

    and Guo, Anhong and Lin, Chi and Grauman, Kristen and Luo, Jiebo and Bigham, Jeffrey P

    Gurari, Danna and Li, Qing and Stangl, Abigale J. and Guo, Anhong and Lin, Chi and Grauman, Kristen and Luo, Jiebo and Bigham, Jeffrey P. , booktitle =. 2018 , volume =. doi:10.1109/CVPR.2018.00380 , url =

    doi:10.1109/cvpr.2018.00380 2018

  12. [12]

    Zihua Wang and Ruibo Li and Haozhe Du and Joey Tianyi Zhou and Yu Zhang and Xu Yang , year=

  13. [13]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    ViSpec: Accelerating Vision-Language Models with Vision-Aware Speculative Decoding , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

  14. [14]

    arXiv preprint arXiv:2410.17247 , year =

    PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction , author =. arXiv preprint arXiv:2410.17247 , year =

    Pith/arXiv arXiv

  15. [15]

    Proceedings of the 33rd ACM International Conference on Multimedia , pages =

    Chen, Hanning and Ni, Yang and Huang, Wenjun and Oh, Hyunwoo and Liu, Yezi and Das, Tamoghno and Imani, Mohsen , title =. Proceedings of the 33rd ACM International Conference on Multimedia , pages =. 2025 , isbn =. doi:10.1145/3746027.3755243 , abstract =

    doi:10.1145/3746027.3755243 2025

  16. [16]

    Yuan Zhang and Chun-Kai Fan and Junpeng Ma and Wenzhao Zheng and Tao Huang and Kuan Cheng and Denis A Gudovskiy and Tomoyuki Okuno and Yohei Nakata and Kurt Keutzer and Shanghang Zhang , booktitle=. Sparse. 2025 , url=

    2025

  17. [17]

    Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , year =

    Beyond Text-Visual Attention: Exploiting Visual Cues for Effective Token Pruning in Vision-Language Models , author =. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , year =

  18. [18]

    What matters when building vision-language models? , year =

    Lauren. What matters when building vision-language models? , year =. Proceedings of the 38th International Conference on Neural Information Processing Systems , articleno =

  19. [19]

    MagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative Decoding , booktitle =

    Ranajoy Sadhukhan and Jian Chen and Zhuoming Chen and Vashisth Tiwari and Ruihang Lai and Jinyuan Shi and Ian En. MagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative Decoding , booktitle =. 2025 , url =

    2025

  20. [20]

    and Chen, Deming and Dao, Tri , title =

    Cai, Tianle and Li, Yuhong and Geng, Zhengyang and Peng, Hongwu and Lee, Jason D. and Chen, Deming and Dao, Tri , title =. Proceedings of the 41st International Conference on Machine Learning , articleno =. 2024 , publisher =

    2024

  21. [21]

    2015 , eprint=

    Microsoft COCO Captions: Data Collection and Evaluation Server , author=. 2015 , eprint=

    2015

  22. [22]

    Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis , year=

    Fu, Chaoyou and Dai, Yuhan and Luo, Yongdong and Li, Lei and Ren, Shuhuai and Zhang, Renrui and Wang, Zihan and Zhou, Chenyu and Shen, Yunhang and Zhang, Mengdan and Chen, Peixian and Li, Yanwei and Lin, Shaohui and Zhao, Sirui and Li, Ke and Xu, Tong and Zheng, Xiawu and Chen, Enhong and Shan, Caifeng and He, Ran and Sun, Xing , booktitle=. Video-MME: Th...

  23. [23]

    A Corpus for Reasoning about Natural Language Grounded in Photographs

    Suhr, Alane and Zhou, Stephanie and Zhang, Ally and Zhang, Iris and Bai, Huajun and Artzi, Yoav. A Corpus for Reasoning about Natural Language Grounded in Photographs. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019. doi:10.18653/v1/P19-1644

    doi:10.18653/v1/p19-1644 2019

  24. [24]

    The Thirteenth International Conference on Learning Representations , year=

    MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding , author=. The Thirteenth International Conference on Learning Representations , year=

  25. [25]

    2026 , url=

    Yuhui Li and Fangyun Wei and Chao Zhang and Hongyang Zhang , booktitle=. 2026 , url=

    2026

  26. [26]

    2026 , eprint=

    RACER: Retrieval-Augmented Contextual Rapid Speculative Decoding , author=. 2026 , eprint=

    2026

  27. [27]

    Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recycling

    Luo, Xianzhen and Wang, Yixuan and Zhu, Qingfu and Zhang, Zhiming and Zhang, Xuanyu and Yang, Qing and Xu, Dongliang. Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recycling. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025. doi:10.18653/v1/2025....

    doi:10.18653/v1/2025.acl-long.338 2025

  28. [28]

    Proceedings of the 29th Symposium on Operating Systems Principles , pages =

    Kwon, Woosuk and Li, Zhuohan and Zhuang, Siyuan and Sheng, Ying and Zheng, Lianmin and Yu, Cody Hao and Gonzalez, Joseph and Zhang, Hao and Stoica, Ion , title =. Proceedings of the 29th Symposium on Operating Systems Principles , pages =. 2023 , isbn =. doi:10.1145/3600006.3613165 , abstract =

    doi:10.1145/3600006.3613165 2023

  29. [29]

    Proceedings of the 37th International Conference on Neural Information Processing Systems , articleno =

    Liu, Haotian and Li, Chunyuan and Wu, Qingyang and Lee, Yong Jae , title =. Proceedings of the 37th International Conference on Neural Information Processing Systems , articleno =. 2023 , publisher =

    2023

  30. [30]

    Proceedings of the 40th International Conference on Machine Learning , articleno =

    Li, Junnan and Li, Dongxu and Savarese, Silvio and Hoi, Steven , title =. Proceedings of the 40th International Conference on Machine Learning , articleno =. 2023 , publisher =

    2023

  31. [31]

    Proceedings of the 40th International Conference on Machine Learning , pages =

    Fast Inference from Transformers via Speculative Decoding , author =. Proceedings of the 40th International Conference on Machine Learning , pages =. 2023 , publisher =

    2023

  32. [32]

    2024 , publisher =

    Miao, Xupeng and Oliaro, Gabriele and Zhang, Zhihao and Cheng, Xinhao and Wang, Zeyu and Wong, Rae Ying Yee and Zhu, Alan and Yang, Lijie and Shi, Xiaoxiang and Shi, Chunan and Chen, Zhuoming and Arfeen, Daiyaan and Abhyankar, Reyna and Jia, Zhihao , booktitle =. 2024 , publisher =

    2024

This paper was first reviewed by grok-4.5 on July 14, 2026.