Energy-Efficient Wireless LLM Inference via Uncertainty and Importance-Aware Speculative Decoding

Jihoon Park; Seong-Lyun Kim; Seungeun Oh

REVIEW 5 major objections 5 minor 17 references

Energy-Efficient Wireless LLM Inference via Uncertainty and Importance-Aware Speculative Decoding

T0 review · 5 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Hybrid LLM inference can cut cloud verifications by uploading only tokens that are both uncertain and contextually important, reaching near-full accuracy at 40.7% lower energy.

desk verdict Sensible filtering idea, but the 40.7% energy savings claim contradicts the paper's own formula—the equation and the table disagree by about 20 points, so the headline result is not supported as written. read the letter →

arxiv 2508.12590 v1 pith:22IF2DVM submitted 2025-08-18 cs.LG cs.AI

Jihoon Park , Seungeun Oh , Seong-Lyun Kim This is my paper

classification cs.LGcs.AI

keywords speculativedecodinghybridlanguagemodeledgeinferenceuncertaintyestimationattentionimportanceenergyefficiencyselectiveuploadwirelesscommunication

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that hybrid LLM inference—a small local model drafting tokens, a large cloud model verifying them—becomes much cheaper if a token is uploaded for verification only when two local signals agree: the drafter is genuinely unsure of the token, and the token's attention pattern marks it as contextually important. On 100 instruction prompts, this combined filter reaches 87.5% BERTScore versus 87.6% for full cloud verification, while cutting uploads from 100% to 38.6% and energy by 40.7%. The result matters because bandwidth- and battery-constrained edge devices cannot afford to send every token's full probability distribution to a cloud LLM; a cheap local gate could make accurate LLM inference practical at the edge. The paper also shows the gate is tunable, trading accuracy against energy through two parameters.

What carries the argument

The load-bearing object is the selective upload policy in Eq. (7), $\delta(t) = 1$ only if both the temperature-perturbation uncertainty $u(t) > \theta_u$ and the top-$k$ attention weight $a^{(L(t)^2-k+1)}_t > \theta_{\mathrm{imp}}(t)$, with $\theta_{\mathrm{imp}}(t) = \max(A_i) - \gamma\,\mathrm{std}(A_i)$. The adaptive importance threshold uses the gap below the maximum attention weight, which lets the policy stay strict when attention is sharp and lenient when attention collapses into a flat distribution.

What would settle it

Run the same $k=3$, $\gamma=1.5$ configuration on a real edge device with a power meter and the same wireless link, and compare measured end-to-end energy and upload rate against Table I's 40.7% savings and 38.6% upload rate; a large discrepancy, or a different ordering of configurations in measured energy, would falsify the energy model. As a second check, repeating the 100 prompts with different random temperature perturbations should reproduce the 87.5% BERTScore; high variance would show the uncertainty signal is not stable.

Watch

Extended reading notes

Core claim

The central discovery is that uncertainty and attention-based importance are complementary upload signals. Uncertainty alone sends many tokens that the local model is unsure about but that do not matter much semantically; importance alone can miss tokens that matter but look confident. The paper's upload policy uploads a draft token only when $u(t) > \theta_u$ and the top-$k$ attention weight exceeds $\theta_{\mathrm{imp}}(t) = \max(A_i) - \gamma\,\mathrm{std}(A_i)$, so the cloud is called precisely when the local draft is both unstable and contextually load-bearing. This keeps BERTScore within 0.1 point of full verification (87.5% versus 87.6%) while reducing LLM invocations enough to save

Load-bearing premise

The energy-saving numbers rest on hand-set per-token power costs (300 W for uplink, 350 W for resampling, 100 W for local inference) rather than measured values; if those costs do not reflect a real device and wireless link, the reported 40.7% energy saving is not guaranteed.

Editorial extensions

If this is right

At $k=3$, $\gamma=1.5$, the combined filter reaches 87.5% BERTScore versus 87.6% for full HLM while saving 40.7% energy and cutting uploads from 100% to 38.6%.
Adding the importance condition to the uncertainty-only baseline improves BERTScore from 85.8% to 87.0%, energy savings from 31.6% to 43.6%, and token throughput from 0.36 to 0.40 tokens/sec.
The parameters $k$ and $\gamma$ form a tunable accuracy-efficiency knob: stricter filtering (higher $k$, lower $\gamma$) reduces LLM usage and raises throughput at modest accuracy cost, e.g. $k=11,\gamma=0.5$ gives 0.08% uploads and 66.0% energy savings at 85.1% BERTScore.
Because fewer uploads also mean fewer resampling events, the scheme reduces wireless uplink load as well as cloud compute, which is the bottleneck the paper targets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the same two-signal gate should transfer to any cheap-draft/expensive-verify pipeline beyond token generation, such as retrieval-augmented generation or tool-call validation, because uncertainty and attention importance are not specific to autoregressive next-token verification.
Editorial inference: the adaptive attention threshold doubles as a lightweight hallucination indicator; if attention collapse tracks factual drift, the same statistic could flag uncertain generations for correction without any cloud round trip.
Editorial inference: the exact 40.7% savings is sensitive to the paper's assumed per-token power costs; a deployment should remeasure those costs on target hardware, and the qualitative result would survive if the relative magnitudes remain similar.
Testable extension: replace temperature-perturbation uncertainty with Monte Carlo dropout or ensemble disagreement and re-run the same instruction prompts; if the importance condition alone explains the quality retention, the uncertainty branch could be simplified or removed.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: the combination of epistemic uncertainty and attention-based importance for selective upload in hybrid speculative decoding is new and plausible, but the paper's central energy claim doesn't survive contact with its own math.

What's actually new: the dual-filter rule in Eq. (7)—upload a draft token only when both uncertainty and top-k attention importance pass their thresholds—extends the authors' earlier U-HLM work in a sensible way. The adaptive importance threshold built around attention collapse is a reasonable idea. The evaluation sweeps k and gamma systematically and the qualitative trade-offs look credible: stricter filtering uploads fewer tokens and costs a little BERT score.

Now the soft spot, and it's load-bearing. Section IV defines uplink energy as eu = lu·B·eps_u, with B = |V|·bprob = 1,024,000 bits per uploaded token. Plugging Table I's upload and reject rates into that formula gives an energy saving of about 61.4% for the headline (k=3, gamma=1.5) configuration, not the reported 40.7%. The 40.7% figure matches only if you drop the bit factor entirely and use eu = lu·eps_u. The equation and the table cannot both be right, and since the abstract advertises 40.7%, this is a serious internal inconsistency, not a cosmetic typo.

There are also reproducibility gaps. theta_u, the perturbation count N, and the temperature range T are never specified, so the uncertainty signal can't be replicated. The energy constants (300/350/100 W) are hand-set rather than measured. And the favorable k/gamma configuration is highlighted from a 100-prompt sweep without error bars—that's post-hoc selection, so the absolute numbers should be treated as indicative at best.

Who gets value from this: researchers working on edge/cloud LLM splitting will find the filtering rule worth reading, but they should not cite the efficiency numbers until the energy model is fixed and the hidden hyperparameters are reported. The core idea is salvageable; the paper is one corrected model and a few error bars away from being a solid short paper.

If an editor asked, I'd say send it to peer review: the combination is new, the flaw is fixable, and a good reviewer could push the authors to make the energy accounting honest. But as submitted, the headline claim should not stand.

Referee Report

5 major / 5 minor

Summary. The paper proposes an opportunistic token-level filtering mechanism for hybrid SLM/LLM (HLM) inference. A draft token is uploaded to the cloud LLM only when both an epistemic-uncertainty condition (u(t) > θ_u) and an attention-importance condition (a k-th largest attention weight exceeds an adaptive threshold θ_imp(t)) are satisfied. The method is evaluated with TinyLlama-1.1B as the SLM and LLaMA-2-7B as the LLM on 100 Alpaca prompts, sweeping k and γ. The authors report near-HLM BERTScore (87.5% vs 87.6%) with 40.7% energy savings and improved token throughput, and claim tunable accuracy-efficiency trade-offs.

Significance. If the quantitative claims were supported, the proposal would be a useful, simple mechanism for reducing wireless uplink traffic and LLM invocation in hybrid inference, and the attention-collapse-motivated adaptive threshold is an interesting ingredient. The paper also reports a broad parameter sweep and uses standard open models, which facilitates comparison. However, the central energy-efficiency claims rest on an energy model with inconsistent units and internal numerical contradictions, and several required hyperparameters are missing. As written, the significance of the findings is not established.

major comments (5)

[IV, energy model; Table I] The energy equation ¯e = e_u + e_r + e_s is dimensionally inconsistent and does not reproduce Table I. With B = 32,000×32 = 1,024,000 bits, ε_u = 300 W, and the HLM and U+I(k=3,γ=1.5) rows, the stated formula gives η ≈ 61.4%, not the reported 40.7%. Conversely, if B is omitted from e_u (i.e., e_u = l_u·300, e_r = l_r·350, e_s = l_s·100), the formula gives η ≈ 16.7%, not 40.7%. Thus the stated model and the headline energy-saving number cannot both be correct. Please define all energy coefficients with proper units (e.g., J/token or J/bit) and recompute every η value in Table I and the abstract.
[III-A, III-C, IV] The method cannot be reproduced because three required hyperparameters are never specified: θ_u in Eq. (7), N in Eq. (4), and the temperature range T from which ζ_i are sampled. The U-only and U+I upload/reject rates in Table I depend directly on these values, so the experiments are not reproducible and cannot be compared with the earlier U-HLM baseline.
[III-C vs IV] The top-k condition is defined inconsistently. Section III-C states that the draft token is eligible when the k-th largest attention weight satisfies a(L^2−k+1) > θ_imp(t), i.e., a single order statistic. Section IV states 'A token is deemed important if at least k of its attention weights exceed θ_imp(t).' These are different events. Since Eq. (7) and all I/U+I results in Table I depend on this condition, please correct the definition and confirm which policy generated the reported numbers.
[II-B, IV] Equation (2) with the stated channel parameters is inconsistent with the HLM throughput in Table I. For bprob = 32, B = 1,024,000 bits, W = 1 MHz, and ρ ≈ 0.128 (using p = 23 dBm, σ² = −104 dBm, d = 2.5 km, α = 4, h = 1), Eq. (2) gives µ(t) ≈ 5.9 s per upload. HLM uploads every token, so its throughput cannot exceed about 0.17 tokens/s; Table I reports 0.25. If the table was generated with bprob = 16 (B = 512,000 bits), this must be stated explicitly.
[IV] The evaluation protocol is post-hoc. All results come from a sweep over k and γ on the same 100 Alpaca prompts, and the paper highlights favorable configurations from that sweep without a held-out test set or repeated trials. This makes the claimed accuracy/efficiency trade-offs unverifiable. Please add a separate test split and report variability (multiple seeds or confidence intervals).

minor comments (5)

[I, abstract] In the introduction, 'reducing energy saving by 40.7%' should read 'reducing energy consumption by 40.7%'.
[Table I] The table columns are hard to parse, especially for the 'I' rows where the γ column appears to be missing or inconsistently placed. Please reformat so each method row maps unambiguously to k, γ, BERT, CS, upload rate, reject rate, η, and λ.
[Throughout] Use 'BERTScore' consistently rather than alternating 'BERT', 'BERT Score', and 'BERT score'.
[Figures] Figures 2 and 3 are referenced but are not included in the manuscript text provided; ensure that all figures are present and legible in the final version.
[III-A] The phrase 'element-wise ascending sort of all values in A(t)' is redundant; simply 'ascending sort' suffices. More importantly, please clarify whether A(t) is the full L×L attention matrix or a single head's attention matrix.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the selective upload policy and energy metrics are empirically evaluated rather than defined by their outputs.

full rationale

The paper's derivation chain is not circular. The selective upload policy in Eq. (7) is defined by combining uncertainty u(t) (Eq. 4, based on temperature perturbation) and importance (top-k attention weight vs. adaptive threshold in Eq. 6). These quantities are computed from the SLM's logits and attention scores; they are not defined in terms of the reported outcomes (BERTScore, throughput, energy savings). The reported metrics are measured after applying the policy, not used to construct the policy. Comparisons to HLM and to the authors' prior U-HLM [6] are empirical baselines; [6] is an external prior publication and is not used as an imported uniqueness theorem or to forbid alternative approaches. The energy model in Section IV uses hand-set per-token power costs ({300, 350, 100} W) and an additive formula; this is an unvalidated modeling assumption that affects the strength of the energy-saving claim, and there is a numerical inconsistency: with B = |V|*bprob = 32,000 * 32 bits = 1,024,000 bits per uploaded token, the stated formula cannot reproduce the Table I eta values (the headline U+I (k=3,gamma=1.5) case would give roughly 61% savings, not 40.7%). However, this is a correctness/arithmetic issue, not circularity: the energy-saving ratio is computed from the model inputs and measured upload/reject rates, not used to define the method. Similarly, the uncertainty threshold theta_u is not specified, but that omission is a reproducibility gap rather than a circular step. No fitted parameter is renamed as a prediction, no equation reduces to its own output by construction, and self-citations are load-bearing only as baselines and prior components, which is standard and not circular.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central efficiency claims rest on a hand-built energy model with unverified constants, plus several unreported hyperparameters in the uncertainty estimator. The free parameters are the coefficients in the energy model and the thresholds governing the upload policy. The domain assumptions are borrowed from prior literature on attention and speculative decoding. No new physical entities are introduced.

free parameters (5)

eps_u, eps_r, eps_s = 300, 350, 100 W
Per-token power costs set empirically in the energy model; changing them changes the energy saving ratio.
k (top-k attention) = grid searched over {3,5,7,9,11}
Upload strictness for importance; tuned on the evaluation prompts.
gamma = grid searched over {0.5,1.0,1.5}
Scaling factor for importance threshold margin; tuned on the evaluation prompts.
theta_u (uncertainty threshold) = not reported
Threshold in Eq. (7) governing uncertainty-based upload, value is unstated.
N and temperature range T = not reported
Number of perturbed softmax samples and temperature range in Eq. (4), unstated.

assumptions (5)

standard math Metropolis-Hastings acceptance criterion from speculative decoding (Leviathan et al.)
The verification step treats draft and target distributions as a proposal and target; invoked in Section II-A without proof.
domain assumption Attention weights indicate token importance
Borrowed from KV-cache reduction literature [10], not independently validated here for this upload policy.
domain assumption Attention collapse causes hallucination and factual drift
Cited from [8] in Section III-B; the paper provides no hallucination measurements.
domain assumption Wireless channel model with Shannon capacity, block fading, and average channel gain h=1
Used to compute transmission time and throughput in Section II-B; fading dynamics are discarded by using the average.
ad hoc to paper Energy model as sum of uplink, resampling, and SLM energies
Defined in Section IV with dimensionally inconsistent terms and hand-set coefficients.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Energy-Efficient Wireless LLM Inference via Uncertainty and Importance-Aware Speculative Decoding." pith.science (2026). https://pith.science/paper/22IF2DVM

@misc{pith2026250812590,
  author       = {Pith},
  title        = {Pith review of: Energy-Efficient Wireless LLM Inference via Uncertainty and Importance-Aware Speculative Decoding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/22IF2DVM}},
  note         = {Machine review of arXiv:2508.12590}
}

read the original abstract

To address the growing demand for on-device LLM inference in resource-constrained environments, hybrid language models (HLM) have emerged, combining lightweight local models with powerful cloud-based LLMs. Recent studies on HLM have primarily focused on improving accuracy and latency, while often overlooking communication and energy efficiency. We propose a token-level filtering mechanism for an energy-efficient importance- and uncertainty-aware HLM inference that leverages both epistemic uncertainty and attention-based importance. Our method opportunistically uploads only informative tokens, reducing LLM usage and communication costs. Experiments with TinyLlama-1.1B and LLaMA-2-7B demonstrate that our method achieves up to 87.5% BERT Score and token throughput of 0.37 tokens/sec while saving the energy consumption by 40.7% compared to standard HLM. Furthermore, compared to our previous U-HLM baseline, our method improves BERTScore from 85.8% to 87.0%, energy savings from 31.6% to 43.6%, and throughput from 0.36 to 0.40. This approach enables an energy-efficient and accurate deployment of LLMs in bandwidth-constrained edge environments.

Figures

Figures reproduced from arXiv: 2508.12590 by the authors.

**Figure 2.** Attention collapse via decoding step t uncertainty-aware HLM approach [6], we quantify uncertainty by applying temperature perturbation to the SLM’s logit outputs. Specifically, for a given decoding step t, we generate N perturbed softmax distributions {x (i) (t)} N i=1 by scaling the logits with randomly sampled temperatures ζi ∈ T , where T is a predefined temperature range. From each perturbed distribution x (i)… view at source ↗

**Figure 3.** Evaluations of U, I, and U+I across various config [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

17 extracted references · 10 canonical work pages

[1]

Harnessing the power of llms in practice: A survey on chatgpt and beyond,

J. Yang, H. Jin, R. Tang, X. Han, Q. Feng, H. Jiang, S. Zhong, B. Yin, and X. Hu, “Harnessing the power of llms in practice: A survey on chatgpt and beyond,” ACM Transactions on Knowledge Discovery from Data, vol. 18, no. 6, pp. 1–32, 2024

work page 2024
[2]

Pushing large language models to the 6g edge: Vision, challenges, and oppor- tunities,

Z. Lin, G. Qu, Q. Chen, X. Chen, Z. Chen, and K. Huang, “Pushing large language models to the 6g edge: Vision, challenges, and oppor- tunities,” arXiv preprint arXiv:2309.16739 , 2023

arXiv 2023
[3]

Hybrid slm and llm for edge-cloud collaborative inference,

Z. Hao, H. Jiang, S. Jiang, J. Ren, and T. Cao, “Hybrid slm and llm for edge-cloud collaborative inference,” in Proceedings of the Workshop on Edge and Mobile F oundation Models , pp. 36–41, 2024

work page 2024
[4]

Fast inference from trans- formers via speculative decoding,

Y . Leviathan, M. Kalman, and Y . Matias, “Fast inference from trans- formers via speculative decoding,” in International Conference on Machine Learning , pp. 19274–19286, PMLR, 2023

work page 2023
[5]

Distillspec: Improving speculative decoding via knowledge distillation,

Y . Zhou, K. Lyu, A. S. Rawat, A. K. Menon, A. Rostamizadeh, S. Ku- mar, J.-F. Kagy, and R. Agarwal, “Distillspec: Improving speculative decoding via knowledge distillation,” arXiv preprint arXiv:2310.08461, 2023

arXiv 2023
[6]

Uncertainty-aware hybrid inference with on-device small and remote large language models,

S. Oh, J. Kim, J. Park, S.-W. Ko, T. Q. Quek, and S.-L. Kim, “Uncertainty-aware hybrid inference with on-device small and remote large language models,” in IEEE International Conference on Machine Learning for Communication and Networking (ICMLCN) , 2025

work page 2025
[7]

Uncertainty-aware opportunistic hybrid language model in wireless robotic systems,

J. Park, Y . Lim, S. Oh, J. Park, and S.-L. Kim, “Uncertainty-aware opportunistic hybrid language model in wireless robotic systems,” in ICML 2025 Workshop on Machine Learning for Wireless Communica- tion and Networks (ML4Wireless) , 2025

work page 2025
[8]

Seeing far and clearly: Mitigating halluci- nations in mllms with attention causal decoding,

F. Tang, C. Liu, Z. Xu, M. Hu, Z. Huang, H. Xue, Z. Chen, Z. Peng, Z. Yang, S. Zhou, et al. , “Seeing far and clearly: Mitigating halluci- nations in mllms with attention causal decoding,” in Proceedings of the Computer Vision and Pattern Recognition Conference , pp. 26147– 26159, 2025

work page 2025

Show all 17 references

[9]

Understanding the metropolis-hastings algorithm,

S. Chib and E. Greenberg, “Understanding the metropolis-hastings algorithm,” The american statistician, vol. 49, no. 4, pp. 327–335, 1995

1995
[10]

Attention score is not all you need for token importance indicator in kv cache reduction: Value also matters,

Z. Guo, H. Kamigaito, and T. Watanabe, “Attention score is not all you need for token importance indicator in kv cache reduction: Value also matters,” arXiv preprint arXiv:2406.12335 , 2024

2024 arXiv
[11]

Latte: Low-precision approximate attention with head-wise trainable threshold for efficient transformer,

J.-P. Wang, M.-G. Lin, and A.-Y . A. Wu, “Latte: Low-precision approximate attention with head-wise trainable threshold for efficient transformer,” in 2024 IEEE 6th International Conference on AI Circuits and Systems (AICAS) , pp. 208–212, IEEE, 2024

2024
[12]

Energon: Toward efficient acceleration of transformers using dynamic sparse attention,

Z. Zhou, J. Liu, Z. Gu, and G. Sun, “Energon: Toward efficient acceleration of transformers using dynamic sparse attention,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 42, no. 1, pp. 136–149, 2022

2022
[13]

Tinyllama: An open-source small language model,

P. Zhang, G. Zeng, T. Wang, and W. Lu, “Tinyllama: An open-source small language model,” arXiv preprint arXiv:2401.02385 , 2024

2024 arXiv
[14]

Llama 2: Open foundation and fine-tuned chat models,

H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, et al. , “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288, 2023

2023 arXiv
[15]

Stanford alpaca: An instruction-following llama model,

R. Taori, I. Gulrajani, T. Zhang, Y . Dubois, X. Li, C. Guestrin, P. Liang, and T. B. Hashimoto, “Stanford alpaca: An instruction-following llama model,” 2023

2023
[16]

Bertscore: Evaluating text generation with bert,

T. Zhang, V . Kishore, F. Wu, K. Q. Weinberger, and Y . Artzi, “Bertscore: Evaluating text generation with bert,” arXiv preprint arXiv:1904.09675, 2019

1904 arXiv
[17]

Joint computation and communication cooperation for energy-efficient mobile edge comput- ing,

X. Cao, F. Wang, J. Xu, R. Zhang, and S. Cui, “Joint computation and communication cooperation for energy-efficient mobile edge comput- ing,” IEEE Internet of Things Journal , vol. 6, no. 3, pp. 4188–4200, 2018

2018

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

Harnessing the power of llms in practice: A survey on chatgpt and beyond,

J. Yang, H. Jin, R. Tang, X. Han, Q. Feng, H. Jiang, S. Zhong, B. Yin, and X. Hu, “Harnessing the power of llms in practice: A survey on chatgpt and beyond,” ACM Transactions on Knowledge Discovery from Data, vol. 18, no. 6, pp. 1–32, 2024

work page 2024

[2] [2]

Pushing large language models to the 6g edge: Vision, challenges, and oppor- tunities,

Z. Lin, G. Qu, Q. Chen, X. Chen, Z. Chen, and K. Huang, “Pushing large language models to the 6g edge: Vision, challenges, and oppor- tunities,” arXiv preprint arXiv:2309.16739 , 2023

arXiv 2023

[3] [3]

Hybrid slm and llm for edge-cloud collaborative inference,

Z. Hao, H. Jiang, S. Jiang, J. Ren, and T. Cao, “Hybrid slm and llm for edge-cloud collaborative inference,” in Proceedings of the Workshop on Edge and Mobile F oundation Models , pp. 36–41, 2024

work page 2024

[4] [4]

Fast inference from trans- formers via speculative decoding,

Y . Leviathan, M. Kalman, and Y . Matias, “Fast inference from trans- formers via speculative decoding,” in International Conference on Machine Learning , pp. 19274–19286, PMLR, 2023

work page 2023

[5] [5]

Distillspec: Improving speculative decoding via knowledge distillation,

Y . Zhou, K. Lyu, A. S. Rawat, A. K. Menon, A. Rostamizadeh, S. Ku- mar, J.-F. Kagy, and R. Agarwal, “Distillspec: Improving speculative decoding via knowledge distillation,” arXiv preprint arXiv:2310.08461, 2023

arXiv 2023

[6] [6]

Uncertainty-aware hybrid inference with on-device small and remote large language models,

S. Oh, J. Kim, J. Park, S.-W. Ko, T. Q. Quek, and S.-L. Kim, “Uncertainty-aware hybrid inference with on-device small and remote large language models,” in IEEE International Conference on Machine Learning for Communication and Networking (ICMLCN) , 2025

work page 2025

[7] [7]

Uncertainty-aware opportunistic hybrid language model in wireless robotic systems,

J. Park, Y . Lim, S. Oh, J. Park, and S.-L. Kim, “Uncertainty-aware opportunistic hybrid language model in wireless robotic systems,” in ICML 2025 Workshop on Machine Learning for Wireless Communica- tion and Networks (ML4Wireless) , 2025

work page 2025

[8] [8]

Seeing far and clearly: Mitigating halluci- nations in mllms with attention causal decoding,

F. Tang, C. Liu, Z. Xu, M. Hu, Z. Huang, H. Xue, Z. Chen, Z. Peng, Z. Yang, S. Zhou, et al. , “Seeing far and clearly: Mitigating halluci- nations in mllms with attention causal decoding,” in Proceedings of the Computer Vision and Pattern Recognition Conference , pp. 26147– 26159, 2025

work page 2025