Keyword Matters: Unveiling the Energy Sensitivity of On-Device LLM Prompting

Haoxin Wang; Ruiyi Tao; Xiaolong Tu

REVIEW 4 major objections 6 minor 12 references

Keyword Matters: Unveiling the Energy Sensitivity of On-Device LLM Prompting

T0 review · 4 major / 6 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Choosing a different verb in a prompt shifts on-device LLM energy draw by up to 103 percent, with 'label' saving over 60% in sentiment tasks.

desk verdict Genuine measurement effort, but the sentiment results are confounded by template changes and the whole study lacks repeats, so the headline savings figures are not yet trustworthy. read the letter →

arxiv 2607.22568 v1 pith:DPZTURH3 submitted 2026-05-31 cs.AI cs.NI

Ruiyi Tao , Xiaolong Tu , Haoxin Wang This is my paper

classification cs.AIcs.NI

keywords energyefficiencyon-deviceLLMpromptengineeringkeywordselectiondecodesmartphonemeasurementsentimentanalysistextgeneration

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that prompt wording—specifically, which imperative verb a user chooses—is a measurable lever on the energy an on-device LLM consumes during generation. Using real voltage and current traces from a smartphone and a single-board computer across five compact models and two tasks, it reports consistent decode-phase energy differences: the most efficient keyword saved between 18.5% and 62.6% of mean decode energy depending on task, while the least efficient cost up to 103% more. If true, this makes prompt engineering a zero-retraining, zero-hardware lever for battery life, complementary to model compression. The paper also argues quality need not suffer: in text generation energy and judged quality are nearly uncorrelated, while in sentiment analysis the shorter, more efficient responses tend to be more accurate.

What carries the argument

The central object is the keyword-replacement prompt set: 50 prompts per task differing only in the imperative verb (10 verbs per task). The measurement chain is the energy decomposition into prefill and decode phases, with decode energy computed from 10 Hz battery voltage/current samples on a phone and an FNIRSI power logger on a single-board computer. The analysis metric is the percentage deviation of each keyword's mean decode energy from the model's overall mean, plus Spearman rank correlations to test cross-model and cross-device consistency.

What would settle it

Run the same 1,000 prompt–model pairs multiple times (at least 10) on the same devices and compute per-keyword energy distributions with confidence intervals; if the mean differences among verbs shrink to within noise, the claimed sensitivity fails. Also, if controlling for response length (e.g., early stopping at equal token counts) removes the keyword–energy correlation, then the effect is entirely length-mediated rather than a distinct 'keyword' effect.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that keyword choice alone—with the same prompt structure, model, device, and decoding settings—shifts decode-phase energy consumption, with the variance stemming largely from changes in response length and reasoning depth that different verbs trigger. Measured deviations range from −62.6% to +103% relative to each model's mean; 'label' is consistently efficient in sentiment analysis, 'create' in text generation, while 'analyze' and 'craft' are consistently inefficient. These differences appear stable within model families (Qwen models show Spearman correlation 0.867 in sentiment) and across devices for closed-ended tasks (Qwen sentiment ρ=0.

Load-bearing premise

That each keyword's mean energy is reliably estimated from a single run per prompt with no error bars or statistical tests; if the 10 Hz energy traces are noisy, the reported savings and rankings could be artifacts.

Editorial extensions

If this is right

If the findings generalize, replacing 'analyze' with 'label' in sentiment prompts could cut decode energy by roughly half on existing devices with no retraining.
Energy-optimal keywords are task- and model-family-dependent: 'create' leads in text generation, 'label' in sentiment, so energy-aware prompt rewriting must be per-task.
Because text-generation energy and response quality were uncorrelated, energy optimization in that setting is nearly free; in sentiment, the relation is inverse, so efficiency and accuracy align.
Consistency within model families (Qwen sentiment ρ=0.867, text ρ=0.588) suggests a 'green prompt' lookup can be built once per model family rather than per device.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Single-run measurements at 10 Hz sampling make the magnitude claims fragile; a repetition study with error bars might shrink or shift the rankings, though the direction—verbs that cut response length save energy—is plausible.
The likely mechanism is response length: energy scales almost linearly with output length, so keywords that make models stop sooner (like 'label' vs 'analyze') save energy; a testable extension is to control for response length and see if keyword-specific energy differences persist beyond length.
The proposed 'green prompt engine' implies a practical on-device rewriter that preserves meaning via entailment checks, but it would need to validate against the paper's limited prompt diversity (Alpaca-GPT4 and Yelp only).
All models share the same q4f16 quantization, so it is unknown how the effect interacts with different bit-widths, quantization schemes, or non-instruction-tuned models.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Honest take: this is a well-intentioned empirical study with real measurement work, but the claims outrun the data. The new thing is a dataset linking prompt verbs to measured decode energy on two edge devices across five small LLMs. The measurement protocol is the strength: real power traces, CPU/GPU locking, per-phase energy split, and a prompt-complexity check for the text-generation set. Credit where due: the complexity validation (r=0.942) and the cross-device sentiment correlation (ρ=0.939) are more than many papers at this level do.

The soft spots are substantial. The sentiment task does not isolate the keyword. The appendix shows the instruction sentence changes across verbs: "Classify the sentiment of the following Yelp review..." versus "Decide whether the sentiment expressed in this Yelp review...". So the headline -62.6% to +103% deviations are confounded with template length and structure. This is not fixable with more repeats; the prompts need a fixed template with only the verb swapped. The text-generation task does isolate the keyword, and there the effect is more modest (-18.5% to +26.5%, mostly single-digit). But every prompt was run once, sampled at 10 Hz, with no error bars or significance tests, so we cannot tell whether the rankings are stable. The paper's own limitations section admits this.

The deeper issue: the paper itself states energy scales almost linearly with output length. So the keyword effect is largely a response-length effect in disguise. That is not fatal—knowing which verbs reliably shorten outputs on a given model is useful engineering info—but it means the contribution is a catalog of verb-specific length effects, not a new principle. And the abstract's "consistent energy differences" conflicts with the paper's own Spearman values: 0.153 cross-model for text, 0.388 for sentiment. Only same-family and cross-device sentiment show strong consistency.

Who is this for? App developers wanting a quick heuristic on prompt wording, and researchers working on LLM energy efficiency as a baseline. It deserves a serious referee, not a desk reject, because the measurement effort is genuine and the confound is fixable. But the current numbers should not be cited without the fixes.

Referee Report

4 major / 6 minor

Summary. This paper reports an empirical measurement study of how replacing the imperative verb in an instruction prompt affects the decode-phase energy consumption of on-device LLMs. Using 1,000 prompts (50 per keyword, 10 keywords per task) across two tasks (text generation, sentiment analysis), five small models, and two edge devices (Orange Pi 5 Pro, Pixel 9a), the authors report per-keyword percentage deviations from each model's mean decode energy, ranging from -18.5% to +26.5% for text generation and -62.6% to +103% for sentiment analysis. They also examine cross-model and cross-device rank-order consistency and correlate accuracy with energy.

Significance. If the results were reliable, they would provide a useful, low-cost lever for energy-efficient prompt engineering and one of the first real-hardware measurements of keyword-level energy sensitivity. Strengths include actual power traces on two platforms with a documented CPU/GPU frequency-locking procedure, a prompt-complexity check for the text-generation set, and an explicit limitations section. However, as reported, the headline numbers are not stable estimates: each prompt is run once, no uncertainty intervals are given, and the sentiment-analysis prompts do not isolate the keyword. The cross-model consistency metric is low and sits uneasily with the abstract. The direction is promising, but the evidence is not yet conclusive.

major comments (4)

[§4.3 and Appendix C] The sentiment-analysis prompt set does not isolate the keyword. The sample prompts show that keyword replacement also changes the instruction template: 'Classify the sentiment of the following Yelp review as positive or negative:' versus 'Decide whether the sentiment expressed in this Yelp review is positive or negative:' differ in sentence length, word order, and phrasing, not only in the verb. Consequently, the -62.6% to +103% deviations in Figure 4 and the strong cross-device correlation in Figure 6 may reflect template-level differences (including prompt length) rather than keyword semantics. The statement in §4.3 that 'this was also done to isolate the impact of keywords' is contradicted by the appendix. Repeats alone would not fix this; the sentiment prompts must be re-designed so only the verb changes.
[Limitations and §4.3] The paper explicitly states that each test was conducted only once. With energy sampled at 10 Hz, a single run gives at most a few samples for short sentiment responses and no estimate of measurement noise. None of the figures or tables reports error bars, confidence intervals, or significance tests. As a result, the reported per-keyword deviations (e.g., 'label' -32.1% to -62.6%; 'analyze' +103% on Gemma) cannot be distinguished from random variation. This is load-bearing for RQ1 and for the practical savings claims. Please repeat each prompt at least several times and report uncertainty (e.g., bootstrapped CIs) and pairwise tests.
[§5.2.1 vs Abstract] The abstract claims 'consistent energy differences across verbs and tasks,' but the paper's own consistency analysis reports average pairwise Spearman coefficients of 0.153 for text generation and 0.388 for sentiment across models, which the text describes as 'very little correlation.' Cross-device text generation is also weak (ρ=0.261 for Qwen, 0.697 for SmolLM). Only sentiment analysis on Qwen shows strong cross-device consistency (ρ=0.939). The claim of consistency needs to be substantially qualified, or the abstract and conclusions revised.
[§3.1 and §5.1] The paper states that total energy scales almost linearly with output length and decode latency, but it does not report per-keyword response lengths or any length-controlled analysis. The 'keyword sensitivity' shown in Figures 3 and 4 may therefore be almost entirely a restatement of the known sensitivity of energy to generated tokens. Presenting response-length statistics and a regression or stratified analysis would clarify whether keyword choice has any effect beyond making the model more or less verbose.

minor comments (6)

[§4.2] The average prefill time of 87,000 ms on Pixel 9a seems implausible for a short prompt; if correct, please explain, otherwise correct the unit/value.
[Appendix C] The 'NOTE: The following prompts produced biased results in the original experiment' is ambiguous. Specify whether these prompts were excluded from the final dataset and why.
[Figure 4] Some extreme values (e.g., Gemma 'analyze' +103%, 'assess' +59.5%) are not annotated; if these correspond to degenerate responses or token-limit hits, state so in text.
[§4.4] The claim that criterion overlap 'has minimal effect on the significance of the overall score' is asserted without supporting analysis; either provide evidence or remove.
[References] References with incomplete author lists (e.g., 'Chenxi Liu and 1 others') should be completed.
[Reproducibility] The paper does not release the final prompt sets or raw energy traces; providing these would materially strengthen reproducibility.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: energy values are measured, not derived from keywords; the only author self-citation is a future-work aside and not load-bearing.

full rationale

The paper's central quantities are hardware measurements: total energy is computed from logged current/voltage traces (E = sum I_i V_i dt), and per-keyword deviations are averages of measured decode-phase energies over repeated prompts. There is no fitted parameter that is later renamed as a prediction, and no equation reduces a claimed output to an input by construction. The paper even describes a design correction in text generation: after noticing that the original multi-prompt-per-keyword dataset gave 'results that were suspiciously consistent across environments,' it switched to the same 50 prompts per keyword so the prompts 'only differ from each other by keyword, isolating its impact.' This is an attempt to remove a confound, not to build the result into the input. The energy–length relation is acknowledged in Section 3.1 ('total energy scales almost linearly with output length and decoding latency') and used in Section 5.1.2 to explain why sentiment-analysis percentage savings are larger ('sentiment analysis generates shorter responses, so small changes in response length can lead to large percentage changes'). That is a mediational explanation of measured data, not a definitional identity: the keyword-condition energy values are still independently measured, and the length effect does not make the keyword comparison circular. The only self-citation is Section 6.3's mention of PlatformX (Tu et al., 2025) as future-work inspiration; it is not load-bearing and no uniqueness theorem is imported. Appendix C does show that sentiment prompts vary instruction wording beyond the verb (e.g., 'Classify the sentiment of the following Yelp review...' vs. 'Decide whether the sentiment expressed in this Yelp review...'), which weakens the claim that keyword impact is isolated in that task, but this is a methodological confound/validity threat, not a circularity of derivation. The appended Limitations section also honestly flags single-run measurements, no repetition, and potential judge conflation; these are reliability concerns, not circularity. Overall, the empirical claims are self-contained and externally falsifiable, so the circularity score is minimal.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claim rests on measurement reliability and on the assumption that response length is the main driver; no fitted parameters or invented entities are introduced.

assumptions (4)

domain assumption Total energy of on-device LLM inference is dominated by the decode phase and scales almost linearly with output length.
Stated in §3.1 and used to focus the analysis on decode energy; it also implies that keyword effects on energy are largely response-length effects.
domain assumption The 50 prompts per keyword form a representative sample and, for sentiment analysis, the keyword is the only meaningful difference.
Assumed in §4.3; Appendix C shows sentiment prompts vary in phrasing beyond the verb, so the assumption is violated.
domain assumption The GPT-4.1-mini judge provides a reliable measure of response quality.
Used in §4.4; the paper itself notes the judge conflated accuracy with other metrics, undermining measurement reliability.
ad hoc to paper A single run per prompt yields a reliable estimate of the keyword mean.
The paper asserts this implicitly and lists the lack of repetition as a limitation.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Keyword Matters: Unveiling the Energy Sensitivity of On-Device LLM Prompting." pith.science (2026). https://pith.science/paper/DPZTURH3

@misc{pith2026260722568,
  author       = {Pith},
  title        = {Pith review of: Keyword Matters: Unveiling the Energy Sensitivity of On-Device LLM Prompting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DPZTURH3}},
  note         = {Machine review of arXiv:2607.22568}
}

read the original abstract

Large Language Models (LLMs) are increasingly deployed on mobile and embedded devices to improve privacy and reduce network latency. Yet on-device inference faces a fundamental constraint: high energy consumption on battery-powered, resource-limited hardware. While model compression and runtime acceleration have been widely studied, the effect of \emph{prompt design} on energy efficiency remains underexplored. This paper presents an empirical study of the relationship between prompt wording and energy consumption for on-device LLMs. Using real power measurements collected on a smartphone, we quantify how linguistic features, particularly imperative keywords and instruction structure, affect decoding length and total energy. Our results show consistent energy differences across verbs and tasks, indicating that prompt engineering is a lightweight lever for improving energy efficiency.

Figures

Figures reproduced from arXiv: 2607.22568 by the authors.

**Figure 2.** Prompt-complexity distribution comparison between the original and keyword-replaced textgeneration prompt dataset. differ from each other by keyword, isolating its impact, also preserving the inherent randomness of language models. Sentiment Analysis: Prompts synthesized using Yelp review data (Yelp, Inc., 2024) with analytical verbs (determine, label, etc.), representing short, deterministic reasoning. These pr… view at source ↗

**Figure 3.** Text generation decode energy deviation from model mean (%) across classification-related keywords on the Orange Pi 5 Pro. Values represent per-keyword deviation from each model’s mean decode energy. findings addressing our three research questions (RQ1–RQ3). We analyze how linguistic variations, particularly keywords, influence the energy consumption of on-device LLMs across 5 models and two task types (text gener… view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Sentiment analysis decode energy deviation from model mean (%) across classification-related keywords on the Orange Pi 5 Pro. Values represent per-keyword deviation from each model’s mean decode energy. generate write create produce construct make output build compose …

**Figure 6.** Figure 6: Sentiment analysis decode energy deviation from model mean for Pixel 9a and Orange Pi 5 Pro, shown for Qwen2.5-1.5B. words for energy efficiency were “create,” “generate,” and “write,” in this order for both models. In text generation, for all keywords but 3 (“craft,”…

**Figure 7.** Figure 7: Text generation decode energy % deviation from model mean and output accuracy on the Orange Pi 5 Pro. word was “label,” which performed well in all 5 models, with an average percentage deviation of -48.64%. Other keywords that performed well include “categorize” (-24.5…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

12 extracted references · 6 linked inside Pith

[3]

Yaniv Leviathan, Matan Kalman, and Yossi Matias

Ef- ficient memory management for large language model serving with pagedattention.arXiv preprint arXiv:2309.06180. Yaniv Leviathan, Matan Kalman, and Yossi Matias

arXiv
[5]

InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2511–2522, Singapore

G-eval: NLG evaluation using gpt-4 with better human align- ment. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2511–2522, Singapore. Association for Com- putational Linguistics. Alexandra Sasha Luccioni, Thomas Viguier, Alex Hernández-García, and 1 others

2023
[6]

https: //ai.meta.com/research/publications/the-l lama-3-herd-of-models/

The llama 3 herd of models. https: //ai.meta.com/research/publications/the-l lama-3-herd-of-models/. Accessed 2025-11-01. Microsoft Research

2025
[7]

https://github.com/mlc- ai/mlc- llm

Mlc-llm: Universal deployment of llms. https://github.com/mlc- ai/mlc- llm . Accessed 2025-11-01. NVIDIA

2025
[8]

https://github.com /NVIDIA/TensorRT-LLM

Tensorrt-llm. https://github.com /NVIDIA/TensorRT-LLM. Accessed 2025-11-01. NVIDIA

2025
[9]

https://github.com/tatsu-lab /stanford_alpaca

Stanford alpaca: Code and data. https://github.com/tatsu-lab /stanford_alpaca. Accessed 2025-11-01. Xiaolong Tu, Dawei Chen, Kyungtae Han, Onur Alt- intas, and Haoxin Wang

2025
[11]

https://busine ss.yelp.com/data/resources/open-dataset/

Yelp open dataset. https://busine ss.yelp.com/data/resources/open-dataset/ . Accessed 2025-11-01. Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba

2025
[12]

Relevance

Large language models are human-level prompt engineers.arXiv preprint arXiv:2211.01910. A GEval Metric Definitions 1GEval( 2name="Relevance", 3criteria="Evaluate whether the actual output meaningfully attempts to address the questions, tasks, or instructions expressed in the input prompt.", 4evaluation_steps=[ 5"Decompose the input prompt into all explici...

arXiv

Show all 12 references

[2020]

Carbontracker: Tracking and predicting the carbon footprint of training deep learning models.arXiv preprint arXiv:2007.03051. Baryluk

2007 arXiv
[2022]

Yelp, Inc

Chain-of-thought prompting elicits reasoning in large language models.arXiv preprint arXiv:2201.11903. Yelp, Inc

arXiv
[2023]

arXiv preprint arXiv:2302.12246

Active prompt- ing with chain-of-thought for large language models. arXiv preprint arXiv:2302.12246. Song Han, Huizi Mao, and William J. Dally

arXiv
[2024]

Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu

Mobilellm: Optimizing sub-billion parameter language models for on-device use.arXiv preprint arXiv:2402.14905. Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu

arXiv

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.

[1] [3]

Yaniv Leviathan, Matan Kalman, and Yossi Matias

Ef- ficient memory management for large language model serving with pagedattention.arXiv preprint arXiv:2309.06180. Yaniv Leviathan, Matan Kalman, and Yossi Matias

arXiv

[2] [5]

InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2511–2522, Singapore

G-eval: NLG evaluation using gpt-4 with better human align- ment. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2511–2522, Singapore. Association for Com- putational Linguistics. Alexandra Sasha Luccioni, Thomas Viguier, Alex Hernández-García, and 1 others

2023

[3] [6]

https: //ai.meta.com/research/publications/the-l lama-3-herd-of-models/

The llama 3 herd of models. https: //ai.meta.com/research/publications/the-l lama-3-herd-of-models/. Accessed 2025-11-01. Microsoft Research

2025

[4] [7]

https://github.com/mlc- ai/mlc- llm

Mlc-llm: Universal deployment of llms. https://github.com/mlc- ai/mlc- llm . Accessed 2025-11-01. NVIDIA

2025

[5] [8]

https://github.com /NVIDIA/TensorRT-LLM

Tensorrt-llm. https://github.com /NVIDIA/TensorRT-LLM. Accessed 2025-11-01. NVIDIA

2025

[6] [9]

https://github.com/tatsu-lab /stanford_alpaca

Stanford alpaca: Code and data. https://github.com/tatsu-lab /stanford_alpaca. Accessed 2025-11-01. Xiaolong Tu, Dawei Chen, Kyungtae Han, Onur Alt- intas, and Haoxin Wang

2025

[7] [11]

https://busine ss.yelp.com/data/resources/open-dataset/

Yelp open dataset. https://busine ss.yelp.com/data/resources/open-dataset/ . Accessed 2025-11-01. Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba

2025

[8] [12]

Relevance

Large language models are human-level prompt engineers.arXiv preprint arXiv:2211.01910. A GEval Metric Definitions 1GEval( 2name="Relevance", 3criteria="Evaluate whether the actual output meaningfully attempts to address the questions, tasks, or instructions expressed in the input prompt.", 4evaluation_steps=[ 5"Decompose the input prompt into all explici...

arXiv