{"id":"1eb6317a-4737-468f-bc29-79c6a5ba7a7b","arxiv_id":"2507.02135","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Independent Android DVFS governors waste energy and time during on-device LLM inference, and a coordinated governor (FUSE) recovers 7.0-36.8% of the lost performance.","lead":"This paper measures how Android's built-in CPU, GPU, and memory power managers slow down on-device AI chatbots, and proposes a new coordinator, FUSE, that pins the chips to better speed combinations. If it works as reported, it could make mobile AI assistants noticeably faster while using the same battery.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"FUSE profiles decode at only 32 tokens but evaluates on decode lengths up to 256; if optimal frequencies shift with KV-cache growth, the claimed TPOT gains rest on an untested assumption.","rationale":"The reader's weakest assumption identifies the general transferability of FUSE's offline-profiled frequencies to deployment, including the untested input-content-agnostic claim. I agree with that identification but sharpen it to a specific, testable dimension: decode-length dependence. The KV-cache grows throughout decoding, changing memory-bandwidth pressure and the CPU/GPU utilization balance, so the optimal frequency combination for decode is not obviously constant across the 32-to-256-token range present in the ShareGPT trace. The paper profiles decode at exactly 32 tokens and does not report whether optimal frequencies vary with decode length, even though Section 4 exhaustively searched multiple length combinations. This is the most load-bearing gap because FUSE's runtime strategy and its headline TPOT improvements depend on a single static decode configuration being near-optimal for all requests. I do not see an internal inconsistency in the measurement methodology, and the downward-spiral mechanism is supported by controlled experiments and a direct frequency trace, so the central governor-inefficiency claim is plausible. The FUSE evaluation would be solidified by reporting optimal frequencies across decode lengths and by comparing against a length-aware variant on a held-out trace. Since this is an addressable validation gap rather than a demonstrated error, the reader's CONDITIONAL verdict remains appropriate.","tokens_in":23095,"tokens_out":5765,"duration_ms":70402,"concrete_test":"Use the Section 4 exhaustive Pin methodology on Pixel 7 with TinyLlama to find the optimal CPU/GPU/memory combination for decode lengths 32, 64, 128, and 256 with prefill fixed at 128 tokens. If the optimal frequency combination differs by more than one frequency step between 32 and 256, the single decode profile in FUSE is not representative. Then run FUSE on a held-out ShareGPT split both as-is and with per-decode-length profiles; if the length-aware variant achieves more than 5% lower TPOT at equal energy-per-token, the input-content-agnostic assumption is falsified and FUSE's reported gains are not validated for longer generations.","verdict_should_be":"UNCHANGED","load_bearing_attack":"FUSE's runtime design assumes the optimal CPU/GPU/memory configuration for a given model is 'input-content-agnostic and primarily affected by the prefill length' (Section 6). The offline search therefore profiles one decode length (32 tokens) and five prefill lengths, then applies those static configurations to every request in the ShareGPT trace, whose decode lengths reach 256 tokens. This is load-bearing because the decode stage's memory-bandwidth demand grows as the KV-cache accumulates: attention reads the full KV-cache each step, so the balance between GPU compute, CPU-side OpenCL feeding, and memory traffic changes with sequence position. Section 4's own Figure 3 shows the energy-latency Pareto optimum varies across the 16 (prefill, decode) length combinations, yet the paper never reports whether the optimal frequencies themselves shift with decode length. If the optimum at 32 decode tokens differs from the optimum at 256 tokens, FUSE's static decode configuration is suboptimal on longer requests, and the reported 25.4-36.8% TPOT reductions are not the achievable optimum. The evaluation cannot detect this because profiling and testing use the same devices, models, and frequency tables, and no held-out or length-aware comparison is reported. The concern is addressable rather than fatal: the core governor-inefficiency measurements in Sections 4-5 are independent of this assumption, but the FUSE contribution is directly contingent on it.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper measures energy and latency of on-device LLM inference (llama.cpp, OpenCL) under Android's default CPU/GPU/memory DVFS governors on Pixel 7 and Pixel 7 Pro, and compares with pinning all 2808 frequency combinations. It reports that the default governors are far from optimal: for the same energy, the optimal pin reduces prefill TTFT by up to 40.4% and decode TPOT by up to 31.8% for TinyLlama, with consistent gains for other models. Controlled experiments show both the GPU governor and EAS independently select overly low frequencies, and that they can trigger a 'downward spiral' where each governor's frequency reduction lowers the other's utilization, cascading both to their minima. The authors then design FUSE, an offline-profiled unified governor that pins a small set of CPU/GPU frequencies (memory governor left at default) based on prefill length. On a 200-request ShareGPT trace, FUSE reduces TTFT by 7.0-16.9% and TPOT by 25.4-36.8% on average with the same energy-per-token across six models.","tokens_in":23369,"tokens_out":9197,"duration_ms":95652,"significance":"The paper's strengths are the unusually thorough measurement campaign (2808 frequency combinations, two phones, six models, two optimization goals, controlled pinning experiments), the concrete and well-documented downward-spiral mechanism, and the release of FUSE as an extension to llama.cpp. If the results generalize, this is an important contribution: it exposes a coordination failure in commodity mobile governors for LLM workloads and demonstrates a practical, low-cost fix with substantial energy/latency gains. The main reservation is that the FUSE evaluation is built on an unvalidated assumption that optimal frequencies are input-content-agnostic and primarily affected by prefill length; because the offline profiling and online evaluation share the same devices, models, and traces, the evaluation does not yet separate the benefit of the search strategy from a simple static pin. With additional sensitivity and generalization experiments, the contribution would be solid.","major_comments":[{"comment":"The assumption that the optimal frequency configuration for a given model is 'input-content-agnostic and primarily affected by the prefill length' is stated but never validated. FUSE profiles decode at a single length of 32 tokens and five prefill lengths, then applies these static configurations to every request in the ShareGPT trace, whose decode lengths reach 256 tokens. During decode, attention reads the full KV-cache at each step, so the balance of CPU-side OpenCL feeding, GPU compute, and memory traffic changes with sequence position; if the optimal frequencies shift with decode length, the static decode setting is suboptimal on longer requests and the reported 25.4-36.8% TPOT reductions are not the achievable optimum. The paper should report whether the optimal CPU/GPU frequencies found in Section 4 vary with decode length (e.g., by repeating the sweep at decode lengths 32, 64, 128, and 256) or evaluate a decode-length-aware variant of FUSE.","section":"Section 6 (Design overview) and Section 6.2"},{"comment":"The frequency search relies on the observed U-shape energy-per-token curves (Figs. 4 and 8) and stops at the first frequency that meets the energy budget. This unimodality assumption is only validated on the same platform and models used by the subsequent evaluation, which makes the 'effectiveness of frequency search' results in Section 6.2 partly circular: the search is tuned to a property measured on the exact devices being evaluated. To strengthen the claim, the authors should validate the U-shape on a held-out model or device, or compare FUSE's found configurations against the exhaustive 2808-combination optimum for at least one model and report the gap to the true Pareto frontier.","section":"Section 6.1"},{"comment":"The paper's headline 40.4% latency improvement comes from the exhaustive pinning search, while FUSE achieves up to 36.8% TPOT and 16.9% TTFT improvements over the default governors. It is not reported how close FUSE's selected configurations come to the exhaustive-search optimum (Pin-Opt) on the same models and settings, so it is unclear whether the 374x search reduction sacrifices a significant portion of the available gain. Adding such a comparison would help readers separate the value of the unified-governor concept from the specific search heuristic.","section":"Section 4 vs Section 6.2"},{"comment":"The testbed pins frequencies on battery-bypassed phones, but the paper does not report whether the pinned frequencies actually held for the entire duration of each run, nor does it report device temperature or run ordering. Thermal throttling is a known confound on mobile devices and can be asymmetric between long high-frequency runs (e.g., the 115.15 s Gov run in Fig. 3) and shorter pinned runs. The authors should report thermal/clock-holder monitoring and describe how they controlled for thermal drift (e.g., randomization or cool-down periods).","section":"Section 3 (Methodology)"}],"minor_comments":[{"comment":"Typos: 'bevahior' (Section 5.4), 'explaiend' (Section 5.3), 'caculating' (Section 2.3), and 'Specicially' (Section 3).","section":"Various"},{"comment":"The first sentence says 'by pinning CPU and memory frequencies', but the experiments in Fig. 6 pin CPU and GPU frequencies (GPU at 471 MHz and CPU at 1826 MHz). Please correct the text.","section":"Section 5.2"},{"comment":"References [46] and [47] are identical (Mobile Foundation Model as Firmware); the duplicate should be removed or replaced with the original citation.","section":"References"},{"comment":"The statement that results for other prefill/decode length combinations are 'similar' is unsupported; either include an appendix figure or qualify the claim.","section":"Section 5.1"},{"comment":"The 200-request ShareGPT sample is a single random draw; the paper should report variance across multiple seeds or bootstrap confidence intervals for the TTFT/TPOT reductions.","section":"Section 6.2"},{"comment":"The abstract's 'up to 40.4% longer prefilling and decoding latency' could be misread as applying to both stages; Section 4 reports 40.4% for prefill TTFT and 31.8% for decode TPOT on TinyLlama. Please attribute the numbers per stage.","section":"Abstract and Section 4"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical systems study that fits the journal's scope. My main request is for the authors to close the self-referential loop in the FUSE evaluation by adding a decode-length sensitivity analysis and an exhaustive-search comparison on at least one model. I would not require new hardware; a carefully designed sensitivity study on the existing testbed should suffice. The duplicate reference and typos are trivial to fix."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere is the short version: this is a solid measurement paper that quantifies a real problem with default DVFS governors on mobile LLM inference. The 40% latency gap at equal energy is convincing, and the downward-spiral mechanism between EAS and the GPU governor is well supported by traces. FUSE, the proposed fix, is simple and works in their tests, but its headline gains rest on an untested assumption about decode length.\n\nWhat is new: this is the first study I know of that looks at the interaction of CPU, GPU, and memory governors during on-device LLM inference. The measurement setup is careful: 2808 frequency combinations, two Pixel devices, six models, battery bypass, fine-grained power. Section 5's isolation experiments are the core contribution—they show each governor's utilization targeting goes wrong on this workload, and the coordinated downward spiral explains why default behavior is so bad. That is a genuinely useful insight for anyone working on mobile systems.\n\nWhere it is soft: FUSE assumes the optimal CPU/GPU frequency for a model depends only on prefill length, not on prompt content or decode position. The offline search profiles decode at 32 tokens and then pins those frequencies for entire requests, some with decode lengths up to 256. Attention traffic grows with KV-cache size, so the compute/memory balance shifts during generation; the optimal frequency likely moves with it. They never test whether that matters. So the 25-37% TPOT improvement is real relative to default governors, but it is not the achievable optimum, and it could degrade on longer generations. They also evaluate on the same platform used for profiling, with no held-out device or prompt set, and no variance reported. These are addressable weaknesses, not fatal ones.\n\nThe basic governor-inefficiency claim, Sections 4-5, stands on its own. The FUSE part is a reasonable engineering follow-up, not a breakthrough.\n\nWho should read this: mobile systems people, anyone building on-device LLM runtimes, and OS folks thinking about coordinated DVFS. It deserves a serious referee—the measurements are reproducible in spirit and the mechanism is worth checking. I would recommend accepting it with a request for stronger external validation of FUSE, specifically a decode-length sweep.\n\nRecommendation: send to peer review. The core empirical findings are solid; FUSE needs more scrutiny.","headline":"A careful measurement study showing mobile DVFS governors fail on LLM inference, with a plausible but under-validated fix.","tokens_in":23926,"tokens_out":3284,"would_cite":true,"duration_ms":36051,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Android's uncoordinated chip governors can add up to 40.4% latency to on-device LLM inference, and a unified governor recovers most of the gap.","keywords":["mobile LLM inference","DVFS governors","energy efficiency","CPU-GPU coordination","frequency pinning","Android power management","downward spiral","FUSE"],"falsifier":"Re-run the full 2,808-combination frequency sweep for a fixed model and prefill length while varying only the prompt text, the battery or thermal condition, or a concurrent background app; if the best frequency combination moves with any of these, then FUSE's offline lookup table is incomplete. A lighter check: take FUSE's chosen configuration on a warm phone and compare it against the best configuration found on a cool phone for the same prompt; if the configurations differ or the latency/energy ordering reverses under one condition, the paper's claim that the optimal configuration is primarily a function of prefill length fails.","tokens_in":22900,"feed_emoji":"⚡","tokens_out":10047,"duration_ms":103996,"temperature":0.7,"pith_summary":"On-device large language model inference uses the phone's CPU, GPU, and memory even when the model is mostly running on the GPU, and Android's DVFS governors (the OS routines that scale each chip's voltage and frequency) tune those components independently. This paper claims that this uncoordinated tuning is far from optimal for LLM workloads: across thousands of pinned frequency combinations on Pixel 7-series phones, the default governors produce up to 40.4% longer prefill and decode latency than some CPU/GPU/memory frequency combination that consumes the same energy, and up to 16.6% more energy at the same latency. The paper traces the inefficiency to a downward spiral in which the CPU and GPU governors each lower their frequency to chase a utilization target, starving the other component and triggering further reductions. It then builds FUSE, a unified governor that profiles each model offline and pins the components to the best frequency pair at runtime. On six models serving a real chat trace, FUSE cuts time-to-first-token by 7.0-16.9% and time-per-output-token by 25.4-36.8% on average while consuming the same energy per token.","feed_headline":"Phone chip governors waste up to 40% of LLM speed","feed_subtitle":"Pinning CPU and GPU to profiled frequencies cuts per-token delay 25-37% and first-token delay 7-17% at equal energy.","key_machinery":"The object that carries the argument is the frequency-pinning testbed and the two-step search that FUSE uses. The testbed writes a chosen frequency into both the minimum and maximum frequency limits of each component's governor, letting the authors enumerate all 2,808 combinations and compare them against governor behavior. The explanatory mechanism is the utilization-threshold feedback loop: EAS and the GPU governor each raise or lower frequency to keep hardware utilization in a vendor-defined range, but during LLM decode the utilization of each component depends on the other component's frequency, so lowering either frequency drops the other's utilization and forces another downward step. FUSE's machinery is the same testbed converted into a runtime: for each model it searches GPU frequency first because GPU frequency dominates latency and energy, keeps at most two candidate GPU frequencies, then fine-tunes CPU frequency while leaving the memory governor untouched because it is already near-optimal.","core_discovery":"The central discovery is that the default triplet of mobile governors—the EAS CPU governor, the Quickstep GPU governor, and the interactive memory governor—is jointly inefficient for LLM inference even though each component is locally sensible. The paper argues this in three steps: a full sweep of 2,808 CPU/GPU/memory frequency combinations shows that many pinned combinations dominate the default governors on both latency and energy; controlled isolation experiments show the GPU governor runs too slow in decode and EAS runs too slow in both stages, while the memory governor is near-optimal; and when EAS and the GPU governor run together, each governor's utilization-target scaling lowers the other's utilization, driving both frequencies down in a cascade to their minimums. FUSE is the constructive consequence: an offline profiling search, which starts from the highest GPU frequency and then fine-tunes CPU frequency, yields a configuration table keyed by prefill length, and runtime lookup pins those frequencies to avoid the spiral. The paper claims this delivers the measured latency and energy gains on Pixel 7 and Pixel 7 Pro devices.","pith_inferences":["The downward spiral likely generalizes beyond LLMs to any interactive mobile workload that alternates bursty GPU phases with a CPU-bound driver or runtime layer, such as image generation, augmented reality, or camera ML; the paper's measurements only cover LLM inference.","The results imply that an OS-level coordinator which shares utilization signals across CPU, GPU, and memory could approximate FUSE's offline optimum without per-model profiling.","A direct testable extension is to profile FUSE's optimal-frequency table under thermal throttling and concurrent background tasks; if the table shifts, the runtime lookup should incorporate thermal or workload features rather than only prefill length."],"forward_implications":["Stock Android DVFS governors can waste up to 40.4% of prefill/decode latency at equal energy, so OS vendors have large headroom to reclaim without changing models or hardware.","The downward-spiral mechanism means fixing one governor in isolation may not help; coordinated frequency decisions across CPU and GPU are required.","A per-model offline frequency search of about 15-31 inference runs can reduce time-to-first-token by 7.0-16.9% and time-per-output-token by 25.4-36.8% on average across six models at equal energy-per-token.","With a fixed latency target, the same approach cuts energy-per-token by 6.9-10.3% on average across models, and total energy on a 200-request trace by up to 14.3%."],"supporting_citations":[{"why":"supplies the inference engine whose CPU, GPU, and memory frequency behavior is measured and extended","marker":"[13]"},{"why":"documents the battery drain of on-device LLM inference and supplies the energy-efficiency motivation","marker":"[21]"},{"why":"describes EAS, the CPU governor whose frequency choices are shown to be too slow for LLM inference","marker":"[19]"},{"why":"provides the Quickstep GPU governor and its utilization-threshold table used to explain low GPU frequencies","marker":"[2]"},{"why":"provides the memory governor's target-frequency formula used in the memory-governor analysis","marker":"[1]"},{"why":"supplies the power monitor used to measure energy-per-token at fine granularity","marker":"[26]"},{"why":"supplies the Llama-2 model used in the frequency sweeps and FUSE evaluation","marker":"[38]"},{"why":"supplies the TinyLlama model used in the isolation and FUSE experiments","marker":"[48]"},{"why":"provides the OpenCL BLAS library through which the inference engine runs GPU workloads on these phones","marker":"[27]"}],"fun_headline_variants":["Mobile DVFS governors cost LLMs up to 40% speed","FUSE: Pin frequencies to speed mobile LLMs by 25-37%","Governor mismatch slows mobile LLMs; FUSE fixes it","Default GPU governor slows LLMs; FUSE boosts speed 25-37%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole design rests on the assumption that the frequency combination found once on an open phone without its battery remains the best choice for all later requests: the paper relies on the optimal settings depending mainly on the model and the prompt length, not on what the prompt says, how hot the chip is, or what else the phone is doing.","fun_headline_variants_meta":{"raw":{"variants":["Mobile DVFS governors cost LLMs up to 40% speed","FUSE: Pin frequencies to speed mobile LLMs by 25-37%","Governor mismatch slows mobile LLMs; FUSE fixes it","Default GPU governor slows LLMs; FUSE boosts speed 25-37%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000644,"raw_usage":{"total_tokens":3019,"prompt_tokens":1059,"completion_tokens":1960,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":675,"completion_tokens_details":{"reasoning_tokens":1880}},"tokens_in":675,"tokens_out":1960,"duration_ms":12566,"temperature":1.0,"reasoning_tokens":1880,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T20:37:47.880973+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the full 2,808-combination frequency sweep for a fixed model and prefill length while varying only the prompt text, the battery or thermal condition, or a concurrent background app; if the best frequency combination moves with any of these, then FUSE's offline lookup table is incomplete. A lighter check: take FUSE's chosen configuration on a warm phone and compare it against the best configuration found on a cool phone for the same prompt; if the configurations differ or the latency/energy ordering reverses under one condition, the paper's claim that the optimal configuration is primarily a function of prefill length fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the inference engine whose CPU, GPU, and memory frequency behavior is measured and extended"},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"documents the battery drain of on-device LLM inference and supplies the energy-efficiency motivation"},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"describes EAS, the CPU governor whose frequency choices are shown to be too slow for LLM inference"},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"provides the Quickstep GPU governor and its utilization-threshold table used to explain low GPU frequencies"},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"provides the memory governor's target-frequency formula used in the memory-governor analysis"},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the power monitor used to measure energy-per-token at fine granularity"},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"provides the OpenCL BLAS library through which the inference engine runs GPU workloads on these phones"}],"review_version":1}