{"id":"4bc77d0d-7219-49c9-ac0f-c5d768f9e16e","arxiv_id":"2412.03621","paper_version":4,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"JPPO++ jointly tunes prompt compression ratio and wireless transmit power with a DRL agent, reporting large latency savings in simulation but with an evaluation that partly reuses its own reward function.","lead":"This paper proposes a framework that compresses long prompts with a small language model before sending them over wireless links to a large language model, and uses reinforcement learning to choose both how much to compress and how much transmit power to use. It claims up to 46.5% service time reduction in simulation, but the evaluation relies on a self-defined fidelity score and shows quality drops on standard benchmarks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Quality-preservation claim rests on a hand-set fidelity metric that is also the DRL reward; the paper's own LongBench results show clear quality loss, so 'maintaining output quality' is unestablished.","rationale":"The reader's weakest_assumption identifies the same load-bearing issue: the fidelity metric f is used both as the evaluation of quality and as the DRL reward, making effectiveness results partly circular. I agree with that diagnosis and with the conclusion that the central claim is not supported. The strongest version of the paper's claim is that JPPO++ maintains output quality while reducing service time; this requires a quality measure that tracks real task performance. The paper's own independent evaluation in Fig. 7 shows substantial degradation on QA F1 and ROUGE-L under 16x compression, which directly contradicts the abstract's 'maintaining output quality.' My stress-test adds two specifics: (1) the components of f are token-overlap and BEP-scaled ratios, with no evidence that these correlate with semantic quality; (2) the threshold f_th in Eq. (19d) is never quantified, so even the weaker notion of 'acceptable loss' is undefined. The timing inconsistencies in Fig. 6 versus the abstract's 46.5% figure are real but secondary; the quality-preservation problem alone is sufficient to invalidate the central claim as stated. I therefore recommend keeping the reader's REJECT verdict unchanged. The framework may be salvageable, but only with an externally validated quality metric, a pre-specified acceptable-loss threshold, and released code or measurements that reconcile the service-time numbers.","tokens_in":18509,"tokens_out":4487,"duration_ms":41374,"concrete_test":"On a held-out sample from LongBench, compute the Pearson/Spearman correlation between the fidelity metric f (Eq. 7) and external task quality (QA F1 and ROUGE-L) across the same grid of compression ratios and power levels used in Tables 3-6. If the correlation is weak (e.g., r < 0.7) or non-positive at the operating points that yield the claimed 17% and 46.5% service-time reductions, then optimizing f does not imply quality preservation and the central quality claim is not established. This check should be run with the same models (GPT-Neo 125M compressor, GPT-J 6B generator) and with confidence intervals over the test samples.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that JPPO++ cuts service time by 17% (and 46.5% under 16x compression) while maintaining output quality. The service-time reductions are plausible arithmetic consequences of compression, but the 'maintaining quality' half of the claim depends entirely on the composite fidelity f = 0.4 f1 + 0.3 f2 + 0.3 f3 (Eq. 7, weights in Table 2). This same f is used as the DRL reward in Section 4.2.1, so the high fidelity/reward values in Fig. 4 and Tables 3-6 show the policy optimizing its own objective, not preserving task-level quality. The components are token overlap (f1, f3) and a BEP-scaled length ratio (f2); none is validated against semantic or task-level quality. The paper's only independent quality evidence, Fig. 7, contradicts the 'maintaining' claim: on Multi-news, no-compression average QA F1 = 0.23 versus JPPO++ = 0.19, and ROUGE-L = 0.25 versus 0.16; on GovReport, F1 = 0.37 versus 0.26 (or 0.31) and ROUGE-L = 0.33 versus 0.18/0.21. Both compressed variants are clearly worse than no compression. Moreover, the acceptable-loss threshold f_th in Eq. (19d) is never specified, so there is no principled bound within which 'acceptable loss' could be judged. The abstract's unconditional claim of maintaining output quality is therefore unsupported; at best the paper demonstrates a latency-quality trade-off with an unspecified and apparently substantial quality cost. A secondary numerical inconsistency strengthens the concern: Fig. 6 reports service-time reductions of 39-52% relative to the single-round baseline, which, combined with the 42.3% reduction of that baseline versus no compression, implies a 65-72% reduction versus no compression, not the claimed 46.5%.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes JPPO and JPPO++, frameworks that jointly optimize SLM-based prompt compression and wireless transmission power for mobile LLM services. A DRL agent selects discrete compression and power levels to maximize a composite fidelity metric subject to energy and latency constraints. The authors report that JPPO++ reduces service time by 17% versus no compression, and that 16x compression reduces service time by 42.3% (JPPO) and 46.5% (JPPO++) while maintaining output quality. The evaluation uses a weighted fidelity score as both the DRL reward and the main effectiveness measure, supplemented by a LongBench task-quality comparison and simulated latency numbers.","tokens_in":18915,"tokens_out":10871,"duration_ms":90994,"significance":"If the reported results were reliable, the paper would address a timely and practical problem: cutting both transmission and inference costs for mobile LLM services by coupling prompt compression with power control. The denoising-inspired iterative compression idea is potentially interesting, and the paper gives a clearly stated optimization problem with reproducible seeds and an explicit Double DQN formulation. However, the central evidence is not reliable: the quality metric is circular, the only independent task-level evaluation contradicts the headline quality-preservation claim, and the reported service-time reductions are internally inconsistent. The paper does not establish its main quantitative claims as submitted.","major_comments":[{"comment":"The composite fidelity f = 0.4 f1 + 0.3 f2 + 0.3 f3 is used both as the DRL reward (Section 4.2.1, Reward) and as the effectiveness metric in Fig. 4 and Section 5.2.1. The agent optimizes exactly the function on which it is later evaluated, so the reported fidelity and reward improvements partly reflect the policy maximizing its own objective rather than an independent measure of output quality. Moreover, f1 (Eq. (4)) and f3 (Eq. (6)) are token-overlap rates, while f2 (Eq. (5)) is alpha times (1 - BEP), so one fidelity component is by construction proportional to the inverse compression ratio. None of these components is validated against semantic or task-level quality. The only independent task-level evidence, Fig. 7, shows no compression outperforming JPPO++ on Multi-news (QA F1 0.23 vs 0.19, ROUGE-L 0.25 vs 0.16) and on GovReport (F1 0.37 vs 0.26/0.31, ROUGE-L 0.33 vs 0.18/0.21). The abstract's unconditional claim of maintaining output quality is therefore unsupported and contradicted by the paper's own data.","section":"Section 4.2.1, Section 5.2.1, Fig. 4, Eqs. (4)-(7), Table 2"},{"comment":"The constraint f > f_th in Eq. (19d) is never assigned a numerical value, and Section 5.2.1 invokes an unexplained 'tolerating up to a 30% fidelity drop.' Without a concrete f_th, the 'acceptable loss' claim is unfalsifiable, and the optimization problem is missing a stated feasibility bound. The 30% figure is a drop in the internal composite f, not in any task-level metric, so it cannot justify the conclusion that output quality is maintained under aggressive compression.","section":"Eq. (19d), Section 5.2.1"},{"comment":"The service-time reduction numbers are mutually inconsistent. The text reports that single-round 16x compression reduces service time by 42.3% and iterative compression by 46.5% relative to no compression, while Fig. 6(b) reports that the iterative method reduces service time by 39-52% relative to the single-round baseline. If both statements were true, the iterative-vs-no-compression reduction would be roughly 65-72% (using the Fig. 6 values) or the iterative-vs-single-round reduction would be about 7% (using 46.5% versus 42.3%), not 39-52%. The abstract's separate 17% reduction figure also does not appear in the body. These inconsistencies undermine the paper's central quantitative claims.","section":"Abstract, Section 5.2.1, Fig. 6"},{"comment":"The experimental evaluation compares JPPO and JPPO++ only against no compression and one-shot compression; it does not compare against the prompt-compression or power-allocation baselines discussed in the related work, such as LLMLingua [16] or PCRL [27]. Because the paper motivates JPPO++ by the limitations of those methods, the absence of any comparison leaves the claimed practical advantage over prior approaches unestablished.","section":"Section 5.2, related work [16], [27]"}],"minor_comments":[{"comment":"The weights are labeled alpha_1, alpha_2, alpha_3 in Table 2 but phi_1, phi_2, phi_3 in Eq. (7); the notation should be made consistent.","section":"Table 2 and Eq. (7)"},{"comment":"The state vector is written as [f_n(eta_n), gamma_n, ] with an empty entry, and the BEP mentioned in the text is missing from the vector; Algorithm 1 also lists 'target privacy parameters (epsilon, delta)' that are not used in the problem.","section":"Section 4.2.1 and Algorithm 1"},{"comment":"The compression action is described as a discrete level from 0 to 4 and the power level from 0 to 9, but Tables 3-6 use rows and columns numbered 1 to 10; the mapping between action levels and table indices should be clarified.","section":"Section 4.2.1 and Tables 3-6"},{"comment":"Table 2 reports 10,000 episodes per test run, but Section 5.2.3 and Fig. 5 report reward convergence over 100 episodes; the relationship between these two numbers should be stated explicitly.","section":"Section 5.1, 5.2.3, Fig. 5"},{"comment":"Eq. (20) is the standard Q-learning update rather than the Double DQN target; the text should clarify that Eq. (22) is the actual Double DQN update used.","section":"Eq. (20) and Eq. (22)"},{"comment":"The LongBench results in Fig. 7 are reported without error bars or statistical significance; given the small differences in the Multi-news category, some measure of variance is needed to support the claim of minimal degradation.","section":"Fig. 7"}],"recommendation":"reject","confidential_remarks":"The paper is clearly written and the problem is timely, but as submitted the central quality-preservation claim is contradicted by the paper's own independent evaluation, the evaluation metric is circular with the optimization objective, and the latency-reduction figures are internally inconsistent. These are load-bearing issues that would require new experiments and a reframing of the central claim, rather than a local revision, so I recommend rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague — quick take on JPPO++ (arXiv:2412.03621). The core idea is sensible: use an on-device SLM to compress prompts before wireless transmission to a cloud LLM, and have a DRL agent choose compression ratio and transmit power jointly. That is a real gap in the literature as far as I can tell, and the denoising-inspired iterative compression schedule (compressing in stages rather than one shot) is a reasonable heuristic, even if it is not a principled new framework. The system model and problem formulation are clearly laid out, and the paper is well-written.\n\nThe trouble is the evaluation. The composite fidelity f (Eq. 7) is used both as the DRL reward and as the effectiveness metric, so high reward/fidelity values in Fig. 4 and Tables 3–6 partly just show that the agent optimizes its own objective. The components are token overlap and a BEP-scaled length ratio; none is validated against real task quality. The one independent check, Fig. 7 on LongBench, shows that both JPPO and JPPO++ are clearly worse than no compression on QA F1 and ROUGE-L (e.g., GovReport F1 drops from 0.37 to 0.26/0.31). So the abstract's unconditional \"maintaining output quality\" is not supported; at best there is a latency-quality trade-off with an unspecified threshold (f_th in Eq. 19d is never given). I also noticed an arithmetic inconsistency: Fig. 6 reports service-time reductions of 39–52% relative to the single-round baseline. Combined with that baseline's 42.3% reduction vs. no compression, that implies a 65–72% reduction vs. no compression, not the claimed 46.5%. Maybe the two numbers refer to different settings, but the paper does not say so. Finally, no code or data are released, and the latency numbers come from the model, not measurements.\n\nWho is this for? A reader working on wireless edge LLM services might get value from the problem framing and the DRL-based joint optimization setup, but the results should not be taken as evidence of quality preservation. I would not cite it for its numbers, but I might cite it as an example of the joint-compression-and-power-allocation problem.\n\nRecommendation: This deserves a serious referee — the topic is relevant and the framework is coherent — but the current version should be rejected. The authors need to decouple the reward from the evaluation metric, report external benchmarks with task-based metrics (which they partially have), give the fidelity threshold, reconcile the timing numbers, and ideally run a real testbed or at least use measured latency profiles. The paper could be resubmitted after that.","headline":"Relevant joint-optimization problem and a clear framework, but the evaluation does not back the central 'maintains quality' claim; the paper's own LongBench results show a real quality cost.","tokens_in":19460,"tokens_out":5368,"would_cite":false,"duration_ms":45957,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Jointly tuning prompt length and transmit power cuts mobile LLM latency by up to 46.5%.","keywords":["large language models","small language models","prompt engineering","power allocation","joint optimization","prompt compression","deep reinforcement learning","denoising-inspired compression"],"falsifier":"Run the same long-document tasks with no compression versus sixteen-fold iterative compression and measure answer quality with a metric independent of token overlap, such as exact-match or human ratings; if output quality falls sharply while the paper's fidelity score stays high, the central quality claim is false. Alternatively, measure the full pipeline on real mobile hardware and a real fading link: if the 17% service-time reduction at maintained quality does not reproduce end-to-end, then the delay model, not the compression method, is the weak link.","tokens_in":18305,"feed_emoji":"⚡","tokens_out":11057,"duration_ms":94981,"temperature":0.7,"pith_summary":"Mobile LLM requests are slowed twice by long prompts: uploading them over wireless links costs time, and the cloud model spends more time processing more tokens. This paper claims that both costs can be cut together by compressing the prompt at the edge with a small language model and, at the same time, choosing the transmit power with a deep reinforcement learning agent. The central move is to compress iteratively, treating the long original prompt as a noisy image and the compressed prompt as the clean image, so that four steps, each halving the prompt, preserve more information than one step that cuts the prompt to a sixteenth of its length. The paper reports that this joint design cuts total service time by 17% relative to no compression while keeping its measured output quality, and by 46.5% when the aggressive one-sixteenth compression ratio is used. A careful reader should care because prompt lengths are growing rapidly with in-context learning and document uploads, making the compression-versus-power trade-off a practical lever for low-latency AI services.","feed_headline":"Prompt plus power tuning cuts mobile LLM latency 46.5%","feed_subtitle":"A learned policy balances prompt length, transmit power, and fidelity to speed up cloud LLM responses on phones.","key_machinery":"The load-bearing object is the composite fidelity metric $f = \\phi_1 f_1 + \\phi_2 f_2 + \\phi_3 f_3$, with fixed weights $(0.4, 0.3, 0.3)$, where $f_1$ is the token overlap between original and compressed prompts, $f_2$ is the compression ratio scaled by $(1 - \\mathrm{BEP}(\\mathrm{SNR}))$, and $f_3$ is the token overlap between the LLM's answer on the compressed and uncompressed prompts. This single scalar appears both in the optimization objective and as the deep-reinforcement-learning reward, so every reported gain in service time is measured against it. The second mechanism is the denoising-inspired compression schedule $\\alpha(t) = \\psi_\\kappa^{-\\sigma(t)}$ with $\\sigma(t)$ chosen as linear, cosine, or quadratic; this identity converts one large, risky compression step into $M$ controlled smaller steps, letting the small language model preserve more information at the same final ratio.","core_discovery":"The paper's central claim is that prompt compression ratio and wireless transmit power are coupled decision variables in mobile LLM serving, and that a learned policy over both outperforms any fixed choice of either. It formalizes the objective as maximizing a composite fidelity score under energy, power, latency, and minimum-fidelity constraints, then solves it with a double deep Q-network whose reward is exactly that score. The discovery that carries the paper is the compression-ratio scheduling idea: instead of compressing a sixteen-fold target in one pass, the framework applies a schedule that removes a little at each step, and the paper finds that four iterative halving steps outperform one direct sixteenth-size compression on fidelity, reward, and total time. It also finds that the shape of the schedule matters, with cosine and quadratic schedules outperforming linear ones on long meeting transcripts, and that the best operating point sits at moderate compression and moderate power, confirming that neither lever can be tuned alone.","pith_inferences":["A natural extension the paper leaves open is to make the schedule shape and the number of iterations decision variables rather than fixed hyperparameters; its own results showing that quadratic and cosine schedules beat linear on structured documents suggest the gains would grow.","The reported quality preservation is only as strong as the token-overlap fidelity measure; a testable check would re-run the same experiments using an external semantic metric or human judgment on the compressed prompts.","The energy model relies on thermal design power and estimated GPU times, so the end-to-end latency and energy trade-off should be validated on real edge hardware before deployment claims are generalized.","The 17%-at-fixed-quality and 46.5%-at-sixteen-fold numbers come from a mix of two datasets and a simulation of the wireless link; a dedicated end-to-end prototype with live radio conditions would show how much of the gain survives outside simulation."],"forward_implications":["If the claims hold, service providers can cut mobile LLM latency by roughly 17% without any quality loss on the paper's fidelity measure, just by adding edge prompt compression and learned power control.","At aggressive compression, iterative refinement dominates one-shot compression: at sixteen-fold compression, the iterative method improves the service-time reduction from 42.3% to 46.5% while also scoring higher on fidelity and the learned reward.","Because the optimal reward occurs at moderate compression and moderate power, static heuristics that fix one of the two variables will leave latency and quality on the table.","The choice of compression schedule is a real degree of freedom: cosine and quadratic schedules outperform linear on long multi-speaker documents, so prompt structure should inform how compression steps are spaced.","The DRL policy converges and adapts to changing channel conditions in simulation, suggesting the same joint state-action design could be retargeted to new networks by swapping the reward and action definitions."],"supporting_citations":[{"why":"Supplies the small-language-model prompt-compression approach that JPPO++ adapts to the wireless setting.","marker":"[16]"},{"why":"Provides the diffusion-model denoising analogy that motivates iterative, multi-step prompt compression.","marker":"[19]"},{"why":"Supplies the GPU energy and execution-time model used to compute encoding energy in the joint objective.","marker":"[31]"},{"why":"Provides the average bit-error-probability expression used in the transmission completeness and fidelity terms.","marker":"[32]"},{"why":"Supplies the double deep Q-network update rule that the paper uses to solve the joint optimization.","marker":"[34]"},{"why":"Provides the long meeting-transcript dataset used in the main fidelity, reward, and latency experiments.","marker":"[36]"},{"why":"Provides the long-context benchmark used for the QA F1 and ROUGE-L comparisons across compression strategies.","marker":"[37]"}],"fun_headline_variants":["Iterative compression plus power tuning cuts LLM latency 46.5%","Two knobs, one policy: JPPO++ cuts mobile LLM delay 46.5%","Denoising-inspired prompt compression and power control slash service time","Adaptive prompt schedule and transmit power minimize mobile LLM latency","Over 46% faster mobile LLMs with joint prompt and power optimization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire optimization and every quality claim run through one hand-weighted fidelity score built from token overlap and transmission error rate; if that score does not track how well the model actually answers the user's question, then the reported latency savings may be accompanied by an unmeasured drop in output quality.","fun_headline_variants_meta":{"raw":{"variants":["Iterative compression plus power tuning cuts LLM latency 46.5%","Two knobs, one policy: JPPO++ cuts mobile LLM delay 46.5%","Denoising-inspired prompt compression and power control slash service time","Adaptive prompt schedule and transmit power minimize mobile LLM latency","Over 46% faster mobile LLMs with joint prompt and power optimization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000481,"raw_usage":{"total_tokens":2409,"prompt_tokens":1006,"completion_tokens":1403,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":622,"completion_tokens_details":{"reasoning_tokens":1303}},"tokens_in":622,"tokens_out":1403,"duration_ms":11106,"temperature":1.0,"reasoning_tokens":1303,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T22:26:09.810610+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same long-document tasks with no compression versus sixteen-fold iterative compression and measure answer quality with a metric independent of token overlap, such as exact-match or human ratings; if output quality falls sharply while the paper's fidelity score stays high, the central quality claim is false. Alternatively, measure the full pipeline on real mobile hardware and a real fading link: if the 17% service-time reduction at maintained quality does not reproduce end-to-end, then the delay model, not the compression method, is the weak link.","supporting_citations":[{"cited_title":"LLMCarbon: Modeling the end-to-end carbon footprint of large language models,","cited_arxiv_id":null,"evidence_quote":"Supplies the GPU energy and execution-time model used to compute encoding energy in the joint objective."},{"cited_title":"Tse and P","cited_arxiv_id":null,"evidence_quote":"Provides the average bit-error-probability expression used in the transmission completeness and fidelity terms."},{"cited_title":"Deep reinforcement learn- ing with double q-learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the double deep Q-network update rule that the paper uses to solve the joint optimization."},{"cited_title":"Meetingbank: A benchmark dataset for meeting summarization,","cited_arxiv_id":null,"evidence_quote":"Provides the long meeting-transcript dataset used in the main fidelity, reward, and latency experiments."}],"review_version":1}