{"id":"79d6ce23-2a3f-4c95-b5a4-0db2b901d2e5","arxiv_id":"2502.02287","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A prompt-tuned LLM with external score feedback solves OFDMA and MU-MIMO resource allocation under QoS and queue constraints, and switches objectives without retraining.","lead":"LLM-RAO uses a large language model with iterative, score-based prompting to allocate spectrum, users, and MIMO modes in an IEEE 802.11ax network while respecting data-rate and queue constraints. In simulations it reports large gains over reinforcement learning and round-robin baselines, and adapts to new objectives by changing only the prompt.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No same-budget non-LLM black-box baseline (random search / hill climbing) with the same external score feedback is tested, so the reported gains cannot be attributed to LLM reasoning.","rationale":"The reader's weakest assumption is exactly the load-bearing concern identified in this stress test: the paper lacks a non-LLM search baseline with the same external score feedback and evaluation budget. The full text confirms that LLM-RAO relies on an OPRO-style closed-loop where the LLM sees exact scores and constraint violations from an external toolkit. Without a matched black-box optimizer as control, the observed performance could be due to the iterative search procedure rather than any distinctive LLM capability. This is not an internal inconsistency, but it is a missing control that directly affects the strength of the central claim. The paper does provide a GA comparator, but that GA is run for days as an approximate global upper bound, so it is not a fair cost-matched baseline, and the absence of error bars or seed variation further weakens quantitative comparisons. The proposed concrete test—replacing the LLM with a simple same-budget search algorithm while keeping the oracle and evaluation protocol identical—would settle whether the LLM is load-bearing. Because the reader already reached CONDITIONAL on essentially this basis, the stress test does not move the verdict; it reinforces it. Therefore the appropriate recommendation is UNCHANGED: the paper remains a conditional acceptance candidate, pending the matched-baseline control and statistical reporting.","tokens_in":7858,"tokens_out":2401,"duration_ms":24129,"concrete_test":"Re-run the four scenario experiments with a control optimizer that is not an LLM—e.g., repeated random feasible-solution sampling, hill climbing with random restarts, or simulated annealing—using the exact same meta-prompt information (objective, constraints, user data), the same external MATLAB/Python score oracle, and the same number of candidate evaluations as LLM-RAO. If no LLM-RAO budget is disclosed, fix a common budget such as 100, 500, or 1000 evaluations. Compare median and distribution over at least 20 independent seeds. Also report LLM-RAO's variance across seeds and inference temperature. If a non-LLM control matches or exceeds LLM-RAO in any scenario, the attribution of the gains to LLM reasoning is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III-C describes LLM-RAO as an iterative optimizer: the LLM proposes solutions, an external toolkit returns the exact objective score and constraint violations, and the history of <solution, score> pairs is fed back for in-context refinement. With that protocol, the intrinsic contribution of the LLM is entangled with the closed-loop search itself. The baseline set in Section IV (Round-Robin, DRL, GA) does not include any non-LLM optimizer that receives the same score feedback over the same number of function evaluations. The GA comparator is explicitly run \"over several days\" as an upper bound, so it is not a matched-cost control, and no iteration count or LLM-call budget is reported for LLM-RAO. Figures 4 and 5 show single curves without error bars or seeds. Consequently, the central claim—that an LLM's reasoning, rather than a generic feedback-driven search, produces the 40%, 80%, and 2.9x gains—is untested. The same loop with random search, hill climbing, or simulated annealing could plausibly match or exceed LLM-RAO on these discrete RA problems.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LLM-RAO, an LLM-based iterative optimizer for uplink resource allocation in IEEE 802.11ax networks. The method wraps an off-the-shelf LLM in an OPRO-style closed loop: an external toolkit evaluates each proposed allocation and returns the exact objective score and constraint violations, and the LLM refines subsequent proposals from the accumulated history. The authors formulate a mixed-integer NP-hard problem with OFDMA RB allocation, SU/MU-MIMO mode selection, queue limits, and QoS constraints, and report simulations over four scenarios with different utility functions, queue models, user counts, and rate constraints. They claim up to 40% improvement over a DRL baseline, up to 80% over round-robin, up to 2.9x over a fixed DRL model under changing environments, and performance 'comparable to GA' as a near-optimal upper bound.","tokens_in":8058,"tokens_out":2125,"duration_ms":22272,"significance":"If the central claim is correct, the paper would provide a meaningful demonstration that a single prompt-tuned LLM can serve as a general constrained optimizer for discrete, QoS-constrained wireless resource allocation problems, adapting to new objectives without retraining. The problem formulation is detailed and the use of exact external score feedback inside an iterative prompting loop is a sensible and potentially useful architecture. The main value is in the adaptability claim: changing the meta-prompt rather than retraining a network is an attractive property for dynamic environments. However, the paper currently does not provide enough experimental control or statistical detail to establish that the reported gains come from the LLM's reasoning rather than from the generic closed-loop search procedure itself.","major_comments":[{"comment":"The baseline set is missing a non-LLM black-box optimizer that receives the same external score feedback and is evaluated under the same number of function evaluations. The reported improvements over DRL and round-robin cannot be attributed to the LLM's reasoning unless LLM-RAO is compared against random search, hill climbing, or simulated annealing driven by the same exact score oracle and the same iteration budget. As written, the central claim—that LLM reasoning, rather than the closed-loop search itself, produces the 40%, 80%, and 2.9x gains—is unsupported.","section":"Section IV"},{"comment":"All reported results are single performance curves with no error bars, no number of seeds, and no variance information. The GA baseline is described as run 'over several days' to obtain a 'naive upper bound,' but no iteration count or wall-clock budget is reported for LLM-RAO, so the 'near-optimal' claim is not accompanied by a quantified optimality gap or a matched evaluation budget. Without these, the numerical claims are not statistically grounded.","section":"Section IV, Figs. 4 and 5"},{"comment":"The inference process is described qualitatively, but key implementation parameters are omitted: the maximum number of optimization iterations, the history sampling size, the LLM sampling temperature, and the exact meta-prompt template. The paper also states in Section III-A that optimization stops when the LLM 'can no longer propose solutions with improved optimization scores or when the maximum number of optimization steps has been reached,' but Section III-C does not specify how this stopping condition is implemented. These details are needed to reproduce the method and to interpret the results as a fair comparison with any search baseline.","section":"Section III-C, Fig. 3"},{"comment":"The adaptability experiment compares LLM-RAO against DRL models whose weights are fixed to those trained for Scenario 1 only. Since Section IV-A shows that per-scenario retrained DRL models perform much better, the 2.9x improvement over a deliberately mismatched fixed DRL model is an expected consequence of the experimental protocol rather than a direct measure of LLM-RAO's adaptability. A stronger comparison would report the gap between LLM-RAO and the per-scenario retrained DRL upper bounds alongside the fixed-model results.","section":"Section IV-B, Fig. 5"}],"minor_comments":[{"comment":"The percentages '40%,' '80%,' and '2.9 times' are stated without specifying against which scenario or metric they are measured; the abstract should either name the scenarios or defer the precise numbers to Section IV.","section":"Abstract"},{"comment":"The QoS constraint R(k) is used in the formulation, but its units (Mbps) are only implied by the simulation setting in Table I. Please state the units explicitly in the problem formulation.","section":"Section II-C, Eq. (2c)"},{"comment":"The version and access date of 'ChatGPT 3.5 Turbo' are not specified. Since LLM behavior changes with model version, this information is important for reproducibility.","section":"Section IV"},{"comment":"The phrase 'delivers the final optimal solution' is too strong for an iterative heuristic; please use 'final solution' or 'best found solution.'","section":"Section III-A"},{"comment":"The paper does not state the number of simulation runs or the random seed policy used to generate user positions and channel realizations. Adding this information and reporting mean and standard deviation would substantially improve the paper.","section":"Section IV"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the journal and addresses a timely topic, but the experimental methodology currently falls short of supporting the headline claims. The missing same-budget non-LLM search baseline is the most serious issue; without it the paper cannot rule out that the gains are due to the closed-loop iterative search rather than the LLM. I would also encourage the editor to require the authors to provide code or at least detailed hyperparameters, since the current description makes reproduction difficult."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper does something genuinely new—applying OPRO-style LLM optimization to a realistically complex joint OFDMA/MU-MIMO resource allocation problem with QoS and queueing constraints. The problem formulation is harder than the simple power-allocation tasks in earlier LLM work, and the adaptation experiments across four scenario configurations are a useful step. But the evaluation gives you no way to attribute the reported gains to the LLM's reasoning rather than to the iterative score-feedback loop itself.\n\nWhat's actually new: the joint problem with finite backlogs, minimum-rate constraints, and changing objectives is a real mixed-integer optimization task, and applying OPRO to it with dynamic scenario transitions is not in the cited prior work. The prompt-based handling of shifting objectives is timely and well motivated.\n\nWhat's done well: constraints are laid out explicitly in (1a)-(1c) and (2c); the four scenario axes (utility, queue, K, R_min) cover the right dimensions of adaptivity; and the GA is honestly described as a days-long upper bound rather than a matched-cost competitor. The authors also acknowledge the hallucination problem and constrain the solution space in the meta-prompt.\n\nWhere it's soft: the stress-test note is right. The only non-LLM optimizers are round-robin, a task-specific DRL, and GA. There is no random search, hill-climbing, or simulated-annealing baseline receiving the same score and constraint feedback over the same number of function evaluations. Without that control, the 40%, 80%, and 2.9x claims are evidence about the closed-loop protocol, not about LLM reasoning. Figures 4 and 5 are single curves with no error bars, the LLM-call budget is never stated, and the DRL baseline is the authors' own prior work. None of this is disqualifying as an idea, but it is under-supported as a demonstration.\n\nBottom line: for someone working on LLM-based network optimization, this is a useful paper with a promising setup and clear limitations. A serious referee could require the missing baseline, error bars, and code. Send it to review; the empirical claim needs fixing, but the core idea deserves scrutiny.","headline":"An OPRO-style LLM optimizer on a realistic 802.11ax resource allocation problem, but the evaluation lacks the non-LLM search control that would make the LLM's contribution measurable.","tokens_in":8598,"tokens_out":2306,"would_cite":false,"duration_ms":22230,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a pre-trained LLM, steered only by prompt-based iterative feedback, can solve a constrained NP-hard wireless resource allocation problem and adapt to changing objectives without retraining, outperforming deep…","keywords":["large language model","resource allocation","task-oriented communication","IEEE 802.11ax","constrained optimization","prompt-based tuning","dynamic wireless environments","deep reinforcement learning"],"falsifier":"Run the closed-loop evaluation on the paper's four scenarios with a simple optimizer such as random search or hill climbing that consumes the same number of candidate evaluations and receives the same external scores; if the simple optimizer matches or exceeds LLM-RAO's objective values, the specific claim that LLM reasoning drives the gains is refuted.","tokens_in":7642,"feed_emoji":"📶","tokens_out":5817,"duration_ms":52591,"temperature":0.7,"pith_summary":"This paper tries to establish that a large language model can act as a general optimizer for constrained wireless resource allocation, a problem class that usually requires specially designed deep learning architectures or heuristic search. It introduces LLM-RAO, which wraps a pretrained LLM in a closed loop: a meta-prompt describes the objective, constraints, users, and channel state; the LLM proposes resource allocations; an external simulator scores them and reports constraint violations; and the score history is fed back so the LLM refines its proposals. Across four scenarios with different objectives, queue lengths, user counts, and QoS constraints, the method claims up to 40% improvement over a deep reinforcement learning baseline, up to 80% over a round-robin analytical scheme, and performance comparable to a genetic algorithm run over several days. Its adaptation claim is that switching scenarios only requires editing the prompt, avoiding the retraining that fixed deep learning models need. The matter is worth caring about because a single flexible optimizer could replace many task-specific models in dynamic networks.","feed_headline":"LLM resource optimizer beats DRL by 40%, nears genetic upper bound","feed_subtitle":"A prompt-tuned LLM adapts to shifting wireless goals without retraining, matching a days-long search.","key_machinery":"The engine is OPRO (Optimization by PROmpting), a closed-loop prompting method. A meta-prompt built by the access point encodes user information (channels, backlogged packets, minimum-rate requirements) and the task description (utility function, constraints, variable ranges such as user indices). The LLM outputs candidate binary allocation matrices; a MATLAB/Python external toolkit evaluates them against the true objective and constraints, returning scores and violation feedback; those <solution, score> pairs are appended to the prompt history, and the LLM uses in-context learning to propose better allocations in the next round. The same mechanism delivers adaptability because retuning the system means rewriting the meta-prompt, not updating network weights.","core_discovery":"The central claim is that prompt-based tuning alone is enough to turn an off-the-shelf LLM into a competitive solver for a mixed-integer NP-hard allocation problem: joint OFDMA resource-block assignment, SU-/MU-MIMO mode selection, per-user minimum-rate QoS, and finite backlogged queues. The authors argue that the LLM's pretrained world knowledge lets it infer data rates and structural patterns from channel and queue data without hand-derived formulas, and the closed-loop score feedback lets it learn constraints in context. In simulations on IEEE 802.11ax uplink with four antennas and twenty megahertz of bandwidth, LLM-RAO is reported to outperform a deliberately tailored deep reinforcement learning baseline by up to 40% and round-robin by up to 80%, while matching the genetic algorithm, whose several-day runtime is treated as a near-optimal upper bound. In the changing-environment test, the same LLM, with only prompt modifications, reaches up to 2.9 times the performance of DRL models trained on one scenario and left fixed.","pith_inferences":["The paper does not include a control where a non-LLM method such as random search or hill climbing receives the same score feedback and the same evaluation budget; until that control is run, part of the reported gain could come from the iterative search loop rather than from the LLM's reasoning.","The same closed-loop structure should transfer to other discrete network optimizations such as beam selection, routing, or channel assignment by swapping the external evaluator, assuming the prompt-engineering effort scales similarly.","The paper does not report per-iteration latency or API cost, so a practical deployment would still need to show that the LLM loop can finish within a scheduling interval; the reported experiments do not establish real-time feasibility.","The no-retraining advantage may depend on the base LLM's pretrained coverage; for radio configurations far outside its training distribution, in-context adaptation could degrade, so generality across radio technologies is an untested extrapolation."],"forward_implications":["If the results hold, an operator can switch between sum-rate maximization, proportional fairness, finite-queue, and minimum-rate scenarios by editing the meta-prompt, so a single LLM instance covers multiple environments without retraining.","The near-genetic-algorithm performance in all four tested scenarios implies that prompt-guided LLM search can rival a heuristic search that requires days of computation, at least at the tested problem sizes.","On the paper's evidence, DRL models trained per scenario are outperformed by up to 40% in objective value, and by up to 2.9 times when the environment shifts away from the training scenario.","The proposed method handles discrete allocation variables and QoS constraints, which the paper identifies as a known weakness of conventional deep learning approaches.","The adaptive advantage is delivered purely through prompt modification with no weight updates, which would remove the retraining bottleneck in dynamic radio access networks."],"supporting_citations":[{"why":"Supplies the OPRO closed-loop prompting method that LLM-RAO is built on.","marker":"[10]"},{"why":"Provides the deep hierarchical reinforcement learning baseline that LLM-RAO reports beating by up to 40%.","marker":"[12]"},{"why":"Provides the genetic algorithm whose several-day run serves as the near-optimal upper-bound comparison.","marker":"[13]"},{"why":"Supplies the IEEE 802.11 channel model used in the MATLAB simulator for all evaluations.","marker":"[11]"},{"why":"Defines the 802.11ax resource-block structure and MU-MIMO constraints that shape the optimization problem.","marker":"[9]"}],"fun_headline_variants":["LLM resource optimizer beats RL by 40%, matches genetic search","Prompt-tuned LLM beats deep RL by 40% in wireless allocation","LLM adapts to shifting wireless goals via prompts, no retraining","Prompt-tuned LLM matches genetic upper bound for wireless allocation","LLM resource optimizer: 40% better than DRL, 2.9x in dynamic scenarios"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The results are taken as showing the LLM's reasoning ability, but the paper never compares against a trivial search baseline, such as random search or hill climbing, that receives the same score feedback and evaluation budget, so the margin attributed to the LLM could instead come from the iterative search loop itself.","fun_headline_variants_meta":{"raw":{"variants":["LLM resource optimizer beats RL by 40%, matches genetic search","Prompt-tuned LLM beats deep RL by 40% in wireless allocation","LLM adapts to shifting wireless goals via prompts, no retraining","Prompt-tuned LLM matches genetic upper bound for wireless allocation","LLM resource optimizer: 40% better than DRL, 2.9x in dynamic scenarios"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000652,"raw_usage":{"total_tokens":3021,"prompt_tokens":1007,"completion_tokens":2014,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":623,"completion_tokens_details":{"reasoning_tokens":1913}},"tokens_in":623,"tokens_out":2014,"duration_ms":12389,"temperature":1.0,"reasoning_tokens":1913,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T12:39:11.136489+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the closed-loop evaluation on the paper's four scenarios with a simple optimizer such as random search or hill climbing that consumes the same number of candidate evaluations and receives the same external scores; if the simple optimizer matches or exceeds LLM-RAO's objective values, the specific claim that LLM reasoning drives the gains is refuted.","supporting_citations":[{"cited_title":"Large language models as optimizers,","cited_arxiv_id":null,"evidence_quote":"Supplies the OPRO closed-loop prompting method that LLM-RAO is built on."},{"cited_title":"Joint optimization on uplink OFDMA and MU-MIMO for IEEE 802.11ax: Deep hierarchical reinforcement learning approach,","cited_arxiv_id":null,"evidence_quote":"Provides the deep hierarchical reinforcement learning baseline that LLM-RAO reports beating by up to 40%."},{"cited_title":"Alexander, An introduction to genetic algorithms for scientists and engineers","cited_arxiv_id":null,"evidence_quote":"Provides the genetic algorithm whose several-day run serves as the near-optimal upper-bound comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the IEEE 802.11 channel model used in the MATLAB simulator for all evaluations."},{"cited_title":"Wireless LAN medium access control (MAC) and physical layer (PHY) specifications,","cited_arxiv_id":null,"evidence_quote":"Defines the 802.11ax resource-block structure and MU-MIMO constraints that shape the optimization problem."}],"review_version":1}