{"id":"798bb350-1d0d-45e6-a331-7bee549c0b50","arxiv_id":"2506.00574","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Prompt-tuned ORANSight state representations improve convergence and slice-level QoS for multi-agent SAC in a simulated O-RAN slicing environment, according to the reported ablation.","lead":"This paper combines a domain-specific language model with trainable prompt tokens to build state representations for multi-agent reinforcement learning in O-RAN network slicing, reporting faster convergence and higher QoS than its baselines. A generalist might read it as a test of whether lightweight prompt tuning can make LLM-augmented DRL practical in real-time radio access networks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline comparison is not statistically grounded: Fig. 3 shows single reward curves with no error bars, and the learnable-prompt count in Fig. 4 is selected on the evaluation curve, so 'consistently outperforms' is unverified.","rationale":"The paper's own text flags the token-count tuning: Section V-C calls the number of context tokens 'a tunable hyperparameter' and highlights the 'dominant maximum' in Fig. 4, which is a selection on the evaluation curve. This is a self-referential limitation that must be weighed. The reader's Fc2 concern is real but secondary: even if Fc2 is well-matched, the lack of error bars means the observed gap could be seed noise; if Fc2 is mismatched, the gap's interpretation changes. Both issues point to the same need for controlled, repeated evaluation. A multi-seed rerun with a pre-registered token count directly tests the central empirical claim and would also reveal whether the reported gains survive when the token count is not tuned on the test metric. I therefore agree with the reader's CONDITIONAL verdict, while disagreeing that the Fc2 distribution shift is the single most load-bearing assumption; the statistical grounding of the comparison is more fundamental.","tokens_in":9550,"tokens_out":8975,"duration_ms":87905,"concrete_test":"Run ORANSight PA-MRL, GPT PA-MRL, and ORANSight MARL for 10 independent seeds each, using a fixed prompt-token count chosen on a validation split rather than on the test curve, and report mean ± standard deviation of cumulative reward over episodes. If the PA-MRL advantage does not exceed the inter-seed spread, the claim of consistent outperformance fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that ORANSight PA-MRL 'consistently outperforms' the two baselines requires that the reward gap in Fig. 3 be real and attributable to the method. The paper provides no seed count, error bars, or significance test; a single MARL run can be dominated by seed noise. Section V-C compounds this by tuning the number of learnable prompt tokens on the evaluation curve and highlighting the 'dominant maximum' in Fig. 4, a form of test-set selection that inflates the reported gain. The frozen adapter Fc2 (Sec. IV-C), trained offline on unprompted [9] representations, introduces a further confound: if h_t = M(p_t ∪ T) lies outside Fc2's training distribution, the improvement could be an artifact of distribution shift rather than prompt-based learning. None of these confounds is controlled, so the empirical evidence does not yet establish the claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript proposes PA-MRL, a prompt-tuned LLM-augmented multi-agent reinforcement learning framework for dynamic O-RAN network slicing. The method uses ORANSight, a domain-specific LLM, together with informal and learnable prompts, and two offline-trained adapter networks Fc1 and Fc2, to fuse raw network state and LLM-generated representations into a unified state for SAC-based MARL agents. The paper reports simulation results over three slices (eMBB, mMTC, URLLC) and six DUs, claiming faster convergence and higher cumulative reward than a GPT-2-based variant and the ORANSight MARL baseline, plus an ablation table and per-user throughput CDFs.","tokens_in":9707,"tokens_out":4455,"duration_ms":40876,"significance":"If the empirical claims were substantiated, the paper would make a useful incremental contribution: parameter-efficient, in-loop adaptation of a domain-specific LLM via learnable prompts for wireless resource allocation, avoiding full fine-tuning. The paper is honest in building on prior work [9] for the simulator, prompt construction, and adapter networks, and it provides an ablation comparing domain-specific vs generic LLMs and prompt vs no-prompt variants. However, the reported evidence is currently too thin to establish the central claim: there are no error bars, seed counts, significance tests, released code, or validation of the frozen adapter under the new prompted distribution. The contribution is promising but needs substantially stronger empirical support.","major_comments":[{"comment":"The central claim that ORANSight PA-MRL \"consistently outperforms\" the other approaches is supported only by a single pair of learning curves with no error bars, seed count, or statistical test. A single MARL run can be dominated by seed noise, especially with stochastic actor-critic training, so the reported gap cannot be distinguished from variance. Please report results over multiple seeds with means and confidence intervals, and ideally a significance test, for cumulative reward and convergence episodes.","section":"V-B, Fig. 3"},{"comment":"The number of learnable prompt tokens is selected on the evaluation curve: the paragraph states that this value \"should be treated as a tunable hyperparameter\" and then highlights the \"dominant maximum\" in Fig. 4. Tuning a hyperparameter on the same data used to report final performance inflates the measured gain. Use a separate validation set or report the full sensitivity curve with a clearly defined training/evaluation split.","section":"V-C, Fig. 4"},{"comment":"Fc2 is trained offline on LLM-generated representations from prior work [9], but Algorithm 1 feeds it h_t = M(p_t ∪ T), i.e., representations produced with the new learnable prompts. The paper provides no evidence that Fc2 remains valid under this distribution shift; if prompted representations fall outside its training distribution, the observed improvement could be an artifact of the mismatch rather than evidence for prompt-based learning. Please retrain or fine-tune the adapters on prompted outputs, or provide an analysis showing that Fc2's embedding space is unaffected by the added prompt tokens.","section":"IV-C, Algorithm 1"},{"comment":"Constraint (1d) allows relaxed allocations that may assign more than one slice to the same RB, and the text asserts that \"final allocations enforce one-slice-per-RB assignments\" without specifying the projection or rounding procedure used. Since this procedure determines feasibility and reward, please state it explicitly or include pseudocode; otherwise the optimization problem solved by the agents is not fully defined.","section":"III-C, Eq. (1d)"},{"comment":"Table I reports QoS improvements and convergence gains relative to a \"plain MARL baseline,\" but that baseline is not defined, is absent from Fig. 3, and the raw values behind the percentages are not given. This makes the ablation difficult to interpret: for example, a gain of 23.66% in URLLC QoS needs the baseline magnitude to be meaningful. Please define the baseline, show its learning curves, and report absolute metric values with variance.","section":"V-D, Table I"}],"minor_comments":[{"comment":"The x-axis of Fig. 3 has no label or units; the text mentions \"after 1000 episodes,\" but the axis appears to end at 30, which is confusing. Please label the axis clearly as training episodes (or thousands of episodes).","section":"V-A, Fig. 3"},{"comment":"The section title \"PROMP-AUGMENTEDMULTIAGENTRL\" has a typo and missing spacing; it should be \"PROMPT-AUGMENTED MULTI-AGENT RL.\"","section":"IV header"},{"comment":"Notation is inconsistent: constraint (1d) uses lambda_l, but the text later refers to a \"penalty term lambda_k\"; please unify the notation.","section":"III-C and IV-C"},{"comment":"The section title \"Number of Context\" should be \"Number of Context Tokens\" for clarity.","section":"V-C"},{"comment":"The components \"LLMSR Dataset\" and the cosine-similarity module in Fig. 2 are not explained in the text; please clarify how the dataset and similarity loss are used.","section":"Fig. 2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript depends heavily on the authors' prior conference paper [9] for the simulator, adapter networks, prompt construction, and the main baseline; the novelty is the learnable-prompt variant. This dependence is acceptable if the comparative evidence is robust, but the current single-seed experiments and selection of the prompt-token count on the evaluation curve substantially weaken the central claim. I would not recommend rejection if the authors supply multi-seed results, a proper validation split, and adapter validation under the prompted distribution; the contribution is incremental but potentially publishable in a venue that values empirical systems papers."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You asked what I think of arXiv:2506.00574. Short version: the idea is sensible, the writing is clear, and the authors are honest about building on their own prior work. But the evaluation is not yet at the level that would make me trust the headline results.\n\nWhat is actually new: learnable prompt tokens added to the ORANSight state representation module, trained jointly with the MARL policy, instead of the informal prompts from their ICC 2025 paper. That is a legitimate extension, and the framework description (Algorithm 1, the SRM in Fig. 2) is readable. They also cite the relevant prompt-tuning and LLM-augmented RL literature. For someone in the O-RAN/LLM space, this is a plausible way to make LLM-based state representations adaptive without fine-tuning the LLM.\n\nSoft spots, in proportion: the biggest one is statistical. Figure 3 shows three single reward curves with no error bars, no seed count, no significance test. The plain MARL baseline is described in the text but absent from the figure, so the \"consistently outperforms\" claim is really an observation about one run of each method. Figure 4 tunes the number of context tokens and highlights the peak on the evaluation curve, which inflates the reported gain. Then there is the frozen adapter issue: Fc2 was trained offline on representations from [9], and here it receives outputs from the LLM with new learnable prompts appended. If the prompt distribution shifts, any measured improvement could be an artifact of the mismatch rather than evidence for prompt-based learning. The paper does not validate or retrain Fc2 on prompted outputs. Minor but worth noting: the paper says the prompts optimize a semantic clustering objective, but no such loss term appears in the algorithm.\n\nThese are not fatal to the concept. The central idea could still hold. But the evidence as presented is preliminary. The paper does not ship code, the simulator is external, and every quantitative claim comes from a single experiment setup.\n\nWho this is for: researchers working on LLM-empowered RL for wireless resource management. A serious referee could usefully demand more runs, the missing baseline, and a discussion of the adapter distribution shift. That is worth a review slot, not a desk reject.\n\nRecommendation: send it to peer review, but prepare to require major revisions on the empirical side before any acceptance.","headline":"A clear, incremental extension of the authors' own ORANSight MARL work, but the experimental evidence is too thin to support the convergence and QoS claims.","tokens_in":10331,"tokens_out":1457,"would_cite":false,"duration_ms":15969,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that adding a few learnable prompt tokens to a frozen O-RAN-specific LLM's state representation lets multi-agent RL slice network resources with faster convergence and better QoS than un-prompted or generic-LLM baselines.","keywords":["O-RAN","network slicing","multi-agent reinforcement learning","large language models","prompt tuning","Soft Actor-Critic","state representation","resource allocation"],"falsifier":"Retrain $F_{c2}$ on prompted LLM outputs and rerun the comparison, or replace the learned prompts with random frozen token embeddings while keeping everything else identical; if the convergence and QoS gaps disappear or do not shrink, the claim that learnable prompts drive the gain is falsified.","tokens_in":9233,"feed_emoji":"📶","tokens_out":12988,"duration_ms":105066,"temperature":0.7,"pith_summary":"O-RAN slicing must divide radio resources among three service classes—high-throughput eMBB, massive mMTC, and low-latency URLLC—while traffic and channel conditions keep changing. This paper claims that a multi-agent Soft Actor-Critic system can learn such slicing policies much faster if its state representations are produced by a frozen O-RAN-specific language model with a few learnable prompt tokens added to each informal text prompt. Those tokens are updated by the RL gradient, so the LLM's semantic clustering of related network metrics (throughput, latency, user density, power) is aligned with the agent's reward instead of being fixed in advance. In a simulated three-slice, six-DU O-RAN environment, the prompt-aligned variant converges sooner, reaches a higher final cumulative reward, and improves slice-level QoS relative to the same LLM without prompts and to a generic language model. The motivation is that this gives real-time adaptability without the cost of fine-tuning a large model.","feed_headline":"Learnable prompts speed up LLM-driven O-RAN slicing","feed_subtitle":"In a simulated three-slice network, prompt-aligned agents beat un-prompted and generic-LLM baselines.","key_machinery":"The load-bearing component is the prompt-augmented State Representation Module (SRM) embedded in the actor and critic paths of a multi-agent Soft Actor-Critic setup. The SRM concatenates two adapters' outputs: $F_{c1}$ on the raw metrics $s_t$ and $F_{c2}$ on $h_t = M(p_t \\cup T)$, where $M$ is ORANSight, a language model pre-trained on O-RAN knowledge. The learnable prompt tokens $T$ are the only part of the LLM pipeline updated by RL gradients, so the frozen model can still produce task-tuned semantic representations; the paper argues the LLM naturally clusters related concepts such as SNR, power, and throughput, making the state easier for agents to interpret.","core_discovery":"The central claim is that prompt alignment, not merely LLM augmentation, is what makes LLM-based state representations useful for dynamic slicing. In PA-MRL, the State Representation Module builds two views of the state: raw network metrics passed through $F_{c1}$, and LLM embeddings passed through $F_{c2}$; the LLM input is the informal prompt $p_t$ concatenated with learnable embeddings $T$, yielding $h_t = M(p_t \\cup T)$. The learnable prompts are optimized with the SAC policy gradient, so the representation is pulled toward both semantic structure and RL reward. The paper evaluates this against ORANSight MARL without prompt alignment and GPT PA-MRL with a generic LLM, and reports that the proposed ORANSight PA-MRL achieves the best convergence, final reward, and per-slice QoS, with the largest improvements in the eMBB and mMTC slices.","pith_inferences":["The authors do not report wall-clock training cost or parameter counts, so their lightweight claim is about convergence and architecture, not measured compute; a direct comparison with LoRA or full fine-tuning under the same compute budget would make the efficiency claim testable.","The authors do not retrain or validate $F_{c1}$ and $F_{c2}$ on prompted outputs; since those adapters were trained offline on un-prompted representations, prompted outputs could be out of distribution, and retraining them or measuring representation shift would separate prompt-driven gains from adapter mismatch.","If the learned prompts encode general semantic structure rather than scenario-specific values, the same prompt embeddings could transfer to new slice configurations or traffic mixes; the paper does not test transfer, but that is a natural next step.","The same contextualization mechanism, freezing a domain LLM and tuning small prompts with the RL objective, could apply to other resource-allocation MDPs with mixed numerical and semantic observations, such as spectrum sharing or edge-computing scheduling."],"forward_implications":["Operators could adapt slicing policies by tuning only the prompt embeddings, leaving the large language model frozen, which is far cheaper than full fine-tuning in a real-time RIC.","Applying the same prompt-aligned state representation at both DU actors and the centralized critic lets coordination use semantically structured state without centralizing the policies.","The number of context tokens behaves as a tunable hyperparameter: too few underfit and too many risk overfitting, so deployments would need a short token-count search.","Replacing the domain-specific ORANSight LLM with GPT-2 in the same pipeline reduces the gain, so both prompt alignment and domain specialization contribute to the reported improvement."],"supporting_citations":[{"why":"Supplies the simulator, mobility model, informal-prompt construction, offline-trained adapters $F_{c1}$/$F_{c2}$, and the ORANSight MARL baseline that PA-MRL extends.","marker":"[9]"},{"why":"ORANSight, the domain-specific LLM pre-trained on O-RAN knowledge whose frozen representations PA-MRL prompts.","marker":"[15]"},{"why":"The O-RAN benchmark used in building and evaluating ORANSight, grounding the claim that the LLM has domain knowledge.","marker":"[16]"},{"why":"Introduces learnable prompt tuning as a parameter-efficient adaptation technique, the method PA-MRL adopts.","marker":"[17]"},{"why":"Prefix-tuning, the continuous-prompt optimization approach that supports the design of learnable prompt embeddings.","marker":"[18]"},{"why":"Soft Actor-Critic, the entropy-regularized off-policy algorithm used for the multi-agent actors and critic.","marker":"[19]"},{"why":"LoRA, a parameter-efficient fine-tuning alternative that motivates the choice of lightweight adaptation over full fine-tuning.","marker":"[13]"}],"fun_headline_variants":["Prompt-tuned LLMs boost O-RAN slicing agility","LLM prompts accelerate O-RAN slicing convergence","Prompt alignment unlocks faster O-RAN slicing","Learnable prompts sharpen LLM state for O-RAN slicing","PA-MRL: prompt-tuned LLMs enable faster O-RAN slicing"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The adapter networks described in Section IV-C were trained offline on outputs from the earlier un-prompted system, and the paper never re-trains or validates them on the new prompted outputs; the comparison assumes those prompted embeddings stay inside the adapters' training distribution.","fun_headline_variants_meta":{"raw":{"variants":["Prompt-tuned LLMs boost O-RAN slicing agility","LLM prompts accelerate O-RAN slicing convergence","Prompt alignment unlocks faster O-RAN slicing","Learnable prompts sharpen LLM state for O-RAN slicing","PA-MRL: prompt-tuned LLMs enable faster O-RAN slicing"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000464,"raw_usage":{"total_tokens":2325,"prompt_tokens":961,"completion_tokens":1364,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":577,"completion_tokens_details":{"reasoning_tokens":1283}},"tokens_in":577,"tokens_out":1364,"duration_ms":9865,"temperature":1.0,"reasoning_tokens":1283,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:02:50.390407+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain $F_{c2}$ on prompted LLM outputs and rerun the comparison, or replace the learned prompts with random frozen token embeddings while keeping everything else identical; if the convergence and QoS gaps disappear or do not shrink, the claim that learnable prompts drive the gain is falsified.","supporting_citations":[{"cited_title":"Llm-augmented deep reinforcement learning for dynamic o-ran network slicing,","cited_arxiv_id":null,"evidence_quote":"Supplies the simulator, mobility model, informal-prompt construction, offline-trained adapters $F_{c1}$/$F_{c2}$, and the ORANSight MARL baseline that PA-MRL extends."},{"cited_title":"Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,","cited_arxiv_id":null,"evidence_quote":"Soft Actor-Critic, the entropy-regularized off-policy algorithm used for the multi-agent actors and critic."},{"cited_title":"Lora: Low-rank adaptation of large language models.,","cited_arxiv_id":null,"evidence_quote":"LoRA, a parameter-efficient fine-tuning alternative that motivates the choice of lightweight adaptation over full fine-tuning."}],"review_version":1}