EvoRIC: Reinforcement Learning Fine-Tuned LLM-empowered RAN Intelligent Control Toward Autonomous O-RAN

Jemin Lee; Lingyan Bao; Tony Q.S. Quek

REVIEW 5 major objections 5 minor 15 references

EvoRIC: Reinforcement Learning Fine-Tuned LLM-empowered RAN Intelligent Control Toward Autonomous O-RAN

T0 review · 5 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A fine-tuned 3B LLM can outperform optimization solvers and far larger models for RAN power allocation.

desk verdict EvoRIC is a well-structured framework paper whose headline generalization claim is unverified because the paper never specifies the PPO training topologies; the quantitative results are conditional. read the letter →

arxiv 2608.06789 v1 pith:AE2FJZOF submitted 2026-08-07 cs.NI

Lingyan Bao , Jemin Lee , Tony Q.S. Quek This is my paper · ORCID

classification cs.NI

keywords openradioaccessnetworkRANintelligentcontrollerlargelanguagemodelreinforcementlearningfine-tuningproximalpolicyoptimizationintegratedandbackhaultopologygeneralizationnear-real-timeinference

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes EvoRIC, a hierarchical O-RAN control architecture in which a compact language model acts as the reinforcement-learning policy and is fine-tuned in the non-real-time RIC, then deployed to near-real-time RICs for fast inference. The claim is that this collect-update-deploy loop gives the LLM wireless-domain decision-making ability without expert-labeled data. Evaluated on integrated access and backhaul power allocation, the fine-tuned 3B model is reported to outperform equal-power allocation, the model-based SCA algorithm, and much larger general-purpose LLMs, while completing inference in 0.56 seconds. If correct, this would make LLM-driven, edge-deployable RAN control practical and reduce the need for topology-specific retraining.

What carries the argument

The machinery is an LLM as the actor inside a PPO reinforcement-learning agent, regularized by a frozen reference model through a KL-divergence penalty. The non-RT RIC aggregates interaction tuples (state prompt, generated action, reward) via the O1 interface, runs PPO updates on the last L transformer blocks, and redeploys via A1. The near-RT RIC serializes network state into a prompt with explicit output-format directives, parses the LLM's text into a power vector, verifies it against the power budget, and falls back to equal power allocation if validation fails. A composite reward adds a format penalty to the throughput objective, which keeps the model from producing unexecutable outputs.

What would settle it

Train EvoRIC-L2 on a held-out topology set that excludes the three reported scenarios, then evaluate on those scenarios: if the throughput advantage over EPA disappears or drops to in-distribution levels, the generalization claim is unsupported.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that a 3B-parameter Llama-3.2-Instruct model, fine-tuned with PPO using only environmental reward, becomes a better power-allocation policy for IAB networks than the equal-power heuristic, the successive convex approximation solver, the un-fine-tuned Llama, and the much larger DeepSeek-V3 and Gemini 1.5 Flash models. EvoRIC-L2, the version with two final transformer blocks trained, achieves these results across three network topologies, adapts to a changed objective through prompt modification alone, and loses almost nothing when one telemetry stream is corrupted. The paper also reports that the KL-regularized PPO update is necessary for stable training, since without it the policy drifts and training becomes numerically unstable.

Load-bearing premise

The cross-topology generalization claim assumes the test scenarios were not seen during fine-tuning, but the paper does not specify which topologies were in the training rollout buffer.

Editorial extensions

If this is right

A single fine-tuned compact LLM could replace topology-specific RL models, cutting the retraining burden in heterogeneous RAN deployments.
Sub-second inference (0.56 s) falls inside the near-RT RIC control window, so LLM-based control is not limited to non-real-time planning.
KL regularization keeps the fine-tuned model close to its pretrained knowledge, which should reduce catastrophic forgetting during continued learning.
Prompt-level intent changes let the same model switch objectives, such as priority-weighted throughput, without parameter updates.
The safety-verification and fallback layer ensures that even hallucinated outputs degrade to a valid baseline, a necessary condition for deployment in critical infrastructure.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper never states whether the three evaluation topologies were part of the PPO rollout buffer; if they were, the reported cross-topology generalization is in-distribution and a held-out topology evaluation is needed.
The 0.56 s inference figure is measured on an RTX 4090 workstation; real edge hardware will likely be slower, so the sub-second claim should be re-verified on target COTS edge servers.
The comparison against DeepSeek and Gemini is zero-shot with prompt engineering; fine-tuning those large models might narrow the gap, though their size would still make near-RT deployment impractical.
The composite reward's format penalty and fail-safe fallback mean part of the measured gain comes from suppressing parse failures; ablating the fallback would quantify how much.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's my take on EvoRIC.

The framework itself is clear and sensible: a non-RT RIC runs PPO fine-tuning on a 3B LLM, the near-RT RIC does inference with a safety-verification fallback. The reward design with format penalty and KL regularization is standard but applied cleanly. I like the intent-adaptation and telemetry-robustness experiments; those go beyond what most papers of this type bother with.

What's actually new is the specific application: a PPO-fine-tuned Llama-3.2-3B as the RL actor for IAB power allocation inside an O-RAN-style split. The baselines are reasonable, and the sub-second inference claim is relevant for the near-RT RIC.

The soft spots are real, and one is load-bearing. The paper's central contribution is cross-topology generalization, but the training topology set is never specified. Figure 2 shows training on M=3, N=5, which is Scenario 2, yet the text never says whether the final model was trained only on that scenario or on all three. If it was trained on all three, Figure 3 demonstrates in-distribution performance and the generalization claim collapses. That is not a minor omission; it is the key claim. The missing error bars and seeds on Figure 3 make it worse, especially in Scenario 3 where the margin over EPA is only 16.6%. The comparison to DeepSeek and Gemini is a bit apples-to-oranges since those are zero-shot via API, but the authors acknowledge the trade-off, so I don't weight that heavily. The latency numbers also come from different hardware, a practical point rather than a scientific one.

If the authors specify the protocol and report variance across seeds, this could become a solid paper. As it stands, the quantitative results are conditional. The framework idea is not wrong, and the architecture is described well enough that a referee can push for the missing details.

My call: send it to peer review rather than desk reject. It deserves referee time, but it needs major revision before acceptance. I would not cite the specific numerical gains until the training protocol is clarified.

Referee Report

5 major / 5 minor

Summary. The paper introduces EvoRIC, a hierarchical O-RAN control framework in which a compact LLM (Llama-3.2-3B) is fine-tuned via proximal policy optimization (PPO) in the non-RT RIC and then deployed in the near-RT RIC for power allocation in IAB networks. The framework serializes network state into structured prompts, uses a composite reward with format penalty and KL regularization, and closes the loop through O1/A1/E2 interfaces. Experiments compare EvoRIC-L2 against EPA, Llama3B-NoFT, DeepSeek-V3, Gemini, and an SCA baseline, reporting throughput gains across three topologies, intent adaptation, and robustness to telemetry corruption.

Significance. If the results hold, the paper demonstrates a practically attractive combination: a 3B-parameter LLM fine-tuned with reinforcement learning can outperform both a conventional heuristic and a model-based solver for IAB power allocation while keeping inference latency below one second, and it can transfer to varying topology sizes without retraining. The closed-loop collect-update-deploy design and the explicit handling of format compliance and safety fallback are well motivated. The paper also includes a limitation statement for the telemetry robustness experiments, which is commendable. However, the empirical claims rest on an under-specified evaluation protocol, and the central generalization claim is not yet demonstrated.

major comments (5)

[Section III-C, Figure 3] The paper's central generalization claim is not verifiable because the training topology distribution is never specified. The training curve in Figure 2 is reported for M=3, N=5, and the evaluation scenarios in Figure 3 include M=3,N=6; M=3,N=5; and M=2,N=3. If the PPO rollouts used any of these scenarios, the corresponding result is in-distribution. Please state exactly which topologies were used for training, whether any evaluation scenarios were held out, and how the reported averages were computed.
[Section III-C, Figure 3 and Table 1] All numerical results are averages over 500 episodes with no confidence intervals, standard deviations, or number of random seeds. The claimed margin in Scenario 3 (16.6% over EPA) may be within sampling noise; without variance bars it is impossible to judge statistical significance. Please report per-scenario confidence intervals or box plots and repeat with at least three seeds.
[Section III-B-4 and Section III-C] The safety verification module falls back to EPA whenever the LLM output is invalid, so the throughput averages in Figure 3 and Table 1 mix successful LLM actions with EPA fallback actions. The paper does not report the fraction of fallback episodes or the throughput conditional on a valid generation. This confound should be disentangled to assess the true contribution of the learned policy.
[Section III-C, Table 1] The SCA-based baseline is under-specified; no details are given on the convex approximation scheme, initialization, iteration count, or convergence tolerance. Given that EvoRIC reportedly outperforms SCA by 10.6% on the weighted-throughput intent, please document the SCA implementation and verify that both methods use the same information and solve the same optimization problem.
[Section III-C and Section IV] The simulator setup is not described sufficiently for reproduction. The paper omits the simulation platform, path-loss exponents, LoS probability parameters, Nakagami-m parameter, number of users per SBS, mobility parameters, episode length, and initial-state distribution. Please provide these details or release the simulator code.

minor comments (5)

[Figure 2 caption] The caption contains an apparent text-encoding artifact (e.g., '/s48/s49/s48/s48/s48 /s50/...'); please repair the caption and ensure the figure is legible.
[Section III-B-4] The phrase 'an fail-safe protocol' should read 'a fail-safe protocol'.
[Section III-C] The notation 'top p' should be made consistent with 'top-p sampling' at first use; also define the PPO hyperparameters (clip ratio, GAE lambda, number of epochs) for reproducibility.
[Section III-C, Scenario 3 discussion] The sentence 'the solver improves upon EPA by only 16.6%' is ambiguous; specify which solver (SCA or EvoRIC) is referred to.
[Section III-C] The inference latency of 0.56 s is reported without specifying the inference hardware, quantization, batch size, or whether prompt parsing is included; please state the test conditions or add a caveat.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: EvoRIC's empirical comparisons are self-contained; the only self-citation is non-load-bearing, and the generalization claim is under-specified but not circular.

full rationale

The paper's derivation chain is empirical rather than analytic: EvoRIC uses PPO to fine-tune Llama-3.2-3B as the actor, with rewards computed from a simulator, and then compares throughput against EPA, Llama3B-NoFT, DeepSeek, Gemini, and an SCA/CVXPY solver. No equation or fitted constant is reused as a prediction: the architecture is not defined in terms of the reported metrics, the reward is not the evaluation metric by construction, and the baselines are externally implemented. The only self-citation, [5] (LLM-hRIC), appears in the introduction as related work describing a prior hierarchical architecture and is not used to justify EvoRIC's central claims; the KL-regularized PPO update is supported by external references [10], [11]. The main weakness adjacent to circularity is that the cross-topology generalization claim in Section III-C (Figure 3) is not verifiable because the paper never states which topology/topologies were used for PPO training: the training curve in Figure 2 is for M=3, N=5 (Scenario 2), so the reader cannot tell whether Scenarios 1-3 are in-distribution or out-of-distribution. This is a missing train/test specification, not a reduction of the claim to its inputs; it should be weighed as a rigor/correctness concern, and the paper itself acknowledges limited scope when it notes that telemetry robustness 'does not establish general robustness to all telemetry faults.' Overall, no load-bearing step is circular.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

No new physical entities are posited. The framework's dependence is concentrated in hand-chosen RL hyperparameters and simulator-fidelity assumptions, plus an unverified transferability assumption across topologies.

free parameters (6)

Number of fine-tuned transformer blocks L = 2 (EvoRIC-L2)
Selected by comparing L=1, 2, and 3 on the same throughput metric; no held-out validation set is described. This choice affects the reported central result.
RL temperature schedule = 1.5 to 0.6 linear decay
Chosen by hand for exploration and exploitation; no sensitivity analysis or external benchmark is provided.
Top-p sampling = 0.95
Exploration hyperparameter with no ablation supporting the specific value.
Format penalty r_format = -5
Hand-chosen penalty to enforce parsable output; no ablation on penalty magnitude is reported.
Learning rate and batch size = 1e-5 and 16
Standard PPO hyperparameters, not tuned against an external reference or ablations.
KL penalty coefficient = not reported
The KL-regularized PPO is central to the stability claims in Section III-C, but the numerical weight of the KL term is never quantified.

assumptions (5)

domain assumption The IAB network model with LoS/NLoS path loss, Nakagami-m fading, Gaussian-Markov mobility, and bottleneck-limited throughput is a faithful proxy for real IAB deployment performance.
Introduced in Section III-A; every reward and throughput number depends on this simulator fidelity, which is not validated against measurements.
domain assumption PPO with KL regularization to a frozen reference model can fine-tune an LLM's control policy without catastrophic forgetting.
Invoked in Sections II-A and III-B; relies on prior work [10,11] and is not independently verified on this task.
domain assumption Throughput is the correct scalar reward for the power allocation objective, with only a format penalty attached.
Section III-B defines r_t = r_th + r_format; the design assumes this reward captures the operator's intent.
standard math KL divergence penalizes large policy updates and stabilizes RL fine-tuning.
Used as a trust-region regularizer in Sections II-A and III-B; standard in RLHF literature.
domain assumption A common model fine-tuned in the non-RT RIC can be deployed across multiple near-RT RICs with different topologies.
Stated in Section II-A.3; the experimental evidence for this is the ambiguous cross-scenario evaluation in Figure 3.

how reviews work

0 comments

Cite this review

Pith. "Pith review of EvoRIC: Reinforcement Learning Fine-Tuned LLM-empowered RAN Intelligent Control Toward Autonomous O-RAN." pith.science (2026). https://pith.science/paper/AE2FJZOF

@misc{pith2026260806789,
  author       = {Pith},
  title        = {Pith review of: EvoRIC: Reinforcement Learning Fine-Tuned LLM-empowered RAN Intelligent Control Toward Autonomous O-RAN},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AE2FJZOF}},
  note         = {Machine review of arXiv:2608.06789}
}

read the original abstract

Despite recent advances in applying artificial intelligence (AI) techniques to radio access network (RAN), critical challenges remain: traditional machine learning (ML) algorithms suffer from limited generalization across varying network topologies, whereas general-purpose large language models (LLMs) face high computational demands and lack domain-specific knowledge. To address these gaps, this article introduces the evolving RAN intelligent controller (RIC) (EvoRIC) framework, a hierarchical architecture that enables continuous evolution by leveraging a non-real-time RIC (non-RT RIC) for global model updates and a near-real-time RIC (near-RT RIC) for local execution, dynamically empowering LLMs with domain-specific decision-making capabilities. Within this framework, we employ a reinforcement learning-based fine-tuning (RLFT) mechanism where an LLM operates as an actor within a proximal policy optimization (PPO) agent. By leveraging the interaction tuples collected from the wireless environment, the LLM's parameters are iteratively updated to align semantic reasoning with rigorous network performance objectives. We evaluate the generalization and efficacy of the proposed EvoRIC framework within integrated access and backhaul (IAB) networks, and finally, discuss the open challenges and future directions of the EvoRIC framework toward realizing autonomous O-RAN.

Figures

Figures reproduced from arXiv: 2608.06789 by the authors.

**Figure 2.** Training curve for EvoRIC with and without KL when [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Average total throughput of the proposed EvoRIC and b [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 10 canonical work pages

[1]

Un- derstanding O-RAN: Architecture, interfaces, algorithms , security, and research challenges,

M. Polese, L. Bonati, S. D’oro, S. Basagni, and T. Melodia , “Un- derstanding O-RAN: Architecture, interfaces, algorithms , security, and research challenges,” IEEE Commun. Surv. Tutor . , vol. 25, no. 2, pp. 1376–1411, Jan. 2023

work page 2023
[2]

Resource allocation for network slicing in open RAN: A hier archical learning approach,

K. Qiao, H. Wang, W. Zhang, D. Y ang, Y . Zhang, and N. Zhang, “Resource allocation for network slicing in open RAN: A hier archical learning approach,” IEEE Trans. Cogn. Commun. Netw. , vol. 11, no. 4, pp. 2584–2600, 2025

work page 2025
[3]

LLM-empowered resource allocation i n wireless communications systems,

W. Lee and J. Park, “LLM-empowered resource allocation i n wireless communications systems,” IEEE Access , vol. 14, pp. 15 260–15 272, 2026

work page 2026
[4]

Prompting Wireless Networks: Reinforced In-Context Learning for Power Control

H. Zhou, C. Hu, D. Y uan, Y . Y uan, D. Wu, X. Liu et al. , “Prompting wireless networks: Reinforced in-context learning for pow er control,” arXiv preprint arXiv:2506.06526 , 2025

work page Pith review arXiv 2025
[5]

LLM- hRIC: LLM-empowered hierarchical RAN intelligent control for O-RAN,

L. Bao, S. Y un, J. Lee, and T. Q. Quek, “LLM- hRIC: LLM-empowered hierarchical RAN intelligent control for O-RAN,” IEEE Commun. Mag. , pp. 1–7, 2026, early access

work page 2026
[6]

LLM4WM: Adap ting LLM for wireless multi-tasking,

X. Liu, S. Gao, B. Liu, X. Cheng, and L. Y ang, “LLM4WM: Adap ting LLM for wireless multi-tasking,” IEEE Trans. Mach. Learn. Commun. Netw., pp. 1–1, 2025

work page 2025
[7]

Empowering large language models in wireless communi cation: A novel dataset and ﬁne-tuning framework,

Y . Lin, R. Zhang, W. Huang, K. Wang, Z. Ding, D. K. C. So, and D. Niy- ato, “Empowering large language models in wireless communi cation: A novel dataset and ﬁne-tuning framework,” IEEE Trans. on Commun. , vol. 73, no. 12, pp. 14 260–14 274, 2025

work page 2025
[8]

Mobile-LLaMA: Instr uction ﬁne-tuning open-source LLM for network analysis in 5G netwo rks,

K. B. Kan, H. Mun, G. Cao, and Y . Lee, “Mobile-LLaMA: Instr uction ﬁne-tuning open-source LLM for network analysis in 5G netwo rks,” IEEE Netw., vol. 38, no. 5, pp. 76–83, Sep. 2024

work page 2024

Show all 15 references

[9]

ORAN-Bench-13K: An open source bench- mark for assessing LLMs in open radio access networks,

P . Gajjar and V . K. Shah, “ORAN-Bench-13K: An open source bench- mark for assessing LLMs in open radio access networks,” in Proc. - IEEE Consum. Commun. Netw. Conf. (CCNC) , Las V egas, NV , USA, Jan. 2025, pp. 1–4

2025
[10]

LLMs are greedy agents: Effects of RL ﬁne-tuning on de cision- making abilities,

T. Schmied, J. Bornschein, J. Grau-Moya, M. Wulfmeier, and R. Pas- canu, “LLMs are greedy agents: Effects of RL ﬁne-tuning on de cision- making abilities,” arXiv preprint arXiv:2504.16078 , 2025

2025 arXiv
[11]

Rethinking KL regu larization in RLHF: From value estimation to gradient optimization,

K. Liu, J. K. Liu, M. Chen, and Y . Liu, “Rethinking KL regu larization in RLHF: From value estimation to gradient optimization,” arXiv preprint arXiv:2510.01555, 2025

2025
[12]

The Llama 3 herd of models,

A. Grattaﬁori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian , A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. V aughan et al. , “The Llama 3 herd of models,” arXiv preprint arXiv:2407.21783 , 2024

2024 arXiv
[13]

Deepseek-v3 technical report,

A. Liu, B. Feng, B. Xue, B. Wang, B. Wu, C. Lu, C. Zhao, C. De ng, C. Zhang, C. Ruan et al., “Deepseek-v3 technical report,” arXiv preprint arXiv:2412.19437, 2024

2024 arXiv
[14]

Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context,

G. Team, P . Georgiev, V . I. Lei, R. Burnell, L. Bai, A. Gul ati, G. Tanzer, D. Vincent, Z. Pan, S. Wang et al., “Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context,” arXiv preprint arXiv:2403.05530, 2024

2024 arXiv
[15]

CVXPY: A Python-embedded model ing lan- guage for convex optimization,

S. Diamond and S. Boyd, “CVXPY: A Python-embedded model ing lan- guage for convex optimization,” Journal of Machine Learning Research , vol. 17, no. 83, pp. 1–5, 2016

2016

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Un- derstanding O-RAN: Architecture, interfaces, algorithms , security, and research challenges,

M. Polese, L. Bonati, S. D’oro, S. Basagni, and T. Melodia , “Un- derstanding O-RAN: Architecture, interfaces, algorithms , security, and research challenges,” IEEE Commun. Surv. Tutor . , vol. 25, no. 2, pp. 1376–1411, Jan. 2023

work page 2023

[2] [2]

Resource allocation for network slicing in open RAN: A hier archical learning approach,

K. Qiao, H. Wang, W. Zhang, D. Y ang, Y . Zhang, and N. Zhang, “Resource allocation for network slicing in open RAN: A hier archical learning approach,” IEEE Trans. Cogn. Commun. Netw. , vol. 11, no. 4, pp. 2584–2600, 2025

work page 2025

[3] [3]

LLM-empowered resource allocation i n wireless communications systems,

W. Lee and J. Park, “LLM-empowered resource allocation i n wireless communications systems,” IEEE Access , vol. 14, pp. 15 260–15 272, 2026

work page 2026

[4] [4]

Prompting Wireless Networks: Reinforced In-Context Learning for Power Control

H. Zhou, C. Hu, D. Y uan, Y . Y uan, D. Wu, X. Liu et al. , “Prompting wireless networks: Reinforced in-context learning for pow er control,” arXiv preprint arXiv:2506.06526 , 2025

work page Pith review arXiv 2025

[5] [5]

LLM- hRIC: LLM-empowered hierarchical RAN intelligent control for O-RAN,

L. Bao, S. Y un, J. Lee, and T. Q. Quek, “LLM- hRIC: LLM-empowered hierarchical RAN intelligent control for O-RAN,” IEEE Commun. Mag. , pp. 1–7, 2026, early access

work page 2026

[6] [6]

LLM4WM: Adap ting LLM for wireless multi-tasking,

X. Liu, S. Gao, B. Liu, X. Cheng, and L. Y ang, “LLM4WM: Adap ting LLM for wireless multi-tasking,” IEEE Trans. Mach. Learn. Commun. Netw., pp. 1–1, 2025

work page 2025

[7] [7]

Empowering large language models in wireless communi cation: A novel dataset and ﬁne-tuning framework,

Y . Lin, R. Zhang, W. Huang, K. Wang, Z. Ding, D. K. C. So, and D. Niy- ato, “Empowering large language models in wireless communi cation: A novel dataset and ﬁne-tuning framework,” IEEE Trans. on Commun. , vol. 73, no. 12, pp. 14 260–14 274, 2025

work page 2025

[8] [8]

Mobile-LLaMA: Instr uction ﬁne-tuning open-source LLM for network analysis in 5G netwo rks,

K. B. Kan, H. Mun, G. Cao, and Y . Lee, “Mobile-LLaMA: Instr uction ﬁne-tuning open-source LLM for network analysis in 5G netwo rks,” IEEE Netw., vol. 38, no. 5, pp. 76–83, Sep. 2024

work page 2024