Pith. sign in

REVIEW 5 major objections 6 minor 34 references

Clipped Affine Policy: Low-Complexity Near-Optimal Online Power Control for Energy Harvesting Communications over Fading Channels

T0 review · 5 major / 6 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A clipped affine power policy—a closed-form rule mixing battery level and inverse channel strength—achieves near-optimal (≤2% loss) online throughput for energy harvesting over fading channels, and its robust variant extends to one-step loo

desk verdict A useful low-complexity policy family for EH fading channels, but the '<2% near-optimal' claim rests on an unquantified value-function approximation and needs stronger empirical backing. read the letter →

arxiv 2601.07622 v2 pith:EJJPYU66 submitted 2026-01-12 cs.IT cs.SYeess.SYmath.IT

Hao Wu , Shengtian Yang , Huiguo Gao , Diao Wang , Jun Chen , Guanding Yu This is my paper
classification cs.ITcs.SYeess.SYmath.IT MSC 90C4068T0594A40
keywords onlinepowercontrolenergyharvestingfadingchannelBellmanequationclippedaffinepolicyrelative-valuefunctionreinforcementlearninglookahead
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper attacks the online power control problem for a point-to-point energy harvesting transmitter with a finite battery and slowly fading channel—an MDP whose Bellman equation resists closed-form solution. The authors propose approximating the relative-value function by a linear-policy-based form, which turns the Bellman update into a concave maximization whose solution is a clipped affine function of battery level and reciprocal channel SNR. From that they derive two policy families, optimistic and robust, and use a small RL module to fit the two free parameters of each (plus a third estimated directly from battery dynamics), yielding low-complexity schemes with at most five parameters. Simulations show the robust clipped affine policy stays within 1–2% of the optimal throughput across Bernoulli, exponential, and uniform energy arrivals and across NSNR values, outperforming neural and shallow RL baselines. The reason this matters: it suggests that the notoriously high-dimensional online power control problem has a simple near-optimal parametric structure, not a curvy one.

What carries the argument

The load-bearing object is the linear-policy-based relative-value approximation ĥ_{q,γ̂}(b)=(1/q) r(γ̂ q b), Eq. (25). It replaces the true relative-value function—unknown, two-dimensional, and defined through a fixed-point equation—with a one-dimensional concave function parameterized by two effective quantities: the slope q of a linear battery-consumption policy and an effective channel SNR γ̂. Substituting ĥ into the Bellman equation produces an optimization problem whose first-order condition is a linear equation in u, so the optimal action is a saturating (clipped) linear function of battery b and 1/γ. The clipping encodes the energy-causality constraint 0≤u≤b and the battery-capacity

What would settle it

Run the robust clipped affine policy with RL-fitted q and γ̂ under a thoroughly non-i.i.d. energy arrival process, e.g., alternating deterministic bursts and silence at low NSNR, and compare its asymptotic throughput (via long-run simulation) to the optimal policy computed by discretized policy iteration; a measured performance loss exceeding the claimed ~2% would refute the universal near-optimality. A more direct check: numerically solve the Bellman equation and compute sup_{b,γ}|h(b,γ)−(1/q)r(γ̂ q b)| for the fitted parameters; a large residual would indicate the derivation's foundation is

Watch

Extended reading notes

Core claim

The central discovery is that the relative-value function h(b,γ) of the Bellman equation can be approximated accurately by (1/q) r(γ̂ q b), where q is an 'effectively equivalent linear-policy slope' and γ̂ an 'effectively equivalent channel SNR coefficient.' Inserting this approximation into the Bellman update turns the per-step action choice into a concave optimization problem whose maximizer is a clipped affine policy: σ(b,γ)=clip(θ0+θ1 b−θ2/γ, 0, b), with clipping bounds depending on battery. The robust version σ_rca(b,γ)=clip((q b−(1−p)/γ+1/γ̂)/(1−p+q),0,b) is the maximin-optimal choice when the energy-arrival distribution is characterized by a mean-to-capacity ratio p. This gives an onl

Load-bearing premise

Near-optimality rests on the unquantified accuracy of the relative-value approximation h(b,γ)≈(1/q) r(γ̂ q b); if that linear-policy-based guess is far from the true Bellman value for some energy distribution or operating regime, the derived clipped affine policies inherit the error, and the paper gives no error bound tying the approximation to the true value function.

Editorial extensions

If this is right

  • The robust clipped affine policy with RL-tuned q and γ̂ attains less than 1% average and 2% maximum throughput loss vs optimal across Bernoulli, exponential, uniform energy arrivals at NMCR 0.1–0.9 and NSNR 0–30 dB.
  • The optimistic variant with one-step energy lookahead outperforms its robust counterpart in accurate-prediction settings and shows ≤0.8% loss, though it degrades when predictions are inaccurate.
  • The policy family extends to one-step channel lookahead (adding an affine dependence on the predicted γ̂=s Γ̇+γ̂0) and joint energy-channel lookahead with only 3–6 parameters, and these extensions inherit the near-optimality when lookahead is accurate.
  • Because the policy is closed-form with bounded parameters, the resulting schemes are lightweight (three to five parameters versus hundreds to thousands for deep RL baselines) and avoid per-step gradient updates.
  • The clipped affine structure reveals an online analogue of directional waterfilling, connecting the online and offline optimal power control problems.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the linear-slope approximation is as accurate as simulations suggest, then the true relative-value function for these problems is close to a one-dimensional concave shape for a wide range of energy distributions, which would imply that the online power control problem lacks the complex non-convex structure often feared; one could test this directly by fitting ĥ to numerically computed value fu
  • The approach's dependence on i.i.d. energy arrivals and channel state, except through lookahead extensions, suggests a promising direction: combining the clipped affine policy with a learned predictor (not just one-step) for non-i.i.d. energy arrivals, decoupling prediction from decision-making as the paper hints in a footnote; the decision part would remain three parameters.
  • A direct generalization to multi-antenna or multi-user settings might start by replacing r(x)=log(1+x) with any increasing concave reward, which the paper notes is all that is needed; whether the clipped affine form survives non-logarithmic rewards is untested but plausible given the concavity of the derivation.
  • The robustness parameter p is essentially the mean-to-capacity ratio of the energy arrival distribution; this suggests a testable extension: in deployments where p can be measured over long windows, the policy may run without any RL, using p and a manually tuned q, γ̂—though the paper does not claim this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper studies online power control for a point-to-point energy-harvesting transmitter with a finite battery and slow block fading. It proposes an approximate relative-value function ĥ(b) = (1/q) r(γ̂ q b) and uses it to derive two closed-form clipped affine policies: an optimistic policy (OCA, Theorem 5) and a robust policy (RCA, Theorem 6). The parameters q and γ̂ are tuned by a lightweight RL algorithm (Algorithm 1), with extensions for one-step energy, channel, and joint lookahead (Algorithms 2–4). The central claim is that the RCA-based schemes, including the RL variants, achieve less than 2% performance loss relative to the optimal policy across Bernoulli, exponential, and uniform energy arrivals, while using at most five parameters. Appendix A proves the supporting theorems for the one-point and Bernoulli cases, and Appendix B derives the OCA/RCA policies for the approximate Problems 3 and 4.

Significance. If the near-optimality claim were rigorously established, the paper would provide a useful low-complexity design: closed-form policies with three to five interpretable parameters, a waterfilling-type interpretation, and a transparent way to inject domain knowledge into RL. The special cases in Eqs. (37)–(40) recovering the optimal offline policy and the two-piece fixed-fraction policy are elegant, and the proofs of Theorems 2–6 appear internally correct for the stated approximate problems. However, the paper's headline generality is not currently supported. The approximation in Eq. (25) has no quantified error bound, the RL parameter fit is a bootstrap on that same approximation, and the optimal benchmark is a coarsely quantized policy. The abstract additionally promises Markov-arrival extensions and named schemes (RCA-OLA-A, RCA-OL) that do not appear in the body. These issues make the central '<2%' claim a simulation observation rather than a verified property.

major comments (5)
  1. [§III, Eq. (25)] The central near-optimality claim rests on the unquantified approximation h̄(b) ≈ (1/q) r(γ̂ q b). Theorems 5 and 6 are exact only for the approximate Problems 3 and 4, not for Problem 1. Theorem 4 verifies Eq. (20) only at a single point b0 = e/q for one-point arrivals. There is no bound on the Bellman residual or on the resulting performance gap. I request either a quantitative approximation-error bound (e.g., bounding the value-function residual in terms of the variation of ρ or the Jensen gap in Eq. (24)) or a restriction of the near-optimality claim to the tested distributions/regimes.
  2. [§III, Theorem 6 and Section VI] The RCA policy is proved optimal for Problem 4 only when ρ(P_E, x) = p for all x ≤ c. This holds for Bernoulli arrivals but not for exponential or uniform arrivals. For E ∼ Exp(λ), ρ(x) = (1 − e^{−x/λ})/(x/λ) is state-dependent. Algorithm 1 estimates a single p via Eq. (43) and the paper gives no bound on the loss caused by replacing ρ(P_E, C_t) with a constant. This is the main analytical gap separating Theorems 5–6 from the claimed '<2%' performance for general i.i.d. energy arrivals.
  3. [§IV, Algorithm 1, lines 11–18] The RL loss L(q, γ̂) minimizes squared TD error in which the target H_i = R_i − ĝ + ĥ_{q,γ̂}(B_i′) itself uses the same parametric approximation ĥ. This is a bootstrapped fit: the loss can be small even when ĥ is far from the true relative-value function. There is no convergence or approximation guarantee. I suggest validating ĥ against the PI-computed value function or reporting Bellman residuals for the tested scenarios, so the reader can see whether the fitted parameters are actually close to the true value function.
  4. [§VI, Table VII and Figs. 3–5] The optimal benchmark is computed with a coarsely quantized MDP (250 battery levels, 50 channel levels, 250 action levels, Footnote 2). At c = 10^4 (Bernoulli NMCR = 0.1, high NSNR), quantization can only lower the optimal benchmark, so the reported OMF > 1 values and the sub-1% losses are biased in favor of the proposed schemes. No confidence intervals, standard errors, or multiple seeds are reported. To support the '<2% across a range of scenarios' claim, the paper needs a refined or error-bounded optimal benchmark and statistical uncertainty quantification.
  5. [Abstract and Introduction vs. body] The abstract promises extensions to 'Markov energy arrivals' and introduces named schemes RCA-OLA-A and RCA-OL. The body does not define RCA-OLA-A or RCA-OL, and Section V explicitly retains the i.i.d. assumption ('For simplicity, we continue to assume that both the energy arrivals and channel SNR coefficients are i.i.d.'). No Markov-arrival algorithm or simulation is presented. The headline results therefore cannot be fully checked from the manuscript as written.
minor comments (6)
  1. [Table I, footnote b] The statement that [23]'s zero-output behavior 'differs significantly' from known optimal policies is an evaluative claim not supported by the numerical comparison in the table. Soften or add a quantitative reference.
  2. [Eq. (20)] The definition of ĥ_q for q = 0 as a limit is fine but slightly nonstandard; clarify that the limit is taken pointwise.
  3. [Section V, Algorithms 2–4] The statement that the i.i.d. assumption 'affects only the probability weights assigned to sample paths' is too strong: the conditional-expectation estimators (48)–(50) are derived under i.i.d. assumptions and do not automatically carry over to Markov arrivals. Justify the extension or remove the claim.
  4. [Throughout] Typos: 'Bernoullli' in the Fig. 3 caption, 'ALGROTHM' in the Table VI heading, and 'underling' in Section VII.
  5. [Table VII] State explicitly over which NSNR points and energy distributions the average and maximum losses are computed, and provide a per-scenario breakdown.
  6. [References] Reference [30] is a Chinese patent with no public English equivalent. If possible, provide a DOI or arXiv/technical-report version so reviewers and readers can access it.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the near-optimality claim is an empirical result validated against an independent optimal baseline, not a consequence of the approximation by construction.

full rationale

The paper's derivation chain is transparent: it starts from the Bellman equation, introduces an explicit linear-policy-based approximation for the relative-value function (Eq. 25), defines optimistic and pessimistic approximate problems (Problems 3 and 4), and derives closed-form clipped affine policies as exact optima of those approximate problems (Theorems 5 and 6). The theorems do not assert optimality for the original problem; they are conditional on special energy-arrival forms (one-point or Bernoulli). For general distributions such as exponential and uniform, the paper applies the same policy with averaged parameters, which is an unquantified heuristic rather than a claimed exact result. The headline '<2% performance loss' is obtained by simulation against a policy-iteration optimal baseline, not by substituting the fitted parameters into the approximation and declaring victory. The RL component tunes q and γ̂ using a TD error in which the same approximate ĥ appears as the target; this is a standard bootstrap in approximate dynamic programming, but it does not make the reported performance a definitional tautology, because the performance metric is measured externally. A few self-citations ([11], [13], [27]) are used to motivate the linear-policy ansatz, but external results ([9]) and the paper's own Theorem 4 provide additional support, so the self-citations are not load-bearing. The main weakness is the absence of an error bound for the Eq. (25) approximation and the state-dependent DMCR mismatch; this is a correctness/rigor concern, not circularity.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The paper introduces no new physical entities or forces. Its novelty relies on effective mathematical parameters q, gamma_hat, e, and p, which are fully captured as free parameters. The main structural assumptions are the parametric value-function ansatz and the substitution of general distributions by effective one-point/Bernoulli objects.

free parameters (5)
  • q (effectively equivalent linear-policy slope) = Learned in (0,1), initialized 0.5 in Algorithm 1
    Defines the approximate value function (25) and appears directly in both clipped affine policies (34), (36). Performance depends critically on this fitted value.
  • gamma_hat (effectively equivalent channel SNR coefficient) = Learned in (0,∞), initialized 1 in Algorithm 1
    Introduced in Eq. (25) to collapse the random channel into an effective constant; appears in both policy formulas and is fitted via RL.
  • e (OCA parameter) = Estimated online via Eq. (41), or set from lookahead in Algorithm 2
    Represents dynamic clipped mean of energy arrivals; the optimistic policy (34) assumes a one-point effective arrival e.
  • p (RCA parameter) = Estimated online via Eq. (43), or set from lookahead in Algorithm 2
    Represents dynamic mean-to-capacity ratio; the robust policy (36) assumes a Bernoulli-like effective arrival with DMCR p.
  • gamma_hat_0 and s (channel-lookahead parameters) = Learned in Algorithms 3 and 4
    In Eq. (50), gamma_hat = s * Gamma_dot + gamma_hat_0, adding two more tunable parameters for the channel-lookahead extension.
assumptions (6)
  • standard math The average-cost Bellman equation (6) is valid and the optimal average reward is attained.
    Theorem 1 invokes [26, Thm. 6.1] to justify the Bellman characterization.
  • domain assumption Energy arrivals and channel SNR coefficients are i.i.d., mutually independent, and known causally at the transmitter.
    MDP formulation in Section II; the paper explicitly assumes (E_t) and (Gamma_t) are independent i.i.d. sequences, except for later lookahead variants.
  • ad hoc to paper The approximate relative-value function (25) is accurate for any energy arrival distribution.
    Eq. (25) replaces E h(b,Γ) by (1/q) r(gamma_hat q b) with q and gamma_hat as effective parameters. No error bound is provided; this is the main ansatz of the paper.
  • ad hoc to paper A general energy arrival distribution can be represented by an effective one-point e (OCA) or effective DMCR p (RCA).
    Theorems 5 and 6 require exact conditions mu_bar(P_E,x)=<e>_<=x or rho_bar(P_E,x)=p, but the algorithms apply them to Bernoulli, exponential, and uniform distributions via estimates (41), (43).
  • ad hoc to paper The Jensen upper/lower bounds used in Problems 3 and 4 preserve near-optimality.
    Eqs. (27)-(31) replace the conditional expectation by bounds; the paper does not bound the resulting performance gap for non-extreme distributions.
  • ad hoc to paper The TD-learning updates for q and gamma_hat converge to parameters that make the clipped affine policy near-optimal.
    Algorithm 1 fits h to a semi-gradient TD target; convergence and the transfer of fitting error to policy performance are empirical, not proven.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Clipped Affine Policy: Low-Complexity Near-Optimal Online Power Control for Energy Harvesting Communications over Fading Channels." pith.science (2026). https://pith.science/paper/EJJPYU66

@misc{pith2026260107622,
  author       = {Pith},
  title        = {Pith review of: Clipped Affine Policy: Low-Complexity Near-Optimal Online Power Control for Energy Harvesting Communications over Fading Channels},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EJJPYU66}},
  note         = {Machine review of arXiv:2601.07622}
}
read the original abstract

This paper studies online power control for battery-limited point-to-point energy harvesting communications over slow block-fading channels. A linear-policy-based approximation is developed for the relative-value function in the Bellman equation of the power control problem. This approximation leads to two fundamental parameterized clipped affine policies: an optimistic policy derived from a certainty-equivalence-type approximation and a robust policy derived from worst-case analysis. For independent and identically distributed energy arrivals and channel states, two families of power control schemes are developed based on the optimistic clipped affine (OCA) and robust clipped affine (RCA) policies, respectively. The proposed adaptive RCA policy based on reinforcement learning (RCA-RL) is further extended to address four scenarios with contextual information: one-step energy lookahead, one-step channel lookahead, one-step joint energy-channel lookahead, and Markov energy arrivals. Extensive simulation results show that the proposed schemes provide a favorable tradeoff between computational complexity and performance. The adaptive RCA policy based on the maximin optimal linear-policy-slope approximation (RCA-OLA-A) and the RCA-RL scheme achieve the best overall performance, while the RCA policy based on the maximin optimal linear policy (RCA-OL) is the best-performing closed-form policy. In particular, RCA-OLA-A, RCA-RL, and the aforementioned RCA-RL extensions achieve less than 2% performance loss relative to the optimal policy across a range of scenarios, consistently outperforming the considered benchmark approaches, including generic reinforcement learning baselines. The RCA-OL policy also performs well with less than 4% performance loss.

Figures

Figures reproduced from arXiv: 2601.07622 by the authors.

Figure 1
Figure 1. A discrete-time energy harvesting wireless communication system. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. An illustration of Algorithm 1 (for σ = σrca). TABLE II PHYSICAL INTERPRETATIONS OF CLIPPED-AFFINE-POLICY PARAMETERS Parameter Definition Physical Interpretation q Eq. (25), Probs. 3 and 4 Approximate fraction of energy con￾sumption (in the next time slot) under the optimal policy γˆ Eq. (25), Probs. 3 and 4 Effectively equivalent channel SNR co￾efficient (in the next time slot) e Thm. 5 Dynamic clipped mean of the … view at source ↗
Figure 3
Figure 3. Performance comparison under Bernoullli energy arrivals. [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Performance comparison under exponential energy arrivals. [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Performance comparison under uniform energy arrivals. [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

34 extracted references · 1 linked inside Pith

  1. [1]

    Energy harvesting wireless communications: A review of recent advances,

    S. Ulukus, A. Yener, E. Erkip, O. Simeone, M. Zorzi, P. Grover, and K. Huang, “Energy harvesting wireless communications: A review of recent advances,”IEEE J. Sel. Areas Commun., vol. 33, no. 3, pp. 360– 381, Mar. 2015

    2015

  2. [2]

    Advances in energy harvesting communications: Past, present, and future challenges,

    M.-L. Ku, W. Li, Y . Chen, and K. J. Ray Liu, “Advances in energy harvesting communications: Past, present, and future challenges,”IEEE Commun. Surveys Tuts., vol. 18, no. 2, pp. 1384–1412, 2016

    2016

  3. [3]

    Sensing, computing, and communications for energy harvesting IoTs: A survey,

    D. Ma, G. Lan, M. Hassan, W. Hu, and S. K. Das, “Sensing, computing, and communications for energy harvesting IoTs: A survey,”IEEE Commun. Surveys Tuts., vol. 22, no. 2, pp. 1222–1250, 2020

    2020

  4. [4]

    Modeling and analy- sis of energy harvesting and smart grid-powered wireless communication networks: A contemporary survey,

    S. Hu, X. Chen, W. Ni, X. Wang, and E. Hossain, “Modeling and analy- sis of energy harvesting and smart grid-powered wireless communication networks: A contemporary survey,”IEEE Trans. Green Commun. Netw., vol. 4, no. 2, pp. 461–496, Jun. 2020

    2020

  5. [5]

    Machine learning applica- tions in energy harvesting internet of things networks: A review,

    O. Alamu, T. O. Olwal, and E. M. Migabo, “Machine learning applica- tions in energy harvesting internet of things networks: A review,”IEEE Access, vol. 13, pp. 4235–4266, 2025

    2025

  6. [6]

    Power control for battery-limited energy harvest- ing communications,

    S. Yang and J. Chen, “Power control for battery-limited energy harvest- ing communications,”Foundations and Trends® in Communications and Information Theory, vol. 22, no. 2-3, pp. 185–393, 2025

    2025

  7. [7]

    Optimal online strategies for an energy harvesting system with Bernoulli energy recharges,

    A. Kazerouni and A. ¨Ozg¨ur, “Optimal online strategies for an energy harvesting system with Bernoulli energy recharges,” inProc. 2015 13th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt). Mumbai, India: IEEE, May 2015, pp. 235–242

    2015

  8. [8]

    Capacity of the AWGN channel with random battery recharges,

    D. Shaviv and A. ¨Ozg¨ur, “Capacity of the AWGN channel with random battery recharges,” inProc. 2015 IEEE International Symposium on Information Theory (ISIT). Hong Kong, Hong Kong: IEEE, Jun. 2015, pp. 136–140

    2015

Show all 34 references
  1. [9]

    Universally near optimal online power control for energy harvest- ing nodes,

    ——, “Universally near optimal online power control for energy harvest- ing nodes,”IEEE J. Sel. Areas Commun., vol. 34, no. 12, pp. 3620–3631, Dec. 2016

    2016

  2. [10]

    On optimal power control for energy harvesting communications with lookahead,

    A. Zibaeenejad, S. Yang, and J. Chen, “On optimal power control for energy harvesting communications with lookahead,”IEEE Trans. Wireless Commun., vol. 21, no. 6, pp. 4054–4067, Jun. 2022

    2022

  3. [11]

    On the optimality of the greedy policy for battery limited energy harvesting communications,

    Y . Wang, A. Zibaeenejad, Y . Jing, and J. Chen, “On the optimality of the greedy policy for battery limited energy harvesting communications,” IEEE Trans. Inf. Theory, vol. 67, no. 10, pp. 6548–6563, Oct. 2021

    2021

  4. [12]

    A maximin optimal online power control policy for energy harvesting communications,

    S. Yang and J. Chen, “A maximin optimal online power control policy for energy harvesting communications,” inProc. ICC 2020 - 2020 IEEE International Conference on Communications (ICC). Dublin, Ireland: IEEE, Jun. 2020, pp. 1–6

    2020

  5. [13]

    A maximin optimal online power control policy for energy harvesting communications,

    ——, “A maximin optimal online power control policy for energy harvesting communications,”IEEE Trans. Wireless Commun., vol. 19, no. 10, pp. 6708–6720, Oct. 2020

    2020

  6. [14]

    Optimal power control policy for fading channels with Bernoulli harvested energy,

    A. Khajepour and A. Zibaeenejad, “Optimal power control policy for fading channels with Bernoulli harvested energy,” in2021 Iran Workshop on Communication and Information Theory (IWCIT). Tehran, Iran, Islamic Republic of: IEEE, May 2021, pp. 1–6

    2021

  7. [15]

    Reinforcement learning for energy harvesting point-to-point communications,

    A. Ortiz, H. Al-Shatri, X. Li, T. Weber, and A. Klein, “Reinforcement learning for energy harvesting point-to-point communications,” in2016 IEEE International Conference on Communications (ICC). Kuala Lumpur, Malaysia: IEEE, May 2016, pp. 1–6. 14

    2016

  8. [16]

    Reinforcement learning exploration algorithms for energy harvesting communications systems,

    A. Masadeh, Z. Wang, and A. E. Kamal, “Reinforcement learning exploration algorithms for energy harvesting communications systems,” in2018 IEEE International Conference on Communications (ICC). Kansas City, MO: IEEE, May 2018, pp. 1–6

    2018

  9. [17]

    Action-bounding for reinforce- ment learning in energy harvesting communication systems,

    H. Kim, H. Yang, Y . Kim, and J. Lee, “Action-bounding for reinforce- ment learning in energy harvesting communication systems,” in2018 IEEE Global Communications Conference (GLOBECOM). Abu Dhabi, United Arab Emirates: IEEE, Dec. 2018, pp. 1–7

    2018

  10. [18]

    Deep reinforcement learning optimal transmission policy for communication systems with energy harvesting and adaptive mqam,

    M. Li, X. Zhao, H. Liang, and F. Hu, “Deep reinforcement learning optimal transmission policy for communication systems with energy harvesting and adaptive mqam,”IEEE Trans. Veh. Technol., vol. 68, no. 6, pp. 5782–5793, Jun. 2019

    2019

  11. [19]

    An actor-critic reinforcement learning approach for energy harvesting communications systems,

    A. Masadeh, Z. Wang, and A. E. Kamal, “An actor-critic reinforcement learning approach for energy harvesting communications systems,” in 2019 28th International Conference on Computer Communication and Networks (ICCCN). Valencia, Spain: IEEE, Jul. 2019, pp. 1–6

    2019

  12. [20]

    Deep deterministic policy gradient (DDPG)-based energy harvesting wireless communications,

    C. Qiu, Y . Hu, Y . Chen, and B. Zeng, “Deep deterministic policy gradient (DDPG)-based energy harvesting wireless communications,” IEEE Internet Things J., vol. 6, no. 5, pp. 8577–8588, Oct. 2019

    2019

  13. [21]

    Shallow reinforcement learning for energy harvesting communications with imperfect channel knowledge,

    H. Kim, J. Lee, W. Shin, and H. V . Poor, “Shallow reinforcement learning for energy harvesting communications with imperfect channel knowledge,”IEEE J. Sel. Topics Signal Process., vol. 15, no. 5, pp. 1258–1271, Aug. 2021

    2021

  14. [22]

    Transmis- sion with energy harvesting nodes in fading wireless channels: Optimal policies,

    O. Ozel, K. Tutuncuoglu, J. Yang, S. Ulukus, and A. Yener, “Transmis- sion with energy harvesting nodes in fading wireless channels: Optimal policies,”IEEE J. Sel. Areas Commun., vol. 29, no. 8, pp. 1732–1743, Sep. 2011

    2011

  15. [23]

    Online power control optimization for wireless transmission with energy harvesting and storage,

    F. Amirnavaei and M. Dong, “Online power control optimization for wireless transmission with energy harvesting and storage,”IEEE Trans. Wireless Commun., pp. 4888–4901, 2016

    2016

  16. [24]

    Neural-network-based power control prediction for solar-powered energy harvesting communications,

    M.-L. Ku and T.-J. Lin, “Neural-network-based power control prediction for solar-powered energy harvesting communications,”IEEE Internet Things J., vol. 8, no. 16, pp. 12 983–12 998, Aug. 2021

    2021

  17. [25]

    Tse and P

    D. Tse and P. Viswanath,Fundamentals of Wireless Communication. Cambridge, UK; New York: Cambridge University Press, 2005

    2005

  18. [26]

    Discrete-time controlled Markov processes with average cost criterion: A survey,

    A. Arapostathis, V . S. Borkar, E. Fern ´andez-Gaucherand, M. K. Ghosh, and S. I. Marcus, “Discrete-time controlled Markov processes with average cost criterion: A survey,”SIAM Journal on Control and Op- timization, vol. 31, no. 2, pp. 282–344, Mar. 1993

    1993

  19. [27]

    On linear power control policies for energy harvesting communications,

    H. M. Garmaroudi, Z. Dou, S. Yang, and J. Chen, “On linear power control policies for energy harvesting communications,” 2022. [Online]. Available: https://arxiv.org/abs/2207.10230

    2022

  20. [28]

    Hern ´andez-Lerma and J.-B

    O. Hern ´andez-Lerma and J.-B. Lasserre,Markov Chains and Invariant Probabilities, ser. Progress in Mathematics. Basel ; Boston: Birkh ¨auser, 2003, no. v. 211

    2003

  21. [29]

    Optimum transmission policies for bat- tery limited energy harvesting nodes,

    K. Tutuncuoglu and A. Yener, “Optimum transmission policies for bat- tery limited energy harvesting nodes,”IEEE Trans. Wireless Commun., vol. 11, no. 3, pp. 1180–1189, Mar. 2012

    2012

  22. [30]

    A power control method for energy harvesting wireless communication systems,

    S. Yang and H. Wu, “A power control method for energy harvesting wireless communication systems,” Chinese Patent CN119 854 923B, Jun. 6, 2025

    2025

  23. [31]

    R. S. Sutton and A. G. Barto,Reinforcement Learning: An Introduction, second edition ed., ser. Adaptive Computation and Machine Learning Series. Cambridge, Massachusetts: The MIT Press, 2018

    2018

  24. [32]

    M. L. Puterman,Markov Decision Processes: Discrete Stochastic Dy- namic Programming, ser. Wiley Series in Probability and Statistics. Hoboken, NJ: Wiley-Interscience, 2005

    2005

  25. [33]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,”

  26. [2017]

    Available: https://arxiv.org/abs/1412.6980

    [Online]. Available: https://arxiv.org/abs/1412.6980

    arXiv

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.