A Reinforcement Learning Framework for Optimizing Age-of-Information in RF-powered Communication Systems

Harpreet S. Dhillon; Mohamed A. Abd-Elmagid; Nikolaos Pappas

REVIEW 3 major objections 4 minor 47 references

A Reinforcement Learning Framework for Optimizing Age-of-Information in RF-powered Communication Systems

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper proves that the age-optimal joint wireless-energy-transfer and scheduling policy for RF-powered monitors is threshold-based in each process's age, and provides a deep Q-learning algorithm that learns it.

desk verdict A genuinely useful multi-source RF-powered AoI paper whose structural results are likely right but whose proof as written leans on a false VIA convergence claim and whose DRL optimality is only validated for N=1. read the letter →

arxiv 1908.06367 v1 pith:57H7XY4M submitted 2019-08-18 cs.IT cs.NImath.IT

Mohamed A. Abd-Elmagid , Harpreet S. Dhillon , Nikolaos Pappas This is my paper · ORCID

classification cs.ITcs.NImath.IT MSC 90C40

keywords Age-of-InformationRFenergyharvestingMarkovdecisionprocessreinforcementlearningthresholdpolicywirelesspoweredcommunicationdeepQ-networkstatusupdatescheduling

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks how a destination node that charges battery-equipped sensors over the radio should spend each time slot: broadcast energy to recharge them, or let one sensor transmit a status update, so that the weighted average age of the monitored processes stays smallest. It models this as an average-cost Markov decision process whose state includes each sensor's battery level, the age of each process at the destination, and both downlink and uplink channel gains, then proves the age-optimal policy is threshold-based. The main theorem states that for each process, once its age crosses a threshold while everything else is fixed, transmitting an update for that process becomes optimal and remains optimal for all larger ages. For a single source, the paper proves the optimal action is also threshold-based in battery level and channel gains. This matters because threshold rules are simple to describe and to implement on energy-starved devices, and because the same analysis shows the throughput-optimal policy has a different, non-age-aware structure.

What carries the argument

The load-bearing object is the Q-function of the average-cost MDP, together with the monotonicity of its value function. Lemma 2 proves the value function is non-increasing in each battery level and channel power gain and non-decreasing in each age. Theorem 1 is carried by the comparison inequality $Q(s_2,\bar a)-Q(s_2,a') \le Q(s_1,\bar a)-Q(s_1,a')$, which transfers optimality from a low-age state to a high-age state. The proof relies on the transition-probability factorization $P(s'|s,a)=P(b'|b,g,h,a)P(A'|A,a)P(g')P(h')$, which decouples battery evolution, age evolution, and channel memorylessness. On the computational side, a deep Q-network approximates the Q-function with experience replay and $\epsilon$-greedy exploration, making the large finite state space tractable.

What would settle it

Run exhaustive value iteration on a small instance, such as two sources with six age levels, five battery levels, and quantized channel gains, and check every pair of states satisfying Theorem 1's conditions; if any pair has $T_j$ optimal at the lower-age state but not at the higher-age state, the threshold theorem is false. The same exhaustive check on a single-source instance would test the all-state-variable threshold claims of Theorem 2.

Watch

Extended reading notes

Core claim

The central discovery is that the jointly optimal wireless-energy-transfer and scheduling policy for minimizing long-term average weighted sum-age has a threshold structure in age. Theorem 1 shows: for two system states that are identical except that process $j$'s age is at least as large in the second state, if transmitting process $j$'s update is optimal in the first state, it is also optimal in the second. Thus, for each process, fixing all other state variables, the optimal decision switches at most once as that process's age grows. For a single source-destination pair, Theorem 2 extends the monotonicity to all state variables: on states where the battery is sufficiently charged, the optimal action is threshold-based in the battery level, the age, and the downlink and uplink channel gains. The paper also analyzes the policy that maximizes average throughput and shows that, although it is threshold-based as well, it differs from the age-optimal policy exactly when the monitored age is large: the age-optimal rule prioritizes updating stale information even at the cost of throughput.

Load-bearing premise

The argument assumes the downlink and uplink fading gains are independent and identically distributed from slot to slot, so the next channel state does not depend on the current one and the Markov state needs no channel memory.

Editorial extensions

If this is right

A multi-source scheduler can be implemented by maintaining one age threshold per process and transmitting for any process whose age exceeds its threshold, instead of solving the MDP online.
For a single source, the optimal decision rule over the high-battery region is fully described by thresholds in battery level, age, and channel gains.
Age-optimal and throughput-optimal policies coincide when the monitored age is small but diverge when the age is large, so throughput-oriented design is safe only for fresh-state regimes.
The proposed deep Q-learning algorithm approaches the optimal average age in numerical experiments, with the remaining gap caused by exploration.
The achievable average weighted sum-age falls as battery capacity grows and rises as update packet size grows.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the policy is threshold-based, the entire decision rule can in principle be stored as at most one age threshold per process per coarse state configuration, which suggests a low-memory implementation even without the neural network.
The same monotonicity proof should transfer to other freshness metrics with the same age evolution, such as peak age or value-of-information variants, but the paper does not make that claim.
A natural test of the model's limits is to run the same learning algorithm under temporally correlated fading; if the threshold structure persists, the i.i.d. assumption is not essential, and if it breaks, the conclusion is tied to memoryless channels.
The structural difference between age- and throughput-optimal policies suggests a practical heuristic: switch from throughput-based to age-based scheduling only when a process's age is stale, with the crossover point identified by simulation.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper is worth a serious look. The model is new and well-chosen: multiple RF-powered sources with finite batteries, i.i.d. fading, and a jointly optimized WET/scheduling policy. The main analytical results—Theorem 1's threshold structure in each AoI for N sources, Theorem 2's full threshold structure for N=1, and the comparison with the throughput-optimal policy—are the kind of clean structural claims that actually help designers. The paper also does a good job positioning itself against the prior EH/AoI literature, and the citation pattern looks fair. Credit where due: Theorem 1 is a real extension to the multi-source RF-powered case, and the throughput comparison in Section V is a nice addition.

The soft spots are real, though. The stress-test note about the VIA is correct. Equation (20) claims the unnormalized value iteration iterates converge to the V that satisfies the average-cost Bellman equation. For an average-cost MDP, unnormalized VIA typically diverges or drifts; it does not converge to the relative value function. Since the proofs of Lemma 2 and Theorems 1-2 use V(s)=lim V^{(m)}(s), the structural results are not rigorously established as written. The repair is standard (relative value iteration, subtracting a reference-state value at each step), and the authors even mention RVIA in passing, so the fix is likely straightforward. But the supplied proof has a genuine gap, and a referee should insist it be fixed.

The DRL claims are also softer than the paper suggests. The algorithm is a standard DQN variant for average-cost MDPs. For N=1 it is compared against RVIA and matches. For N>1, the learned policy is used to "verify" the threshold structure and to produce design curves, but there is no optimal baseline, so calling it age-optimal is not justified by the experiments. The paper also omits code, data, hyperparameters, and any error bars. That would matter less if the analytical results were fully rigorous, but as is, the empirical verification is doing more work than it can support.

Minor: the weak accessibility condition used in Lemma 1 is asserted without proof. It is probably true for this finite MDP, but it should be shown or cited precisely.

Bottom line: the core ideas are sound and the structural results are likely correct, but the paper needs a revision—fix the VIA/RVIA proof, and either provide reproducible code or temper the DRL optimality claims. This deserves peer review, not a desk reject. I would cite it for Theorem 1 once the proof is cleaned up.

Referee Report

3 major / 4 minor

Summary. The paper studies a discrete-time monitoring system in which N source nodes are powered exclusively by RF energy transfer from a destination node, and each source observes a physical process whose freshness at the destination is measured by an Age-of-Information (AoI) metric. The authors formulate the joint design of wireless energy transfer and update scheduling as a finite-state, finite-action average-cost MDP, and they propose a deep Q-learning algorithm (Algorithm 1) to approximate the age-optimal policy. The main analytical contributions are structural: Theorem 1 claims that the age-optimal policy is threshold-based in each AoI component for general N; Theorem 2 claims that for N=1 the optimal policy is threshold-based in all state variables on a battery-sufficient subset of states; Theorem 3 gives an analogous result for the throughput-optimal policy. Numerical experiments illustrate the threshold structure, compare age-optimal and throughput-optimal policies, and evaluate the proposed DRL algorithm against RVIA for N=1.

Significance. If established, the threshold characterization is a valuable result: it gives a compact description of an otherwise high-dimensional optimal policy and permits a concrete comparison between age-optimal and throughput-optimal behavior. The paper is also to be credited for making its modeling assumptions explicit, particularly the i.i.d. quasi-static fading assumption and the finite-state discretization, and for constructing the MDP formulation and Bellman equations in a self-contained way. However, the central analytical proof as written relies on an invalid convergence assertion for unnormalized value iteration, and the numerical verification for N>1 does not clearly state whether the plotted policy is the true optimal policy or a learned approximation. The structural results appear plausible and are likely repairable by switching to relative value iteration, but the proof and the numerical verification need correction before the claims can be accepted.

major comments (3)

[Section IV, Eq. (18)-(20)] The proof of Lemma 2 and the threshold theorems is based on unnormalized value iteration, and Eq. (20) states that the iterates V^(m)(s) converge to a solution V(s) of the average-cost Bellman equation (12). For an average-cost finite MDP this is false in general: for a single-state, single-action MDP with cost 1, the iteration gives V^(m)=m, so the limit is not finite. Since Lemma 2, Theorem 1, Theorem 2, and Theorem 3 in Section V all use V(s)=lim_m V^(m)(s), the structural results are not rigorously established as written. The repair is standard: replace the VIA step with relative value iteration, e.g., W^(m+1)(s) = T W^(m)(s) - T W^(m)(s_ref) for a fixed reference state s_ref, and initialize W^(0) as a constant so that the induction base holds. Because the subtracted term is independent of s, the monotonicity comparisons in (21)-(24) are preserved. This is a required fix, not merely a presentational change.
[Section VI-A, Figs. 2-3] The text describes these figures as presenting "the structure of the age-optimal policy," but it does not state how the plotted actions were computed. For the N=2 case, the parameters in the caption give about 1.68 million states (per source: 6 AoI values, 6 battery levels, 6 downlink channel levels, 6 uplink channel levels), so it is not clear whether the figure comes from an exact RVIA computation or from the deep Q-network. If the actions are generated by the learned DRL policy, the figure demonstrates only that the learned policy has threshold-like behavior; it does not verify Theorem 1 for the true optimal policy. The authors should state the computation method explicitly, and for any N>1 verification they should either use the exact RVIA solution on a tractable instance or clearly label the plot as a property of the learned policy.
[Section III-C, Algorithm 1 and Eq. (15)-(16)] The paper claims that the proposed DRL algorithm learns the age-optimal policy in a computationally efficient manner, but no convergence guarantee is provided for the average-cost deep Q-learning update when the Q-function is approximated by a neural network. The target in (16) includes min_{a in A(\bar s)} Q_k(\bar s,a|\beta_k), which is a moving target that depends on the current network weights; with function approximation, the tabular Q-learning convergence conditions cited in the paper do not apply, and the stopping criterion "until convergence" in Algorithm 1 is not specified. The empirical evidence in Fig. 5 is limited to a single N=1 instance and shows only a small gap, not proof of convergence to the optimal policy. The authors should either add formal conditions under which convergence to pi* is guaranteed or explicitly state that the convergence of the DRL algorithm is an empirical observation rather than a proven property.

minor comments (4)

[Section IV, proof of Lemma 2] The monotonicity of V with respect to g_j and h_j is asserted in a single sentence rather than proved by the same induction used for b_j; please expand the argument, including the effect of the changing action feasibility when h_j changes, or state that the proof follows by repeating the b_j argument verbatim.
[Section V-B, Remark 4] In the sentence beginning "according to Lemma 3, we note that pi*(s)=H," the reference to Lemma 3 appears to be a typo; the conclusion follows from the definition of \bar A_th,1 and the threshold property of the age-optimal policy, not from the throughput-optimal value-function lemma.
[Section III-B, Lemma 1] The assertion that the weak accessibility condition holds, and hence that a solution to Bellman's equations exists, is made without proof or a specific citation; a short justification or a precise reference to the corresponding condition in [45] would strengthen the formulation.
[Page 3, Section I-A] There is a recurring typo "throughout-optimal policies" that should read "throughput-optimal policies."

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the threshold theorems follow from the Bellman optimality equations rather than being fit, assumed, or imported from self-citations.

full rationale

The paper's central claim is that the age-optimal policy for the RF-powered monitoring MDP has a threshold structure (Theorems 1 and 2). This is derived analytically: Lemma 2 establishes monotonicity of the value function from the Bellman equation (12) and the transition model (8)-(11), and Theorem 1 then proves the AoI-threshold property from Q-function differences (25)-(26), while Theorem 2 extends the argument to all state variables in the single-source case using Lemma 2. No parameter is fitted to a subset of data and then presented as a prediction: the structural properties are consequences of the Bellman optimality conditions, not of the DRL algorithm. The DRL algorithm in Section III-C is a computational method for the same average-cost MDP, and the numerical plots in Section VI-A are corroborative illustrations of the analytically proven structure rather than the source of the proof. The self-citations in the references, including the conference version [1] and background articles [2], [18], [20], are not load-bearing: no uniqueness theorem and no ansatz is imported from them to force the threshold conclusion. The quasi-static iid fading assumption (Section II-A, Eq. (8)) is an explicitly stated modeling premise, not a definitional circularity. The stronger concern in the proof is the correctness of Eq. (20), where unnormalized VIA iterates are asserted to converge to the value function of an average-cost MDP; that is a mathematical correctness issue, not circularity. Overall, the derivation chain is self-contained and the score is 0.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central claims rest on standard MDP theory plus domain modeling assumptions about i.i.d. fading, generate-at-will sampling, finite discretization, and perfect CSI knowledge. No free parameters are fitted to data; the DRL hyperparameters are not reported and are not part of the analytical claims.

assumptions (5)

standard math Existence of an optimal stationary deterministic policy and Bellman equations for finite-state average-cost MDPs
Used in Lemma 1 and throughout Section III-B; standard result from Bertsekas [45].
domain assumption Channel gains g_i and h_i are i.i.d. across slots with known distributions
Section II-A; enables factorized transition (8) and all monotonicity proofs.
domain assumption Generate-at-will: a scheduled source creates a fresh update at slot start; AoI resets to 1 after a successful transmission
Section II-B, Eq. (4); used in the Q-function expressions.
domain assumption Finite battery with energy quanta (ceil for transmission cost, floor for harvest) and finite AoI cap Amax
Section III-B; makes state space finite, necessary for average-cost MDP results.
domain assumption Destination has perfect current CSI and statistical future CSI; P is large enough that cross-source uplink harvesting is negligible
Section II-A and Eq. (1); simplifies harvested energy to eta*P*g_i and supports the MDP action space.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A Reinforcement Learning Framework for Optimizing Age-of-Information in RF-powered Communication Systems." pith.science (2026). https://pith.science/paper/57H7XY4M

@misc{pith2026190806367,
  author       = {Pith},
  title        = {Pith review of: A Reinforcement Learning Framework for Optimizing Age-of-Information in RF-powered Communication Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/57H7XY4M}},
  note         = {Machine review of arXiv:1908.06367}
}

read the original abstract

In this paper, we study a real-time monitoring system in which multiple source nodes are responsible for sending update packets to a common destination node in order to maintain the freshness of information at the destination. Since it may not always be feasible to replace or recharge batteries in all source nodes, we consider that the nodes are powered through wireless energy transfer (WET) by the destination. For this system setup, we investigate the optimal online sampling policy (referred to as the age-optimal policy) that jointly optimizes WET and scheduling of update packet transmissions with the objective of minimizing the long-term average weighted sum of Age-of-Information (AoI) values for different physical processes (observed by the source nodes) at the destination node, referred to as the sum-AoI. To solve this optimization problem, we first model this setup as an average cost Markov decision process (MDP). Due to the extreme curse of dimensionality in the state space of the formulated MDP, classical reinforcement learning algorithms are no longer applicable to our problem. Motivated by this, we propose a deep reinforcement learning (DRL) algorithm that can learn the age-optimal policy in a computationally-efficient manner. We further characterize the structural properties of the age-optimal policy analytically, and demonstrate that it has a threshold-based structure with respect to the AoI values for different processes. We extend our analysis to characterize the structural properties of the policy that maximizes average throughput for our system setup, referred to as the throughput-optimal policy. Afterwards, we analytically demonstrate that the structures of the age-optimal and throughput-optimal policies are different. We also numerically demonstrate these structures as well as the impact of system design parameters on the optimal achievable average weighted sum-AoI.

Figures

Figures reproduced from arXiv: 1908.06367 by the authors.

**Figure 1.** AoI evolution vs. time when N = 1 and Amax,1 = 4. III. PROBLEM FORMULATION AND PROPOSED SOLUTION A. Problem Statement Our objective is to obtain the optimal policy, which specifies the actions taken at different states of the system over time, achieving the minimum average weighted sum-AoI, i.e., sum of AoI values for different processes at the destination. Particularly, a policy π = {π0, π1, · · · } is a sequence … view at source ↗

**Figure 2.** Structure of the age-optimal policy when [PITH_FULL_IMAGE:figures/full_fig_p023_2.png] view at source ↗

**Figure 3.** Structure of the age-optimal policy when [PITH_FULL_IMAGE:figures/full_fig_p024_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Comparison between the age-optimal and throughput-optimal policies when [PITH_FULL_IMAGE:figures/full_fig_p024_4.png]

**Figure 5.** Figure 5: Convergence of deep reinforcement learning algorithm when [PITH_FULL_IMAGE:figures/full_fig_p026_5.png]

**Figure 6.** Figure 6: Impact of size of update packets and capacity of batteries on the achievable average weighted sum-AoI by the deep [PITH_FULL_IMAGE:figures/full_fig_p027_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 43 canonical work pages

[1]

Online age-minimal sampling policy for RF-powered IoT networks,

M. A. Abd-Elmagid, H. S. Dhillon, and N. Pappas, “Online age-minimal sampling policy for RF-powered IoT networks,” to be presented in IEEE Globecom , Dec. 2019

work page 2019
[2]

On the role of age-of-information in internet of things,

M. A. Abd-Elmagid, N. Pappas, and H. S. Dhillon, “On the role of age-of-information in internet of things,” IEEE Commun. Magazine, to appear. Available online: arxiv.org/abs/1812.08286

arXiv
[3]

Real-time status: How often should one update?

S. Kaul, R. Yates, and M. Gruteser, “Real-time status: How often should one update?” in Proc., IEEE INFOCOM , 2012

work page 2012
[4]

Real-time status updating: Multiple sources,

R. D. Yates and S. Kaul, “Real-time status updating: Multiple sources,” in Proc., IEEE Intl. Symposium on Information Theory, 2012

work page 2012
[5]

Age of information under random updates,

C. Kam, S. Kompella, and A. Ephremides, “Age of information under random updates,” in Proc., IEEE Intl. Symposium on Information Theory , 2013

work page 2013
[6]

Optimizing age-of-information in a multi-class queueing system,

L. Huang and E. Modiano, “Optimizing age-of-information in a multi-class queueing system,” in Proc., IEEE Intl. Symposium on Information Theory , 2015

work page 2015
[7]

On the age of information in status update systems with packet management,

M. Costa, M. Codreanu, and A. Ephremides, “On the age of information in status update systems with packet management,” IEEE Trans. on Info. Theory , vol. 62, no. 4, pp. 1897–1910, 2016

work page 1910
[8]

Age-of-information in the presence of error,

K. Chen and L. Huang, “Age-of-information in the presence of error,” in Proc., IEEE Intl. Symposium on Information Theory, 2016

work page 2016

Show all 47 references

[9]

Is the zero-wait policy always optimum for information freshness (peak age) or throughput?

B. Barakat, S. Keates, I. Wassell, and K. Arshad, “Is the zero-wait policy always optimum for information freshness (peak age) or throughput?” IEEE Commun. Letters , vol. 23, no. 6, pp. 987–990, June 2019

2019
[10]

Age and value of information: Non-linear age case,

A. Kosta, N. Pappas, A. Ephremides, and V . Angelakis, “Age and value of information: Non-linear age case,” in Proc., IEEE Intl. Symposium on Information Theory , 2017

2017
[11]

Age of information in multiple sensing of a single source,

A. Javani and Z. Wang, “Age of information in multiple sensing of a single source,” 2019, available online: arxiv.org/abs/1902.01975

2019 arXiv
[12]

Update or wait: How to keep your data fresh,

Y . Sun, E. Uysal-Biyikoglu, R. D. Yates, C. E. Koksal, and N. B. Shroff, “Update or wait: How to keep your data fresh,” IEEE Trans. on Info. Theory , vol. 63, no. 11, pp. 7492–7508, 2017

2017
[13]

Optimizing data freshness, throughput, and delay in multi-server information- update systems,

A. M. Bedewy, Y . Sun, and N. B. Shroff, “Optimizing data freshness, throughput, and delay in multi-server information- update systems,” in Proc., IEEE Intl. Symposium on Information Theory , 2016

2016
[14]

Minimizing the age of information in broadcast wireless networks,

I. Kadota, E. Uysal-Biyikoglu, R. Singh, and E. Modiano, “Minimizing the age of information in broadcast wireless networks,” in Proc., Allerton Conf. on Commun., Control, and Computing , 2016. 29

2016
[15]

Beneﬁts of coding on age of information in broadcast networks,

X. Chen and S. S. Bidokhti, “Beneﬁts of coding on age of information in broadcast networks,” 2019, available online: arxiv.org/abs/1904.10077

2019 arXiv
[16]

Minimizing age-of-information in multi-hop wireless networks,

R. Talak, S. Karaman, and E. Modiano, “Minimizing age-of-information in multi-hop wireless networks,” in Proc., Allerton Conf. on Commun., Control, and Computing , 2017

2017
[17]

Maximizing energy efﬁciency of cognitive wireless sensor networks with constrained age of information,

A. Valehi and A. Razi, “Maximizing energy efﬁciency of cognitive wireless sensor networks with constrained age of information,” IEEE Trans. on Cognitive Commun. and Networking , vol. 3, no. 4, pp. 643–654, Dec 2017

2017
[18]

Average peak age-of-information minimization in UA V-assisted IoT networks,

M. A. Abd-Elmagid and H. S. Dhillon, “Average peak age-of-information minimization in UA V-assisted IoT networks,” IEEE Trans. on Veh. Technology, vol. 68, no. 2, pp. 2003–2008, Feb. 2019

2003
[19]

Age-optimal trajectory planning for UA V-assisted data collection,

J. Liu, X. Wang, B. Bai, and H. Dai, “Age-optimal trajectory planning for UA V-assisted data collection,” in Proc., IEEE INFOCOM Workshops, 2018

2018
[20]

Deep reinforcement learning for minimizing age-of- information in UA V-assisted networks,

M. A. Abd-Elmagid, A. Ferdowsi, H. S. Dhillon, and W. Saad, “Deep reinforcement learning for minimizing age-of- information in UA V-assisted networks,”to be presented in IEEE Globecom , Dec. 2019

2019
[21]

Timely status update in internet of things monitoring systems: An age-energy tradeoff,

Y . Gu, H. Chen, Y . Zhou, Y . Li, and B. Vucetic, “Timely status update in internet of things monitoring systems: An age-energy tradeoff,” IEEE Internet of Things Journal , vol. 6, no. 3, pp. 5324–5335, June 2019

2019
[22]

Joint status sampling and updating for minimizing age of information in the Internet of Things,

B. Zhou and W. Saad, “Joint status sampling and updating for minimizing age of information in the Internet of Things,” IEEE Tran. on Commun. , to appear. Available online: arxiv.org/abs/1807.04356

arXiv
[23]

Ultra-reliable low-latency vehicular networks: Taming the age of information tail,

M. K. Abdel-Aziz, C.-F. Liu, S. Samarakoon, M. Bennis, and W. Saad, “Ultra-reliable low-latency vehicular networks: Taming the age of information tail,” in Proc., IEEE Globecom , 2018

2018
[24]

Age of information in Two-hop multicast networks,

B. Buyukates, A. Soysal, and S. Ulukus, “Age of information in Two-hop multicast networks,” in Proc., IEEE Asilomar , 2018

2018
[25]

Timely updates over an erasure channel,

R. D. Yates, E. Najm, E. Soljanin, and J. Zhong, “Timely updates over an erasure channel,” in Proc., IEEE Intl. Symposium on Information Theory) , 2017

2017
[26]

Energy efﬁcient and throughput optimal CSMA scheme,

A. Maatouk, M. Assaad, and A. Ephremides, “Energy efﬁcient and throughput optimal CSMA scheme,” IEEE/ACM Trans. on Networking, vol. 27, no. 1, pp. 316–329, Feb 2019

2019
[27]

Minimizing age of information with soft updates,

M. Bastopcu and S. Ulukus, “Minimizing age of information with soft updates,” 2018, available online: arxiv.org/abs/1812.08148

2018 arXiv
[28]

Age-based scheduling: Improving data freshness for wireless real-time trafﬁc,

N. Lu, B. Ji, and B. Li, “Age-based scheduling: Improving data freshness for wireless real-time trafﬁc,” in ACM Intl. Symposium on Mobile Ad Hoc Networking and Computing , 2018, pp. 191–200

2018
[29]

Age minimization of multiple ﬂows using reinforcement learning,

H. B. Beytur and E. Uysal, “Age minimization of multiple ﬂows using reinforcement learning,” in Intl. Conf. on Computing, Networking and Commun. (ICNC) , 2019

2019
[30]

Sampling for remote estimation through queues: Age of information and beyond,

T. Z. Ornee and Y . Sun, “Sampling for remote estimation through queues: Age of information and beyond,” 2019, available online: arxiv.org/abs/1902.03552

2019 arXiv
[31]

Average age of information with hybrid ARQ under a resource constraint,

E. T. Ceran, D. G ¨und¨uz, and A. Gy ¨orgy, “Average age of information with hybrid ARQ under a resource constraint,” IEEE Trans. on Wireless Commun. , vol. 18, no. 3, pp. 1900–1913, March 2019

1900
[32]

Lazy is timely: Status updates by an energy harvesting source,

R. D. Yates, “Lazy is timely: Status updates by an energy harvesting source,” in Proc., IEEE Intl. Symposium on Information Theory, 2015

2015
[33]

Timely updates in energy harvesting two-hop networks: Ofﬂine and online policies,

A. Arafa and S. Ulukus, “Timely updates in energy harvesting two-hop networks: Ofﬂine and online policies,” IEEE Trans. on Wireless Commun., vol. 18, no. 8, pp. 4017–4030, Aug. 2019

2019
[34]

Age of information under energy replenishment constraints,

B. T. Bacinoglu, E. T. Ceran, and E. Uysal-Biyikoglu, “Age of information under energy replenishment constraints,” in Proc., Information Theory and its Applications (ITA) , 2015, pp. 25–31

2015
[35]

Age of information minimization for an energy harvesting source with updating erasures: With and without feedback,

S. Feng and J. Yang, “Age of information minimization for an energy harvesting source with updating erasures: With and without feedback,” 2018, available online: arxiv.org/abs/1808.05141. 30

2018 arXiv
[36]

Age-minimal transmission for energy harvesting sensors with ﬁnite batteries: Online policies,

A. Arafa, J. Yang, S. Ulukus, and H. V . Poor, “Age-minimal transmission for energy harvesting sensors with ﬁnite batteries: Online policies,” 2018, available online: arxiv.org/abs/1806.07271

2018 arXiv
[37]

Achieving the age-energy tradeoff with a ﬁnite-battery energy harvesting source,

B. T. Bacinoglu, Y . Sun, E. Uysal-Bivikoglu, and V . Mutlu, “Achieving the age-energy tradeoff with a ﬁnite-battery energy harvesting source,” in Proc., IEEE Intl. Symposium on Information Theory , 2018

2018
[38]

Sening information through status updates,

A. Baknina, S. Ulukus, O. Oze, J. Yang, and A. Yener, “Sening information through status updates,” in Proc., IEEE Intl. Symposium on Information Theory , 2018, pp. 2271–2275

2018
[39]

Optimal status update for age of information minimization with an energy harvesting source,

X. Wu, J. Yang, and J. Wu, “Optimal status update for age of information minimization with an energy harvesting source,” IEEE Trans. on Green Commun. and Networking , vol. 2, no. 1, pp. 193–204, 2018

2018
[40]

Average age of information for status update systems with an energy harvesting server,

S. Farazi, A. G. Klein, and D. R. Brown, “Average age of information for status update systems with an energy harvesting server,” in Proc., IEEE INFOCOM Workshops , 2018

2018
[41]

Control of status updates for energy harvesting devices that monitor processes with alarms,

G. Stamatakis, N. Pappas, and A. Traganitis, “Control of status updates for energy harvesting devices that monitor processes with alarms,” 2019, available online: arxiv.org/abs/1907.03826

2019 arXiv
[42]

Age of information in a multiple access channel with heterogeneous trafﬁc and an energy harvesting node,

Z. Chen, N. Pappas, E. Bj ¨ornson, and E. G. Larsson, “Age of information in a multiple access channel with heterogeneous trafﬁc and an energy harvesting node,” in Proc., IEEE INFOCOM Workshops , 2019

2019
[43]

Online transmission policy in wireless powered networks with urgency-aware age of information,

Y . Lu, K. Xiong, P. Fan, Z. Zhong, and K. B. Letaief, “Online transmission policy in wireless powered networks with urgency-aware age of information,” in Proc., Intl. Wireless Commun. Mobile Computing Conf. , 2019

2019
[44]

Average age of information in wireless powered sensor networks,

I. Krikidis, “Average age of information in wireless powered sensor networks,” IEEE Wireless Commun. Letters , 2019

2019
[45]

Dynamic programming and optimal control 3rd edition, volume ii,

D. P. Bertsekas, “Dynamic programming and optimal control 3rd edition, volume ii,” Belmont, MA: Athena Scientiﬁc , 2011

2011
[46]

Human-level control through deep reinforcement learning,

V . Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis, “Human-level control through deep...

2015
[47]

W. B. Powell, Approximate Dynamic Programming: Solving the curses of dimensionality . John Wiley & Sons, 2007, vol. 703

2007

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Online age-minimal sampling policy for RF-powered IoT networks,

M. A. Abd-Elmagid, H. S. Dhillon, and N. Pappas, “Online age-minimal sampling policy for RF-powered IoT networks,” to be presented in IEEE Globecom , Dec. 2019

work page 2019

[2] [2]

On the role of age-of-information in internet of things,

M. A. Abd-Elmagid, N. Pappas, and H. S. Dhillon, “On the role of age-of-information in internet of things,” IEEE Commun. Magazine, to appear. Available online: arxiv.org/abs/1812.08286

arXiv

[3] [3]

Real-time status: How often should one update?

S. Kaul, R. Yates, and M. Gruteser, “Real-time status: How often should one update?” in Proc., IEEE INFOCOM , 2012

work page 2012

[4] [4]

Real-time status updating: Multiple sources,

R. D. Yates and S. Kaul, “Real-time status updating: Multiple sources,” in Proc., IEEE Intl. Symposium on Information Theory, 2012

work page 2012

[5] [5]

Age of information under random updates,

C. Kam, S. Kompella, and A. Ephremides, “Age of information under random updates,” in Proc., IEEE Intl. Symposium on Information Theory , 2013

work page 2013

[6] [6]

Optimizing age-of-information in a multi-class queueing system,

L. Huang and E. Modiano, “Optimizing age-of-information in a multi-class queueing system,” in Proc., IEEE Intl. Symposium on Information Theory , 2015

work page 2015

[7] [7]

On the age of information in status update systems with packet management,

M. Costa, M. Codreanu, and A. Ephremides, “On the age of information in status update systems with packet management,” IEEE Trans. on Info. Theory , vol. 62, no. 4, pp. 1897–1910, 2016

work page 1910

[8] [8]

Age-of-information in the presence of error,

K. Chen and L. Huang, “Age-of-information in the presence of error,” in Proc., IEEE Intl. Symposium on Information Theory, 2016

work page 2016