pith. sign in

arxiv: 1907.03826 · v1 · pith:EHZIDB6Hnew · submitted 2019-07-08 · 💻 cs.IT · cs.NI· math.IT

Control of Status Updates for Energy Harvesting Devices that Monitor Processes with Alarms

George Stamatakis , Nikolaos Pappas , Apostolos Traganitis This is my paper

Pith reviewed 2026-05-25 00:35 UTC · model grok-4.3

classification 💻 cs.IT cs.NImath.IT
keywords energy harvestingage of informationstatus updatesMarkov decision processalarm statesstochastic processes
0
0 comments X

The pith

Energy harvesting status update systems derive optimal policies that reserve energy for alarm states.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper derives optimal transmission policies for an energy harvesting device that monitors a stochastic process switching between normal and alarm states. It introduces separate Age of Information variables for each state and models the decision problem as a Markov Decision Process whose cost function applies higher penalties during alarms. Numerical evaluation of the resulting policies demonstrates that they hold back energy in advance of alarm periods. A sympathetic reader would care because the work shows how to manage update timing when energy is scarce and information value spikes during critical states.

Core claim

We derive optimal transmission policies in an energy harvesting status update system. The system monitors a stochastic process which can be either in a normal or in an alarm state of operation. We capture the freshness of status updates for each state by introducing two Age of Information variables and extend the definition of AoI to account for state changes. We formulate the problem as a Markov Decision Process which utilizes a transition cost function that applies linear and non-linear penalties based on AoI and the state. Numerical evaluation illustrates the policies' effectiveness for reserving energy in anticipation of future alarm states.

What carries the argument

Markov Decision Process with two Age of Information variables and a state-dependent transition cost function that applies higher penalties in the alarm state.

If this is right

  • Optimal policies can be obtained by solving the MDP for given energy arrival and state transition statistics.
  • The policies explicitly trade off immediate update cost against the value of stored energy for future alarms.
  • Numerical results confirm that state-aware reservation improves performance over policies that treat both states equally.
  • The dual-AoI extension allows the cost function to penalize staleness differently depending on the current process state.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same MDP structure could be used to study continuous-time versions or systems with more than two process states.
  • The approach may transfer to sensor networks where events trigger higher urgency without an explicit alarm label.
  • Threshold-type policies extracted from the MDP value function might admit simpler online implementations than full dynamic programming.

Load-bearing premise

The demand for status updates is higher when the stochastic process is in the alarm state, which is used to define the transition cost function.

What would settle it

A numerical run or simulation in which the alarm state occurs and the derived policy delivers fewer timely updates than a simple threshold policy that ignores state information.

Figures

Figures reproduced from arXiv: 1907.03826 by Apostolos Traganitis, George Stamatakis, Nikolaos Pappas.

Figure 1
Figure 1. Figure 1: An EH status update system for a stochastic process [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The states of the stochastic process and transition [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The first sub-figure presents the evolution of the [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Impact of transmission success probability [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Impact of different combinations of stochastic process’s [PITH_FULL_IMAGE:figures/full_fig_p005_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Visualization of the optimal actions when the stochastic [PITH_FULL_IMAGE:figures/full_fig_p006_7.png] view at source ↗
read the original abstract

In this work, we derive optimal transmission policies in an energy harvesting status update system. The system monitors a stochastic process which can be either in a normal or in an alarm state of operation. We capture the freshness of status updates for each state of the stochastic process by introducing two Age of Information (AoI) variables and extend the definition of AoI to account for the state changes of the stochastic process. We formulate the problem at hand as a Markov Decision Process which, under the assumption that the demand for status updates is higher when the stochastic process is in the alarm state, utilizes a transition cost function that applies linear and non-linear penalties based on AoI and the state of the stochastic process. Finally, we evaluate numerically the derived policies and illustrate their effectiveness for reserving energy in anticipation of future alarm states.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper derives optimal transmission policies for an energy harvesting status update system monitoring a stochastic process with normal and alarm states. It introduces two AoI variables (one per state) and extends the AoI definition to account for state changes. The problem is formulated as an MDP whose transition cost applies linear and non-linear AoI penalties that are higher in the alarm state (under the explicit modeling assumption of greater update demand then). The derived policies are evaluated numerically to illustrate energy reservation in anticipation of alarms.

Significance. If the MDP derivation and solution are correct, the work contributes a state-dependent extension of AoI together with an energy-aware control policy that anticipates higher-urgency periods. The numerical evaluation, even if illustrative, provides concrete evidence of the reservation behavior and is a positive feature of the manuscript.

major comments (2)
  1. [MDP formulation] MDP formulation section: the state space (two AoI values plus energy level), transition probabilities, and exact form of the linear/non-linear cost function are not specified with sufficient detail or equations to permit reproduction or independent verification of the claimed optimal policies. This is load-bearing for the central claim.
  2. [Numerical results] Numerical results section: no parameter values for the underlying stochastic process, no description of the MDP solver or discretization, and no tabulated policy or cost values are provided, so the illustration of energy-reservation behavior cannot be assessed for robustness or sensitivity.
minor comments (2)
  1. [Abstract / Introduction] The abstract and introduction could more explicitly reference prior AoI work on energy harvesting to situate the two-AoI extension.
  2. [System model] Notation for the two AoI processes and the state-change extension should be introduced with a clear diagram or table.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed review and constructive suggestions. We agree that the MDP formulation and numerical results sections require additional explicit detail to support reproducibility. We will revise the manuscript accordingly.

read point-by-point responses

  1. Referee: [MDP formulation] MDP formulation section: the state space (two AoI values plus energy level), transition probabilities, and exact form of the linear/non-linear cost function are not specified with sufficient detail or equations to permit reproduction or independent verification of the claimed optimal policies. This is load-bearing for the central claim.

    Authors: We agree that the current presentation does not provide sufficient equations for independent verification. In the revised manuscript we will add an explicit definition of the state space S = {(Δ_n, Δ_a, E)}, where Δ_n and Δ_a are the two AoI processes and E is the discrete energy level. We will also state the transition probabilities p(s'|s,a) derived from the underlying two-state Markov chain of the monitored process and give the precise cost function c(s,a), which applies a linear penalty αΔ_n when the process is normal and a state-dependent non-linear penalty β(Δ_a)^2 when the process is in alarm (with β > α reflecting higher demand). revision: yes

  2. Referee: [Numerical results] Numerical results section: no parameter values for the underlying stochastic process, no description of the MDP solver or discretization, and no tabulated policy or cost values are provided, so the illustration of energy-reservation behavior cannot be assessed for robustness or sensitivity.

    Authors: We accept that the numerical section is currently illustrative only and lacks the requested specifics. In the revision we will supply the concrete parameter values (transition probabilities of the normal/alarm Markov chain, energy arrival rate, penalty coefficients α and β), describe the solution method (relative value iteration on a discretized state space), and include a table of selected optimal actions and long-run average costs for representative energy levels to allow assessment of the energy-reservation behavior. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation is self-contained

full rationale

The paper formulates an MDP with an explicitly declared cost function (linear/non-linear AoI penalties weighted by alarm vs. normal state) and derives optimal policies via standard MDP solution methods. The assumption on update demand is stated upfront and used to define the cost; no parameter is fitted to data and then renamed as a prediction, no self-citation chain supports a load-bearing uniqueness claim, and no equation reduces to its own input by construction. Numerical evaluation is presented only as illustration. The derivation chain therefore stands on the stated model without internal reduction.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the domain assumption that update demand is higher in the alarm state and on the MDP formulation itself; no free parameters or invented entities are mentioned.

axioms (1)
  • domain assumption Demand for status updates is higher when the stochastic process is in the alarm state
    Invoked to construct the transition cost function with linear and non-linear penalties.

pith-pipeline@v0.9.0 · 5674 in / 1017 out tokens · 20081 ms · 2026-05-25T00:35:48.163535+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

23 extracted references · 23 canonical work pages · 2 internal anchors

  1. [1]

    Research directions for the internet of things,

    J. A. Stankovic, “Research directions for the internet of things,” IEEE Internet of Things Journal , vol. 1, no. 1, pp. 3–9, 2014

    work page 2014

  2. [2]

    Worldwide and regional internet of things (IoT) 2014–2020 forecast: A virtuous circle of proven value and demand,

    D. Lund, C. MacGillivray, V . Turner, and M. Morales, “Worldwide and regional internet of things (IoT) 2014–2020 forecast: A virtuous circle of proven value and demand,” International Data Corporation (IDC), Tech. Rep, vol. 1, 2014

    work page 2014

  3. [3]

    Real-time status: How often should one update?

    S. Kaul, R. Yates, and M. Gruteser, “Real-time status: How often should one update?” in IEEE INFOCOM , March 2012

    work page 2012

  4. [4]

    Remote estimation of the wiener process over a channel with random delay,

    Y . Sun, Y . Polyanskiy, and E. Uysal-Biyikoglu, “Remote estimation of the wiener process over a channel with random delay,” in IEEE ISIT , June 2017

    work page 2017

  5. [5]

    Update or wait: How to keep your data fresh,

    Y . Sun, E. Uysal-Biyikoglu, R. D. Yates, C. E. Koksal, and N. B. Shroff, “Update or wait: How to keep your data fresh,” IEEE Transactions on Information Theory , vol. 63, no. 11, pp. 7492–7508, November 2017

    work page 2017

  6. [6]

    Optimal Policies for Status Update Generation in a Wireless System with Heterogeneous Traffic

    G. Stamatakis, N. Pappas, and A. Traganitis, “Optimal policies for status update generation in a wireless system with heterogeneous traffic,” arXiv:1810.03201, 2018

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  7. [7]

    Age and value of information: Non-linear age case,

    A. Kosta, N. Pappas, A. Ephremides, and V . Angelakis, “Age and value of information: Non-linear age case,” in IEEE ISIT , June 2017

    work page 2017

  8. [8]

    Sampling for data freshness optimization: Non- linear age functions,

    Y . Sun and B. Cyr, “Sampling for data freshness optimization: Non- linear age functions,” Journal of Communications and Networks (JCN) , vol. 21, no. 3, pp. 204–219, June 2019

    work page 2019

  9. [9]

    Closed-Form Analysis of Non-Linear Age-of-Information in Status Updates with an Energy Harvesting Transmitter

    X. Zheng, S. Zhou, Z. Jiang, and Z. Niu, “Closed-form analysis of non- linear age-of-information in status updates with an energy harvesting transmitter,” arXiv preprint arXiv:1906.00192 , 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1906

  10. [10]

    Wireless energy harvesting for the internet of things,

    P. Kamalinejad, C. Mahapatra, Z. Sheng, S. Mirabbasi, V . C. M. Leung, and Y . L. Guan, “Wireless energy harvesting for the internet of things,” IEEE Communications Magazine , vol. 53, no. 6, pp. 102–108, June 2015

    work page 2015

  11. [11]

    Internet of hybrid energy harvesting things,

    O. B. Akan, O. Cetinkaya, C. Koca, and M. Ozger, “Internet of hybrid energy harvesting things,” IEEE Internet of Things Journal , vol. 5, no. 2, pp. 736–746, 2017

    work page 2017

  12. [12]

    Age of informa- tion under energy replenishment constraints,

    B. T. Bacinoglu, E. T. Ceran, and E. Uysal-Biyikoglu, “Age of informa- tion under energy replenishment constraints,” in ITA, February 2015

    work page 2015

  13. [13]

    Lazy is timely: Status updates by an energy harvesting source,

    R. D. Yates, “Lazy is timely: Status updates by an energy harvesting source,” in IEEE ISIT , June 2015

    work page 2015

  14. [14]

    Age-minimal transmission in energy harvesting two-hop networks,

    A. Arafa and S. Ulukus, “Age-minimal transmission in energy harvesting two-hop networks,” in IEEE GLOBECOM , December 2017

    work page 2017

  15. [15]

    Scheduling status updates to minimize age of information with an energy harvesting sensor,

    B. T. Bacinoglu and E. Uysal-Biyikoglu, “Scheduling status updates to minimize age of information with an energy harvesting sensor,” in IEEE ISIT, June 2017

    work page 2017

  16. [16]

    Age-minimal online policies for energy harvesting sensors with random battery recharges,

    A. Arafa, J. Yang, and S. Ulukus, “Age-minimal online policies for energy harvesting sensors with random battery recharges,” in IEEE ICC, May 2018

    work page 2018

  17. [17]

    Optimal status update for age of information minimization with an energy harvesting source,

    X. Wu, J. Yang, and J. Wu, “Optimal status update for age of information minimization with an energy harvesting source,” IEEE Transactions on Green Communications and Networking , vol. 2, no. 1, pp. 193–204, March 2018

    work page 2018

  18. [18]

    Average age of information for status update systems with an energy harvesting server,

    S. Farazi, A. G. Klein, and D. R. Brown, “Average age of information for status update systems with an energy harvesting server,” in IEEE INFOCOM Workshops, April 2018

    work page 2018

  19. [19]

    On the role of age-of-information in internet of things,

    M. A. Abd-Elmagid, N. Pappas, and H. S. Dhillon, “On the role of age-of-information in internet of things,” arXiv:1812.08286, 2018

    work page arXiv 2018

  20. [20]

    Average age of information in wireless powered sensor networks,

    I. Krikidis, “Average age of information in wireless powered sensor networks,” IEEE Wireless Communications Letters , vol. 8, no. 2, pp. 628–631, April 2019

    work page 2019

  21. [21]

    Age of information in a multiple access channel with heterogeneous traffic and an energy harvesting node,

    Z. Chen, N. Pappas, E. Bj ¨ornson, and E. G. Larsson, “Age of information in a multiple access channel with heterogeneous traffic and an energy harvesting node,” in IEEE INFOCOM Workshops , April 2019

    work page 2019

  22. [22]

    Reinforcement learning to minimize age of information with an energy harvesting sensor with HARQ and sensing cost,

    E. T. Ceran, D. G ¨und¨uz, and A. Gy ¨orgy, “Reinforcement learning to minimize age of information with an energy harvesting sensor with HARQ and sensing cost,” in IEEE INFOCOM Workshops , April 2019

    work page 2019

  23. [23]

    Bertsekas, Dynamic Programming and Optimal Control, V ol

    D. Bertsekas, Dynamic Programming and Optimal Control, V ol. II . Athena Scientific Belmont, MA, 2012, vol. 2

    work page 2012