Pith. sign in

REVIEW 2 major objections 7 minor 41 references

Age of Incorrect Information for Pull-Based State Estimation of General Markov Sources

T0 review · 2 major / 7 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read In pull-based estimation of a Markov source, every reachable belief is determined by the last observed state and the elapsed no-success duration, reducing a POMDP to an exactly equivalent countable-state MDP.

desk verdict Theorem 1 is the real thing: under the synchronized exact-revelation reset, every reachable belief reduces to (last observed state, elapsed no-success duration), and the paper turns the pull-based AoII POMDP into a countable fully observed MDP with explicit truncation control. read the letter →

arxiv 2608.13248 v1 pith:QD3NXFIP submitted 2026-08-13 cs.IT cs.NImath.IT

Marco Zanni , Mohamad Assaad , Touraj Soleymani This is my paper · ORCID
classification cs.ITcs.NImath.IT
keywords ageofincorrectinformationpull-basedremotestateestimationMarkovsourcebeliefdecisionprocesspartiallyobservableno-successcyclereductionrestlessmulti-armedbanditAoIItruncationerrorbound
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper studies when a remote monitor should pull updates from a Markov source it cannot see, when the cost is the age of incorrect information (AoII) plus a charge per request. Its central claim is that, under the paper's assumptions, the monitor's full belief state is always determined by two observable numbers: the last source state successfully received and how many time slots have passed without a success. Because those two numbers are enough, the problem becomes an exactly equivalent countable-state Markov decision process rather than a decision process over an uncountable belief space. The paper then proves that truncating the elapsed time at a finite threshold $H$ gives a finite dynamic program with an explicit error bound, and it derives structured near-optimal policies for reliable and unreliable links, as well as a multi-source scheduling extension.

What carries the argument

The load-bearing object is the no-success cycle parameterization $b_{i,n}$, a belief indexed by the last observed source state and the elapsed no-success duration. The parameterization carries the proof because a successful update reveals the exact state and resets the belief to a point mass, while a failed or idle slot is a null observation that leaves the source distribution unchanged; the null-observation recursion sends $b_{i,n}$ to $b_{i,n+1}$, and a successful observation of state $k$ sends it to $b_{k,1}$, keeping the family closed. The reduced MDP's costs and transitions are expressed through the expected posterior AoII $g_i(n)$ and the transition matrix $P$, and the truncation error of Theorem 2 is bounded by explicitly computable quantities $C_n$ and $M_H$.

What would settle it

Run the belief recursion from the paper's Eq. (9) for a fixed, explicitly given transition matrix and enumerate all histories up to a finite depth; if any reachable belief is not of the form $b_{i,n}$, Theorem 1 is false. A structurally different test: replace the perfect-revelation observation in Eq. (3) by a noisy or partial observation and check whether the reachable belief set becomes uncountable, which would confirm the reduction's dependence on exact state revelation.

Watch

Extended reading notes

Core claim

The central claim is Theorem 1: every prior belief reachable from the synchronized initial condition is either $b_0$ or a belief $b_{i,n}$, where $i$ is the last successfully observed source state and $n$ is the number of time slots elapsed without a success. The proof shows that the family $\{b_{i,n}\}$ is closed under both possible observations: a successful transmission revealing state $k$ sends the belief to $b_{k,1}$, and a null observation, an idle slot or a failed pull, sends it to $b_{i,n+1}$. Along the way the paper derives the source marginal $p_i(n)=e_i P^n$ and the expected posterior AoII $g_i(n)$, and it proves $0 \leq g_i(n) \leq n$. Because the observable pair $(i,n)$ is a sufficient state, the pull-based AoII problem is exactly equivalent to a fully observed countable-state MDP on $Y = \mathcal{X} \times \mathbb{N}_{\ge 1}$.

Load-bearing premise

The reduction depends on the monitor starting with exact knowledge of the source, on every successful update revealing the complete source state, and on failed pulls carrying no information beyond an idle slot; if any of these breaks, the closed belief family $b_{i,n}$ need not exist and the countable-state reduction can fail.

Editorial extensions

If this is right

  • A pull-based scheduler can be implemented with a table indexed by $(i,n)$ rather than by a continuous belief; only the latest observed state and the elapsed no-success time need to be stored and updated online.
  • For any prescribed tolerance, truncating the elapsed time at $H$ yields a finite MDP whose optimal value differs from the exact value by at most $\gamma^{H+1}M_H$, so the approximation is controlled by a one-dimensional search over $H$.
  • For reliable links, the optimal policy is a look-up table of waiting times $m_i^*$: after observing state $i$, the monitor stays idle for $m_i^*-1$ slots and then transmits.
  • For unreliable links, persistent transmit-until-success policies come with closed-form regenerative bounds on the suboptimality gap, and the MAP estimate stabilizes to the stationary mode after a finite number of slots, enabling an exact finite-memory estimator table.
  • For multiple independent sources, the reduced state makes each source a countable arm of a restless multi-armed bandit; under a simple condition on the discount factor and success probability the arm is indexable, and an approximate index policy computed from a small number of anchor states matches the exact index policy in the numerical experiments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: if the channel is changed so that transmission success depends on the source state, no-success cycles no longer collapse to $(i,n)$; the paper's own reasoning suggests the countable reduction would need a new closure property and would generally fail.
  • Inference: the same two-parameter state should apply to any freshness or correctness metric that is a deterministic function of the joint distribution of source state and elapsed error duration, not only to AoII.
  • Inference: the numerical projections suggest that for stable, persistent sources the optimal policy is nearly threshold in the expected AoII; a direct test would be to compute the optimal policy for a family of stable chains and check whether a threshold description holds exactly.
  • Inference: the paper gives separate computable bounds for truncation, persistent-policy suboptimality, and early stationary switch; combining them into a single end-to-end memory-versus-performance certificate for a given set of parameters is a natural next step the paper does not take.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 7 minor

Summary. The paper considers pull-based remote state estimation of a general finite-state Markov source under an age-of-incorrect-information (AoII) cost, where the monitor observes the source only through (possibly failed) pull requests and uses MAP estimation. It formulates the problem as a belief-MDP, proves that every prior belief reachable from a synchronized initial condition is parameterized by the last successfully observed state and the elapsed no-success duration, and thus obtains an exact countable-state MDP on X × N≥1. On this basis it develops a finite truncation with an explicit error bound and a rule for selecting the truncation level; for reliable links it shows that an optimal policy is a waiting-time table; for unreliable links it proposes a persistent policy with regenerative performance bounds; it proves finite-time MAP stabilization and introduces a hybrid estimator with a computable value-difference bound; and it extends the framework to multiple sources as a restless multi-armed bandit with a sufficient indexability condition and an approximate Whittle-index policy. Numerical experiments compare the persistent policy, Whittle index policy, approximate Whittle index policy, and random scheduling baselines.

Significance. If the results hold, this is a substantial contribution: it turns a POMDP over a belief simplex into a fully observed countable MDP with a finite, error-controlled approximation, for arbitrary asymmetric multi-state Markov sources rather than only binary or symmetric ones. The paper is strong in several specifics: Theorem 1's closure argument is clean and the resulting reduction is genuinely exact under the stated model; Theorem 2 gives a usable, parameter-based truncation bound; Theorem 3 is a correct contraction argument for the reliable-link case; Lemma 1 and Theorem 6 are internally consistent; and the paper does not fit parameters to data or calibrate predictions to targets. The main qualifications are the scope of the synchronized exact-revelation reset and the correctness of the lower-bound part of the persistent-policy certificate in Theorem 4; these are the subjects of my major comments.

major comments (2)
  1. [§IV-A, Eq. (9) and Eq. (3)] The central reduction that every reachable belief has the form b_{i,n} is proved only for the synchronized point-mass initial belief in Eq. (9) and for successful transmissions that reveal the complete source state in Eq. (3). If the initial belief is non-degenerate, or if a successful packet reveals only a noisy or partial observation of the source, the null-observation recursion is no longer deterministic on a two-parameter family and the countable reduced-state MDP, the truncation bound of Theorem 2, the waiting-time table of Theorem 3, the persistent policy of Theorem 4, and the multi-source index policies all lose their exact foundation. This is a scoping condition rather than an internal inconsistency, but it is the load-bearing boundary of the model. I recommend adding a prominent scope paragraph and making the abstract and conclusions carry the same qualifier, so that 'every reachable belief' is not read as a statement about arbitrary initial beliefs.
  2. [Appendix D-B, Eqs. (105)–(108) and Eq. (54)] The lower bound L_{ν,T} for the optimal policy is obtained by replacing the continuation cost after T unsuccessful slots by V_min. This replacement is a valid relaxation only if the true continuation value from every state (i,T+1) is at least V_min. If V_min is interpreted as min_i V_i(1), the minimum over post-regeneration states, the needed inequality V_i(T+1) ≥ min_j V_j(1) is not proved and is not obviously true in general; if V_min is instead interpreted as a global minimum over all reduced states, then the subsequent vertex-reduction step at the minimizing state i_0 is not covered, because C_{i,T}(u) assumes a cycle starting at n=1. As written, therefore, the proof does not establish that L_{ν,T} is a lower bound on the optimal cost, and the certificate B_{ν,K,T}(m) in Eq. (56) may not be a guaranteed suboptimality gap. Please either prove the needed monotonicity of the value function or modify the relaxation so that the lower bound is valid under the stated assumptions.
minor comments (7)
  1. [Abstract and Conclusions] The abstract and Section X state that 'every reachable belief' is represented by the last observed state and elapsed time, omitting the qualifier 'from the synchronized initial condition' that appears in Theorem 1; please align the wording.
  2. [§IX-B] The text first says the approximate Whittle table is built using 10 anchor beliefs and that the exact table contains 122 beliefs, but later says the AWIP uses 30 anchor beliefs instead of computing indices for all 366 reachable belief states; these numbers should be reconciled.
  3. [§IV-A and §IV-B] There are small editing issues: 'let the timeline' should be 'Let the timeline', and 'N 0' should be typeset as N_{\ge 1} (or N_{\ge 0}, depending on the intended set).
  4. [Table II] The header 'PolicyJ ¯J95% CI half-width forJ' is missing column separators and should be reformatted, e.g., as 'Policy | J | J̄ | 95% CI half-width'.
  5. [Algorithm 2] The pseudocode contains typos such as 'forally' in lines 3 and 12; please correct them.
  6. [Lemma 2 and Theorem 5] Lemma 2 and the bound in Theorem 5 use λ/s and therefore assume s>0, while the model in Eq. (3) allows s=0; please either state s>0 in Section VII or handle the degenerate case separately.
  7. [Theorem 4] Theorem 4 is stated for post-regeneration initial distributions ν, but the original problem starts from the synchronized belief b_0, where V^λ(b_0)=γV_{i_0}(1). The γ factor that scales the gap for b_0 should be stated explicitly so that the theorem is directly applicable to the optimization problem of Section III.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: Theorem 1 and all performance bounds are derived from the model assumptions; the only caveat is an unspecified choice of m in one numerical validation, which is not a circular step.

full rationale

The central reduction (Theorem 1) is derived rather than assumed: starting from the synchronized point-mass initial belief in Eq. (9) and the perfect-revelation channel model in Eq. (3), Appendix A proves by induction that the successor maps U and F close the family {b_i,n}, so the two-parameter representation follows from the information structure of the model. The truncation error bound in Theorem 2 is computed from the stated model parameters using an always-transmit policy to bound the discarded continuation; no parameter is fitted to the quantity being bounded. The reliable-link waiting-time table in Theorem 3 follows from a Bellman contraction argument, and the Whittle indexability condition in Theorem 6 is proved from the relaxed Bellman equations. The multi-source numerical comparisons are against a random baseline and do not fit the optimized index policies. The only mild concern is the persistent-policy validation in Section IX-A: Section VI permits the waiting-time vector m to be chosen either by minimizing the upper bound U_{nu,K}(m) or by solving the original MDP. If the latter option produced the m used in Table II, that particular numerical near-optimality claim would be partly by construction. However, the paper does not state which method was used, so this is an unconfirmed scoping concern and it does not affect the analytical theorems. Self-citations are present but not load-bearing: reference [7] is background, and reference [13] defines AoII; neither supplies the reduction theorem or the bounds.

Assumptions & free parameters 0 free parameters · 7 assumptions · 0 invented entities

No free parameters fitted to data; P, s, gamma, and lambda are system inputs. The approximation parameters H, kappa, and K_j are chosen with explicit error bounds or as algorithmic heuristics, not calibrated to match results. No new physical or mathematical entities are postulated; the hybrid estimator and persistent policy are algorithms, not invented entities.

assumptions (7)
  • domain assumption Synchronized initial condition: b0(i,Delta)=1{i=i0,Delta=0} before time slot 0 (Eq. 9)
    Used to prove Theorem 1; without an exact reset, reachable beliefs need not be of the form b_{i,n} and the countable-state reduction is not established.
  • domain assumption Successful transmission reveals the complete source state and arrives within the same slot; success probability s is constant and independent of source and history (Eq. 3)
    Makes null observations uninformative and no-success cycles deterministic, the mechanism behind the (i,n) parameterization.
  • domain assumption Monitor uses MAP estimator with a fixed deterministic tie-breaking rule (Eqs. 7, 12, 34)
    AoII reset and mismatch matrices D_i(n) are defined through argmax estimates; all bounds are computed for this estimator.
  • domain assumption Source transition matrix P is known to the monitor and time-homogeneous
    The recursion uses e_i P^n and the error bound uses P through the constant q.
  • domain assumption Ergodicity of P and unique mode of the stationary distribution for Section VII
    Lemma 1 and the hybrid estimator rely on e_i P^n converging uniformly to a unique most-likely state; not needed for Theorems 1 and 2.
  • domain assumption Discounted infinite-horizon objective with linear transmission cost lambda (Eqs. 22-24)
    Bellman contraction, the waiting-time theorem, and all bounds require gamma in (0,1) and the additive cost structure.
  • domain assumption Independent source-monitor arms in the multi-source RMAB (Section VIII)
    The Lagrangian decoupling and Whittle index computation rely on independence across arms; coupling or shared channel effects would break the construction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Age of Incorrect Information for Pull-Based State Estimation of General Markov Sources." pith.science (2026). https://pith.science/paper/QD3NXFIP

@misc{pith2026260813248,
  author       = {Pith},
  title        = {Pith review of: Age of Incorrect Information for Pull-Based State Estimation of General Markov Sources},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QD3NXFIP}},
  note         = {Machine review of arXiv:2608.13248}
}
read the original abstract

We study pull-based remote state estimation of an arbitrary, multi-state Markov source while accounting for both freshness and correctness attributes of information. To that end, we formulate a discounted optimization problem in terms of the age of incorrect information (AoII), and express it as a joint source-AoII belief Markov decision process (MDP) under maximum a posteriori (MAP) estimation. We then exploit the information structure of the model and prove that every reachable belief is represented by the last successfully observed source state and the number of time slots elapsed since that observation. For numerical computation, we truncate the elapsed no-success duration at a finite level and derive an explicit error bound and a criterion for selecting the truncation parameter. For reliable links, we show that an optimal policy can be represented by a look-up table of waiting times. For unreliable links, we propose a persistent policy and derive computable performance bounds. We also show that the MAP estimate stabilizes after a finite number of time slots. To further reduce memory requirements, we introduce a hybrid estimator with an early stationary switch and derive a computable bound on the resulting difference in performance. Finally, we extend the framework to multiple sources, formulate the scheduling problem as a restless multi-armed bandit, establish a sufficient condition for indexability, and develop an approximate Whittle index policy based on interpolation. Our numerical results illustrate the structure of the optimal single-source policy, evaluate the performance of the multi-source policies, and verify that the proposed heuristic policies closely approach the optimal solution while substantially reducing computational efforts.

Figures

Figures reproduced from arXiv: 2608.13248 by the authors.

Figure 1
Figure 1. Information flow and update order within one decision slot. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Projection of the optimal single-source action over reachable beliefs for the [PITH_FULL_IMAGE:figures/full_fig_p019_2.png] view at source ↗
Figure 3
Figure 3. Projection of the optimal single-source action over reachable beliefs for the [PITH_FULL_IMAGE:figures/full_fig_p019_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Projection of the optimal single-source action over reachable beliefs for the [PITH_FULL_IMAGE:figures/full_fig_p019_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 40 canonical work pages

  1. [1]

    Real-time status: How often should one update?

    S. Kaul, R. D. Yates, and M. Gruteser, “Real-time status: How often should one update?” inProc. IEEE INFOCOM, 2012, pp. 2731–2735

    2012

  2. [2]

    The age of information: Real-time status updating by multiple sources,

    R. D. Yates and S. K. Kaul, “The age of information: Real-time status updating by multiple sources,”IEEE Transactions on Information Theory, vol. 65, no. 3, pp. 1807–1827, Mar. 2019

    work page 2019

  3. [3]

    Age of information: An introduction and survey,

    R. D. Yates, Y . Sun, D. R. B. III, S. K. Kaul, E. Modiano, and S. Ulukus, “Age of information: An introduction and survey,”IEEE Journal on Selected Areas in Communications, vol. 39, no. 5, pp. 1183–1210, May 2021

    work page 2021

  4. [4]

    Update or wait: How to keep your data fresh,

    Y . Sun, E. Uysal-Biyikoglu, R. D. Yates, C. E. Koksal, and N. B. Shroff, “Update or wait: How to keep your data fresh,”IEEE Transactions on Information Theory, vol. 63, no. 11, pp. 7492–7508, Nov. 2017

    work page 2017

  5. [5]

    Sampling of the Wiener process for remote estimation over a channel with random delay,

    Y . Sun, Y . Polyanskiy, and E. Uysal, “Sampling of the Wiener process for remote estimation over a channel with random delay,”IEEE Transactions on Information Theory, vol. 66, no. 2, pp. 1118–1135, Feb. 2020

    work page 2020

  6. [6]

    Sampling and remote estimation for the Ornstein–Uhlenbeck process through queues: Age of information and beyond,

    T. Z. Ornee and Y . Sun, “Sampling and remote estimation for the Ornstein–Uhlenbeck process through queues: Age of information and beyond,” IEEE/ACM Transactions on Networking, vol. 29, no. 5, pp. 1962–1975, Oct. 2021

    work page 1962

  7. [7]

    Age of Information Optimization for Status Updates in Integrated Sensing and Communication Systems

    M. Zanni, M. Assaad, and T. Soleymani, “Age of information optimization for status updates in integrated sensing and communication systems,”arXiv preprint arXiv:2605.24714, 2026

    work page Pith review arXiv 2026

  8. [8]

    Query age of information: Freshness in pull-based communication,

    F. Chiariotti, J. Holm, A. E. Kalør, B. Soret, S. K. Jensen, T. B. Pedersen, and P. Popovski, “Query age of information: Freshness in pull-based communication,”IEEE Transactions on Communications, vol. 70, no. 3, pp. 1606–1622, Mar. 2022

    work page 2022

Show all 41 references
  1. [9]

    Detecting state transitions of a Markov source: Sampling frequency and age trade-off,

    J. P. Champati, M. Skoglund, M. Jansson, and J. Gross, “Detecting state transitions of a Markov source: Sampling frequency and age trade-off,”IEEE Transactions on Communications, vol. 70, no. 5, pp. 3081–3095, May 2022

    2022

  2. [10]

    Query-based sampling of heterogeneous CTMCs: Modeling and optimization with binary freshness,

    N. Akar and S. Ulukus, “Query-based sampling of heterogeneous CTMCs: Modeling and optimization with binary freshness,”IEEE Transactions on Communications, vol. 72, no. 12, pp. 7705–7714, Dec. 2024

    2024

  3. [11]

    Uncertainty-of-information scheduling: A restless multiarmed bandit framework,

    G. Chen, S. C. Liew, and Y . Shao, “Uncertainty-of-information scheduling: A restless multiarmed bandit framework,”IEEE Transactions on Information Theory, vol. 68, no. 9, pp. 6151–6173, Sep. 2022

    2022

  4. [12]

    An index policy for minimizing the uncertainty-of-information of Markov sources,

    G. Chen and S. C. Liew, “An index policy for minimizing the uncertainty-of-information of Markov sources,”IEEE Transactions on Information Theory, vol. 70, no. 1, pp. 698–721, Jan. 2024

    2024

  5. [13]

    The age of incorrect information: A new performance metric for status updates,

    A. Maatouk, S. Kriouile, M. Assaad, and A. Ephremides, “The age of incorrect information: A new performance metric for status updates,”IEEE/ACM Transactions on Networking, vol. 28, no. 5, pp. 2215–2228, Oct. 2020

    2020

  6. [14]

    Semantic communications in networked systems: A data significance perspective,

    E. Uysal, O. Kaya, A. Ephremides, J. Gross, M. Codreanu, P. Popovski, M. Assaad, G. Liva, A. Munari, B. Soret, T. Soleymany, and K. H. Johansson, “Semantic communications in networked systems: A data significance perspective,”IEEE Network, vol. 36, no. 4, pp. 233–240, Jul./Aug. 2022

    2022

  7. [15]

    On the relationship between mean absolute error and age of incorrect information in the estimation of a piecewise linear signal over noisy channels,

    S. Saha, H. S. Makkar, V . B. Sukumaran, and C. R. Murthy, “On the relationship between mean absolute error and age of incorrect information in the estimation of a piecewise linear signal over noisy channels,”IEEE Communications Letters, vol. 26, no. 11, pp. 2576–2580, Nov. 2022

    2022

  8. [16]

    On the cost of consecutive estimation error: Significance-aware non-linear aging,

    J. Luo and N. Pappas, “On the cost of consecutive estimation error: Significance-aware non-linear aging,”IEEE Transactions on Information Theory, vol. 71, no. 10, pp. 7976–7989, Oct. 2025

    2025

  9. [17]

    The age of incorrect information: An enabler of semantics-empowered communication,

    A. Maatouk, M. Assaad, and A. Ephremides, “The age of incorrect information: An enabler of semantics-empowered communication,”IEEE Transactions on Wireless Communications, vol. 22, no. 4, pp. 2621–2635, Apr. 2023

    2023

  10. [18]

    Minimizing the age of incorrect information for real-time tracking of Markov remote sources,

    S. Kriouile and M. Assaad, “Minimizing the age of incorrect information for real-time tracking of Markov remote sources,” inProc. IEEE International Symposium on Information Theory (ISIT), 2021, pp. 2978–2983

    2021

  11. [19]

    When to pull data from sensors for minimum age of incorrect information,

    S. Kriouile and M. Assaad, “When to pull data from sensors for minimum age of incorrect information,” inProc. International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt), 2023, pp. 603–610

    2023

  12. [20]

    Minimizing age of incorrect information over a channel with random delay,

    Y . Chen and A. Ephremides, “Minimizing age of incorrect information over a channel with random delay,”IEEE/ACM Transactions on Networking, vol. 32, no. 4, pp. 2752–2764, Aug. 2024. 22

    2024

  13. [21]

    Age of incorrect information with hybrid ARQ under a resource constraint for n-ary symmetric Markov sources,

    K. Bountrogiannis, A. Ephremides, P. Tsakalides, and G. Tzagkarakis, “Age of incorrect information with hybrid ARQ under a resource constraint for n-ary symmetric Markov sources,”IEEE Transactions on Networking, vol. 33, no. 2, pp. 640–653, Apr. 2025

    2025

  14. [22]

    Minimizing functions of age of incorrect information for remote estimation,

    I. Cosandal, S. Ulukus, and N. Akar, “Minimizing functions of age of incorrect information for remote estimation,” inProc. IEEE Global Communications Conference (GLOBECOM), 2025, pp. 5417–5422

    2025

  15. [23]

    Semantic-aware sampling and transmission in real-time tracking systems: A POMDP approach,

    A. Zakeri, M. Moltafet, and M. Codreanu, “Semantic-aware sampling and transmission in real-time tracking systems: A POMDP approach,”IEEE Transactions on Communications, vol. 73, no. 7, pp. 4898–4913, Jul. 2025

    2025

  16. [24]

    Remote monitoring of two-state Markov sources via random access channels: An information freshness vs. state estimation entropy perspective,

    G. Cocco, A. Munari, and G. Liva, “Remote monitoring of two-state Markov sources via random access channels: An information freshness vs. state estimation entropy perspective,”IEEE Journal on Selected Areas in Information Theory, vol. 4, pp. 651–666, 2023

    2023

  17. [25]

    Modeling AoII in push- and pull-based sampling of continuous-time Markov chains,

    I. Cosandal, N. Akar, and S. Ulukus, “Modeling AoII in push- and pull-based sampling of continuous-time Markov chains,” inProc. IEEE INFOCOM Workshops, 2024, pp. 267–272

    2024

  18. [26]

    Multi-threshold AoII-optimum sampling policies for continuous-time Markov chain information sources,

    I. Cosandal, N. Akar, and S. Ulukus, “Multi-threshold AoII-optimum sampling policies for continuous-time Markov chain information sources,”IEEE Transactions on Information Theory, vol. 71, no. 9, pp. 6968–6988, Sep. 2025

    2025

  19. [27]

    The optimal control of partially observable Markov processes over a finite horizon,

    R. D. Smallwood and E. J. Sondik, “The optimal control of partially observable Markov processes over a finite horizon,”Operations Research, vol. 21, no. 5, pp. 1071–1088, Sep. 1973

    1973

  20. [28]

    Planning and acting in partially observable stochastic domains,

    L. P. Kaelbling, M. L. Littman, and A. R. Cassandra, “Planning and acting in partially observable stochastic domains,”Artificial Intelligence, vol. 101, no. 1–2, pp. 99–134, May 1998

    1998

  21. [29]

    Partially observable minimum-age scheduling: The greedy policy,

    Y . Shao, Q. Cao, S. C. Liew, and H. Chen, “Partially observable minimum-age scheduling: The greedy policy,”IEEE Transactions on Communications, vol. 70, no. 1, pp. 404–418, Jan. 2022

    2022

  22. [30]

    Pragmatic communication for remote control of finite-state Markov processes,

    P. Talli, E. D. Santi, F. Chiariotti, T. Soleymani, F. Mason, A. Zanella, and D. G ¨und¨uz, “Pragmatic communication for remote control of finite-state Markov processes,”IEEE Journal on Selected Areas in Communications, vol. 43, no. 7, pp. 2589–2603, Jul. 2025

    2025

  23. [31]

    Age of estimates: When to submit jobs to a Markov machine to maximize revenue,

    S. Liyanaarachchi and S. Ulukus, “Age of estimates: When to submit jobs to a Markov machine to maximize revenue,” inProc. International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt), 2026, pp. 1–8

    2026

  24. [32]

    Joint age-state belief is all you need: Minimizing AoII via pull-based remote estimation,

    I. Cosandal, S. Ulukus, and N. Akar, “Joint age-state belief is all you need: Minimizing AoII via pull-based remote estimation,” inProc. IEEE International Conference on Communications Workshops (ICC Workshops), 2025, pp. 1098–1103

    2025

  25. [33]

    Which sensor to observe? timely tracking of a joint Markov source with model predictive control,

    I. Cosandal, S. Ulukus, and N. Akar, “Which sensor to observe? timely tracking of a joint Markov source with model predictive control,” inProc. IEEE International Symposium on Information Theory (ISIT), 2025, pp. 1–6

    2025

  26. [34]

    Multi-stage structured estimators for information freshness,

    S. Liyanaarachchi, S. Ulukus, and N. Akar, “Multi-stage structured estimators for information freshness,” inProc. IEEE INFOCOM, 2026, pp. 1–6

    2026

  27. [35]

    Beyond martingale estimators: Structured estimators for maximizing information freshness in query-based update systems,

    S. Liyanaarachchi, S. Ulukus, and N. Akar, “Beyond martingale estimators: Structured estimators for maximizing information freshness in query-based update systems,”arXiv preprint arXiv:2601.22098, 2026

    2026

  28. [36]

    Restless bandits: Activity allocation in a changing world,

    P. Whittle, “Restless bandits: Activity allocation in a changing world,”Journal of Applied Probability, vol. 25, no. A, pp. 287–298, 1988. APPENDIXA PROOF OFTHEOREM1 We first prove that the set{b i,n}is closed under every possible observation. Suppose that the current belief i...

    1988

  29. [37]

    We examine the process until the next successful transmission or until time slotT, whichever occurs first

    Regenerative relaxation:For every finite horizonT≥1and every statei∈ X, the optimal value satisfies V ∗ i ≥inf u∈[0,1]T {Ci,T (u) + ΓT (u)V ∗ min}.(105) To prove this, fix an admissible policy and suppose that the latest successful transmission observed statei. We examine the ...

  30. [38]

    Vertex reduction:LetA T :={0,1} T . For everyi∈ X, everyz≥0, and everyT≥1, inf u∈[0,1]T {Ci,T (u) + ΓT (u)z}= min a∈AT {Ci,T (a) + ΓT (a)z}(106) and inf u∈[0,1]T Ci,T (u) 1−Γ T (u) = min a∈AT Ci,T (a) 1−Γ T (a) .(107) To prove this, fix all components ofuexceptu r. Since Qn(u)...

  31. [39]

    Regenerative discrepancy bound under a common policy:FixE∈ {map,hyb}and a deterministic stationary policyϕ whose firstRactions in each cycle are represented by a tableu∈ U E R , that is, ui(n) =ϕ(i, n), i∈ X, n= 1, . . . , R. In particular, by Lemma 2, this property is satisfi...

  32. [40]

    By Lemma 2,u ∗ hyb ∈ Uhyb R

    Comparison of the two optimal post-regeneration values:Letϕ ∗ hyb be an optimal deterministic policy for the hybrid model, and letu ∗ hyb be its first-Raction table. By Lemma 2,u ∗ hyb ∈ Uhyb R . Applying the previous bound to the common policyϕ ∗ hyb and then maximizing overU...

  33. [41]

    Hence, the passive action must be optimal for both estimators and V ∗ map(b0) =γV map i0 (1), V ∗ hyb,κ(b0) =γV hyb i0 (1)

    Synchronized initial belief:Atb 0, the passive action has zero cost and leads to(i 0,1)after the source transition, while the active action provides the same information and additionally paysλ. Hence, the passive action must be optimal for both estimators and V ∗ map(b0) =γV m...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.