Pith. sign in

REVIEW 4 major objections 4 minor 28 references

Timely Remote Estimation with Memory at the Receiver

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A receiver that stores multiple packets estimates non-Markovian sources more accurately than one that keeps only the freshest packet.

desk verdict A clear AoI-vector buffer model with honest provenance, but the headline numerical result is not reproducible without the missing MSE derivation. read the letter →

arxiv 2501.02099 v1 pith:FPBOZB32 submitted 2025-01-03 eess.SP

Sirin Chakraborty , Yin Sun This is my paper · ORCID
classification eess.SP
keywords ageofinformationAoIvectorremoteestimationreceiverbufferrestlessmulti-armedbanditMaximumGainFirstKalmanfilternon-Markoviansources
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a remote estimator can do better when the receiver keeps a small buffer of recently received packets rather than only the freshest one. For sources whose current value depends on more than the latest observation, the age vector of all buffered packets becomes the state that drives both estimation and scheduling. The authors formulate the scheduling problem as a restless multi-armed bandit, relax it with a Lagrange multiplier, and derive a Maximum Gain First policy that selects sensors by the Q-value gain of transmission. Numerical results for an AR(4) process with a Kalman filter show that a two-packet buffer reduces average estimation error relative to the one-packet baseline.

What carries the argument

The central object is the age-of-information vector Δ_n(t) = (Δ_{n,1}(t), . . . , Δ_{n,b}(t)) of the b packets stored for sensor n. Together with the buffer contents, it forms the state of a restless multi-armed bandit; the scheduling rule is the Maximum Gain First (MGF) policy, which at each slot transmits to at most M sensors with the largest non-negative gain α_{n,λ*}(Δ) = Q*_{n,λ*}(Δ,0) − Q*_{n,λ*}(Δ,1), the Q-value benefit of sending. Because the gain comes from a Lagrangian-relaxed dual solution rather than a Whittle index, the policy needs no indexability condition. The AoI vector is what lets the scheduler reason about the value of all stored packets, not just the freshest.

What would settle it

Recompute the Kalman-filter mean-squared error for the AR(4) source with buffer size 2 at specific age pairs, such as Δ1=1 and Δ2=3, and compare with the b=1 curve at the same ages; if the b=2 estimate is not strictly better for the age pairs the scheduler actually visits, the buffer advantage reported in Figure 3 would not hold.

Watch

Extended reading notes

Core claim

The central claim is that receiver-side memory improves remote estimation for non-Markovian sources, and that the improvement can be captured by an age-of-information vector rather than by tracking full packet contents. The paper constructs a scheduling policy, Maximum Gain First, based on the Q-value difference α_{n,λ*}(Δ) = Q*_{n,λ*}(Δ,0) − Q*_{n,λ*}(Δ,1), computed from a Lagrangian relaxation of the restless multi-armed bandit, and shows numerically that with buffer size 2 the optimized average mean-squared error is lower than with buffer size 1 for an AR(4) source under erasure channels. This extends the conventional b=1 AoI framework, suited to Markovian sources, to sources with longer temporal dependence. The authors also note that this appears to be the first AoI-vector-based scheduling formulation with a receiver-side buffer.

Load-bearing premise

The entire buffer advantage rests on the numerically generated curves in Figure 2 that relate the AoI vector to Kalman-filter mean-squared error; if those curves are inaccurate or the age-vector state reduction is invalid, the simulated performance gap in Figure 3 collapses.

Editorial extensions

If this is right

  • For non-Markovian sources, receiver memory is structurally useful: with buffer size 2, the optimized average estimation error falls below the b=1 baseline across the simulated transmission probabilities.
  • The AoI vector is a sufficient state for scheduling: the scheduler needs only the ages of the stored packets, not their values, to make transmission decisions.
  • The Maximum Gain First policy applies to restless multi-armed bandits where Whittle indexability is hard to verify, because it only requires solving relaxed per-arm MDPs and computing Q-value differences.
  • The estimation error as a function of the AoI vector can be non-monotonic, so a fresher packet does not always help more than an older packet that completes the information available in the buffer.
  • For a two-sensor, one-channel system with an AR(4) source, the two-packet buffer improves the optimized average estimation error compared with the standard one-packet setup.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the effect is robust, the optimal buffer size may scale with the source's memory order and the channel's packet-loss probability; a source with longer temporal dependence could benefit from b>2, while an i.i.d. source would gain nothing from any buffer.
  • The MGF policy's independence from indexability suggests it may extend to multiple channels and nonlinear estimators beyond Kalman filtering, though the paper only simulates one channel and two sensors.
  • Because the MSE is non-monotonic in the AoI vector, the scheduler implicitly trades freshness against informativeness: transmitting a fresh packet may be less valuable than transmitting a packet that fills a gap in the buffer's history.
  • The buffer's benefit can be viewed as a form of implicit prediction: stored packets act as a finite memory that lets the estimator exploit autocorrelation without changing the underlying transmission protocol.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper studies a remote estimation system in which N sensors transmit status updates over M shared wireless channels to a receiver equipped with a finite buffer per sensor. The receiver stores the b most recently received packets from each sensor, and the state of each source is an AoI vector describing the ages of the buffered packets. The transmission scheduling problem is formulated as a discounted infinite-horizon RMAB, relaxed via Lagrangian dual decomposition, and solved by a Maximum Gain First (MGF) policy that selects sensors with the largest Q-value differences. A numerical example with an AR(4) source and a Kalman filter at the receiver is used to claim that a buffer of size 2 reduces the average estimation error relative to a buffer of size 1, and the abstract and conclusion state that receiver buffering improves estimation for non-Markovian sources.

Significance. If the result holds, the paper provides a useful extension of AoI-based scheduling from a scalar age to an age vector, and the MGF approach offers a way to avoid the indexability requirement of Whittle-index policies in a restless bandit formulation. The authors are also transparent that the MGF policy is adopted from their prior work [22], [23]. However, the central numerical claim rests on a single AR(4) example whose underlying MSE curves are not specified in the manuscript, and the term "non-Markovian" is applied to a source that is in fact fourth-order Markov. The paper would be more convincing if the evaluation included reproducible details, additional source models, and scheduling baselines; the current manuscript does not yet support the strength of the abstract's general claim.

major comments (4)
  1. [Section V, Figure 2] The functions err(Δ1) and err(Δ1,Δ2) are the immediate costs in the Bellman equations (12)-(13), and through (15) they determine the MGF gains and the entire buffer comparison in Figure 3. The manuscript does not give the state-space representation used for the AR(4) model in (18), the measurement equation, the noise covariances, the Kalman filtering equations, the procedure for computing the MSE from the AoI vector, or the number of Monte Carlo runs with error bars. Since these curves are the sole numerical input to the scheduling policy, the central result cannot be reproduced or independently verified from the information provided.
  2. [Abstract and Section V] The paper's main claim concerns "non-Markovian sources," but the only source in the evaluation is the fourth-order autoregressive process X_{n,t} = 0.1 X_{n,t-1} + 0.8 X_{n,t-4} + W_{n,t}. An AR(4) process is a fourth-order Markov process and can be recast as a first-order Markov process by augmenting the state with lagged values. Thus the term "non-Markovian" is misleading, and the general claim is supported neither analytically nor by a broader set of source models. The authors should either qualify the claim or provide evidence for sources that are not finite-order Markov.
  3. [Section V, Figure 3] The comparison between buffer sizes b=1 and b=2 uses the proposed MGF policy, but no alternative scheduling policies are evaluated. Without baselines such as a greedy policy, a Whittle-index policy when available, or a simple round-robin scheme, it is unclear whether the observed improvement is due to the buffer memory itself or to the specific scheduling rule. Since the abstract's conclusion is about the benefit of buffering, the experiment should separate the effect of the buffer from the effect of the scheduling policy.
  4. [Section IV-C] The MGF policy is adopted from [22], [23] and is defined as the solution of the Lagrangian-relaxed problem, but the manuscript provides no performance guarantee, optimality gap, or asymptotic optimality argument for the original constrained problem (5)-(6). The policy is therefore heuristic for the considered model, and the absence of any bound weakens the claim that the proposed scheduling is "optimized" in the numerical section. A short discussion of the known theoretical status of MGF-type policies would help the reader calibrate the strength of the conclusions.
minor comments (4)
  1. [Equation (5)] Equation (5) contains a limit superior over T of an infinite sum over t=0 to infinity; with a fixed discount factor γ<1 the limit superior is redundant. This likely reflects an editing error from an average-cost formulation, and it should be clarified.
  2. [Figure 2 caption] The caption states "AoI Δ1(t) vs estimation error with fixed Δ2(t)=3," but the b=1 case has no second age component. Please specify which curves correspond to b=1 and b=2 and what the fixed age means for the single-packet case.
  3. [Section I] The phrase "the data at the receiver may become outdated" is colloquial; also "an estimator whose estimation performance" is awkward. A brief proofreading pass would improve clarity.
  4. [Section II] The related-work paragraph mentions "non-indexable scheduling policies" and then says the MGF policy was also called the Optimal Lagrange Index policy in [27], LP-Index policy in [28], and Gain Index policy in [26]. Adding one sentence explaining the relation among these names and the proposed policy would help readers who are not familiar with this literature.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the buffer-benefit conclusion is a numerical result computed from an independently defined cost function, not a fitted input or a self-citation chain.

full rationale

The paper's derivation chain is self-contained. The cost errn(Delta) is defined in (4) as the minimum estimation loss given the AoI vector, and the scheduling problem (5)-(6) is solved via Lagrangian relaxation and Bellman equations (12)-(13). The MGF gains in (15) are computed from these Q-functions using the per-state costs that Figure 2 reports as Kalman-filter MSE for the AR(4) source (18). Nothing in the Bellman recursion or the gain definition embeds the final b=2-vs-b=1 comparison; the comparison in Figure 3 is an evaluation of the same policy on two buffer sizes, not a prediction derived from the cost input. The only explicit self-citation is the adoption of the 'Net-gain Maximization' policy from the authors' [22],[23], transparently renamed 'Maximum Gain First'; this is a methodological reuse, and the policy's performance is tested numerically rather than inferred from the citation, so it does not constitute load-bearing circularity. The omitted derivation/error analysis for Figure 2's MSE curves is a reproducibility limitation, not a circular step, and the description of AR(4) as 'non-Markovian' is loose terminology (it is fourth-order Markov), but neither affects the circularity verdict. Therefore no circular reduction can be exhibited.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central claims rest on the AoI-vector state being sufficient for scheduling, known transmission success probabilities, exact or convergent DP and subgradient solution, and correct MSE curves. These are standard modeling assumptions but are not proven or fully specified.

assumptions (5)
  • domain assumption Transmission success indicators c_n(t) are i.i.d. across time and sensors with known probabilities p_n.
    Invoked in Section III-A when defining p_n and the AoI transition (2)-(3). The MDP formulation depends on these probabilities being known and independent of scheduling.
  • domain assumption The estimation error err_n(Delta_n(t)) is a deterministic function of the AoI vector only.
    Equation (4) defines err_n as the minimum expected loss conditioned on the AoI vector, making the AoI vector a sufficient state for scheduling. If the true error depended on other memory, the MDP would be misspecified.
  • domain assumption The per-sensor subproblem (11) can be solved exactly by dynamic programming over the infinite AoI-vector state space.
    Section IV-B states 'we use a dynamic programming algorithm' without specifying truncation, discretization, or convergence. The MGF gains in (15) depend on this solution.
  • domain assumption The Kalman filter MSE curves in Figure 2 are computed correctly for the AR(4) process.
    Section V presents the MSE as a function of AoI vector; these curves drive the MGF gains and the final comparison, but no derivation, simulation details, or error analysis is provided.
  • domain assumption The stochastic subgradient ascent in (14) converges to the optimal dual variable lambda*.
    Equation (14) updates lambda using infinite-horizon aggregated action counts; the paper does not prove convergence or specify stopping criteria or step size schedule.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Timely Remote Estimation with Memory at the Receiver." pith.science (2026). https://pith.science/paper/FPBOZB32

@misc{pith2026250102099,
  author       = {Pith},
  title        = {Pith review of: Timely Remote Estimation with Memory at the Receiver},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FPBOZB32}},
  note         = {Machine review of arXiv:2501.02099}
}
read the original abstract

In this study, we consider a remote estimation system that estimates a time-varying target based on sensor data transmitted over wireless channel. Due to transmission errors, some data packets fail to reach the receiver. To mitigate this, the receiver uses a buffer to store recently received data packets, which allows for more accurate estimation from the incomplete received data. Our research focuses on optimizing the transmission scheduling policy to minimize the estimation error, which is quantified as a function of the age of information vector associated with the buffered packets. Our results show that maintaining a buffer at the receiver results in better estimation performance for non-Markovian sources.

Figures

Figures reproduced from arXiv: 2501.02099 by the authors.

Figure 1
Figure 1. System Model propose a buffer-based remote estimation model. In this model, the receiver maintains a buffer of size b for each sensor n, storing the b most recently received data packets. After each successful packet delivery from sensor n, the newly received data packet is stored in the buffer, while the oldest packet is discarded to maintain the fixed buffer size b. This setup extends the conventional concept of A… view at source ↗
Figure 2
Figure 2. AoI ∆1(t) vs estimation error with fixed ∆2(t) = 3 [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Average estimation error vs transmission probability [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 25 canonical work pages

  1. [22]

    Learning and communications co-design for remote inference systems: Feature length selection and transmission scheduling,

    M. K. C. Shisher, B. Ji, I.-H. Hou, and Y . Sun, “Learning and communications co-design for remote inference systems: Feature length selection and transmission scheduling,” IEEE Journal on Selected Areas in Information Theory , vol. 4, pp. 524–538, 2023

    2023

  2. [23]

    Context- aware status updating: Wireless scheduling for maximizing situational awareness in safety-critical systems,

    T. Z. Ornee, M. K. C. Shisher, C. Kam, and Y . Sun, “Context- aware status updating: Wireless scheduling for maximizing situational awareness in safety-critical systems,” in IEEE MIL- COM 2023 , 2023, pp. 194–200

    work page 2023

  3. [1]

    Remote estimation of the wiener process over a channel with random delay,

    Y . Sun, Y . Polyanskiy, and E. Uysal-Biyikoglu, “Remote estimation of the wiener process over a channel with random delay,” in IEEE ISIT , 2017, pp. 321–325

    work page 2017

  4. [2]

    Sampling for data freshness optimization: Non-linear age functions,

    Y . Sun and B. Cyr, “Sampling for data freshness optimization: Non-linear age functions,” Journal of Communications and Networks, vol. 21, no. 3, pp. 204–219, 2019. Fig. 3. Average estimation error vs transmission probability pn

    work page 2019

  5. [3]

    AoI-penalty minimization for networked control systems with packet loss,

    M. Kl ¨ugel, M. H. Mamduhi, S. Hirche, and W. Kellerer, “AoI-penalty minimization for networked control systems with packet loss,” in IEEE INFOCOM AoI Workshop , 2019, pp. 189–196

    work page 2019

  6. [4]

    Sampling and remote estimation for the Ornstein-Uhlenbeck process through queues: Age of infor- mation and beyond,

    T. Z. Ornee and Y . Sun, “Sampling and remote estimation for the Ornstein-Uhlenbeck process through queues: Age of infor- mation and beyond,” IEEE/ACM Transactions on Networking , vol. 29, no. 5, pp. 1962–1975, 2021

    work page 1962

  7. [5]

    A Whittle index policy for the remote estimation of multiple continuous Gauss-Markov processes over parallel channels,

    T. Z. Ornee and Y . Sun, “A Whittle index policy for the remote estimation of multiple continuous Gauss-Markov processes over parallel channels,” in ACM MobiHoc, 2023, pp. 91–100

    work page 2023

  8. [6]

    Real-time status: How often should one update?

    S. Kaul, R. Yates, and M. Gruteser, “Real-time status: How often should one update?” In 2012 Proceedings IEEE INFO- COM, 2012, pp. 2731–2735

    work page 2012

Show all 28 references
  1. [7]

    Age of information: An introduction and survey,

    R. D. Yates, Y . Sun, D. R. Brown, S. K. Kaul, E. Modiano, and S. Ulukus, “Age of information: An introduction and survey,” IEEE Journal on Selected Areas in Communications , vol. 39, no. 5, pp. 1183–1210, 2021

    2021

  2. [8]

    Lazy is timely: Status updates by an energy harvesting source,

    R. D. Yates, “Lazy is timely: Status updates by an energy harvesting source,” in IEEE ISIT , 2015, pp. 3008–3012

    2015

  3. [9]

    Update or wait: How to keep your data fresh,

    Y . Sun, E. Uysal-Biyikoglu, R. D. Yates, C. E. Koksal, and N. B. Shroff, “Update or wait: How to keep your data fresh,” IEEE Transactions on Information Theory , vol. 63, no. 11, pp. 7492–7508, 2017

    2017

  4. [10]

    The age of correlated features in supervised learning based fore- casting,

    M. K. C. Shisher, H. Qin, L. Yang, F. Yan, and Y . Sun, “The age of correlated features in supervised learning based fore- casting,” in IEEE INFOCOM AoI Workshop , 2021, pp. 1–8

    2021

  5. [11]

    How does data freshness affect real-time supervised learning?

    M. K. C. Shisher and Y . Sun, “How does data freshness affect real-time supervised learning?” In ACM MobiHoc, 2022, pp. 31–40

    2022

  6. [12]

    Minimizing age of information in multiaccess-edge- computing-assisted iot networks,

    A. Muhammad, I. Sorkhoh, M. Samir, D. Ebrahimi, and C. Assi, “Minimizing age of information in multiaccess-edge- computing-assisted iot networks,” IEEE Internet of Things Journal, vol. 9, no. 15, pp. 13 052–13 066, 2021

    2021

  7. [13]

    The age of incorrect information: A new performance metric for status updates,

    A. Maatouk, S. Kriouile, M. Assaad, and A. Ephremides, “The age of incorrect information: A new performance metric for status updates,” IEEE/ACM Transactions on Networking , vol. 28, no. 5, pp. 2215–2228, 2020

    2020

  8. [14]

    Two freshness metrics for local cache refresh,

    J. Zhong, R. D. Yates, and E. Soljanin, “Two freshness metrics for local cache refresh,” in IEEE ISIT , 2018, pp. 1924–1928

    2018

  9. [15]

    Consistency of value of information: Effects of packet loss and time delay in networked control systems tasks,

    T. Soleymani, J. S. Baras, S. Wang, S. Hirche, and K. H. Johansson, “Consistency of value of information: Effects of packet loss and time delay in networked control systems tasks,” arXiv preprint arXiv:2403.11932 , 2024

    2024 arXiv

  10. [16]

    On the credibility of information flows in real-time wireless networks,

    D. Guo and I.-H. Hou, “On the credibility of information flows in real-time wireless networks,” in 2019 International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOPT) , 2019, pp. 1–8

    2019

  11. [17]

    Kalman filtering with intermit- tent observations,

    B. Sinopoli, L. Schenato, M. Franceschetti, K. Poolla, M. I. Jordan, and S. S. Sastry, “Kalman filtering with intermit- tent observations,” IEEE transactions on Automatic Control , vol. 49, no. 9, pp. 1453–1464, 2004

    2004

  12. [18]

    Optimal estimation in networked control sys- tems subject to random delay and packet drop,

    L. Schenato, “Optimal estimation in networked control sys- tems subject to random delay and packet drop,” IEEE trans- actions on automatic control , vol. 53, no. 5, pp. 1311–1317, 2008

    2008

  13. [19]

    A Whittle index approach to minimizing functions of age of information,

    V . Tripathi and E. Modiano, “A Whittle index approach to minimizing functions of age of information,” in 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton) , 2019, pp. 1160–1167

    2019

  14. [20]

    Scheduling to minimize age of incorrect information with imperfect channel state informa- tion,

    Y . Chen and A. Ephremides, “Scheduling to minimize age of incorrect information with imperfect channel state informa- tion,” Entropy, vol. 23, no. 12, p. 1572, 2021

    2021

  15. [21]

    Timely commu- nications for remote inference,

    M. K. C. Shisher, Y . Sun, and I.-H. Hou, “Timely commu- nications for remote inference,” IEEE/ACM Transactions on Networking, vol. 32, no. 5, pp. 3824–3839, 2024

    2024

  16. [24]

    Restless bandits: Activity allocation in a changing world,

    P. Whittle, “Restless bandits: Activity allocation in a changing world,” Journal of applied probability, vol. 25, no. A, pp. 287– 298, 1988

    1988

  17. [25]

    On the asymptotic optimality of greedy index heuristics for multi-action restless bandits,

    D. J. Hodge and K. D. Glazebrook, “On the asymptotic optimality of greedy index heuristics for multi-action restless bandits,” Adv. Appl. Probab., vol. 47, no. 3, pp. 652–667, 2015

    2015

  18. [26]

    An index policy for minimizing the uncertainty-of-information of Markov sources,

    G. Chen and S. C. Liew, “An index policy for minimizing the uncertainty-of-information of Markov sources,” IEEE Trans- actions on Information Theory , vol. 70, no. 1, pp. 698–721, 2024

    2024

  19. [27]

    Index policies and perfor- mance bounds for dynamic selection problems,

    D. B. Brown and J. E. Smith, “Index policies and perfor- mance bounds for dynamic selection problems,” Management Science, vol. 66, no. 7, pp. 3029–3050, 2020

    2020

  20. [28]

    Linear program-based poli- cies for restless bandits: Necessary and sufficient conditions for (exponentially fast) asymptotic optimality,

    N. Gast, B. Gaujal, and C. Yan, “Linear program-based poli- cies for restless bandits: Necessary and sufficient conditions for (exponentially fast) asymptotic optimality,” Mathematics of Operations Research , 2023

    2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.