{"id":"e0214f3b-6dc4-4809-b3e7-f8b960e4c382","arxiv_id":"1908.01503","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A control-aware age-of-information scheduler solved by value iteration on a Markov decision process reduces estimation error in multi-loop networked control compared to age-only and greedy schedulers.","lead":"This paper presents a centralized scheduler for wireless networked control that uses how old the received information is and how unstable each control loop is to decide which loop transmits next. The proposed policy is computed with a Markov decision process and is shown in simulation to lower average estimation error relative to an age-only scheduler and a greedy error scheduler.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central infinite-horizon optimality claim is unproven: Algorithm 1 solves the truncated MDP_M, and no ATAS convergence proof, finiteness check, or error bound is given; the abstract overstates the result.","rationale":"The strongest load-bearing concern is not the value iteration itself but the identification of the finite-M optimal policy with the solution of the infinite-horizon problem. The reader's weakest_assumption points to the same place, and the paper's own footnote 3 and Section IV-D corroborate it. I agree with the CONDITIONAL verdict: the algorithmic contribution is well structured and the simulations are repeatable, but the abstract's optimality claim needs either a proof of ATAS convergence/error bound or a softened wording. I considered whether the original objective may be infinite for the tested parameters; while this is a sharper possibility, the necessary tail analysis is not in the paper, so the safe formulation is that finiteness and convergence are unverified rather than definitely false. A concrete simulation of the untruncated discounted cost and a check of Sennott's conditions would settle the issue. If the check fails, the claim should be restricted to the finite-state approximation; if it passes, the abstract can be retained. Thus no verdict change from the reader is needed.","tokens_in":10252,"tokens_out":20262,"duration_ms":227546,"concrete_test":"Run the Section IV-B setup with M=25, γ=0.9 and simulate the original untruncated age process for T=10^6 slots under the DES policy; compute the cumulative discounted cost Σ_{t=0}^T γ^t (1/N)Σ_i g_i(Δ_i[t]). If the series has not converged (e.g., it keeps growing with T) or if the per-subsystem tail estimate P(Δ_i≥k) and E[g_i(Δ_i)] fail to make γ^tE[g_i(Δ_i[t])]→0, then (8) is not finite for the claimed policy and the ATAS approximation is not justified. A complementary analytical check: instantiate Sennott's ATAS conditions for transition (2) and cost (6); if any condition fails, exhibit a parameter set where the finite-M policy differs from the true optimal policy by an arbitrarily large amount as M grows.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's abstract and title claim a stationary policy that minimizes the discounted infinite-horizon cost Jπ(s0) in (8) over the countable state space S = N_+^N. What Algorithm 1 actually computes is the exact optimal policy of the finite MDP_M obtained in Section III-A by clamping every age at M and redistributing excess transition probability back to state M. For the finite-M optimum to be a faithful approximation of the original problem one needs the ATAS value functions to converge to V* (or an explicit suboptimality bound). The paper cites Sennott [16] but never verifies that the specific transition law (2) and exponential-type cost (6) satisfy the ATAS conditions. This gap is load-bearing: g_i(Δ) grows roughly as A_i^{2Δ}, so the contribution of ages beyond M is not automatically negligible, and for γ close to 1 and unstable A_i the discounted sum (8) is not even shown to be finite. Footnote 3 concedes that the algorithm solves only 'the approximated problem optimally for a given γ and an augmentation level M', and Section IV-D shows M=15 is strictly worse than M=25 (Fig. 7), demonstrating that truncation level materially changes the policy. Without a convergence theorem or a computable gap, the abstract's unqualified 'minimizes control error over an infinite horizon' and the 'first work' claim go beyond what is proven. The finite-MDP result itself appears correct; the problem is the leap from finite truncation to infinite-horizon optimality.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper considers a centralized wireless scheduler that, at each slot, selects up to R of N sensors to transmit state measurements to their controllers over an erasure channel. It formulates the problem as a discounted infinite-horizon MDP whose state is the vector of ages of information (AoI) and whose per-step cost is the sum of per-loop mean-square estimation errors g_i(Δ_i), derived for LTI plants with Gaussian disturbances. Since the state space is countably infinite, the authors introduce a finite 'augmentation type approximating sequence' by truncating each AoI at M, and apply value iteration to obtain a stationary deterministic policy (DES). They claim this policy minimizes the infinite-horizon control error and simulate it against a γ-optimal AoI scheduler and a greedy error scheduler. The finite-state value iteration itself is standard, but the paper does not prove that the truncated problem approximates the original infinite-horizon problem.","tokens_in":10481,"tokens_out":8422,"duration_ms":88672,"significance":"If the optimality claim could be substantiated, the paper would be a useful contribution: it gives an explicit control-aware age penalty, a clean MDP formulation, and a systematic DP approach to multi-loop scheduling. The simulation study is also reasonably transparent, including a sensitivity analysis over M, and footnote 3 is honest about the approximation. However, the main advertised result, infinite-horizon optimality, is not established, and the benchmark scheduling rule is mis-specified in Eq. (13). The contribution is therefore currently a heuristic finite-state scheduler with promising simulations, not a proven optimal scheduler for the original problem. The value of the paper would increase substantially if the authors added a formal ATAS convergence argument or an explicit suboptimality bound, and corrected the benchmark definition.","major_comments":[{"comment":"The central claim that Algorithm 1 minimizes the infinite-horizon cost Jπ(s0) in (8) is not proven. Algorithm 1 solves the finite MDP_M of Section III-A, and footnote 3 concedes that it solves the approximated problem optimally for a given γ and an augmentation level M. The paper cites Sennott [16] for the ATAS framework but never verifies that the specific transition law (2) and cost (6) satisfy the conditions under which the ATAS value functions converge to V*, nor does it give an optimality gap. This is not a technicality: for A_i = 1.9, g_i(Δ) grows like 1.9^{2Δ}, so for γ close to 1 the discounted sum in (8) may be infinite under policies that starve a sub-system; the original problem is not even shown to be finite. Section IV-D, especially Fig. 7, demonstrates that the truncation level materially changes the policy. To support the abstract and title, the authors must either prove convergence and finiteness or restrict all optimality claims to the finite approximated MDP.","section":"§III-A, §III-B, Abstract"},{"comment":"Equation (13) defines the greedy error scheduler as a[t] = arg max_{a∈A} { Σ_i p_i g_i(Δ_i[t]) }. The objective is independent of a, so the arg max is not well-defined; as written every admissible action is a maximizer. The text says the scheduler prioritizes sub-systems with the highest mean squared error, so the intended expression presumably includes the scheduling variables, e.g., a[t] = arg max_{a∈A} Σ_i a_i p_i g_i(Δ_i[t]). Because Fig. 3 and the simulations depend on this benchmark, the definition must be corrected before the comparison claims in Section IV-C can be evaluated.","section":"Eq. (13)"},{"comment":"In Algorithm 1, the extracted policy is π*(s) ← arg min_{a∈A} J(s). Since J(s) does not depend on a, this does not define a policy; the greedy action should be computed from the Bellman expression in (11), e.g., π*(s) ← arg min_{a∈A} { C(s,a) + γ Σ_{s'} P_{ss'}(a) J(s') }. The sentence before the algorithm states this correctly, but the algorithm as printed is not implementable and must be fixed.","section":"Algorithm 1, line 10"}],"minor_comments":[{"comment":"The statement 'ai[t] = qi[t], ∀i ∈ N+' uses N+ as an index set; it should be '∀i = 1, ..., N' to avoid confusion with the state-space notation.","section":"§II-D"},{"comment":"The conditional expectation in (4) is missing a closing bracket: it should read E[xi[t] | xi[t−Δ_i[t]]].","section":"§II-C, Eq. (4)"},{"comment":"The redistribution distribution f_{s'}(s,a,ŝ,M) is not specified; the clamping rule used in Fig. 2 should be stated formally as an instance of (9).","section":"§III-A, Definition 2"},{"comment":"The phrase 'DES results in a concave shape with respect to Δ as γ increases' should read 'with respect to γ' or 'as a function of γ'.","section":"§IV-C"},{"comment":"The statement that value iteration converges for any initial value function J0 is only valid for the finite-state discounted problem with bounded costs; this should be stated explicitly, since the original infinite-state problem may have unbounded costs.","section":"§III-B"},{"comment":"The stopping threshold θ = 0.1 is quite large relative to the costs (of order 10^2) and no discussion of the induced value-function error is given; a smaller threshold or an error bound would make the reported policies more reproducible.","section":"§IV-B"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a conference-style paper whose main gap is between the claimed infinite-horizon optimality and the actual finite-truncation result. If the authors can add a convergence/gap result or qualify the claims, it could be publishable after revision. The comparison to prior work would also be more credible once Eq. (13) and Algorithm 1 line 10 are corrected."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper is worth engaging, but the abstract promises more than the math delivers. The finite-state scheduler is solid; the infinite-horizon claim is not.\n\nWhat's actually new: formulation of multi-loop NCS scheduling as a discounted MDP with AoI states and control-error costs, solved by value iteration. The cost derivation from estimation error (eq. 6) is standard and clean. The comparison against an AoI-only scheduler and a greedy error scheduler is sensible, and the simulations are thorough enough to reproduce. I also appreciate the honest footnote admitting that the algorithm solves the approximated problem, and the discussion of M in Sec IV-D showing the truncation level changes results.\n\nThe soft spot is exactly that gap. Algorithm 1 computes the optimal policy for the truncated MDP_M. To license the abstract's claim of minimizing control error over an infinite horizon, you need the ATAS value functions to converge to the original value function or an explicit suboptimality bound. The paper cites Sennott but doesn't verify the conditions for this specific transition law and cost. That's not a nitpick: g_i(Δ) grows like A_i^{2Δ}, so for unstable A_i and γ close to 1 the infinite discounted sum may not even be finite. The fact that M=15 vs M=25 changes the policy (Fig. 7) confirms the truncation matters. Without a convergence result, the unqualified abstract and the 'first work' claim overshoot.\n\nThis is fixable. The cleanest response is to soften the claims: state the result as optimal for the augmented finite MDP, and present the infinite-horizon result as a heuristic with simulation support. Alternatively, add a sufficient condition or a numerical gap. Either way, the core contribution—using control-aware age costs in an MDP scheduler—remains useful.\n\nWho this is for: people working on AoI-based scheduling for networked control, and anyone who wants a benchmark of control-aware scheduling. It deserves a serious referee, because the problem is well-motivated and the method is mostly sound. I'd send it to review with the expectation that the authors will need to either add a bound or rewrite the claims.","headline":"Genuinely useful scheduler for multi-loop NCS, but the abstract's infinite-horizon optimality claim is not supported by the truncated-MDP proof.","tokens_in":11080,"tokens_out":2280,"would_cite":true,"duration_ms":21578,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A scheduling policy built on age-of-information states minimizes discounted control error across multiple wireless control loops, outperforming both age-optimal and greedy error baselines in simulation.","keywords":["age of information","networked control systems","Markov decision process","wireless scheduling","discounted infinite horizon","value iteration","estimation error","age penalty"],"falsifier":"Run the same five-loop scenario with $M$ doubled, for example $M=50$, and compare the value functions and simulated average error; if the policy changes substantially or the error rises, the truncation has not converged. More directly, for a small case with $N=1$ or $N=2$, enumerate policies on the untruncated state space, compute their exact discounted costs, and check whether the truncated policy's Bellman optimality equation is violated by more than the convergence threshold at any reachable state.","tokens_in":9980,"feed_emoji":"📡","tokens_out":2697,"duration_ms":31028,"temperature":0.7,"pith_summary":"The paper tries to establish that a centralized wireless scheduler for multiple feedback control loops can be designed directly around the control cost, using age-of-information (AoI) as the only state information. It models the scheduling problem as a Markov decision process in which each loop's mean-square estimation error is written as a function of its AoI, then applies value iteration to obtain a stationary, deterministic, discounted-optimal policy. The claimed payoff is lower average quadratic estimation error than the state-of-the-art age-optimal scheduler and greedy error scheduler, without any communication beyond each loop's current age. A sympathetic reader would care because this is a concrete step toward making AoI a true cross-layer interface for control, not just a freshness metric.","feed_headline":"Age-based scheduler cuts control error in shared wireless loops","feed_subtitle":"A discounted value-iteration policy over age-of-information states beats age-optimal and greedy baselines for multi-loop control.","key_machinery":"The carrying object is the coupled pair of an augmented MDP and a control-aware age penalty. The state is the vector of per-loop ages $\\Delta_i[t]$, capped at $M$, and the cost is the mean-square estimation error $g_i(\\Delta_i[t])$, which is increasing in age because the open-loop prediction error accumulates through the unstable plant dynamics. Transitions are Bernoulli packet successes, and any probability mass that would leave the finite state space at the cap is folded back onto the cap state. Value iteration on the Bellman equation then produces the stationary policy.","core_discovery":"For a given discount factor $\\gamma$ and augmentation level $M$, the paper claims that Algorithm 1 returns a stationary deterministic policy that minimizes the discounted infinite-horizon sum of network-induced mean-square estimation errors for the finite-state augmented MDP, with cost $g_i(\\Delta_i[t]) = \\sum_{r=0}^{\\Delta_i[t]-1} \\operatorname{tr}\\big((A_i^T)^r A_i^r \\Sigma_i\\big)$ per loop. The paper also claims, based on Monte Carlo simulation, that this discounted error scheduler (DES) attains lower average quadratic error than the $\\gamma$-optimal AoI scheduler and the greedy error scheduler in the five-loop, single-resource scenario it studies.","pith_inferences":["If a formal optimality gap for the truncation could be established, the same value-iteration design would justify itself beyond the specific five-loop scenario; the paper currently relies on simulation to show that the approximation is faithful enough.","A natural testable extension is the undiscounted average-cost regime, where the discount factor is removed; the paper's concave error curves suggest the discounted problem may not directly extrapolate to the average-cost optimum.","The method's state-space size $M^N$ makes the exact value-iteration approach expensive for many loops; a reader might infer that factored or approximate dynamic programming over per-loop age distributions is the next step.","Because the cost $g_i$ is derived purely from open-loop prediction error, the same scheduler design could be applied to any remote estimation task whose error grows monotonically with staleness, not just LTI control loops."],"forward_implications":["The scheduler can be computed offline once the plant matrices, noise covariances, and packet success probabilities are known, and then applied online using only each loop's current AoI.","Increasing the discount factor makes the scheduler more farsighted and, in the simulated setup, monotonically lowers the average quadratic error, suggesting that long-horizon planning matters for control quality.","The control-aware age penalty, rather than raw age, is what drives the performance gain: the age-optimal baseline achieves the best freshness but much worse estimation error.","The augmentation level $M$ is a genuine complexity-performance trade-off: larger $M$ improves the approximation, but the improvement saturates, so a moderate $M$ can capture most of the benefit.","The same MDP formulation can absorb additional per-action costs, such as communication energy or resource prices, without changing the solution method."],"supporting_citations":[{"why":"Supplies the $\\gamma$-optimal AoI scheduler baseline that the proposed DES is compared against and outperforms in estimation error.","marker":"[10]"},{"why":"Supplies the control-aware greedy error scheduler baseline that the proposed DES is compared against and outperforms.","marker":"[14]"},{"why":"Provides the dynamic programming and value iteration convergence results used to justify Algorithm 1.","marker":"[15]"},{"why":"Provides the augmentation-type approximating sequence framework used to truncate the infinite state space to $\\{1,\\dots,M\\}^N$.","marker":"[16]"},{"why":"Provides the earlier derivation, reused here, that expresses state estimation error as a function of AoI for LTI systems.","marker":"[12]"},{"why":"Supports the assumption that sensors replace their queued packet with a fresher measurement when not granted a transmission slot.","marker":"[4]"}],"fun_headline_variants":["Discounted age-penalty scheduling cuts wireless control error","AoI-based scheduler minimizes infinite-horizon control error","Optimal policy for age penalty in multi-loop control","Age-aware scheduler beats baselines in wireless loops","First AoI scheduler for infinite-horizon multi-loop control"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The scheduler's optimality relies on the assumption that truncating very large ages at a bound $M$ and folding the lost probability back into that bound does not materially change which decisions are best.","fun_headline_variants_meta":{"raw":{"variants":["Discounted age-penalty scheduling cuts wireless control error","AoI-based scheduler minimizes infinite-horizon control error","Optimal policy for age penalty in multi-loop control","Age-aware scheduler beats baselines in wireless loops","First AoI scheduler for infinite-horizon multi-loop control"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000238,"raw_usage":{"total_tokens":1468,"prompt_tokens":859,"completion_tokens":609,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":475,"completion_tokens_details":{"reasoning_tokens":530}},"tokens_in":475,"tokens_out":609,"duration_ms":6288,"temperature":1.0,"reasoning_tokens":530,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T15:10:12.945333+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same five-loop scenario with $M$ doubled, for example $M=50$, and compare the value functions and simulated average error; if the policy changes substantially or the error rises, the truncation has not converged. More directly, for a small case with $N=1$ or $N=2$, enumerate policies on the untruncated state space, compute their exact discounted costs, and check whether the truncated policy's Bellman optimality equation is violated by more than the convergence threshold at any reachable state.","supporting_citations":[{"cited_title":"Age of information: Design and analysis of optimal scheduling algorithms,","cited_arxiv_id":null,"evidence_quote":"Supplies the $\\gamma$-optimal AoI scheduler baseline that the proposed DES is compared against and outperforms in estimation error."},{"cited_title":"Control-aware uplink resource allocation for cyber-physical systems in wireless networks,","cited_arxiv_id":null,"evidence_quote":"Supplies the control-aware greedy error scheduler baseline that the proposed DES is compared against and outperforms."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the dynamic programming and value iteration convergence results used to justify Algorithm 1."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the augmentation-type approximating sequence framework used to truncate the infinite state space to $\\{1,\\dots,M\\}^N$."},{"cited_title":"Age-of- information vs. value-of-information scheduling for cellular networked control systems,","cited_arxiv_id":null,"evidence_quote":"Provides the earlier derivation, reused here, that expresses state estimation error as a function of AoI for LTI systems."},{"cited_title":"Age of information with packet management,","cited_arxiv_id":null,"evidence_quote":"Supports the assumption that sensors replace their queued packet with a fresher measurement when not granted a transmission slot."}],"review_version":1}