{"id":"c7679ac4-45f4-4713-91cd-6fa2568a7480","arxiv_id":"2508.10872","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"A2C reinforcement learning tunes five orbital elements for ground coverage, but the claimed superiority over PPO is unsupported by inconsistent reward numbers and an underspecified environment.","lead":"This paper trains a reinforcement learning agent (A2C) to adjust five satellite orbital parameters so the orbit passes near a target on Earth, and compares it to the PPO algorithm. The authors report large speedups for A2C, but the claimed improvements do not match the paper's own numbers.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"A2C-superiority ratio is contradicted by the paper's own numbers: 10.0 vs 9.263025 is ~8% higher, not 5.8x; timestep ratios differ between abstract and conclusion.","rationale":"The reader's strongest claim is the abstract's assertion of A2C superiority, and their verdict is REJECT. My stress test identifies the most load-bearing support for that claim: the reported numerical comparison itself. Tables 4 and 5 and the training text do not produce the claimed 5.8x reward multiplier or consistent timestep counts; the abstract and conclusion use different values. This is not an interpretive disagreement or a matter of tuning—it is an internal inconsistency that directly undercuts the central claim. The missing d_target definition in Eq. (8) is important for reproducibility and for evaluating the absolute rewards, but the arithmetic failure is sufficient to invalidate the headline ratio even if the environment were fully specified. I therefore agree with the reader's REJECT verdict and would keep it unchanged, but I locate the decisive problem in the inconsistent reported numbers rather than in the underspecified reward distance. A simple recomputation of the ratios from the paper's own tables settles the concern; if the authors supply corrected multipliers and raw training data, the claim could be reassessed, but as submitted the evidence does not support the abstract's quantitative claims.","tokens_in":8500,"tokens_out":6076,"duration_ms":64011,"concrete_test":"Recompute the ratios directly from Tables 4–5 and the stated training lengths: reward ratio = 10.0/9.263025 ≈ 1.08 (~8% higher, not 5.8x); timestep ratios are 63,000/2,000 = 31.5 and 61,440/2,240 ≈ 27.4. If the corrected values do not reproduce the abstract's 5.8x and 31.5x claims, the headline is falsified by the paper's own data. Optionally, locate the missing definition of d_target in Eq. (8); without it, no reward value is independently computable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on a quantitative comparison: the abstract states A2C achieves '5.8x higher cumulative rewards (10.0 vs 9.263025)' and converges in '31.5x fewer timesteps (2,000 vs 63,000)'. The parenthetical reward values give 10.0/9.263025 ≈ 1.08, i.e. about 8% higher, and an absolute difference of 0.737—not 5.8 under any standard reading. The conclusion instead reports '73.6% higher cumulative rewards' and '27.4x fewer timesteps (2,240 vs 61,440)', and these timestep counts also disagree with the abstract's 2,000 vs 63,000. Since the entire argument for A2C over PPO depends on these numbers, the headline claim is not supported by the paper's own data. Secondary, but also important: Eq. (8) uses a distance d_target that is never defined, so the absolute reward values cannot be independently reproduced either. The arithmetic inconsistency alone, however, is sufficient to void the claimed 5.8x advantage.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a reinforcement-learning framework for LEO orbital parameter planning. It defines a custom OpenAI Gymnasium environment whose state is five Keplerian elements, designs a shaped reward for ground-target coverage, safety distance, and altitude validity, and trains an A2C agent with a plateau-detection callback. The paper's central claim is that A2C outperforms PPO by 5.8× in cumulative reward while converging in 31.5× fewer timesteps, and thus validates actor-critic methods over trust-region approaches for continuous orbital control. The paper includes hyperparameter tables, reward equations, training curves, and a single prediction table for each algorithm.","tokens_in":8900,"tokens_out":6647,"duration_ms":75735,"significance":"If the claimed result were solid, a TLE-initialized RL environment with rapid convergence would be a useful step for responsive LEO mission planning. The authors address a relevant problem and provide a concrete MDP formulation, with plausible use of Keplerian elements and publicly available TLE bounds. However, the current manuscript does not support its central quantitative claims: the reported rewards contradict the stated multiplier, the environment and reward formulas are underspecified, and the evaluation rests on single runs with no statistical validation. As written, the paper's contribution is not independently reproducible and its main comparison is not trustworthy.","major_comments":[{"comment":"The headline comparison is contradicted by the paper's own data. 10.0 vs 9.263025 is about 8% higher (10.0/9.263025 ≈ 1.08), not '5.8× higher' as the abstract claims, nor '73.6% higher' as the conclusion claims. The timestep counts also disagree: the abstract says 2,000 vs 63,000, while the conclusion says 2,240 vs 61,440, and the Training section says 2,500 vs 62,000. Because the entire A2C-superiority argument rests on these numbers, the central claim is unsupported by the manuscript itself.","section":"Abstract; Conclusion; Tables 4–5"},{"comment":"The ground-target reward Rt depends on d_target, but d_target is never defined. Without a definition of the distance between the orbit and a ground target coordinate, the reward cannot be computed, the optimization objective is not meaningful, and the A2C/PPO comparison cannot be reproduced. Likewise, dmin in Eq. (4) for the safety margin is undefined. These are not minor omissions; they make the reward function underspecified.","section":"Section 3.3, Eq. (8); Eq. (4)"},{"comment":"Eq. (11) defines Re,i = Re + Ri, but Re and Ri are never defined. Eq. (13) introduces weights wc, ws, wt and adds Re,i to a weighted sum, but no values or ranges are given for any of these weights. Eq. (15) divides by 5 while listing only Ps, Pr, Pt, Pe,i (four terms) and leaves Pr undefined. As written, the final reward formula is not implementable, so the learning curves and cumulative rewards are not auditable.","section":"Section 3.3, Eqs. (11), (13), (15); Observation Space"},{"comment":"The observation space defines 'Coverage Error' as a discrete binary value (0 or 1), while Eq. (1) computes a normalized coverage error as coverage_error / max(1e-6, hmax - hmin). If coverage_error is binary, the normalized error is either 0 or a tiny number, which does not yield the smooth altitude-deviation penalty described in the text. The equation needs a continuous definition of altitude deviation to be meaningful.","section":"Section 2.2, Observation Space; Eq. (1)"},{"comment":"The evaluation comprises a single training run per algorithm with no seeds, no error bars, and no statistical summary. The sentence claiming that the A2C agent 'consistently meets mission objectives across diverse target coordinates' is not supported by any table or protocol; only one target scenario is reported. Moreover, 'Objectives Met' is derived from the same reward function the agent maximizes, so reaching the reward clip (10.0) is reported as success. The observed 8% reward difference is well within the range expected from a single noisy run and does not establish algorithmic superiority.","section":"Section 5, 'Prediction & Reward Curve Analysis'; Tables 4–5"}],"minor_comments":[{"comment":"The action-space paragraph says 'The PPO-based agent has decisive control' and later refers to the 'PPO-agent' during training, although the paper's algorithm is A2C. This appears to be leftover text from a PPO-based version and should be corrected.","section":"Section 3.2, Action Space"},{"comment":"The definition of true anomaly duplicates the definition of argument of perigee: both are described as 'the angle between the ascending node and the orbit's point of closest approach to Earth.' The text should distinguish these distinctly.","section":"Section 3.1, Orbital Elements"},{"comment":"The sentence 'we will release the data and code soon' appears in the abstract. This is not a technical claim and is better placed in a reproducibility statement or omitted until the materials are actually available.","section":"Abstract"},{"comment":"There are several typographical and formatting inconsistencies, including 'OpenAI' vs 'OpenAI', inconsistent capitalization ('Our simulation', 'It enables'), and the arXiv references lacking proper version identifiers. These should be cleaned up.","section":"Throughout"},{"comment":"The illustration is credited to Tafanidis et al. 2025; if the figure is reproduced from that source, the necessary permission and a clear attribution statement should be provided.","section":"Figure 1"}],"recommendation":"reject","confidential_remarks":"The paper appears to be an early-stage or workshop-style submission. The main quantitative claim is arithmetically inconsistent with the reported data, and the environment and reward equations are too underspecified to permit reproduction. The single-run evaluation and self-referential success criterion further weaken the comparison. I would not recommend encouraging a revision without a fundamentally new experimental setup and a corrected, fully specified environment."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague—short take: this is a reasonable test of A2C vs PPO on a custom orbital-coverage MDP, but the sellers' numbers don't hold up under a simple ratio check. The paper is worth a referee's time, but only if the authors fix the arithmetic and define the environment properly.\n\nWhat's actually new: the specific comparison of A2C and PPO on a TLE-seeded, Keplerian-element action space for ground coverage. That's not in the cited literature as far as I can tell. The environment design is sensible: actions on five orbital elements, observations with binary flags for validity, and a shaped reward with altitude, safety, and target terms. The use of TLE initialization bounds is a nice touch. They were also upfront that code/data will be released (though 'soon' is doing work).\n\nThe soft spots are central. The abstract claims 5.8x higher cumulative reward and 31.5x fewer timesteps, but the parenthetical values (10.0 vs 9.263) are ~8% higher, and the conclusion says 73.6% and 27.4x with timestep counts (2,240 vs 61,440) that disagree with the abstract (2,000 vs 63,000). So the headline advantage is not supported. Add to that a single seed with no error bars, an undefined d_target in Eq. (8) (so the reward is not reproducible), and sub-rewards Pe, Pi, Pr that never get defined. The 'objectives met' binary is also keyed to the same reward threshold, so the success criterion is self-referential. None of this is fatal to the research question, but it's fatal to the paper's stated claims as written.\n\nWho this is for: readers who want a quick view of how actor-critic behaves on a simplified orbit-planning MDP, not anyone who needs evidence that A2C beats PPO for real mission planning.\n\nRecommendation: send it to peer review, but a referee should require corrected numbers, multi-seed results, and a full reward definition before this can be accepted.","headline":"A real A2C-vs-PPO comparison on an orbit-design MDP, but the headline speedup is contradicted by the paper's own numbers and the environment is under-defined.","tokens_in":9279,"tokens_out":3355,"would_cite":false,"duration_ms":37362,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that an A2C reinforcement-learning agent can tune five Keplerian orbital elements to place an orbit over a ground target, and that it does so with a cumulative reward of 10.0 in about 2,000 timesteps, versus 9.263025 for PP","keywords":["reinforcement learning","A2C","PPO","orbital mechanics","Keplerian elements","terrestrial coverage","LEO mission planning","TLE"],"falsifier":"Check the arithmetic in Table 4: $10.0/9.263025 \\approx 1.08$, which is neither the abstract's 5.8x nor the conclusion's 73.6% gain, so the multiplier itself can be settled by division. To test the behavioral claim, define how $d_{\\text{target}}$ in Eq. (8) is computed from the orbital elements, then rerun A2C and PPO over multiple random seeds; if A2C's cumulative reward and timestep-to-convergence are not consistently better, the central comparison fails.","tokens_in":1568,"feed_emoji":"🛰️","tokens_out":1888,"duration_ms":132516,"temperature":0.7,"pith_summary":"This paper sets out to show that a reinforcement-learning agent using the Advantage Actor-Critic (A2C) algorithm can choose a satellite orbit that passes within a required radius of a specified ground target, and can do so far more efficiently than Proximal Policy Optimization (PPO). The authors build a custom environment in which the agent adjusts five Keplerian orbital elements—semi-major axis, eccentricity, inclination, right ascension of the ascending node, and argument of perigee—using a reward that combines target proximity, altitude validity, and safety-buffer distance. They report that A2C reaches a cumulative reward of 10.0 in about 2,000 timesteps, while PPO reaches 9.263025 in about 63,000 timesteps, and that the A2C policy meets its mission objectives. If this result is correct, it would make actor-critic reinforcement learning an attractive engine for fast, adaptive low-Earth-orbit mission planning.","feed_headline":"A2C tunes satellite orbits in 31x fewer training steps","feed_subtitle":"Five Keplerian parameters get adjusted to cover a ground target, reaching PPO-level reward in about 2,000 steps.","key_machinery":"The central machinery is a Markov decision process over five classical Keplerian elements inside a custom environment, paired with a shaped reward function. The reward combines a coverage-altitude term, a safety-buffer term using a hyperbolic tangent of the proximity margin, and an exponential ground-target term $R_t = e^{-3 d_{\\text{target}}/\\sigma}$; a multiplicative bonus and a sharp penalty then shape the final reward. The learning side uses A2C with orthogonally initialized LeakyReLU policy and value networks, state-dependent exploration, vectorized parallel environments, and a plateau-detection callback that force-resets the orbital configuration when progress stalls.","core_discovery":"The paper's central claim is that the A2C algorithm—synchronous actor-critic with parallel vectorized environments—learns to adjust five orbital elements to achieve terrestrial coverage, and that this policy is both better and dramatically more sample-efficient than PPO on the same Markov decision process. The evidence offered is a head-to-head table: A2C yields a cumulative reward of 10.0 after roughly 2,300 training timesteps, versus PPO's 9.263025 after roughly 61,000 timesteps, with both agents reported to meet mission objectives. The proposed explanation is that unconstrained policy updates and parallel exploration let A2C take large steps toward high-reward orbital configurations immed","pith_inferences":["The paper's own numbers need a second look: 10.0 versus 9.263025 is about an 8% reward difference, not the 5.8x stated in the abstract or the 73.6% stated in the conclusion, so the multiplier should be recomputed from the raw table before the headline result is used.","The paper does not isolate the cause of the timestep gap: the custom plateau-reset callback and state-dependent exploration could be responsible for A2C's apparent speed, so a component ablation would be needed to attribute the gain to the algorithm itself.","The method as formulated has a single scalarized reward, so it cannot trade off multiple ground targets; a natural extension would be a multi-target or constellation formulation that returns a coverage schedule rather than one orbit.","If the roughly 2,000-step convergence is reproducible across seeds and target coordinates, the same environment could be used for online replanning, where an operator re-optimizes elements whenever a target is added or removed."],"forward_implications":["If A2C's sample-efficiency claim holds, mission planners could obtain candidate orbital elements with thousands rather than tens of thousands of environment steps, making retargeting fast when ground objectives change.","The same MDP formulation could be reused for collision-aware planning by extending the safety-buffer reward to multiple active satellites and debris objects.","The policy's output is an explicit set of five Keplerian elements, so it can feed directly into standard orbit propagation and TLE-generation tools without additional representation learning.","Rapid convergence would let operators run the optimizer at deployment time for different target coordinates, rather than precomputing a fixed orbit.","The comparison strengthens the case that on-policy actor-critic methods, not only trust-region methods, are viable for continuous control in orbital mechanics."],"supporting_citations":[{"why":"Provides the asynchronous actor-critic family that A2C implements and motivates the parallel-exploration advantage the paper claims.","marker":"Mnih et al. 2016"},{"why":"Defines PPO's clipped surrogate objective, the baseline algorithm in the comparison.","marker":"Schulman et al. 2017b"},{"why":"Introduces TRPO and the trust-region reasoning the paper contrasts with A2C's unconstrained updates.","marker":"Schulman et al. 2017a"},{"why":"Defines the Gymnasium API that shapes the custom orbital environment's action, observation, and reward interface.","marker":"Towers et al. 2024"},{"why":"Prior application of reinforcement learning to low-Earth-orbit satellite management, which the paper extends to coverage-oriented orbit selection.","marker":"Kyuroson et al. 2024"}],"fun_headline_variants":["A2C beats PPO in satellite orbit planning with 5.8x reward","A2C tunes orbits in 31x fewer steps for Earth coverage","Reinforcement learning: A2C optimizes satellite orbits faster","A2C learns orbital parameters in 2,000 steps, vs PPO's 63,000","A2C outperforms PPO for satellite coverage path planning"],"cache_read_input_tokens":11264,"weakest_assumption_plain":"The argument assumes the reward function faithfully encodes mission success, but Eq. (8) never defines how $d_{\\text{target}}$ is computed from the orbital elements, so the objective the agents optimize cannot be independently reproduced or checked.","fun_headline_variants_meta":{"raw":{"variants":["A2C beats PPO in satellite orbit planning with 5.8x reward","A2C tunes orbits in 31x fewer steps for Earth coverage","Reinforcement learning: A2C optimizes satellite orbits faster","A2C learns orbital parameters in 2,000 steps, vs PPO's 63,000","A2C outperforms PPO for satellite coverage path planning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000572,"raw_usage":{"total_tokens":2583,"prompt_tokens":826,"completion_tokens":1757,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":570,"completion_tokens_details":{"reasoning_tokens":1654}},"tokens_in":570,"tokens_out":1757,"duration_ms":13332,"temperature":1.0,"reasoning_tokens":1654,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T20:13:13.829293+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Check the arithmetic in Table 4: $10.0/9.263025 \\approx 1.08$, which is neither the abstract's 5.8x nor the conclusion's 73.6% gain, so the multiplier itself can be settled by division. To test the behavioral claim, define how $d_{\\text{target}}$ in Eq. (8) is computed from the orbital elements, then rerun A2C and PPO over multiple random seeds; if A2C's cumulative reward and timestep-to-convergence are not consistently better, the central comparison fails.","supporting_citations":[{"cited_title":"A.; Satpute, S.; and Nikolakopoulos, G","cited_arxiv_id":null,"evidence_quote":"Prior application of reinforcement learning to low-Earth-orbit satellite management, which the paper extends to coverage-oriented orbit selection."}],"review_version":1}