{"id":"b466a3a6-cda0-450f-8aa3-ae3c0c3dcdf1","arxiv_id":"2507.18883","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"This paper reports the first stable partial-observability training on Humanoid-v4, using a parallel history encoder that matches full-state TD3 performance in most tested state-removal settings.","lead":"A reinforcement learning agent trained in the Gymnasium Humanoid-v4 simulator learns to walk forward at near full-observation performance even when the simulator withholds one-third to two-thirds of the state variables. If the results hold, a fixed-length history encoder could reduce the sensor information humanoid robots need without relying on recurrent neural networks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Causal role of the 'novel history encoder' is untestable: the encoder is never specified, code and hyperparameters are absent, and the paper itself defers architecture details to a separate report.","rationale":"The load-bearing condition for the abstract's claim is that the novel history encoder, not some incidental detail, causes the partial-observability success. That condition fails at the level of verifiability: the encoder is absent from the manuscript. I agree with the reader's REJECT but for a slightly different reason than the reader's stated 'weakest_assumption' about realism of simulated partial observability. The realism concern is also real: deleting constant mass/inertia or known torques is much easier than sensor noise, latency, or unobservable disturbances. But the more immediate blocker is that we cannot tell what algorithm produced the numbers. If code and the separate report are provided, and if the simple history-stacking ablation shows a large gap, the central claim would be supported; if the gap is small, the novelty claim collapses. The reader's weakest_assumption and my concern are complementary; both point toward non-verifiability, but the causal attribution gap is the one that makes the current submission unassessable. With the code and ablations, the paper could move to a conditional accept; without them, REJECT is appropriate. No fraud is implied; the issue is that the key artifact is missing.","tokens_in":5325,"tokens_out":4512,"duration_ms":48756,"concrete_test":"Request the separate architecture report and code (or release them); then run a direct ablation on the same TD3 framework: replace the parallel history encoder with a plain concatenation of the last k raw observations fed to an MLP, keeping sequence length, normalization, and all hyperparameters identical, on Remove VM and Remove VF for 3M steps with five seeds. If the plain history-stacking baseline reaches within 10% of the reported mean final reward, then the reported success does not depend on the novel encoder. As a secondary check, verify that RMF and ODERMF receive the same observation history format and normalization before comparing curves.","verdict_should_be":"REJECT","load_bearing_attack":"The central positive claim is that a novel parallel history encoder is the key to successful partial-observation humanoid learning. The evidence for this is a set of training curves and tables for an otherwise-described TD3 variant, but the encoder itself is not defined: no sequence length, layer structure, positional encoding, integration with the policy/value networks, or loss is given. The paper states, 'Further details will be provided in a separate report that discusses the architecture and mechanisms of our method,' and says only that 'most hyperparameters follow the original TD3 configuration.' This is an explicit self-declared gap in the proof of the main claim. Because the method is incompletely specified, the comparison against RMF and ODERMF cannot assign credit to the encoder: any advantage might come from simple history stacking, observation normalization, the shared codebase, or hyperparameter choices. The claim 'the encoder enables performance on par with fully observed baselines' is therefore not an established result. This is an internal verifiability problem, not a disagreement with consensus; the manuscript lacks a checkable description of the very mechanism that its title and abstract credit.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper claims the first stable training of a humanoid locomotion policy under partial observability in Gymnasium Humanoid-v4, using a 'novel history encoder' that processes fixed-length observation sequences in parallel and is integrated into a model-free TD3-style algorithm. Experiments remove one or two semantic blocks from the 348-dimensional MuJoCo state vector (velocity, mass/inertia, force), with joint positions always retained, and compare against two recurrent baselines (RMF and ODERMF) and a full-state TD3 baseline. The authors report that the proposed method matches or exceeds full-state TD3 in several configurations, extends to 3 million gradient steps, and adapts to body-mass perturbations. The numerical tables are internally consistent and show large margins over the recurrent baselines in the one-attribute-removal settings, but the manuscript does not specify the encoder architecture, omits the recurrent baselines from the two-attribute experiments, reports very high variance in two-attribute results, and provides no ablation or analysis supporting the causal role of the encoder.","tokens_in":5468,"tokens_out":5054,"duration_ms":58322,"significance":"If the central claim is correct, this would be a useful empirical result: a model-free, memory-based method that succeeds at high-dimensional humanoid POMDP control with reduced observation dimensionality, plus a clean taxonomy of which state attributes matter. The one-attribute-removal comparisons are concrete and the observation-attribute decomposition is a good idea. However, the significance is currently limited by three factors. First, the method itself is not specified, so the result cannot be reproduced or even checked; the paper explicitly defers the architecture to a separate report. Second, the two-attribute results, which are those closest to the paper's 'one-third of states' claim, lack baseline comparisons and have large seed variance, so the 'stable training' claim is overstated. Third, the encoder's causal role is asserted rather than evidenced; no ablation, code, or analysis distinguishes the encoder from hyperparameter choices or simple history stacking. These are not cosmetic issues; they bear directly on the paper's main contribution.","major_comments":[{"comment":"The central mechanism of the paper, the 'novel history encoder,' is never specified. The manuscript says only that 'most hyperparameters follow the original TD3 configuration' and that 'further details will be provided in a separate report' about the architecture. There is no description of the sequence length, network depth, positional encoding, normalization, or how the encoder is integrated with the actor and critic. Without this information, the comparison against RMF and ODERMF cannot be reproduced, and credit cannot be assigned to the encoder as opposed to observation preprocessing, the shared codebase, or hyperparameters. Because the abstract and title attribute the success to this encoder, this missing specification is a load-bearing gap that must be fixed in the manuscript itself.","section":"Results, second paragraph"},{"comment":"The two-attribute results, which are the evidence for the paper's 'one-third of states' claim, are presented without the RMF and ODERMF baselines. The text says those baselines fail when one attribute is removed and therefore are not included, but this does not demonstrate their behavior in the two-attribute settings; it is possible that some two-attribute configurations are easier or harder for recurrent methods in a way that changes the comparison. As a result, the claim that the proposed method is uniformly superior under two-attribute removal is not supported. The full-state TD3 baseline is also not repeated in Table 3, making the comparison to 'full observability' less direct; the authors reference the same TD3 number, but a reader cannot see the statistic alongside the partial-observation rows.","section":"Table 3 and 'Removal of Two Observation Attributes'"},{"comment":"The reported standard deviations are extremely large relative to the means: for example, Ours (1M) Remove VM has 974.6 ± 535.3 and Ours (1M) Remove VF has 1382.1 ± 730.9; the 3M Remove VM value is 2978.3 ± 1310.6. These values are consistent with runs that frequently fail and only occasionally succeed, and they undermine the paper's stated claim that 'our method remains robust across all partial observability configurations' and the abstract's phrase 'stable training.' The paper should report per-seed curves or at least confidence intervals, and the claim of stability should be qualified for configurations with high variance.","section":"Table 3, Remove VM and Remove VF rows"},{"comment":"The paper hypothesizes that the encoder 'reconstructs essential contextual information from recent observations,' but no evidence is offered for this mechanism. There is no latent-state probing, no ablation comparing the parallel encoder against a simple concatenation of recent observations, no control experiment varying the history length, and no analysis showing that the encoder recovers the removed velocity or mass information. The observed behavior could plausibly arise from the retained position features alone plus memory, or from the extra computation of processing a sequence. The causal claim in the abstract ('the key to this success is a novel history encoder') therefore goes beyond what the experiments demonstrate.","section":"Discussion and Abstract"}],"minor_comments":[{"comment":"The environment name is written as 'Mujoco Humanoid-v4'; the standard name is 'MuJoCo Humanoid-v4' and the capitalization should be corrected.","section":"Results, first paragraph"},{"comment":"The table reports only the average return for each mass-variation scenario, although the text says each scenario was evaluated ten times. Reporting standard deviations or individual trial values would allow the reader to assess the reliability of the mass-adaptation claim.","section":"Table 4"},{"comment":"The paper states that all models use 'matched network capacity and consistent hyperparameters,' but it does not specify the hidden sizes, LSTM layers, or optimizer details for RMF and ODERMF. Such details are needed for a fair comparison and for replication.","section":"Results, 'Removal of One Observation Attribute'"},{"comment":"The claim that 'no prior work has demonstrated stable training' is a strong literature claim. The manuscript does not include a systematic search or comparison of prior partial-observability results on this exact benchmark; a more carefully qualified statement or a reference to an existing negative result would be appropriate.","section":"Abstract and Introduction"},{"comment":"Figures 1 and 2 would be easier to interpret if the captions stated the number of seeds, the error-band convention, and the smoothing procedure used to plot the curves.","section":"Figure captions"}],"recommendation":"reject","confidential_remarks":"The core problem is not that the results are necessarily wrong, but that the manuscript as submitted does not permit anyone to check them: the method is deferred to an unnamed separate report, no code or hyperparameters are provided, and the key causal claim is not supported by ablation or analysis. Even if the authors were to add an architecture section, the two-attribute experiments need baseline comparisons and variance reporting before the central claims could be evaluated. This is beyond a minor revision. I would be willing to reconsider if a substantially expanded version with full method details, code, baselines, and variance-aware claims is submitted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"For you in one breath: this is the first time I've seen stable partial-observability training on Humanoid-v4 with full-state-level returns, and the margin over RMF and ODERMF is large enough to take seriously. But the paper as written cannot support its own headline, because the thing it credits for the result—the \"novel history encoder\"—is never actually described. The authors say, in the paper itself, that architecture and mechanism details will be in a separate report. That is an honest admission, but it is also a load-bearing hole: with no sequence length, layer structure, positional encoding, or integration into the policy and value networks, there is no way to verify the encoder exists as claimed, no way to reproduce the curves, and no way to assign credit to the encoder versus simple history stacking, observation normalization, or hyperparameter tuning.\n\nWhat is genuinely good: the one-attribute removal tables are internally consistent, the comparison against RMF and ODERMF is credible in direction and size, and the mass-variation evaluation is a useful robustness check. The citation pattern looks fine—RMF and ODERMF are real, relevant baselines, and the self-citation issue is not present. The benchmark result, if the method actually works, is a meaningful step for POMDP continuous control.\n\nThe soft spots beyond the missing encoder: the two-attribute experiments omit the baselines altogether, so we cannot see whether the recurrent methods would also partially improve with more training. The variance in the two-attribute tables is high, especially Remove VM at 3M (2978 ± 1310), which the text acknowledges with \"meaningful performance\" but should be weighed against the abstract's \"comparable\" wording. And the partial-observability model itself is just dimension deletion with joint positions always kept; no sensor noise, delay, or latent variables. That is a legitimate simplification, but it means the real-world transfer claim is weaker than the paper suggests.\n\nMy bottom line: the paper deserves a serious referee, not a desk reject in the strong sense, because the reported numbers are specific and potentially important. But a referee should be told to demand the full method, code, and baseline results before accepting anything. As submitted, I would not recommend acceptance; I would recommend a major revision that puts the architecture in the paper.\n\nThis is a paper for people working on memory and partial observability in continuous control, and for editors who want to push the community toward actually specifying model-free POMDP algorithms. If the authors provide the omitted material, this could become a useful benchmark paper.","headline":"The empirical result looks real and novel, but the paper is not self-contained: the central history encoder is never specified, so the main causal claim cannot be checked.","tokens_in":6019,"tokens_out":2509,"would_cite":false,"duration_ms":31978,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A parallel history encoder lets model-free reinforcement learning succeed at humanoid locomotion under partial observability, matching full-state performance with as little as one-third of the state.","keywords":["humanoid locomotion","partial observability","history encoder","parallel temporal encoding","model-free reinforcement learning","state reconstruction","mass adaptation","Humanoid-v4 benchmark"],"falsifier":"On the same humanoid task, keep the same available dimensions but corrupt every retained observation with zero-mean noise and a one-step delay, and randomly drop readings from one joint. If the encoder-based policy's final return falls to the level of the recurrent baselines, the success depends on clean missing dimensions rather than on true reconstruction under partial observability.","tokens_in":5077,"feed_emoji":"🦿","tokens_out":9357,"duration_ms":89835,"temperature":0.7,"pith_summary":"This paper tries to show that high-dimensional humanoid locomotion can be learned when the controller sees only part of the robot's state, a setting where prior reinforcement-learning methods have failed. In the Humanoid-v4 benchmark, a model-free policy using a fixed-length parallel history encoder reaches the same level of performance as a baseline with full state access, while seeing only one-third to two-thirds of the simulator's 348-dimensional state. The same policy also keeps working when body-part masses change, which the authors take as evidence that the encoder reconstructs enough context from recent observations. If true, this would make humanoid control far less dependent on complete, low-noise sensing.","feed_headline":"Humanoid learns to walk using only one-third of its state data","feed_subtitle":"A fixed-length parallel history encoder lets a model-free policy match full-state performance with much less information","key_machinery":"The carrying mechanism is the parallel history encoder: a network that processes a fixed-length sequence of past observations all at once, assigning equal weight to each time step rather than compressing history sequentially the way recurrent networks do. Integrated into a standard model-free actor-critic algorithm, this encoder gives the policy temporal context from which it can infer state information missing from the current observation. The paper's hypothesis is that this reconstruction of essential context is what makes stable locomotion possible.","core_discovery":"The central claim is that partial observability does not prevent successful humanoid control if the policy is given a fixed window of recent observations and a parallel encoder that treats every time step in that window equally. Removing one or two semantic blocks of the state—velocity, mass/inertia, or force—while always keeping joint positions, the encoder-based method trains stably and matches or beats a full-state baseline in several configurations; recurrent sequential baselines do not learn in the same settings. The paper also reports that training with body masses varied between 50% and 100% of default values still produces a policy that handles ±50% mass changes on every body part. It concludes that the encoder reconstructs essential latent context from recent observations, and that velocity is the most critical missing attribute while force is the least.","pith_inferences":["Beyond the paper: the block-removal protocol leaves the retained channels clean; adding sensor noise, dropout, or delay to those channels would test whether the encoder's reconstruction survives realistic sensing imperfections.","Beyond the paper: because the window is fixed-length and parallel, the encoder has no mechanism for very long-range dependencies; tasks where a hidden quantity drifts over hundreds of steps may still require recurrent or state-space memory.","Beyond the paper: the full-state baseline is a single model-free algorithm; comparisons with other full-state methods and with a state-inference oracle would clarify how much of the gap the encoder actually closes."],"forward_implications":["If the claim holds, humanoid controllers could be built with far fewer sensors, since the method works with 34–73% of the original state dimensions.","Sensor design should prioritize joint positions and velocity, since removing velocity hurts most and removing force hurts least.","Architecture matters more than memory length: sequential recurrent processing fails on this task, while equal-weight parallel history succeeds.","Body-mass and inertia information are not strictly needed for locomotion, so policies trained without them may transfer to robots with altered or unknown mass distributions.","Configurations that remove velocity and mass converge slowly but partially recover with more training, so the encoder's reconstruction improves with experience."],"supporting_citations":[{"why":"supplies the recurrent LSTM baseline that the paper's encoder outperforms in the humanoid partial-observation setting.","marker":"(13)"},{"why":"supplies the ODE-based recurrent baseline used as a second comparison point.","marker":"(14)"},{"why":"defines the Humanoid-v4 environment, its 348-dimensional state, and the observation structure used in all experiments.","marker":"(21)"}],"fun_headline_variants":["Humanoid RL succeeds with one-third states via history encoder","First partial-observation humanoid locomotion in Gymnasium","Humanoid walks with 2/3 states missing, matches full-state","History encoder enables humanoid RL under partial observability","Partial observability conquered for humanoid RL with encoder"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that removing whole blocks of simulator state, while always keeping joint positions, faithfully represents what partial observability means; if real sensing adds noise, delay, occlusion, or hidden quantities that positions cannot recover, the reported performance may not transfer.","fun_headline_variants_meta":{"raw":{"variants":["Humanoid RL succeeds with one-third states via history encoder","First partial-observation humanoid locomotion in Gymnasium","Humanoid walks with 2/3 states missing, matches full-state","History encoder enables humanoid RL under partial observability","Partial observability conquered for humanoid RL with encoder"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000224,"raw_usage":{"total_tokens":1435,"prompt_tokens":897,"completion_tokens":538,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":513,"completion_tokens_details":{"reasoning_tokens":457}},"tokens_in":513,"tokens_out":538,"duration_ms":5622,"temperature":1.0,"reasoning_tokens":457,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:06:50.391704+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On the same humanoid task, keep the same available dimensions but corrupt every retained observation with zero-mean noise and a one-step delay, and randomly drop readings from one joint. If the encoder-based policy's final return falls to the level of the recurrent baselines, the success depends on clean missing dimensions rather than on true reconstruction under partial observability.","supporting_citations":[],"review_version":2}