{"id":"719995bf-c6e4-4795-b584-fbb1141df73d","arxiv_id":"2504.21731","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A PPO-based reinforcement learning agent learns to position a single 3D UI in simulated mixed reality, keeping it visible, collision-free, and near a moving user in both familiar and novel indoor scenes.","lead":"This paper trains a reinforcement learning agent to automatically move and position a 3D user interface in mixed reality, keeping it visible and reachable for a simulated walking user. It is an early feasibility study that shows RL can learn such placement, but it has not been tested with real users or compared with existing optimization methods.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (2) reachability reward has no negative exponent, so it rewards distance rather than a 0.5 m target; reported reachable placement may not follow from the stated reward.","rationale":"The reader identified the simulated-user assumption as the weakest point, which is a valid external-validity concern and is acknowledged by the authors. My pass found a more immediate internal correctness problem in Eq. (2): the printed reachability reward has no negative sign in the exponent, so it mathematically favors moving the UI far away from the user rather than maintaining a 0.5 m reachable distance. This directly affects the interpretation of Table 1's distance offset and the claim that the agent maximizes accumulated reward by keeping UI reachable. I am not claiming the authors necessarily implemented the reward incorrectly; a missing minus sign in the manuscript is plausible. But the burden is on the paper to state the reward correctly and show that the reported behavior follows from it. This does not overturn the entire feasibility study, because the visibility and non-collision results may still stand, so the verdict remains conditional; it should be conditional on correcting Eq. (2) and rerunning or re-reporting the reward function.","tokens_in":12431,"tokens_out":6277,"duration_ms":66206,"concrete_test":"Recompute R_reachability from Eq. (2) at Dist = 0.5 and Dist = 1.0 with the UI in front of the user. If the formula is implemented as printed, R_reach(1.0) = 0.349 > R_reach(0.5) = 0.1, proving the reward does not encode a 0.5 m target. Then inspect the shipped Unity/ML-Agents reward function for a missing minus sign in the exponent; if a minus sign exists, Eq. (2) is a typo and the evaluation should be rerun with the corrected equation to verify Table 1; if no minus sign exists, the reported distance offset likely arises from constraints other than the stated reachability reward and the 'reachable distance' conclusion should be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing issue is internal to the reward design, not the acknowledged limitation of the simulated user. In Section 3.2.3, Eq. (2) defines R_reachability as 0.1·exp((Dist−0.5)^4 / 0.05) when the UI is in front of the user. Because the exponent has no minus sign, the reward is minimized (0.1) at Dist = 0.5 m and grows super-exponentially as Dist moves away: at Dist = 1.0 m, R = 0.1·e^1.25 ≈ 0.349, and at Dist = 2.0 m it is astronomically larger. This is the opposite of a reachability kernel centered at a half-meter target. The paper says the reward was inspired by a kernel function [10], but the printed form is not a kernel around 0.5. The evaluation reports mean distance offsets of 0.04–0.31 m (Table 1), so either the implementation has a minus sign that is missing from the paper, in which case Eq. (2) and the numerical results must be reconciled, or the policy stays near 0.5 m because of other reward terms or environment geometry, in which case the central 'maximize accumulated reward' claim and the explanation of the reachability result are not supported. This is load-bearing because the whole feasibility argument rests on the agent optimizing the designed reward; a reward that literally rewards flying away from the user cannot demonstrate optimized 3D UI placement without further explanation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript presents a deep reinforcement learning approach for continuous 3D UI placement in mixed reality. The agent observes the user's pose, distance to the environment, and other state information, then outputs a 3D force vector to move a single virtual content item via physics simulation. The policy is trained with PPO in two Unity-simulated Replica scenes using a composite reward that combines visibility, reachability, physicality, and stability terms. A preliminary evaluation is conducted in the two training scenes and two held-out validation scenes, with and without a moving obstacle, reporting high visible UI percentage, non-collision percentage, and small distance offsets. The paper positions this as the first RL-based 3D UI placement in MR and discusses future directions such as multi-agent RL and RLHF.","tokens_in":12744,"tokens_out":5029,"duration_ms":53544,"significance":"If the results hold, the paper offers a promising alternative to per-frame optimization for adaptive MR interfaces, enabling continuous placement decisions without re-solving an objective at each step. The authors are explicit about limitations, including the simulated user's fidelity, the absence of comparison to optimization methods, and the observed instability of the UI. The paper provides a concrete training and evaluation setup that can be reproduced and extended. However, the central feasibility claim is currently undermined by a likely error in the published reachability reward equation, the lack of baselines or statistical variation, and the fact that the evaluation metrics are essentially the same as the trained reward terms. These issues must be addressed before the claims can be assessed fairly.","major_comments":[{"comment":"The reachability reward is written as R_reachability = 0.1 * exp(((Dist - 0.5)^4) / 0.05) when the UI is in front of the user. Because the exponent has no minus sign, this function grows without bound as Dist moves away from 0.5 m: at Dist = 1.0 m it is approximately 0.349, and at Dist = 2.0 m it is astronomically large. This is the opposite of a kernel peaked at a 0.5 m target. Table 1 reports distance offsets of 0.04 to 0.31 m, so either the implementation contains a minus sign missing from the manuscript, in which case Eq. (2) must be corrected, or the reported distances do not follow from the stated reward, in which case the claim that the agent maximizes the designed reward is unsupported. This issue is load-bearing because the entire feasibility argument rests on the semantics of the reward function.","section":"Section 3.2.3, Eq. (2)"},{"comment":"The evaluation reports only single values for each heuristic and environment, with no standard deviations, confidence intervals, number of seeds, or number of evaluation episodes. Since PPO is stochastic and training used six parallel instances, it is unknown whether these values are robust across runs. Furthermore, there is no baseline comparison, such as a random policy, a heuristic like 'keep the UI in front of the user at 0.5 m', or an optimization-based method. Without such comparisons, the claim that RL 'demonstrates the potential' for 3D UI placement cannot be distinguished from trivial or heuristic behavior. The authors acknowledge the missing comparison in Section 5, but this is a central aspect of the paper's stated contribution and should be addressed.","section":"Section 4.1/4.2, Table 1"},{"comment":"The evaluation heuristics (visible UI %, non-collision %, distance offset, speed) are direct aggregates of the trained reward terms (visibility, physicality, reachability, stability). The positive results therefore partly reflect the fact that the agent was optimized for these very metrics. This does not invalidate the approach, but it does mean the evaluation provides limited evidence about the general utility of the placement. An ablation study or an independent outcome measure, such as task performance, user preference, or ergonomic models, is needed to substantiate the claim of adaptive, useful placement.","section":"Section 4.1 vs Section 3.2.3"}],"minor_comments":[{"comment":"The sentence 'These include, but and not limited to' contains a typo; it should read 'but are not limited to.'","section":"Section 3.1"},{"comment":"The caption describes a histogram with 'the reward distribution (x-axis) in relation to the total number of steps (y-axis)', but a histogram typically places the binned variable on the x-axis and counts on the y-axis; please clarify the axes.","section":"Figure 2(c)"},{"comment":"The caption does not state whether the reported values are means, over how many time steps or episodes, or how many evaluation runs were performed; please specify this to make the results interpretable.","section":"Table 1 caption"},{"comment":"The statement that 'the drops of visibility/physicality were capped at 3.12%' is ambiguous; it should specify the baseline (e.g., compared to the static condition for the same environment) and clarify whether the comparison is across all environments or only validation environments.","section":"Section 4.2"},{"comment":"The authors state that thresholds for V_c and P_percent were determined empirically but do not report the chosen values; please include these values or a reference to a supplementary resource to improve reproducibility.","section":"Section 3.2.3"}],"recommendation":"major_revision","confidential_remarks":"The reachability reward equation issue is the most serious concern; it is likely a typographical omission of a minus sign, but it must be resolved because it undermines the central claim. I recommend that the authors provide the exact equation used in the implementation and, if possible, a short code snippet. The lack of baselines and variance is a weakness for a full paper, though more understandable for an extended abstract; still, the authors should be asked to add at least a random-policy or heuristic baseline and to report standard deviations or multiple seeds. The paper is otherwise clearly written and the future directions are thoughtful."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my read on 2504.21731. In its own cited context, this is the first application of model-free RL (PPO) to continuous 3D placement of a world-anchored UI for a mobile MR user. That novelty is modest but real, and the paper is honest about being an initial exploration. The design choices are standard and mostly sensible: Unity + ML-Agents, Replica scenes, ray observations, a three-dimensional force action, and a reward that combines visibility, reachability, physicality, and stability. The held-out simulation results—around 85% visibility and 82–94% non-collision in unseen scenes, with small performance drops under a moving obstacle—are plausible as feasibility evidence.\n\nWhat the paper does well is scope itself. It acknowledges the simulated-user limitation, reports training curves, and lists concrete future work, including the missing baseline comparisons and user studies. That's the right shape for an extended abstract.\n\nThe soft spots are real, and one of them is load-bearing. In Section 3.2.3, Eq. (2) defines the reachability reward as 0.1·exp(((Dist−0.5)^4)/0.05) when the UI is in front of the user. No minus sign. That function is minimized at Dist = 0.5 m and grows super-exponentially away from it—it literally rewards the UI for moving far away. Yet Table 1 reports mean distance offsets of 0.04–0.31 m from the half-meter baseline, meaning the trained policy kept the UI near the target. Those two facts cannot both be true under the stated reward. Either the implementation has a missing minus sign (a typo in the paper) or the reported placement behavior doesn't follow from the stated objective. The authors need to reconcile this; as printed, a reader cannot verify the central claim.\n\nThe other issues are more moderate. The evaluation has no comparison against heuristic or optimization baselines (the authors say so themselves), no seed-level variance, a single UI type, and the simulated user only stops on a grid. The circularity concern—the heuristics mirror the reward terms—is worth noting but not fatal for a feasibility study; it just means they showed the agent can optimize those proxies, not that the proxies equal user value. The empirically tuned thresholds are also standard practice in RL, though sensitivity analysis would help.\n\nIf I were the editor, I would send this to review—the topic is timely and the approach, once the equation is fixed, is a reasonable stepping stone for the subfield. I wouldn't cite it in its current form, and I'd want major revision before acceptance. For a reading group working on adaptive MR or view management, it's a useful example of what goes into RL reward design, precisely because of the sign error.","headline":"A plausible first step toward RL-driven 3D UI placement in MR, but the printed reachability reward contradicts the reported results and must be fixed before the feasibility claim can be verified.","tokens_in":13296,"tokens_out":4689,"would_cite":false,"duration_ms":45139,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A deep reinforcement learning agent can keep a mixed-reality 3D menu visible, reachable, and collision-free as the user moves.","keywords":["mixed reality","reinforcement learning","3D UI placement","adaptive user interface","mobile scenarios","proximal policy optimization","view management","simulation training"],"falsifier":"Run the trained policy on recorded trajectories of real users walking through an office or living room, either in a mixed-reality headset or by replaying their tracked poses in the same simulator, and measure the fraction of time the UI is visible, collision-free, and within arm's reach. If visibility falls below the simulated 84% or the UI frequently collides or drifts out of reach, then the simulated-user assumption is the point of failure.","tokens_in":12260,"feed_emoji":"🥽","tokens_out":8035,"duration_ms":69100,"temperature":0.7,"pith_summary":"This paper asks whether deep reinforcement learning can replace per-frame optimization for placing a 3D user interface in mixed reality as a user moves through an indoor space. It trains a policy-gradient agent in a physics simulator that applies a 3D force vector to a piece of virtual content every few steps, with a reward balancing visibility, reachability, non-collision, and stability. In simulated tests across multiple indoor scenes, the agent keeps the interface visible 84–93% of the time, avoids object collisions most steps, and stays within arm's reach, even with a moving obstacle. The authors conclude that a single learned policy could adapt UI placement on the go, though scalability to real users remains untested.","feed_headline":"RL keeps mixed-reality menus visible, reachable, collision-free","feed_subtitle":"A single trained agent adapts a world-anchored UI in 3D as the user walks, without per-frame optimization.","key_machinery":"The central object is a proximal policy optimization (PPO) actor–critic agent acting in a physics-based simulator. At each step the agent's observation includes 22 rays cast around the content that report hits against the environment mesh, the content's position, rotation, direction, and velocity, its distance to the user, and the user's position and direction. The action is a three-dimensional continuous force vector in $[-1,1]^3$ applied to the content, letting the physics engine accelerate or decelerate it. The reward is $R = R_{\\text{visibility}} + R_{\\text{reachability}} + R_{\\text{physicality}} + R_{\\text{stability}}$, with scale and thresholds tuned empirically. This machinery recasts UI placement as a sequential control problem: instead of re-solving an optimization whenever the user moves, the agent learns a policy that maps observations directly to placement adjustments.","core_discovery":"The authors demonstrate that a model-free reinforcement learning agent, trained with proximal policy optimization, can learn a policy that continuously maneuvers a single world-anchored 3D interface in a dynamic mixed-reality setting. The policy takes as input the content's surroundings (ray casts), its own position and motion, and the user's position and facing direction, then emits a force vector that the physics simulation applies to the content. The reward is a sum of four components: visibility (pixel percentage of the content in the user's view), reachability (an exponential function of distance when the content is in front of the user), physicality (penalty for overlap with the environment), and stability (penalty for fast content when the user is still). In the paper's preliminary evaluation, the trained agent generalizes to unseen rooms and to a randomly moving obstacle, keeping visible UI percentage above roughly 83% and non-collision percentage above roughly 82% while staying near a half-meter distance baseline. From this the authors conclude that reinforcement learning is a viable alternative to optimization-based methods for sequential, context-aware 3D UI placement in mobile mixed reality.","pith_inferences":["A natural next experiment is to record real pedestrian trajectories with a head-mounted device and replay them through the same simulator; if the visibility and reachability metrics stay near the simulated values, the grid-based user model may be sufficient for training.","The force-vector action space behaves like a learned PD controller for UI position; a simple spring-to-view-center baseline with collision avoidance would quantify how much the RL training adds over a hand-tuned controller.","The paper's reported high average UI speed suggests the stability reward is too weak when the user is moving; conditioning the penalty on user velocity rather than only on user stillness would be a cheap, testable fix.","The same training loop could be extended to multiple UI elements by treating each as an independent agent with a shared reward, testing whether multi-agent cooperation emerges without redesigning observations."],"forward_implications":["A single trained policy can run at interactive rates on a mobile device, deciding the UI position every few steps without recomputing an optimization from scratch.","The policy's behavior transfers to room layouts it never trained on, with only a modest drop in visibility, suggesting that simulation-based training in a few scenes may cover unseen indoor spaces.","The agent can compensate for an unexpected moving obstacle by trading a little visibility for collision avoidance, indicating the approach may handle bystanders or moving furniture.","The hand-crafted reward is modular: adding semantic, ergonomic, or preference-based terms later would reshape placement without changing the learning algorithm.","Because evaluation is entirely simulated, the paper's direct claim is about a simulated mobile user; real-world generalization is explicitly left open."],"supporting_citations":[{"why":"Supplies the scanned indoor room meshes used to build the training and validation environments.","marker":"[41]"},{"why":"Provides the simulation-to-training bridge that lets the agent interact with the physics environment during PPO training.","marker":"[27]"},{"why":"Defines the proximal policy optimization algorithm that the agent is trained with.","marker":"[39]"},{"why":"Earlier RL work on AR label placement in a static viewport; this paper extends that idea to world-anchored 3D content under a moving user.","marker":"[8]"},{"why":"Optimization-based MR layout adaptation whose visibility and reachability objectives inform the reward design.","marker":"[10]"},{"why":"An adaptive UI toolkit with multiple objectives that motivates the reward components for physicality and reachability.","marker":"[17]"},{"why":"A context-aware online adaptation system that represents the optimization-based approach the paper contrasts with RL.","marker":"[32]"},{"why":"Pareto-optimal layout work for MR that defines similar multiple objectives, cited as related optimization methodology.","marker":"[26]"}],"fun_headline_variants":["RL agent adapts 3D UI placement in mixed reality as you walk","Single PPO policy keeps MR menus visible, reachable, collision-free","Reinforcement learning steers 3D UI in AR, avoids obstacles","Deep RL learns to keep world-anchored content in view and reachable"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything rests on the assumption that a simulated user stopping at random grid points facing random directions is a faithful enough proxy for a real person walking with intent, so the policy learned from that simulation will serve actual users; the authors acknowledge this mismatch in their problem formulation.","fun_headline_variants_meta":{"raw":{"variants":["RL agent adapts 3D UI placement in mixed reality as you walk","Single PPO policy keeps MR menus visible, reachable, collision-free","Reinforcement learning steers 3D UI in AR, avoids obstacles","Deep RL learns to keep world-anchored content in view and reachable"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000278,"raw_usage":{"total_tokens":1627,"prompt_tokens":893,"completion_tokens":734,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":509,"completion_tokens_details":{"reasoning_tokens":652}},"tokens_in":509,"tokens_out":734,"duration_ms":7088,"temperature":1.0,"reasoning_tokens":652,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:54:14.868491+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the trained policy on recorded trajectories of real users walking through an office or living room, either in a mixed-reality headset or by replaying their tracked poses in the same simulator, and measure the fraction of time the UI is visible, collision-free, and within arm's reach. If visibility falls below the simulated 84% or the UI frequently collides or drifts out of reach, then the simulated-user assumption is the point of failure.","supporting_citations":[],"review_version":1}