{"id":"0e6b8c52-1c1a-415d-9142-0e748b534bd7","arxiv_id":"2508.11022","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"The paper introduces GhostObjects, interactive world-aligned virtual twins in AR that users manipulate to specify robot instructions, including multi-object lasso selection and snap-to-default placement.","lead":"This paper proposes instructing robots by letting users grab and move life-size virtual twins of physical objects in augmented reality, instead of using natural-language commands or direct teleoperation. A reader might find it useful as a concept for end-user robot programming, though no user study or robot experiment is included.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim that GhostObject manipulations become executable robot instructions is unsupported: Section 3.2 sends 'action and spatial parameters' to the robot with no mapping or execution, and Section 4 defers all evaluation.","rationale":"I agree with the reader that the load-bearing assumption is the unverified translation from GhostObject manipulations to executable robot actions. The paper contains no implementation or evaluation of this pipeline; Section 3.2 merely asserts that parameters are 'sent to the robot,' and Section 4 confirms further evaluations are future work. This is sufficient to make the central claim conditional rather than established. A secondary but related gap is that Section 2.3 claims spatially aligned virtual twins created from Meta Quest spatial anchors, but does not explain how per-object GhostObjects (e.g., foam blocks, bottle) are detected or associated with anchor positions; this strengthens the need for an end-to-end prototype test. The proposed concrete test—an end-to-end execution with a real robot measuring placement error—would settle both. The paper is a reasonable design contribution, so a CONDITIONAL verdict is appropriate; I see no basis to change it.","tokens_in":4866,"tokens_out":6724,"duration_ms":71252,"concrete_test":"Implement the robot-side execution for the pick-and-place scenario: export the transformed GhostObject poses (position, orientation, scale) from the AR system and feed them to a motion planner and real manipulator; run at least 20 trials and measure end-effector placement error against the user-specified targets. If the exported poses cannot be converted to executable commands, or if placement error exceeds the task tolerance, the central claim fails. For the filling scenario, the same test would require a defined mapping from the deformed GhostObject to a pour amount and a successful physical pour.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2 states 'Once the user's instructions are complete, the action and spatial parameters are sent to the robot.' This is the sole bridge from AR manipulation to robot instruction; no robot-side mapping, planner interface, grasp generation, or execution result is described or referenced. Section 4 says 'we intend to conduct further evaluations of the efficacy of using GhostObjects in real-world robot tasks,' confirming that no robot execution was performed. The abstract's claims of precisely specifying physical goals and enabling tasks beyond simple pick-and-place are therefore unsupported by any measurement or implementation. The deformation example (filling a bottle, Section 3.2) is especially ungrounded: a filled GhostObject specifies a liquid level, but the action semantics—e.g., what the robot should pour, how much, and how to map the deformation to an executable skill—is entirely undefined. If this translation is lossy or absent, the system manipulates virtual objects but does not instruct robots.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GhostObjects: world-aligned, life-size virtual twins of physical objects in augmented reality, which users manipulate directly to specify robot instructions. It describes single-object raycast selection, multi-object lasso selection, spatial alignment via Meta Quest Space Setup and MRUK, direct manipulation with simulated physics, and a snap-to-default feature. Two illustrative walkthroughs are provided: relocating foam blocks to a basket, and specifying a water fill level by deforming a GhostObject. The abstract claims that direct manipulation of GhostObjects lets users \"precisely specify physical goals and spatial parameters\" and enables tasks beyond simple pick-and-place; the paper's conclusion defers further evaluation of GhostObjects in real-world robot tasks.","tokens_in":5001,"tokens_out":3146,"duration_ms":37831,"significance":"The core idea is timely and potentially useful: a co-located AR interaction modality in which end users specify robot goals by manipulating virtual twins, rather than through language, demonstration, or teleoperation, could lower the barrier for personalized spatial instructions. The paper's concrete contributions include the raycast and lasso selection interaction design, the use of headset spatial anchors to avoid manual calibration, and the snap-to-default mechanism for reusing prior configurations. These are promising design elements for an HRI/AR systems paper. However, the present manuscript is a 3-page adjunct description with illustrative walkthroughs only: there is no running robot, no end-to-end execution, no user study, and no measurement of selection accuracy, placement error, or task completion. As a result, the abstract's precision and task-generality claims are currently unsubstantiated. The contribution is better described as an interaction concept with an implemented AR prototype fragment, not a validated method for instructing robots.","major_comments":[{"comment":"The central claim that GhostObject manipulations become executable robot instructions is not supported. Section 3.2 states \"Once the user's instructions are complete, the action and spatial parameters are sent to the robot,\" but no robot-side mapping, planner interface, grasp generation, action semantics, or execution result is described or referenced. In particular, the bottle-filling example is ungrounded: a deformed water GhostObject specifies a target liquid level, yet the paper does not explain how that deformation is parsed into a robot skill (what to pour, how much, from where, or with what end-effector). Without this translation, the system moves virtual objects but does not, as demonstrated, instruct a robot. The claims of \"precise\" specification and \"tasks beyond simple pick-and-place\" in the abstract are therefore not evidenced. Please either provide a concrete implementation of the robot-side interpreter and a demonstration of it executing a GhostObject-specified task, or explicitly scope the contribution to the AR goal-specification interface rather than to robot instruction.","section":"Section 3.2, Figure 1"},{"comment":"The paper's own conclusion states that the authors \"intend to conduct further evaluations of the efficacy of using GhostObjects in real-world robot tasks.\" This admission confirms that no real robot was used in the work. Yet the abstract and Introduction make quantitative and general claims: users \"precisely specify physical goals and spatial parameters,\" and the approach gives \"greater precision\" than language commands. None of these claims is backed by a user study, a robot execution log, a placement error measurement, a selection accuracy measurement, or a comparison with HoloSpot, ImageInThat, or other baselines. The precision claim is especially load-bearing because it is the stated advantage over natural-language interfaces. At minimum, the paper needs a controlled demonstration in which a real robot carries out a GhostObject-specified task and the resulting placement or state-change error is reported; otherwise the claims should be weakened to \"intended to enable\" and \"may provide precision,\" with the concept presented as a work-in-progress.","section":"Section 4, Conclusion and Future Work"},{"comment":"The implementation description is too thin to establish that the system works as described or to permit replication. The lasso selection is defined only informally: \"intersection points connect\" and \"objects that overlap this volume are grouped,\" but the paper does not specify how the closed boundary is triangulated into a selection volume, how occlusions are handled, or what happens when an object is only partially inside the lasso. Similarly, direct manipulation of deformable GhostObjects (e.g., the \"liquid level\" change in Section 3.2) is mentioned without describing how the deformation is modeled, recognized, or mapped to a parameter. The spatial alignment paragraph says MRUK provides anchor positions for spawning GhostObjects, but no alignment accuracy or calibration procedure is reported. These omissions make it difficult to judge whether the interaction techniques are implementable or whether the walkthroughs are scripted illustrations. Please provide concrete algorithmic or implementation details for lasso volume construction, deformation-to-parameter mapping, and anchor-based alignment, or clearly label the walkthroughs as conceptual mockups.","section":"Sections 2.1–2.4"}],"minor_comments":[{"comment":"Figure 2's caption mentions the lasso intersecting a sofa, carpet, and table, but the accompanying illustration is hard to parse; a clearer overlay showing the lasso boundary, the raycast, and the resulting selection volume would improve understanding.","section":"Section 2.2, Figure 2"},{"comment":"The hardware and software stack is identified as Meta Quest Space Setup and MRUK, but the specific headset model, SDK versions, and coordinate-frame conventions are not given; these details matter for reproducibility of the spatial alignment.","section":"Section 2.3"},{"comment":"The \"water GhostObject\" is not clearly defined: it is unclear whether it is a distinct virtual object, a deformation of a bottle GhostObject, or a separate state layer, and the figure should illustrate the deformation gesture and the resulting desired fill level.","section":"Section 3.2, Figure 4"},{"comment":"The paper introduces GhostAR [2] in one sentence and then names the new concept \"GhostObjects,\" which is terminologically confusing; a brief sentence explicitly distinguishing the two would help readers.","section":"Introduction"},{"comment":"Reference [4] lists \"IEEE, IEEE, Japan\" and should be cleaned up; also, several cited arXiv/technical reports lack consistent version or access details.","section":"References"},{"comment":"The phrase \"In sum, we propose instructing robots by directly interacting with virtual twins\" is repeated with slight variation in the same paragraph; condensing would improve readability.","section":"Section 1"}],"recommendation":"major_revision","confidential_remarks":"This is a 3-page UIST Adjunct style contribution that currently reads as a work-in-progress teaser. The strength is the design space: co-located, life-size virtual twins with multi-object selection and snap-to-default are interesting interaction primitives for robot goal specification. The weakness is that the abstract claims an end-to-end robot instruction method while the paper provides neither a robot-side mapping nor any execution evidence, and the conclusion explicitly defers evaluation. The load-bearing issues are fixable in a revision, either by adding a working robot pipeline with at least one executed task and basic accuracy measurements, or by recalibrating the claims to describe an AR goal-specification technique that is intended for future robot integration. No concerns about novelty disclosure or citation behavior; the related work coverage appears appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [colleague],\n\nRead the GhostObjects UIST Adjunct piece. Short version: it is a clearly written concept paper for a direct-manipulation AR robot instruction modality, and the specific combination — co-located, life-size, selectable virtual twins with lasso and snap-to-default — is not in the prior work I know. The authors position it carefully against HoloSpot, Marcer, ImageInThat, and X-HR Training. Credit where due: the interaction design is concrete, the walkthroughs are legible, and the writing is honest that this is early work.\n\nThe soft spot is exactly what the stress-test says. The abstract claims users 'precisely specify physical goals and spatial parameters,' but there is no user study, no robot execution, no measurement of placement error, and no description of how the manipulated GhostObject state is translated into robot actions. Section 3.2 says 'action and spatial parameters are sent to the robot' — that is the entire bridge, and it is a black box. The bottle-filling deformation example is the weakest: a liquid-level GhostObject does not by itself define what the robot should do. This is not a fatal flaw for a 3-page concept paper, but the claims need softening or the paper needs at least a minimal proof-of-concept robot run.\n\nI do not think the stress-test is wrong. It is the correct reading. The paper's own conclusion defers real-world evaluation, so the right framing is 'design exploration with plausible value, unverified performance claims.'\n\nWho is this for? Researchers in AR/robot programming, especially those working on co-located instruction interfaces. It would be a reasonable poster or demo at UIST; it is not yet a full conference paper. A serious referee should engage with it, because the design space is real and the combination is new, and the path to validation is clear: add a small user study, a real robot mapping, and error measurements, and the idea could become a solid short paper.\n\nRecommendation: send to peer review, yes. It deserves referee time, but the reviewers should hold the authors to softening 'precisely' and providing at least one execution trace.\n\nBest,\n[your name]","headline":"A clean, well-scoped AR robot-instruction concept whose new combination is real, but whose 'precise specification' claim is unsupported by any robot execution or user measurement.","tokens_in":5547,"tokens_out":2173,"would_cite":false,"duration_ms":22879,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Moving a life-size virtual twin in augmented reality is proposed as a precise way to instruct robots, replacing language, demonstration, and teleoperation.","keywords":["human-robot interaction","robot instructions","end-user robot programming","augmented reality","mixed reality","direct manipulation","spatial alignment","virtual twins"],"falsifier":"Connect the AR headset to a real robot and have a user lasso two objects, drag their twins to a shelf, and raise a bottle twin's water level to a chosen mark; then run the generated instructions and measure the final object poses and fill level. If the robot's placement error exceeds the tolerance the user can see in the AR view, or if the deformation parameter does not reach the robot, then the paper's claim that direct manipulation precisely specifies physical goals fails at execution.","tokens_in":4642,"feed_emoji":"🥽","tokens_out":6139,"duration_ms":61046,"temperature":0.7,"pith_summary":"The paper argues that people should program robots by directly manipulating life-size virtual copies of real objects in augmented reality, rather than by typing commands, demonstrating motions, or teleoperating the robot. It claims that because these GhostObjects are world-aligned and overlaid on the actual objects, a user's grab, drag, lasso, and release actions can precisely specify both what should happen and where. The payoff would be an instruction channel that feels direct: move the ghost, and the robot has its goal. The paper demonstrates the interactions in a headset walkthrough and leaves the robot-side translation and user evaluation to future work.","feed_headline":"Move a virtual twin to command a real robot","feed_subtitle":"No language or teleoperation: life-size AR copies of objects carry precise spatial goals to the robot.","key_machinery":"The central mechanism is the GhostObject: a life-size virtual twin spawned at the position of its physical counterpart using the headset's spatial anchors, so it stays aligned with the real world and needs no repeated calibration. The argument rides on three interaction features. Raycast selection grabs the first object the controller points at; lasso selection sweeps the controller to enclose multiple objects whose twins then move as a group; snap-to-default stores a pose and draws an arched trajectory to it, letting release along that path return the twin to its home location. The same manipulation language lets users deform twins—the water level in a bottle—to encode parameters of state-changing tasks.","core_discovery":"In the authors' own framing, the central claim is that users can issue personalized robot instructions by directly manipulating GhostObjects—spatially aligned, life-size virtual twins of physical objects—and that the resulting poses and deformations constitute precise specifications of physical goals and spatial parameters. Selecting an object is done by raycast or by drawing a lasso in the real scene; selected twins can be grabbed, moved, snapped back to a stored default along an arched trajectory, or deformed, for example by raising the liquid level in a bottle's twin. This is presented as covering not just pick-and-place but state-changing tasks such as filling. Direct manipulation is offered as the alternative to language, demonstration, and teleoperation because it removes the mental mapping between an abstract command and the physical layout.","pith_inferences":["A testable extension the paper does not run is a head-to-head comparison measuring final object-pose error for GhostObject instructions versus language instructions and 2D drag instructions on the same rearrangement task.","The deferred robot-side mapping is where the method stands or falls: one could treat each final GhostObject pose as a target in the robot's coordinate frame and measure end-to-end execution error.","Co-located life-size manipulation is likely to matter most for tasks where the user judges aesthetics or fit against the real environment, such as hanging a painting or arranging furniture, rather than tasks with fixed coordinates.","Synthetic or reconstructed twins from a single photo could generalize the interaction to objects not currently in the room, turning GhostObjects into a spatial instruction language for future-layout or remote planning."],"forward_implications":["If GhostObject manipulation works as claimed, a user can specify a group rearrangement with a single lasso gesture rather than issuing one command per object.","Snap-to-default gives a natural way to express 'put it back where it belongs,' with the stored default pose acting as the robot's goal and redefinable by the user.","Because the twins are co-located and life-size, instructions are authored in the same space the robot will operate in, removing the need for a miniature or 2D proxy that users must mentally map onto reality.","Deformation of twins extends the same direct-manipulation grammar beyond pick-and-place to parameters such as fill levels, so state-changing tasks could be specified without language.","If the precision claim holds, this offers a middle path between ambiguous language commands and high-workload teleoperation for end-user robot programming."],"supporting_citations":[{"why":"contrast baseline: a language-based agent with high-level semantic grounding, which the paper argues carries ambiguity and discoverability problems that direct manipulation avoids.","marker":"[1]"},{"why":"prior embodied authoring system that relies on user demonstration and understanding of robot capabilities; GhostObjects is positioned against this cost.","marker":"[2]"},{"why":"nearest prior drag-and-drop instruction interface, but in a miniature non-co-located world; the paper's co-located life-size twins extend it.","marker":"[7]"},{"why":"the direct-manipulation interface theory used to justify that grabbing and moving twins minimizes the mismatch between user intention and task execution.","marker":"[9]"},{"why":"example of AR robot instructions that still use language and non-interactive visualizations; GhostObjects adds interactivity.","marker":"[10]"},{"why":"2D image-dragging instruction baseline that the paper extends into a 3D co-located scene.","marker":"[12]"},{"why":"prior AR human-robot training where virtual twins are manipulated but are not spatially aligned with physical counterparts; the paper's alignment is the differentiator.","marker":"[16]"}],"fun_headline_variants":["GhostObjects: move AR twins to command real robots","Directly manipulate life-size AR twins to teach robots","No language needed: AR ghost twins carry robot goals","Drag virtual twins in AR to set precise robot tasks","Instruct robots via spatial AR twins: GhostObjects"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The approach assumes that a robot system can reliably translate the final poses and deformations of the manipulated virtual twins into executable actions with the intended spatial parameters, a mapping the paper mentions in one sentence and never tests.","fun_headline_variants_meta":{"raw":{"variants":["GhostObjects: move AR twins to command real robots","Directly manipulate life-size AR twins to teach robots","No language needed: AR ghost twins carry robot goals","Drag virtual twins in AR to set precise robot tasks","Instruct robots via spatial AR twins: GhostObjects"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000252,"raw_usage":{"total_tokens":1477,"prompt_tokens":775,"completion_tokens":702,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":391,"completion_tokens_details":{"reasoning_tokens":626}},"tokens_in":391,"tokens_out":702,"duration_ms":7355,"temperature":1.0,"reasoning_tokens":626,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:28:59.137138+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Connect the AR headset to a real robot and have a user lasso two objects, drag their twins to a shelf, and raise a bottle twin's water level to a chosen mark; then run the generated instructions and measure the final object poses and fill level. If the robot's placement error exceeds the tolerance the user can see in the AR view, or if the deformation parameter does not reach the robot, then the paper's claim that direct manipulation precisely specifies physical goals fails at execution.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"contrast baseline: a language-based agent with high-level semantic grounding, which the paper argues carries ambiguity and discoverability problems that direct manipulation avoids."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"example of AR robot instructions that still use language and non-interactive visualizations; GhostObjects adds interactivity."}],"review_version":2}