{"id":"a35059e5-e49f-4979-a17b-3f2f20f84d7a","arxiv_id":"2508.08774","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A position paper proposing a four-module memory-augmented AR agent framework that uses stored scene graphs of past user experiences to personalize task guidance.","lead":"This paper proposes a conceptual framework for augmented reality agents that store a user's past experiences as scene graphs and use them to give personalized step-by-step guidance. It describes four modules, an implementation roadmap, and a planned user study, but no working system, code, or experimental results.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The alignment between stored and live scene graphs is asserted, not demonstrated; without it, the framework's value collapses.","rationale":"The reader's weakest assumption—faithful scene-graph construction and alignment—is indeed the most load-bearing. I partially agree: my stress-test centers more specifically on the absence of an alignment algorithm and the citation gap for MLLM-based scene-graph generation, both of which are concrete and testable. The paper honestly frames itself as a conceptual framework with a future evaluation plan, so this is not a case of overclaiming an implemented system. However, Section 7 states as fact that the system 'supports adaptive guidance,' which is not established by any experiment. A proof-of-concept on public egocentric data would settle whether the central alignment step is feasible. This does not change the reader's CONDITIONAL verdict; it sharpens the condition under which the claim would be accepted.","tokens_in":10412,"tokens_out":4640,"duration_ms":49618,"concrete_test":"Build a minimal offline prototype on a public egocentric video dataset (e.g., Ego4D or EgoProceL): (1) construct scene-graph sequences for one recorded episode (memory) and a separate re-enactment of the same procedure (current); (2) implement a graph-sequence alignment between the two (e.g., temporal graph matching with edit distance); (3) report per-step progress accuracy and top-1 next-step accuracy. If alignment cannot reach, say, >=80% per-step accuracy on clean re-enactments, or degrades sharply when sensor noise or off-task actions are injected, the framework's core assumption fails. If it succeeds, the concern is resolved in the paper's favor.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—that the proposed four-module framework 'supports adaptive guidance grounded in user-specific workflows' (Sec. 7)—rests on the Spatiotemporal Reasoning Module's ability to align a noisy, streamed current scene graph with a stored scene-graph sequence and to infer intent, track progress, and select next steps (Sec. 4.5). This is the load-bearing step: even if perception and memory construction were perfect, the framework's benefit would collapse if this alignment is unreliable. Yet the paper specifies no alignment algorithm, no graph-similarity metric, no handling of skipped/reordered/repeated steps, and no error recovery for off-task behavior beyond the assertion in Sec. 4.5.2 that such behavior is 'distinguished' by interpreting short-term patterns. The works cited in Sec. 2.2 and Sec. 4.3 establish scene-graph generation, not temporal alignment or personalized procedural matching. Moreover, Sec. 4.3 attributes direct MLLM-based scene-graph generation to [21,29,45], none of which are MLLM-based; this weakens the perception pipeline's stated support. Since no experiments are reported, the 'supports adaptive guidance' conclusion is an existence conjecture, not an established result.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a conceptual framework for memory-augmented AR agents that provide personalized task assistance by recalling and reasoning over a user's long-term interaction history. It identifies a limitation of current AR agents (lack of persistent, spatiotemporal user memory), and proposes four interconnected modules: Perception, Memory, Spatiotemporal Reasoning, and Actuator. Scene graphs are chosen as the unified representation for both stored episodic memories and live egocentric observations. The paper also provides a two-phase recording/recall scenario, an implementation roadmap using Unity and GPT-4o-realtime, a planned user-study evaluation, and a target cooking-recall application with additional use cases. No implementation, prototype, or empirical evaluation is reported; Section 5 is explicitly a future plan.","tokens_in":10698,"tokens_out":4240,"duration_ms":49182,"significance":"If the proposed framework were realized, it would address a genuine gap: current AR assistants generally support immediate tasks but do not leverage user-specific long-term workflows. The modular decomposition and the use of scene graphs as a shared representation are sensible and provide a concrete starting point for building such systems. The roadmap and target application give the proposal actionable specificity. However, the paper's central claim that the framework 'supports adaptive guidance grounded in user-specific workflows' (Section 7) is not established by any evidence; it remains an existence conjecture. The main strength is the conceptual synthesis, not a validated system. The paper is clearly written and the use cases are well chosen, but the load-bearing reasoning and alignment steps need significantly more specification or empirical grounding before the central claim can be accepted.","major_comments":[{"comment":"The paper's title, abstract, and conclusion assert that the framework 'can provide personalized task assistance' and 'supports adaptive guidance grounded in user-specific workflows.' No implementation or user study is presented; Section 5 is only a plan. This is a causal/existence claim that is not supported by the manuscript's evidence. The authors should either provide at least a proof-of-concept implementation/feasibility study, or explicitly reframe the contribution as a proposed framework and hypothesis with the conclusion appropriately qualified.","section":"§1, §7"},{"comment":"The load-bearing step of the framework is the alignment between a streamed, noisy current scene graph and a stored scene-graph sequence. This component is asserted in §4.5.2 ('interprets short-term behavioral patterns... and aligns them with expected procedural steps') and in §5.1 ('aligns the current scene graph with recalled memory entities'), but no algorithm, graph-similarity metric, or handling of skipped/reordered/repeated steps is specified. Similarly, the claimed distinction of off-task behavior is not operationalized. Without this specification, the framework's core mechanism is untestable and the conclusion in §7 is not justified. Please provide a concrete alignment/robustness design or evidence of feasibility.","section":"§4.5.2, §5.1"},{"comment":"The Perception Module cites [21,29,45] as recent advances in MLLMs that 'generate scene graph structures directly from complex, multimodal inputs.' These papers (Action Genome, video relation detection with spatio-temporal global context, and target-adaptive context aggregation) are not MLLM-based; they are supervised scene-graph-generation methods using convolutional/transformer architectures. This misattribution weakens the stated support for the perception pipeline. Please correct the citations and accurately describe the state of the art in MLLM-based scene graph generation, or qualify the feasibility claim accordingly.","section":"§4.3"},{"comment":"The proposed evaluation plan measures task completion rate, completion time, NASA-TLX, and user satisfaction, but it does not measure the framework's internal validity: memory construction fidelity, scene-graph alignment accuracy, step-detection precision/recall, or robustness to off-task actions. Without such metrics, even a successful between-group user study cannot attribute benefits to the memory-augmented architecture rather than to the novelty of the system or the content of instructions. Please add component-level evaluation metrics (e.g., graph edit distance, alignment error, misstep detection rate) to the plan.","section":"§5.2"},{"comment":"The memory construction phase assumes that a user's past procedure can be faithfully captured as a sequence of scene graphs via offline processing of egocentric video and sensor data. This assumption is load-bearing because the entire Recall phase depends on the constructed memory. The paper offers no evidence that current scene-graph-generation methods achieve the required fidelity on noisy, long-form egocentric recordings, nor does it discuss failure modes (e.g., missing objects, misdetected relations, temporal segmentation errors). Please either provide a feasibility analysis or explicitly state this as an open research challenge that the framework does not solve.","section":"§3, §4.4"}],"minor_comments":[{"comment":"The notation for the dynamic scene graph appears garbled (e.g., 'Gt �Vt�Et�'). Please ensure proper typesetting and define the node/edge notation more clearly.","section":"§4.2.2"},{"comment":"The phrase 'we set the base simulation engine with Unity' is awkward; consider 'we use Unity as the base simulation engine.'","section":"§5.1"},{"comment":"The text under the example recipe title appears as corrupted/non-rendered characters ('����� ������� ����'). Please verify the figure's font embedding and character encoding.","section":"Figure 2"},{"comment":"The author affiliations and names contain spacing artifacts (e.g., 'Y onsei University', 'T aeyoon Kwon'). Please correct these typographical issues.","section":"Author list"}],"recommendation":"major_revision","confidential_remarks":"This is a position/vision paper rather than a systems or empirical paper. If the journal's scope includes such contributions, the main issue is the gap between the strong claims in the abstract/conclusion and the absence of feasibility evidence. The self-citations [26,53] are used for two key design choices (personalization value and LLM reasoning over scene graphs); they are relevant but should be independently supported. No concerns about academic integrity beyond the citation misattribution noted in the major comments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a position paper, not an implemented system. What is actually new is the specific combination: a four-module AR agent architecture with scene-graph episodic memory and a two-phase recording/recall workflow for personalized task assistance. That combination is not present in the cited literature, and the paper presents it clearly. It also does a good job motivating the problem: current AR assistants handle immediate tasks but not long-term personalized routines, and the cooking/organizing/rehab/lab use cases make the intended setting concrete. The modular decomposition is sensible, and the paper is honest that the evaluation strategy in Section 5 is a plan, not results. If the goal is to frame a research agenda, this works.\n\nThe soft spots are concentrated in the Spatiotemporal Reasoning Module. The whole value of the framework depends on aligning a noisy streamed scene graph with a stored scene-graph sequence well enough to infer intent, track progress, and suggest the next step. Section 4.5.2 asserts this can be done—including handling off-task behavior—but gives no algorithm, no similarity metric, no treatment of skipped/reordered/repeated steps, and no error recovery. The cited works in Section 2.2 and 4.3 establish scene graph generation, not temporal alignment. There is also a real citation slip: Section 4.3 attributes MLLM-based scene graph generation to [21,29,45], but those are pre-LLM vision papers. Two of the load-bearing supporting citations, [53] and [26], are the authors' own prior work; that is not disqualifying, but the feasibility case rests on a narrow base. Since there are no experiments, the strong claims in Sections 1 and 7—that current systems fail and this framework supports adaptive personalized guidance—are existence conjectures.\n\nThese are not fatal flaws for a vision paper, but they matter if the paper is positioned as a research contribution. As written, it is a well-scoped plan with a reasonable architecture and an honest roadmap. It deserves a serious referee, not a desk rejection, but a referee should push the authors on the alignment assumption and ask either for a proof-of-concept or for language that clearly separates hypothesis from result. I would not cite it as evidence in my own work, but I would bring it to a reading group focused on AR agents or memory-augmented systems for discussion.","headline":"A clear, honest position paper for memory-augmented AR agents; the four-module architecture is new as a combination, but the hypothesis is not yet backed by evidence and the load-bearing scene-graph alignment step is asserted rather than demonstrated.","tokens_in":11152,"tokens_out":3239,"would_cite":false,"duration_ms":31610,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that AR assistants need a persistent memory of the user's own experiences, and proposes a four-module framework that stores past procedures as scene graph sequences and aligns live context against them to give personalized","keywords":["augmented reality","memory-augmented agents","scene graphs","spatiotemporal reasoning","personalized task assistance","multimodal large language models","egocentric video","procedural memory"],"falsifier":"Run a controlled repeat-task study: users perform a personalized multi-step task twice (e.g., cooking their own recipe) with AR glasses; build scene-graph memories from the first pass, then have the system predict each next step during the second pass. If predicted next steps do not beat a fixed text-instruction baseline, or if accuracy collapses when the user is interrupted or the camera moves, the central claim is contradicted.","tokens_in":10346,"feed_emoji":"🥽","tokens_out":7792,"duration_ms":74672,"temperature":0.7,"pith_summary":"The paper argues that augmented-reality assistants fail at complex, personalized multi-step tasks because they cannot retain and reuse a user's own past experiences. It proposes a four-module architecture—perception, memory, spatiotemporal reasoning, and actuation—built around storing recorded egocentric activities as sequences of scene graphs. In a later recall phase, the agent aligns a live scene graph against the stored sequence to infer the user's task, track their progress, and suggest the next step based on how the user previously did the task. If the architecture works as claimed, AR assistance could shift from generic instructions to guidance that reproduces a user's personal cooking, organizing, health, or laboratory workflows. The paper is a design proposal: it supplies the framework, an implementation roadmap, and an evaluation plan rather than a tested system.","feed_headline":"A four-module design gives AR assistants memory of past tasks","feed_subtitle":"The framework stores your recorded workflows as scene graphs and aligns live views to them, so guidance matches how you actually work.","key_machinery":"The dynamic scene graph $G_t = (\\mathcal{V}_t, \\mathcal{E}_t)$, a structured snapshot in which nodes denote objects, user hands, actions, and UI elements with multimodal features, and directed edges denote observed physical interactions, attentional cues, or planned guidance relations. A temporal sequence of such graphs forms an episode memory, and the same graph format is used for both stored memories and live perception, making it the shared substrate that lets the Spatiotemporal Reasoning Module align past and present context and plan guidance.","core_discovery":"On its own terms, the paper's central claim is that the reason current AR agents fall short on personalized tasks is not perception or language grounding but the absence of persistent, spatiotemporal memory, and that this memory can be supplied by converting recorded egocentric experiences into sequences of scene graphs. The framework's four modules—Perception, Memory, Spatiotemporal Reasoning, and Actuator—are organized around that claim. The load-bearing operation happens in the Spatiotemporal Reasoning Module: it aligns the incoming scene graph with the stored episode to infer task intent, identify the current stage, and generate an action plan, which the Actuator renders as AR overlays o","pith_inferences":["Beyond the paper: the same recording-then-alignment pattern could be applied to egocentric assembly or repair tasks, and a proof-of-concept on existing long-form egocentric video would test the core assumption before any head-mounted display deployment.","The authors leave implicit that if scene graphs are a workable memory substrate, the stored procedure could be distilled into a natural-language workflow; retrieval and alignment might then be done by text reasoning, which is easier to evaluate and debug.","A longer-horizon consequence the paper does not develop: memory value compounds with use, so the same architecture becomes more personalized over time, making memory capacity, forgetting, and user privacy first-order design questions rather than add-ons.","A stricter evaluation than the proposed text-only baseline would be to compare the full memory agent against the same agent with memory access removed, isolating whether the memory module itself drives any gains apart from the AR visualization or language model."],"forward_implications":["If the framework is sound, an AR agent can reproduce a user's own prior procedure—such as a personal recipe—rather than giving generic instructions.","The two-phase recording and recall design turns ordinary egocentric recordings into reusable procedural memory, so personalized assistance does not require the user to describe their workflow in words.","Scene graph memory makes retrieval spatial and procedural: the system can match by object layout and action sequence, not only by verbal query.","The planned user study comparing memory-augmented guidance with fixed on-screen text would give a direct test of whether personalization improves task completion rate, completion time, workload, and satisfaction.","The same framework generalizes beyond cooking to household organizing, personalized health routines, and repeatable laboratory experiments."],"supporting_citations":[{"why":"Establishes the idea of representing actions as compositions of spatiotemporal scene graphs, the representational basis for episode memory.","marker":"[21]"},{"why":"Shows how video can be parsed into spatiotemporal relational structures, grounding the offline scene graph construction from egocentric recordings.","marker":"[29]"},{"why":"Supplies a method for aggregating temporal context in video scene graph generation, used for converting recorded experiences into memory.","marker":"[45]"},{"why":"Demonstrates that large language models can understand and reason over scene graphs, supporting the design choice of scene graphs as the reasoning interface.","marker":"[53]"},{"why":"Provides an egocentric action-scene-graph dataset and pipeline for long-form video, used in the perception module's training alternative.","marker":"[39]"},{"why":"Motivates storing user-specific semantic knowledge and routine patterns for personalized assistance, the paper's central goal.","marker":"[26]"},{"why":"Represents a current AR memory-augmented assistant limited to short-term interactions, the limitation the framework is designed to overcome.","marker":"[14]"},{"why":"Shows a proactive AR assistant without long-term memory, serving as the comparison point for the proposed memory-based personalization.","marker":"[27]"}],"fun_headline_variants":["AR agents with scene-graph memory personalize help","Align live views to stored scene graphs for personal AR","Memory module enables spatiotemporal reasoning in AR","Scene graphs give AR assistants memory of past tasks","Design: AR agents recall workflows via scene graphs"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The whole proposal rests on whether a user's past actions and surroundings can be captured, from egocentric video, as a faithful sequence of structured scene snapshots, and whether the live view can be matched against that stored sequence accurately enough to infer the task, the current step, and the right next step.","fun_headline_variants_meta":{"raw":{"variants":["AR agents with scene-graph memory personalize help","Align live views to stored scene graphs for personal AR","Memory module enables spatiotemporal reasoning in AR","Scene graphs give AR assistants memory of past tasks","Design: AR agents recall workflows via scene graphs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000681,"raw_usage":{"total_tokens":2938,"prompt_tokens":758,"completion_tokens":2180,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":502,"completion_tokens_details":{"reasoning_tokens":2109}},"tokens_in":502,"tokens_out":2180,"duration_ms":20930,"temperature":1.0,"reasoning_tokens":2109,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T21:19:58.402458+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a controlled repeat-task study: users perform a personalized multi-step task twice (e.g., cooking their own recipe) with AR glasses; build scene-graph memories from the first pass, then have the system predict each next step during the second pass. If predicted next steps do not beat a fixed text-instruction baseline, or if accuracy collapses when the user is interrupted or the camera moves, the central claim is contradicted.","supporting_citations":[{"cited_title":"Action genome: Actions as compositions of spatio- temporal scene graphs","cited_arxiv_id":null,"evidence_quote":"Establishes the idea of representing actions as compositions of spatiotemporal scene graphs, the representational basis for episode memory."},{"cited_title":"Beyond short-term snippet: Video relation de- tection with spatio-temporal global context","cited_arxiv_id":null,"evidence_quote":"Shows how video can be parsed into spatiotemporal relational structures, grounding the offline scene graph construction from egocentric recordings."},{"cited_title":"Tar- get adaptive context aggregation for video scene graph gen- eration","cited_arxiv_id":null,"evidence_quote":"Supplies a method for aggregating temporal context in video scene graph generation, used for converting recorded experiences into memory."},{"cited_title":"LLM Meets Scene Graph: Can Large Language Models Understand and Generate Scene Graphs? A Benchmark and Empirical Study","cited_arxiv_id":"2505.19510","evidence_quote":"Demonstrates that large language models can understand and reason over scene graphs, supporting the design choice of scene graphs as the reasoning interface."},{"cited_title":"Action scene graphs for long- form understanding of egocentric videos","cited_arxiv_id":null,"evidence_quote":"Provides an egocentric action-scene-graph dataset and pipeline for long-form video, used in the perception module's training alternative."},{"cited_title":"Satori: Towards proactive ar assistant with belief-desire-intention user modeling","cited_arxiv_id":null,"evidence_quote":"Shows a proactive AR assistant without long-term memory, serving as the comparison point for the proposed memory-based personalization."}],"review_version":1}