{"id":"e11d6457-30b1-4793-8b8e-691d9618ceee","arxiv_id":"2509.06269","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"REMI combines a personal causal graph, counterfactual traversal, and schema-based planning with an LLM to generate explainable, personalized lifestyle recommendations, claiming higher personalization and causal accuracy than baselines.","lead":"A proposed architecture adds a personal causal knowledge graph, causal reasoning, and schema-based planning to LLM-powered lifestyle agents, aiming to make advice both personalized and explainable. The paper reports that this design outperforms memory-only and no-planner baseline agents on two new similarity-based metrics, but the evaluation is thin and the metrics reward the very content the system is prompted to echo back.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Empirical claim is undermined by self-referential metrics: CRA/PSS score whether the LLM echoes the causal factors and memories that REMI itself injects into the prompt, so the reported baseline superiority does not establish value of the causal schema.","rationale":"The paper is a system-design proposal, and the architecture is internally coherent. The reader's rejection is based primarily on unverified graph reliability and lack of artifacts. I agree with the rejection, but I locate the most load-bearing weakness one step earlier. The empirical claims in the abstract and §6.3 are operationalized by PSS and CRA, which are computed against content the system itself inserts into the LLM prompt. This makes the metrics self-scoring rather than measurements of personalization or causal correctness. Section 4.4.1 shows the prompt contains the retrieved memories and causal factors; §4.4.2 explicitly instructs the LLM to explain steps with reference to those injected factors. Under these conditions, high CRA is expected for any instruction-following model; the memory-only baseline's 0.0 CRA is a definitional consequence of not receiving a causal-factor block, not a measured absence of causal accuracy. This is an internal validity problem: even granting the graph, the reasoner, and the planner, the reported margins do not show that these components add value. It also explains the absence of per-scenario tables, confidence intervals, human evaluation, and artifacts: without them the reader cannot tell whether the claims are real or prompt artifacts. The graph-construction assumption (§4.1) is indeed unvalidated and load-bearing for real-world deployment, but the metric circularity affects the only evidence offered for the strongest claim, so it is the more urgent concern. A single control experiment—a text-oracle baseline that receives the same context and causal-factor text without the machinery—would resolve it. If the control matches REMI, the empirical case collapses to a prompt-engineering demonstration; if it does not, the architecture has at least passed a meaningful falsification test. Because the evidence as published does not support the abstract's comparative claim, I concur with REJECT and see no change to the reader's verdict.","tokens_in":11161,"tokens_out":4463,"duration_ms":50383,"concrete_test":"Run a control condition on the same 28 scenarios: feed the exact retrieved-memory, causal-factor, and plan text that REMI emits to a plain LLM (no graph traversal, no schema planner), asking it to answer the user query. Compute PSS/CRA on those outputs. If the text-oracle matches REMI's 0.85–0.92 PSS and 0.4–0.8 CRA, the metrics are measuring prompt-faithfulness only, and the reported superiority is an artifact of prompt construction.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that CSM agents give more context-aware, user-aligned recommendations than baselines—is supported only by PSS and CRA, and both metrics are close to tautological given the system design. In §4.4.1, the orchestrator explicitly injects '[Retrieved memory]' blocks, a '[Causal factors]' list, and a '[Plan]' into the LLM prompt. PSS (Eq. 1) then measures cosine similarity between those very context items and response chunks; CRA (Eq. 2) measures similarity between the injected causal factors and the response. A model that simply paraphrases its prompt will score at or near the threshold, and this is exactly the behavior the authors instruct: §4.4.2 says the LLM 'effectively justifies its advice by drawing on that injected trace.' The 0.0 CRA of the memory-only baseline is therefore by construction—it receives no causal-factor block to echo—not evidence that it lacks causal reasoning. The reported REMI-vs-baseline margins (0.4–0.8 CRA) reflect differences in prompt composition, output length, or instruction-following, not in the graph, traversal, or schema. The 28-scenario evaluation is undocumented: no per-prompt outputs, no confidence intervals, no human validation, no code/data. Even if the personal causal graph in §4.1 were perfectly accurate, this experiment would not demonstrate the architecture's contribution. The graph-reliability concern identified by the reader is real, but it is secondary: the published evidence cannot distinguish a working causal reasoner from a prompt template.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents REMI, a Causal Schema Memory (CSM) architecture for personalized lifestyle recommendation agents. The system combines a personal causal knowledge graph, a causal reasoner performing goal-directed traversal and counterfactual checks, a schema-based planner, and an LLM orchestrator. To evaluate the architecture, the authors introduce two metrics, Personalization Salience Score (PSS) and Causal Reasoning Accuracy (CRA), and report results on 28 scenarios comparing REMI with a memory-only RAG baseline and an ablated CSM without the schema planner. The central claim is that CSM-based agents provide more context-aware, user-aligned recommendations than baseline LLM agents.","tokens_in":11559,"tokens_out":4173,"duration_ms":44262,"significance":"If the empirical claims were valid, the architecture would be a useful contribution to hybrid memory-augmented LLM agents, showing how a structured personal causal graph and schema-based planning can be combined with LLM generation while preserving an explicit explanation trace. The modular decomposition is clear, and the idea of using an LLM to score causal paths while keeping generation grounded in an external graph is potentially valuable. However, the paper supplies no code, data, prompts, or full result tables, and the two proposed metrics are constructed so that the injected context is exactly what is measured, making the headline comparison close to tautological. The work is therefore better positioned as a system description or position paper than as a validated empirical contribution.","major_comments":[{"comment":"The evaluation metrics are circular by construction. In §4.4.1 the prompt context explicitly includes '[Retrieved memory]', '[Causal factors]', and '[Plan]' blocks. Equation (1) then measures cosine similarity between response chunks and those same context items, and Eq. (2) measures similarity between the injected causal-factor list and the response embedding. Because the LLM is instructed to draw on that injected trace (§4.4.2), a system that merely paraphrases its prompt will trivially attain high PSS and CRA. The memory-only baseline receives no causal-factor block, so its CRA = 0.0 is by construction, not evidence of absent causal reasoning. The reported 0.4–0.8 CRA advantage therefore does not establish that the graph, traversal, or schema contribute anything beyond prompt composition. To support the central claim, the metrics need independent validation (e.g., human judgments of w","section":"§4.4.1, §5.1, §5.2"},{"comment":"The quantitative evidence is not documented. The paper reports only aggregate ranges ('PSS 0.85–0.92', 'CRA 0.4–0.8', baselines '0.68–0.82' and '0.0') over 28 scenarios, with no per-scenario scores, no confidence intervals, no significance tests, and no data/prompt/code release. Section 6.1 gives one illustrative scenario (PSS 0.92, CRA 0.60), while the dog-naming scenario in §6.2 reports no scores. The claim that REMI 'outperformed the baselines in every case' is not verifiable. Additionally, the arbitrary threshold τ = 0.7 in Eqs. (1)–(2) is not calibrated against human judgment or any ground-truth reference, and no sensitivity analysis is provided. The experiment is therefore not reproducible from the manuscript alone.","section":"§6.3"},{"comment":"The correctness of the system rests on the personal causal graph, but graph construction is underspecified. Edges are described as 'user-input or learned' (§4.1), yet no learning procedure, validation, or error analysis is given. The reasoner also permits the LLM to 'insert a hypothesized link' (§4.2.2) and uses LLM-based path scoring and self-reflection (§4.2.3, §4.2.5), with no evidence that these LLM judgments are reliable. Every downstream traversal, counterfactual check, schema instantiation, and explanation inherits the graph's mistakes, so the missing graph-reliability evidence is load-bearing. The authors should report graph size, how edges and confidence weights were obtained in the 28 scenarios, and a sensitivity or robustness analysis to graph noise.","section":"§4.1, §4.2.2"}],"minor_comments":[{"comment":"The indicator notation in Eq. (1) is nonstandard and should be defined in prose. Also, CRA uses the overall response embedding while PSS uses a per-chunk max, so the aggregation procedures are inconsistent; specify whether sim(f,R) is a max over chunks or a mean.","section":"Eqs. (1)–(2)"},{"comment":"The 'What Should I Name My Dog' scenario is presented as part of the evaluation but reports no quantitative PSS/CRA scores. Either provide the scores or label it explicitly as a qualitative example.","section":"§6.2"},{"comment":"Figure 1 is referenced in the architecture description but is not visible in the manuscript text. The architecture diagram should be included or the components described in enough detail to stand alone.","section":"§4"},{"comment":"The fine-tuned dual-encoder model is not described (training data, loss function, evaluation). Since this model controls node mapping and schema retrieval, provide these details or a reference to the training procedure.","section":"§4.2.1"},{"comment":"Minor typos and formatting issues: 'upto' appears instead of 'up to', 'networkx' should be 'NetworkX', and the '⊮' symbol in Eq. (1) should be a standard indicator function. The threshold τ is introduced without justification.","section":"Throughout"},{"comment":"The paper acknowledges that the authors 'rely on prompt quality' and that hallucination avoidance is an ongoing focus. A small safety/alignment evaluation, even qualitative, would strengthen the claims about trustworthiness and explainability.","section":"§7.4.2"}],"recommendation":"reject","confidential_remarks":"The stress-test concern is accurate: the PSS and CRA metrics are essentially tautological given the prompt design, so the empirical section does not establish the central claim. The manuscript reads more like a workshop system description than a journal article. I would be open to a revised submission if the authors added a non-circular evaluation (e.g., human judgments, held-out causal graph accuracy) and released the data, prompts, and code. There is no concern about novelty disclosure or citation behavior."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The REMI paper is a system design proposal worth reading for its clean assembly of known ideas: a personal causal knowledge graph, graph traversal with counterfactual checks, schema-based planning, and LLM orchestration. The authors are honest about limitations, and the modular architecture is clearly thought through. That part is fine.\n\nThe problem is the evaluation. The two new metrics, PSS and CRA, are circular by construction. In §4.4.1 the orchestrator explicitly injects retrieved memories, extracted causal factors, and the draft plan into the LLM prompt. PSS then measures cosine similarity between those exact context items and the response chunks; CRA does the same for the causal factor list. So REMI is scored on how faithfully the LLM echoes its own prompt. The memory-only baseline has 0.0 CRA simply because no causal block is in its prompt—not because it lacks causal reasoning. The reported margins are artifacts of prompt composition, not evidence about the causal graph or schema.\n\nThe evaluation is also thin: 28 undocumented scenarios, no code, data, prompts, or graph artifacts, no error bars or statistical tests. The graph-reliability concern is real but secondary. Even a perfect graph would not rescue the experiment as it stands.\n\nWhat is genuinely new: the specific combination and the attempt to define metrics for personalization and explainability. Those metrics could be salvaged if redesigned to compare against human judgment or real outcomes, and if the authors separate what is in the prompt from what is measured.\n\nSo my verdict: plausible architecture, but the central empirical claim is unsupported. I would not cite this as evidence for REMI's superiority. For peer review, I see it as a borderline desk-reject—the value is in the blueprint, not the results. If the authors release artifacts and revalidate the metrics, I'd reconsider.","headline":"Coherent modular architecture for causal schema memory, but the empirical claims rest on metrics that measure whether the LLM paraphrases its own prompt; without released artifacts or non-circular validation, the central comparison is not established.","tokens_in":12040,"tokens_out":2590,"would_cite":false,"duration_ms":26461,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a personal causal knowledge graph, paired with schema-based planning, lets a lifestyle agent give advice that is both deeply personalized and causally explainable.","keywords":["causal reasoning","personalization","knowledge graph","schema-based planning","explainable AI","large language models","lifestyle recommendation","memory architecture"],"falsifier":"Have domain experts label the true causal factors for a set of user scenarios, then compute Causal Reasoning Accuracy for REMI and a retrieval-only agent against those external labels. The central claim stands only if REMI beats the retrieval-only baseline on that ground truth rather than on semantic similarity to the causal factors the pipeline itself generated.","tokens_in":11036,"feed_emoji":"🧠","tokens_out":7438,"duration_ms":72970,"temperature":0.7,"pith_summary":"REMI is an architecture for lifestyle recommendation agents that stores a user's life events and habits as a personal causal graph, then walks that graph to find why a problem occurs before assembling a plan. The paper's claim is that causal-schema memory beats ordinary retrieval-augmented LLM agents on both personalization and explainability: recommendations reflect the user's own context, and each suggested step can be traced back to a cause in the graph. On 28 scenarios the authors report high Personalization Salience Scores and Causal Reasoning Accuracy in the 0.4–0.8 range, while a memory-only baseline scores zero on causal accuracy, which they read as evidence that retrieval alone cannot ground causal explanations. The contribution is the combination: causal graph memory, graph-of-thought traversal with counterfactual checks, schema instantiation, and an LLM restricted to phrasing the final answer.","feed_headline":"Causal memory graphs make AI lifestyle advice fit the user","feed_subtitle":"New architecture traces each recommendation to your sleep, stress, and caffeine—and beats retrieval-only agents on causal accuracy.","key_machinery":"The load-bearing object is the Causal Schema Memory: a user-specific directed graph whose nodes are events or states and whose edges carry cause-effect labels with confidence weights, paired with a library of abstract plan schemas. The graph supplies the 'why' — goal mapping, causal traversal, counterfactual checks, and explanation tracing all read from it — while the schema library supplies the 'what to do', giving each recommendation a fixed structure that is then filled in with the user's personal causal factors.","core_discovery":"The central claim is that personalization and explainability improve when the agent's long-term memory is a personal causal knowledge graph rather than raw text or vectors. Given a query, the reasoner maps it onto graph nodes, enumerates causal chains up to three hops, inserts hypothesized links when personal data is sparse, and scores paths by LLM plausibility. The schema planner then retrieves a plan template and instantiates it with the identified causes, verifying the plan with a simulated intervention on the graph: if removing or mitigating a cause node would break the link to the problem node, the plan is judged sound. The LLM orchestrator only turns the assembled context into fluent t","pith_inferences":["Beyond the paper: the CRA metric measures semantic overlap between the response and the causal factors the pipeline itself produced, so a stronger test would score both REMI and baselines against externally verified causes for the same user logs.","Beyond the paper: if the personal causal graph is noisy or incomplete, every downstream benefit degrades, and cold-start users with sparse graphs may receive mostly commonsense hypotheses; the paper acknowledges this but does not quantify it.","Beyond the paper: the same graph-plus-schema design should transfer to other personal domains with structured data, such as finance or education, as the paper hints.","Beyond the paper: the counterfactual verification step could be turned into a training signal, rewarding plans whose simulated graph intervention removes the target effect, converting the LLM heuristic judge into a measurable objective."],"forward_implications":["If REMI works as reported, a lifestyle agent can answer 'why' with a traceable causal chain instead of a generic tip list, with each plan step mapped to a graph edge.","Retrieval-based memory can achieve surface-level personalization, but the paper's zero causal-accuracy result for the memory-only agent implies that explanation quality requires an explicit causal representation.","Schema-based planning is what makes causal accuracy consistent; the ablated version without the schema planner scores moderately and inconsistently, suggesting templates impose useful structure.","Because the components are modular, better causal discovery methods or richer plan libraries can be dropped in without redesigning the agent.","The proposed PSS and CRA metrics give a repeatable, quantitative way to compare personalization and explainability across recommendation agents."],"supporting_citations":[{"why":"Supplies the agent pattern of interleaving an LLM's reasoning with tool actions that REMI builds on.","marker":"[4]"},{"why":"Defines retrieval-augmented generation, which the memory-only baseline instantiates and which REMI augments with causal reasoning.","marker":"[6]"},{"why":"Shows long-range factual recall from token-indexed chunks, motivating the separation of parametric knowledge from external memory.","marker":"[7]"},{"why":"Provides the nearest-neighbor language model memory precedent that REMI contrasts with graph-based causal memory.","marker":"[8]"},{"why":"Supplies the pre-trained embedding model that is fine-tuned and used for query-to-node goal mapping and schema retrieval.","marker":"[9]"},{"why":"Implements the vector search used to pull semantically similar user memories into the orchestration context.","marker":"[10]"},{"why":"Motivates the personalization target by showing that generic LLM advice overlooks person-specific factors in health domains.","marker":"[2]"},{"why":"Motivates the explainability target by quantifying uncertainty in natural language explanations of large language models.","marker":"[1]"}],"fun_headline_variants":["Causal graphs in memory make AI lifestyle tips more personal","Why causal memory beats raw text for AI lifestyle advice","AI lifestyle coach uses your causal graph for tailored plans","REMI: causal schema memory boosts AI advice personalization","Personal causal graphs improve AI's lifestyle recommendations"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The whole pipeline assumes the user's causal map — which events lead to which outcomes, and how strongly — is accurate, but the paper does not describe how that map is built or verified beyond saying edges are user-input or learned.","fun_headline_variants_meta":{"raw":{"variants":["Causal graphs in memory make AI lifestyle tips more personal","Why causal memory beats raw text for AI lifestyle advice","AI lifestyle coach uses your causal graph for tailored plans","REMI: causal schema memory boosts AI advice personalization","Personal causal graphs improve AI's lifestyle recommendations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000691,"raw_usage":{"total_tokens":2957,"prompt_tokens":724,"completion_tokens":2233,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":468,"completion_tokens_details":{"reasoning_tokens":2171}},"tokens_in":468,"tokens_out":2233,"duration_ms":17034,"temperature":1.0,"reasoning_tokens":2171,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T23:50:19.473544+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have domain experts label the true causal factors for a set of user scenarios, then compute Causal Reasoning Accuracy for REMI and a retrieval-only agent against those external labels. The central claim stands only if REMI beats the retrieval-only baseline on that ground truth rather than on semantic similarity to the causal factors the pipeline itself generated.","supporting_citations":[{"cited_title":"Retrieval- augmented generation for knowledge-intensive nlp tasks,","cited_arxiv_id":null,"evidence_quote":"Defines retrieval-augmented generation, which the memory-only baseline instantiates and which REMI augments with causal reasoning."},{"cited_title":"Improving language models by retrieving from trillions of tokens,","cited_arxiv_id":null,"evidence_quote":"Shows long-range factual recall from token-indexed chunks, motivating the separation of parametric knowledge from external memory."},{"cited_title":"Generalization through memorization: Nearest neighbor language models,","cited_arxiv_id":null,"evidence_quote":"Provides the nearest-neighbor language model memory precedent that REMI contrasts with graph-based causal memory."},{"cited_title":"Minilm: Deep self- attention distillation for task-agnostic compression of pre-trained transformers,","cited_arxiv_id":null,"evidence_quote":"Supplies the pre-trained embedding model that is fine-tuned and used for query-to-node goal mapping and schema retrieval."},{"cited_title":"Graph-augmented llms for personalized health insights: A case study in sleep analysis,","cited_arxiv_id":null,"evidence_quote":"Motivates the personalization target by showing that generic LLM advice overlooks person-specific factors in health domains."},{"cited_title":"Quantifying uncertainty in natural language explanations of large language models,","cited_arxiv_id":null,"evidence_quote":"Motivates the explainability target by quantifying uncertainty in natural language explanations of large language models."}],"review_version":1}