{"id":"92847483-8242-41d3-bc1a-f20c90480531","arxiv_id":"2608.11354","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A five-stage LLM pipeline infers explainable beliefs and personas from browsing traces, and these inferred profiles match or beat interview-derived profiles on several downstream prediction tasks.","lead":"The paper builds a pipeline that reads a person's browsing actions and produces a written profile of their beliefs and personality, then uses that profile to predict their next moves and shopping attitudes. This matters because personalized systems usually need interviews or surveys to understand a user, and this pipeline attempts to get that understanding from ordinary clicks and searches.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"RQ1's inferred-vs-GT persona comparison is format-confounded; the paper itself admits the +50.6% margin may be prompt-format alignment, leaving the central claim unverified.","rationale":"The paper's central claim that inferred personas match or exceed ground-truth personas rests on RQ1, but the comparison is not format-controlled. The inferred persona is a natural-language profile rich in specific behavioral episodes, while the GT persona is assembled from interviews, demographics, and personality assessments as an abstract trait summary. The paper's own Discussion (Section 4.7) acknowledges that the +50.6% margin may partly reflect prompt-format alignment and leaves the format-controlled comparison to future work. This is a missing control for the headline result, not a minor caveat: if the episodic format is what drives action-prediction gains, the result does not show that inferred personas capture the user better than interview-based personas. The reader's faithfulness concern is real but secondary for the outcome claim, because a persona can be unfaithful in its trait interpretations yet still contain the episodic facts that drive next-action prediction. The format confound directly undermines the stated comparison. I therefore keep the verdict CONDITIONAL: the paper should provide the format-controlled comparison, plus error bars for Table 1, before the abstract's claim is taken at face value. This is consistent with the reader's conditionality but for a more specific, testable reason.","tokens_in":18927,"tokens_out":8173,"duration_ms":68793,"concrete_test":"Run the format-controlled comparison described as future work in Section 4.7: a 2x2 design crossing persona source (inferred vs GT) with representation format (episodic vs trait-summary). Rewrite GT personas into the episode-grounded style used for inferred personas, and convert inferred personas into trait-summary style, then re-run RQ1 on all four conditions. If the inferred source still beats GT after format equalization, the +50.6% survives; if the advantage tracks format rather than source, the central claim must be restated as a format effect. Report paired bootstrap or per-user CIs for the 15-user sample.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing premise of RQ1 is that comparing inferred personas to GT personas isolates persona quality. It does not: inferred personas are episode-grounded natural-language profiles, GT personas are abstract trait summaries (Section 4.7). The paper explicitly concedes that 'the +50.6% margin may partly reflect prompt-format alignment with the action-prediction task' and that a format-controlled comparison is future work. Since RQ1 is the primary evidence for the abstract's 'inferred personas match or exceed ground-truth personas,' the headline claim is currently not established. The advantage could result from conditioning on concrete decision episodes rather than from inferential fidelity of the persona. Additionally, Table 1 reports 15 users / 90 sessions / 985 instances without error bars, so the 25.69 vs 17.06 difference has no stated uncertainty. This is a missing control for the central claim, not a stylistic objection.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes an Inverse Theory of Mind (IToM) pipeline that reconstructs a user's decision context from raw interaction traces, uses an LLM to generate counterfactual belief statements, optimizes and deduplicates these beliefs, and synthesizes them into a structured natural-language persona through multi-hypothesis aggregation. The pipeline is evaluated on the OPeRA dataset across four tasks: next action prediction, held-out shopping category prediction, Big Five personality inference, and shopping attitude survey alignment. The authors report that inferred personas match or exceed ground-truth interview-based personas on next action prediction, that multi-hypothesis reasoning reverses a negative correlation between LLM-based personality estimates and ground truth, and that the resulting persona transfers to a VisionOS spatial banking prototype. The paper is clearly written and honestly acknowledges several limitations, but the central empirical claims rely on small samples and on at least one comparison that the authors themselves concede is format-confounded.","tokens_in":19158,"tokens_out":3482,"duration_ms":32511,"significance":"If the central claims held, the contribution would be significant: a behavioral-trace-to-persona pipeline that is interpretable, modality-agnostic, and reusable across tasks would address a real bottleneck in recommender systems and adaptive interfaces. The use of an external dataset with ground-truth interviews and personality assessments is a genuine strength, as are the direct comparisons against external published Big Five norms and the transparent reporting of LLM call costs. The multi-hypothesis aggregation idea is interesting and, if properly validated, could be a useful corrective to the known tendency of LLMs to produce stereotyped personality judgments. However, the current evidence is not yet at the level of the paper's headline claims: sample sizes are small (N=46, 27, 15), several comparisons lack uncertainty estimates, and the key RQ1 comparison is explicitly admitted to be confounded by prompt format. The paper is therefore best viewed as a promising proposal that first needs a format-controlled validation and more careful statistical reporting.","major_comments":[{"comment":"The paper's headline claim that inferred personas 'match or exceed ground-truth personas' is not established by RQ1. The GT persona is an abstract trait summary, whereas the inferred persona is an episode-grounded natural-language profile; Section 4.7 explicitly concedes that the +50.6% relative gain on Claude Opus 4.6 'may partly reflect prompt-format alignment with the action-prediction task' and that a format-controlled comparison is future work. Because Table 1 is the primary evidence for the abstract's central claim, the comparison needs a control that rewrites GT personas in the inferred-persona format, and an ablation that removes episodic content from inferred personas. The table should also report bootstrap confidence intervals or a significance test over the 15 users / 90 sessions / 985 instances, since the 25.69 vs. 17.06 difference has no stated uncertainty.","section":"§4.3, Table 1; §4.7"},{"comment":"The claimed benefit of multi-hypothesis aggregation for Big Five prediction is the sign reversal of the average Pearson correlation (−0.103 or −0.008 to +0.103). However, the aggregate bootstrap CI is reported as [−0.05, +0.26], which includes zero and negative values, and the only per-trait correlation approaching significance is Extraversion (r=+0.280, p=0.060) without any multiple-comparison correction. With N=46, this does not establish that the sign reversal is a stable population-level effect. I ask for per-trait confidence intervals, a bootstrap distribution of the aggregate correlation, and a statement of how many of the five traits have CIs excluding zero.","section":"§4.5, Table 4 and Figure 1"},{"comment":"The adaptive prior strength α_d and the confidence weights w_h are central to the reported personality and attitude results, but they are underspecified. The text says only that α_d 'scales inversely with mean evidence confidence,' with no formula, no default values, and no description of whether α_d or w_h were calibrated on the 46 evaluation users. If these weights were chosen after inspecting the results, then RQ3 and RQ4 are not held-out evaluations. Please specify the exact computation of α_d and w_h, report their values, and include a sensitivity analysis over a plausible range of α_d; alternatively, derive α_d from a pre-registered rule.","section":"§3.8, Eq. (5)"},{"comment":"There is an internal inconsistency in what 'Multi-hyp persona' means. Section 4.3 defines the condition as the 'best-scored hypothesis from Stage 5,' whereas the pipeline's stated contribution in Section 3.8 is confidence-weighted aggregation across M hypotheses via Eq. (5). The RQ1 results therefore do not evaluate the multi-hypothesis aggregation mechanism at all; they evaluate a single selected hypothesis. The authors should either evaluate the aggregate persona in RQ1 or explicitly restrict the multi-hypothesis claim to RQ3/RQ4.","section":"§4.3 and §3.8"},{"comment":"The load-bearing premise that LLM-generated belief statements faithfully reflect the user's actual decision process is not tested. Section 3.6.3 says hallucination is 'constrained by construction,' but grounding a statement in a reconstructed percept does not guarantee that the statement matches what the user actually believed, and the paper cites known LLM Theory-of-Mind failures in Section 3.1. Without a fidelity check against the OPeRA interview rationales (which the dataset provides), the reconstructed percepts and personas could be coherent stereotypes. A human-annotation study on a sample of belief statements, or a comparison of inferred rationales against the interview rationales, would directly address this concern.","section":"§3.2, §3.6.3"}],"minor_comments":[{"comment":"The implementation details omit the embedding model's pooling and normalization choices for the MMR computation; adding these would improve reproducibility.","section":"§4.2"},{"comment":"The Hit@3=100% result for the multi-hypothesis method on 27 users would benefit from a bootstrap confidence interval; with only 27 users, a single error changes the metric substantially.","section":"§4.4, Table 2"},{"comment":"The text says 'both methods achieve MAE=0.945' after listing Questionnaire and Direct Inference; consider stating the MAE once and explicitly noting that the two baselines coincide, to avoid the appearance of an error.","section":"§4.5, Table 4"},{"comment":"The VisionOS case study is labeled a proof-of-concept, but the caption claims that 'the persona determines' the layout; softening this to 'is used to determine' or reporting a user study would prevent overstatement.","section":"§5"},{"comment":"Several hyperparameter values (λ=0.3, θ_dedup=0.85, K=50, M=7, temperature 0.85) are given without ablations or sensitivity checks; a short paragraph discussing their stability would strengthen the paper.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is honest and the pipeline is interesting, but the central comparative claim against ground-truth personas is explicitly conceded to be format-confounded, and the multi-hypothesis aggregation result is statistically fragile at N=46. I would be willing to review a revised version that adds a format-controlled RQ1 comparison, uncertainty estimates throughout, and full specification of the aggregation weights. The paper also currently undercuts its 'multi-hypothesis' framing by using a single best hypothesis in RQ1."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper to know: an IToM pipeline that turns raw browsing traces into natural-language belief statements and then a structured persona, evaluated on OPeRA. The concept is genuinely interesting and the authors are upfront about its limits, but the headline result—that inferred personas beat interview-based ground-truth personas—is not actually established. The paper itself concedes the +50.6% action-generation margin may come from prompt-format alignment; GT personas are abstract trait summaries, inferred personas are episode-grounded narratives. No format-controlled comparison is run. So the central claim is conditional at best.\n\nWhat is new: the five-stage pipeline (summarize, perceive, infer, optimize, synthesize) is a sensible assembly of existing building blocks, and the multi-hypothesis synthesis with confidence-weighted aggregation toward population priors is a legitimately novel mechanism. The percept reconstruction stage—rebuilding what alternatives the user saw—is a good idea, and the MMR belief selection is reasonable. The authors also do several things right: they use an external dataset with real ground truth, they include sensible baselines in RQ2, they scope RQ3 as a population-level claim rather than per-trait, and the limitation section is unusually candid, including the format-confound admission.\n\nSoft spots, in proportion. The biggest is RQ1: small sample (N=15), no error bars, and the format confound. A missing control, not a stylistic quibble. Second, the aggregation parameter alpha_d in Eq. 5 is described as adaptive but its values are never reported; we don't know if it was tuned on the same data. Third, the whole approach rests on the LLM's counterfactual belief statements being faithful to actual user reasoning, and no faithfulness or hallucination metric is reported—the paper cites LLM ToM failures and then relies on the LLM as an approximate inference engine. Fourth, no code or data artifact is shipped, so the cost numbers ($79/user) and the pipeline itself aren't independently checkable. These are addressable, not fatal.\n\nWho gets value: anyone working on LLM-based user modeling, persona synthesis, or cold-start personalization for novel interfaces. The paper is worth a serious referee: the approach is clearly described, the evaluation is honest, and the limitations are stated rather than hidden. I'd send it to review, but the reviewers should push hard on RQ1's format control and on reporting alpha_d and the belief-faithfulness checks.","headline":"The IToM pipeline is a well-described, honest contribution to LLM-based user modeling, but its headline claim of beating ground-truth personas rests on a format-confounded comparison the authors themselves concede.","tokens_in":19641,"tokens_out":3461,"would_cite":true,"duration_ms":26879,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Behavioral traces alone can yield a persona that matches or beats interview-built personas on next-action prediction.","keywords":["Inverse Theory of Mind","persona synthesis","recommender systems","abductive inference","LLM reasoning","Big Five personality","generative UI","user modeling"],"falsifier":"If a user's retrospective think-aloud rationale for a specific action disagrees with the LLM's inferred belief for that action at chance level, or if feeding the pipeline only the chosen action without the reconstructed alternative set leaves downstream accuracy unchanged, then the inverse-reasoning mechanism is not what carries the result.","tokens_in":18764,"feed_emoji":"🧠","tokens_out":6025,"duration_ms":66320,"temperature":0.7,"pith_summary":"The paper tries to establish that a user's 'why' — the beliefs and decision style behind a click — can be recovered from ordinary interaction traces without interviews or surveys. It proposes an Inverse Theory of Mind pipeline that reconstructs what a user saw and chose, generates natural-language counterfactual belief statements, and composes those beliefs into a portable persona. On a 46-user online shopping dataset with interview and personality ground truth, the inferred personas match or exceed interview-based personas at predicting the next action, improve Big Five prediction over population norms, and align with self-reported shopping attitudes. The payoff for a sympathetic reader is a user model that is interpretable, auditable, and transferable to new interface modalities such as generative layouts or spatial interfaces, where click histories alone are not enough.","feed_headline":"Browsing logs alone can build a persona that beats interviews","feed_subtitle":"Five-stage inverse reasoning turns raw clicks into belief-grounded personas, improving next-action and personality prediction.","key_machinery":"The load-bearing object is the reconstructed percept: the decision context in which an action occurred, rendered as the main choices, sub choices, other available options, and the user's actual choice in natural language. Each inferred belief must cite a chosen option and a rejected alternative from that percept, which grounds the counterfactual reasoning and is intended to dilute hallucination. The pipeline's second load-bearing mechanism is multi-hypothesis persona synthesis: it generates several distinct personality interpretations of the same evidence, scores each by quality and per-dimension evidence confidence, and combines them with an adaptive population prior, so weak evidence regresses toward published norms instead of an overconfident stereotype. Belief optimization with maximal marginal relevance selects a compact, diverse belief set in between.","core_discovery":"The central claim is that persona-level understanding can be inferred, not elicited: a five-stage pipeline (Summarize, Perceive, Infer, Optimize, Synthesize) converts raw action events into a structured persona, and that persona, on one frontier LLM backend, raises exact next-action generation accuracy from 17.06% with the ground-truth interview persona to 25.69% with the inferred persona, a 50.6% relative gain. The same pipeline lowers Big Five mean absolute error to 0.762, below the 0.836 of predicting population norms, and reaches 76.6% normalized accuracy on shopping-attitude items. A further claim is that single-interpretation LLM inference from behavioral personas is systematically biased on this cohort, producing predictions anti-correlated with ground truth, while aggregating several diverse persona hypotheses with confidence weighting reverses the correlation sign. The paper reads the action-prediction advantage as evidence of a representational-format effect rather than proof that inferred personas understand users better than interviews, and it grounds that reading in the episodic, belief-anchored format of the inferred personas.","pith_inferences":["Beyond the paper: if the pipeline's belief statements turn out to be largely stereotype-driven, the reported downstream gains could survive even when the inverse-reasoning story is false; a belief-level faithfulness test against users' own retrospective rationales would separate the two.","Beyond the paper: the action-generation margin may partly be a prompt-format artifact of episodic phrasing, as the paper itself notes; a format-controlled comparison that rewrites ground-truth personas into the same episodic style would isolate the user-modeling contribution.","Beyond the paper: the percept-reconstruction step should generalize to traces without page HTML, such as eye tracking in spatial interfaces, gesture logs, or typed queries, where a test of the same pipeline on a non-shopping domain is the natural next experiment.","Beyond the paper: if persona portability holds, recommender systems could adopt a compute-once, query-anywhere pattern, paying a one-time inference cost per user and then reusing the persona across every adaptive surface, changing the cost model of personalization."],"forward_implications":["Interview- and survey-based persona construction becomes optional for many downstream tasks, since behaviorally inferred personas carry enough signal for next-action, attitude, and category prediction.","A single natural-language persona can be reused across tasks and interface modalities without retraining, so a persona built from screen browsing can steer layout and content in a spatial interface.","Multi-hypothesis aggregation provides a floor against LLM stereotyping: when behavioral evidence is weak, predictions regress to population priors instead of confident anti-correlated guesses.","Episode-grounded, belief-anchored personas are less likely to inject role-play noise into step-level action prediction than abstract trait summaries.","Held-out shopping category prediction reaches Hit@3 of 100% and NDCG@5 of 0.701, suggesting the persona generalizes to sessions it was not built from."],"supporting_citations":[{"why":"Supplies the shopping-traces-plus-interviews dataset and the next-action prediction protocol the evaluation builds on.","marker":"[76]"},{"why":"Action understanding as inverse planning is the conceptual basis for reasoning backward from actions to beliefs.","marker":"[2]"},{"why":"Formalizes Theory of Mind as inverse reinforcement learning, supporting the abductive objective.","marker":"[30]"},{"why":"Defines maximal marginal relevance, the diversity-selection criterion used to compress candidate beliefs.","marker":"[7]"},{"why":"Evidence that LLMs show human-like everyday causal reasoning, motivating their use as approximate inverse reasoners.","marker":"[65]"},{"why":"Documents LLM Theory of Mind failures that the pipeline's grounding and aggregation are designed to mitigate.","marker":"[10]"},{"why":"Shows role-play persona prompts can hurt zero-shot reasoning, supporting the episodic-format design and the reading of the ground-truth comparison.","marker":"[33]"},{"why":"The questionnaire-administration baseline in the persona-bias ablation.","marker":"[15]"},{"why":"The direct-inference baseline in the persona-bias ablation.","marker":"[74]"}],"fun_headline_variants":["Clickstream personas beat interview-based ones in next-action tasks","Inverse Theory of Mind: clicks infer beliefs that outperform surveys","Multiple persona hypotheses reverse bias and beat single-shot inference","Browsing logs alone build personas that outperform interview-based ones","From web actions to user beliefs: a pipeline that outperforms interviews"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the LLM's counterfactual belief statements faithfully reflect the user's actual decision process rather than plausible-sounding stereotypes, and the paper reports no direct test of that faithfulness.","fun_headline_variants_meta":{"raw":{"variants":["Clickstream personas beat interview-based ones in next-action tasks","Inverse Theory of Mind: clicks infer beliefs that outperform surveys","Multiple persona hypotheses reverse bias and beat single-shot inference","Browsing logs alone build personas that outperform interview-based ones","From web actions to user beliefs: a pipeline that outperforms interviews"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000511,"raw_usage":{"total_tokens":2518,"prompt_tokens":1009,"completion_tokens":1509,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":625,"completion_tokens_details":{"reasoning_tokens":1425}},"tokens_in":625,"tokens_out":1509,"duration_ms":21249,"temperature":1.0,"reasoning_tokens":1425,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:12:39.370295+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"If a user's retrospective think-aloud rationale for a specific action disagrees with the LLM's inferred belief for that action at chance level, or if feeding the pipeline only the chosen action without the reconstructed alternative set leaves downstream accuracy unchanged, then the inverse-reasoning mechanism is not what carries the result.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Formalizes Theory of Mind as inverse reinforcement learning, supporting the abductive objective."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Evidence that LLMs show human-like everyday causal reasoning, motivating their use as approximate inverse reasoners."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows role-play persona prompts can hurt zero-shot reasoning, supporting the episodic-format design and the reading of the ground-truth comparison."},{"cited_title":"Ones, Lihong He, and Xiaolin Xu","cited_arxiv_id":null,"evidence_quote":"The direct-inference baseline in the persona-bias ablation."}],"review_version":1}