{"id":"70612edd-a886-4353-b367-38a25aecbe69","arxiv_id":"2608.13454","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A pattern-conditioned LLM method predicts a person's next verbal communicative intent from longitudinal everyday conversations, beating non-personalized baselines by up to 29% in an LLM-judge evaluation.","lead":"Researchers used smartwatch recordings of over 1,000 hours of everyday conversations to build a large language model that anticipates how each of 14 people will respond next in a conversation. The approach could enable proactive AI assistants that nudge users before they repeat unwanted verbal patterns.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim requires strictly out-of-sample pattern mining, but the paper never states a temporal split; C3 may be evaluated on patterns induced from the target episode itself.","rationale":"The reader's weakest assumption already identifies the same load-bearing issue: the evaluation must be out-of-sample, with patterns mined only from conversations before the target turn, and the paper does not explicitly state this split. My reading of Section 4.1 and Appendix B.1.1 finds no mechanism that enforces temporal exclusion for C3, and the cross-participant transfer control does not repair the problem because it only removes the participant's identity, not the target episode from the own-condition pattern store. This is the most direct threat to the central claim that person-specific verbal behavior can be predicted: if the pattern store can contain the answer, the comparison measures retrieval, not prediction. I do not see an internal inconsistency in the method itself, only a missing and essential procedural guarantee. A single reanalysis with a strict chronological split would settle the concern, which is why I would keep the paper in CONDITIONAL rather than moving to accept or reject. A secondary issue, the '1,000+ hours' description, is also worth correcting: 15,066 utterances averaging 49 words imply roughly 700k words, far below what 1,000 hours of naturalistic conversation would contain, but this factual error does not by itself overturn the predictive claim. The LLM-generated ground truth and LLM judge are additional sources of model-mediated evaluation, but they are less decisive than the temporal-leakage question, because the judge is given the actual utterance and the human evaluation partially validates the judge. Overall, the conditional verdict stands, pending the temporal-split check.","tokens_in":14605,"tokens_out":5909,"duration_ms":66893,"concrete_test":"For each participant, order all episodes by timestamp. For each target turn t in episode e, re-mine or update patterns using only episodes that end before e begins, and activate those patterns on the context C_t as in Section 4.1. Recompute the main LLM-judge scores for C3, Zero-Shot, and All-In-Context, and recompute the cross-participant transfer condition using the same training-only pattern sets. If C3's advantage over Zero-Shot falls below roughly +10% or its absolute score drops by more than about 0.05 relative to 0.597, the reported gains are inflated by target-episode leakage; if C3 retains its advantage, the out-of-sample concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.1 defines behavioral patterns as mined 'from longitudinal conversations,' and the extraction prompt in Appendix B.1.1 supplies 'Conversation Data' with no instruction to exclude the target episode or any later turns within it. Section 5 reports prediction scores without documenting a temporal split between pattern mining and prediction. This matters because the only personalization channel in C3 is the set of activated patterns: if that set contains evidence from the same episode being predicted, then the reported +28.9% gain over Zero-Shot and the +29.8% gap over the cross-participant condition can be explained by memorization rather than by person-specific generalization. The cross-participant control does not rule this out, because own-condition patterns can contain the target episode while transferred patterns cannot. Figure 3's 'accumulated performance' curve suggests chronological evaluation, but the paper does not state that the pattern store at time t excludes episodes after time t. The load-bearing assumption is therefore that target turns never appear in the pattern-mining corpus; this assumption is unstated and, as written, the protocol appears to allow its violation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Pattern-Conditioned Prediction (\"Situational Reasoning\"), a training-free LLM-based method that mines interpretable IF-THEN-EXCEPT behavioral patterns from longitudinal smartwatch conversations and uses those patterns to predict a user's next verbal behavior, framed as a Verbal Response Mode (VRM) communicative intention rather than surface wording. The manuscript contributes a longitudinal dataset of 1,000+ hours from 14 participants (9,901 utterances after cleaning), an evaluation comparing the method against Zero-Shot, All-In-Context, and Natural-Language Summary baselines using an LLM judge and a human-rater validation, a cross-participant transfer condition, and semi-structured interviews on users' perceptions of anticipatory behavioral support. The headline result is an LLM-judge score of 0.597 for Pattern-Conditioned Prediction versus 0.463 for Zero-Shot and 0.502 for All-In-Context, with a drop to 0.460 in the cross-participant condition, plus larger gains on participant-flagged intention-to-change patterns. The central claim is that person-specific verbal behavior can be predicted from longitudinal conversational data.","tokens_in":14790,"tokens_out":5573,"duration_ms":57061,"significance":"If the evaluation evidence holds, this is a useful step toward anticipatory, context-aware personal AI: the paper combines a real longitudinal wearable dataset with an interpretable, inspectable pattern representation, and it attempts several good validation practices, including a cross-participant control and a human-rater study with 40 raters. The direction is novel relative to reactive assistants and to fixed dialogue-act predictors, and the emphasis on situation-conditioned regularities is theoretically grounded in Mischel and Shoda's cognitive-affective system theory. However, the current evidence is not yet sufficient to support the central claim because the evaluation protocol leaves open a leakage path between pattern mining and prediction, the ground-truth labels and the primary judge share the same LLM and VRM prior, a fallback component is undefined, and the reported statistical significance is unsupported. These are fixable issues, but they are load-bearing for the paper's main conclusion.","major_comments":[{"comment":"The temporal split between pattern mining and prediction is never stated. Section 4.1 describes patterns as mined \"from longitudinal conversations\" and updated \"as new interactions arrive,\" and the extraction prompt in Appendix B.1.1 supplies \"Conversation Data\" without any instruction to exclude the target episode or later episodes; Section 5 reports prediction scores without documenting the split. If the pattern store for a given target turn includes evidence from that same episode, the reported +28.9% gain over Zero-Shot and the +29.8% gap over the cross-participant condition can be explained by memorization rather than by person-specific generalization. The cross-participant control does not rule this out, because own-condition patterns can contain the target episode while transferred patterns cannot. Figure 3's accumulated-performance curve suggests a chronological evaluation, but the paper does not state that the pattern store at time t excludes episodes after time t. Please specify the split explicitly and report results under a strictly out-of-sample protocol, e.g., patterns mined only from episodes that end before the target turn begins.","section":"§4.1, §5, Fig. 3"},{"comment":"The ground-truth behavioral descriptions are LLM-generated with the same VRM prompt used for predictions, and the primary judge is also an LLM prompted with the same taxonomy. The paper states that the ground truth B_t is \"generated from the actual utterance using the same prompt,\" and the GPT-5 judge evaluates pragmatic function, specificity, and compositionality using the same VRM categories. This shared prior can inflate alignment scores: the predictor, the ground-truth generator, and the judge may all prefer the same stylistic and categorical conventions. The human evaluation partially mitigates this concern and the cross-participant condition helps, but the human raters evaluate predictions against the same LLM-generated ground truths on only 200 sampled scenarios, and the paper does not report human VRM annotations of the ground truths themselves. Please report agreement between LLM-generated ground truths and independent human VRM annotations, and/or report results on a subset with human-annotated ground truth.","section":"§5, Appendix B.1.2"},{"comment":"The claim that Pattern-Conditioned Prediction \"significantly outperforms\" the baselines is not supported by any inferential statistic. Means and standard deviations are reported (C3: M=0.597, SD=0.350; Zero-Shot: M=0.463, SD=0.330; All-In-Context: M=0.502, SD=0.339), but there are no p-values, confidence intervals, effect sizes, or statement of the unit of analysis (turns versus participants). With 14 participants and repeated per-turn measurements, a per-participant paired test (e.g., Wilcoxon signed-rank or paired t-test on participant-level means) with the corresponding test statistic and effect size is needed to justify \"significantly outperforms\" in the abstract and Section 5.1.","section":"§5.1"},{"comment":"The fallback mechanism used when no behavioral pattern activates is undefined. Figure 6 states that participants with lower activation rates \"rely more frequently on the fallback prediction mechanism when no relevant behavioral pattern is activated,\" but the method description in Section 4.1 never defines this fallback or explains how it is invoked during evaluation. Because the reported C3 score aggregates cases with and without activated patterns, the result depends on this unspecified component. Please specify the fallback behavior, and report performance separately for turns with at least one activated pattern and turns handled by the fallback.","section":"§4.1, Fig. 6"}],"minor_comments":[{"comment":"The Natural-Language Summary baseline is introduced in Section 4 and included in the human evaluation, but its LLM-judge scores are not reported in the Section 5.1 results table; please add them for completeness.","section":"§5.1"},{"comment":"The text refers to \"Appendix 4\" for per-participant results and examples, but the appendix is labeled \"A Additional Results\" and the relevant material appears in Figures 5 and 6; the cross-reference should be corrected.","section":"§5.1"},{"comment":"The judge prompt asks for \"a single aggregated score weighting dimensions by relevance to this case\" without a fixed aggregation rule; please specify the aggregation formula or report dimension-wise scores separately, since the aggregate is central to the reported numbers.","section":"Appendix B.1.2"},{"comment":"The human evaluation reports Kendall's tau = 0.83 as agreement among raters, but Kendall's tau is a rank-correlation coefficient; please clarify whether this is inter-rater agreement or agreement between human rankings and the LLM judge, and report an appropriate inter-rater reliability statistic such as Krippendorff's alpha.","section":"§5.1"},{"comment":"The data-cleaning step removes turns with fewer than 4 words, which the paper justifies as excluding trivial exchanges; please report how many of the excluded turns were user turns versus other-speaker turns, since the final user/other balance (57%/43%) may shift after cleaning.","section":"§3"}],"recommendation":"major_revision","confidential_remarks":"The paper is a strong fit for UIST, and the dataset and human-centered framing are valuable. My recommendation of major revision is driven by the unresolved temporal-leakage question and the shared-LLM evaluation loop; these are fixable with a stricter evaluation protocol and additional analyses, so I do not see a reason to reject at this stage. I would also ask the editors to ensure the relationship to the authors' prior Mind Mapper work [5] is clearly delineated in the revised version, since the paper positions itself as an extension of that work."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is a real step forward in a crowded space: it actually collects longitudinal conversational data from 14 people wearing always-on smartwatches (~1,000 hours, 15,066 utterances), mines IF-THEN-EXCEPT patterns, and shows that conditioning prediction on those patterns beats zero-shot and in-context baselines. The cross-participant control (dropping from 0.597 to 0.460) and a 40-rater human evaluation give the claim of person-specificity independent support. I'd credit the authors for also doing interviews, which are thoughtful.\n\nBut the central evaluation has a load-bearing ambiguity. Nowhere in Section 4.1 or 5 does the paper state that pattern mining is strictly out-of-sample — that the pattern store at time t excludes episodes after t. The extraction prompt in Appendix B.1.1 simply supplies 'Conversation Data,' with no instruction to exclude the target turn. If the target episode is in the mining corpus, then the +28.9% gain over zero-shot is mostly memorization, not prediction. The cross-participant control doesn't fix this, because only own-condition patterns could contain the target episode. Figure 3's 'accumulated performance' curve hints at chronological evaluation, but the paper never says it.\n\nThere's also the LLM-mediated loop: ground-truth behavioral descriptions and the judge both use the same VRM taxonomy and same model family, so shared stylistic priors can inflate alignment scores. The human rater comparison is a partial break, but it's on a sample and ranks methods rather than replicating the full scoring. The authors report 'significantly outperforms' without significance tests. And '1,000+ hours' is misleading if it's recording hours rather than speech hours — the cleaned corpus is 9,901 utterances, which is substantial but not what the abstract implies.\n\nNone of this is fatal to the idea. The method is sensible, the dataset is rare, and the qualitative results are worth reading. But the paper is not acceptable as is. I'd send it to peer review with a clear request: state the temporal split explicitly, show results with patterns mined only from prior data, release the prompts and at least derived patterns, add significance tests, and fix the hours description. If the temporal split is already chronological, the authors should say so and show the protocol; if not, the experiment needs to be redone.\n\nWho should read it: people working on wearable sensing, user modeling, and proactive assistance. It deserves a serious referee, but the referee should demand the revision I've outlined.","headline":"A substantial wearable-conversation dataset and an interpretable pattern-conditioning method, but the evaluation never states that pattern mining is out-of-sample, so the headline gains may be inflated.","tokens_in":15354,"tokens_out":2309,"would_cite":false,"duration_ms":24180,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Person-specific verbal behavior can be predicted from a user's own conversation history using mined IF-THEN-EXCEPT patterns, beating both zero-shot and all-in-context baselines on an LLM judge.","keywords":["verbal behavior prediction","longitudinal conversation","large language models","situational reasoning","IF-THEN-EXCEPT patterns","wearable conversation capture","person-specific prediction","anticipatory AI"],"falsifier":"Re-run the evaluation with an explicit temporal cutoff that forbids pattern mining from any conversation overlapping the target turn, and generate ground-truth behavioral descriptions with human annotators or a different model rather than the VRM prompt shared with the predictor; if the 0.597 vs 0.463 margin collapses or reverses, the claim of person-specific prediction is not supported.","tokens_in":14377,"feed_emoji":"🗣️","tokens_out":7853,"duration_ms":70275,"temperature":0.7,"pith_summary":"This paper tries to establish that a person's verbal behavior—the communicative intention behind their next utterance, not the exact words—can be predicted from their own longitudinal conversation history. Using over 1,000 hours of naturalistic speech recorded by 14 participants wearing smartwatches, the authors introduce Pattern-Conditioned Prediction, which mines situation-dependent IF-THEN-EXCEPT rules from a user's past conversations and uses the rules activated by the current context to prompt an LLM to describe the user's likely next verbal tendency. In their evaluation this method scores 0.597 on an LLM-based judge, outperforming a zero-shot baseline (0.463, +28.9%) and an all-in-context baseline that ingests the full raw history (0.502, +18.9%). When patterns from a randomly assigned other participant are substituted, the score drops to 0.460, suggesting the predictions are person-specific rather than generic. If correct, the approach would give proactive AI systems a way to anticipate a user's likely next communicative move and intervene before regretful behavior unfolds.","feed_headline":"Mined chat patterns predict your next conversational move","feed_subtitle":"A new LLM method mines IF-THEN-EXCEPT rules from a person's own 1,000-hour history and beats generic baselines by up to 29%.","key_machinery":"The central object is the contrastive IF-THEN-EXCEPT behavioral pattern: IF [situational condition], THEN [behavioral tendency], BUT NOT when [exception situation]. Each pattern links a situation (when, where, with whom, internal state) to a VRM-grounded communicative tendency, carries a probability estimated from the balance of supporting and contradicting instances, and is merged and refined as new conversations accumulate. At inference time, patterns whose antecedents match the current context and whose exception conditions are absent are activated, weighted by probability, and fed to an LLM together with the current conversation to generate a natural-language prediction of the user's next verbal tendency. This representation is the load-bearing piece: it converts raw long-form history into a small, interpretable, situation-specific rule set that the LLM can reason over, avoiding the brittleness of conditioning directly on long transcripts.","core_discovery":"The paper's central claim is that person-specific verbal behavior can be predicted from longitudinal conversational data by representing the user's behavioral regularities as contrastive situational rules of the form 'IF [situation], THEN [verbal behavior], BUT NOT when [exception]' and conditioning an LLM's prediction on the subset of rules that match the current conversational context. The authors call this Situational Reasoning and evaluate it as Pattern-Conditioned Prediction. Against ground-truth behavioral descriptions generated from the actual next utterance using the same VRM prompt, Pattern-Conditioned Prediction achieves an LLM-judge score of 0.597 (SD 0.350), beating zero-shot (0.463) and all-in-context (0.502); on turns flagged by participants as reflecting behaviors they want to change, the gain is larger (0.858 vs 0.607 zero-shot). A cross-participant transfer condition drops to 0.460, and the approach's performance improves with accumulated data while baselines stay flat—evidence that the improvement comes from person-specific pattern mining rather than generic conversational ability.","pith_inferences":["A direct extension would be to test whether the same IF-THEN-EXCEPT representation transfers to non-verbal behavior, such as spending or procrastination, captured by other sensors, since the representation is not speech-specific.","The pattern-activation fallback mechanism suggests a natural test: measure prediction quality as a function of how many patterns are activated per turn, since the paper hints that low activation correlates with weaker performance.","One could design a prospective study where the system's predictions are shown to the user in real time and the user's ability to change the predicted behavior is measured, testing the implicit assumption that anticipation creates a window for intervention.","Because the judge rewards specificity and compositionality, the 0.597 score may partly reflect the judge's stylistic preferences; an adversarial evaluation with a judge trained to detect pattern-based generic descriptions would separate pattern fidelity from judge artifact."],"forward_implications":["Assistants could issue just-in-time, in-the-moment nudges when a recognized situation begins to unfold, before the user's habitual response occurs.","The same pattern representation could let users inspect, correct, and set goals for their own inferred behavioral tendencies, since patterns are readable rules rather than opaque embeddings.","Performance improving with accumulation implies longer deployments would yield better anticipation, and that the approach should keep getting more personalized over time.","The larger gains on intention-to-change turns suggest behavior-change support is the most promising near-term application.","The cross-participant drop to 0.460 indicates the effect is person-specific rather than a generic conversation-modeling boost."],"supporting_citations":[{"why":"Previous work that introduced behavioral pattern mining from everyday conversations, which Situational Reasoning extends with the prediction framing.","marker":"[5]"},{"why":"Defines the Verbal Response Mode taxonomy used to frame verbal behavior as communicative intention rather than surface form.","marker":"[32]"},{"why":"Supplies the situation-dependent view of personality that motivates mining situation-behavior associations from longitudinal data.","marker":"[20]"},{"why":"Establishes the longitudinal personalization baseline paradigm that the all-in-context condition operationalizes.","marker":"[27]"},{"why":"Provides the LLM-as-a-judge evaluation method used to score predictions along the three alignment dimensions.","marker":"[43]"},{"why":"Provides the LLM used for pattern mining, activation, and prediction.","marker":"[8]"},{"why":"Documents LLM difficulty with long contexts, motivating the symbolic pattern representation over raw history.","marker":"[16]"},{"why":"Prior user-modeling approach that motivates the natural-language summary baseline and longitudinal user representation.","marker":"[31]"}],"fun_headline_variants":["LLM predicts your next words from 1,000 hours of your own chats","Your chat history trains AI to anticipate your responses","Mining your chat rules helps AI foresee your next move","AI learns your behavioral patterns to predict what you'll say next","Forecast your own verbal behavior with LLM trained on your chats"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes the mined patterns come only from conversations that happened before the target turn and that the LLM judge does not favor predictions that share its own text-generation style; the paper does not explicitly state the temporal split for pattern mining, and ground-truth labels are produced by an LLM using the same prompt as the predictors, so a leak of target-episode information or shared stylistic priors would inflate the reported margins.","fun_headline_variants_meta":{"raw":{"variants":["LLM predicts your next words from 1,000 hours of your own chats","Your chat history trains AI to anticipate your responses","Mining your chat rules helps AI foresee your next move","AI learns your behavioral patterns to predict what you'll say next","Forecast your own verbal behavior with LLM trained on your chats"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001042,"raw_usage":{"total_tokens":4403,"prompt_tokens":989,"completion_tokens":3414,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":605,"completion_tokens_details":{"reasoning_tokens":3327}},"tokens_in":605,"tokens_out":3414,"duration_ms":25193,"temperature":1.0,"reasoning_tokens":3327,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T10:37:07.443782+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the evaluation with an explicit temporal cutoff that forbids pattern mining from any conversation overlapping the target turn, and generate ground-truth behavioral descriptions with human annotators or a different model rather than the VRM prompt shared with the predictor; if the 0.597 vs 0.463 margin collapses or reverses, the claim of person-specific prediction is not supported.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Previous work that introduced behavioral pattern mining from everyday conversations, which Situational Reasoning extends with the prediction framing."},{"cited_title":"1992.Describing talk: A taxonomy of verbal response modes","cited_arxiv_id":null,"evidence_quote":"Defines the Verbal Response Mode taxonomy used to frame verbal behavior as communicative intention rather than surface form."},{"cited_title":"2025.Gemini 2.5 Pro Model Card","cited_arxiv_id":null,"evidence_quote":"Provides the LLM used for pattern mining, activation, and prediction."},{"cited_title":"Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang","cited_arxiv_id":null,"evidence_quote":"Documents LLM difficulty with long contexts, motivating the symbolic pattern representation over raw history."}],"review_version":1}