{"id":"18b98e1a-e5f7-4506-93c5-3850fa938663","arxiv_id":"2501.01915","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Social Process models, which treat each conversation group as a meta-learning task and condition forecasts on a short context of the same group, can interpolate to unseen synthetic group dynamics but do not extrapolate beyond training variety.","lead":"This paper tests a meta-learning approach that forecasts the future behavior of a whole conversation group, such as who will speak next, by first observing a short sample of that group's past interactions. The method adapts to new groups only when their behavior falls inside the range of behaviors seen during training, which the authors demonstrate on synthetic head-movement and turn-taking data.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim that SP models generalize to unseen groups is not supported by the reported experiments: all evaluations are synthetic, and the paper's own results show only interpolation within training-distribution dynamics, with extrapolation failing (Section VI.B, Table II).","rationale":"The reader's weakest assumption (stationarity) is real but not the most decisive. The decisive problem is that the experiments are designed so that 'unseen groups' are always drawn from the same generative distribution as training, and the paper's own cross-distribution test shows extrapolation failure. The conditional verdict is appropriate: the manuscript should either deliver the promised real-world evaluation or explicitly limit the central claim to interpolation within training-distribution dynamics. I keep the verdict unchanged.","tokens_in":30531,"tokens_out":6375,"duration_ms":64016,"concrete_test":"Add an experiment on real conversational data (e.g., MatchNMingle or CMU Panoptic Haggling) following the Section V protocol: for each held-out conversation group, build context C from the group's first portion of interactions (observed-future pairs) and evaluate log-likelihood on later, disjoint windows; compare SP against the non-meta VED baseline and an NP baseline. Report per-group and aggregate log-likelihood with error bars over multiple seeds. If SP fails to beat VED on held-out real groups, or if the results are not reported, the abstract and Section V should be rescoped to state that generalization is demonstrated only within the training distribution.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in the abstract and Section V is that SP models 'generalize to unseen groups at evaluation' by conditioning on a context set C of the same group's observed-future pairs. The experiments in Section VI.B do not establish this claim for genuinely unseen dynamics. (1) The glancing-behavior datasets are i.i.d. sinusoids; a 'meta-sample' is just a random subset of sequences from a two-type generative process, so 'unseen group' reduces to 'new sample from the same two-type distribution.' The separated-context experiment shows the model can infer which of two training-time types a context belongs to—a binary classification, not adaptation to a novel group process. (2) In the speaking-turn datasets, groups are generated from a small set of latent rules (clockwise/anticlockwise; dominating person), and held-out groups share those same rules. The only cross-distribution test, Table II, evaluates models trained on Dual/Dual-random/Full-random on the Dominating dataset; the paper itself concludes (Section VI.B, Figure 11) that Dual and Dual-random models fail to generalize, and the Full-random model succeeds only because Dominating dynamics are a subset of Full-random dynamics—i.e., interpolation. (3) The Introduction (Section I) promises 'considerably stronger real-world experiments with larger and more expressive datasets,' but no real-world results appear in Section VI. Thus the unqualified 'generalization to unseen groups' claim is not supported; the support is limited to interpolation among training-time dynamics, which the authors themselves state. The stationarity assumption in Section V is a related weakness but secondary: even with stationarity, the reported evidence does not demonstrate adaptation to novel group dynamics.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper formalizes Social Cue Forecasting (SCF), a meta-learning task in which each conversation group is treated as a separate task, and proposes Social Process (SP) models, Neural-Process-style sequence models that condition forecasts on a context set of observed-future pairs from the same group. An Attentive Social Process variant adds cross-attention over context sequences, and the method includes an offset encoding for non-contiguous observed/future windows and auxiliary geometric losses. The authors evaluate on two families of synthetic datasets (glancing behavior and speaking turns), reporting log-likelihood and mean errors and analyzing latent-space structure. The central claim is that conditioning on a group's context set yields generalization to unseen groups at evaluation.","tokens_in":30843,"tokens_out":3426,"duration_ms":35668,"significance":"If the central claim were established, the formulation would be a useful bridge between meta-learning and multiparty interaction forecasting, and the paper contains several genuine strengths: the synthetic tasks are carefully constructed so that the latent type is identifiable, the ELBO objective is evaluated on held-out meta-samples, the authors are explicit that extrapolation fails, and code, data, and trained models are released. However, the validation does not currently support the advertised real-world generalization. The only cross-distribution experiment shows interpolation, and the introduction promises real-world experiments that do not appear in the manuscript. The contribution is therefore best assessed as a well-executed synthetic analysis with an overstated generalization claim.","major_comments":[{"comment":"The Introduction states that this paper extends previous work with 'considerably stronger real-world experiments with larger and more expressive datasets,' but Section VI contains only synthetic experiments; no real-world results appear anywhere in the manuscript. This is a direct mismatch between the stated contribution and the reported content. The authors should either supply the promised real-world experiments or revise the contribution statement to describe the synthetic generalization analysis accurately.","section":"Section I and Section VI"},{"comment":"The abstract and Section V claim that conditioning on a context set leads to 'generalization to unseen groups,' but the only cross-distribution experiment, Table II, shows that models trained on Dual and Dual-random fail on the Dominating dataset and that the Full-random model succeeds only because Dominating dynamics are a subset of Full-random dynamics. The paper's own conclusion in Section VI.B is that the model can interpolate between known social behaviors but has difficulty extrapolating to out-of-distribution data. The unqualified claim of generalization to unseen groups is therefore not supported for dynamics outside the training distribution; either the claims should be qualified or the evaluation should include held-out groups generated by genuinely novel dynamics.","section":"Section VI.B, Table II, Figure 11"},{"comment":"The load-bearing assumption stated in Section V, that 'the underlying stochastic process generating social behaviors does not evolve over time,' justifies the use of a single group-level latent z and a static context set C. This assumption is not tested anywhere in the manuscript, and the synthetic datasets satisfy it by construction because each group's generative rule is fixed throughout. The paper should either provide an experiment with time-varying group dynamics or explicitly scope the method's validity to stationary interactions, since the meta-learning conditioning mechanism would otherwise fail under drift.","section":"Section V"}],"minor_comments":[{"comment":"There is a typo in 'simultaneusly'; it should be 'simultaneously'.","section":"Section I"},{"comment":"There is a typo in the heading 'monanidc'; it should be 'monadic'.","section":"Section II"},{"comment":"The table headers 'Mixed context', 'Type I context', and 'Type III Context' are used without an explicit explanation of what each column represents; adding a sentence in the text or a footnote would improve readability.","section":"Section VI.B, Table I"},{"comment":"The phrase 'under the random context regime and no-pool configuration' is used without defining these configurations in the main text or appendix; please define them or remove the reference.","section":"Appendix A"},{"comment":"The notation in Equation 10 uses \\hat{s}_l and \\hat{s}_q before explaining that \\hat{s} := \\log \\hat{\\sigma}^2; the explanation should be moved before or integrated into the equation.","section":"Section V, Equation 10"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to be an extension of the authors' prior conference paper [35], and the promised real-world experiments may be reported there. If so, the authors should clearly delineate what is new in this manuscript and avoid claiming real-world experiments that are not present. The overclaim in the abstract and introduction is the main obstacle to publication; the synthetic analysis itself is careful and worth preserving."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is an extension paper that does something rare—it runs controlled synthetic experiments to test where the method actually fails, and then says so. The headline claim in the abstract ('generalization to unseen groups') is stronger than the evidence, but the paper's own Section VI.B conclusion is honest: the model interpolates between training-time dynamics and does not extrapolate.\n\nWhat's new: the posterior collapse analysis (mixed vs. separated context), the latent-space interpolation between two glance types (Figure 8), and the cross-dataset speaking-turn results (Table II, Figure 11) are not in the prior work [35] as far as I can tell. The synthetic tasks are thoughtfully constructed: two modalities with controlled amounts of context information, so you can actually see whether the latent variable is doing work. The paper ships code and data, and the baselines (NP, ANP, VED) are reasonable for the comparison. The Full-random/Dominating result is presented with the right caveat—Dominating is a subset of Full-random dynamics, so success there is interpolation, and the Dual/Dual-random failures show the limit.\n\nSoft spots: the abstract and introduction promise 'considerably stronger real-world experiments' and claim generalization to unseen groups, but all of Section VI is synthetic. The 'unseen group' in the glancing experiments is either a new sample from the same two-type distribution (mixed context) or binary classification of which training-time type a context comes from (separated context). The only cross-distribution test is Table II, which shows interpolation, not adaptation to genuinely novel dynamics. Stationarity (Section V) is a real assumption, but secondary: even if it holds, the evidence doesn't support novel-group adaptation. Minor: metrics are mean/std over sequences, not over training seeds, so we don't know seed sensitivity. Also the intro's promise of real-world experiments should be removed or fulfilled.\n\nBottom line: as an analysis of NP-style models for group behavior forecasting, the paper is solid and the negative result is useful. As a claim of adaptive generalization to unseen groups, it overreaches. The fix is either rescope the claims to synthetic interpolation or add the promised real-world evaluation.\n\nI'd send it to review—the experiments are careful enough that a referee can push on the claim-reality gap and get a better paper. I'd cite it for the interpolation/extrapolation findings if I work on NP generalization. Not sure I'd bring it to a general reading group, but for a social cues/forecasting group, yes.","headline":"Honest synthetic analysis showing SP models interpolate but don't extrapolate; the 'unseen groups' claim overreaches, but the paper deserves review.","tokens_in":31381,"tokens_out":2681,"would_cite":true,"duration_ms":27203,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper tries to show that treating every conversation group as a meta-learning task, and conditioning forecasts on a short context of the same group's behavior, lets a model adapt to groups it never saw during training.","keywords":["meta-learning","social cue forecasting","neural processes","multiparty interaction forecasting","conversation dynamics","latent variable model","human behavior forecasting","generalization"],"falsifier":"Within the paper's own synthetic speaking-turn setup, create a group whose conversation rule changes halfway through an interaction, then measure whether the Social Process model's forecast log-likelihood on the post-switch target window stays high when the context set is drawn only from the pre-switch period; if it collapses, the stationarity assumption that underlies the group-level latent variable is violated.","tokens_in":30342,"feed_emoji":"💬","tokens_out":7438,"duration_ms":69174,"temperature":0.7,"pith_summary":"The paper is trying to establish that group-level social forecasting can adapt to unseen conversation groups if each group is treated as its own meta-learning task. The proposed Social Process models learn to condition a probabilistic forecast of all participants' future low-level cues on a short context set of that same group's observed-future behavior pairs, so that at evaluation a new group can be handled by supplying a context, with no retraining. The authors argue this matters because even the same person behaves differently across different groups, and standard supervised social forecasting models struggle to generalize to new groups. Their synthetic experiments support the mechanism but also bound it: the model can interpolate between known social behavior types and learns meaningful latent representations when contexts are informative, yet it extrapolates poorly to out-of-distribution dynamics, so the claimed adaptation to unseen groups works only when the unseen group's dynamics resemble a subset or combination of training dynamics.","feed_headline":"Group meta-learning adapts forecasts to unseen conversations","feed_subtitle":"A short context of a group's own behavior steers predictions of who speaks and how heads turn, no retraining needed.","key_machinery":"The central mechanism is the context-conditioned predictive distribution p(Y|X,C), built on the Neural Process latent-variable setup: a group-level latent z sampled from q(z|C) injects the group's identity into decoding, and an optional deterministic path r_C, with cross-attention in the Attentive Social Process variant, carries context directly. Around this core, sequence encoders produce per-participant embeddings by combining a self-encoder with a partner encoder that pools, from the target participant's frame of reference, relative quaternion orientation, relative location, and relative speaking status; offset encodings based on sinusoidal positional encodings inject the time gap between observed and future windows, and decoding is autoregressive with a Gaussian observation model plus geometric auxiliary losses for pose and speaking status.","core_discovery":"The paper claims to introduce and formalize Social Cue Forecasting (SCF), jointly predicting a distribution over future multimodal cues (pose, head orientation, speaking status) for all members of a conversation group from their preceding cues, and to solve it with Social Process models, a meta-learning family that predicts p(Y|X,C) by conditioning on a context set C of the same group's past observed-future sequence pairs. The experiments on synthetic glancing and speaking-turn data show that the proposed models, particularly the GRU-based Social Process, give better log-likelihood and uncertainty estimates than non-meta-learning and Neural Process baselines. The paper also demonstrates that when contexts carry information about the behavior type, the model maps different group types to separated latent distributions, learns a semantic latent axis, and can interpolate between known behaviors; however, the same experiments show that generalization to unseen groups is limited to interpolation, since models trained on narrower behavior sets fail to produce sensible forecasts for a new, more complex set of group dynamics.","pith_inferences":["If the interpolation result generalizes, the context set is best understood not as a source of new dynamics but as a selector over a learned library of dynamics; a direct test would be to measure whether the latent axis learned on synthetic glancing behavior appears on real conversations with graded head-turn amplitudes.","The stationarity assumption is the most exposed point: real group interactions drift, so a natural extension is to make the latent variable time-dependent and re-estimate it from a rolling context window, which the paper's fixed-behavior synthetic setting cannot distinguish.","The partner-encoding design, which transforms partners' cues into the target participant's frame before pooling, is a transferable building block that could be tested in other multi-agent forecasting tasks such as traffic or team sports, where each agent's future depends on how it perceives others' positions and headings.","The paper's interpolation conclusion suggests a practical deployment rule: before trusting forecasts for a new group, verify that the group's observed dynamics fall within the convex hull of training dynamics, for example by checking the posterior distance of its context encoding to training encodings."],"forward_implications":["A social robot or agent could adapt to a new group's interaction style after observing a short context of that group, without training a separate model for the group.","Forecasts come with calibrated uncertainty estimates, which the paper argues is necessary because one observed sequence can lead to multiple socially valid futures.","When the context is informative, the model's latent space organizes groups by behavior type and supports interpolation between known behaviors, as shown in the separated-context glancing experiment.","Generalization to unseen groups is bounded by training diversity: models trained on a wider variety of group dynamics adapt better to a new dynamic than models trained on narrow dynamics.","The Social Cue Forecasting formulation, with non-contiguous observed and future windows and explicit offset encodings, is designed to support social-science tasks such as forecasting lagged synchrony, mimicry, and disengagement."],"supporting_citations":[{"why":"It defines the Neural Process generative process and the ELBO objective (Equation 3) that the Social Process family builds on.","marker":"[103]"},{"why":"It defines Attentive Neural Processes, whose cross-attention mechanism the ASP variant adopts to produce query-specific context representations.","marker":"[106]"},{"why":"It is the earlier Social Processes paper that this work extends with synthetic generalization analysis and stronger experiments.","marker":"[35]"},{"why":"It supplies the sequence-to-sequence formulation used to encode observed sequences and decode future sequences.","marker":"[104]"},{"why":"It supplies the RNN encoder-decoder (GRU) backbone used in the sequence-based Social Process variants.","marker":"[105]"},{"why":"It is the source of the relative-feature pooling approach that the paper extends from pedestrian positions to conversation behavior for encoding partners from the target participant's perspective.","marker":"[76]"},{"why":"It supplies the homoscedastic uncertainty-based geometric loss used to regress quaternion orientation and location.","marker":"[114]"},{"why":"It provides the sinusoidal encoding scheme that the paper repurposes to encode the offset between observed and future windows.","marker":"[115]"}],"fun_headline_variants":["Meta-learning predicts group conversations it's never seen","Group context adapts AI forecasts to new conversations","Social Process models forecast unseen group interactions","No retraining: meta-learned group behavior forecasts","Adaptive forecasting for new conversation groups"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise, stated in Section V, is that the stochastic process generating a group's social behavior does not evolve over time, so a single context set and a single latent variable can keep representing the group's future.","fun_headline_variants_meta":{"raw":{"variants":["Meta-learning predicts group conversations it's never seen","Group context adapts AI forecasts to new conversations","Social Process models forecast unseen group interactions","No retraining: meta-learned group behavior forecasts","Adaptive forecasting for new conversation groups"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000891,"raw_usage":{"total_tokens":3861,"prompt_tokens":978,"completion_tokens":2883,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":594,"completion_tokens_details":{"reasoning_tokens":2815}},"tokens_in":594,"tokens_out":2883,"duration_ms":21168,"temperature":1.0,"reasoning_tokens":2815,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:14:38.659391+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Within the paper's own synthetic speaking-turn setup, create a group whose conversation rule changes halfway through an interaction, then measure whether the Social Process model's forecast log-likelihood on the post-switch target window stays high when the context set is drawn only from the pre-switch period; if it collapses, the stationarity assumption that underlies the group-level latent variable is violated.","supporting_citations":[{"cited_title":"Sequence to Sequence Learning with Neural Networks","cited_arxiv_id":null,"evidence_quote":"It supplies the sequence-to-sequence formulation used to encode observed sequences and decode future sequences."}],"review_version":1}