{"id":"828bbfcd-0134-4da9-86da-6b4e7faae84b","arxiv_id":"2607.07387","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":6,"one_line_summary":"An LLM-driven agent-based model with multi-round dialogue reproduces non-linear social influence patterns in vaccination opinion dynamics, with memory increasing resistance and prompt diversity increasing adoption.","lead":"This paper builds an agent-based simulation where AI language models play the roles of people discussing vaccination, testing how memory and communication style affect opinion spread. It matters because it tests whether AI agents can reproduce known social dynamics like polarization and persuasion thresholds, which could make social simulation more realistic.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"The 'emergence' claim in §3.3 is undermined by a concrete circularity path: the LLM receives numeric opinion scores in its prompts and may apply a learned theory-consistent mapping rather than generating patterns from interaction dynamics.","rationale":"The reader correctly identified the circularity concern as the most load-bearing issue. I refine it by making the mechanism concrete: the LLM receives numeric opinion scores in its prompts, so the mapping from input scores to output scores — which generates the scatter plot patterns — could be a learned application of social judgment theory rather than an emergent property of dialogue. This is more specific than 'the training data contains the theory'; it identifies the exact pathway (numeric opinion scores in prompts → LLM applies learned mapping → non-linear patterns appear in scatter plots) through which circularity operates. The reader's verdict of CONDITIONAL is appropriate. The framework is a reasonable proof-of-concept demonstrating that LLM-driven ABMs can produce qualitatively interesting opinion dynamics. But the 'emergence' and 'level 3 validation' claims overreach the evidence without addressing the circularity confound. The paper also lacks code/data release, sensitivity analysis on α=0.5, and the reported filter rate for LLM refusals — all mentioned by the reader and all valid secondary concerns. No adjustment to the verdict is needed; the reader's assessment is well-calibrated.","tokens_in":9782,"tokens_out":3889,"duration_ms":193493,"concrete_test":"Run the simulation with a modified reflection prompt that omits the neighbor's numeric opinion score, providing only the dialogue transcript and the agent's own current opinion. Recompute the Figure 5 scatter plots and Table 3 quadrant proportions. If the repulsive quadrant share (currently 31.0% baseline) and the threshold non-linearity persist, the patterns are extracted from dialogue content itself, strengthening the emergence claim. If the repulsion share drops substantially (e.g., below 15%) or the threshold effect flattens, the LLM was primarily using the numeric disparity to apply a learned theory-consistent mapping, and the 'emergence' claim weakens. Additionally, report whether vaccinated agents (o=1.0) are included in the Figure 5 data points.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim (Section 3.3) is that non-linear influence patterns — threshold effects, repulsive influence — 'emerge naturally from LLM-based interactions' and are 'not imposed exogenously.' However, the dialogue and reflection prompts (Section 2.3) feed the LLM explicit numeric opinion scores for both agents. The reflection module then asks the LLM to output a new numeric opinion score. The scatter plot patterns in Figure 5 are determined by this mapping from input opinion scores to output opinion scores. If Qwen3-8B has learned from training data that 'large opinion differences lead to resistance' (as described in the social judgment theory and bounded confidence literature the paper itself cites — Sherif & Hovland 1961, Jager & Amblard 2005), then the threshold and repulsion patterns are the LLM reproducing a learned textual regularity, not an emergent property of multi-round dialogue. The paper does not discuss or test this alternative explanation. This is the most load-bearing concern because it directly challenges the word 'emergent' in the strongest claim: the patterns could be a memorized theory applied via numeric inputs rather than a product of the interaction dynamics the framework is designed to study. A secondary issue compounds this: the opinion space is bounded at [-1,1] and vaccinated agents are pinned at o=1.0, which could create artificial non-linearity in the scatter plots (Figure 5) independent of LLM behavior — agents near the boundary have asymmetric movement possibilities that could mimic threshold effects. The paper does not clarify whether vaccinated agents are excluded from the Figure 5 scatter plots.","agreement_with_reader":"agree"},"referee_report":{"model":"glm-5.2","summary":"This paper introduces an LLM-driven agent-based modeling framework using Qwen3-8B to simulate vaccination opinion dynamics. Agents engage in multi-round dialogues, reflect on conversations to update their opinion scores, and update opinions via a mathematical aggregation layer (Eq. 1). The authors evaluate four scenarios varying cognitive modules (memory, prompt diversity) across 95 agents over 10 time steps. They report that memory reduces vaccination rates while prompt diversity increases them, and claim that non-linear social influence patterns (assimilative and repulsive influence, threshold effects) emerge naturally from LLM-based interactions rather than from imposed rules, suggesting potential for level 3 validation of agent-based models.","tokens_in":10054,"tokens_out":1413,"duration_ms":252853,"significance":"The paper addresses a timely question in computational social science: how LLM-based dialogue can replace static numerical rules in agent-based opinion dynamics models. The modular experimental design (Table 2) and the micro-level scatter analysis (Figure 5, Table 3) are commendable attempts to move beyond aggregate outcome reporting to mechanism-level inspection. The framework produces falsifiable, quadrant-based predictions about assimilative versus repulsive influence. The use of synthetic population data with heterogeneous social networks provides demographic grounding.","major_comments":[{"comment":"§3.3, final paragraph: The claim that non-linear influence patterns 'emerge naturally from LLM-based interactions' and are 'not imposed exogenously' is not adequately supported. The reflection module (§2.3) feeds the LLM explicit numeric opinion scores for both agents and asks it to output a new numeric opinion score. If Qwen3-8B has learned from training data that large opinion differences lead to resistance (as described in the social judgment theory and bounded confidence literature the paper itself cites — Sherif & Hovland 1961, Jager & Amblard 2005), then the threshold and repulsion patterns in Figure 5 could be the LLM reproducing a learned textual regularity rather than an emergent property of multi-round dialogue. The paper does not discuss this alternative explanation. This is load-bearing because the word 'emergent' is central to the paper's strongest claim. The authors should:","section":null},{"comment":"§2.2.1, Table 1: The openness parameter α=0.5 is fixed for all agents based on Yin et al. (2024), but that work uses a different model architecture (rule-based ABM, not LLM-driven). Since α directly controls the magnitude of opinion change in Eq. (1), the scatter patterns in Figure 5 and the quadrant statistics in Table 3 are partly determined by this fixed parameter. No sensitivity analysis is provided. The paper should either vary α and show that the qualitative patterns persist, or explicitly acknowledge that the reported non-linearities are conditional on this parameter choice.","section":null},{"comment":"§2.3 and §3.3: The opinion space is bounded at [-1,1] and vaccinated agents are pinned at o=1.0. This boundary could create artificial non-linearity in the scatter plots (Figure 5) independent of LLM behavior — agents near the upper boundary have asymmetric room for opinion movement, which could inflate the apparent repulsion share in QII+QIV. The paper does not control for this boundary effect when interpreting the scatter patterns. The authors should either exclude interactions involving agents near the boundary from the scatter analysis, or demonstrate that the non-linear patterns persist away from the boundaries.","section":null},{"comment":"§3, verification paragraph: The filtering of LLM refusals ('As an AI' responses) is mentioned, but the paper does not report what fraction of responses were filtered, whether the filter rate varies across scenarios, or whether filtering introduces systematic bias (e.g., filtering may remove interactions where the LLM was uncomfortable with extreme opinion distances, biasing the remaining data toward assimilation). Since the filtered interactions are part of the dialogue-reflection pipeline that generates the opinion changes analyzed in §3.3, this is relevant to the validity of the micro-level results. The authors should report filter rates per scenario and discuss potential bias.","section":null}],"minor_comments":[{"comment":"Table 3 caption references 'Fig.8A–D' but the corresponding figures are labeled Figure 5A–D in the text. This inconsistency should be corrected.","section":null},{"comment":"§2.2.1: The initial opinion distribution is described as N(0, 0.3²) in one place and N(0, 0.3) in another (§2.2.2). These are different distributions. The notation should be consistent.","section":null},{"comment":"§2.3, Eq. (1): The term dO_{i,j,t} is described as 'the direct opinion shift derived from the reflection score following the dialogue,' but the precise computation from the reflection output to dO is not specified. This makes the equation difficult to reproduce.","section":null},{"comment":"§3.1: The claim that 'opinion polarization emerges under all four scenarios' (Figure 3C) is partly an artifact of the design: vaccinated agents are pinned at o=1.0, which mechanically increases the standard deviation. The authors should note this confound when interpreting the polarization trend.","section":null},{"comment":"Figure 2: The example dialogue is useful but the figure quality should be improved for readability in print.","section":null},{"comment":"§2.2.1: The number of dialogue rounds is fixed at two. The paper's title emphasizes 'multi-round communication,' but two rounds is minimal. The authors should justify this choice or discuss how the number of rounds might affect the results.","section":null},{"comment":"The paper would benefit from citing and comparing to Chuang et al. (2024), which is referenced in the introduction but not discussed in relation to the paper's findings on LLM-based opinion dynamics.","section":null}],"recommendation":"major_revision","confidential_remarks":"The circularity concern raised in the stress-test note is legitimate and load-bearing. The paper's central claim of 'emergence' is undermined by the fact that the LLM receives numeric opinion scores in its prompts and may apply a learned theory-consistent mapping. This is not fatal — the authors can address it by acknowledging the confound, running control experiments (e.g., removing numeric scores from prompts), or softening the emergence claim. But it must be addressed before the paper can be accepted. The other major comments (α sensitivity, boundary effects, filter bias) are also fixable with additional analysis or discussion."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for a careful and constructive report. The comments identify four substantive issues: (1) the 'emergence' claim may be confounded by the LLM reproducing learned textual regularities from social judgment theory; (2) the fixed openness parameter α=0.5 lacks sensitivity analysis; (3) boundary effects from the [-1,1] opinion space may inflate apparent non-linearity in scatter plots; and (4) LLM refusal filtering rates and potential bias are not reported. We agree that all four points require revision. We can fully address points 1, 3, and 4 with textual additions and supplementary analysis. Point 2 requires a sensitivity analysis that we can partially conduct but may be constrained by computational resources for a full sweep.","responses":[{"response":"The referee raises a valid and important concern. We agree that the distinction between genuine emergence and reproduction of learned textual regularities is critical, and our current manuscript does not adequately address this alternative explanation. In the revision, we will revise the language in §3.3 to qualify the emergence claim more carefully. Specifically, we will replace 'emerges naturally from LLM-based interactions' with language acknowledging that the observed non-linear patterns arise from the LLM's generative process without being explicitly coded as numerical rules, while noting that the LLM may have internalized regularities from its training corpus that align with social judgment theory. We will add a dedicated paragraph discussing this confound explicitly, noting that our framework does not impose a bounded-confidence threshold or a repulsion rule as a mathematical function (as in Jager & Amblard 2005), but that the LLM's behavior may reflect learned associations from training data that encode similar patterns. We believe this is still a meaningful result—the non-linearities arise from natural language dialogue rather than from an imposed numerical rule—but we agree the word 'emergent' overstates the claim without further evidence. We will also note that disentangling these explanations would require controlled experiments (e.g., ablating the numeric opinion scores from the reflection prompt) that we leave to future work.","revision_made":"yes","referee_comment":"§3.3, final paragraph: The claim that non-linear influence patterns 'emerge naturally from LLM-based interactions' and are 'not imposed exogenously' is not adequately supported. The reflection module feeds the LLM explicit numeric opinion scores and asks it to output a new numeric opinion score. If Qwen3-8B has learned from training data that large opinion differences lead to resistance (as in social judgment theory), the threshold and repulsion patterns could be the LLM reproducing a learned textual regularity rather than an emergent property of multi-round dialogue."},{"response":"We agree that the lack of sensitivity analysis for α is a gap, and that citing Yin et al. (2024) for this parameter value is insufficient given the architectural difference between their rule-based ABM and our LLM-driven framework. We will conduct a sensitivity analysis by running the baseline and combined scenarios with α values of 0.3, 0.5, and 0.7, and will report whether the qualitative patterns (presence of repulsive interactions, threshold effects) persist across these values. We expect the quadrant proportions to be robust to α because α scales the magnitude of opinion changes uniformly in Eq. (1) without changing their sign, meaning the quadrant classification (which depends on the sign of Δo) should be largely unaffected. However, the boundary effects and the distribution of points within quadrants may shift. We will add this analysis as a supplementary section and explicitly acknowledge in §2.2.1 that the reported patterns are conditional on this parameter choice. If computational constraints prevent a full sweep across all four scenarios, we will at minimum report results for the baseline scenario and discuss the expected generalization.","revision_made":"partial","referee_comment":"§2.2.1, Table 1: The openness parameter α=0.5 is fixed for all agents based on Yin et al. (2024), but that work uses a different model architecture (rule-based ABM, not LLM-driven). Since α directly controls the magnitude of opinion change in Eq. (1), the scatter patterns in Figure 5 and the quadrant statistics in Table 3 are partly determined by this fixed parameter. No sensitivity analysis is provided."},{"response":"This is a well-taken point. The boundary at [-1,1] and the pinning of vaccinated agents at o=1.0 do create asymmetric movement potential that could confound the scatter analysis. In the revision, we will address this in two ways. First, we will re-run the quadrant analysis excluding interactions where either agent's opinion is within 0.15 of either boundary (i.e., |o| > 0.85), and report the revised quadrant proportions. If the repulsion share remains substantially above zero in this restricted sample, this demonstrates that the non-linear patterns are not purely boundary artifacts. Second, we will add a discussion paragraph in §3.3 explicitly acknowledging the boundary effect and its potential to inflate the QII+QIV share, and we will report both the full-sample and boundary-excluded statistics in Table 3 so readers can assess the magnitude of this confound. We note that the vaccinated-agent pinning is a design choice reflecting the irreversibility of vaccination decisions, but we agree it should not contaminate the micro-level influence analysis without explicit acknowledgment.","revision_made":"yes","referee_comment":"§2.3 and §3.3: The opinion space is bounded at [-1,1] and vaccinated agents are pinned at o=1.0. This boundary could create artificial non-linearity in the scatter plots (Figure 5) independent of LLM behavior — agents near the upper boundary have asymmetric room for opinion movement, which could inflate the apparent repulsion share in QII+QIV."},{"response":"We agree this information should be reported. In the revision, we will add a table reporting the filter rate (percentage of responses containing refusal phrases such as 'As an AI' or 'I cannot provide') per scenario per time step, along with the total number of filtered interactions. We will also discuss whether the filter rate varies systematically across scenarios (e.g., whether the memory scenario, which involves more context, triggers more refusals). Regarding potential bias, we will add a discussion noting that if the LLM tends to refuse more often in interactions involving extreme opinion distances or sensitive vaccination arguments, the filtered sample could be biased toward assimilative interactions, which would mean our reported repulsion shares are conservative estimates. We will also report the filter rate broken down by opinion distance bins to check for this pattern. If the data show no systematic relationship between opinion distance and filter rate, we will state this explicitly; if a relationship exists, we will acknowledge the direction of the bias.","revision_made":"yes","referee_comment":"§3, verification paragraph: The filtering of LLM refusals is mentioned, but the paper does not report what fraction of responses were filtered, whether the filter rate varies across scenarios, or whether filtering introduces systematic bias."}],"tokens_in":9750,"tokens_out":1474,"duration_ms":185913,"standing_objections":[]},"desk_editor":{"model":"glm-5.2","letter":"The paper builds an LLM-driven agent-based model for vaccine opinion dynamics using Qwen3-8B, with multi-round dialogue, reflection, memory, and prompt diversity modules. The ablation across four module configurations is clean — 10 runs each, identical initial conditions — and the finding that memory suppresses opinion change while prompt diversity amplifies it is a concrete, useful result. The micro-level scatter plots (Figure 5) examining opinion disparity against opinion change, decomposed into assimilative vs. repulsive quadrants, are the best part of the paper. That decomposition is a genuinely nice analytical move that other LLM-ABM papers haven't done as explicitly.","headline":"LLM-driven ABM for vaccine opinion dynamics: functional framework with a real circularity problem in the 'emergence' claim","tokens_in":10545,"tokens_out":1021,"would_cite":false,"duration_ms":76468,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"LLM agents reproduce polarization without being told to","keywords":["LLM agents","agent-based modeling","opinion dynamics","vaccine hesitancy","social influence","polarization","social judgment theory","bounded confidence"],"falsifier":"If the LLM were prompted with a system instruction that explicitly forbids any opinion movement away from a neighbor's position (no repulsion), and the scatter plots still showed substantial second/fourth quadrant activity, the claim of emergent repulsive influence would be undermined. Conversely, if a different LLM architecture not trained on social science literature produced the same patterns, the emergence claim would be strengthened.","tokens_in":9865,"feed_emoji":"🧠","tokens_out":914,"duration_ms":142191,"temperature":0.7,"pith_summary":"The paper introduces an agent-based modeling framework where individual agents, powered by a large language model (Qwen3-8B), engage in multi-round natural language dialogues about vaccination. After each conversation, agents reflect on the exchange and update their opinion scores, which are then aggregated through a mathematical update function to drive macro-level opinion dynamics. The framework tests four scenarios by toggling a memory module (agents retain past interaction histories) and a prompt diversity module (agents adopt varied communication styles based on their demographic profiles). The central claim is that non-linear social influence patterns—specifically assimilative influence (opinions moving toward a neighbor's), repulsive influence (opinions moving away), and opinion-change thresholds—emerge from the LLM's dialogue and reflection generation without being imposed as rules. The paper argues this demonstrates validity at level 3 of agent-based model validation, where simulated micro-level behaviors qualitatively match empirically documented human social influence patterns.","feed_headline":"LLM agents reproduce polarization without being told to","feed_subtitle":"Simulated vaccine conversations spontaneously generate assimilation, repulsion, and opinion thresholds matching classic social psychology—no","key_machinery":"Qwen3-8B","core_discovery":"When LLM-driven agents converse and reflect, they spontaneously produce both assimilative influence (31-41% of interactions depending on scenario) and repulsive influence (29-39%), along with a threshold effect where moderate opinion differences lead to convergence but extreme differences trigger resistance. This matches the predictions of social judgment theory (Sherif and Hovland, 1961) and bounded confidence models (Jager and Amblard, 2005), but the paper claims these patterns arise from the language model's conversational generation rather than from exogenously imposed rules. Additionally, the memory module increases repulsive responses (from 31.0% to 38.6%) and lowers vaccination rates,","pith_inferences":["The paper's claim of emergence rests on the LLM not having been explicitly prompted to produce assimilative or repulsive behavior. However, the LLM's training data likely contains descriptions of social judgment theory, bounded confidence models, and polarization dynamics. If the model has internalized these patterns as textual regularities, the 'emergence' may be circular: the LLM reproduces the ","The fixed openness parameter α=0.5 and the mathematical aggregation layer (Equation 1) constrain how LLM-generated opinion shifts propagate through the network. The non-linear patterns observed in the scatter plots could be partly shaped by this hybrid architecture rather than purely by the LLM's dialogue generation. Disentangling the LLM's contribution from the aggregation mechanism would require","The finding that memory amplifies repulsive influence suggests a testable hypothesis: agents with longer memory windows should show progressively higher repulsion rates and stronger polarization, as accumulated negative experiences compound resistance. This could be tested by varying memory length as a continuous parameter.","The 95-agent scale limits the ability to distinguish between genuine emergent polarization and finite-size effects. Repulsive influence at 31% in a small population could produce apparent polarization that would wash out or intensify at larger scales, making the scaling experiment the authors propose a critical validation step."],"forward_implications":[],"fun_headline_variants":["LLM agents spontaneously reproduce classic opinion polarization patterns","Memory makes LLM agents more resistant to opposing vaccine views","LLM agents show assimilation and repulsion without programmed rules","Vaccine opinion dynamics emerge from LLM agent conversation alone","Memory module lowers vaccination acceptance in LLM agent simulations"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The paper claims that non-linear social influence patterns (assimilation, repulsion, thresholds) 'emerge naturally from LLM-based interactions,' but this assumes the LLM is not simply reproducing patterns it learned from training data that already contains descriptions of these exact social psychology theories.","fun_headline_variants_meta":{"raw":{"variants":["LLM agents spontaneously reproduce classic opinion polarization patterns","Memory makes LLM agents more resistant to opposing vaccine views","LLM agents show assimilation and repulsion without programmed rules","Vaccine opinion dynamics emerge from LLM agent conversation alone","Memory module lowers vaccination acceptance in LLM agent simulations"]},"model":"glm-5.2","effort":"low","cost_usd":0.0,"raw_usage":{"total_tokens":599,"prompt_tokens":521,"completion_tokens":78,"prompt_tokens_details":null},"tokens_in":521,"tokens_out":78,"duration_ms":50173,"temperature":1.0,"reasoning_tokens":null,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-09T12:16:26.554402+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"If the LLM were prompted with a system instruction that explicitly forbids any opinion movement away from a neighbor's position (no repulsion), and the scatter plots still showed substantial second/fourth quadrant activity, the claim of emergent repulsive influence would be undermined. Conversely, if a different LLM architecture not trained on social science literature produced the same patterns, the emergence claim would be strengthened.","supporting_citations":[],"review_version":1}