{"id":"75b51b01-3030-47e6-b2b6-abd6369ab5b0","arxiv_id":"2411.19635","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"A simulated social network with LLM agents shows a Q-learning bot can gain about 10% of available followers, with the authors arguing that a small action space and self-observation are key to stable learning.","lead":"This paper builds a simulated social media platform where AI agents post opinions and follow or unfollow each other, then trains one bot with reinforcement learning to win followers by choosing what to say. The authors report that giving the bot a small set of opinion choices and letting it track its own follower count makes the training more stable.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No ablation isolates action-space restriction or self-observation, and the reward is generated by the same hand-set homophily rule the action directly controls; the headline causal claims are not identified by the reported experiments.","rationale":"I read the paper in good faith: the simulation pipeline is coherent, the code is released, and the learning curves are plausible. The reader's REJECT is warranted, but I locate the decisive weakness in the missing factorial control rather than primarily in the homophily rule. The link-update rule is a real validity threat and is inconsistent with §2.3's 'no assumed mechanism' claim, yet even a fully realistic simulator cannot support a causal statement about factors that were never varied. The eight-scenario design varies LLM temperature, follow/unfollow mode, and initial visibility; none is the claimed factor. A 2x2 ablation with a random-action baseline would settle whether the headline claims have empirical support. Because no such control appears, the central contribution is unsupported as written, consistent with the reader's REJECT. I would keep the verdict unchanged.","tokens_in":14540,"tokens_out":10222,"duration_ms":88188,"concrete_test":"Run a 2x2 factorial control within the released code: restricted versus unrestricted action space crossed with state including versus excluding the target's follower count, keeping all other parameters fixed, and compare mean follower gain over 500 episodes. If the unrestricted or no-self-observation arm matches the restricted/self-observation arm, the §1.6 claim fails. To rule out the homophily artifact in the same protocol, also run a uniform-random action policy across the existing eight scenarios; if that random baseline attains the reported 10% gain, the follower reward is not evidence of RL-driven influence.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central assertion in §1.6, repeated in §6.4, is that limiting the action space and incorporating self-observation are key factors for stable opinion-leader generation. That assertion requires a contrast that the experiments never provide. All eight scenarios in §5.14 use the same predefined five-category action dictionary (§4.7) and the same state that always includes the target's follower count (§4.6). The 'Part-/Full-Observable' contrast varies the agent's initial follow list, not self-observation, so no arm tests the claimed factors. The second headline result, the '10% of total available follower population' gain, is generated entirely by Algorithms 2 and 3: follow links are added when opinion categories match for num_same consecutive steps (probability 0.8) and removed when categories differ (probability 0.5). Because the RL action directly selects one of those opinion categories, the follower reward is a direct consequence of this hand-coded homophily rule rather than evidence of linguistically mediated influence. This also contradicts §2.3's claim that the method does not prescribe an influence mechanism. The only baseline, described in §5.16 as a flat line, is asserted in prose rather than measured, so the reported learned advantage has no calibrated comparison.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript introduces a simulated social media environment in which LLM-driven agents produce topic-specific posts, have their opinions elicited via perplexity scores, and establish or break follow links based on opinion-category similarity. A target agent is trained with Q-learning to choose among five predefined opinion categories in order to maximize its follower count. Experiments are run on Gender and Drug topics across eight scenarios that vary LLM decoding settings, follow/unfollow dynamics, and initial observability, reporting rewards of roughly one to three followers in a 20-agent population. The paper claims that restricting the action space and incorporating self-observation are key to stable opinion-leader generation, and that RL achieves a gain of 10% of the total available follower population.","tokens_in":14721,"tokens_out":5838,"duration_ms":52021,"significance":"The combination of LLM post generation, perplexity-based opinion measurement, and Q-learning in a synthetic social network is a potentially useful prototype for ethically exploring topic-specific opinion dynamics, and the paper provides detailed algorithms and a public code repository. If the central claims were established, the framework could be a valuable testbed for studying influence processes with real-data alignment. However, the experimental design never varies the two factors named in the headline claims, the baseline is asserted rather than measured, and the reward mechanism is a direct consequence of the hand-coded homophily rule. The current evidence is therefore not sufficient to support the paper's stated contributions.","major_comments":[{"comment":"The central claim that limiting the action space and incorporating self-observation are key factors is not tested by any ablation. All eight scenarios use the same five-category action dictionary (§4.7) and the same state vector that always includes the target's follower count (§4.6); the Part-/Full-Observable contrast in §5.14 varies whether the target initially follows one or all agents, not whether self-observation is present. Without conditions that vary action-space size and the presence of self-observation, the experiments cannot support this headline contribution.","section":"§1.6, §5.11, §5.14"},{"comment":"The reward is the change in follower count, and follow links are created or removed only when opinion categories match or differ across consecutive steps, with hand-set probabilities (follow 0.8 and unfollow 0.5, per §5.14). Since the RL action directly selects the opinion category, the follower reward is a direct consequence of this hand-coded homophily rule rather than evidence of linguistically mediated influence. This also contradicts §2.3, which states that the method does not prescribe a particular mechanism for how influence spreads; Algorithms 2 and 3 prescribe precisely such a mechanism.","section":"§3.23–3.24, Algorithms 2–3, §4.5"},{"comment":"The baseline is asserted, not measured. The statement that a no-RL agent \"would be a statistical flat line with no followers gained over time\" is presented without a corresponding experiment, and no random-policy, fixed-action, or random-initial-condition comparator is reported. Consequently, the reported 10% follower gain (§5.15) cannot be attributed to RL, and the convergence of the learning curves (§5.17) only shows that training stabilized rather than that the agent outperformed a baseline.","section":"§5.16"},{"comment":"The claim that convergence of reward learning curves indicates that \"the RL algorithm successfully identified optimal solutions\" is not justified. Q-learning convergence guarantees apply to stationary single-agent Markov decision processes, whereas this is a non-stationary multi-agent environment in which all other agents' posts and opinions evolve partly in response to the target agent. No convergence theorem, optimality check, or comparison with exhaustive policy evaluation is provided, so the word \"optimal\" in the conclusions is unsupported.","section":"§5.17, §6.4"}],"minor_comments":[{"comment":"The parameter num_same, which controls the number of consecutive matching opinion categories required for a follow or unfollow event, is not specified in Table 3 or in the main text; its value and choice should be reported.","section":"§3.23, Algorithms 2–3"},{"comment":"The relationship between epsilon and exploration rate is unclear, and Algorithm 4 refers to \"a policy derived from Q\" without defining the exploration schedule; please specify the exact action-selection rule.","section":"Table 3, Algorithm 4"},{"comment":"The claim that the simulation result \"aligns with the polarity distribution of the actual dataset\" is supported only by qualitative inspection of the plotted distributions; a quantitative comparison would strengthen the validation.","section":"§1.4, Figure 3"},{"comment":"The captions of the learning curves should state the number of repeated runs and explicitly define the shaded areas as variance or confidence intervals, so that the reported stability can be judged.","section":"Figures 6–7"},{"comment":"The state description lists \"self-observation\" as a component, but the formal state definition in §4.6 only mentions followed agents' opinion states and the target's follower count; please clarify what the self-observation component is.","section":"§4.6, Figure 2, Table 3"},{"comment":"There are several typographical issues, including \"Table.2 shows the our setting\" after Table 2 and \"filed experiments\" in §2.2; a careful proofread is needed.","section":"Various"}],"recommendation":"reject","confidential_remarks":"The manuscript describes a working prototype and makes its code available, which is commendable, but the experimental design does not test the paper's central claims and the reward mechanism makes the reported RL behavior largely predetermined by the hand-coded homophily rule. A future version that repositions the contribution as a tool for studying homophily-driven synthetic dynamics, adds proper baselines and ablations, and tempers the optimality language could be of interest, but that would require a substantial reworking beyond a standard major revision. The journal may also wish to confirm that all simulation parameters, including num_same and the exploration schedule, are fully documented."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this is a working simulation pipeline with a real topic dataset and released code, but the headline finding is not actually tested. The paper says restricting the action space and adding self-observation drive stable opinion-leader generation, yet every RL run uses the same fixed five-category dictionary and always includes the agent's follower count in the state. The 'Part-/Full-Observable' contrast changes how many other agents the target follows initially, not whether it observes itself. So the central causal claim has no experimental contrast behind it.\n\nWhat is genuinely useful: the environment is concrete. LLM post generation, perplexity-based opinion scores, homophily-based link updates, and a Q-learning target agent all fit together cleanly. The action dictionary trick, where RL picks an opinion category and the LLM only renders text, is a sensible way to keep the decision space tractable. The code is on GitHub, the data is from Kialo, and the authors run two topics with multiple link dynamics and report reward curves with variance over five repetitions. As a sandbox for experimenting with opinion dynamics, it is a reasonable starting point.\n\nThe soft spots are real and they are load-bearing. The reward is follower change, and followers are created by Algorithms 2 and 3: agents start following when their opinion categories match for a few consecutive steps, with probability 0.8, and unfollow on mismatch. The RL action directly selects the opinion category. So the learned 'influence' is largely the agent discovering the homophily rule, not linguistically mediated persuasion. Section 2.3 claims the method does not prescribe an influence mechanism, but the link-update rule is exactly such a prescription. The baseline is asserted in prose ('statistical flat line') rather than measured. The '10% follower gain' is an artifact of this setup, not a general result.\n\nNone of this kills the framework as a simulation tool, but it means the paper's second contribution, the discovery about action-space restriction and self-observation, is unsupported as stated. Fixing it would require real ablations: a larger or unrestricted action space, a state without the agent's own follower count, and a baseline that is actually computed rather than described.\n\nWho should read it: people working on LLM-agent social simulations, especially those building RL-controlled bots. It is not ready as-is; the claims need to be scaled back or the experiments need to deliver the missing contrasts. A serious referee should see it, because the infrastructure is real and the flaws are fixable. If I were handling it, I would send it out with the expectation of major revision, and I would ask specifically for the ablations.","headline":"A useful simulation pipeline with released code, but the headline claims about action-space restriction and self-observation are not tested by the reported experiments.","tokens_in":15315,"tokens_out":2047,"would_cite":false,"duration_ms":18284,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A reinforcement-learned LLM bot learns to gain 10% of a synthetic network's followers.","keywords":["agent-based modeling","large language models","reinforcement learning","Q-learning","opinion dynamics","social influence","opinion leader","simulated social network"],"falsifier":"Run the same RL setup with the follow probability changed to 0.5 and the unfollow probability changed to 0.8, and with the consecutive-match requirement relaxed to a single-step match; if the learned policy and the reported 10% follower gain change sharply or vanish, the result is an artifact of the link rule rather than learned influence. A complementary check is to compare the RL bot's final follower count, across many random seeds, against a non-learning bot that picks uniformly among the same five opinion categories while using the same LLM for post generation.","tokens_in":14229,"feed_emoji":"🤖","tokens_out":4336,"duration_ms":36606,"temperature":0.7,"pith_summary":"The paper tries to establish that a simulated online social network made of large language model agents can serve as an ethical testbed for studying opinion influence, and that a reinforcement learning agent can learn to grow its follower count inside that world. The central claims are that a Q-learning bot can gain about 10% of the total available follower population and that stable opinion-leader behavior requires two design choices: limiting the bot's action space to five predefined opinion categories and letting it observe its own follower count. If these findings hold, researchers can run controlled influence and misinformation experiments without real-world manipulation, and the framework can be extended to topics where real social data does not yet exist.","feed_headline":"RL-trained LLM bot gains 10% of a synthetic network's followers","feed_subtitle":"A simulated social network shows restricting action space and self-observation stabilizes opinion-leader learning.","key_machinery":"The central machinery is a three-stage simulation loop — LLM post generation, opinion elicitation, and link update — with a Q-learning control loop on top. Post generation uses GPT-3.5 prompts containing the agent's prior posts and the posts of accounts it follows; opinion elicitation converts each post into a polarity score by comparing conditional perplexities of a pro sentence and a con sentence, producing a value in [0,1] that is binned into five opinion categories; link update applies a homophily rule that adds a follow link when two agents stay in the same opinion category over consecutive steps, with hand-set follow (0.8) and unfollow (0.5) probabilities. The RL bot observes the opinion states of the accounts it follows plus its own follower count, chooses one of the five opinion categories as its action, and receives the change in followers as reward. This combination matters because it lets a linguistic choice be translated, through the link rule, into a measurable network outcome that the RL can optimize.","core_discovery":"On its own terms, the discovery is a framework plus a design finding: an LLM-driven bot placed among LLM-driven users can learn, through reinforcement learning, which of five opinion stances to voice in order to maximize followers. Across eight scenario combinations — two topics, two LLM creativity settings, two link-update dynamics, and two observability settings — the authors report convergence in the reward curves and follower gains rising from 1 to 2–3 in a population of 20 agents, which they state as a 10% gain of the total available follower population. They attribute the stable emergence of this opinion-leader behavior to constraining the action space to a dictionary of real posts rather than letting the LLM generate arbitrary tweets, and to including the bot's own follower count as part of its observed state.","pith_inferences":["An untested implication is that most of the 10% follower gain may come from simply choosing the opinion category the population already holds, rather than from adaptive sequencing; a fair comparison would pit the RL bot against a non-learning bot that randomizes among the same five categories, which the paper does not report.","The reward landscape is produced entirely by a homophily rule with fixed probabilities, so the result is best understood as a property of that rule; real platforms add recommendation algorithms, multi-topic ties, and bot detection, so transferring the 'stable opinion leader' conclusion to real networks would require validating the link rule against observed follow behavior.","A natural extension would be to replace the hand-set follow and unfollow probabilities with values fitted to a real social network's follow-interaction data; if the RL bot's advantage over a random baseline disappears under fitted probabilities, the claimed gain would be exposed as a simulation artifact."],"forward_implications":["If the paper's claims are correct, researchers gain a way to run ethics-safe, topic-specific experiments on how opinions form and how influence leaders emerge, using the kialo.com-derived gender and drug datasets.","The design findings give concrete guidance for building such agents: limiting the action space and adding self-observation improve convergence, while full network observability reduces variance even when the surrounding agents behave unpredictably.","A trained influence-seeking bot in this framework could be used both to simulate how a targeted persuasion campaign spreads and to test detection methods against such campaigns, a dual-use point the paper itself raises.","Because the framework uses real posts to build the action dictionary, it can be pointed at other topics where public debate exists, and the authors also propose extending it to topics like 6G acceptance where real datasets are absent."],"supporting_citations":[{"why":"Supplies the kialo.com debate dataset and the conditional-perplexity method used to elicit each agent's opinion polarity.","marker":"Betz (2022)"},{"why":"Provides the homophily premise that users follow others with similar opinions, which Algorithm 2 and Algorithm 3 implement as the environment's link update rule.","marker":"Zhou et al. (2009)"},{"why":"Provides the Q-learning update rule that the target bot uses to select opinion categories and maximize followers.","marker":"Watkins & Dayan (1992)"},{"why":"Establishes that pretrained LLMs generate human-like text without fine-tuning, the basis for using GPT-3.5 to produce agent posts.","marker":"Brown et al. (2020)"},{"why":"Supplies the GPT-3.5 model actually used for post generation and opinion elicitation in the simulations.","marker":"OpenAI (2022)"}],"fun_headline_variants":["LLM bot uses reinforcement learning to gain 10% of followers in simulation","RL-LLM bot gains 10% follower share by constraining action space","Self-observation and limited actions let RL bot win 10% of network followers","In a synthetic network, an RL-LLM bot learns to win 10% of followers","Constrained actions and self-observation key to RL bot gaining 10% followers"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire reward the agent learns from is produced by the rule that a user follows another user when their opinion categories match over consecutive time steps, using hand-set follow and unfollow probabilities of 0.8 and 0.5.","fun_headline_variants_meta":{"raw":{"variants":["LLM bot uses reinforcement learning to gain 10% of followers in simulation","RL-LLM bot gains 10% follower share by constraining action space","Self-observation and limited actions let RL bot win 10% of network followers","In a synthetic network, an RL-LLM bot learns to win 10% of followers","Constrained actions and self-observation key to RL bot gaining 10% followers"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001029,"raw_usage":{"total_tokens":4333,"prompt_tokens":939,"completion_tokens":3394,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":555,"completion_tokens_details":{"reasoning_tokens":3285}},"tokens_in":555,"tokens_out":3394,"duration_ms":16788,"temperature":1.0,"reasoning_tokens":3285,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:59:10.181892+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same RL setup with the follow probability changed to 0.5 and the unfollow probability changed to 0.8, and with the consecutive-match requirement relaxed to a single-step match; if the learned policy and the reported 10% follower gain change sharply or vanish, the result is an artifact of the link rule rather than learned influence. A complementary check is to compare the RL bot's final follower count, across many random seeds, against a non-learning bot that picks uniformly among the same five opinion categories while using the same LLM for post generation.","supporting_citations":[{"cited_title":"& Zhang, C","cited_arxiv_id":null,"evidence_quote":"Provides the homophily premise that users follow others with similar opinions, which Algorithm 2 and Algorithm 3 implement as the environment's link update rule."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Q-learning update rule that the target bot uses to select opinion categories and maximize followers."},{"cited_title":"D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A","cited_arxiv_id":null,"evidence_quote":"Establishes that pretrained LLMs generate human-like text without fine-tuning, the basis for using GPT-3.5 to produce agent posts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the GPT-3.5 model actually used for post generation and opinion elicitation in the simulations."}],"review_version":1}