Pith. sign in

REVIEW 3 major objections 4 minor 97 references

IO Factory: Simulating AI-Enabled Influence Campaigns at Scale

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that AI-enabled influence campaigns can be simulated as traceable ten-phase lifecycles inside a controlled social platform, with every step from public action to measured belief shift preserved as inspectable evidence.

desk verdict A useful, carefully scoped simulation framework for AI-enabled influence campaigns, held back only by missing artifacts and an unvalidated LLM judge at the center of its measurement. read the letter →

arxiv 2608.10920 v1 pith:4UAWZMI2 submitted 2026-08-11 cs.AI

Lukasz Olejnik , Wenchao Dong , Jonas R. Kunst , Signe Riemer-S{o}rensen , Tobias Herb , Meeyoung Cha , Daniel Thilo Schroeder This is my paper · ORCID
classification cs.AI
keywords AI-enabledinfluenceAIswarmsmulti-agentsocialsimulationLLMagentsinformationoperationslifecycleexposuremeasurementredteamingdirectionallift
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that an AI-enabled influence campaign should be treated as a single traceable process rather than a scattering of posts, and that this process can be studied inside a controlled simulation. IO Factory represents the full campaign lifecycle in ten phases, from reconnaissance and narrative design through amplification, adaptation, and evaluation, inside a simulated social platform with up to 100,000 agents. The framework keeps public platform activity separate from measurement, records each exposure, asks a language-model judge to score relevance, stance, confidence, and persuasiveness, and then applies a deterministic update rule to simulated civilian beliefs. In matched runs, active campaigns produced movement toward the configured target on all three measured constructs, and every step from action to evidence remains inspectable. The stated purpose is reproducible red-team analysis and benchmark construction, not a claim about real-world persuasion.

What carries the argument

The load-bearing mechanism is the campaign lifecycle instantiated as ten phases (reconnaissance, narrative design, infrastructure, content production, laundering, integration, amplification, absorption, adaptation, evaluation), which gate which actors may act, what can become visible, and what evidence is recorded. On top of it runs the measurement pipeline, which ensures that content affects simulated civilian state only after it is made visible, recorded as an exposure, scored by an LLM judge on relevance, stance, confidence, and persuasiveness, and passed through the deterministic update rule $$\Delta_{i,c,e}=p_e \tau_{i,s(e)} u_i r_{i,e}\lambda_c d_{e,c}\gamma_{e,c},\qquad $x^{{\mathrm{new}}$}_{i,c}=\mathrm{clip}($x^{{\mathrm{old}}$}_{i,c}+\Delta_{i,c,e}).$$ The comparison mechanism is directional lift $L_{s,c}$, the active-run change in mean construct value minus the matched-baseline change, sign-aligned to the campaign target. Together these components keep message production, visibility, exposure, interpretation, and state change separate, so campaign volume is never conflated with exposure or effect.

What would settle it

Take a sample of exposure records from a completed run, have human annotators rate the same four dimensions the judge scores, and recompute directional lift using human ratings in the update rule; if the lifts shrink, reverse, or lose significance, the central measurement claim fails.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that a campaign lifecycle can be made into a controllable, inspectable simulation object. The reported runs show that IO Factory can execute full campaign timelines at scale and preserve an evidence path from public action and non-public coordination to exposure records, judge readings, deterministic state updates, and outcome summaries. All three primary construct endpoints moved in the target direction: directional lift of 0.132 for support for eating insects, 0.130 for trust in Russia, and sign-adjusted lift of 0.336 for lower trust in public institutions, each significant at p<0.001 across 13 matched baseline-active replicates. The paper is explicit that these are simulator-scale measurements under declared assumptions, not estimates of real-world persuasion or operational effectiveness.

Load-bearing premise

The whole measurement of belief movement depends on the language-model judge's ratings of relevance, stance, confidence, and persuasiveness; if those ratings do not meaningfully capture the intended constructs, the reported directional lift is an artifact of the configured pipeline.

Editorial extensions

If this is right

  • Researchers and red teams can design a campaign by configuring constructs, phases, actor permissions, and exposure rules, and receive a recorded run in which every outcome traces back to specific actions, exposures, and judge readings.
  • The matched baseline protocol isolates the intervention's contribution from ordinary simulated dynamics, so similar final lift can be decomposed into different exposure and update profiles rather than read as simple posting volume.
  • The same campaign model can be tested under different platform assumptions by varying graphs, feeds, or discovery rules while holding measurement fixed, supporting sensitivity analysis.
  • Full lifecycles execute at 100,000-agent scale, making population-scale campaign simulation feasible for defensive exercises and benchmark construction.
  • Results are simulator-scale under declared assumptions; external claims would require calibration of judge outputs against human annotations and sensitivity analysis across models, as the paper itself states.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: if a shared benchmark standard emerged around this lifecycle representation, different groups could compare red-team scenarios directly; the paper calls for this but does not claim to establish it.
  • Beyond the paper: the paper's distinction between human-facing and machine-facing exposure, left to future work, could turn retrieval-augmented generation or web-scale training-data poisoning into campaign vectors inside the same framework, since those are also exposure paths.
  • Beyond the paper: because the judge readings feed a deterministic update rule, swapping the LLM judge for hand-coded persuasion parameters would let IO Factory reproduce classical opinion-dynamics models, offering a way to validate the pipeline against known dynamics.
  • Beyond the paper: the unvalidated judge readings imply a concrete test; if human-annotated ratings disagree with judge readings on a fixed exposure sample, the reported lifts should be treated as pipeline behavior rather than evidence about influence.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces IO Factory, a framework for simulating AI-enabled influence campaigns as traceable lifecycle processes inside a simulated social platform. It separates a control plane, a simulation plane, and an evaluation plane; defines civilian, IO-operator, and manager actor roles; implements a ten-phase campaign lifecycle; and measures campaign influence as directional lift in configured civilian construct values. The measurement pipeline uses an LLM judge to produce structured readings (relevance, stance, confidence, persuasiveness) that feed a deterministic update rule, and the comparison protocol uses matched baseline and active runs with 13 replicates of 10,000 civilians, plus a claimed 100,000-agent validation run. The paper explicitly frames all results as simulator-scale and not as estimates of real-world persuasion.

Significance. If the framework works as described, IO Factory is a valuable contribution to red-team scenario design, benchmark construction, and sensitivity analysis for AI-enabled influence research. The strengths are the explicit separation of the platform environment from the campaign model, the provenance trail from action through exposure to judge reading and state update, and the careful matched-replicate inference with bootstrap intervals and exact sign-flip tests. The paper is also commendably explicit about its limitations, including the statement that LLM judges are not ground truth. The central architectural claim is defensible, but the empirical demonstration is currently conditional on an unvalidated measurement instrument, and the formal update rule is underspecified in one load-bearing respect. The framework's potential is substantial, but the measurement claims need further support before the results can be taken as evidence of simulated influence rather than internal pipeline consistency.

major comments (3)
  1. [Section 6, Eq. (1)] The update rule as written contains no term for judged relevance, yet the prose states that content judged irrelevant to the construct should produce no movement and that a post can contribute to the measure only after passing the update rule. Please specify how an irrelevant exposure maps to zero update, either by adding a relevance gate factor to Eq. (1) or by explicitly defining p_e to be zero for irrelevant content. As written, the formal rule is incomplete and the otherwise-clean deterministic pipeline is underspecified.
  2. [Section 7 and Section 8] The central empirical claim of measured movement in configured belief variables rests entirely on LLM judge outputs (persuasiveness, stance, confidence, relevance) that are neither human-calibrated nor tested for sensitivity across judge prompts or model choices. Section 8 acknowledges that LLM judges are not ground truth and promises future calibration, but the Section 7 lifts are presented as the main empirical results. Because the same model family generates the campaign content and judges it, positive lift is close to by-construction: active runs inject content aimed at the target constructs, and the judge reads that content as persuasive. To make the empirical demonstration informative rather than merely an internal consistency check, the authors should provide at least a sensitivity analysis over judge settings or a small human-annotated validation subset; otherwise the results show only that the pipeline moves numbers as configured.
  3. [Section 7, large-scale run] The 100,000-agent validation run is asserted without details or artifacts, and the manuscript does not state code or data availability despite claiming that IO Factory supports reproducible research. Please include an artifact availability statement with run configurations, prompts, and a minimal example, or clearly mark the absence of a public release. Without this, the scale claim and the provenance architecture cannot be verified by independent readers.
minor comments (4)
  1. [Section 7, Figure 4] The agenda-share measure is reported only for active conditions, and the text notes that baseline configurations did not run agenda accounting. Please add a sentence in the main text explaining why baseline agenda share is absent and what that implies for interpreting the agenda-share values as a campaign effect.
  2. [Table 2] The Polarization diagnostic is reported as mean +/- SD but is never defined. Please state how polarization is computed, for example as the standard deviation or variance of civilian construct values.
  3. [Section 7, bootstrap procedure] The phrase 'paired-bootstrap intervals over the 13 matched baseline-active replicates' is clear, but please report the bootstrap resampling procedure, including the number of resamples, for reproducibility.
  4. [Section 6, Eq. (1), repetition weight] The notation r_i,e is introduced as the repetition weight for repeated exposure to the same content, but the subscript e appears to refer to the exposure event rather than the content item. Please clarify whether repeated exposures to the same content by the same civilian are aggregated before computing n_i,e.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: directional lift is a declared simulator output under a configured deterministic rule, and the unvalidated LLM judge is a measurement-validity limitation explicitly acknowledged by the paper.

full rationale

The paper's central contribution is architectural traceability: a controlled simulation environment with a deterministic construct-update rule, provenance records, and matched baseline comparison. The reported directional-lift values are not framed as external predictions; Section 8 explicitly states that 'LLM judges are not ground truth' and that the results are 'simulator-scale measurements under declared assumptions.' The Section 6 update equation consumes judge readings as configured inputs, but the sign and magnitude of the lift are not fixed by construction: they depend on stochastic content generation, feed exposure, and judge outputs, and the matched-baseline subtraction could in principle yield negative lift. No fitted parameter is renamed as a prediction, and no load-bearing premise is justified exclusively by the authors' own prior work; references [1,2,3,11] appear as contextual motivation rather than as evidence for the framework's outputs. The acknowledged absence of human calibration for the LLM judge is a construct-validity limitation, not a circular derivation; likewise, the omission of an explicit relevance multiplier from the update equation is a specification gap, not circularity. Therefore the derivation chain is self-contained as a simulation methodology.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The framework introduces no new physical or conceptual entities, but it relies heavily on configured free parameters and domain assumptions. The reported directional-lift results depend on six classes of unstated parameter values and on the untested assumption that LLM judge readings are meaningful measurements of the simulated constructs.

free parameters (6)
  • construct-specific update rate lambda_c
    Configured per construct before runs; controls how much a judged exposure moves civilian construct values. Appears in the update equation in Section 6 and is not reported in the paper.
  • civilian-level update weight u_i
    Per-civilian multiplier in the update rule, listed as a mutable state field in Section 4 but no values or distributions are reported.
  • repetition decay factor rho and floor rho_min
    Configured decay parameters in the repetition weight r_i,e in Section 6; values not reported.
  • susceptibility and resilience parameters
    Mentioned as actor state fields in Section 4; determine how civilians react but values are not specified.
  • lifecycle transition thresholds
    Minimum steps and thresholds for content production, laundering, integration, adaptation in Table 1; values not reported.
  • judge prompt and model settings = Gemma 4 31B
    The LLM judge is a measurement instrument; prompt templates, temperature, and judge schema are not provided, so the measurement is not reproducible.
assumptions (5)
  • domain assumption LLM judge outputs are valid measurements of relevance, stance, confidence, and persuasiveness for the simulated constructs.
    The entire measurement pipeline in Section 6 relies on judge readings without human validation; the paper notes judges are not ground truth in Section 8.
  • domain assumption The deterministic update rule in Section 6 is an acceptable model of how exposure changes civilian beliefs.
    The rule is a modeling choice, not derived from data or theory; it determines all reported lift values.
  • domain assumption The ten-phase lifecycle and transition gates from Table 1 capture the operational structure of influence campaigns.
    The lifecycle is grounded in staged models but is instantiated with arbitrary thresholds and phase order.
  • domain assumption Matched baseline runs isolate the effect of IO operators from other simulator dynamics.
    The comparison in Section 6 assumes that the only relevant difference between active and baseline runs is the presence of the campaign actors.
  • standard math Standard arithmetic and clipping operations are correct.
    The update and lift formulas use basic arithmetic, which is not in dispute.

how reviews work

0 comments
Cite this review

Pith. "Pith review of IO Factory: Simulating AI-Enabled Influence Campaigns at Scale." pith.science (2026). https://pith.science/paper/4UAWZMI2

@misc{pith2026260810920,
  author       = {Pith},
  title        = {Pith review of: IO Factory: Simulating AI-Enabled Influence Campaigns at Scale},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4UAWZMI2}},
  note         = {Machine review of arXiv:2608.10920}
}
read the original abstract

We introduce IO Factory, an AI-driven framework for simulating information and influence campaigns as fully integrated, traceable processes. The threat of digital manipulation now extends beyond persuasive text from individual language models to AI swarms, i.e., persistent groups of coordinated agents that adapt to platform feedback and disguise organized campaigns as ordinary social interaction. Because such campaigns cannot be identified from isolated messages alone, they must be analyzed across a continuous spectrum of planning, platform action, exposure, interpretation, measurement, and adaptation. IO Factory represents this process inside a controlled simulated platform, linking actor roles, platform actions, exposure records, structured model-based evaluations, and configured changes in the simulated population. We implement the architecture and evaluate it across configurations of up to 100,000 agents. The results show that IO Factory executes campaign timelines at scale and produces inspectable evidence of exposure and measured movement in configured belief variables. By recording the actors, objectives, action constraints, exposure paths, and measurement rules used in each run, IO Factory supports reproducible research and red-team analysis of coordinated influence.

Figures

Figures reproduced from arXiv: 2608.10920 by the authors.

Figure 1
Figure 1. High-level IO Factory architecture. The run controller orchestrates experiment execution across the platform environment, actors, study models, measurement, and evidence layers. Provenance links pre￾serve the path from public action through exposure and state updates to comparison and audit records. The control plane contains the run controller, which orchestrates execution across all other components. It initialize… view at source ↗
Figure 2
Figure 2. Simulation time coordination. IO Factory advances the authoritative simulation step; campaign phases span multiple steps and change at transition checks. OASIS supplies platform-compatible timestamps but does not drive the campaign lifecycle. At each simulation step, the current campaign phase is stored as part of the run state. It gates which actor categories can be selected, which actions they may propose, and whi… view at source ↗
Figure 3
Figure 3. Mean construct value trajectories under matched baseline and active conditions. Lines average the 13 matched replicates and ribbons show bootstrap 95% intervals. Panels A and B share one active run and one scale, and panel C shows the separate decrease-target run on its own scale. ∆ is the final-step active-minus-baseline change from run start, shown with its paired-bootstrap 95% interval and exact sign-flip signifi… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Agenda share in measured civilian￾authored posts under active conditions. Panels show the two-construct run (A) and the public-institutions run (B). Lines average 13 matched replicates with bootstrap 95% ribbons and final topic-share labels. Exposure, phase, and discou…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

97 extracted references · 69 canonical work pages

  1. [1]

    How malicious AI swarms can threaten democracy,

    D. T. Schroeder, M. Cha, A. Baronchelli, N. Bostrom, N. A. Christakis, D. Garcia, A. Gold- enberg, Y. Kyrychenko, K. Leyton-Brown, N. Lutz, G. Marcus, F. Menczer, G. Pennycook, D. G. Rand, M. Ressa, F. Schweitzer, D. Song, C. Summerfield, A. Tang, J. J. Van Bavel, S. van der Linden, and J. R. Kunst, “How malicious AI swarms can threaten democracy,” Scienc...

    2026

  2. [2]

    AI propaganda factories with language models,

    L. Olejnik, “AI propaganda factories with language models,” 2025. [Online]. Available: https://arxiv.org/abs/2508.20186

    arXiv 2025

  3. [3]

    Emergent coordinated behaviors in networked LLM agents: Modeling the strategic dynamics of information operations,

    G. M. Orlando, J. Ye, V. La Gatta, M. Saeedi, V. Moscato, E. Ferrara, and L. Luceri, “Emergent coordinated behaviors in networked LLM agents: Modeling the strategic dynamics of information operations,” inProceedings of the ACM Web Conference 2026 (WWW ’26). New York, NY, USA: Association for Computing Machinery, 2026, dubai, United Arab Emirates, April 13...

    arXiv 2026

  4. [4]

    OASIS: Open agents social interaction simulations on one million agents,

    Z. Yang, Z. Zhang, Z. Zheng, Y. Jiang, Z. Gan, Z. Wang, Z. Ling, J. Chen, M. Ma, B. Dong, P. Gupta, S. Hu, Z. Yin, G. Li, X. Jia, L. Wang, B. Ghanem, H. Lu, C. Lu, W. Ouyang, Y. Qiao, P. Torr, and J. Shao, “OASIS: Open agents social interaction simulations on one million agents,” 2024. [Online]. Available: https://arxiv.org/abs/2411.11581

    arXiv 2024

  5. [5]

    Generative agents: Interactive simulacra of human behavior,

    J. S. Park, J. C. O’Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein, “Generative agents: Interactive simulacra of human behavior,” inProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, 2023

    2023

  6. [6]

    GenSim: A general social simulation platform with large language model based agents,

    J. Tang, H. Gao, X. Pan, L. Wang, H. Tan, D. Gao, Y. Chen, X. Chen, Y. Lin, Y. Li, and oth- ers, “GenSim: A general social simulation platform with large language model based agents,” inProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Associa- tion for Computational Linguistics: Human Language Technologies (System Demonstra...

    2025

  7. [7]

    Large language models empowered agent-based modeling and simulation: A survey and perspectives,

    C. Gao, X. Lan, N. Li, Y. Yuan, J. Ding, Z. Zhou, F. Xu, and Y. Li, “Large language models empowered agent-based modeling and simulation: A survey and perspectives,”Humanities and Social Sciences Communications, vol. 11, p. 1259, 2024

    2024

  8. [8]

    Out of one, many: Using language models to simulate human samples,

    L. P. Argyle, E. C. Busby, N. Fulda, J. Gubler, C. Rytting, and D. Wingate, “Out of one, many: Using language models to simulate human samples,”Political Analysis, vol. 31, no. 3, pp. 337–351, 2023. 16

    2023

Show all 97 references
  1. [9]

    Using large language models to simulate multiple humans and replicate human subject studies,

    G. V. Aher, R. I. Arriaga, and A. T. Kalai, “Using large language models to simulate multiple humans and replicate human subject studies,” inProceedings of the 40th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 202, 2023, pp....

    2023

  2. [10]

    Simulating opinion dynamics with networks of LLM-based agents,

    Y.-S. Chuang, A. Goyal, N. Harlalka, S. Suresh, R. Hawkins, S. Yang, D. Shah, J. Hu, and T. Rogers, “Simulating opinion dynamics with networks of LLM-based agents,” inFindings of the Association for Computational Linguistics: NAACL 2024, 2024, pp. 3326–3346

    2024

  3. [11]

    Puppets or partners? governing cyborg propaganda in the digital public square,

    J. R. Kunst, K. Bierwiaczonek, M. Cha, O. V. Ebrahimi, M. Fawcett-Atkinson, A. Følstad, A. Gollwitzer, N. K¨ obis, G. Marcus, J. Roozenbeek, D. T. Schroeder, J. J. Van Bavel, S. van der Linden, R. White, and L. L. Wilhelmsen, “Puppets or partners? governing cyborg propaganda i...

    2026 arXiv

  4. [12]

    AI and the future of disinformation campaigns: Part 1: The RICHDATA framework,

    Center for Security and Emerging Technology, “AI and the future of disinformation campaigns: Part 1: The RICHDATA framework,” Georgetown University Center for Security and Emerging Technology, Tech. Rep., 2021. [Online]. Available: https: //cset.georgetown.edu/publication/ai-a...

    2021

  5. [13]

    Phase-based tactical analysis of online operations,

    Carnegie Endowment for International Peace, “Phase-based tactical analysis of online operations,” 2023. [Online]. Available: https://carnegieendowment.org/research/2023/03/p hase-based-tactical-analysis-of-online-operations

    2023

  6. [14]

    DISARM framework,

    DISARM Foundation, “DISARM framework,” n.d. [Online]. Available: https://github.com /DISARMFoundation/DISARMframeworks/

  7. [15]

    Joint publication 3-13.2: Military information support operations,

    Joint Chiefs of Staff, “Joint publication 3-13.2: Military information support operations,” United States Department of Defense, Tech. Rep., n.d. [Online]. Available: https: //www.esd.whs.mil/Portals/54/Documents/FOID/Reading%20Room/Joint Staff/Military I nformation Support Op...

  8. [16]

    Microsoft digital defense report 2022: Cyber influence operations,

    Microsoft, “Microsoft digital defense report 2022: Cyber influence operations,” 2022. [Online]. Available: https://www.microsoft.com/en-gb/security/business/microsoft-digital-defense-r eport-2022-cyber-influence-operations

    2022

  9. [17]

    The rise of social bots,

    E. Ferrara, O. Varol, C. Davis, F. Menczer, and A. Flammini, “The rise of social bots,” Communications of the ACM, vol. 59, no. 7, pp. 96–104, 2016

    2016

  10. [18]

    Uncovering coordinated networks on social media: Methods and case studies,

    D. Pacheco, P.-M. Hui, C. Torres-Lugo, B. T. Truong, A. Flammini, and F. Menczer, “Uncovering coordinated networks on social media: Methods and case studies,” inProceedings of the International AAAI Conference on Web and Social Media, vol. 15, no. 1, 2021, pp. 455–466. [Online...

    2021

  11. [19]

    Temporal dynamics of coordinated online behavior: Stability, archetypes, and influence,

    S. Tardelli, L. Nizzoli, M. Tesconi, M. Conti, P. Nakov, G. Da San Martino, and S. Cresci, “Temporal dynamics of coordinated online behavior: Stability, archetypes, and influence,” Proceedings of the National Academy of Sciences, vol. 121, no. 20, p. e2307038121, 2024

    2024

  12. [21]

    Exposing secondary infektion,

    Graphika, “Exposing secondary infektion,” Graphika, Tech. Rep., 2020. [Online]. Available: https://www.graphika.com/reports/exposing-secondary-infektion/

    2020

  13. [22]

    Information laundering in the nordic-baltic region,

    NATO Strategic Communications Centre of Excellence, “Information laundering in the nordic-baltic region,” NATO Strategic Communications Centre of Excellence, Tech. Rep., n.d. [Online]. Available: https://stratcomcoe.org/publications/information-laundering-in-the -nordic-baltic...

  14. [23]

    Adding a “d

    Brookings Institution, “Adding a “d” to the ABC disinformation framework,” n.d. [Online]. Available: https://www.brookings.edu/articles/adding-a-d-to-the-abc-disinformation-frame work/

  15. [24]

    Red teaming language models with language models,

    E. Perez, S. Huang, F. Song, T. Cai, R. Ring, J. Aslanides, A. Glaese, N. McAleese, and G. Irving, “Red teaming language models with language models,” inProceedings of EMNLP 2022, 2022, pp. 3419–3448

    2022

  16. [25]

    Cyber ranges and security testbeds: Scenarios, functions, tools and architecture,

    M. M. Yamin, B. Katt, and V. Gkioulos, “Cyber ranges and security testbeds: Scenarios, functions, tools and architecture,”Computers & Security, vol. 88, p. 101636, 2020

    2020

  17. [26]

    The malicious use of artificial intelligence: Forecasting, prevention, and mitigation,

    M. Brundage, S. Avin, J. Clark, H. Toner, P. Eckersley, B. Garfinkel, A. Dafoe, P. Scharre, T. Zeitzoff, and B. Filar, “The malicious use of artificial intelligence: Forecasting, prevention, and mitigation,” Tech. Rep., 2018. [Online]. Available: https://arxiv.org/abs/1802.07228

    2018 arXiv

  18. [27]

    Release strategies and the social impacts of language models,

    I. Solaiman, M. Brundage, J. Clark, A. Askell, A. Herbert-Voss, J. Wu, A. Radford, G. Krueger, J. W. Kim, and S. Kreps, “Release strategies and the social impacts of language models,” Tech. Rep., 2019. [Online]. Available: https://arxiv.org/abs/1908.09203

    2019 arXiv

  19. [28]

    The diamond model for influence operations analysis,

    Recorded Future, “The diamond model for influence operations analysis,” Recorded Future, Tech. Rep., n.d. [Online]. Available: https://assets.recordedfuture.com/Whitepapers/diamon d-model-influence-operations-analysis.pdf

  20. [29]

    Allied joint doctrine for information operations,

    NATO Standardization Office, “Allied joint doctrine for information operations,” North Atlantic Treaty Organization, Tech. Rep., 2023. [Online]. Available: https://assets.publishin g.service.gov.uk/media/650c03bf52e73c000d9425bb/AJP 10 1 Info Ops UK web.pdf

    2023

  21. [30]

    Adversarial misuse of generative AI,

    Google Threat Intelligence Group, “Adversarial misuse of generative AI,” Jan. 2025. [Online]. Available: https://cloud.google.com/blog/topics/threat-intelligence/adversarial-misuse-gener ative-ai

    2025

  22. [31]

    G. S. Jowett and V. O’Donnell,Propaganda & Persuasion, 7th ed. SAGE, 2018. [Online]. Available: https://books.google.com/books/about/Propaganda Persuasion.html?id=v wYt AEACAAJ

    2018

  23. [32]

    Information disorder: Toward an interdisciplinary framework for research and policy making,

    C. Wardle and H. Derakhshan, “Information disorder: Toward an interdisciplinary framework for research and policy making,” Council of Europe, Tech. Rep., 2017. [Online]. Available: https://edoc.coe.int/en/media/7495-information-disorder-toward-an-interdisciplinary-frame work-f...

    2017

  24. [33]

    Political communication, computational propaganda, and autonomous agents: Introduction,

    S. C. Woolley and P. N. Howard, “Political communication, computational propaganda, and autonomous agents: Introduction,”International Journal of Communication, vol. 10, pp. 4882–4890, 2016. [Online]. Available: https://ijoc.org/index.php/ijoc/article/view/6298 18

    2016

  25. [34]

    The global disinformation order: 2019 global inventory of organised social media manipulation,

    S. Bradshaw and P. N. Howard, “The global disinformation order: 2019 global inventory of organised social media manipulation,” Oxford Internet Institute, Tech. Rep., 2019. [Online]. Available: https://www.oii.ox.ac.uk/news-events/reports/the-global-disinformation-order-2 019-g...

    2019

  26. [35]

    Computational experiments for complex social systems-part i: The customization of computational model,

    X. Xue, F. Chen, D. Zhou, X. Wang, M. Lu, and F.-Y. Wang, “Computational experiments for complex social systems-part i: The customization of computational model,”IEEE Transactions on Computational Social Systems, vol. 9, no. 5, pp. 1330–1345, 2022

    2022

  27. [36]

    A standard protocol for describing individual-based and agent-based models,

    V. Grimm, U. Berger, D. L. DeAngelis, J. G. Polhill, J. Giske, and S. F. Railsback, “A standard protocol for describing individual-based and agent-based models,”Ecological Modelling, vol. 198, no. 1-2, pp. 115–126, 2006

    2006

  28. [37]

    Verification and validation of simulation models,

    R. G. Sargent, “Verification and validation of simulation models,”Journal of Simulation, vol. 7, no. 1, pp. 12–24, 2013

    2013

  29. [38]

    Exposure to ideologically diverse news and opinion on facebook,

    E. Bakshy, S. Messing, and L. A. Adamic, “Exposure to ideologically diverse news and opinion on facebook,”Science, vol. 348, no. 6239, pp. 1130–1132, 2015

    2015

  30. [39]

    Exposure to the russian internet research agency foreign influence campaign on twitter in the 2016 US election and its relationship to attitudes and voting behavior,

    G. Eady, T. Paskhalis, J. Zilinsky, R. Bonneau, J. Nagler, and J. A. Tucker, “Exposure to the russian internet research agency foreign influence campaign on twitter in the 2016 US election and its relationship to attitudes and voting behavior,”Nature Communications, vol. 14, p...

    2016

  31. [40]

    G-eval: NLG evaluation using GPT-4 with better human alignment,

    Y. Liu, D. Iter, Y. Xu, S. Wang, R. Xu, and C. Zhu, “G-eval: NLG evaluation using GPT-4 with better human alignment,” inProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2023, pp. 2511–2522. [Onli...

    2023

  32. [41]

    Judging LLM-as-a-judge with MT-bench and chatbot arena,

    L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica, “Judging LLM-as-a-judge with MT-bench and chatbot arena,”arXiv preprint arXiv:2306.05685, 2023

    2023 arXiv

  33. [42]

    Threat report: The state of influence operations 2017-2020,

    Meta, “Threat report: The state of influence operations 2017-2020,” Meta, Tech. Rep., 2021. [Online]. Available: https://about.fb.com/wp-content/uploads/2021/05/IO-Threat-Repor t-May-20-2021.pdf

    2017

  34. [43]

    Reports on foreign information manipulation and interference threats,

    European External Action Service, “Reports on foreign information manipulation and interference threats,” European External Action Service, Tech. Rep., 2025. [Online]. Available: https://www.eeas.europa.eu/eeas/4th-eeas-annual-report-foreign-information-m anipulation-and-inter...

    2025

  35. [44]

    Information integrity and countering foreign information manipulation and interference (FIMI),

    ——, “Information integrity and countering foreign information manipulation and interference (FIMI),” n.d. [Online]. Available: https://www.eeas.europa.eu/eeas/information-integrity-a nd-countering-foreign-information-manipulation-interference-fimi en

  36. [45]

    New efforts to disrupt DRAGONBRIDGE spam activity,

    Google Threat Analysis Group, “New efforts to disrupt DRAGONBRIDGE spam activity,”

  37. [46]

    Seeing through a GLASSBRIDGE: Understanding the digital marketing ecosystem spreading pro-PRC influence operations,

    Google Threat Intelligence Group, “Seeing through a GLASSBRIDGE: Understanding the digital marketing ecosystem spreading pro-PRC influence operations,” 2024. [Online]. 19 Available: https://cloud.google.com/blog/topics/threat-intelligence/glassbridge-pro-prc-influ ence-operations

    2024

  38. [47]

    Available: https://blog.google/threat-analysis-group/google-disrupted-drago nbridge-activity-q1-2024/

    [Online]. Available: https://blog.google/threat-analysis-group/google-disrupted-drago nbridge-activity-q1-2024/

    2024

  39. [48]

    Justice department leads efforts to disrupt covert russian government-operated social media bot farm,

    United States Department of Justice, “Justice department leads efforts to disrupt covert russian government-operated social media bot farm,” 2024. [Online]. Available: https://www.justice.gov/archives/opa/pr/justice-department-leads-efforts-among-federal-int ernational-and-pri...

    2024

  40. [49]

    Ghostwriter update: Cyber espionage group UNC1151 likely conducts ghostwriter influence activity,

    Mandiant, “Ghostwriter update: Cyber espionage group UNC1151 likely conducts ghostwriter influence activity,” 2021. [Online]. Available: https://cloud.google.com/blog/topics/threat-i ntelligence/espionage-group-unc1151-likely-conducts-ghostwriter-influence-activity

    2021

  41. [50]

    Master of puppets: Uncovering the DoppelG¨ anger pro-russian influence campaign,

    Sekoia.io, “Master of puppets: Uncovering the DoppelG¨ anger pro-russian influence campaign,”

  42. [51]

    Justice department disrupts covert russian government-sponsored foreign malign influence operation targeting audiences in the united states and elsewhere,

    ——, “Justice department disrupts covert russian government-sponsored foreign malign influence operation targeting audiences in the united states and elsewhere,” 2024. [Online]. Available: https://www.justice.gov/archives/opa/pr/justice-department-disrupts-covert-rus sian-gover...

    2024

  43. [52]

    The F AIR guiding principles for scientific data management and stewardship,

    M. D. Wilkinson, M. Dumontier, I. J. Aalbersberg, G. Appleton, M. Axton, and A. Baak, “The F AIR guiding principles for scientific data management and stewardship,”Scientific Data, vol. 3, p. 160018, 2016

    2016

  44. [53]

    Available: https://blog.sekoia.io/master-of-puppets-uncovering-the-doppelg anger-pro-russian-influence-campaign/

    [Online]. Available: https://blog.sekoia.io/master-of-puppets-uncovering-the-doppelg anger-pro-russian-influence-campaign/

  45. [54]

    National Academies Press, 2019

    National Academies of Sciences, Engineering, and Medicine,Reproducibility and Replicability in Science. National Academies Press, 2019

    2019

  46. [55]

    Disinformation as collaborative work: Surfacing the participatory nature of strategic information operations,

    K. Starbird, A. Arif, and T. Wilson, “Disinformation as collaborative work: Surfacing the participatory nature of strategic information operations,”Proceedings of the ACM on Human- Computer Interaction, vol. 3, no. CSCW, pp. 1–26, 2019

    2019

  47. [56]

    Benkler, R

    Y. Benkler, R. Faris, and H. Roberts,Network Propaganda: Manipulation, Disinformation, and Radicalization in American Politics. Oxford University Press, 2018

    2018

  48. [57]

    The russian “firehose of falsehood

    C. Paul and M. Matthews, “The russian “firehose of falsehood” propaganda model: Why it might work and options to counter it,” RAND Corporation, Tech. Rep., 2016. [Online]. Available: https://www.rand.org/pubs/perspectives/PE198.html

    2016

  49. [58]

    Intelligence-driven computer network defense informed by analysis of adversary campaigns and intrusion kill chains,

    E. M. Hutchins, M. J. Cloppert, and R. M. Amin, “Intelligence-driven computer network defense informed by analysis of adversary campaigns and intrusion kill chains,” Lockheed Martin, Tech. Rep., 2011. [Online]. Available: https://www.lockheedmartin.com/content/d am/lockheed-ma...

    2011

  50. [59]

    Disinformation’s spread: Bots, trolls and all of us,

    K. Starbird, “Disinformation’s spread: Bots, trolls and all of us,”Nature, vol. 571, p. 449, 2019

    2019

  51. [60]

    The breakout scale: Measuring the impact of influence operations,

    B. Nimmo, “The breakout scale: Measuring the impact of influence operations,” Brookings Institution, Tech. Rep., 2020. [Online]. Available: https://www.brookings.edu/articles/the-b reakout-scale-measuring-the-impact-of-influence-operations/

    2020

  52. [61]

    The spread of true and false news online,

    S. Vosoughi, D. Roy, and S. Aral, “The spread of true and false news online,”Science, vol. 359, no. 6380, pp. 1146–1151, 2018

    2018

  53. [62]

    Toward an information operations kill chain,

    B. Schneier, “Toward an information operations kill chain,” 2019. [Online]. Available: https://www.schneier.com/essays/archives/2019/04/toward an informatio.html 20

    2019

  54. [63]

    The role of social networks in information diffusion,

    E. Bakshy, I. Rosenn, C. Marlow, and L. Adamic, “The role of social networks in information diffusion,” inProceedings of the 21st International Conference on World Wide Web, 2012, pp. 519–528

    2012

  55. [64]

    The spread of behavior in an online social network experiment,

    D. Centola, “The spread of behavior in an online social network experiment,”Science, vol. 329, no. 5996, pp. 1194–1197, 2010

    2010

  56. [65]

    Fake news on twitter during the 2016 U.S. presidential election,

    N. Grinberg, K. Joseph, L. Friedland, B. Swire-Thompson, and D. Lazer, “Fake news on twitter during the 2016 U.S. presidential election,”Science, vol. 363, no. 6425, pp. 374–378, 2019

    2016

  57. [66]

    A 61-million-person experiment in social influence and political mobilization,

    R. M. Bond, C. J. Fariss, J. J. Jones, A. D. I. Kramer, C. Marlow, J. E. Settle, and J. H. Fowler, “A 61-million-person experiment in social influence and political mobilization,”Nature, vol. 489, pp. 295–298, 2012

    2012

  58. [67]

    Exposure to opposing views on social media can increase political polarization,

    C. A. Bail, L. P. Argyle, T. W. Brown, J. P. Bumpus, H. Chen, M. B. F. Hunzaker, J. Lee, M. Mann, F. Merhout, and A. Volfovsky, “Exposure to opposing views on social media can increase political polarization,”Proceedings of the National Academy of Sciences, vol. 115, no. 37, p...

    2018

  59. [68]

    How do social media feed algorithms affect attitudes and behavior in an election campaign?

    A. M. Guess, N. Malhotra, J. Pan, P. Barber´ a, H. Allcott, and T. Brown, “How do social media feed algorithms affect attitudes and behavior in an election campaign?”Science, vol. 381, no. 6656, pp. 398–404, 2023

    2023

  60. [69]

    Reshares on social media amplify political news but do not detectably affect beliefs or opinions,

    ——, “Reshares on social media amplify political news but do not detectably affect beliefs or opinions,”Science, vol. 381, no. 6656, pp. 404–408, 2023

    2023

  61. [70]

    Beyond the headlines: On the efficacy and effectiveness of misinformation interventions,

    J. Roozenbeek, M. Remshard, and Y. Kyrychenko, “Beyond the headlines: On the efficacy and effectiveness of misinformation interventions,”advances.in/psychology, vol. 2, no. 1, p. e24569, 2024. [Online]. Available: https://advances.in/psychology/10.56296/aip00019/

    2024 doi

  62. [71]

    Less than you think: Prevalence and predictors of fake news dissemination on facebook,

    A. Guess, J. Nagler, and J. Tucker, “Less than you think: Prevalence and predictors of fake news dissemination on facebook,”Science Advances, vol. 5, no. 1, p. eaau4586, 2019

    2019

  63. [72]

    Misinformation and its correction: Continued influence and successful debiasing,

    S. Lewandowsky, U. K. H. Ecker, C. M. Seifert, N. Schwarz, and J. Cook, “Misinformation and its correction: Continued influence and successful debiasing,”Psychological Science in the Public Interest, vol. 13, no. 3, pp. 106–131, 2012

    2012

  64. [73]

    The truth about the truth: A meta-analytic review of the truth effect,

    A. Dechˆ ene, C. Stahl, J. Hansen, and M. W¨ anke, “The truth about the truth: A meta-analytic review of the truth effect,”Personality and Social Psychology Review, vol. 14, no. 2, pp. 238– 257, 2010

    2010

  65. [74]

    The psychological drivers of misinformation belief and its resistance to correction,

    U. K. H. Ecker, S. Lewandowsky, J. Cook, P. Schmid, L. K. Fazio, N. Brashier, P. Kendeou, E. K. Vraga, and M. A. Amazeen, “The psychological drivers of misinformation belief and its resistance to correction,”Nature Reviews Psychology, vol. 1, pp. 13–29, 2022

    2022

  66. [75]

    The psychology of fake news,

    G. Pennycook and D. G. Rand, “The psychology of fake news,”Trends in Cognitive Sciences, vol. 25, no. 5, pp. 388–402, 2021

    2021

  67. [76]

    Frequency and the conference of referential validity,

    L. Hasher, D. Goldstein, and T. Toppino, “Frequency and the conference of referential validity,” Journal of Verbal Learning and Verbal Behavior, vol. 16, no. 1, pp. 107–112, 1977

    1977

  68. [77]

    Knowledge does not protect against illusory truth,

    L. K. Fazio, N. M. Brashier, B. K. Payne, and E. J. Marsh, “Knowledge does not protect against illusory truth,”Journal of Experimental Psychology: General, vol. 144, no. 5, pp. 993–1002, 2015. 21

    2015

  69. [78]

    Repetition increases perceived truth equally for plausible and implausible statements,

    L. K. Fazio, D. G. Rand, and G. Pennycook, “Repetition increases perceived truth equally for plausible and implausible statements,”Psychonomic Bulletin & Review, vol. 26, no. 5, pp. 1705–1710, 2019

    2019

  70. [79]

    Computational agent-based models in opinion dynamics: A survey on social simulations and empirical studies,

    Y.-S. Chuang and T. T. Rogers, “Computational agent-based models in opinion dynamics: A survey on social simulations and empirical studies,”arXiv preprint arXiv:2306.03446, 2023. [Online]. Available: https://arxiv.org/abs/2306.03446

    2023 arXiv

  71. [80]

    The effects of repetition frequency on the illusory truth effect,

    A. Hassan and S. J. Barber, “The effects of repetition frequency on the illusory truth effect,” Cognitive Research: Principles and Implications, vol. 6, no. 1, p. 38, 2021

    2021

  72. [81]

    The illusory truth effect: A review of how repetition increases belief in misinformation,

    J. Udry and S. J. Barber, “The illusory truth effect: A review of how repetition increases belief in misinformation,”Current Opinion in Psychology, vol. 56, p. 101736, 2024

    2024

  73. [82]

    Heuristic versus systematic information processing and the use of source versus message cues in persuasion,

    S. Chaiken, “Heuristic versus systematic information processing and the use of source versus message cues in persuasion,”Journal of Personality and Social Psychology, vol. 39, no. 5, pp. 752–766, 1980

    1980

  74. [83]

    Measurement schmeasurement: Questionable measurement prac- tices and how to avoid them,

    J. K. Flake and E. I. Fried, “Measurement schmeasurement: Questionable measurement prac- tices and how to avoid them,”Advances in Methods and Practices in Psychological Science, vol. 3, no. 4, pp. 456–465, 2020

    2020

  75. [84]

    The elaboration likelihood model of persuasion,

    R. E. Petty and J. T. Cacioppo, “The elaboration likelihood model of persuasion,” inAdvances in Experimental Social Psychology, L. Berkowitz, Ed. Academic Press, 1986, vol. 19, pp. 123– 205

    1986

  76. [85]

    Datasheets for datasets,

    T. Gebru, J. Morgenstern, B. Vecchione, J. W. Vaughan, H. Wallach, H. Daum´ e III, and K. Crawford, “Datasheets for datasets,”Communications of the ACM, vol. 64, no. 12, pp. 86–92, 2021

    2021

  77. [86]

    Heuristic processing can bias systematic processing: Effects of source credibility, argument ambiguity, and task importance on attitude judgment,

    S. Chaiken and D. Maheswaran, “Heuristic processing can bias systematic processing: Effects of source credibility, argument ambiguity, and task importance on attitude judgment,”Journal of Personality and Social Psychology, vol. 66, no. 3, pp. 460–473, 1994

    1994

  78. [87]

    Bayesian methods for media mix modeling with carryover and shape effects,

    Y. Jin, Y. Wang, Y. Sun, D. Chan, and J. Koehler, “Bayesian methods for media mix modeling with carryover and shape effects,” Google Inc., Tech. Rep., 2017. [Online]. Available: https://research.google/pubs/bayesian-methods-for-media-mix-modeling-with-carryover-and -shape-effects/

    2017

  79. [88]

    Answering the call for a standard reliability measure for coding data,

    A. F. Hayes and K. Krippendorff, “Answering the call for a standard reliability measure for coding data,”Communication Methods and Measures, vol. 1, no. 1, pp. 77–89, 2007. 22

    2007

  80. [89]

    Model cards for model reporting,

    M. Mitchell, S. Wu, A. Zaldivar, P. Barnes, L. Vasserman, B. Hutchinson, E. Spitzer, I. D. Raji, and T. Gebru, “Model cards for model reporting,” inProceedings of the Conference on Fairness, Accountability, and Transparency, 2019, pp. 220–229

    2019

  81. [90]

    Closing the AI accountability gap: Defining an end-to-end frame- work for internal algorithmic auditing,

    I. D. Raji, A. Smart, R. N. White, M. Mitchell, T. Gebru, B. Hutchinson, J. Smith-Loud, D. Theron, and P. Barnes, “Closing the AI accountability gap: Defining an end-to-end frame- work for internal algorithmic auditing,” inProceedings of the ACM Conference on Fairness, Account...

    2020

  82. [91]

    Social influence and opinions,

    N. E. Friedkin and E. C. Johnsen, “Social influence and opinions,”The Journal of Mathematical Sociology, vol. 15, no. 3-4, pp. 193–206, 1990

    1990

  83. [92]

    What will it take to fix benchmarking in natural language understanding?

    S. R. Bowman and G. E. Dahl, “What will it take to fix benchmarking in natural language understanding?” inProceedings of NAACL-HLT 2021, 2021, pp. 4843–4855

    2021

  84. [93]

    Reaching a consensus,

    M. H. DeGroot, “Reaching a consensus,”Journal of the American Statistical Association, vol. 69, no. 345, pp. 118–121, 1974

    1974

  85. [94]

    Poisoning retrieval corpora by injecting ad- versarial passages,

    Z. Zhong, Z. Huang, A. Wettig, and D. Chen, “Poisoning retrieval corpora by injecting ad- versarial passages,” inProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, pp. 13764–13775

    2023

  86. [95]

    Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning,

    G. Pennycook and D. G. Rand, “Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning,”Cognition, vol. 188, pp. 39–50, 2019

    2019

  87. [96]

    The agenda-setting function of mass media,

    M. E. McCombs and D. L. Shaw, “The agenda-setting function of mass media,”Public Opinion Quarterly, vol. 36, no. 2, pp. 176–187, 1972

    1972

  88. [98]

    Poisoning web-scale training datasets is practical,

    N. Carlini, M. Jagielski, C. A. Choquette-Choo, D. Paleka, W. Pearce, H. Anderson, A. Terzis, K. Thomas, and F. Tram` er, “Poisoning web-scale training datasets is practical,” in2024 IEEE Symposium on Security and Privacy (SP), 2024, pp. 407–425. 23

    2024

  89. [2024]

    Available: https://arxiv.org/abs/2408.01257 17

    [Online]. Available: https://arxiv.org/abs/2408.01257 17

    arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.