REVIEW 3 major objections 4 minor 2 cited by
Revisiting Rogers' Paradox in the Context of Human-AI Interaction
T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read Mirror-AI offers no long-term boost to collective knowledge.
desk verdict The headline result is the classic Rogers paradox wearing an AI costume — the AI is defined as a mirror of the population mean, so no-improvement follows by construction — but the extensions on critical learning, update schedules, and deskilling are genuinely new and worth a referee's time. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is an extended Rogers-style agent-based network in which each of 1000 agents can learn individually, copy a random human, or copy an abstract AI. The load-bearing object is the AI's adaptation rule: $p_{\mathrm{AI}}^{\mathrm{OK}} := q^{\mathrm{OK}}$, meaning the AI's probability of being adapted at the next step equals the population's mean adaptation from the previous step. This makes the AI a perfect, immediate aggregator of the population's knowledge, i.e., pure social learning from everyone at once. A second mechanism is the negative-feedback parameter $\kappa$: agents who learn from the AI have their individual-learning success multiplied by $\kappa$, starting at 1 and decaying by a factor of 0.9 each time they use the AI, which models deskilling.
What would settle it
Run the same 1000-agent network but give the AI a training lag (its adaptation level reflecting the population several timesteps earlier), aggregate a non-representative subset of the population instead of the mean, or let it explore with moderate success; if any of these changes pushes the equilibrium population world understanding above the individual-learning-only baseline of about 0.58, the central claim that a population-mirroring AI cannot help would be shown to depend on the perfect-aggregation idealization. The paper's own Figure 6 provides a partial check: an AI with high individual-learning success clearly raises collective understanding.
Extended reading notes
Core claim
The central claim is an 'AI Rogers' Paradox': the widespread availability of AI systems that are cheap, reliable, and trained on all human data may not, on their own, improve the population's collective world model in the long run. In the paper's simulation, a zero-cost AI node whose adaptation level is set to the mean adaptation of the population at the previous timestep behaves like another social-learning channel; the equilibrium population fitness stays at the same value as when only individual learning is available. The finding holds even when the AI is made cheaper relative to human social learning or more reliable in transmitting the population's behavior. The paper also establishes that the paradox can be broken by giving the AI its own individual learning (exploration) when that learning is successful, and that the collective outcome depends on how humans and model builders choose to interact with and update the AI.
Load-bearing premise
The whole no-benefit result rests on the AI being a perfect and immediate aggregator of the population's current knowledge, with its adaptation level each step set to the population mean of the previous step; if real AI is stale, biased, or capable of independent exploration, the conclusion does not follow.
Editorial extensions
If this is right
- An AI that merely mirrors the population will not dissolve Rogers' Paradox; long-run collective world understanding stays at the individual-learning-only equilibrium.
- Critical social learning, overriding the AI's output and switching to individual learning when social learning fails, raises collective world understanding and also reduces reliance on the AI.
- An AI that performs its own (individual) exploration can substantially improve collective understanding when its success rate is high and cost low, but can harm collective understanding when it frequently learns badly.
- Too-infrequent AI updates degrade the equilibrium, with stronger effects in a faster-changing world; beyond a saturation point, more frequent updates add little.
- When learning from the AI weakens the learner's own individual-learning ability, the collective equilibrium can drop below baseline unless humans retain a more expensive human social alternative.
Reading between the lines
- An implied testable extension: if the AI aggregates a biased subset of the population rather than the true mean, the equilibrium should shift; the paper's framing suggests equity of data inclusion is not just a fairness issue but a determinant of collective learning outcomes.
- The negative-feedback result points to a testable empirical prediction: populations that lean heavily on AI assistance for routine problem-solving may show a measurable decline in unaided problem-solving skill over time, and this decline could feed back into the quality of the data the AI learns from.
- The update-saturation result implies a practical cost-saving guideline for model builders: once the environment-change rate is known, there is an update frequency beyond which additional updates do not improve collective outcomes, so resources can be saved without harming population-level understanding.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper extends Alan Rogers' classic model of cultural learning to a setting in which a single abstract AI system is added to a population of human learners. The AI learns socially from the entire population, and humans may learn individually, from other humans, or from the AI. The authors report that this addition does not raise the equilibrium level of collective world understanding above the individual-learning baseline, which they call an 'AI Rogers' Paradox'. They then simulate several interventions—critical social learning, the option to override AI output, variable AI update schedules, AI individual learning, and a negative-feedback deskilling mechanism—and find that critical appraisal and AI individual learning can improve the equilibrium, while pure reliance on a population-mirroring AI cannot.
Significance. If the central result were a genuinely new phenomenon, the paper would be of broad interest to cultural evolution, AI-assisted decision-making, and human-AI interaction research. The paper has some genuine strengths: it re-instantiates the classic Rogers baseline and the Enquist critical-social-learning framework, it presents analytical derivations for the baseline human-only case in the supplement, it commits to releasing simulation code, and Section 5.6 explicitly acknowledges limitations of the aggregate fitness notion and of the population-mean learning assumption. However, the headline 'AI Rogers' Paradox' is, on close reading, a direct corollary of the defining assumption p^OK_AI := q^OK, rather than a new equilibrium phenomenon. The paper's own Figure 6 shows that an AI with independent individual learning can substantially improve collective understanding, and Section 5.6 concedes that real systems learn from biased data rather than from the population mean. The contribution is therefore best understood as a conditional model exploration, not as a general statement about AIs trained on human data.
major comments (3)
- [§2.2 and Supplement, 'Introducing AI to the Network'] The central no-benefit result is forced by the model definition rather than derived or discovered. The supplement states that the AI's adaptation level is set to the mean adaptation status of the population, i.e., p^OK_AI := q^OK. Under this rule the AI is a zero-lag, unbiased aggregator of the population's previous-timestep knowledge, so it carries no information beyond what is already contained in the population mean. The invariance of the equilibrium at 0.58 is then a direct corollary of the classic Rogers/Enquist result, not a new 'AI Rogers' Paradox'. The supplement derives Eqs. 4-5 only for human social learners and gives no analogous closed-form derivation for Eq. 3; without that derivation the paper cannot separate a genuinely novel network effect from a relabeling of the existing equilibrium.
- [§2.2 and Supplement, timing of AI update] The timing of the AI update is under-specified in a way that matters for the claimed result. The text says the AI learns from the entire population at the end of each timestep and then humans learn from the AI, but it does not state whether the environment change at the start of the next timestep is applied to the AI's stored knowledge. In the human social-learning channel, §2.1 explicitly discounts the teacher's knowledge by (1-u), giving p^{OK→OK}_s = 0.99. The AI rule p^OK_AI := q^OK applies no such discount, so the AI is artificially more reliable than a human teacher. The simulation code and the analytic statement should specify whether the AI's snapshot is taken before or after the environment-change step; if the snapshot is taken before, the correct rule would be p^OK_AI := (1-u) q^OK, which would require re-running the simulations and re-deriving the claimed invariance.
- [§3.2.2, Figure 6, and §5.6] The paper's own results undermine the generality of the headline claim. Figure 6 shows that when the AI can learn individually with low cost and high success, equilibrium collective understanding rises from 0.58 to about 0.93 in the baseline condition and to about 0.89 in the critical-social-learning condition. Section 5.6 similarly concedes that real AI systems learn from biased or non-representative data rather than from the population mean. Therefore the abstract's statement that cheap AI systems trained on all human data 'may not, on its own in the long-term, improve our collective world model' is not a robust property of AI-trained-on-human-data; it is a conditional statement about a perfect population-mean mirror. The abstract, introduction, and conclusion should present the result as conditional on this mirror assumption, with the same emphasis given to the counterexample in Figure 6.
minor comments (4)
- [§5.6 and §6] There are several typos: 'fan intuition nad further thinking' should be 'and', and 'demand work work across' should be 'demand work across'.
- [§3.2.1 and Supplement, 'How Often Should an AI System Update Its Model of the World?'] The notation is confusing and inconsistent: the environment change probability is called u in §2.1, but the supplement reuses u for the AI's update probability, and the main text presents the update rule as p^{x→qOK}_AI := 1-cλs and p^{x→x}_AI := cλs, where cλs was introduced as a cost. Please use distinct symbols for update probability, update cost, and environment change rate, and give a clean equation for the AI's state after an update.
- [Figure 7 and §4] The caption says 'Learners start in the low equilibrium but phase out the AI after a period of time' without identifying which panel (left, center, or right) this refers to, and the text does not clearly explain how the strategy-replication dynamics implement this phase-out; please clarify.
- [References] The citation 'N. S and D. N. S. (2021)' is incomplete and nonstandard; please replace it with the full author or institutional name and report number.
Circularity Check
The headline 'AI Rogers' Paradox' is fixed by the definition p^OK_AI := q^OK: the AI is a zero-lag mirror of the population mean, so the invariance of the equilibrium is an algebraic consequence, not an independent finding.
-
self definitional
[Section 2.2 and Eq. (3); Supplement 'Introducing AI to the Network']
"succeeds with probability equal to adaptation level of the AI, pOK AI := qOK ... At the end of each timestep, the AI system learns socially from the entire population and matches the corresponding probability distribution of strategies (i.e., the AI's adaptation level is set to the mean adaptation status of the population). Running these extended simulations we again find that the average population fitness does not increase relative to the individual learning-only case."
The AI's success probability is, by definition, the current population mean q^OK. Substituting p^OK_AI=E[q^OK] into the paper's own equilibrium expression Eq. (3) makes the AI term proportional to E[q^OK] itself; the fixed point is then the individual-learning value regardless of what fraction learns from the AI (any channel whose output is the population mean carries no information beyond that mean). The central 'AI Rogers' Paradox' conclusion therefore follows immediately from the assignment p^OK_AI := q^OK rather than from the dynamics simulated in Figure 3.
-
self definitional
[Section 3.1.1; Supplement Figure 10]
"when we implement such a strategy – by making the AI system unavailable when the expected adaptation value of learning from it is lower than the expected adaptation value of learning individually (E[pAI ]OK < E[pOK i ]) – into our network model, we find that the population equilibria of world understanding do not change."
Because p^OK_AI = q^OK and the model's baseline equilibrium is q^OK=0.58 while p^OK_i=0.627, the condition E[p_AI]<E[p_i] is always true; the 'critical' strategy simply never activates the AI, reducing the network to the individual-only case. The null result is the same definitional identity restated as an intervention outcome, not an independent discovery about appraisal-based engagement with AI.
full rationale
The paper is self-contained and its base Rogers simulation is a legitimate replication; there is no load-bearing self-citation and the strategy simulations (update schedules, override/critical learning implementation, negative feedback, AI individual learning) are independently coded. However, the central claim of Section 2.2 is definitionally forced: the AI agent is constructed so that its adaptation level equals the population mean, p^OK_AI := q^OK. Under that construction, adding the AI as a learning channel cannot change the population mean at equilibrium by any mechanism other than relabeling; the simulation in Figure 3 is a consistency check, not evidence for the substantive claim that AIs trained on all human data will fail to improve collective understanding. The paper's own Figure 6 and Section 5.6 concede that an AI that individually learns, or that learns from biased rather than mean data, changes or invalidates the result. The no-benefit conclusion is therefore a conditional theorem about a perfect mirror, and the headline paradox is the input assumption wearing the guise of an output.
Assumptions & free parameters
free parameters (9)
- environment change rate u =
0.01 (baseline); 0.1, 0.5 (sweeps)
- individual learning success probability z_i =
0.66
- individual learning cost c_i =
0.05
- survival probabilities sOK / sNotOK =
0.93 / 0.85
- social learning cost c_s =
0
- mutation rate for strategy inheritance =
0.005
- AI update probability =
swept in Fig. 5 (label conflated with u in the text)
- AI individual learning cost and success (c_lambda_i, z_AI) =
swept over 0-1 grid in Fig. 6
- negative feedback decay factor kappa =
0.9 per AI-learning event
assumptions (6)
- domain assumption Rogers' (1988) population dynamics: agents survive proportionally to fitness and are replenished, with learned behaviors and strategies inherited with mutation.
- domain assumption The world has a single optimal 'OK' behavior that changes with probability u each timestep, and social learning is one timestep delayed.
- ad hoc to paper AI adaptation level equals the population mean adaptation at the previous timestep (p^OK_AI := q^OK).
- domain assumption Critical social learning success is p^OK_cs = 1 - (1-p^OK_s)(1-p^OK_i); agents override the AI output after observing failure.
- ad hoc to paper Negative feedback: each AI social-learning event multiplies the agent's individual-learning efficacy by 0.9, with no recovery mechanism.
- domain assumption A single 'collective world model' exists and fitness is measured by the expected proportion of adapted agents q^OK.
invented entities (2)
-
Abstract 'AI agent' that learns socially from the entire population at each timestep
-
Individual learning penalty kappa_j (deskilling/negative feedback)
Cite this review
Pith. "Pith review of Revisiting Rogers' Paradox in the Context of Human-AI Interaction." pith.science (2026). https://pith.science/paper/GRNOV2HJ
@misc{pith2026250110476,
author = {Pith},
title = {Pith review of: Revisiting Rogers' Paradox in the Context of Human-AI Interaction},
year = {2026},
howpublished = {\url{https://pith.science/paper/GRNOV2HJ}},
note = {Machine review of arXiv:2501.10476}
}
read the original abstract
Humans learn about the world, and how to act in the world, in many ways: from individually conducting experiments to observing and reproducing others' behavior. Different learning strategies come with different costs and likelihoods of successfully learning more about the world. The choice that any one individual makes of how to learn can have an impact on the collective understanding of a whole population if people learn from each other. Alan Rogers developed simulations of a population of agents to study these network phenomena where agents could individually or socially learn amidst a dynamic, uncertain world and uncovered a confusing result: the availability of cheap social learning yielded no benefit to population fitness over individual learning. This paradox spawned decades of work trying to understand and uncover factors that foster the relative benefit of social learning that centuries of human behavior suggest exists. What happens in such network models now that humans can socially learn from AI systems that are themselves socially learning from us? We revisit Rogers' Paradox in the context of human-AI interaction to probe a simplified network of humans and AI systems learning together about an uncertain world. We propose and examine the impact of several learning strategies on the quality of the equilibrium of a society's 'collective world model'. We consider strategies that can be undertaken by various stakeholders involved in a single human-AI interaction: human, AI model builder, and society or regulators around the interaction. We then consider possible negative feedback loops that may arise from humans learning socially from AI: that learning from the AI may impact our own ability to learn about the world. We close with open directions into studying networks of human and AI systems that can be explored in enriched versions of our simulation framework.
Figures
Figures from the paper (7 more)
Forward citations
Cited by 2 Pith papers
-
Human-AI Synergy Supports Collective Creative Search
Hybrid human-AI groups outperform all-human and all-AI groups on a semantic word-search task while keeping guess diversity as high as all-human groups.
-
Documenting Deployment with Fabric: A Repository of Real-World AI Governance
Fabric is a public repository of 20 deployed AI systems with co-designed workflow diagrams, plus a four-level human oversight and five-level institutional oversight taxonomy.
Reference graph
Works this paper leans on
-
[4]
A. Chan, R. Salganik, A. Markelius, C. Pang, N. Rajkumar, D. Krasheninnikov, L. Langosco, Z. He, Y. Duan, M. Carroll, et al. Harms from increasingly agentic algorithmic systems. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency , pages 651–666,
work page 2023
-
[10]
When Should You Override from an AI System? We next imagine that the agent can tell whether they became adapted after attempting to learn: that is, the agent can evaluate how successful a learning interaction was, in line with [Enquist et al., 2007]. We extend the simulation to consider the case where agents can override the outputs of the AI system by le...
work page 2007
-
[11]
W. Gurnee and M. Tegmark. Language models represent space and time. arXiv preprint arXiv:2310.02207,
-
[12]
S. Hao, Y. Gu, H. Ma, J. J. Hong, Z. Wang, D. Z. Wang, and Z. Hu. Reasoning with language model is planning with world model. arXiv preprint arXiv:2305.14992,
-
[14]
URL http:// arxiv.org/abs/1706.06551. J. Hernández-Orallo and K. Vold. Ai extenders: the ethical and societal implications of humans cognitively extended by ai. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pages 507–513,
arXiv 2019
- [15]
- [16]
-
[17]
V . Lai, C. Chen, A. Smith-Renner, Q. V . Liao, and C. Tan. Towards a science of human-ai decision making: An overview of design space in empirical human-subject studies. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pages 1369–1385,
work page 2023
Show all 25 references
-
[18]
R. Liu, J. Geng, J. C. Peterson, I. Sucholutsky, and T. L. Griffiths. Large language models assume people are more rational than we really are. arXiv preprint arXiv:2406.17055,
-
[19]
doi: 10.2760/376602(online)
ISSN 1831-9424 (online). doi: 10.2760/376602(online). A. Salinas, P . Shah, Y. Huang, R. McCormack, and F. Morstatter. The unequal opportunities of large language models: Examining demographic biases in job recommendations by chatgpt and llama. In Proceedings of the 3rd ACM Co...
-
[20]
Silver, T
D. Silver, T. Hubert, J. Schrittwieser, I. Antonoglou, M. Lai, A. Guez, M. Lanctot, L. Sifre, D. Ku- maran, T. Graepel, et al. Mastering chess and shogi by self-play with a general reinforcement learning algorithm. arXiv preprint arXiv:1712.01815,
-
[21]
Snell, J
C. Snell, J. Lee, K. Xu, and A. Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314,
-
[22]
Sucholutsky, R
I. Sucholutsky, R. M. Battleday, K. M. Collins, R. Marjieh, J. Peterson, P . Singh, U. Bhatt, N. Jacoby, A. Weller, and T. L. Griffiths. On the informativeness of supervision signals. In Uncertainty in Artificial Intelligence, pages 2036–2046. PMLR, 2023a. 19 I. Sucholutsky, L...
-
[24]
K. Vafa, J. Y. Chen, A. Rambachan, J. Kleinberg, and S. Mullainathan. Evaluating the world model implicit in a generative model. In Neural Information Processing Systems, 2024a. K. Vafa, A. Rambachan, and S. Mullainathan. Do large language models perform the way people expect?...
-
[25]
L. Wong, G. Grand, A. K. Lew, N. D. Goodman, and V . K. e. a. Mansinghka. From word models to world models: Translating from natural language to the probabilistic language of thought. arXiv preprint arXiv:2306.12672, pages arXiv–2306,
-
[1993]
doi: 10.1017/S0140525X0003123X. M. Tomasello, M. Carpenter, J. Call, T. Behne, and H. Moll. Understanding and sharing intentions: The origins of cultural cognition. Behavioral and brain sciences, 28(5):675–691,
-
[2007]
Gabriel, A
I. Gabriel, A. Manzini, G. Keeling, L. A. Hendricks, and V . e. a. Rieser. The ethics of advanced ai assistants. arXiv preprint arXiv:2404.16244,
-
[2017]
Ehsan, Q
U. Ehsan, Q. V . Liao, M. Muller, M. O. Riedl, and J. D. Weisz. Expanding explainability: Towards social transparency in ai systems. In Proceedings of the 2021 CHI conference on human factors in computing systems, pages 1–19,
2021
-
[2018]
Doshi-Velez and B
F. Doshi-Velez and B. Kim. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608,
-
[2019]
Z. Guo, Y. Wu, J. D. Hartline, and J. Hullman. A decision theoretic framework for measuring ai reliance. In The 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 221–236,
2024
-
[2020]
URL https://www
doi: 10.1073/pnas.1912341117. URL https://www. pnas.org/doi/abs/10.1073/pnas.1912341117. 15 J. Andreas. Language models as agent models. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 5769–5779,
2022 doi
-
[2021]
ISBN 9781450384735. U. Bhatt, V . Chen, K. M. Collins, P . Kamalaruban, E. Kallina, A. Weller, and A. Talwalkar. Learning personalized decision support policies. arXiv preprint arXiv:2304.06701,
-
[2022]
K. M. Collins, M. Barker, M. Espinosa Zarlenga, N. Raman, and U. e. a. Bhatt. Human uncertainty in concept-based ai systems. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, pages 869–889,
2023
-
[2023]
K. M. Collins, V . Chen, I. Sucholutsky, H. R. Kirk, M. Sadek, H. Sargeant, A. Talwalkar, A. Weller, and U. Bhatt. Modulating language model experiences through frictions. In NeurIPS 2024 Workshop on Behavioral Machine Learning, 2024a. K. M. Collins, A. Q. Jiang, S. Frieder, L...
2024 arXiv
-
[2024]
Bhatt, J
U. Bhatt, J. Antorán, Y. Zhang, Q. V . Liao, and P . e. a. Sattigeri. Uncertainty as a form of transparency: Measuring, communicating, and using uncertainty. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pages 401–413, New York, NY, USA,
2021
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.