pith. sign in

arxiv: 1907.04534 · v1 · pith:63VJ74J2new · submitted 2019-07-10 · 💻 cs.CY · cs.AI

The Role of Cooperation in Responsible AI Development

Amanda Askell , Miles Brundage , Gillian Hadfield This is my paper

Pith reviewed 2026-05-24 23:42 UTC · model grok-4.3

classification 💻 cs.CY cs.AI
keywords responsible AIcollective action problemsAI industry cooperationcompetitive pressuresAI safetyAI ethicsindustry self-regulation
0
0 comments X

The pith

Competitive pressures may cause AI companies to underinvest in safety and positive impact, requiring solutions to collective action problems among firms.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that market competition can push AI developers to reduce spending on safety, security, and beneficial social effects in order to stay ahead of rivals. A sympathetic reader would therefore conclude that responsible AI cannot be achieved through isolated company decisions and instead depends on coordinated efforts to resolve shared underinvestment incentives. The authors examine conditions known to support cooperation in such dilemmas and derive practical strategies for AI firms to collaborate on responsibility measures.

Core claim

Competitive pressures could incentivize AI companies to underinvest in ensuring their systems are safe, secure, and have a positive social impact. Ensuring that AI systems are developed responsibly may therefore require preventing and solving collective action problems between companies. Several key factors improve the prospects for cooperation in collective action problems, and these can be used to identify strategies that improve the prospects for industry cooperation on the responsible development of AI.

What carries the argument

Collective action problems among AI companies, in which each firm's incentive to underinvest in responsibility conflicts with the shared interest in safe and beneficial systems; the paper applies known factors that aid cooperation to generate industry strategies.

If this is right

  • AI firms may need shared mechanisms to jointly fund and enforce safety standards that no single company would adopt alone.
  • Industry agreements could block a race to lower responsibility standards driven by short-term competitive gains.
  • Cooperation is more likely when firms can monitor each other's actions and expect repeated future interactions.
  • Strategies should target factors that raise the cost of defection and increase the value of mutual compliance.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same logic could apply to other fast-moving technologies where rivalry affects ethical or safety investments.
  • External rules or neutral conveners might create protected channels for companies to coordinate without legal risk.
  • Market data on investment levels versus competitive intensity could test whether the collective action framing holds in practice.

Load-bearing premise

Competitive pressures are the dominant reason AI companies underinvest in responsible development.

What would settle it

An industry-wide audit or dataset showing high levels of safety investment by AI companies even under strong market competition, or data indicating that underinvestment arises primarily from technical limits or regulation rather than rivalry.

read the original abstract

In this paper, we argue that competitive pressures could incentivize AI companies to underinvest in ensuring their systems are safe, secure, and have a positive social impact. Ensuring that AI systems are developed responsibly may therefore require preventing and solving collective action problems between companies. We note that there are several key factors that improve the prospects for cooperation in collective action problems. We use this to identify strategies to improve the prospects for industry cooperation on the responsible development of AI.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 2 minor

Summary. The paper argues that competitive pressures could incentivize AI companies to underinvest in ensuring their systems are safe, secure, and have positive social impact. It therefore suggests that responsible AI development may require preventing and solving collective action problems between companies. The authors identify several key factors that improve prospects for cooperation in such problems and use these to outline strategies for industry cooperation on responsible AI.

Significance. If the conditional argument holds, the paper offers a clear conceptual framing of responsible AI as a potential collective-action issue, drawing on standard economic reasoning to highlight cooperation factors and practical strategies. This could usefully inform policy discussions and industry self-regulation efforts, though the contribution remains primarily analytical rather than empirical or model-based.

minor comments (2)
  1. The manuscript would benefit from explicit citations to foundational works on collective action (e.g., Ostrom or Axelrod) when listing the key cooperation factors, to strengthen the grounding of the proposed strategies.
  2. Section discussing strategies could include brief, concrete illustrations from other industries (e.g., pharmaceutical safety standards or environmental agreements) to make the recommendations more actionable.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their review and recommendation of minor revision. The referee's summary accurately reflects the paper's argument that competitive pressures in AI development can create collective action problems, and that identifying factors for cooperation can inform strategies for responsible AI.

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper advances a conditional conceptual argument drawn from standard economic theory on collective action problems and competitive incentives. No equations, fitted parameters, self-citations, or uniqueness claims appear in the provided text or abstract. The central claim does not reduce to any input by definition or construction; it applies external economic principles to the AI domain without circular reduction.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The argument rests on standard domain assumptions from economics about competition and public goods without introducing new parameters, entities, or ad-hoc axioms specific to this paper.

axioms (1)
  • domain assumption Competitive pressures in markets can lead firms to underinvest in safety and responsibility as a form of public good.
    This is the core premise stated in the abstract that motivates the need for collective action solutions.

pith-pipeline@v0.9.0 · 5591 in / 1152 out tokens · 18336 ms · 2026-05-24T23:42:41.197529+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Why Open Source? A Game-Theoretic Analysis of the AI Race

    cs.GT 2026-04 unverdicted novelty 6.0

    A game-theoretic R&D race model shows that pure Nash equilibria for open-sourcing decisions exist and are computationally tractable in both discrete and continuous settings.

  2. Racing to Release: Priority, Congestion, and Community Recognition in Open-Source LLM Ecosystems

    cs.SI 2026-04 unverdicted novelty 4.0

    Later derivative LLM releases and those in more crowded competitive environments on Hugging Face receive weaker community recognition.

Reference graph

Works this paper leans on

37 extracted references · 37 canonical work pages · cited by 2 Pith papers · 5 internal anchors

  1. [1]

    Concrete Problems in AI Safety

    (Accessed on 03/12/2019). Dario Amodei, Chris Olah, et al. Concrete problems in ai safe ty. arXiv preprint arXiv:1606.06565 ,

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  2. [2]

    Competition and collusion in the ameri can automobile industry: The 1955 price war

    Timothy F Bresnahan. Competition and collusion in the ameri can automobile industry: The 1955 price war. The Journal of Industrial Economics , pages 457–482,

    work page 1955

  3. [3]

    C., Steinhardt, J., Flynn, C., h\' E igeartaigh, S

    Miles Brundage, Shahar Avin, et al. The Malicious Use of Arti ficial Intelligence: Forecasting, Prevention, and Mitigation. arXiv e-prints, art. arXiv:1802.07228, Feb

    work page arXiv

  4. [4]

    Compensation an d risk incentives in banking and finance

    Jian Cai, Kent Cherny, Todd Milbourn, et al. Compensation an d risk incentives in banking and finance. Economic Commentary, (2010-13),

    work page 2010

  5. [5]

    URL https://doi.org/10 .5465/amr.2007.25275684

    doi: 10.5465/amr.2007.25275684. URL https://doi.org/10 .5465/amr.2007.25275684. 18 D Chalmers. The singularity: A philosophical analysis. Science fiction and philosophy: From time travel to superintelligence , pages 171–224,

    work page doi:10.5465/amr.2007.25275684 2007

  6. [6]

    Paul Christiano

    (Accessed on 03/13/2019). Paul Christiano. Hyperbolic growth. The sideways view, https://sideways-view.com/2017/10/04/ hyperbolic-growth/, Oct

    work page 2019

  7. [7]

    Paul Christiano

    (Accessed on 03/21/2019). Paul Christiano. Takeoff speeds. The sideways view, https://sideways-view.com/2018/02/24/ takeoff-speeds/, Feb

    work page 2019

  8. [8]

    Jack Clark and Dario Amodei

    (Accessed on 03/21/2019). Jack Clark and Dario Amodei. Faulty reward functions in the w ild. Open AI Blog, https://openai.com/ blog/faulty-reward-functions/, Dec

    work page 2019

  9. [9]

    Iain M Cockburn, Rebecca Henderson, and Scott Stern

    (Accessed on 03 /13/2019). Iain M Cockburn, Rebecca Henderson, and Scott Stern. The imp act of artificial intelligence on innovation. Technical report, National Bureau of Economic Research,

    work page 2019

  10. [10]

    Nina W Cornell, Roger G Noll, and Barry R Weingast

    (Accessed on 03/12/2019). Nina W Cornell, Roger G Noll, and Barry R Weingast. Safety reg ulation

    work page 2019

  11. [11]

    Towards A Rigorous Science of Interpretable Machine Learning

    Finale Doshi-V elez and Been Kim. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 ,

    work page internal anchor Pith review Pith/arXiv arXiv

  12. [12]

    URL https://doi.org/10.1016/ 0166-4972(95)96593-i

    doi: 10.1016/0166-4972(95)9659 3-i. URL https://doi.org/10.1016/ 0166-4972(95)96593-i. Jon Elster. Rationality, morality, and collective action. Ethics, 96(1):136–155,

    work page doi:10.1016/0166-4972(95)9659

  13. [13]

    Regulating artificial i ntelligence: Proposal for a global solution

    Olivia J Erdélyi and Judy Goldsmith. Regulating artificial i ntelligence: Proposal for a global solution. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, a nd Society, pages 95–101. ACM,

    work page 2018

  14. [14]

    Shanti Gamper-Rabindran and Stephen R Finger

    (Accessed on 04/01/2019). Shanti Gamper-Rabindran and Stephen R Finger. Does industr y self-regulation reduce pollution? responsible care in the chemical industry. Journal of Regulatory Economics , 43(1):1–30,

    work page 2019

  15. [15]

    URL https://doi.org/10 .2307/2297442

    doi: 10.2307/2297442. URL https://doi.org/10 .2307/2297442. 20 John C Harsanyi. A simplified bargaining model for the n-pers on cooperative game. International Economic Review, 4(2):194–220,

    work page doi:10.2307/2297442

  16. [16]

    U RL https://doi.org/10.1177/ 1043463189001001006

    doi: 10.1177/1043463189001001006. U RL https://doi.org/10.1177/ 1043463189001001006. Spencer Henson and Julie Caswell. Food safety regulation: a n overview of contemporary issues. F ood policy, 24(6):589–603,

    work page doi:10.1177/1043463189001001006

  17. [17]

    The problems of collective action: A n ew approach

    Katharina Holzinger. The problems of collective action: A n ew approach. MPI Collective Goods Preprint, (2003/2),

    work page 2003

  18. [18]

    AI safety via debate

    Geoffrey Irving, Paul Christiano, and Dario Amodei. Ai safe ty via debate. arXiv preprint arXiv:1805.00899,

    work page internal anchor Pith review Pith/arXiv arXiv

  19. [19]

    URL http://www.jstor.org/stable/2006714

    ISSN 00028282. URL http://www.jstor.org/stable/2006714 . Victoria Krakovna. Clopen ai: Openness in different aspect s of ai development. https://vkrakovna. wordpress.com/2016/08/01/clopen-ai-openness-in-diff erent-aspects-of-ai-development/, Aug

    work page arXiv 2016

  20. [20]

    Victoria Krakovna

    (Accessed on 06/27/2019). Victoria Krakovna. Specification gaming examples in ai. htt ps://vkrakovna.wordpress.com/2018/04/ 02/specification-gaming-examples-in-ai/, April

    work page 2019

  21. [21]

    Sarah Kreps

    (A ccessed on 05/14/2019). Sarah Kreps. Is this a sputnik moment for artificial intellig ence? or why the cold war past should not be prologue (unpublished manuscript)

    work page 2019

  22. [22]

    AI Safety Gridworlds

    21 Jan Leike, Miljan Martic, Victoria Krakovna, Pedro A Ortega , Tom Everitt, Andrew Lefrancq, Laurent Orseau, and Shane Legg. Ai safety gridworlds. arXiv preprint arXiv:1711.09883 ,

    work page internal anchor Pith review Pith/arXiv arXiv

  23. [23]

    The newfound power of tech workers

    Talya Minsberg. The newfound power of tech workers. The New Y ork Times, https://www.nytimes. com/2019/03/02/business/tech-employees-protests.html, Mar

    work page 2019

  24. [24]

    Kara Morgan

    (Accessed on 03/14/2019). Kara Morgan. Development of a preliminary framework for inf orming the risk analysis and risk management of nanoparticles. Risk Analysis: An International Journal , 25(6):1621–1635,

    work page 2019

  25. [25]

    Martin A Nowak

    doi: 10.1145/2644257. Martin A Nowak. Five rules for the evolution of cooperation. science, 314(5805):1560–1563,

    work page doi:10.1145/2644257

  26. [26]

    (Accessed on 04/01/2019). Pedro A. Ortega, Vishal Maini, et al. Building safe artificia l intelligence: specification, robustness, and assurance. https://medium.com/@deepmindsafetyresearc h/ building-safe-artificial-intelligence-52f5f75058f1, S ep

    work page 2019

  27. [27]

    Cullen O’Keefe

    (Accessed on 03/13/2019). Cullen O’Keefe. Antitrust-compliant ai industry self-reg ulation. forthcoming

    work page 2019

  28. [28]

    The windfall clause

    Cullen O’Keefe et al. The windfall clause. forthcoming 2019 . Christine Parker. Meta-regulation: legal accountability for corporate social responsibility

    work page 2019

  29. [29]

    Schelling

    T.C. Schelling. Micromotives and Macrobehavior. W . W . Norton, 2006 [1978]. ISBN 9780393069778. URL https://books.google.com/books?id=DenWKRgqzWMC. Thomas C Schelling. The strategy of conflict . Harvard university press,

    work page 2006

  30. [30]

    URL https://doi.org/10.10 38/d41586-018-07501-y

    doi: 10.1038/d41586-018-07501-y. URL https://doi.org/10.10 38/d41586-018-07501-y. Murray Shanahan. The technological singularity. MIT Press,

    work page doi:10.1038/d41586-018-07501-y

  31. [31]

    Facial recognition technology: The need for pub lic regulation and corporate responsibility

    Brad Smith. Facial recognition technology: The need for pub lic regulation and corporate responsibility. https://blogs.microsoft.com/on-the-i ssues/2018/07/13/ facial-recognition-technology-the-need-for-public-r egulation-and-corporate-responsibility/, Jul 2018a. (Accessed on 03/20/2019). Brad Smith. Facial recognition: It’s time for action. Microsoft on ...

    work page 2018

  32. [32]

    Who is liable for a death caused by a self-drivi ng car? https://www.theatlantic.com/ technology/archive/2018/03/can-you-sue-a-robocar/55 6007/, Mar

    The Atlantic. Who is liable for a death caused by a self-drivi ng car? https://www.theatlantic.com/ technology/archive/2018/03/can-you-sue-a-robocar/55 6007/, Mar

    work page 2018

  33. [33]

    coopetition

    (Accessed on 03/12/2019). Wenpin Tsai. Social structure of “coopetition” within a mul tiunit organization: Coordination, competition, and intraorganizational knowledge sharing. Organization science, 13(2):179–190,

    work page 2019

  34. [34]

    If a pre-trial risk ass essment tool does not satisfy these criteria, it needs to stay out of the courtroom | electronic f rontier foundation

    Hayley Tsukayama and Jamie Williams. If a pre-trial risk ass essment tool does not satisfy these criteria, it needs to stay out of the courtroom | electronic f rontier foundation. EFF, https://www.eff. org/deeplinks/2018/11/ if-pre-trial-risk-assessment-tool-does-not-satisfy- these-criteria-it-needs-stay, Nov

    work page 2018

  35. [35]

    (Accessed on 03/13/2019). Roman V . Y ampolskiy. Taxonomy of pathways to dangerous AI.arXiv preprint arXiv:1511.03246 ,

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  36. [36]

    Remco Zwetsloot and Allan Dafoe

    (Accessed on 03/21/2019). Remco Zwetsloot and Allan Dafoe. Thinking About Risks From A I: Accidents, Misuse and Structure, Feb

    work page 2019

  37. [37]

    Lawfare, https://www.lawfareblog.com/ thinking-about-risks-ai-accidents-misuse-and-struct ure (Accessed 03/07/2019). 23

    work page 2019