REVIEW 12 cited by
On the Societal Impact of Open Foundation Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Foundation models are powerful technologies: how they are released publicly directly shapes their societal impact. In this position paper, we focus on open foundation models, defined here as those with broadly available model weights (e.g. Llama 2, Stable Diffusion XL). We identify five distinctive properties (e.g. greater customizability, poor monitoring) of open foundation models that lead to both their benefits and risks. Open foundation models present significant benefits, with some caveats, that span innovation, competition, the distribution of decision-making power, and transparency. To understand their risks of misuse, we design a risk assessment framework for analyzing their marginal risk. Across several misuse vectors (e.g. cyberattacks, bioweapons), we find that current research is insufficient to effectively characterize the marginal risk of open foundation models relative to pre-existing technologies. The framework helps explain why the marginal risk is low in some cases, clarifies disagreements about misuse risks by revealing that past work has focused on different subsets of the framework with different assumptions, and articulates a way forward for more constructive debate. Overall, our work helps support a more grounded assessment of the societal impact of open foundation models by outlining what research is needed to empirically validate their theoretical benefits and risks.
Forward citations
Cited by 12 Pith papers
-
Opening the Scope of Openness in AI
A taxonomy of 98 openness concepts from across disciplines reveals that AI openness discussions overemphasize access, inspection, reuse, and organic behavior while underrepresenting fairness, diversity, autonomy, and ...
-
Who Does Withholding Delay? A Game-Theoretic Model of Open-Weight AI Release Under Asymmetric Proliferation
For dual-use AI, withholding helps only if it delays harmful actors more than defenders; the paper derives a substitution-rate threshold that decides when open release beats control.
-
Video Deepfake Abuse: How Company Choices Predictably Shape Misuse Patterns
A few open-weight video models and distribution platforms dominate the creation and spread of NSFW AI video, making developer and platform choices the main intervention points for reducing non-consensual deepfake abuse.
-
User Privacy and Large Language Models: An Analysis of Frontier Developers' Privacy Policies
All six leading U.S. AI chatbot developers, as of May 2025, appear to train their models on users' chat data by default, often without clear opt-out options.
-
Attestable Audits: Verifiable AI Safety Benchmarks Using Trusted Execution Environments
A TEE-based protocol for cryptographically verifiable AI safety benchmark results, demonstrated on Llama-3.1 with AWS Nitro Enclaves.
-
From Reddit to Generative AI: Evaluating Large Language Models for Anxiety Support Fine-tuned on Social Media Data
Fine-tuning GPT-3.5 and Llama 2 on r/Anxiety posts improves readability but raises toxicity and bias while reducing empathy and reflection.
-
Distributed and Decentralised Training: Technical Governance Challenges in a Shifting AI Landscape
A policy analysis distinguishing distributed and decentralised AI training, arguing decentralised training may erode detectability and shutdownability while compute controls remain relevant.
-
7B Fully Open Source Moxin-LLM/VLM -- From Pretraining to GRPO-based Reinforcement Learning Enhancement
The authors trained and openly released a 7B LLM, an instruction-tuned variant, a GRPO-based reasoning variant, and a VLM, claiming competitive or superior performance on zero-shot, few-shot, CoT, and VLM benchmarks.
-
The Reality of AI and Biorisk
A review of the evidence finds no current, statistically significant uplift in biorisk from LLMs or AI biological tools, but the underlying studies are too nascent to justify strong conclusions.
-
Position Paper: Model Access should be a Key Concern in AI Governance
Model access decisions should be studied and coordinated through a dedicated research field, with recommendations for evaluators, companies, governments, and international bodies.
-
Dimensions of Generative AI Evaluation Design
A proposed taxonomy of seven dimensions (setting, task type, input source, interaction style, duration, metric type, scoring method) for designing and comparing generative AI evaluations.
-
AI Safety Frameworks Should Include Procedures for Model Access Decisions
The paper introduces Responsible Access Policies as a recommended component of frontier AI safety frameworks for governing model access.
Discussion (0). Continue with ORCID to comment.