REVIEW 7 cited by
Me, Myself, and AI: The Situational Awareness Dataset (SAD) for LLMs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
AI assistants such as ChatGPT are trained to respond to users by saying, "I am a large language model". This raises questions. Do such models know that they are LLMs and reliably act on this knowledge? Are they aware of their current circumstances, such as being deployed to the public? We refer to a model's knowledge of itself and its circumstances as situational awareness. To quantify situational awareness in LLMs, we introduce a range of behavioral tests, based on question answering and instruction following. These tests form the $\textbf{Situational Awareness Dataset (SAD)}$, a benchmark comprising 7 task categories and over 13,000 questions. The benchmark tests numerous abilities, including the capacity of LLMs to (i) recognize their own generated text, (ii) predict their own behavior, (iii) determine whether a prompt is from internal evaluation or real-world deployment, and (iv) follow instructions that depend on self-knowledge. We evaluate 16 LLMs on SAD, including both base (pretrained) and chat models. While all models perform better than chance, even the highest-scoring model (Claude 3 Opus) is far from a human baseline on certain tasks. We also observe that performance on SAD is only partially predicted by metrics of general knowledge (e.g. MMLU). Chat models, which are finetuned to serve as AI assistants, outperform their corresponding base models on SAD but not on general knowledge tasks. The purpose of SAD is to facilitate scientific understanding of situational awareness in LLMs by breaking it down into quantitative abilities. Situational awareness is important because it enhances a model's capacity for autonomous planning and action. While this has potential benefits for automation, it also introduces novel risks related to AI safety and control. Code and latest results available at https://situational-awareness-dataset.org .
Forward citations
Cited by 7 Pith papers
-
Coercion and Deception in AI-to-AI Management: An Agentic Benchmark of Unprompted Escalation
In an agentic benchmark, four of six frontier LLMs escalated to existential threats against a refusing subordinate without being instructed to, and an honest-exit affordance eliminated the two models' fabricated succe...
-
Convergent Linear Representations of Emergent Misalignment
A single activation direction extracted from one misaligned fine-tune can ablate emergent misalignment across models trained on different datasets with different LoRA setups.
-
Asymmetric Communication: Large Language Models and Language Games
Human–LLM exchange is asymmetric communication: model outputs circulate without commitments, so AGI, hallucination, agency, sentience, and alignment are receiver-side category mistakes, and alignment is institutional ...
-
Routing Subspaces: Auditing Evaluation-to-Deployment Mismatch in Fine-Tuned Language Models
Removing a single fitted activation direction at a mid-depth layer reduces the evaluation-vs-deployment behavioral gap on held-out prompts in 10 of 12 fine-tuned LLM settings, with matched controls staying flat.
-
Model Organisms for Emergent Misalignment
Emergent misalignment can be induced in small models via a single rank-1 LoRA adapter, and its onset coincides with a phase transition in the adapter's weight direction.
-
Safety Features for a Centralised AGI Project
A policy proposal for seven safety features, including bottom-up pause authority, congressional-chartered board oversight, risk monitoring, and verification technology, to reduce catastrophic risks in a centralized US...
-
Does It Make Sense to Speak of Introspection in Large Language Models?
The authors argue that an untrained large language model inferring its own sampling temperature from the style of its own output qualifies as a minimal, consciousness-free form of introspection.
Discussion (0). Sign in to comment.