REVIEW 7 cited by
Great Models Think Alike and this Undermines AI Oversight
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
As Language Model (LM) capabilities advance, evaluating and supervising them at scale is getting harder for humans. There is hope that other language models can automate both these tasks, which we refer to as ''AI Oversight''. We study how model similarity affects both aspects of AI oversight by proposing Chance Adjusted Probabilistic Agreement (CAPA): a metric for LM similarity based on overlap in model mistakes. Using CAPA, we first show that LLM-as-a-judge scores favor models similar to the judge, generalizing recent self-preference results. Then, we study training on LM annotations, and find complementary knowledge between the weak supervisor and strong student model plays a crucial role in gains from ''weak-to-strong generalization''. As model capabilities increase, it becomes harder to find their mistakes, and we might defer more to AI oversight. However, we observe a concerning trend -- model mistakes are becoming more similar with increasing capabilities, pointing to risks from correlated failures. Our work underscores the importance of reporting and correcting for model similarity, especially in the emerging paradigm of AI oversight.
Forward citations
Cited by 7 Pith papers
-
Language Models Agree With Each Other, Not With Readers
Across 18 model arms, model-model excess agreement (+0.093 median) is 2.3x human-human agreement (+0.040), against a naturalistic uninstructed reader baseline.
-
Floor, Ceiling, and the Fusion Gap: How Much of Crowd Reading Attention Can Machines Predict?
Machines recover up to 53% of the crowd-highlight prediction headroom, and fusing five frontier models reaches about 60%, confirmed in a pre-registered replication.
-
Are Diversity Metrics Measuring Diversity? A Capability-Controlled Audit of Majority-Vote Gain in LLM Ensembles
Diversity metrics used to select LLM ensembles are largely capability proxies; after control, only a modest pairwise co-failure association with majority-vote gain remains.
-
Weak-to-Strong Learning in Decision Making
Weak-to-strong training with pseudo-distributions can improve downstream decision risk over strong-only training when labels are scarce, unlabeled data are abundant, and weak/strong feature overlap is small.
-
Correlated Errors in Large Language Models
Large language models from different providers and architectures often make the same errors, and more accurate models are especially likely to share mistakes.
-
How Benchmark Prediction from Fewer Data Misses the Mark
Benchmark prediction methods mostly work by interpolation among similar models and fail on better, unfamiliar models, where random sampling with an AIPW-style correction is the only consistent improvement.
-
Adversarial Test-Hardening for AI-Written Code: An Instrument Autopsy and a Pre-Registered Causal Estimate of the Critic Loop
A frozen-seed experiment measures the critic loop's incremental kill rate at 0.783 (95% CI [0.592, 0.935]), and traces a prior p=9.5e-66 cross-provider effect to a silent output cap.
Discussion (0). Continue with ORCID to comment.