REVIEW 10 cited by
Evaluating the World Model Implicit in a Generative Model
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Recent work suggests that large language models may implicitly learn world models. How should we assess this possibility? We formalize this question for the case where the underlying reality is governed by a deterministic finite automaton. This includes problems as diverse as simple logical reasoning, geographic navigation, game-playing, and chemistry. We propose new evaluation metrics for world model recovery inspired by the classic Myhill-Nerode theorem from language theory. We illustrate their utility in three domains: game playing, logic puzzles, and navigation. In all domains, the generative models we consider do well on existing diagnostics for assessing world models, but our evaluation metrics reveal their world models to be far less coherent than they appear. Such incoherence creates fragility: using a generative model to solve related but subtly different tasks can lead to failures. Building generative models that meaningfully capture the underlying logic of the domains they model would be immensely valuable; our results suggest new ways to assess how close a given model is to that goal.
Forward citations
Cited by 10 Pith papers
-
ICLR: In-Context Learning of Representations
As in-context examples grow, Llama-3.1-8B reorganizes its concept representations to mirror the connectivity structure of a graph defined entirely in context.
-
What Does it Mean for a Neural Network to Learn a "World Model"?
Defines a world model as a simple commutative-diagram factorization through an intermediate representation, with conditions that the model be learned and emergent rather than inherited from input or output.
-
Modeling Open-World Cognition as On-Demand Synthesis of Probabilistic Models
A hybrid language-model and probabilistic-program architecture predicts human judgments on novel open-world reasoning vignettes better than language-model-only baselines.
-
Potemkin Understanding in Large Language Models
LLMs frequently pass definition questions yet fail to use the same concepts in classification, generation, and editing tasks, a gap the authors call potemkin understanding.
-
The Limits of Predicting Agents from Behaviour
Observed behavior only weakly constrains an intentional agent's choices under distribution shift, and its perceived fairness and harm cannot be identified from behavior alone.
-
Quo Vadis, World Modeling?
An agent-centric reframing of world modeling, replacing physical state prediction with 'information transitions' organized into six proxy functions and three empowerment levels.
-
Behavioural vs. Representational Systematicity in End-to-End Models: An Opinionated Survey
A survey showing that common systematic generalization benchmarks measure behavioural systematicity, not the representational systematicity that Fodor and Pylyshyn's challenge requires, and mapping them onto Hadley's ...
-
Performance Gains of LLMs With Humans in a World of LLMs Versus Humans
A commentary argues that medical research should stop running evanescent LLM-versus-human comparisons and instead study human-LLM collaboration, supported by a literature review showing rapid model turnover and tiny e...
-
Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluation
A meta-review of about 110 critical studies finds nine systemic weaknesses in AI benchmarking and concludes that benchmarks are receiving disproportionate trust in AI governance.
-
Emergent Stack Representations in Modeling Counter Languages Using Transformers
A small transformer trained on counter languages encodes the current stack depth in its final-layer activations, recoverable by simple probing classifiers.
Discussion (0). Continue with ORCID to comment.