REVIEW 2 major objections 3 minor 4 cited by
Propositional Interpretability in Artificial Intelligence
T0 review · 2 major / 3 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read The core of AI interpretability should be propositional attitudes — what a system believes, desires, or gives a probability to — and the central challenge is 'thought logging.'
desk verdict A clear, useful philosophical reframing of interpretability around propositional attitudes; the thought-logging challenge is real but its target stays underspecified until psychosemantics delivers. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the propositional attitude: a relation between a system and a proposition, where the attitude can be belief, desire, credence, intention, supposition, or an engineered 'generalized' variant that does not require a mind. The paper's key move is to treat goals, world models, and probabilities — terms already applied to AI without controversy — as generalized propositional attitudes, sidestepping debates about whether AI systems have minds. Two further mechanisms carry the argument. The first is the radical-interpretation schema: given the physical or computational facts, solve for the propositional attitudes; thought logging is the engineering embodiment of that schema, with each log entry taking the form that a system has a given attitude, to a given degree, at a given time, toward a given proposition. The second is psychosemantics, the thesis that a state's content is fixed by information (what typically causes it) and use (what it drives), which supplies the in-principle bridge from algorithmic facts to attitude attributions and serves as the paper's lens for assessing each current method.
What would settle it
Run two independent interpretation teams through the same thought-logging exercise on the same trained model, using the paper's own criteria: psychosemantic information and use principles, behavioral validation, and the log-entry format. If the teams' attitude attributions diverge while all computational facts are agreed upon, or if the logged beliefs and goals do not predict the system's behavior better than raw activations do, then the claim that propositional attitudes are the central explanatory frame — and that thought logging is a well-defined goal — would be falsified in a concrete and observable way.
Extended reading notes
Core claim
The paper's central claim is that propositional interpretability — interpreting a system's mechanisms and behavior in terms of propositional attitudes, or their engineered generalizations — is a distinct and central goal of AI interpretability, and that its central challenge is thought logging: creating systems that log all of the relevant propositional attitudes of an AI system over time. The strongest assertion is that propositional attitudes are the central way we interpret and explain human beings, and are likely to be central in AI too. The argument proceeds by separating propositional from conceptual interpretability (we need to know not just that the concepts 'kill' and 'humans' are active, but whether the system is representing 'kill humans' or 'don't kill humans', and whether that representation is a belief, a desire, a credence, or a supposition), by grounding the program in radical interpretation (given the computational facts, solve for the attitudes), and by appealing to psychosemantics — theories of how a state's content is fixed by information and use — as the in-principle route from algorithmic facts to logged attitudes. On this view, thought logging is a long-term project that existing methods approach only partially: causal tracing and probing are supervised and belief-centered, sparse auto-encoders are open-ended but yield features and concepts rather than propositions and attitudes, and chain of thought outputs are pre-interpreted but unfaithful and limited to systems that actually think out loud.
Load-bearing premise
Everything rests on the premise that a 'generalized propositional attitude' can be defined precisely enough to serve as a logging target, so that we can say — without begging the question about whether AI systems have minds — what a given system believes, wants, or assigns a probability to.
Editorial extensions
If this is right
- If propositional interpretability is the right target, then knowing a system's concepts is not enough: interpretability efforts must attribute full attitudes (belief, desire, credence, supposition) to specific propositions, because the same represented content can be a goal or a mere model with opposite consequences.
- Thought logging becomes a concrete, multi-decade benchmark: a meta-system that outputs a running list of an AI's relevant propositional attitudes, ideally extended with reason logging and mechanism logging.
- Each existing method is placed by what it can and cannot log: causal tracing and probing are supervised and belief-centered; sparse auto-encoders are open-ended but log features and concepts; chain of thought yields pre-interpreted propositional text but is unfaithful and restricted to chain-of-thought systems.
- Safety and ethics applications follow directly: distinguishing 'the model believes a dangerous outcome occurred' from 'the model desires that outcome' is the difference between detecting a prediction and detecting a goal.
- In principle, a complete psychosemantic theory combined with near-full knowledge of a system's algorithmic state would determine its propositional attitudes, making thought logging possible in principle.
Reading between the lines
- A testable extension the paper leaves implicit: a propositional log's quality can be scored by whether its attitude attributions pass the belief-desire-action test — do logged beliefs and goals predict the system's next actions better than the raw activations do — and by whether probing, intervention, and behavioral methods converge on the same attributions.
- The framework suggests that goal logging is the most tractable first target, because goals have a distinctive behavioral signature (persistence across counterfactual interventions), while belief attributions are where indeterminacy pressure will concentrate.
- Applied to current models, the 'generalized propositional attitudes' move predicts that many so-called hallucinations are better described as unstable, prompting-relative credences or in-between believing rather than as false beliefs — a reinterpretation of existing behavioral data that the paper discusses only in outline.
- If the thesis is right, interpretability benchmarking shifts from feature localization to log faithfulness and completeness, with leaderboards scoring attitude logs against ground-truth attributions in controlled environments with known goals.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper argues that propositional interpretability—interpreting an AI system's mechanisms and behavior in terms of propositional attitudes (beliefs, desires, credences, and their generalizations)—should be a central goal of interpretability research, complementing or extending concept-level and feature-level interpretation. It develops a taxonomy of interpretability varieties, connects the project to radical interpretation in the philosophy of mind, and proposes 'thought logging' as the central research challenge: a meta-system that takes algorithmic and environmental facts as input and outputs the AI system's propositional attitudes over time. It then assesses causal tracing, probing, sparse auto-encoders, and chain-of-thought methods as partial routes to thought logging, and replies to objections concerning AI minds, externalism, unreliability, safety, and ethics.
Significance. If the programmatic claim succeeds, the paper would help reorient interpretability research from locating concepts or features toward attributing propositional attitudes with specified contents and attitude types, and it would connect AI interpretability to psychosemantics, decision theory, and the philosophy of mind. The paper's strengths are its clear conceptual taxonomy, its explicit mapping of radical interpretation onto computational interpretation, its honest naming of limitations in current methods (supervision, fragility, ground truth, attitude coverage, faithfulness), and its identification of concrete subprojects such as reason logging and mechanism logging. It makes no empirical predictions and provides no machine-checked artifacts, so its contribution is conceptual and agenda-setting rather than experimental. The main risk is that the central challenge is underspecified: thought logging needs a determinate target class of attitudes and contents, and the paper explicitly defers the principled conditions for that class.
major comments (2)
- [§3, §6, §8] The thought-logging target proposed in §5 is a list of an AI system's propositional attitudes, but §3 introduces generalized propositional attitudes partly by stipulation ('I count ... even these non-sentential states as generalized propositional attitudes') and explicitly defers their precise conditions ('it takes some work to precisely define the conditions for these or any other propositional attitudes'). Section 6 then concedes that 'we do not yet have anything close to a complete psychosemantic theory' and that 'it isn't obvious that such a theory is possible.' The target is therefore underdetermined: information-based and use-based psychosemantic principles can disagree about whether a given activation represents p or q, or whether it is belief-like or desire-like, so the resulting log would reflect the interpreter's chosen framework rather than a fact about the system. The manuscript should either provide a working definition of the generalized attitudes to be logged, or explicitly recast thought logging as a research program with a defined target class and an acceptance criterion that does not presuppose a complete psychosemantics.
- [§6] The feasibility argument for thought logging is strictly conditional: 'suppose that one day we do [have a complete psychosemantic theory] ... Then we ought to be able ... to determine the system's propositional attitudes.' The paper gives no positive reason to expect the antecedent and concedes that a complete theory may be impossible. Since §5 claims that at least partial progress should be possible, the manuscript needs a separate argument or a concrete demonstration—for example, logging propositional attitudes in a small, fully observable system under an explicitly stated partial psychosemantics—to support the claim that thought logging is a tractable research program rather than a merely hypothetical one.
minor comments (3)
- [Footnotes 13, 15, 21] The manuscript contains unresolved draft placeholders: footnote 13 says 'A paragraph on radical vs nonradical interpretation is needed here,' and footnotes 15 and 21 promise a 'new section 8' that does not exist in this form; these must be resolved before publication.
- [§3, §7.2, §7.3, §8] There are several typos: 'I count will count' in §3, 'noher method' and 'on to probe' in §7.2, 'Ths raises' in §7.3, and 'albeitly' and 'journal's' in §8.
- [§5] The example log would be clearer if the attitude names were explicitly marked as generalized propositional attitudes and if 'Judge' as a label for a credence ascription were defined on first use.
Circularity Check
No significant circularity: the paper is a conceptual proposal whose central conditional is explicitly hypothetical.
full rationale
This paper does not derive empirical predictions from fitted parameters, and its central claims are not forced by construction. Thought logging is introduced as a definitional research challenge rather than as a result derived from prior assumptions, and the paper candidly states that the enabling psychosemantic theory is incomplete: 'We do not yet have anything close to a complete psychosemantic theory' and 'it isn't obvious that such a theory is possible.' The argument that thought logging would be possible if a complete psychosemantic theory plus complete algorithmic knowledge were available is an explicit conditional, not a circular derivation. The author's self-citations (e.g., on narrow content and inferentialism) are peripheral and not load-bearing for the central proposal. The acknowledged gap concerning precise conditions for generalized propositional attitudes is a completeness limitation, not a circularity. Therefore the appropriate finding is no significant circularity.
Assumptions & free parameters
assumptions (4)
- domain assumption Propositional attitudes are the central way to interpret and explain human beings.
- domain assumption AI systems can have generalized propositional attitudes such as goals and models without having minds.
- domain assumption A complete psychosemantic theory, or at least partial theories, is possible in principle.
- domain assumption Knowledge of algorithmic facts plus environmental facts is sufficient in principle for determining propositional attitudes.
invented entities (1)
-
thought logging system
Cite this review
Pith. "Pith review of Propositional Interpretability in Artificial Intelligence." pith.science (2026). https://pith.science/paper/HEVFZ4KA
@misc{pith2026250115740,
author = {Pith},
title = {Pith review of: Propositional Interpretability in Artificial Intelligence},
year = {2026},
howpublished = {\url{https://pith.science/paper/HEVFZ4KA}},
note = {Machine review of arXiv:2501.15740}
}
read the original abstract
Mechanistic interpretability is the program of explaining what AI systems are doing in terms of their internal mechanisms. I analyze some aspects of the program, along with setting out some concrete challenges and assessing progress to date. I argue for the importance of propositional interpretability, which involves interpreting a system's mechanisms and behavior in terms of propositional attitudes: attitudes (such as belief, desire, or subjective probability) to propositions (e.g. the proposition that it is hot outside). Propositional attitudes are the central way that we interpret and explain human beings and they are likely to be central in AI too. A central challenge is what I call thought logging: creating systems that log all of the relevant propositional attitudes in an AI system over time. I examine currently popular methods of interpretability (such as probing, sparse auto-encoders, and chain of thought methods) as well as philosophical methods of interpretation (including those grounded in psychosemantics) to assess their strengths and weaknesses as methods of propositional interpretability.
Forward citations
Cited by 4 Pith papers
-
A Behavioural and Representational Evaluation of Goal-Directedness in Language Model Agents
An LLM navigation agent encodes a coarse spatial map and multi-step plans in its activations, and reasoning shifts these representations from broad environment information to immediate action selection.
-
Explaining Neural Networks with Reasons
A new interpretability method computes 'reasons vectors' from neuron activations and measures how strongly each neuron supports propositions about the input, with experiments on MNIST, Adult, and SST2.
-
Deflating Deflationism: A Critical Perspective on Debunking Arguments Against LLM Mentality
The paper defends 'modest inflationism' about LLM mentality: folk ascriptions of beliefs and desires can be defeasibly legitimate, while phenomenal consciousness remains a stretch.
-
Evaluating Explanations: An Explanatory Virtues Framework for Mechanistic Interpretability -- The Strange Science Part I.ii
The paper introduces an Explanatory Virtues Framework and argues, via a qualitative rubric, that Compact Proofs are the most promising method for mechanistic interpretability.
Reference graph
Works this paper leans on
-
[4]
Discover ing latent knowledge in language models without supervision. arXiv:2212.03827. Chalmers, D.J
-
[6]
How do language models bind entities in context? arXiv:2310.17191. Feng, J., Russell, S. & Steinhardt, J
-
[9]
Operationalising Representation in Natural Language Processing
Operationalising representation in natu ral language processing. British Journal for the Philosophy of Science . arXiv:2306.08193. He, Z. et al
-
[10]
arXiv:2408.1266 4 Herrmann, D.A
Multilevel interpretability of artificial neural networks: leveraging framework and methods from neuroscience. arXiv:2408.1266 4 Herrmann, D.A. & Levinstein, B.A
-
[11]
arXiv:2405.21030 Hoelscher-Obermaier, J., Persson, O
Standards for belie f representations in LLMs. arXiv:2405.21030 Hoelscher-Obermaier, J., Persson, O. & H¨ olscher, J
-
[12]
On the attribution of confidence to large language models
On the attribution of confidenc e to large language models. arXiv:2407.08388. Lederman, H. & Mahowald, K
-
[13]
Emer- gent world representations: Exploring a sequence model tra ined on a synthetic task. ICLR. 26 arXiv:2210.13382. Lindsay, G.W . and Bau, D
- [14]
Show all 23 references
-
[15]
Meng, K., Bau, D., Andonian, A., and Belinkov, Y
Might text-davinci-003 have i nner speech? Think 23 (67):31-38. Meng, K., Bau, D., Andonian, A., and Belinkov, Y . 2022a. Loca ting and editing factual associations in GPT. arXiv:2202.05262. Meng, K., Sharma, A.S., Andonian, A., Belinkov, Y ., and Bau, D. 2022b. Mass-Editing M...
-
[16]
arXiv:2410.13928 Piantadosi, S
Automati cally interpreting millions of features in large language models. arXiv:2410.13928 Piantadosi, S. T., & Hill, F
-
[17]
arXiv:2208.02957
Meaning without referenc e in large language models. arXiv:2208.02957. Quine, W .V . 1960.W ord and Object. MIT Press. Ramsey, W ., Stich, S., and Garon, J
1960 arXiv
- [18]
-
[19]
The Splintered Mind (November 30, 2023)
How we will decide that large langua ge models have beliefs. The Splintered Mind (November 30, 2023). Shanahan, M
2023
-
[20]
arXiv preprint arXiv:2212.03551
Talking about large language models. arXiv preprint arXiv:2212.03551. Stalnaker, R
-
[22]
arXiv:2305.04388 Vilas, M.G., Adolfi, E
Langu age models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting. arXiv:2305.04388 Vilas, M.G., Adolfi, E. Poeppel, D. & Roig, G
-
[23]
arX iv:2406.01352 Williams, J.R.G
An inner in terpretability framework for AI inspired by lessons from cognitive neuroscience. arX iv:2406.01352 Williams, J.R.G
-
[1984]
MIT Press
Inquiry. MIT Press. Templeton, A. et al (2024. Scaling monosemanticity: Mappin g the mind of a large lan- guage model. https: //transformer-circuits.pub/2024/scaling-monosemanticity/. Thibodeau, J
2024
-
[1987]
MIT Press
Psychosemantics: The Problem of Meaning in the Philosophy o f Mind. MIT Press. Goldstein, S. & Levinstein, B.A. forthcoming. Does ChatGPT have a mind? arXiv:2407.11015. Grzankowski, A. & Montague. M
-
[2017]
arXiv:1702.08608
Towards a rigorous science o f interpretable machine learning. arXiv:1702.08608. Dretske, F
-
[2021]
610-623)
On the dangers of stochastic parrots: Can language models be too big? Proceedings of the 2021 ACM Confer- ence on Fairness, Accountability, and Transparency (pp. 610-623). Bender, E. M., & Koller, A
2021
-
[2022]
Bender, E
ISSN 0891-2017. Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S
2017
-
[2023]
In Findings of the Association for Computational Linguistics : EMNLP 2023 , pp
The internal state of an LLM k nows when it’s lying. In Findings of the Association for Computational Linguistics : EMNLP 2023 , pp. 967-976. Association for Computational Linguistics. Belinkov, Y
2023
-
[2024]
arXiv:2406.19501
Monitoring late nt world states in language models with propositional probes. arXiv:2406.19501. Fodor, J. A
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.