Use and usability: concepts of representation in philosophy, neuroscience, cognitive science, and computer science
Pith reviewed 2026-05-10 12:31 UTC · model grok-4.3
The pith
Representations are understood at three levels based on whether their information is useful, in usable format, and actually used.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Building on four aspects of information and use, existing perspectives on neural representations are organized into three levels: Representations as Information (Level 1); Representations as Usable (Level 2); and Representations as Used (Level 3). The account helps appreciate the diversity of notions of neural representation and clarify the appropriate notion for specific investigations.
What carries the argument
The taxonomy of three levels of representation supported by the four aspects of information, usefulness, usable format, and downstream use.
If this is right
- Studies of neural codes can be classified by whether they address mere information presence or actual usability.
- Models in AI can be evaluated on which level of representation they implement.
- Philosophical arguments about representation can be mapped to these levels to resolve apparent disagreements.
- Investigators can choose the level appropriate to their research question to avoid mismatch.
Where Pith is reading between the lines
- This organization could reveal gaps in current research where certain levels are understudied.
- It might suggest new experiments that test the transition from usable to used representations.
- Connecting to neighboring problems like the role of representations in learning or decision making could extend the framework.
Load-bearing premise
The four aspects comprehensively capture the relevant dimensions of representation across the disciplines without significant omissions.
What would settle it
Discovery of a conception of representation in one of the fields that fits none of the three levels or requires a fifth aspect of use.
read the original abstract
Representations play a central role in the study of both biological and artificial intelligence, as well as philosophy of mind. Across neuroscience, computer science, and philosophy, a recurring theme is that representations not only carry information but should be ``useful'' for or ``usable'' by an agent in some sense. Here, we review how the ``usefulness'' of representations has been conceptualized and how it figures into different conceptions of representation. We identify and explore four aspects of use and usability: representations generally carry \textit{information}; that information may or may not be \textit{useful} and it may or may not be encoded in a usable \textit{format}; and the representations may or may not be \textit{used downstream}. Building on these four aspects of information and use, we then organize existing perspectives on neural representations into three levels: Representations as Information (Level 1); Representations as Usable (Level 2); and Representations as Used (Level 3). Our account is meant to give readers an appreciation for the diversity of notions of ``neural representation,'' help them navigate the vast and multi-disciplinary literature on the topic, and help them clarify the appropriate notion of representation for their own investigations.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript reviews how usefulness and usability figure into conceptions of representation across philosophy, neuroscience, cognitive science, and computer science. It identifies four aspects—representations carry information; that information may or may not be useful; it may or may not be encoded in a usable format; and representations may or may not be used downstream—and employs these to organize existing perspectives on neural representations into three levels: Representations as Information (Level 1), Representations as Usable (Level 2), and Representations as Used (Level 3). The account is intended to help readers appreciate the diversity of notions and navigate the multidisciplinary literature.
Significance. If the taxonomy holds, the paper supplies a coherent organizational framework for a sprawling interdisciplinary topic without introducing new parameters, axioms, or empirical claims. Its value lies in the explicit linkage of the four aspects to the three levels and the cross-field synthesis, which can assist researchers in selecting defensible notions of representation for their own work. The absence of self-referential derivations or fitted quantities is a strength for a review of this type.
minor comments (2)
- A summary table mapping the four aspects to the three levels, with one or two canonical citations per cell, would improve readability and make the organizational claim easier to apply.
- The manuscript should explicitly note any disciplinary literatures (e.g., embodied cognition or social epistemology) that fall outside the four-aspect framing, even if only to delimit scope.
Simulated Author's Rebuttal
We thank the referee for their positive assessment of the manuscript, including the recognition of its value as a cross-disciplinary organizational framework for notions of neural representation. We appreciate the recommendation for minor revision and note that no specific major comments were raised in the report.
Circularity Check
No significant circularity in this conceptual review and taxonomy
full rationale
This paper is a review and synthesis that identifies four aspects of representation (information carried, usefulness, usable format, downstream use) drawn from existing multi-disciplinary literature and uses them to organize prior perspectives into three levels. The central contribution is the coherence and utility of this framing rather than any derivation, model, or empirical claim. No equations, fitted parameters, or self-citation chains appear; the argument rests on external citations and does not reduce any claim to its own inputs by construction. This is the expected non-finding for a self-contained organizational review.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Representations carry information that can be evaluated for usefulness and usability.
Reference graph
Works this paper leans on
-
[1]
Fallon FT, Ryan TJ, Krakauer JW, What Are We T alking about? Clarifying the Fuzzy Concept of Representation in Neuroscience and Beyond; 2023
work page 2023
-
[2]
Three Aspects of Representation in Neuroscience
Baker B, Lansdell B, Kording KP . Three Aspects of Representation in Neuroscience. T rends in Cognitive Sciences 2022 Nov;26(11):942–958
work page 2022
-
[3]
Investigating the Concept of Representation in the Neural and Psychological Sciences
Favela LH, Machery E. Investigating the Concept of Representation in the Neural and Psychological Sciences. Frontiers in Psychology 2023 Jun;14:1–13
work page 2023
-
[4]
Commentary: Investigating the Concept of Representation in the Neural and Psychological Sciences
Richmond A. Commentary: Investigating the Concept of Representation in the Neural and Psychological Sciences. Frontiers in Psychology 2023;14
work page 2023
-
[5]
Putting Representations to Use
Cao R. Putting Representations to Use. Synthese 2022 Apr;200(2):151
work page 2022
-
[6]
Deflating Mental Representation
Egan F . Deflating Mental Representation. The Jean Nicod Lectures, Cambridge, Massachusetts: The MIT Press; 2025
work page 2025
- [7]
-
[8]
The Embodied Mind: Cognitive Science and Human Experience
Varela FJ, Thompson E, Rosch E. The Embodied Mind: Cognitive Science and Human Experience. 14. print. ed. Cam- bridge, Mass.: MIT Press; 1993
work page 1993
-
[9]
Untangling Invariant Object Recognition
DiCarlo JJ, Cox DD. Untangling Invariant Object Recognition. T rends in Cognitive Sciences 2007;11(8):333–341
work page 2007
-
[10]
Interpreting Encoding and Decoding Models
Kriegeskorte N, Douglas PK. Interpreting Encoding and Decoding Models. Current Opinion in Neurobiology 2019 Apr;55:167–179
work page 2019
-
[11]
Ivanova AA, Schrimpf M, Anzellotti S, Zaslavsky N, Fedorenko E, Isik L, Beyond Linear Regression: Mapping Models in Cognitive Neuroscience Should Align with Research Goals. arXiv; 2022
work page 2022
-
[12]
Representation Learning: A Review and New Perspectives
Bengio Y, Courville A, Vincent P . Representation Learning: A Review and New Perspectives. IEEE T ransactions on Pattern Analysis and Machine Intelligence 2013;35(8):1798–1828
work page 2013
-
[13]
A Framework for the Quantitative Evaluation of Disentangled Representations
Eastwood C, Williams CKI. A Framework for the Quantitative Evaluation of Disentangled Representations. ICLR 2018
work page 2018
-
[14]
T owards a Definition of Disentangled Represen- tations
Higgins I, Amos D, Pfau D, Racaniere S, Matthey L, Rezende D, et al. T owards a Definition of Disentangled Represen- tations. arXiv 2018;p. 1–29
work page 2018
-
[15]
Deep Learning and the Information Bottleneck Principle
Tishby N, Zaslavsky N. Deep Learning and the Information Bottleneck Principle. arXiv 2015
work page 2015
-
[16]
Emergence of Invariance and Disentanglement in Deep Representations
Achille A, Soatto S. Emergence of Invariance and Disentanglement in Deep Representations. Information Theory and Applications Workshop 2018;19:1–34
work page 2018
-
[17]
Clarifying the Conceptual Dimensions of Representation in Neuroscience
Pohl S, Walker EY, Barack DL, Lee J, Denison RN, Block N, et al. Clarifying the Conceptual Dimensions of Representation in Neuroscience. Nature Reviews Neuroscience 2026 Mar;p. 1–16
work page 2026
-
[18]
Grounded Cognition: Past, Present, and Future
Barsalou LW. Grounded Cognition: Past, Present, and Future. T opics in Cognitive Science 2010 Oct;2(4):716–724
work page 2010
-
[19]
The Modal—Amodal Distinction in the Debate on Conceptual Format
Haimovici S. The Modal—Amodal Distinction in the Debate on Conceptual Format. Philosophies 2018 Jun;3(2):7–19
work page 2018
-
[20]
Grounding Action Representations
Weber AM, Vosgerau G. Grounding Action Representations. Review of Philosophy and Psychology 2012 Mar;3(1):53– 69
work page 2012
-
[21]
Ramsey W. Representation Reconsidered. Cambridge: Cambridge University Press; 2007
work page 2007
-
[22]
Alain G, Bengio Y, Understanding Intermediate Layers Using Linear Classifier Probes. arXiv; 2018. 26 Baker, Lange, Richmond et al
work page 2018
-
[23]
Fast Readout of Object Identity from Macaque Inferior T emporal Cortex
Hung CP , Kreiman G, Poggio T, DiCarlo JJ. Fast Readout of Object Identity from Macaque Inferior T emporal Cortex. Science 2005 Nov;310(5749):863–866
work page 2005
-
[24]
Ritchie JB, Kaplan DM, Klein C. Decoding the Brain: Neural Representation and the Limits of Multivariate Pattern Analysis in Cognitive Neuroscience. The British Journal for the Philosophy of Science 2019 Jun;70(2):581–607
work page 2019
-
[25]
Diedrichsen J, Kriegeskorte N. Representational Models: A Common Framework for Understanding Encoding, Pattern- Component, and Representational-Similarity Analysis, vol. 13; 2017
work page 2017
-
[26]
What Representational Similarity Measures Imply about Decodable Information
Harvey SE, Lipshutz D, Williams AH. What Representational Similarity Measures Imply about Decodable Information. In: UniReps: 2nd Edition of the Workshop on Unifying Representations in Neural Models; 2024
work page 2024
-
[27]
Peeling the Onion of Brain Representations
Kriegeskorte N, Diedrichsen J. Peeling the Onion of Brain Representations. Annual Review of Neuroscience 2019;42:407–432
work page 2019
-
[28]
Decoding the Cognitive Map: Ensemble Hippocampal Sequences and Decision Making
Wikenheiser AM, Redish AD. Decoding the Cognitive Map: Ensemble Hippocampal Sequences and Decision Making. Current Opinion in Neurobiology 2015 Jun;32:8–15
work page 2015
-
[29]
Neuroscience Needs Behavior: Correcting a Reductionist Bias
Krakauer JW, Ghazanfar AA, Gomez-Marin A, Maciver MA, Poeppel D. Neuroscience Needs Behavior: Correcting a Reductionist Bias. Neuron Perspective 2017;93(3):480–490
work page 2017
-
[30]
Contents, Vehicles, and Complex Data Analysis in Neuroscience
Burnston DC. Contents, Vehicles, and Complex Data Analysis in Neuroscience. Synthese An International Journal for Epistemology, Methodology and Philosophy of Science 2020;199:1617–1639
work page 2020
-
[31]
Single-T rial Neural Dynamics Are Dominated by Richly Varied Movements
Musall S, Kaufman MT, Juavinett AL, Gluf S, Churchland AK. Single-T rial Neural Dynamics Are Dominated by Richly Varied Movements. Nature Neuroscience 2019 Oct;22(10):1677–1686
work page 2019
-
[32]
Context-Dependent Computation by Recurrent Dynamics in Prefrontal Cortex
Mante V, Sussillo D, Shenoy KV, Newsome WT. Context-Dependent Computation by Recurrent Dynamics in Prefrontal Cortex. Nature 2013;503(7474):78–84
work page 2013
-
[33]
Richmond A. What Is a Theory of Neural Representation For? Synthese An International Journal for Epistemology, Methodology and Philosophy of Science 2025;205(14)
work page 2025
-
[34]
Caruana R. Multitask Learning. Machine Learning 1997 Jul;28(1):41–75
work page 1997
-
[35]
Exploiting T ask Relatedness for Multiple T ask Learning
Ben-David S, Schuller R. Exploiting T ask Relatedness for Multiple T ask Learning. In: Goos G, Hartmanis J, Van Leeuwen J, Schölkopf B, Warmuth MK, editors. Learning Theory and Kernel Machines, vol. 2777 Berlin, Heidelberg: Springer Berlin Heidelberg; 2003.p. 567–580
work page 2003
-
[36]
Roberts SF, Koditschek DE, Miracchi LJ. Examples of Gibsonian Affordances in Legged Robotics Research Using an Empirical, Generative Framework. Frontiers in Neurorobotics 2020;14(February)
work page 2020
-
[37]
The Affordance Competition Hypothesis: A Framework for Embodied Behavior
Cisek P . The Affordance Competition Hypothesis: A Framework for Embodied Behavior. In: Klatzky RL, MacWhinney B, Behrmann M, editors. Embodiment, Ego-Space, and Action, 0 ed. Psychology Press; 2008
work page 2008
-
[38]
Frankle J, Carbin M, The Lottery Ticket Hypothesis: Finding Sparse, T rainable Neural Networks. arXiv; 2019
work page 2019
-
[39]
Wu HG, Miyamoto YR, Castro LNG, Ölveczky BP , Smith MA. T emporal Structure of Motor Variability Is Dynamically Regulated and Predicts Motor Learning Ability. Nature Neuroscience 2014;17(2):312–321
work page 2014
-
[40]
Neural Spiking for Causal Inference and Learning
Lansdell BJ, Kording KP . Neural Spiking for Causal Inference and Learning. PLOS Computational Biology 2023 Apr;19(4):e1011005
work page 2023
-
[41]
The Information Bottleneck Method
Tishby N, Pereira FC, Bialek W. The Information Bottleneck Method. Proceedings of the 37th Allerton Conference on Communication, Control, and Computing 1999;p. 368–377
work page 1999
-
[42]
Visual Representations: Defining Properties and Deep Approximations
Soatto S, Chiuso A. Visual Representations: Defining Properties and Deep Approximations. ICLR 2016;p. 1–20. Baker, Lange, Richmond et al. 27
work page 2016
-
[43]
Natural Image Statistics and Neural Representation
Simoncelli EP , Olshausen BA. Natural Image Statistics and Neural Representation. Annual Review of Neuroscience 2001 Mar;24(Volume 24, 2001):1193–1216
work page 2001
-
[44]
Efficient Representation as a Design Principle for Neural Coding and Computation
Bialek W, de Ruyter van Stevenick RR, Tishby N. Efficient Representation as a Design Principle for Neural Coding and Computation. IEEE International Symposium on Information Theory 2007;p. 659–663
work page 2007
-
[45]
a Olshausen B, Field DJ. Sparse Coding with an Incomplete Basis Set: A Strategy Employed by V1? Vision Research 1997;37(23):3311–3325
work page 1997
-
[46]
Possible Principles Underlying the T ransformations of Sensory Messages
Barlow HB. Possible Principles Underlying the T ransformations of Sensory Messages. Sensory Communication 1961;p. 216–234
work page 1961
-
[47]
Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems
Dayan P , Abbott LF . Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems. Se- jnowski TJ, Poggio T, editors, London: MIT Press; 2001
work page 2001
-
[48]
T oward a Unified Theory of Efficient, Predictive, and Sparse Coding
Chalk M, Marre O, Tkačik G. T oward a Unified Theory of Efficient, Predictive, and Sparse Coding. Proceedings of the National Academy of Sciences 2018;115(1):186–191
work page 2018
-
[49]
Savage LJ. The Foundations of Statistics. Courier Corporation; 1972
work page 1972
-
[50]
Decision Theory: What "Should" the Nervous System Do? Science 2008;606(2007)
Körding KP . Decision Theory: What "Should" the Nervous System Do? Science 2008;606(2007)
work page 2008
-
[51]
Probabilistic Interpretation of Population Codes
Zemel RS, Dayan P , Pouget A. Probabilistic Interpretation of Population Codes. Neural Computation 1998;10(2):403– 430
work page 1998
-
[52]
Statistically Optimal Perception and Learning: From Behavior to Neural Repre- sentations
Fiser J, Berkes P , Orbán G, Lengyel M. Statistically Optimal Perception and Learning: From Behavior to Neural Repre- sentations. T rends in cognitive sciences 2010 Mar;14(3):119–30
work page 2010
-
[53]
Probabilistic Brains: Knowns and Unknowns
Pouget A, Beck JM, Ma WJ, Latham PE. Probabilistic Brains: Knowns and Unknowns. Nature Neuroscience 2013;16(9):1170–1178
work page 2013
-
[54]
Psychosemantics: The Problem of Meaning in the Philosophy of Mind
Fodor JA. Psychosemantics: The Problem of Meaning in the Philosophy of Mind. The MIT Press; 1987
work page 1987
-
[55]
Knowledge & the Flow of Information
Dretske FI. Knowledge & the Flow of Information. 4. print ed. A Bradford Book, Cambridge, Mass: MIT Pr; 1986
work page 1986
-
[56]
Language, Thought, and Other Biological Categories: New Foundations for Realism
Millikan RG. Language, Thought, and Other Biological Categories: New Foundations for Realism. MIT Press; 1987
work page 1987
-
[57]
Knowledge in Perspective: Selected Essays in Epistemology
Sosa E. Knowledge in Perspective: Selected Essays in Epistemology. Cambridge University Press; 1991
work page 1991
-
[58]
Virtues of the Mind: An Inquiry into the Nature of Virtue and the Ethical Foundations of Knowledge
Zagzebski LT. Virtues of the Mind: An Inquiry into the Nature of Virtue and the Ethical Foundations of Knowledge. Cambridge University Press; 1996
work page 1996
-
[59]
towards Reconstruction in Epistemology
Susan H, William D, Evidence and Inquiry. towards Reconstruction in Epistemology. Oxford: Blackwell Publishers; 1993
work page 1993
-
[60]
How Does the Brain Solve Visual Object Recognition? Neuron 2012;73(3):415–434
DiCarlo JJ, Zoccolan D, Rust NC. How Does the Brain Solve Visual Object Recognition? Neuron 2012;73(3):415–434
work page 2012
-
[61]
Explicit Information for Category-Orthogonal Object Properties Increases along the Ventral Stream
Hong H, Y amins DLK, Majaj NJ, DiCarlo JJ. Explicit Information for Category-Orthogonal Object Properties Increases along the Ventral Stream. Nature Neuroscience 2016;19(4)
work page 2016
-
[62]
Poirazi P , Mel B. Memory Capacity of Linear vs. Nonlinear Models of Dendritic Integration. In: Advances in Neural Information Processing Systems, vol. 12 MIT Press; 1999
work page 1999
-
[63]
Normalization as a Canonical Neural Computation
Carandini M, Heeger DJ. Normalization as a Canonical Neural Computation. Nature Reviews Neuroscience 2012;13:51–62
work page 2012
-
[64]
Conjectures Regarding the Nonlinear Geometry of Visual Neurons
Golden JR, Vilankar KP , Wu MCK, Field DJ. Conjectures Regarding the Nonlinear Geometry of Visual Neurons. Vision Research 2016 Mar;120:74–92. 28 Baker, Lange, Richmond et al
work page 2016
-
[65]
Are Dendrites Conceptually Useful? Neuroscience 2022
Larkum ME. Are Dendrites Conceptually Useful? Neuroscience 2022
work page 2022
-
[66]
Soft Matching Distance: A Metric on Neural Representations That Captures Single-Neuron T uning
Khosla M, Williams AH. Soft Matching Distance: A Metric on Neural Representations That Captures Single-Neuron T uning. In: Proceedings of UniReps: The First Workshop on Unifying Representations in Neural Models PMLR; 2024. p. 326–341
work page 2024
-
[67]
Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologies
Achille A, Eccles T, Matthey L, Burgess CP , Watters N, Lerchner A, et al. Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologies. Advances in Neural Information Processing Systems 2018;2018- Decem(Mdl):9873–9883
work page 2018
-
[68]
A Sober Look at the Unsupervised Learning of Disen- tangled Representations and Their Evaluation
Locatello F, Bauer S, Lucic M, Rätsch G, Gelly S, Schölkopf B, et al. A Sober Look at the Unsupervised Learning of Disen- tangled Representations and Their Evaluation. The Journal of Machine Learning Research 2020 Jan;21(1):209:8629– 209:8690
work page 2020
-
[69]
van Steenkiste S, Locatello F, Schmidhuber J, Bachem O. Are Disentangled Representations Helpful for Abstract Visual Reasoning? Advances in Neural Information Processing Systems 2019;32(NeurIPS)
work page 2019
-
[70]
The Role of Disentanglement in Generalization
Montero ML, Ludwig CJ, Costa RP , Malhotra G, Bowers J. The Role of Disentanglement in Generalization. ICLR 2021
work page 2021
-
[71]
Lippl S, Stachenfeld K, When Does Compositional Structure Yield Compositional Generalization? A Kernel Theory. arXiv; 2024
work page 2024
-
[72]
Hyvarinen A, Khemakhem I, Morioka H, Nonlinear Independent Component Analysis for Principled Disentanglement in Unsupervised Deep Learning. arXiv; 2023
work page 2023
-
[73]
T oward Causal Representation Learning
Scholkopf B, Locatello F, Bauer S, Ke NR, Kalchbrenner N, Goyal A, et al. T oward Causal Representation Learning. Proceedings of the IEEE 2021;109(5):612–634
work page 2021
-
[74]
Learning Optimal Representations with the Decodable Information Bottle- neck
Dubois Y, Kiela D, Vedantam R, Schwab DJ. Learning Optimal Representations with the Decodable Information Bottle- neck. arXiv 2020
work page 2020
-
[75]
Harvard University Press; 1975
Fodor J, The Language of Thought. Harvard University Press; 1975
work page 1975
-
[76]
The Language of Thought Hypothesis
Aydede M. The Language of Thought Hypothesis. In: Zalta E, editor. Stanford Encyclopedia of Philosophy; 2010
work page 2010
-
[77]
Quilty-Dunn J, Porot N, Mandelbaum E. The Best Game in T own: The Re-emergence of the Language of Thought Hypothesis across the Cognitive Sciences. Behavioral and Brain Sciences 2022 Dec;p. 1–55
work page 2022
-
[78]
The Medium and the Message in Mental Imagery: A Theory
Kosslyn SM. The Medium and the Message in Mental Imagery: A Theory. Psychological Review 1981;88(1):46–66
work page 1981
-
[79]
Mental Rotation of Three-Dimensional Objects
Shepard RN, Metzler J. Mental Rotation of Three-Dimensional Objects. Science 1971;171(972):701–703
work page 1971
-
[80]
Iconic Representation: Maps, Pictures, and Perception
Burge T. Iconic Representation: Maps, Pictures, and Perception. In: Wuppuluri S, Doria FA, editors. The Map and the T erritory Cham: Springer International Publishing; 2018.p. 79–100
work page 2018
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.