Contextual Role Modulates Object Representational Geometry in the Human Brain
Pith reviewed 2026-05-25 03:00 UTC · model grok-4.3
The pith
The same object recruits different brain networks and organizes its internal geometry by action affordances or semantic dimensions depending on whether it is the target of a goal-directed action or a passive scene element.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
When the same objects function as targets of goal-directed actions they engage a parietal action network centered in the supramarginal and postcentral gyri and their representations are organized along action affordance and hand posture affordance dimensions; when the same objects are passive elements they recruit a distributed occipito-temporal network and their representations align with semantic dimensions; visual representational structure is invariant to context, and representational content outside the context-specific networks retains context-invariance.
What carries the argument
Context-specific representational geometry that produces a double dissociation: action-target objects sorted by affordance dimensions inside parietal networks versus passive objects sorted by semantic dimensions inside occipito-temporal networks.
If this is right
- Object representations are dynamically remapped according to moment-to-moment contextual relevance within a naturalistic scene.
- Flexibility in representational geometry and invariance in visual structure operate at different levels of the same system.
- Representational content outside the networks most strongly engaged by each context remains largely context-invariant.
- Action affordance and hand posture dimensions organize geometry only inside the networks recruited when objects are action targets.
Where Pith is reading between the lines
- The same logic could be tested by asking whether other contextual manipulations, such as whether an object is to be grasped versus verbally described, produce analogous dissociations in geometry.
- If the dissociation holds, computational models of object recognition would need separate pathways or modulation terms that switch representational axes according to current task relevance.
- The finding suggests that studies using only static or decontextualized images may miss the affordance-based organization that appears under naturalistic action conditions.
Load-bearing premise
The naturalistic movie clips cleanly separate the difference between passive and action-target objects without low-level visual features, motion, or attention differences also driving which networks are engaged or how their geometries are arranged.
What would settle it
A controlled experiment that presents the identical objects in passive versus action-target roles while matching low-level visual statistics, motion energy, and attentional demands across conditions and still fails to produce the reported double dissociation in representational geometry would falsify the claim that contextual role alone drives the remapping.
read the original abstract
The human brain represents objects in a way that is both invariant across instances and flexible enough to support different contexts and tasks. Yet it remains unknown how object representations are dynamically remapped as the same object shifts across contextual roles. Here we combined fMRI with naturalistic movie viewing to investigate how the same objects are represented when they are passive elements in the scene versus the targets of goal-directed actions. When objects were action targets, they engaged a parietal action network centered in the supramarginal and postcentral gyri, while passive objects recruited a distributed occipito-temporal network involved in visual object recognition. Within the networks most strongly encoding objects in their respective contexts, representational geometry showed a double dissociation: target object representations were organized by action affordance and hand posture affordance dimensions, while passive object representations aligned with semantic dimensions. In addition, visual representational structure was invariant to context. Outside those context-specific brain networks, representational content retained context-invariance, indicating that flexibility and invariance operate at different levels of the same representational system. Together, these findings demonstrate neural remapping of object representational geometries in a manner that depends on moment-to-moment changes in the contextual relevance of objects within a naturalistic scene.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper uses fMRI during naturalistic movie viewing to examine how the same objects are represented when they serve as passive scene elements versus targets of goal-directed actions. It reports that action targets preferentially engage a parietal action network (supramarginal and postcentral gyri) whose representational geometry is organized along action-affordance and hand-posture dimensions, while passive objects engage an occipito-temporal visual-object network whose geometry aligns with semantic dimensions. Visual representational structure remains invariant to context, and context-invariance is preserved outside the context-specific networks, indicating that flexibility and invariance operate at different levels of the same system.
Significance. If the central double-dissociation result survives appropriate confound controls, the work provides direct evidence that contextual role dynamically remaps object representational geometry in a network-specific manner within ecologically valid stimuli. The naturalistic design and the reported dissociation between context-sensitive and context-invariant representational levels are strengths that could advance models of flexible object coding.
major comments (2)
- [Abstract] Abstract and implied Methods: The double-dissociation claim (target objects organized by action/hand-posture affordances in parietal cortex vs. semantic dimensions for passive objects in occipito-temporal cortex) is load-bearing for the central thesis, yet the manuscript provides no description of stimulus matching, motion-energy regression, attentional salience controls, or eye-tracking data to isolate contextual role from co-varying low-level visual and motion features that differ between action-target and passive conditions.
- [Results] Results (double-dissociation geometry analysis): The paper must report participant numbers, the precise RSA or geometry metrics used, the statistical tests establishing the double dissociation, and any correction for multiple comparisons across networks and dimensions; without these details the reported organization by affordance vs. semantic dimensions cannot be evaluated for robustness.
Simulated Author's Rebuttal
We thank the referee for their constructive comments, which have helped us strengthen the methodological transparency of the manuscript. We address each major point below and indicate the revisions made.
read point-by-point responses
-
Referee: [Abstract] Abstract and implied Methods: The double-dissociation claim (target objects organized by action/hand-posture affordances in parietal cortex vs. semantic dimensions for passive objects in occipito-temporal cortex) is load-bearing for the central thesis, yet the manuscript provides no description of stimulus matching, motion-energy regression, attentional salience controls, or eye-tracking data to isolate contextual role from co-varying low-level visual and motion features that differ between action-target and passive conditions.
Authors: We agree that explicit documentation of these controls is necessary. The revised Methods section now details stimulus matching for low-level visual features (luminance, contrast, and spatial frequency) between matched object instances across conditions, performed via automated frame-by-frame video analysis. Motion-energy regressors derived from the movie stimuli were included in the first-level GLM. Post-hoc analysis using a computational saliency model confirmed comparable attentional salience between conditions. Eye-tracking data were not collected; however, the context-invariant geometry observed in early visual cortex and the network-specific nature of the dissociation provide converging evidence against low-level confounds. These additions are incorporated in the revision. revision: partial
-
Referee: [Results] Results (double-dissociation geometry analysis): The paper must report participant numbers, the precise RSA or geometry metrics used, the statistical tests establishing the double dissociation, and any correction for multiple comparisons across networks and dimensions; without these details the reported organization by affordance vs. semantic dimensions cannot be evaluated for robustness.
Authors: We apologize for the initial omission of these reporting details. The revised manuscript now states that data were collected from 24 participants. Representational geometry was quantified via RSA using Pearson correlation distance on object-specific activation patterns, with subsequent multidimensional scaling for visualization. The double dissociation was tested with a linear mixed-effects model including context (target vs. passive) and dimension type (affordance vs. semantic) as factors; the critical interaction was evaluated with permutation testing (10,000 iterations) and FDR correction across the four networks and three dimensions examined. Exact statistics and corrected p-values are now reported in Results. revision: yes
Circularity Check
No circularity: purely empirical fMRI analysis with no derivations or self-referential reductions
full rationale
The manuscript reports empirical results from fMRI during naturalistic movie viewing, identifying context-dependent network engagement and representational geometries via standard data-driven methods (e.g., RSA-style geometry analyses). No equations, parameter fitting, predictions derived from fitted inputs, or self-citation chains are described that would reduce any claim to its own inputs by construction. The double-dissociation and invariance findings are presented as outcomes of the data rather than analytic necessities. This is the expected non-finding for an observational neuroimaging study without theoretical modeling steps.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption fMRI BOLD signals can be used to recover representational geometry via similarity analysis
Lean theorems connected to this paper
-
IndisputableMonolith/Foundation/RealityFromDistinction.leanreality_from_one_distinction unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
representational geometry showed a double dissociation: target object representations were organized by action affordance and hand posture affordance dimensions, while passive object representations aligned with semantic dimensions. In addition, visual representational structure was invariant to context.
-
IndisputableMonolith/Cost/FunctionalEquation.leanwashburn_uniqueness_aczel unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
voxel-wise encoding models using five feature spaces: passive objects, target objects, action labels, hand synergy weights, and motion energy
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Reference graph
Works this paper leans on
-
[1]
Hebart, M. N. et al. THINGS: A database of 1,854 object concepts and more than 26,000 naturalistic object images. PloS One 14, e0223792 (2019)
work page 2019
-
[2]
Kriegeskorte, N. et al. Matching categorical object representations in inferior temporal cortex of man and monkey. Neuron 60, 1126–1141 (2008)
work page 2008
-
[3]
Haxby, J. V. et al. A common, high-dimensional model of the representational space in human ventral temporal cortex. Neuron 72, 404–416 (2011)
work page 2011
-
[4]
Mishkin, M., Ungerleider, L. G. & Macko, K. A. Object vision and spatial vision: two cortical pathways. Trends Neurosci. 6, 414–417 (1983)
work page 1983
-
[5]
Merigan, W. H. & Maunsell, J. How parallel are the primate visual pathways? Annu. Rev. Neurosci. https://psycnet.apa.org/record/1993-32418-001 (1993)
work page 1993
-
[6]
Goodale, M. A. & Milner, A. D. Separate visual pathways for perception and action. Trends Neurosci. 15, 20–25 (1992)
work page 1992
-
[7]
Grill-Spector, K., Kourtzi, Z. & Kanwisher, N. The lateral occipital complex and its role in object recognition. Vision Res. 41, 1409–1422 (2001)
work page 2001
-
[8]
The Representation of Object Concepts in the Brain
Martin, A. The Representation of Object Concepts in the Brain. Annu. Rev. Psychol. 58, 25–45 (2007)
work page 2007
-
[9]
Grill-Spector, K. & Weiner, K. S. The functional architecture of the ventral temporal cortex and its role in categorization. Nat. Rev. Neurosci. 15, 536–548 (2014)
work page 2014
-
[10]
De Haan, E. H. & Cowey, A. On the usefulness of ‘what’and ‘where’pathways in vision. Trends Cogn. Sci. 15, 460–466 (2011)
work page 2011
-
[11]
Gallivan, J. P., Cant, J. S., Goodale, M. A. & Flanagan, J. R. Representation of object weight in human ventral visual cortex. Curr. Biol. 24, 1866–1873 (2014)
work page 2014
-
[12]
Kravitz, D. J., Saleem, K. S., Baker, C. I. & Mishkin, M. A new neural framework for visuospatial processing. Nat. Rev. Neurosci. 12, 217–230 (2011). 27
work page 2011
-
[13]
Burgess, N., Maguire, E. A., Spiers, H. J. & O’Keefe, J. A temporoparietal and prefrontal network for retrieving the spatial context of lifelike events. Neuroimage 14, 439–453 (2001)
work page 2001
-
[14]
Rizzolatti, G. & Matelli, M. Two different streams form the dorsal visual system: anatomy and functions. Exp. Brain Res. 153, 146–157 (2003)
work page 2003
-
[15]
Binkofski, F. & Buxbaum, L. J. Two action systems in the human brain. Brain Lang. 127, 222– 229 (2013)
work page 2013
-
[16]
Xu, Y. A Tale of Two Visual Systems: Invariant and Adaptive Visual Information Representations in the Primate Brain. Annu. Rev. Vis. Sci. 4, 311–336 (2018)
work page 2018
-
[17]
Kravitz, D. J., Saleem, K. S., Baker, C. I., Ungerleider, L. G. & Mishkin, M. The ventral visual pathway: an expanded neural framework for the processing of object quality. Trends Cogn. Sci. 17, 26–49 (2013)
work page 2013
-
[18]
Ralph, M. A. L., Jefferies, E., Patterson, K. & Rogers, T. T. The neural and computational bases of semantic cognition. Nat. Rev. Neurosci. 18, 42–55 (2017)
work page 2017
-
[19]
Culham, J. C. & Valyear, K. F. Human parietal cortex in action. Curr. Opin. Neurobiol. 16, 205– 212 (2006)
work page 2006
-
[20]
Caspers, S. et al. The human inferior parietal cortex: cytoarchitectonic parcellation and interindividual variability. Neuroimage 33, 430–448 (2006)
work page 2006
-
[21]
Caspers, S. et al. The human inferior parietal lobule in stereotaxic space. Brain Struct. Funct. 212, 481–495 (2008)
work page 2008
-
[22]
Buxbaum, L. J., Kyle, K. M. & Menon, R. On beyond mirror neurons: Internal representations subserving imitation and recognition of skilled object-related actions in humans. Cogn. Brain Res. 25, 226–239 (2005)
work page 2005
-
[23]
Buxbaum, L. J., Johnson -Frey, S. H. & Bartlett -Williams, M. Deficient internal models for planning hand–object interactions in apraxia. Neuropsychologia 43, 917–929 (2005)
work page 2005
-
[24]
Rizzolatti, G. & Craighero, L. THE MIRROR -NEURON SYSTEM. Annu. Rev. Neurosci. 27, 169–192 (2004). 28
work page 2004
-
[25]
Tranel, D., Kemmerer, D., Adolphs, R., Damasio, H. & Damasio, A. R. NEURAL CORRELATES OF CONCEPTUAL KNOWLEDGE FOR ACTIONS. Cogn. Neuropsychol. 20, 409–432 (2003)
work page 2003
-
[26]
Mahon, B. Z. & Almeida, J. Reciprocal interactions among parietal and occipito -temporal representations support everyday object -directed actions. Neuropsychologia 198, 108841 (2024)
work page 2024
-
[27]
Binder, J. R., Desai, R. H., Graves, W. W. & Conant, L. L. Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. Cereb. Cortex 19, 2767–2796 (2009)
work page 2009
- [28]
-
[29]
Hebart, M. N., Bankson, B. B., Harel, A., Baker, C. I. & Cichy, R. M. The representational dynamics of task and object processing in humans. elife 7, e32816 (2018)
work page 2018
-
[30]
Xu, Y. et al. Doctor, teacher, and stethoscope: neural representation of different types of semantic relations. J. Neurosci. 38, 3303–3317 (2018)
work page 2018
-
[31]
Wang, X. et al. Representational similarity analysis reveals task -dependent semantic influence of the visual word form area. Sci. Rep. 8, 3047 (2018)
work page 2018
-
[32]
Kiefer, M. Perceptual and semantic sources of category -specific effects: Event -related potentials during picture and word categorization. Mem. Cognit. 29, 100–116 (2001)
work page 2001
-
[33]
Harel, A., Kravitz, D. J. & Baker, C. I. Task context impacts visual object processing differentially across the cortex. Proc. Natl. Acad. Sci. 111, (2014)
work page 2014
-
[34]
Bar, M. & Ullman, S. Spatial Context in Recognition. Perception 25, 343–352 (1996)
work page 1996
-
[35]
Harel, A. & Bentin, S. Stimulus type, level of categorization, and spatial-frequencies utilization: implications for perceptual categorization hierarchies. J. Exp. Psychol. Hum. Percept. Perform. 35, 1264 (2009). 29
work page 2009
-
[36]
Schyns, P. G. & Oliva, A. Dr. Angry and Mr. Smile: When categorization flexibly modifies the perception of faces in rapid visual presentations. Cognition 69, 243–265 (1999)
work page 1999
-
[37]
Beck, D. M. & Kastner, S. Stimulus context modulates competition in human extrastriate cortex. Nat. Neurosci. 8, 1110–1116 (2005)
work page 2005
-
[38]
Chen, A. J.-W. et al. Goal-directed attention alters the tuning of object-based representations in extrastriate cortex. Front. Hum. Neurosci. 6, 187 (2012)
work page 2012
-
[39]
Gazzaley, A., Cooney, J. W., McEvoy, K., Knight, R. T. & D’esposito, M. Top -down enhancement and suppression of the magnitude and speed of neural activity. J. Cogn. Neurosci. 17, 507–517 (2005)
work page 2005
-
[40]
O’Craven, K. M., Downing, P. E. & Kanwisher, N. fMRI evidence for objects as the units of attentional selection. Nature 401, 584–587 (1999)
work page 1999
-
[41]
Murray, S. O. & Wojciulik, E. Attention increases neural selectivity in the human lateral occipital complex. Nat. Neurosci. 7, 70–74 (2004)
work page 2004
-
[42]
Desimone, R. & Duncan, J. Neural mechanisms of selective visual attention. Annu. Rev. Neurosci. 18, 193–222 (1995)
work page 1995
-
[43]
Miller, E. K., Freedman, D. J. & Wallis, J. D. The prefrontal cortex: categories, concepts and cognition. Philos. Trans. R. Soc. Lond. B. Biol. Sci. 357, 1123–1136 (2002)
work page 2002
-
[44]
Çukur, T., Nishimoto, S., Huth, A. G. & Gallant, J. L. Attention during natural vision warps semantic representation across the human brain. Nat. Neurosci. 16, 763–770 (2013)
work page 2013
-
[45]
Toneva, M., Stretcu, O., Póczos, B., Wehbe, L. & Mitchell, T. M. Modeling task effects on meaning representation in the brain via zero -shot meg prediction. Adv. Neural Inf. Process. Syst. 33, 5284–5295 (2020)
work page 2020
-
[46]
D., Eickenberg, M., Nunez-Elizalde, A
La Tour, T. D., Eickenberg, M., Nunez-Elizalde, A. O. & Gallant, J. L. Feature-space selection with banded ridge regression. NeuroImage 264, 119728 (2022)
work page 2022
-
[47]
Smith, F. W. & Goodale, M. A. Decoding visual object categories in early somatosensory cortex. Cereb. Cortex 25, 1020–1031 (2015). 30
work page 2015
-
[48]
Caspers, S., Zilles, K., Laird, A. R. & Eickhoff, S. B. ALE meta-analysis of action observation and imitation in the human brain. Neuroimage 50, 1148–1167 (2010)
work page 2010
-
[49]
R., Walbrin, J., Akwayena, E., Almeida, J
Caglar, L. R., Walbrin, J., Akwayena, E., Almeida, J. & Mahon, B. Z. Object -directed action representations are componentially built in parietal cortex. Proc. Natl. Acad. Sci. 122, e2421032122 (2025)
work page 2025
-
[50]
Cant, J. S. & Goodale, M. A. Attention to form or surface properties modulates different regions of human occipitotemporal cortex. Cereb. Cortex 17, 713–731 (2007)
work page 2007
-
[51]
Peelen, M. V. & Downing, P. E. Selectivity for the Human Body in the Fusiform Gyrus. J. Neurophysiol. 93, 603–608 (2005)
work page 2005
-
[52]
Schwarzlose, R. F., Baker, C. I. & Kanwisher, N. Separate face and body selectivity on the fusiform gyrus. J. Neurosci. 25, 11055–11059 (2005)
work page 2005
-
[53]
Weiner, K. S. & Grill-Spector, K. Not one extrastriate body area: using anatomical landmarks, hMT+, and visual field maps to parcellate limb -selective activations in human lateral occipitotemporal cortex. Neuroimage 56, 2183–2199 (2011)
work page 2011
-
[54]
Contribution of the posterior parietal cortex in reaching, grasping, and using objects and tools
Vingerhoets, G. Contribution of the posterior parietal cortex in reaching, grasping, and using objects and tools. Front. Psychol. 5, 151 (2014)
work page 2014
-
[55]
Garcea, F. E. et al. Domain-specific diaschisis: lesions to parietal action areas modulate neural responses to tools in the ventral stream. Cereb. Cortex 29, 3168–3181 (2019)
work page 2019
-
[56]
Chao, L. L. & Martin, A. Representation of manipulable man -made objects in the dorsal stream. Neuroimage 12, 478–484 (2000)
work page 2000
-
[57]
Mahon, B. Z. et al. Action-related properties shape object representations in the ventral stream. Neuron 55, 507–520 (2007)
work page 2007
- [58]
-
[59]
Xiao, S. et al. C-Pack: Packed Resources For General Chinese Embeddings. in Proceedings of the 47th International ACM SIGIR Conference on Research and Development in 31 Information Retrieval 641–649 (Association for Computing Machinery, New York, NY, USA, 2024). doi:10.1145/3626772.3657878
-
[60]
Dirani, J. & Pylkkänen, L. The time course of cross -modal representations of conceptual categories. Neuroimage 277, 120254 (2023)
work page 2023
-
[61]
Dirani, J. & Pylkkänen, L. MEG evidence that modality -independent conceptual representations contain semantic and visual features. J. Neurosci. 44, (2024)
work page 2024
-
[62]
Gibson, J. J. The theory of affordances. Hilldale USA 1, 67–82 (1977)
work page 1977
-
[63]
Yee, E. & Thompson-Schill, S. L. Putting concepts into context. Psychon. Bull. Rev. 23, 1015– 1027 (2016)
work page 2016
-
[64]
Mahon, B. Z. What is embodied about cognition? Lang. Cogn. Neurosci. 30, 420–429 (2015)
work page 2015
-
[65]
Nastase, S. A. et al. Attention selectively reshapes the geometry of distributed semantic representation. Cereb. Cortex 27, 4277–4291 (2017)
work page 2017
-
[66]
Kanwisher, N. & Wojciulik, E. Visual attention: Insights from brain imaging. Nat. Rev. Neurosci. 1, 91–100 (2000)
work page 2000
-
[67]
David, S. V., Hayden, B. Y., Mazer, J. A. & Gallant, J. L. Attention to Stimulus Features Shifts Spectral Tuning of V4 Neurons during Natural Vision. Neuron 59, 509–521 (2008)
work page 2008
-
[68]
Mazer, J. A. & Gallant, J. L. Goal -Related Activity in V4 during Free Viewing Visual Search: Evidence for a Ventral Stream Visual Salience Map. Neuron 40, 1241–1250 (2003)
work page 2003
-
[69]
Stuss, D. T. & Benson, D. F. The Frontal Lobes and Control of Cognition and Memory. in The Frontal Lobes Revisited (Psychology Press, 1987)
work page 1987
- [70]
-
[71]
Damen, D. et al. Rescaling Egocentric Vision: Collection, Pipeline and Challenges for EPIC - KITCHENS-100. Int. J. Comput. Vis. 130, 33–55 (2022)
work page 2022
-
[72]
Damen, D. et al. Scaling Egocentric Vision: The EPIC -KITCHENS Dataset. Preprint at https://doi.org/10.48550/arXiv.1804.02748 (2018). 32
work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.1804.02748 2018
-
[73]
CVAT. ai Corporation. Computer Vision Annotation Tool (CVAT). Zenodo https://doi.org/10.5281/ZENODO.12771595 (2024)
-
[74]
Zhang, F. et al. MediaPipe Hands: On -device Real -time Hand Tracking. Preprint at https://doi.org/10.48550/arXiv.2006.10214 (2020)
-
[75]
O., Deniz, F., Dupré la Tour, T., Visconti di Oleggio Castello, M
Nunez-Elizalde, A. O., Deniz, F., Dupré la Tour, T., Visconti di Oleggio Castello, M. & Gallant, J. L. Pymoten: scientific Python package for computing motion energy features from video. Zenodo (2021)
work page 2021
-
[76]
Cox, R. W. AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Comput. Biomed. Res. 29, 162–173 (1996)
work page 1996
-
[77]
Cox, R. W. & Hyde, J. S. Software tools for analysis and visualization of fMRI data. NMR Biomed. 10, 171–178 (1997)
work page 1997
- [78]
-
[79]
Gao, J. S., Huth, A. G., Lescroart, M. D. & Gallant, J. L. Pycortex: an interactive surface visualizer for fMRI. Front. Neuroinformatics 9, 23 (2015)
work page 2015
-
[80]
Lancaster, J. L. et al. Automated labeling of the human brain: A preliminary report on the development and evaluation of a forward -transform method. Hum. Brain Mapp. 5, 238–242 (1997)
work page 1997
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.