MEG Evidence That Modality-Independent Conceptual Representations Encode Visual but Not Lexical Representations

Julien Dirani; Liina Pylkk\"anen

REVIEW 2 major objections 1 minor 2 references

MEG Evidence That Modality-Independent Conceptual Representations Encode Visual but Not Lexical Representations

T0 review · 2 major / 1 minor · reviewed 2026-05-25 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Modality-independent conceptual representations encode visual features but not lexical ones.

desk verdict The abstract claims modality-independent MEG representations carry visual but not lexical content via cross-condition NN decoding, yet supplies none of the controls needed to confirm the latent space is truly modality-agnostic. read the letter →

arxiv 2312.10916 v2 pith:O5IZ4SM2 submitted 2023-12-18 q-bio.NC

Julien Dirani , Liina Pylkkänen This is my paper

classification q-bio.NC

keywords modality-independentrepresentationsMEGdecodingconceptualvisualfeatureslexicalcross-conditionsemanticknowledge

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tests whether representations that support the same concept across pictures and words are purely abstract or include perceptual or word-specific content. Researchers recorded MEG brain signals during picture and word presentations of the same concepts, then trained neural network classifiers to isolate patterns that transfer across the two input types. These shared patterns aligned with models of visual features but showed no alignment with models of lexical properties such as word frequency or length. The result indicates that even modality-independent access to meaning draws on visual processing. Readers care because the work clarifies how the brain maintains semantic knowledge that is not tied to any single sense.

What carries the argument

Neural network classifiers that learn latent modality-independent representations from MEG signals via word-picture cross-condition decoding.

What would settle it

If the cross-condition latent representations fail to predict visual feature models or succeed in predicting lexical feature models, the claim that modality-independent representations contain visual but not lexical content would not hold.

Watch

Extended reading notes

Core claim

Using word-picture cross-condition decoding on MEG data, neural network classifiers extracted latent modality-independent representations. Comparison against semantic, sensory, and lexical feature models showed that these representations contain visual information but no detectable lexical contribution. The findings indicate that perceptual processes participate in the encoding of modality-independent conceptual representations while lexical representations do not.

Load-bearing premise

The neural network classifiers isolate truly modality-independent latent representations without residual information from the specific input modality or overfitting to the training conditions.

Editorial extensions

If this is right

Conceptual access from words recruits visual feature processing even without a picture present.
Lexical properties such as word form do not form part of the shared modality-independent representations.
Perceptual processes contribute to the encoding of conceptual knowledge that functions across input modalities.
Semantic knowledge stored independently of modality is not strictly amodal.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The result suggests that grounded-cognition accounts should include visual overlap within cross-modal semantic codes.
The same decoding approach could be applied to test whether auditory features appear in modality-independent representations for sound-related concepts.
Disruption of visual cortex might affect semantic tasks performed with words alone if the visual content is functionally relevant.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

read the letter

The punchline is that this MEG study finds visual features in cross-modal conceptual representations but no lexical ones, reached by training neural nets on word/picture cross-decoding then matching the latent spaces to explicit feature models. What is new is the specific visual-versus-lexical dissociation tested with this NN latent-space approach; the abstract notes it is not in the prior literature they cite. The setup is direct and the question is cleanly posed, with MEG data suited to timing questions about conceptual access. The approach earns credit for trying to move beyond simple amodal claims by pitting perceptual and lexical models against the same data. The soft spot is exactly the one the stress-test flags: the abstract gives no architecture, regularization, held-out validation, or permutation baselines, so it is impossible to check whether the latent representations are free of residual modality-specific leakage. Without those controls the visual match could reflect overfitting rather than shared content, and the null lexical result is equally hard to interpret. This paper is for labs working on semantic memory and multimodal integration. A reader focused on whether perceptual content is embedded in supposedly amodal concepts could find the framing useful once the methods are shown. It deserves peer review because the question is substantive and the method combination is novel enough to merit checking the details rather than a desk reject.

Referee Report

2 major / 1 minor

Summary. The paper uses MEG recordings during word and picture presentation of concepts, applies cross-condition neural-network decoding to extract latent modality-independent representations, and then compares those representations to models of semantic, visual, and lexical features. It concludes that the modality-independent representations contain visual but not lexical content, supporting a role for perceptual processes in conceptual knowledge.

Significance. If the cross-condition decoding procedure demonstrably isolates representations free of modality-specific leakage, the result would be a substantive contribution to debates on amodal versus grounded conceptual representations by providing direct neural evidence that visual features participate in modality-independent semantic codes.

major comments (2)

[Abstract/Methods] Abstract and Methods: the central claim that the learned latent representations are modality-independent (and therefore that subsequent model comparisons can be interpreted as evidence for visual but not lexical content) rests on the unverified assumption that the neural-network classifiers have eliminated residual modality-specific information; no architecture, regularization, loss function, held-out validation accuracy, or permutation baseline is described.
[Results] Results: the reported absence of lexical contributions and presence of visual contributions are only interpretable once it is shown that the cross-condition decoding does not simply overfit to the training modality; without explicit controls (e.g., within-modality decoding accuracies or feature ablation), the pattern could reflect incomplete isolation rather than the content of modality-independent codes.

minor comments (1)

[Abstract] The abstract would be clearer if it stated the number of participants, number of concepts, and the exact statistical threshold used for the model comparisons.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments, which highlight important points for clarifying our cross-condition decoding procedure. We address each major comment below and will revise the manuscript to incorporate the requested details and controls.

read point-by-point responses

Referee: [Abstract/Methods] Abstract and Methods: the central claim that the learned latent representations are modality-independent (and therefore that subsequent model comparisons can be interpreted as evidence for visual but not lexical content) rests on the unverified assumption that the neural-network classifiers have eliminated residual modality-specific information; no architecture, regularization, loss function, held-out validation accuracy, or permutation baseline is described.

Authors: We agree that the Methods section lacks sufficient detail on the neural network classifiers. In the revised manuscript we will add a full specification of the architecture (layers, units, activations), regularization, loss function, held-out validation accuracies on the cross-condition task, and permutation baselines. These additions will directly address the assumption of modality-independence and allow readers to evaluate residual modality-specific leakage. revision: yes
Referee: [Results] Results: the reported absence of lexical contributions and presence of visual contributions are only interpretable once it is shown that the cross-condition decoding does not simply overfit to the training modality; without explicit controls (e.g., within-modality decoding accuracies or feature ablation), the pattern could reflect incomplete isolation rather than the content of modality-independent codes.

Authors: We acknowledge that explicit controls are needed to rule out overfitting to the training modality. We will add within-modality decoding accuracies to the Results for direct comparison with cross-condition performance. Where the data permit, we will also include feature-ablation results to further isolate visual versus lexical contributions. These revisions will strengthen the claim that the observed pattern reflects modality-independent content. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: empirical MEG decoding with external model comparisons

full rationale

The paper reports results from cross-condition MEG decoding using neural network classifiers to extract latent representations, followed by comparisons to separate semantic/sensory/lexical feature models. No equations, fitted parameters renamed as predictions, self-definitional constructs, or load-bearing self-citations appear in the abstract or described method. The central claims rest on decoding performance metrics that are externally falsifiable against held-out data and independent feature models. This matches the default case of a self-contained empirical study.

Assumptions & free parameters 0 free parameters · 0 assumptions · 0 invented entities

Abstract-only review yields limited visibility into modeling choices; no explicit free parameters, axioms, or invented entities are stated.

how reviews work

0 comments

Cite this review

Pith. "Pith review of MEG Evidence That Modality-Independent Conceptual Representations Encode Visual but Not Lexical Representations." pith.science (2026). https://pith.science/paper/O5IZ4SM2

@misc{pith2026231210916,
  author       = {Pith},
  title        = {Pith review of: MEG Evidence That Modality-Independent Conceptual Representations Encode Visual but Not Lexical Representations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O5IZ4SM2}},
  note         = {Machine review of arXiv:2312.10916}
}

read the original abstract

The semantic knowledge stored in our brains can be accessed from different stimulus modalities. For example, a picture of a cat and the word "cat" both engage similar conceptual representations. While existing research has found evidence for modality-independent representations, their content remains unknown. Modality-independent representations could be abstract, or they might be perceptual or even lexical in nature. We used a novel approach combining word/picture cross-condition decoding with neural network classifiers that learned latent modality-independent representations from MEG data. We then compared these representations to models representing semantic, sensory, and lexical features. Results show that modality-independent representations are not strictly amodal; rather, they also contain visual representations. There was no evidence that lexical properties contributed to the representation of modality-independent concepts. These findings support the notion that perceptual processes play a fundamental role in encoding modality-independent conceptual representations. Conversely, lexical representations did not appear to partake in modality-independent semantic knowledge.

Figures

Figures reproduced from arXiv: 2312.10916 by the authors.

**Figure 1.** Analysis pipeline. (1) Cross-condition decoding in which neural network classifiers trained on one modality were tested on the other modality, for all pairs of timepoints. This allowed us to map the clusters of timepoints (ttrain,ttest) where modality-independent representations of basic-level concepts were activated. (2) Classifiers with successful cross-condition generalization were assumed to have learned latent … view at source ↗

**Figure 2.** The activation timing of modality-independent representations of basic-level concepts. A-B: Accuracy scores at each time-point for classifiers trained and tested within each modality. The shaded regions indicate time points where classifier accuracy was above chance at the group level. C: Cross-condition decoding results where models trained on the MEG data from the words were tested on MEG data from the pictures fo… view at source ↗

**Figure 3.** RSA results investigating the content of modality [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

2 extracted references · 2 canonical work pages

[1]

Deep residual learning for image recognition

He, K., et al. Deep residual learning for image recognition. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. 21. McRae, K., et al., Semantic feature production norms for a large set of living and nonliving things. Behavior research methods, 2005. 37(4): p. 547-559. 22. Balota, D., et al., The English lexicon project...

work page 2016
[2]

Adam: A Method for Stochastic Optimization

Perrin, F., et al., Spherical splines for scalp potential and current density mapping. Electroencephalography and clinical neurophysiology, 1989. 72(2): p. 184-187. 41. Kingma, D.P. and J. Ba, Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. 42. Maris, E. and R. Oostenveld, Nonparametric statistical testing of EEG-and MEG-...

work page Pith review arXiv 1989

Pith tools

Reviewed May 25, 2026 · model on record in the stance chip above.

[1] [1]

Deep residual learning for image recognition

He, K., et al. Deep residual learning for image recognition. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. 21. McRae, K., et al., Semantic feature production norms for a large set of living and nonliving things. Behavior research methods, 2005. 37(4): p. 547-559. 22. Balota, D., et al., The English lexicon project...

work page 2016

[2] [2]

Adam: A Method for Stochastic Optimization

Perrin, F., et al., Spherical splines for scalp potential and current density mapping. Electroencephalography and clinical neurophysiology, 1989. 72(2): p. 184-187. 41. Kingma, D.P. and J. Ba, Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. 42. Maris, E. and R. Oostenveld, Nonparametric statistical testing of EEG-and MEG-...

work page Pith review arXiv 1989