BrainWavLM: Fine-tuning Speech Representations with Brain Responses to Language

Aditya R. Vaidya; Alexander G. Huth; Nishitha Vattikonda; Richard J. Antonello

REVIEW 3 major objections 6 minor 1 cited by

BrainWavLM: Fine-tuning Speech Representations with Brain Responses to Language

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuning WavLM on brain responses with LoRA lifts speech encoding accuracy by 12.5% over the frozen pretrained model.

desk verdict Brain fine-tuning with LoRA is a solid incremental contribution that deserves serious review, but the 12.5% headline rests on one held-out story and needs statistical support. read the letter →

arxiv 2502.08866 v1 pith:DB6RKJCS submitted 2025-02-13 cs.CL

Nishitha Vattikonda , Aditya R. Vaidya , Richard J. Antonello , Alexander G. Huth This is my paper · ORCID

classification cs.CL

keywords speechencodingmodelsfMRIpredictionlow-rankadaptationLoRAWavLMsemanticrepresentationsauditorycortexself-supervised

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that speech encoding models, which predict how the brain responds to spoken language, can be substantially improved by fine-tuning an existing self-supervised speech model directly on fMRI brain responses. Using low-rank adaptation (LoRA) to keep the update efficient, the authors produce BrainWavLM, which improves average whole-cortex encoding performance by 12.5% over the pretrained WavLM Base+ model and trains more stably than full fine-tuning. The improvement comes at the cost of performance in early auditory cortex, but selectively fine-tuning on that region restores its performance while largely keeping gains elsewhere. Fine-tuned models also transfer across subjects and show stronger semantic content without explicit word-level annotations, suggesting brain data can serve as an annotation-free supervisory signal for building more brain-like speech representations.

What carries the argument

The central mechanism is the low-rank adaptation (LoRA) procedure applied to WavLM's attention matrices ($W_Q$, $W_K$, $W_V$) plus a rank-100 linear bottleneck on the 9th-layer hidden states, trained with a loss that maximizes spatial correlation between predicted and actual fMRI volumes. This lets the model adjust its representations to brain data with few learnable parameters (166K LoRA parameters plus a per-subject linear bottleneck), and the paper shows this is more stable than full fine-tuning and sufficient to match its performance.

What would settle it

Compute encoding performance of the same BrainWavLM models on a larger, independently collected set of held-out stories (or another fMRI narrative dataset) and check whether the average 12.5% improvement over the pretrained linearized model persists; a substantial shrinkage or reversal would undermine the claim that brain fine-tuning yields best-in-class speech encoding.

Watch

Extended reading notes

Core claim

Fine-tuning a WavLM Base+ speech model end-to-end on fMRI responses, with LoRA updates to the attention matrices and a low-rank linear readout, produces encoding models that predict held-out brain responses better than the linearized baseline. The average gain is 12.5% over the pretrained model, and it is more stable than full fine-tuning. This gain is uneven: low-level auditory cortex loses some accuracy, but fine-tuning specifically on auditory-cortex voxels reverses that loss while preserving most of the whole-cortex gain. Models fine-tuned on one subject improve encoding in other subjects, and linear probes show the representations become more semantic and less acoustic, comparable to fine-tuning on large language model features, all without annotations.

Load-bearing premise

The numerical gains are measured on a single held-out story per participant, with validation on two other stories; if those particular stories are not representative of narrative speech in general, the reported improvements and tradeoffs may not hold for other stories.

Editorial extensions

If this is right

If correct, the standard linearized encoding model pipeline can be superseded by end-to-end fine-tuning even on neuroscience-scale datasets, yielding more accurate voxelwise predictions.
Brain responses can serve as an annotation-free supervisory signal that makes speech representations more semantic, matching the effect of supervision from a large language model.
The auditory-cortex tradeoff is a tunable property: choosing which voxels to fine-tune on controls whether the model favors low-level acoustic fidelity or high-level semantic content.
Cross-subject transfer of fine-tuned models suggests that at least part of the learned representation captures shared, subject-general brain-like structure, not just individual noise.
The stability benefit of LoRA over full fine-tuning matters for reproducibility in small-data regimes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same LoRA fine-tuning recipe could be applied to other self-supervised speech or language models (e.g., larger checkpoints) to test whether brain-supervision gains scale with model size.
The observed semantic/acoustic tradeoff suggests a Pareto frontier: one could fine-tune a family of models with different cortical targets and select among them for a desired balance, which may be useful for brain-computer interfaces that decode either acoustics or meaning.
Because the gains are measured on only three held-out stories, a natural next step is to evaluate on a larger battery of unseen narratives to quantify how much of the improvement is story-specific versus truly generalizable.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read this if you work on speech encoding or brain-tuning. The genuine additions over Moussa et al. 2024 (which they cite) are the LoRA-based fine-tuning, the auditory-cortex-targeted experiment that sidesteps the AC tradeoff, and the cross-subject transfer results. The evaluation protocol is thoughtful—they re-fit a linearized encoding model at each epoch to select the validation-best checkpoint, which sidesteps the train/test objective mismatch.

What it does well: the AC-targeted experiment is the most interesting scientific result. Fine-tuning only on AC improves AC encoding while mostly preserving gains elsewhere, which supports the voxel-imbalance explanation. The cross-subject transfer is a genuine plus and goes beyond prior work. The probes are a reasonable way to show the representations became more semantic and less acoustic, and the LLaMA-tuned baseline is a fair comparison for semantic supervision.

Where it is soft: the headline 12.5% improvement is computed on a single held-out story per subject (wheretheressmoke), with two stories for validation, n=3 subjects, and no significance tests. The abstract says 'significantly better' but the paper presents no confidence intervals or per-story variability. Cross-subject transfer still uses the same single test story, so it tests transfer across brains, not across narrative content. The 'best-in-class' claim overreaches: the comparisons are WavLM-based plus one LLaMA-tuned model, not the field. And code/weights are promised only for after publication, so the figures are not independently checkable now.

The stress-test concern about the test story is fair and lands. A serious referee should ask for a multi-story test set or a bootstrap over stories, and a control that matches the rank-100 bottleneck against pretrained features. I would not call this fatal—the direction of the effect is internally consistent and the AC result is mechanistically plausible—but the headline number is not yet pinned down.

Bottom line: a solid within-subfield contribution that deserves a serious referee. It is not a desk reject, and after revision (more test stories, stats, code release) it could be a useful citable result.

Referee Report

3 major / 6 minor

Summary. The paper proposes BrainWavLM, a WavLM Base+ speech model fine-tuned with low-rank adaptation (LoRA) on fMRI responses to natural narrative speech. Fine-tuning uses a spatial-correlation loss and a rank-100 linear bottleneck to predict voxel responses, and the best epoch is selected by re-fitting a linearized encoding model on validation stories. On a single held-out story per subject, the authors report that whole-cortex fine-tuning improves average voxelwise encoding correlation by 12.5% over the frozen WavLM features, at the cost of auditory-cortex (AC) performance; selective fine-tuning on AC restores AC performance. The fine-tuned features are also reported to transfer across subjects and, according to linear probes, to become more semantic and less acoustic.

Significance. If the improvement is robust, the paper makes a useful contribution: it demonstrates that LoRA fine-tuning can improve speech encoding models, that low-level and high-level cortical areas impose competing objectives during brain fine-tuning, and that brain responses can act as an unsupervised semantic supervisory signal. The use of the public LeBel/Tang dataset, the clearly described fine-tuning and evaluation protocol, and the inclusion of cross-subject transfer and probe analyses are strengths. However, the current evidence is limited by the single-story test set and by the absence of a capacity-matched control for the rank-100 bottleneck, so the headline claims are not yet fully established.

major comments (3)

[2.1, 3.1] The central 12.5% improvement claim is measured on a single held-out story ('wheretheressmoke') per subject, with two stories ('fromboyhoodtofatherhood' and 'onapproachtopluto') used only for validation; all subsequent analyses in Sections 3.2-3.4 are evaluated on this same test story. With approximately 100 stories available per subject, this design does not test generalization to novel narrative material, and no per-story variability, confidence intervals, or significance tests are reported. I recommend using multiple held-out stories (e.g., leave-several-stories-out cross-validation) or at least reporting per-story and per-subject results with appropriate statistics before the headline claim is considered established.
[2.3.1, 3.1] The fine-tuned BrainWavLM model includes a rank-100 linear bottleneck in the output projection, whereas the pre-trained baseline is a linearized encoding model fit to the full WavLM features with ridge regression. The reported improvement may therefore be due in part to the bottleneck itself rather than to the LoRA brain fine-tuning. The paper should include a control in which the same rank-100 bottleneck is applied to the frozen pre-trained WavLM features, with the projection fit on training data, before the linearized encoding model is evaluated. The LLaMA-tuned baseline already shows that most of the gain (11.1% vs. 12.5%) can be obtained by fine-tuning toward LLaMA features with the same setup, which further motivates this control.
[3.3] The cross-subject transfer results in Figure 3 use the same single test story ('wheretheressmoke') for all subjects, so they demonstrate robustness across subjects for that story but not across stories. This limitation should be acknowledged explicitly, and the transfer claim would be considerably strengthened by re-evaluating the transferred features on multiple held-out stories.

minor comments (6)

[1] The abstract and introduction state that fine-tuned models yield 'significantly better' prediction performance, but no significance tests are reported; please qualify the claim or add appropriate statistical tests.
[2.3.1] The sentence 'the first 100 principal components of the pre-trained model’s encoding weights capture more than 70% of the variance across voxels' is unclear; please specify over which dimension PCA is computed and how this justifies the rank-100 bottleneck.
[Figure 1B] The error bars are SEM across n=3 subjects after subtracting each subject's mean; with three subjects this provides little information about between-story variability, so please show individual subject points as well.
[2.3.3] The LLaMA-tuned baseline is described as using layer 18, which had the highest encoding performance; please specify how this layer was selected and whether the selection was made on held-out data.
[Conclusion] The claim of producing 'best-in-class speech encoding models' is not supported by comparisons to other published encoding models; please temper this claim or add quantitative comparisons.
[2.1, Figure 2C caption] There are minor typographical errors: '17.8 h–19.7 hhours' should read '17.8–19.7 hours', and 'V oxels' in the Figure 2C caption should read 'Voxels'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claims rest on out-of-sample evaluations with held-out stories, and no equation reduces to its inputs.

full rationale

The paper's central claims are empirical comparisons on a held-out story ('wheretheressmoke') and two validation stories, with model selection performed on the validation set before test evaluation (Sec. 2.1, 2.3.2). The end-to-end training loss (Eq. 2, spatial correlation) is explicitly distinct from the evaluation metric (Eq. 1, temporal correlation), and the authors state this difference, so the reported gains are not identities. The semantic-probe experiment trains probes on 26 stories and tests on the three held-out stories, so the strengthened semantic representations are measured out-of-sample rather than read off the training objective. Cross-subject transfer fits encoding models on each target subject's training data and evaluates on that subject's held-out story, so transfer is not forced. Self-citations (LeBel et al. dataset; Antonello et al. layer selection) are to public data and a published empirical result; the layer choice is held fixed for both the linearized baseline and the fine-tuned model, so it does not predetermine the reported improvement. No equation in the paper reduces to its input, and no fitted parameter is renamed as a prediction. Therefore no circular step is exhibited.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim relies on standard encoding-model assumptions, a small set of hand-chosen hyperparameters, and a single held-out story. No new physical or theoretical entities are introduced.

free parameters (3)

LoRA rank r = 4
Rank of the low-rank updates to WQ, WK, WV; chosen without a reported hyperparameter sweep (Section 2.3.1).
Linear bottleneck rank = 100
Rank of the final projection; justified by the first 100 principal components of the pretrained encoding weights capturing more than 70% of voxel variance, not by encoding performance (Section 2.3.1).
Training hyperparameters = lr=1e-4, batch=50, epochs=20
Fixed values reported in Section 2.3.1; no sweep or sensitivity analysis shown.

assumptions (5)

domain assumption fMRI BOLD responses are linearly predictable from delayed speech features via ridge regression
Core encoding model assumption used throughout Section 2.2; inherited from prior work and not tested here.
domain assumption WavLM layer 9 hidden states are the best fixed substrate for brain encoding
Layer 9 chosen based on Antonello et al. 2023 (Section 2.3.1); no re-derivation.
ad hoc to paper A 100-dimensional linear bottleneck preserves enough information for accurate brain prediction
Supported only by PCA variance on pretrained weights, not by an encoding comparison with or without the bottleneck (Section 2.3.1).
domain assumption The LeBel et al. atlas correctly localizes auditory cortex
AC voxels are defined by this atlas in Section 3.2.
ad hoc to paper The single held-out story is representative of natural narrative speech
All test-set numbers come from one story; representativeness is assumed, not demonstrated (Section 2.1).

how reviews work

0 comments

Cite this review

Pith. "Pith review of BrainWavLM: Fine-tuning Speech Representations with Brain Responses to Language." pith.science (2026). https://pith.science/paper/DB6RKJCS

@misc{pith2026250208866,
  author       = {Pith},
  title        = {Pith review of: BrainWavLM: Fine-tuning Speech Representations with Brain Responses to Language},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DB6RKJCS}},
  note         = {Machine review of arXiv:2502.08866}
}

read the original abstract

Speech encoding models use auditory representations to predict how the human brain responds to spoken language stimuli. Most performant encoding models linearly map the hidden states of artificial neural networks to brain data, but this linear restriction may limit their effectiveness. In this work, we use low-rank adaptation (LoRA) to fine-tune a WavLM-based encoding model end-to-end on a brain encoding objective, producing a model we name BrainWavLM. We show that fine-tuning across all of cortex improves average encoding performance with greater stability than without LoRA. This improvement comes at the expense of low-level regions like auditory cortex (AC), but selectively fine-tuning on these areas improves performance in AC, while largely retaining gains made in the rest of cortex. Fine-tuned models generalized across subjects, indicating that they learned robust brain-like representations of the speech stimuli. Finally, by training linear probes, we showed that the brain data strengthened semantic representations in the speech model without any explicit annotations. Our results demonstrate that brain fine-tuning produces best-in-class speech encoding models, and that non-linear methods have the potential to bridge the gap between artificial and biological representations of semantics.

Figures

Figures reproduced from arXiv: 2502.08866 by the authors.

**Figure 1.** Encoding performance of BrainWavLM models fine-tuned on fMRI responses. (A) Cortical map of the change in encoding performance from the pre-trained WavLM model to the highest-performing BrainWavLM model (measured by performance on the validation set) on the test set. Corresponds to model fine-tuned with LoRA in (B). Results shown are for subject S03. (B) Encoding performance on the test set for the pre-trained model… view at source ↗

**Figure 2.** Models adapt to representations in subsets of cortex. (A) Encoding performance was computed for models fine-tuned on the whole cortex or fine-tuned just on auditory cortex (AC). Cortical maps show the difference in encoding performance on one subject. Only voxels with encoding performance above 0.15 for the pre-trained model are shown. The model fine-tuned on AC has higher performance in language-selective areas in … view at source ↗

**Figure 3.** Fine-tuned models transfer between fMRI subjects. Percent improvement in encoding performance (averaged across voxels) of the models fine-tuned using one subject’s fMRI responses compared to the pre-trained model. (A) Models were fine-tuned using fMRI responses from the whole cortex. (B) Models were fine-tuned using fMRI responses from auditory cortex (AC). Performance was only measured on the voxels within auditory… view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Model representations change after fine-tuning. Probe improvement over the pre-trained model, averaged across subjects. Transformer layer 0 is WavLM’s convolutional waveform encoder. Error bars indicate SEM across subjects. (A) Acoustic probes linearly predict filterba…

**Figure 5.** Figure 5: Percent Improvement in Encoding Performance for Models Fine-tuned with and without [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

**Figure 6.** Figure 6: AC Fine-tuning Improvement over Whole Cortex Fine-tuning for subjects S01 and S02. [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

**Figure 7.** Figure 7: Difference in Encoding Performance (Right Hem. Fine-tuning – Left Hem. Fine-tuning) [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

**Figure 8.** Figure 8: Acoustic and semantic probe performance, shown separately for WavLM models fine [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Abstraction Induces the Brain Alignment of Language and Speech Models
cs.CL 2026-02 unverdicted novelty 6.0 of 10

Middle layers of language and speech models align with brain responses due to high intrinsic dimension reflecting semantic abstraction rather than next-token prediction.

Reference graph

Works this paper leans on

22 extracted references · 6 canonical work pages · cited by 1 Pith paper

[8]

doi: 10.1038/s41597-023-02437-z

ISSN 2052-4463. doi: 10.1038/s41597-023-02437-z. B. Lipkin, G. Tuckute, J. Affourtit, H. Small, Z. Mineroff, H. Kean, O. Jouravlev, L. Rakocevic, B. Pritchett, M. Siegelman, C. Hoeflin, A. Pongos, I. A. Blank, M. K. Struhl, A. Ivanova, S. Shan- non, A. Sathe, M. Hoffmann, A. Nieto-Castañón, and E. Fedorenko. Probabilistic atlas for the language network ba...

doi:10.1038/s41597-023-02437-z
[9]

doi: 10.1038/s41597-022-01645-3

ISSN 2052-4463. doi: 10.1038/s41597-022-01645-3. J. Millet, C. Caucheteux, P. Orhan, Y . Boubenec, A. Gramfort, E. Dunbar, C. Pallier, and J.-R. King. Toward a realistic model of speech processing in the brain with self-supervised learning, June

doi:10.1038/s41597-022-01645-3
[10]

Narratives

ISSN 1053-8119. doi: 10.1016/j.neuroimage.2010.07.073. S. A. Nastase, Y .-F. Liu, H. Hillman, A. Zadbood, L. Hasenfratz, N. Keshavarzian, J. Chen, C. J. Honey, Y . Yeshurun, M. Regev, M. Nguyen, C. H. C. Chang, C. Baldassano, O. Lositsky, E. Simony, M. A. Chow, Y . C. Leong, P. P. Brooks, E. Micciche, G. Choe, A. Goldstein, T. Vanderwal, Y . O. Halchenko,...

doi:10.1016/j.neuroimage.2010.07.073 2010
[12]

doi: 10.1016/j.cub.2011.08.031

ISSN 0960-9822. doi: 10.1016/j.cub.2011.08.031. S. R. Oota, E. Çelik, F. Deniz, and M. Toneva. Speech language models lack important brain- relevant semantics. In L.-W. Ku, A. Martins, and V . Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8503–8528, Bangkok, Thailand, Aug

doi:10.1016/j.cub.2011.08.031 2011
[14]

doi: 10.1101/2024.05.15.594172

ISSN 2692-8205. doi: 10.1101/2024.05.15.594172. A. Pasad, J.-C. Chou, and K. Livescu. Layer-wise Analysis of a Self-supervised Speech Representa- tion Model. arXiv:2107.04734 [cs, eess], Oct

arXiv 2024
[15]

Pennington, R

J. Pennington, R. Socher, and C. Manning. Glove: Global Vectors for Word Representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar,

work page 2014
[17]

doi: 10.1038/s41593-023-01304-9

ISSN 1546-1726. doi: 10.1038/s41593-023-01304-9. H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample. LLaMA: Open and Efficient Foundation Language Models, Feb

doi:10.1038/s41593-023-01304-9
[18]

doi: 10.1371/journal.pbio

ISSN 1545-7885. doi: 10.1371/journal.pbio. 3002366. A. R. Vaidya, S. Jain, and A. Huth. Self-Supervised Models of Audio Effectively Explain Human Cortical Responses to Speech. In Proceedings of the 39th International Conference on Machine Learning, pages 21927–21944. PMLR, June

doi:10.1371/journal.pbio

Show all 22 references

[19]

Wehbe, A

L. Wehbe, A. Vaswani, K. Knight, and T. Mitchell. Aligning context-based statistical models of language with brain activity during reading. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages 233–243, Doha, Qatar, Oct

2014
[20]

doi: 10.3115/v1/D14-1030

Association for Computational Linguistics. doi: 10.3115/v1/D14-1030. H. Weissbart and A. E. Martin. The structure and statistics of language jointly shape cross-frequency neural dynamics during spoken language comprehension. Nature Communications, 15(1):8850, Oct

doi
[21]

doi: 10.1038/s41467-024-53128-1

ISSN 2041-1723. doi: 10.1038/s41467-024-53128-1. M. C.-K. Wu, S. V . David, and J. L. Gallant. Complete functional characterization of sensory neurons by system identification. Annual Review of Neuroscience , 29(1):477–505, July

doi
[2006]

doi: 10.1146/annurev.neuro.29.051605.113024

ISSN 0147-006X, 1545-4126. doi: 10.1146/annurev.neuro.29.051605.113024. S.-w. Yang, P.-H. Chi, Y .-S. Chuang, C.-I. J. Lai, K. Lakhotia, Y . Y . Lin, A. T. Liu, J. Shi, X. Chang, G.- T. Lin, T.-H. Huang, W.-C. Tseng, K.-t. Lee, D.-R. Liu, Z. Huang, S. Dong, S.-W. Li, S. Watana...
[2007]

doi: 10.1038/nrn2113

ISSN 1471-0048. doi: 10.1038/nrn2113. 9 E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen. LoRA: Low-Rank Adaptation of Large Language Models, Oct

doi
[2011]

doi: 10.1073/pnas.1112937108. A. Goldstein, A. Dabush, B. Aubrey, M. Schain, S. A. Nastase, Z. Zada, E. Ham, Z. Hong, A. Feder, H. Gazula, E. Buchnik, W. Doyle, S. Devore, P. Dugan, D. Friedman, M. Brenner, A. Hassidim, O. Devinsky, A. Flinker, and U. Hasson. Brain embeddings ...

doi
[2014]

doi: 10.3115/v1/D14-1162

Association for Computational Linguistics. doi: 10.3115/v1/D14-1162. 10 D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlicek, Y . Qian, P. Schwarz, J. Silovsky, G. Stemmer, and K. Vesely. The kaldi speech recognition toolkit. In IEEE 20...

doi
[2016]

doi: 10.1038/nature17637

ISSN 1476-4687. doi: 10.1038/nature17637. A. A. Ivanova, M. Schrimpf, S. Anzellotti, N. Zaslavsky, E. Fedorenko, and L. Isik. Beyond linear regression: Mapping models in cognitive neuroscience should align with research goals. Neurons, Behavior, Data analysis, and Theory, 1, Aug

doi
[2017]

doi: 10.1523/JNEUROSCI.3267-16.2017

ISSN 0270-6474, 1529-2401. doi: 10.1523/JNEUROSCI.3267-16.2017. E. Fedorenko, M. K. Behr, and N. Kanwisher. Functional specificity for high-level linguistic processing in the human brain. Proceedings of the National Academy of Sciences , 108(39): 16428–16433, Sept

2017 doi
[2020]

doi: 10.1126/science.aaz3468. R. Antonello, A. Vaidya, and A. Huth. Scaling laws for language encoding models in fMRI.Advances in Neural Information Processing Systems, 36:21895–21907, Dec

doi
[2021]

doi: 10.1038/s41597-021-01033-3

ISSN 2052-4463. doi: 10.1038/s41597-021-01033-3. S. Nishimoto, A. T. Vu, T. Naselaris, Y . Benjamini, B. Yu, and J. L. Gallant. Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies. Current Biology, 21(19):1641–1646, Oct

doi
[2022]

doi: 10.51628/001c.37507

ISSN 2690-2664. doi: 10.51628/001c.37507. D. P. Kingma and J. Ba. Adam: A Method for Stochastic Optimization. arXiv:1412.6980 [cs], Jan

arXiv
[2023]

doi: 10.1038/s41562-022-01516-2

ISSN 2397-3374. doi: 10.1038/s41562-022-01516-2. S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao, J. Wu, L. Zhou, S. Ren, Y . Qian, Y . Qian, J. Wu, M. Zeng, and F. Wei. WavLM: Large-Scale Self- Supervised Pre-Training for Full Stack S...
[2024]

doi: 10.18653/v1/2024.acl-long.462

Association for Computational Linguistics. doi: 10.18653/v1/2024.acl-long.462. O. Ozernov-Palchik, A. M. O’Brien, E. Jiachen Lee, H. Richardson, R. Romeo, B. Lipkin, H. Small, J. Capella, A. Nieto-Castañón, R. Saxe, J. D. E. Gabrieli, and E. Fedorenko. Precision fMRI reveals t...

2024 doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [8]

doi: 10.1038/s41597-023-02437-z

ISSN 2052-4463. doi: 10.1038/s41597-023-02437-z. B. Lipkin, G. Tuckute, J. Affourtit, H. Small, Z. Mineroff, H. Kean, O. Jouravlev, L. Rakocevic, B. Pritchett, M. Siegelman, C. Hoeflin, A. Pongos, I. A. Blank, M. K. Struhl, A. Ivanova, S. Shan- non, A. Sathe, M. Hoffmann, A. Nieto-Castañón, and E. Fedorenko. Probabilistic atlas for the language network ba...

doi:10.1038/s41597-023-02437-z

[2] [9]

doi: 10.1038/s41597-022-01645-3

ISSN 2052-4463. doi: 10.1038/s41597-022-01645-3. J. Millet, C. Caucheteux, P. Orhan, Y . Boubenec, A. Gramfort, E. Dunbar, C. Pallier, and J.-R. King. Toward a realistic model of speech processing in the brain with self-supervised learning, June

doi:10.1038/s41597-022-01645-3

[3] [10]

Narratives

ISSN 1053-8119. doi: 10.1016/j.neuroimage.2010.07.073. S. A. Nastase, Y .-F. Liu, H. Hillman, A. Zadbood, L. Hasenfratz, N. Keshavarzian, J. Chen, C. J. Honey, Y . Yeshurun, M. Regev, M. Nguyen, C. H. C. Chang, C. Baldassano, O. Lositsky, E. Simony, M. A. Chow, Y . C. Leong, P. P. Brooks, E. Micciche, G. Choe, A. Goldstein, T. Vanderwal, Y . O. Halchenko,...

doi:10.1016/j.neuroimage.2010.07.073 2010

[4] [12]

doi: 10.1016/j.cub.2011.08.031

ISSN 0960-9822. doi: 10.1016/j.cub.2011.08.031. S. R. Oota, E. Çelik, F. Deniz, and M. Toneva. Speech language models lack important brain- relevant semantics. In L.-W. Ku, A. Martins, and V . Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8503–8528, Bangkok, Thailand, Aug

doi:10.1016/j.cub.2011.08.031 2011

[5] [14]

doi: 10.1101/2024.05.15.594172

ISSN 2692-8205. doi: 10.1101/2024.05.15.594172. A. Pasad, J.-C. Chou, and K. Livescu. Layer-wise Analysis of a Self-supervised Speech Representa- tion Model. arXiv:2107.04734 [cs, eess], Oct

arXiv 2024

[6] [15]

Pennington, R

J. Pennington, R. Socher, and C. Manning. Glove: Global Vectors for Word Representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar,

work page 2014

[7] [17]

doi: 10.1038/s41593-023-01304-9

ISSN 1546-1726. doi: 10.1038/s41593-023-01304-9. H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample. LLaMA: Open and Efficient Foundation Language Models, Feb

doi:10.1038/s41593-023-01304-9

[8] [18]

doi: 10.1371/journal.pbio

ISSN 1545-7885. doi: 10.1371/journal.pbio. 3002366. A. R. Vaidya, S. Jain, and A. Huth. Self-Supervised Models of Audio Effectively Explain Human Cortical Responses to Speech. In Proceedings of the 39th International Conference on Machine Learning, pages 21927–21944. PMLR, June

doi:10.1371/journal.pbio