Pith. sign in

REVIEW 5 major objections 6 minor 34 references

Optimizing fMRI Data Acquisition for Decoding Natural Speech with Limited Participants

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper argues that for decoding natural speech from fMRI in small cohorts, deep phenotyping—more recorded hours per participant—outperforms adding more participants, and that multi-subject training with a shared backbone yields no…

desk verdict Solid contrastive-decoding results for natural speech fMRI, but the deep-phenotyping recommendation gets ahead of the evidence: the multi-subject null is one architecture, one data subset. read the letter →

arxiv 2505.21304 v1 pith:OWQ5MPS4 submitted 2025-05-27 q-bio.NC cs.CLcs.LG

Louis Jalouzot , Alexis Thual , Yair Lakretz , Christophe Pallier , Bertrand Thirion This is my paper
classification q-bio.NCcs.CLcs.LG
keywords fMRIdecodingnaturalspeechdeepphenotypingcontrastivelearningLLMembeddingsmulti-subjectretrievalaccuracysyntaxandsemantics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Researchers planning fMRI studies of language face a practical tradeoff: scan a few people for many hours or many people for a few hours each. This paper argues that when the goal is decoding the natural speech a person hears, the first option wins in the small-cohort regime. Training deep networks to predict language-model text embeddings from brain activity with a contrastive retrieval objective yields 27% average top-10 accuracy for the three participants with the most data (36% for the best), far above a 1.6% prior baseline and near-zero chance. Accuracy rises with recorded hours per participant and shows no plateau at 13.5 hours, whereas adding other participants to training gives no improvement and the amount of story overlap across participants does not matter. The authors conclude that inter-subject variability is the binding constraint, so acquisition should favour deep phenotyping unless cohorts become substantially larger.

What carries the argument

The load-bearing mechanism is contrastive retrieval training of a brain decoder on LLM2Vec chunk embeddings, evaluated as top-10 accuracy against a retrieval set of about two thousand chunks per test fold. The contrastive loss pulls the predicted embedding toward the true chunk and pushes it away from other chunks in the same batch, and the pipeline's gains come from four tuned components: a 6-second lag for the haemodynamic delay, an 8-second context of preceding text, averaging 4 brain volumes for temporal smoothing, and pre-selecting 4096 voxels by their Ridge encoding $R^2$. For the multi-subject comparison, the central object is a decoder with a subject-specific first linear layer followed by a shared backbone, tested at hidden widths 64 and 4096; this architecture is what the no-multi-subject-gain claim rests on.

What would settle it

Re-run the multi-subject comparison on the same eight participants with functional alignment or a hierarchical Bayesian model instead of subject-specific linear layers, using the same held-out test stories: if per-subject top-10 accuracy beats the single-subject decoders for most participants, the paper's central negative result is an artefact of its architecture. Independently, extend the three deeply phenotyped participants past 13.5 recorded hours and check whether top-10 accuracy keeps rising; a plateau would falsify the claim that per-participant data volume still has headroom.

Watch

Extended reading notes

Core claim

The central discovery is a practical scaling result for neural speech decoding: with eight participants and up to 13.5 hours of story-listening fMRI per person, per-participant data volume drives retrieval accuracy, whereas adding subjects does not. A decoder trained per subject with a contrastive objective to predict LLM2Vec chunk embeddings from 4096 encoding-selected voxels achieves 27% average top-10 accuracy for the three most-recorded participants and 36% for subject 3, against a 1.6% BrainLLM baseline and near-zero chance. In the multi-subject setup, a shared backbone with subject-specific first layers fails to beat single-subject decoders across all 255 subsets of the eight participants, and the overlap of stories heard during training has negligible effect. The authors further report that decoders discriminate syntactic structure more sharply than semantic content, and that stories with conversational, simple syntax are decoded far better than reflective, complex ones. From this they conclude that the limiting factor is inter-subject variability, so acquisition should favour deep phenotyping unless cohorts become substantially larger.

Load-bearing premise

The conclusion that multi-subject training does not help assumes that the subject-specific-layer-plus-shared-backbone decoder fairly represents multi-subject learning; if functional alignment or hierarchical Bayesian sharing works better, the recommendation to prefer deep phenotyping over larger cohorts loses its main support.

Editorial extensions

If this is right

  • Per-participant recorded hours are the main driver of decoding accuracy in small cohorts: 13.5 hours of training data give 27% average top-10 accuracy, versus 6% for roughly 4 hours.
  • Adding more participants to a shared-backbone decoder will not by itself improve individual decoding accuracy in this regime, so acquisition plans should not trade per-person hours for cohort size.
  • Because story overlap across participants barely matters, experimenters can use diverse or even different stimuli per participant, and datasets recorded in different languages could be pooled if the text embeddings are language-agnostic.
  • Since decoders separate syntactic structure better than semantic content, improving decoding of reflective, semantically dense stories likely needs new objectives or representations rather than simply more of the same data.
  • Accuracy has not plateaued at 13.5 hours, so extending the same participants with more listening sessions is expected to yield further gains before any multi-subject advantage appears.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A cost-budget reading not stated in the paper: if fixed total acquisition hours are the constraint, the data favour concentrating them into as few participants as possible, at least until a per-person saturation point is measured; this is testable by simulating scan-budget allocation with this dataset's scaling curve.
  • The multi-subject null result is likely tied to the subject-specific-layer architecture; functional alignment or hierarchical Bayesian sharing remains a plausible route to cross-subject gains, so the strongest general claim is about today's architectures, not about multi-subject learning in principle.
  • The negligible overlap effect implies that multi-site and cross-language pooling could be a practical path out of the limited-participant regime, turning the bottleneck from participant recruitment into embedding alignment across languages and recording sites.
  • The syntax-over-semantics asymmetry could be an artefact of the LLM2Vec representation rather than a property of neural speech signals; comparing decoders trained on syntax-probing versus semantics-probing targets would separate these explanations.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper investigates data acquisition strategies for fMRI-based decoding of natural speech, using the LeBel et al. (2023) dataset of 8 participants. The authors train deep neural networks with a contrastive objective to predict LLM text embeddings from fMRI activity and evaluate performance with top-10 retrieval accuracy. They report that single-subject decoders scale with the amount of training data per participant, achieving 27% average top-10 accuracy for the three deeply phenotyped subjects (max 36% for subject 3), which far exceeds their stated chance level and the BrainLLM baseline. They also report that multi-subject training does not improve decoding accuracy relative to single-subject models, that the degree of stimulus overlap across subjects has negligible effect, and that decoders appear to capture syntactic structure better than semantic content. Based on these findings, the authors recommend deep phenotyping over collecting data from more participants and conclude that leveraging multi-subject data for natural speech decoding likely requires deeper phenotyping or a substantially larger cohort.

Significance. If the results hold, the paper provides a useful empirical benchmark for fMRI speech decoding, demonstrating that contrastive learning with LLM embeddings can yield high retrieval accuracy from relatively few participants. The scaling result (Figure 2) is a concrete, falsifiable finding that supports the deep-phenotyping strategy in limited-cohort regimes. The comparisons between single-subject and multi-subject training and the stimulus-overlap experiment address practically important design questions. The paper uses a public dataset, applies proper session- and story-holdout cross-validation, and reports confidence intervals, which strengthens the credibility of the per-subject scaling claims. However, the central negative result on multi-subject training is currently narrower than the abstract claims, and the reported chance level contains a quantitative error that affects several interpretations.

major comments (5)
  1. [§2.1, Figure 2, §3.1] The reported chance level of 0.05% is incorrect for the evaluation protocol. With a retrieval set of approximately 2000 chunks and top-10 accuracy, random chance is 10/2000 = 0.5%. The value 0.05% corresponds to a top-1 chance level. This error appears in the Figure 2 caption, the BrainLLM baseline discussion, and the Results section, where the baseline is described as 'far above the chance-level of 0.05%'. The authors should correct this value and recompute the corresponding statements about how much above chance the decoders perform, since the correct chance level is an order of magnitude higher.
  2. [§3.2, Figure 4, Abstract] The central negative result on multi-subject training does not support the abstract conclusion that 'leveraging multi-subject for natural speech decoding likely requires deeper phenotyping or a substantially larger cohort.' Figure 4 explicitly excludes the extra data available for subjects 1–3, so the multi-subject comparison is performed at roughly 4 hours of training data per subject, not in the 13.5-hour deep-phenotyping regime in which the headline 27% accuracy is achieved. The claim that multi-subject training does not improve decoding in the deep-phenotyping regime is therefore untested. The authors should either run the multi-subject experiment using the full deep-phenotyping data for subjects 1–3 or clearly restrict the conclusion to the lower-data regime.
  3. [§2.2, §3.2, §5] The multi-subject architecture tested here (a subject-specific first layer followed by a shared backbone, with no explicit inter-subject alignment) is a limited instantiation of multi-subject learning. The Discussion and Limitations already concede that advanced functional alignment or hierarchical Bayesian models could change the result. Since the recommendation to favor deep phenotyping over larger cohorts rests on this negative evidence, the paper should either provide additional multi-subject evidence (e.g., with a functional-alignment method) or temper the conclusion to the specific architecture and preprocessing pipeline evaluated.
  4. [§3.2, Figure 4] In Figure 4, for each subject and each number of training subjects, the authors report the best accuracy obtained across all 255 combinations of subjects. Taking the maximum over combinations introduces an optimistic bias and makes the displayed 'best accuracy' curve difficult to compare with the single-subject baseline. The paper should report the distribution (e.g., mean and range across combinations) or justify why the best-case selection is the appropriate comparison, and discuss how this selection affects the conclusion that multi-subject training is not beneficial.
  5. [§3.3.2, Figure 6] The conclusion that the decoder 'better differentiates between syntactically dissimilar chunks than semantically dissimilar ones' is based on comparing the normalized slopes of two different similarity metrics (Levenshtein distance on POS-tagged chunks vs. GloVe bag-of-words cosine). These metrics have different scales, distributions, and noise properties, so the observed difference in profile steepness could be an artifact of the metrics rather than a property of the decoder. The authors should provide a more direct and controlled comparison, for example by using a single representational space or by calibrating the two metrics on matched random baselines, before drawing the syntactic-vs-semantic conclusion.
minor comments (6)
  1. [§2.1] In the sentence 'We do not to explicitly model inter-subject variability', there is a typo: 'do not to' should be 'do not'.
  2. [Figure 2] The x-axis label '0 103 30 min' is unclear; it should be formatted as '10^3' and the corresponding time annotation should be explicit (e.g., '~30 min').
  3. [§3.1] The statement 'we achieve an average top-10 accuracy of 27%' should explicitly specify that this average is over subjects 1, 2, and 3, and it would be helpful to report the per-subject values for all eight participants in a table rather than only in figures.
  4. [§3.1] The claim 'To our knowledge, these results represent the first successful decoding of natural speech from fMRI data using a contrastive objective' is strong. The authors should verify this claim against prior fMRI-based contrastive decoding work and, if it stands, provide a more precise statement of what specifically is novel.
  5. [Table 2] The row 'Total number of layers 3 Subject specific layer + Linear + Residual' is ambiguous; rephrase to make the layer structure clear, e.g., '3 layers: one subject-specific layer, one linear layer, one residual layer'.
  6. [§5] The Limitations section correctly acknowledges that the findings may not hold for larger cohorts, but this acknowledgement is not carried into the abstract or conclusion. The abstract should be aligned with the scope of the evidence presented in the paper.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's claims rest on held-out retrieval predictions, external baselines, and its own experiments, not on fitted parameters relabeled as predictions or on load-bearing self-citations.

full rationale

The paper is an empirical study without a derivation chain that reduces to its inputs. Its central retrieval accuracies (27% average top-10 for deeply phenotyped subjects; 6% for shorter-acquisition subjects) are measured on held-out test data: Section 2.1 states 'we split the data into training, validation and test sets, and ensure that data from each of the three sets were acquired on different fMRI runs,' and Figure 2 specifies that the retrieval set was 'acquired on different MRI sessions than the training data, and come from different stories than that of the training set.' Voxel selection (top 4096 by encoding R2 on validation) and Bayesian hyperparameter search are performed on training/validation data only, so no fitted parameter is later relabeled as a prediction without independent test evaluation. The multi-subject negative result and the stimuli-overlap result are likewise direct comparisons on test splits. Citations to prior work by overlapping authors (Thual et al. 2022/2023; Aggarwal et al. 2024; d'Ascoli et al. 2024) appear as background or as possible future directions in the Discussion and are not load-bearing: the conclusions do not depend on the truth of those cited results. The paper is also benchmarked against the external BrainLLM baseline of Ye et al. (2025). Two non-circular weaknesses are visible: the stated 0.05% chance level appears inconsistent with a roughly 2000-chunk retrieval set (top-10 chance would be ~0.5%), and the multi-subject comparison in Figure 4 was run without the extra data available for subjects 1-3, which limits the fairness of the single-subject versus multi-subject comparison. These are correctness/interpretation concerns, not circularity, because neither involves a prediction that is equivalent by construction to an input or an argument that reduces to a self-citation. Accordingly, no circular steps are identified.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The paper's conclusions rest on standard empirical assumptions about the LLM embeddings, the public fMRI dataset, and the chosen network architecture. No new theoretical entities or parameters fitted to the central claim are introduced; all listed hyperparameters were tuned on a validation subset.

free parameters (6)
  • context_length = 3 chunks (6s)
    Tuned via Bayesian search on subjects 1-3; affects the text representation and thus decoding performance.
  • temporal_smoothing = 4 volumes (8s)
    Tuned; affects fMRI preprocessing and signal-to-noise ratio.
  • lag = 6s
    Tuned; accounts for the hemodynamic response delay.
  • top_encoding_voxels = 4096
    Chosen based on encoding performance; determines input dimensionality.
  • temperature = 0.7
    Tuned for the contrastive loss.
  • hidden_dim = 64 or 4096
    Model capacity; both sizes tested to examine interaction with multi-subject training.
assumptions (4)
  • domain assumption LLM2Vec text embeddings capture sufficiently rich linguistic features for decoding.
    The entire decoding target is defined by these embeddings; if they miss syntax or semantics, the decoder cannot recover it. See Section 2.1 where the target Y_t = h(Z_t) is defined.
  • domain assumption The retrieval top-10 accuracy is a valid proxy for decoding quality.
    Evaluation is based on ranking chunks in a retrieval set; this metric does not directly measure text reconstruction quality, as acknowledged in Limitations.
  • domain assumption The LeBel dataset's fMRI preprocessing (MNI normalization, standard scaling, voxel selection) preserves the signal relevant to speech decoding.
    Results depend on the preprocessing pipeline; see Section 2.1.
  • domain assumption The subject-specific layer architecture fairly represents multi-subject training approaches.
    The null result on multi-subject training is conditional on this architecture; the authors acknowledge that other alignment methods could change the conclusion.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Optimizing fMRI Data Acquisition for Decoding Natural Speech with Limited Participants." pith.science (2026). https://pith.science/paper/OWQ5MPS4

@misc{pith2026250521304,
  author       = {Pith},
  title        = {Pith review of: Optimizing fMRI Data Acquisition for Decoding Natural Speech with Limited Participants},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OWQ5MPS4}},
  note         = {Machine review of arXiv:2505.21304}
}
read the original abstract

We investigate optimal strategies for decoding perceived natural speech from fMRI data acquired from a limited number of participants. Leveraging Lebel et al. (2023)'s dataset of 8 participants, we first demonstrate the effectiveness of training deep neural networks to predict LLM-derived text representations from fMRI activity. Then, in this data regime, we observe that multi-subject training does not improve decoding accuracy compared to single-subject approach. Furthermore, training on similar or different stimuli across subjects has a negligible effect on decoding accuracy. Finally, we find that our decoders better model syntactic than semantic features, and that stories containing sentences with complex syntax or rich semantic content are more challenging to decode. While our results demonstrate the benefits of having extensive data per participant (deep phenotyping), they suggest that leveraging multi-subject for natural speech decoding likely requires deeper phenotyping or a substantially larger cohort.

Figures

Figures reproduced from arXiv: 2505.21304 by the authors.

Figure 1
Figure 1. Method for decoding natural speech from fMRI activity A. Decoding setup Deep Neural Networks are trained with a contrastive objective to predict text representations (derived from Large Language Models embeddings) from fMRI activity recorded as participants listened to natural speech. Key parameters include context length c, the number of prior chunks added to the text representations, lag τ, the delay between neura… view at source ↗
Figure 3
Figure 3. Setup comparison Impact of various elements of the decoding setup on decoding performance. We start from a very crude version of our setup, namely ”Base”, which is essentially a simple MLP trained with MSE loss on BERT latents. Then each row corresponds to the previous setup with a modification described by its blue label. We display the top-10 accuracy obtained when training on subjects 1, 2 and 3 with SSLs and the… view at source ↗
Figure 4
Figure 4. Impact of the number of subject used in the training set Multi-subject decoders were trained with subject￾specific layers for each of the 255 possible combinations of the 8 subjects. Then for each subject (color) and each number of subjects (x-axis), we display the best accuracy (y-axis) obtained with any of the combinations including this subject. We test small decoders (left pane, hidden dimension 64) and large on… view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Impact of training stimuli overlap We train multi-subject decoders with subject-specific layers on subjects 1, 2 and 3 while varying the ratio of overlapping stimuli between the subjects. The graphics display the increment in accuracy over single-subject decoders (y ax…
Figure 6
Figure 6. Figure 6: Profiles of average syntactic/semantic similarities Syntactic (left) and semantic (right) similarities between the ground-truth text chunks and the ∼2k candidate chunks from a retrieval set sorted by decreasing cosine similarity of their representation to the predicted…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

34 extracted references · 16 canonical work pages

  1. [1]

    Across-subject ensemble-learning alleviates the need for large samples for fMRI decoding

    H. Aggarwal, L. Al-Shikhley , and B. Thirion. Across-subject ensemble-learning alleviates the need for large samples for fMRI decoding. arXiv, 2024. doi:10.48550/ARXIV.2407.12056

    work page Pith review arXiv doi:10.48550/arxiv.2407.12056 2024

  2. [2]

    Akiba, S

    T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , 2019

    2019

  3. [3]

    E. J. Allen, G. St-Yves, Y. Wu, J. L. Breedlove, J. S. Prince, L. T. Dowdle, M. Nau, B. Caron, F. Pestilli, I. Charest, J. B. Hutchinson, T. Naselaris, and K. Kay. A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence. Nature Neuroscience, 25 0 (1): 0 116--126, Jan. 2022. ISSN 1546-1726. doi:10.1038/s41593-021-00962-x. URL...

    doi:10.1038/s41593-021-00962-x 2022

  4. [4]

    ascoli d'Ascoli , C

    S. ascoli d'Ascoli , C. Bel, J. Rapin, H. J. Banville, Y. Benchetrit, C. Pallier, and J.-R. King. Decoding individual words from non-invasive brain recordings across 723 participants. Dec. 2024

    work page 2024

  5. [5]

    BehnamGhader, V

    P. BehnamGhader, V. Adlakha, M. Mosbach, D. Bahdanau, N. Chapados, and S. Reddy. LLM2Vec : Large Language Models Are Secretly Powerful Text Encoders . 2024. doi:10.48550/ARXIV.2404.05961

    arXiv doi:10.48550/arxiv.2404.05961 2024

  6. [6]

    Y. Dai, Z. Yao, C. Song, Q. Zheng, W. Mai, K. Peng, S. Lu, W. Ouyang, J. Yang, and J. Wu. MindAligner : Explicit Brain Functional Alignment for Cross-Subject Visual Decoding from Limited fMRI Data . Feb. 2025

    work page 2025

  7. [7]

    Devlin, M.-W

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding . In Proceedings of the 2019 Conference of the North , pages 4171--4186, Minneapolis, Minnesota, 2019. Association for Computational Linguistics. doi:10.18653/v1/N19-1423

    doi:10.18653/v1/n19-1423 2019

  8. [8]

    Défossez, C

    A. Défossez, C. Caucheteux, J. Rapin, O. Kabeli, and J.-R. King. Decoding speech from non-invasive brain recordings, Oct. 2023. URL http://arxiv.org/abs/2208.12266

    arXiv 2023

Show all 34 references
  1. [9]

    Esteban, C

    O. Esteban, C. J. Markiewicz, R. W. Blair, C. A. Moodie, A. I. Isik, A. Erramuzpe, J. D. Kent, M. Goncalves, E. DuPre, M. Snyder, H. Oya, S. S. Ghosh, J. Wright, J. Durnez, R. A. Poldrack, and K. J. Gorgolewski. fMRIPrep : a robust preprocessing pipeline for functional MRI . N...

    2019 doi

  2. [10]

    Who " Is Saying

    E. Formisano, F. De Martino, M. Bonte, and R. Goebel. " Who " Is Saying " What "? Brain-Based Decoding of Human Voice and Speech . Science, 322 0 (5903): 0 970--973, Nov. 2008. ISSN 0036-8075, 1095-9203. doi:10.1126/science.1164318

    2008 doi

  3. [11]

    Gelman, J

    A. Gelman, J. B. Carlin, H. S. Stern, D. B. Dunson, A. Vehtari, and D. B. Rubin. Bayesian Data Analysis . Chapman and Hall/CRC , Nov. 2013. ISBN 978-0-429-11307-9. doi:10.1201/b16018

    2013 doi

  4. [12]

    J. V. Haxby, J. S. Guntupalli, A. C. Connolly, Y. O. Halchenko, B. R. Conroy, M. I. Gobbini, M. Hanke, and P. J. Ramadge. A common, high-dimensional model of the representational space in human ventral temporal cortex. Neuron, 72 0 (2): 0 404--416, Oct. 2011. ISSN 1097-4199. d...

    2011 doi

  5. [13]

    Kamitani and F

    Y. Kamitani and F. Tong. Decoding the visual and subjective contents of the human brain. Nature neuroscience, 8 0 (5): 0 679–685, 2005

    2005

  6. [14]

    LeBel, L

    A. LeBel, L. Wagner, S. Jain, A. Adhikari-Desai, B. Gupta, A. Morgenthal, J. Tang, L. Xu, and A. G. Huth. A natural language fMRI dataset for voxelwise encoding models. Scientific Data, 10 0 (1): 0 555, Aug. 2023. ISSN 2052-4463. doi:10.1038/s41597-023-02437-z. URL https://www...

    2023 doi

  7. [15]

    Levenshtein

    V. Levenshtein. Binary codes capable of correcting deletions, insertions, and reversals. Soviet physics. Doklady, 1965. URL https://www.semanticscholar.org/paper/Binary-codes-capable-of-correcting-deletions

    1965

  8. [16]

    J. C. Mazziotta, A. W. Toga, A. Evans, P. Fox, and J. Lancaster. A Probabilistic Atlas of the Human Brain : Theory and Rationale for Its Development : The International Consortium for Brain Mapping ( ICBM ). NeuroImage, 2 0 (2, Part A): 0 89--101, June 1995. ISSN 1053-8119. do...

    1995

  9. [17]

    Mentzelopoulos, E

    G. Mentzelopoulos, E. Chatzipantazis, A. G. Ramayya, M. J. Hedlund, V. P. Buch, K. Daniilidis, K. P. Kording, and F. Vitale. Neural decoding from stereotactic EEG : Accounting for electrode variability across subjects. Nov. 2024

    2024

  10. [18]

    Mokady, A

    R. Mokady, A. Hertz, and A. H. Bermano. ClipCap : CLIP Prefix for Image Captioning , Nov. 2021. URL http://arxiv.org/abs/2111.09734

    2021 arXiv

  11. [19]

    Ogawa, T

    S. Ogawa, T. M. Lee, A. R. Kay, and D. W. Tank. Brain magnetic resonance imaging with contrast dependent on blood oxygenation. Proceedings of the National Academy of Sciences of the United States of America, 87 0 (24): 0 9868--9872, Dec. 1990. ISSN 0027-8424. URL https://www.n...

    1990

  12. [20]

    Ozcelik and R

    F. Ozcelik and R. VanRullen. Natural scene reconstruction from fMRI signals using generative latent diffusion, June 2023. URL http://arxiv.org/abs/2303.05334

    2023 arXiv

  13. [21]

    Paszke, S

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala. Pytorch: An imperative style, high-per...

    2019

  14. [22]

    Pedregosa, G

    F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine Learning in Python . Journal of Machine Learnin...

    2011

  15. [23]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever. Learning Transferable Visual Models From Natural Language Supervision , Feb. 2021. URL http://arxiv.org/abs/2103.00020

    2021 arXiv

  16. [24]

    M. Reid, N. Savinov, D. Teplyashin, D. Lepikhin, T. Lillicrap, J.-B. Alayrac, R. Soricut, A. Lazaridou, O. Firat, J. Schrittwieser, I. Antonoglou, R. Anil, S. Borgeaud, A. M. Dai, K. Millican, E. Dyer, M. Glaese, T. Sottiaux, B.-j. Lee, F. Viola, M. Reynolds, Y. Xu, J. Molloy,...

    2024

  17. [25]

    P. S. Scotti, A. Banerjee, J. Goode, S. Shabalin, A. Nguyen, E. Cohen, A. J. Dempster, N. Verlinde, E. Yundler, D. Weisberg, K. A. Norman, and T. M. Abraham. Reconstructing the Mind 's Eye : fMRI -to- Image with Contrastive Learning and Diffusion Priors , May 2023. URL http://...

    2023 arXiv

  18. [26]

    P. S. Scotti, M. Tripathy, C. K. T. Villanueva, R. Kneeland, T. Chen, A. Narang, C. Santhirasegaran, J. Xu, T. Naselaris, K. A. Norman, and T. M. Abraham. MindEye2 : Shared - Subject Models Enable fMRI - To - Image With 1 Hour of Data , Mar. 2024. URL http://arxiv.org/abs/2403.11207

    2024 arXiv

  19. [27]

    Tang and A

    J. Tang and A. G. Huth. Semantic language decoding across participants and stimulus modalities. Current Biology, page S0960982225000545, Feb. 2025. ISSN 09609822. doi:10.1016/j.cub.2025.01.024

    2025 doi

  20. [28]

    J. Tang, A. LeBel, S. Jain, and A. G. Huth. Semantic reconstruction of continuous language from non-invasive brain recordings. Nature Neuroscience, 26 0 (5): 0 858--866, May 2023. ISSN 1546-1726. doi:10.1038/s41593-023-01304-9. URL https://www.nature.com/articles/s41593-023-01...

    2023 doi

  21. [29]

    Thual, H

    A. Thual, H. Tran, T. Zemskova, N. Courty, R. Flamary, S. Dehaene, and B. Thirion. Aligning individual brains with fused unbalanced Gromov Wasserstein . In Neural Information Processing Systems , June 2022

    2022

  22. [30]

    Thual, Y

    A. Thual, Y. Benchetrit, F. Geilert, J. Rapin, I. Makarov, H. Banville, and J.-R. King. Aligning brain functions boosts the decoding of visual semantics in novel subjects, 2023. URL http://arxiv.org/abs/2312.06467

    2023 arXiv

  23. [31]

    Tuckute, N

    G. Tuckute, N. Kanwisher, and E. Fedorenko. Language in Brains , Minds , and Machines . Annual Review of Neuroscience, 47 0 (1): 0 277--301, Aug. 2024. ISSN 0147-006X, 1545-4126. doi:10.1146/annurev-neuro-120623-101142

    2024 doi

  24. [32]

    F. R. Willett, E. M. Kunz, C. Fan, D. T. Avansino, G. H. Wilson, E. Y. Choi, F. Kamdar, L. R. Hochberg, S. Druckmann, K. V. Shenoy, and J. M. Henderson. A high-performance speech neuroprosthesis. Jan. 2023. doi:10.1101/2023.01.21.524489

    2023 doi

  25. [33]

    O. Yadan. Hydra - a framework for elegantly configuring complex applications. Github, 2019. URL https://github.com/facebookresearch/hydra

    2019

  26. [34]

    Z. Ye, Q. Ai, Y. Liu, M. de Rijke, M. Zhang, C. Lioma, and T. Ruotsalo. Generative language reconstruction from brain recordings. Communications Biology, 8 0 (1): 0 346, 2025

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.