Pith. sign in

REVIEW 3 major objections 5 minor 36 references

CognitionCapturer: Decoding Visual Stimuli From Human EEG Signal With Multimodal Information

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read CognitionCapturer reconstructs seen images by aligning EEG to image, text, and depth embeddings, then decoding through a frozen diffusion model.

desk verdict Solid image-only EEG decoding with a novel multi-expert framework, but the headline multimodal claim rests on an OR-rule upper bound and reconstruction metrics are mixed. read the letter →

arxiv 2412.10489 v2 pith:RBTXOFSB submitted 2024-12-13 cs.CV cs.AIeess.SP

Kaifan Zhang , Lihuo He , Xin Jiang , Wen Lu , Di Wang , Xinbo Gao This is my paper · ORCID
classification cs.CVcs.AIeess.SP
keywords EEGdecodingvisualstimulusreconstructionmultimodalcontrastivelearningmodalityexpertencoderdiffusionpriorCLIPembeddingspacedepthestimationimagecaptioning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that EEG signals carry visual information beyond what direct EEG-image alignment captures, and that this information can be tapped by aligning EEG to multiple derived modalities: captions, depth maps, and the images themselves. It reports that this multimodal alignment reaches 35.64% top-1 accuracy on a 200-way zero-shot classification task under an any-expert-correct rule, compared with 26.1% for a single-modality baseline, and that reconstructed images improve on both low-level and high-level similarity metrics. The authors argue that separate modality experts prevent fine-grained structural information from being overshadowed by coarse semantic information, and that the trained embeddings can drive a frozen generative model without fine-tuning. If the claim holds, EEG-based visual decoding becomes more practical and more extendable than image-pair-only approaches.

What carries the argument

The central mechanism is the Modality Expert Encoder family: for each EEG-modality pair, a dedicated network maps EEG into the embedding space of that modality's pretrained encoder. The alignment is trained with a modified InfoNCE loss that uses the image index as a label, pulling all repetitions of the same image together and avoiding the contradiction of identical pairs being both attracted and repelled. A diffusion prior trained from scratch with a mean-squared-error loss then maps the EEG embeddings into CLIP space, and the generation stage uses SDXL-Turbo with three IP-Adapters, full for image, style-only for text, and layout-only for depth, to combine semantic and structural information without fine-tuning the generative model.

What would settle it

Re-run the 200-way classification using a single fused decision rule, such as summing the three experts' logits or choosing the expert with the highest confidence, instead of the any-expert-correct rule; if fused top-1 does not beat the image-only expert's 33.30%, the multimodal gain is an artifact of the scoring rule.

Watch

Extended reading notes

Core claim

The central claim is that 'beyond-image-modality' information in EEG can be recovered by building one Modality Expert Encoder for each of three aligned modalities, rather than a single EEG-image contrastive model. Each expert projects EEG into the same embedding space as a pretrained encoder for the image, the BLIP2 caption, or the DepthAnything depth map, using a modified InfoNCE loss in which all EEG repetitions of the same image are pulled together. After training, a diffusion prior maps each embedding into CLIP image space, and SDXL-Turbo with dedicated IP-Adapters for image, style, and layout fuses the three embeddings to generate the reconstruction. The paper reports that the full multimodal system outperforms its own image-only expert and the ATM and NICE baselines in 200-way classification, and that the reconstructed images score higher on pixel-level and semantic metrics than single-modality reconstructions. It also claims that the text and depth experts attend to broader occipital and temporal brain regions than the image expert, supporting the idea that different modalities expose different neural content.

Load-bearing premise

The load-bearing premise is that the text and depth embeddings derived from each image carry information not already present in the CLIP image embedding, so aligning EEG to them adds 'beyond-image' signal rather than duplicating the same visual information.

Editorial extensions

If this is right

  • If the central claim is correct, EEG decoders can be trained without requiring exact image-EEG pairs as the only supervision; aligned captions and depth maps provide additional weak labels.
  • The EEG embeddings produced by the expert encoders support zero-shot classification and retrieval directly, not just generation.
  • Because the generative stage is frozen, swapping in a newer CLIP-compatible image generator or adapter should improve reconstruction without retraining the EEG side.
  • Adding further modalities, such as object masks, audio, or scene graphs, should continue to separate semantic from structural information, since each expert is trained independently.
  • Per-subject training on roughly 30 minutes of GPU time makes the approach practical for personalized decoding settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: The reported gain of the full system over the image-only expert (35.64% vs 33.30% top-1) should be rechecked under a single fused decision rule; if fused accuracy does not exceed the image expert, the benefit of text and depth may be an artifact of the any-expert-correct scoring.
  • Inference: Because CLIP image embeddings already encode depth cues, an ablation with shuffled depth maps would isolate whether the depth expert contributes genuinely new signal or merely a second route to the same visual information.
  • Inference: The Grad-CAM observation that the depth expert favors the right inferior temporal lobe suggests a testable prediction: a depth-trained decoder should be more sensitive than an image-trained decoder to disruption of that region.
  • Inference: The framework's modality-agnostic design implies that auxiliary modalities generated from the same images could be plugged in without architectural changes, which the paper does not demonstrate.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes CognitionCapturer, a framework for decoding visual stimuli from EEG by training separate Modality Expert Encoders to align EEG with image, text, and depth embeddings, then mapping the EEG embeddings to CLIP space via a diffusion prior and generating images with pretrained SDXL-Turbo and IP-Adapters. Experiments on Thing-EEG report 200-way zero-shot classification and reconstruction metrics, and the paper claims that CognitionCapturer outperforms state-of-the-art methods both qualitatively and quantitatively.

Significance. If the claims were fully supported, this would be a useful contribution to EEG-based visual decoding, showing that auxiliary multimodal embeddings can improve classification and reconstruction. The framework is extensible, and the authors have released code, which are strengths. However, the quantitative evidence as presented is not sufficient to support the headline claims: the classification gain over single-modality systems rests on an OR-rule upper bound, and the reconstruction comparison against the ATM baseline is mixed or unfavorable on several metrics. The paper's significance therefore depends on substantial revision of both the evaluation method and the claims.

major comments (3)
  1. [Results and Discussion, Table 2] The 'CognitionCapturer (all)' top-1 accuracy of 35.64% is computed with an OR rule: a test sample is counted correct if any one of the three Modality Expert Encoders classifies it correctly, as stated in the table footnote. This is an upper bound, not the accuracy of any deployable fused classifier. The abstract and conclusion claim that CognitionCapturer 'outperforms state-of-the-art methods both qualitatively and quantitatively', and the only number in Table 2 that beats ATM's 26.1% is this upper bound. The image-only row (33.30%) is a legitimate comparison, but the multimodal gain from 33.30% to 35.64% is an artifact of the OR rule. The paper should report a real fusion accuracy (e.g., an ensemble or a learned fusion) or clearly reframe the claim.
  2. [Table 3 and Supplementary Table 6] The reconstruction results do not support the claim of outperforming previous work quantitatively. In Table 3, the all-modality system is worse than the image-only system on AlexNet(2) and AlexNet(5) (0.754/0.623 vs 0.813/0.671), directly contradicting the main contribution statement that reconstruction performance surpasses 'that of any single modality'. In addition, the supplementary Table 6 reports ATM values of AlexNet(5)=0.866, Inception=0.734, and CLIP=0.786, all higher than CognitionCapturer-all (0.623, 0.669, 0.715). The main text statement that the method 'outperforms previous work (Li et al. 2024) in the fine-grained alignment of reconstructed visual stimuli' is not supported by these numbers; the authors need to reconcile the comparison or clearly specify in which metrics their method is superior.
  3. [Introduction and Modality Expert Encoder (Section 3.1)] The paper's central premise is that text and depth modalities provide 'beyond-image-modality' information that improves EEG decoding. However, the paper itself cites Zhang et al. (2022) to state that CLIP image embeddings already contain depth information, and the text and depth embeddings are deterministically derived from the same images via BLIP2 and DepthAnything. The observed OR-rule classification gain could simply reflect that different experts make different errors rather than that they extract complementary EEG information. To justify the claimed contribution, the authors should provide evidence that a single image-only expert of matched capacity cannot achieve the same or better performance, or directly quantify whether the text/depth embeddings add information beyond the CLIP image embedding in the EEG decoding task.
minor comments (5)
  1. [Table 2 caption] The caption says 'acc±std' but no standard deviations are shown anywhere in the table; please report the standard deviations or correct the caption.
  2. [Table 3] The ATM baseline is absent from Table 3 and appears only in supplementary Table 6. Since the paper compares against ATM throughout, please include the ATM row in the main table or clearly cross-reference the supplementary values.
  3. [Figure 3 caption] The caption states that 'selected reconstruction results from subject-08' are shown, but the selection criteria are not described; please clarify whether these are representative, best, or cherry-picked examples.
  4. [Supplementary, Modality Expert Encoder] The main text refers to 'STConv' while the supplementary refers to 'TSConv' for the same module; please unify the terminology.
  5. [Section 4.4 (Grad-CAM analysis)] The claim that different Modality Expert Encoders focus on different brain regions is based on visual inspection of Grad-CAM maps; a quantitative comparison (e.g., spatial centroid distance or overlap coefficient) would make the claim more rigorous.

Circularity Check

1 steps flagged · score 6.0 of 10

Multimodal classification gain is an OR-rule upper bound defined into the metric; image-only SOTA remains independent.

  1. self definitional [Results and Discussion, Classification Performance; Table 2 caption and footnote]
    "(In the calculation of CognitionCapturer (all)’s classification accuracy, if any Modality Expert Encoder correctly classifies a sample, the sample is considered correctly classified.) ... we combined the top-5 results from three modalities, as shown in the upper bound row of Table 2. The results indicate ... the model gains access to more complementary information ... Note: This does not represent the actual accuracy that can be achieved in practice."

    The 'all' accuracy is defined as the union of the three Modality Expert Encoders' predictions, so it is an upper bound on any real fused classifier. The reported multimodal gain (35.64 vs 33.30 top-1) is therefore guaranteed by the scoring rule: the union of three correct sets is necessarily at least as large as the image-only correct set. No actual fused classifier achieves 35.64, and the paper admits this in the footnote. Presenting this row as evidence that 'the model gains access to more complementary information' reduces an empirical claim to the definition of the evaluation metric. The complementary-information conclusion is not derived from data; it is constructed by the OR rule.

full rationale

The paper's core derivation is otherwise self-contained: the Modality Expert Encoders are trained with an InfoNCE-style loss on held-out concepts, the diffusion prior is trained from scratch to map EEG embeddings into CLIP space, and reconstruction uses frozen SDXL/IP-Adapters with multiple independent evaluation metrics (PixCorr, SSIM, AlexNet, Inception, SwAV, plus CLIP). The CLIP-space training and CLIP evaluation overlap somewhat, but the reported reconstruction metrics include several that are not directly optimized, so that alone is not a forced circularity. The references to prior work are mostly methodological (e.g., following Song et al.'s training strategy); the one co-authored citation (Song et al. 2024, which includes Xinbo Gao) is not load-bearing for the main novelty. The significant circular step is the 'CognitionCapturer (all)' classification row: its accuracy is defined as an OR over the three experts, making the multimodal improvement over any single expert true by construction. Since this row is used to support the claim that multiple modalities contribute complementary information, that particular quantitative claim reduces to the definition of the metric. The image-only accuracy (33.30 vs ATM's 26.1) does provide independent evidence for state-of-the-art classification, and the reconstruction results are mixed rather than uniformly favorable, so the overall score is partial circularity rather than total.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the assumption that EEG contains decodable multimodal information, that CLIP is an appropriate shared embedding space, and that the 'any modality correct' evaluation rule meaningfully measures complementary information. No new physical entities are introduced.

free parameters (4)
  • InfoNCE temperature tau = not explicitly reported
    Temperature in loss functions (2)-(3) controls alignment sharpness; value chosen without sensitivity analysis.
  • Diffusion prior guidance scale = 7.5
    Guidance scale for classifier-free guidance during reconstruction, chosen by hand.
  • IP-Adapter scales = image 1.0, text/depth layout/style 0 as appropriate
    Hand-set to emphasize layout/style per modality.
  • Batch size and learning rate = batch 1024, lr 3e-4 for contrastive; batch 512 for diffusion prior
    Hyperparameters tuned on subject-08 only (Table 5).
assumptions (4)
  • domain assumption EEG signals contain decodable visual information when averaged over repetitions.
    All of the method relies on this premise; established in prior work, not verified here.
  • domain assumption CLIP embedding space is a suitable shared space for EEG, image, text, and depth embeddings.
    Used throughout to make contrastive learning and diffusion prior possible.
  • domain assumption The text (BLIP2) and depth (DepthAnything) embeddings of the stimuli are accurate and complementary.
    These embeddings are generated off-the-shelf and assumed to carry task-relevant information.
  • ad hoc to paper The 'any modality correct' accuracy is a meaningful upper bound for achievable fusion accuracy.
    This evaluation rule is introduced solely to demonstrate complementary information; it is not a practical classifier and is not used to produce a real fused prediction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CognitionCapturer: Decoding Visual Stimuli From Human EEG Signal With Multimodal Information." pith.science (2026). https://pith.science/paper/RBTXOFSB

@misc{pith2026241210489,
  author       = {Pith},
  title        = {Pith review of: CognitionCapturer: Decoding Visual Stimuli From Human EEG Signal With Multimodal Information},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RBTXOFSB}},
  note         = {Machine review of arXiv:2412.10489}
}
read the original abstract

Electroencephalogram (EEG) signals have attracted significant attention from researchers due to their non-invasive nature and high temporal sensitivity in decoding visual stimuli. However, most recent studies have focused solely on the relationship between EEG and image data pairs, neglecting the valuable ``beyond-image-modality" information embedded in EEG signals. This results in the loss of critical multimodal information in EEG. To address this limitation, we propose CognitionCapturer, a unified framework that fully leverages multimodal data to represent EEG signals. Specifically, CognitionCapturer trains Modality Expert Encoders for each modality to extract cross-modal information from the EEG modality. Then, it introduces a diffusion prior to map the EEG embedding space to the CLIP embedding space, followed by using a pretrained generative model, the proposed framework can reconstruct visual stimuli with high semantic and structural fidelity. Notably, the framework does not require any fine-tuning of the generative models and can be extended to incorporate more modalities. Through extensive experiments, we demonstrate that CognitionCapturer outperforms state-of-the-art methods both qualitatively and quantitatively. Code: https://github.com/XiaoZhangYES/CognitionCapturer.

Figures

Figures reproduced from arXiv: 2412.10489 by the authors.

Figure 1
Figure 1. We believe that for image-EEG pairs, relying [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overall framework of CognitionCapturer. 1: In the contrastive learning stage, different EEG-Modality data pairs are [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Visual Comparison. Selected reconstruction results from subject-08 show that our reconstructed visual stimuli exhibit [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Reconstruction results of CognitionCapturer on [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: (A) The Grad-CAM results from different Modal [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Subject-08’s Best, Medium, and Worst images selected based on the Pixcorr metric [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Subject-08’s Best, Medium, and Worst images selected based on the SwAV metric. Visual Stimuli Reconstructed Stimuli Visual Stimuli Reconstructed Stimuli Visual Stimuli Reconstructed Stimuli Best Medium Worst [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Subject-05’s Best, Medium, and Worst images selected based on the Pixcorr metric [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Subject-05’s Best, Medium, and Worst images selected based on the SwAV metric. Low-level High-level Subject Pixcorr↑ SSIM↑ AlexNet(2) ↑ AlexNet(5) ↑ Inception↑ CLIP↑ SwAV↓ 1 0.148 0.334 0.741 0.626 0.666 0.711 0.592 2 0.147 0.344 0.764 0.618 0.661 0.725 0.590 3 0.140 0…
Figure 10
Figure 10. Figure 10: The input topographies of the EEG signals for all subjects, along with the brain regions attended to by the different [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Grad-CAM visualizations of the image regions attended to by different Modality Expert Encoders on example [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 19 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    J.; St-Yves, G.; Wu, Y.; Breedlove, J

    Allen, E. J.; St-Yves, G.; Wu, Y.; Breedlove, J. L.; Prince, J. S.; Dowdle, L. T.; Nau, M.; Caron, B.; Pestilli, F.; Charest, I.; et al. 2022. A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence. Nature neuroscience, 25(1): 116--126

    work page 2022

  4. [4]

    Benchetrit, Y.; Banville, H.; and King, J.-R. 2024. Brain decoding: toward real-time reconstruction of visual perception. arXiv:2310.19812

    arXiv 2024

  5. [5]

    Cherti, M.; Beaumont, R.; Wightman, R.; Wortsman, M.; Ilharco, G.; Gordon, C.; Schuhmann, C.; Schmidt, L.; and Jitsev, J. 2023. Reproducible scaling laws for contrastive language-image learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2818--2829

    2023

  6. [6]

    D \'e fossez, A.; Caucheteux, C.; Rapin, J.; Kabeli, O.; and King, J.-R. 2023. Decoding speech perception from non-invasive brain recordings. Nature Machine Intelligence, 5(10): 1097--1107

    work page 2023

  7. [7]

    J.; and Cox, D

    DiCarlo, J. J.; and Cox, D. D. 2007. Untangling invariant object recognition. Trends in cognitive sciences, 11(8): 333--341

    work page 2007

  8. [8]

    Du, C.; Fu, K.; Li, J.; and He, H. 2023. Decoding Visual Neural Representations by Multimodal Learning of Brain-Visual-Linguistic Features. IEEE Transactions on Pattern Analysis and Machine Intelligence

    work page 2023

Show all 36 references
  1. [9]

    Epstein, R.; and Kanwisher, N. 1998. A cortical representation of the local visual environment. Nature, 392(6676): 598--601

    1998

  2. [10]

    T.; Dwivedi, K.; Roig, G.; and Cichy, R

    Gifford, A. T.; Dwivedi, K.; Roig, G.; and Cichy, R. M. 2022. A large and rich EEG dataset for modeling human visual object recognition. NeuroImage, 264: 119754

    2022

  3. [11]

    Gu, Z.; Jamison, K.; Kuceyeski, A.; and Sabuncu, M. R. 2024. Decoding natural image stimuli from fMRI data with a surface-based convolutional network. In Medical Imaging with Deep Learning, 107--118. PMLR

    2024

  4. [12]

    He, K.; Fan, H.; Wu, Y.; Xie, S.; and Girshick, R. 2020. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 9729--9738

    2020

  5. [13]

    N.; Contier, O.; Teichmann, L.; Rockter, A

    Hebart, M. N.; Contier, O.; Teichmann, L.; Rockter, A. H.; Zheng, C. Y.; Kidder, A.; Corriveau, A.; Vaziri-Pashkam, M.; and Baker, C. I. 2023. THINGS-data, a multimodal collection of large-scale datasets for investigating object representations in human brain and behavior. Eli...

    2023

  6. [14]

    N.; Naselaris, T.; Prenger, R

    Kay, K. N.; Naselaris, T.; Prenger, R. J.; and Gallant, J. L. 2008. Identifying natural images from human brain activity. Nature, 452(7185): 352--355

    2008

  7. [15]

    Li, D.; Wei, C.; Li, S.; Zou, J.; and Liu, Q. 2024. Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion. arXiv:2403.07721

    2024 arXiv

  8. [16]

    Li, J.; Li, D.; Savarese, S.; and Hoi, S. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, 19730--19742. PMLR

    2023

  9. [17]

    S.; Ahmed, H.; Ilyevsky, T

    Li, R.; Johansen, J. S.; Ahmed, H.; Ilyevsky, T. V.; Wilbur, R. B.; Bharadwaj, H. M.; and Siskind, J. M. 2020. The perils and pitfalls of block design for EEG classification experiments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(1): 316--333

    2020

  10. [18]

    Liu, D.; Dai, W.; Zhang, H.; Jin, X.; Cao, J.; and Kong, W. 2023 a . Brain-machine coupled learning method for facial emotion recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(9): 10703--10717

    2023

  11. [19]

    Liu, Y.; Ma, Y.; Zhou, W.; Zhu, G.; and Zheng, N. 2023 b . BrainCLIP: Bridging Brain and Visual-Linguistic Representation Via CLIP for Generic Natural Visual Stimulus Decoding. arXiv:2302.12971

    2023 arXiv

  12. [20]

    A.; Boroditsky, L.; and Clark, A

    Lupyan, G.; Rahman, R. A.; Boroditsky, L.; and Clark, A. 2020. Effects of language on visual perception. Trends in cognitive sciences, 24(11): 930--944

    2020

  13. [21]

    C.; Sadato, N.; and Kamitani, Y

    Miyawaki, Y.; Uchida, H.; Yamashita, O.; Sato, M.-a.; Morito, Y.; Tanabe, H. C.; Sadato, N.; and Kamitani, Y. 2008. Visual image reconstruction from human brain activity using a combination of multiscale local image decoders. Neuron, 60(5): 915--929

    2008

  14. [22]

    J.; Kay, K

    Naselaris, T.; Prenger, R. J.; Kay, K. N.; Oliver, M.; and Gallant, J. L. 2009. Bayesian reconstruction of natural images from human brain activity. Neuron, 63(6): 902--915

    2009

  15. [23]

    W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

    Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, 8748--8763. PMLR

    2021

  16. [24]

    Ramesh, A.; Dhariwal, P.; Nichol, A.; Chu, C.; and Chen, M. 2022. Hierarchical Text-Conditional Image Generation with CLIP Latents. arXiv:2204.06125

    2022 arXiv

  17. [25]

    Rauschecker, J. P. 1995. Compensatory plasticity and sensory substitution in the cerebral cortex. Trends in neurosciences, 18(1): 36--43

    1995

  18. [26]

    Ren, Z.; Li, J.; Xue, X.; Li, X.; Yang, F.; Jiao, Z.; and Gao, X. 2021. Reconstructing seen image from brain activity by visually-guided cognitive representation and adversarial learning. NeuroImage, 228: 117602

    2021

  19. [27]

    Sauer, A.; Lorenz, D.; Blattmann, A.; and Rombach, R. 2023. Adversarial Diffusion Distillation. arXiv:2311.17042

    2023 arXiv

  20. [28]

    Scotti, P.; Banerjee, A.; Goode, J.; Shabalin, S.; Nguyen, A.; Dempster, A.; Verlinde, N.; Yundler, E.; Weisberg, D.; Norman, K.; et al. 2024. Reconstructing the mind's eye: fMRI-to-image with contrastive learning and diffusion priors. Advances in Neural Information Processing...

    2024

  21. [29]

    R.; Cogswell, M.; Das, A.; Vedantam, R.; Parikh, D.; and Batra, D

    Selvaraju, R. R.; Cogswell, M.; Das, A.; Vedantam, R.; Parikh, D.; and Batra, D. 2017. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, 618--626

    2017

  22. [30]

    Song, Y.; Liu, B.; Li, X.; Shi, N.; Wang, Y.; and Gao, X. 2024. Decoding Natural Images from EEG for Object Recognition . In International Conference on Learning Representations

    2024

  23. [31]

    Takagi, Y.; and Nishimoto, S. 2023. High-Resolution Image Reconstruction With Latent Diffusion Models From Human Brain Activity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 14453--14463

    2023

  24. [32]

    van den Oord, A.; Li, Y.; and Vinyals, O. 2019. Representation Learning with Contrastive Predictive Coding. arXiv:1807.03748

    2019 arXiv

  25. [33]

    N.; Kaiser, .; and Polosukhin, I

    Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. Advances in neural information processing systems, 30

    2017

  26. [34]

    Yang, L.; Kang, B.; Huang, Z.; Xu, X.; Feng, J.; and Zhao, H. 2024. Depth anything: Unleashing the power of large-scale unlabeled data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10371--10381

    2024

  27. [35]

    Ye, H.; Zhang, J.; Liu, S.; Han, X.; and Yang, W. 2023. IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models. arXiv:2308.06721

    2023 arXiv

  28. [36]

    Zhang, R.; Zeng, Z.; Guo, Z.; and Li, Y. 2022. Can language understand depth? In Proceedings of the 30th ACM International Conference on Multimedia, 6868--6874

    2022

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.