{"id":"f2a68b8c-fe7c-40b4-b74e-0bf302d3d766","arxiv_id":"2506.11123","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"Layer-wise sparse autoencoder features from vision models are cosine-matched to voxel-level fMRI responses, creating a training-free voxel-to-feature dictionary for the ventral visual pathway.","lead":"The paper trains sparse autoencoders on vision model activations and lines up the resulting features with fMRI responses in the human visual cortex, reporting similarities up to 0.76. A generalist might read it because it promises a training-free way to map what vision models compute onto brain areas, though the statistical evidence is thinner than the headline.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 0.76 headline similarity is the maximum over many voxel-unit-layer comparisons and has no null baseline, so shared stimulus statistics or selection could produce it by chance.","rationale":"The paper attempts something valuable: using SAEs trained without brain data as a fixed, interpretable dictionary, then aligning unit activation profiles to voxel fMRI profiles on shared natural images. If the alignment is real, it is a genuinely training-free voxel-wise bridge. There is qualitative evidence in favor: the top-image selectivity for face/body/scene/word/food units is consistent with ROI priors, and the layer alignment broadly matches FactorTopy and Max R2 (Table 3). I therefore do not think the method is vacuous. The load-bearing weakness is that the quantitative headline—maximum cosine similarity 0.76, and the Table 1 max/avg values—comes from a massive implicit search (voxels × units × layers) with no null distribution and no multiple-comparison control. The alternative explanation is concrete: any set of model features that covaries with natural-image statistics, even randomly initialized SAEs, will contain some unit whose activation profile matches some voxel's fMRI profile by chance; z-scoring does not remove this. The reader's weakest_assumption identified exactly this, and I agree. The authors' own Limitations section (Section 5) concerns SAE training hyperparameters and does not address statistical validity; the NeurIPS checklist claims significance for Table 1, but the table reports no error bars or tests. The proposed permutation test on Equation 4 is the minimal check that would settle whether the concern lands. If the observed maxima exceed the selection null, the central claim survives; if not, the abstract's 'strong correspondence' and 'direct bridge' need to be retracted or substantially weakened. Until then, the appropriate verdict stays conditional, unchanged from the reader's assessment.","tokens_in":29325,"tokens_out":6477,"duration_ms":87152,"concrete_test":"Construct a permutation null for Equation 4. For each subject, take the fMRI response matrix B (images × voxels) and the layer-wise SAE activation matrices Z^{(j)}. Null samples: permute the image-order of B jointly across voxels (preserving cross-voxel covariance), and separately permute B's rows independently per voxel (destroying covariance); for each permutation, recompute M_simi^{(j)} for all j, units, and voxels and record (i) the global maximum and (ii) the mean over voxels of their per-unit/layer maxima. Repeat at least 200 times. If the observed 0.76 and the Table 1 max/avg values fall within the null's 95% interval, the headline correspondence is not established. A useful control is the same pipeline with raw model activations or random linear projections in place of SAE units; if max similarity is comparable, the SAE dictionary adds nothing beyond shared image statistics.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on Equation 4: cosine similarity between a voxel's z-scored fMRI profile across images and a SAE unit's z-scored activation profile across the same images. The paper reports a maximum of 0.76, selected over V voxels (thousands), K=16d units per layer (12,288 for ViT-B/16), and L layers (~12). No permutation, surrogate, or analytic null is provided, so a large selection maximum cannot be interpreted. This is not a technicality: natural-image statistics guarantee positive correlations between many model features and many voxels even with no true model-brain correspondence, because both respond to low-level image content (luminance, edges, spatial frequency). Z-scoring removes each unit's and voxel's mean and scale across images but does not remove the shared image structure. The per-voxel average maximum ~0.26 in Table 1 is precisely the quantity that needs a null; the global 0.76 is the extreme of an enormous search. The paper's own NeurIPS checklist says statistical significance is shown in Table 1, but Table 1 contains no error bars, p-values, or null comparisons. Until a null distribution is provided, the phrase 'strong activation correspondence' and the abstract's 'direct bridge' overstate what Equation 4 demonstrates.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces SAE-BrainMap, a framework that trains sparse autoencoders (SAEs) on the layer activations of several vision models (ViT-B/16 CLIP, ImageNet, MAE, DiNOv2, SAM, ResNet50 CLIP) without any brain data, and then correlates the SAE unit activation profiles with voxel-level fMRI responses from the Natural Scenes Dataset using cosine similarity. The authors report a maximum similarity of 0.76, construct a voxel dictionary by assigning to each voxel the most similar SAE unit, and use this dictionary to argue that SAE units preserve the functional structure of visual ROIs, to establish a layer-wise brain-model alignment, and to visualize model information processing on the cortical surface. The main claims are that SAE features provide a direct, downstream-task-free bridge between deep network representations and the human ventral visual pathway.","tokens_in":29605,"tokens_out":3122,"duration_ms":39190,"significance":"If the central similarity result is statistically validated, the paper offers a conceptually attractive approach: features extracted by SAEs trained purely on model activations, with no supervised brain-encoding objective, might align voxel-wise with cortical fMRI responses. The work is also broad in scope, covering multiple architectures, public data, and several analyses, and the SAEs themselves are trained without brain data, which avoids the most obvious form of circularity. However, the key quantitative claims currently lack the statistical baselines needed to distinguish genuine structure from shared stimulus statistics and selection effects. The paper is therefore potentially interesting and publishable in a revised form, but the headline 'strong activation correspondence' is not yet supported by the evidence as presented.","major_comments":[{"comment":"The headline maximum similarity of 0.76 is the maximum over subjects, voxels, SAE units, layers, and models, and Table 1 reports no null distribution, p-values, or error bars for either the maximum or the per-voxel average (~0.26). Because natural images share low-level statistics (luminance, edges, spatial frequency), positive cosine similarities between model features and voxel responses are expected even with no true model-brain correspondence, and the maximum of an enormous search is especially uninterpretable. The paper's own NeurIPS checklist states that statistical significance is illustrated in Tables 1-3, but those tables contain no error bars or significance tests. The authors should provide a permutation or surrogate baseline (e.g., shuffling image labels, using units from untrained models, or an analytic null for the maximum of many correlations) and report selection-corrected statistics; without this, the abstract's 'strong activation correspondence' and 'direct bridge' overstate what Eq. (4) demonstrates.","section":"Section 3.2, Eq. (4); Table 1"},{"comment":"The claim that SAE units preserve the functional structure of ROIs is validated by computing an RSA score between the voxel dictionary and 'brain encoder weights' that were themselves trained to predict fMRI responses from the same model activations (Appendix C). Since the voxel dictionary is constructed by an argmax over the same similarity matrix used throughout the paper, and since both the dictionary and the ground-truth encoder weights derive from the same model activation statistics, the reported RSA values (highest 0.516) may reflect shared ancestry rather than an independent correspondence between SAE features and brain function. The authors should validate the dictionary against held-out fMRI data, or against an independently defined functional atlas, and should also report a null distribution for the RSA scores.","section":"Section 4.3, Eq. (6); Table 2"},{"comment":"The brain-model layer alignment C is obtained by taking an argmax over layers of the per-layer maximum over SAE units, and the 'slope' metric is computed on this selected alignment without any null model. The similarities to FactorTopy and Max R2 are presented without confidence intervals, and the text states that the results are 'well correlated with previous methods' without a statistical comparison. The authors should report bootstrap intervals across subjects and voxels, and compare the slope and alignment similarity against a permutation baseline in which layer labels are shuffled, before claiming a 'clear and consistent hierarchical correspondence' between model layers and the human visual cortex.","section":"Section 4.4, Eq. (5); Table 3"}],"minor_comments":[{"comment":"There are numerous typos and grammatical errors that should be corrected: 'Relate Work' (Section 2), 'constrainted' (Introduction), 'dosen't' (checklist item 3), 'V oxels' (multiple occurrences), 'V oxel Dictionary' (figure captions), and inconsistent capitalization throughout.","section":"Throughout"},{"comment":"The notation is inconsistent: Eq. (1) defines Z with a ReLU nonlinearity, but Eq. (4) computes Z_zscore = Zscore(W A + b) without the ReLU. The definition of the norm in Eq. (4) is also ambiguous (Frobenius norm vs. column-wise norms), and the matrix dimensions in the cosine expression should be stated explicitly.","section":"Section 3.1, Eqs. (1)-(4)"},{"comment":"The text refers to 'Figure 4' for the unit selectivity visualization and for the voxel dictionary / brain encoder similarity matrix, but the relevant figures appear to be Figures 4 and 5 in the paper; the references should be corrected and the figures should be cross-checked.","section":"Section 4.3, Figure 5 and Figure 4"},{"comment":"The text says 'Table 2 presents the cosine similarity between our alignment results and two voxel-wise mapping methods,' but the table referenced is Table 3. The similarity columns are also not clearly defined: the reader should be told whether the values are correlations over voxel labels, IoU-like scores, or something else.","section":"Section 4.4, Table 3"},{"comment":"The limitations section mentions only the fixed SAE hyperparameters. It does not mention the lack of statistical baselines, the selection over many comparisons, or the fact that the brain-encoder validation is model-dependent. These are important limitations that should be acknowledged.","section":"Section 5, Limitations"}],"recommendation":"major_revision","confidential_remarks":"The paper is clearly a work in progress, and the NeurIPS checklist answers are not always consistent with the content (e.g., item 7 says statistical significance is shown, but the tables contain no error bars or tests). The central idea is interesting and the computational investment is substantial, but the main quantitative claim rests on a maximum correlation selected over thousands of units and voxels with no null baseline. I would not reject the paper: the issue is fixable with additional analyses. However, if the authors cannot provide a valid null distribution or selection-corrected statistics, the 'strong activation correspondence' claim should be substantially weakened. Also, I did not see the code referenced in the checklist; reproducibility would benefit from an explicit link or archive."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The new thing here is real: using SAE unit dictionaries for voxel-wise model-brain alignment without fitting a brain encoder is a new application, and the qualitative results (units matching FFA/EBA/RSC/VWFA/FOOD selectivity, layer maps that reproduce the known ventral hierarchy) look convincing at the eyeball level. If the similarity numbers held up, this would be a handy training-free tool for comparing any vision model to the human ventral stream. The writing is clear, the NSD preprocessing is standard, and testing six architectures is a reasonable scope. The layer-to-cortex alignment reproducing earlier encoding results is a solid sanity check, not a contradiction.\n\nThe load-bearing weakness is exactly what the stress test flagged: the 0.76 headline is the maximum over thousands of voxels, 12,288 units per layer, and a dozen layers, with no permutation null or multiple-comparison correction. The per-voxel average max around 0.26 in Table 1 is the quantity that needs a null, because natural image statistics will make many model features and many voxels positively correlated by chance. The NeurIPS checklist says Table 1 shows statistical significance, but Table 1 contains no error bars, p-values, or null comparisons. That mismatch between claim and evidence is the main problem.\n\nThe second soft spot is the voxel dictionary validation. The brain encoder weights used as 'ground truth' are fitted from the same model features, and the RSA scores around 0.5 may partly reflect that both dictionaries are linear functions of the same activations. The main SAE-to-fMRI similarity itself is not circular, but the validation is less independent than it appears. Third, the limitations section only mentions SAE hyperparameters, not the missing null or selection issue. Minor issues: no public code despite the checklist saying code is in supplemental, and some notation typos in Equations 4 and 5.\n\nWho this is for: people in model interpretability and fMRI encoding who want a voxel-wise bridge that does not require training a downstream encoder. They will find the SAE-BrainMap construction useful even if the headline number is not yet interpretable. The fix is straightforward: add a permutation null (shuffle voxel-unit assignments, or use image-label permutations), report corrected maxima and error bars, and reframe the abstract to match what the statistics support. This paper deserves a serious referee because the application is novel and the core idea is promising; it needs heavy revision, not rejection on novelty grounds.","headline":"Genuinely new SAE-to-fMRI alignment method whose headline 0.76 similarity lacks any null baseline, so the core claim overreaches but the tool is worth referee time.","tokens_in":30126,"tokens_out":1798,"would_cite":false,"duration_ms":25497,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Sparse autoencoder units trained without brain data match human visual cortex fMRI responses, with cosine similarity up to 0.76.","keywords":["sparse autoencoders","model-brain alignment","ventral visual pathway","fMRI responses","visual cortex","representational similarity","voxel dictionary","interpretability"],"falsifier":"A permutation test that shuffles the voxel-unit correspondence, or replaces trained SAE features with random features, and recomputes the maximum cosine similarities; if random features still yield maxima around 0.7, the claimed strong correspondence would reflect shared stimulus statistics rather than learned representations.","tokens_in":1560,"feed_emoji":"🧠","tokens_out":4351,"duration_ms":85760,"temperature":0.7,"pith_summary":"The paper claims that sparse autoencoders trained purely on vision-model activations, with no brain data, produce units whose activation patterns track voxel-level fMRI responses in the human ventral visual pathway when both are elicited by the same natural images; the maximum cosine similarity reaches 0.76. On this basis it builds a voxel dictionary that assigns each voxel its most similar SAE feature, showing that SAE units preserve the functional organization of known brain regions and display selectivity consistent with those regions. It then maps each model layer to cortical areas and uses the brain as a spatial template to trace how visual information transforms across layers. If correct, this yields a direct, downstream-task-free bridge between deep networks and the human visual cortex, using features learned without supervision from brain responses.","feed_headline":"SAE units match brain fMRI voxels up to 0.76","feed_subtitle":"Sparse autoencoders trained without brain data align with the human ventral visual pathway, mapping layers to cortex.","key_machinery":"The central object is a sparse autoencoder trained per layer with tied encoder-decoder weights, reconstructing activations as sparse positive combinations of interpretable feature units: $Z = \\mathrm{ReLU}(WA+b)$, with loss $\\|\\hat{A}-A\\|_2^2 + \\alpha\\|Z\\|_1$. The load-bearing comparison is the cosine similarity between z-scored SAE unit activations and z-scored voxel fMRI responses across the shared image set, $M^{(j)}_{\\mathrm{simi}} = \\cos(B^{\\top}/\\|B\\|_2, Z^{(j)}_{\\mathrm{zscore}}/\\|Z^{(j)}_{\\mathrm{zscore}}\\|_2)$, normalized over images. This similarity matrix drives two constructions: a voxel dictionary $D^{(j)}[x] = W[\\arg\\max_y M_{x,y}]$ assigning each voxel its most similar feature, and a voxel-wise layer alignment $C[x] = \\arg\\max_j \\max_y M^{(j)}_{x,y}$. Projecting dictionary activations $S^{(j)} = A^{(j)}D^{(j)}$ onto the cortex lets the brain act as a spatial template for tracing information flow through the model.","core_discovery":"Training layer-wise sparse autoencoders on model activations from natural images yields features whose z-scored activation patterns correlate strongly with z-scored fMRI responses of individual voxels, with cosine similarity up to 0.76. The most similar SAE feature per voxel forms a 'voxel dictionary' that preserves ROI functional structure, and the best-matching layer per voxel reconstructs the classic hierarchy, with early layers aligning to early visual cortex and deeper layers to higher-level visual areas. For CLIP ViT-B/16, the voxel-dictionary activation path suggests low-level features are transformed into abstract semantic information in early layers and later reconstructed alongside high-level information. The paper claims this constitutes a direct, downstream-task-free alignment between deep neural network representations and the human visual cortex.","pith_inferences":["The reported 0.76 maximum is selected across thousands of voxel-unit-layer comparisons, and the paper gives no permutation or null baseline; a natural test is whether randomly initialized SAEs or shuffled voxel labels produce similar maxima driven by shared stimulus statistics.","If the alignment survives such nulls, the method becomes a zero-shot cortical mapper: any new model layer could be localized on the cortex without training a brain encoder, as long as paired image-fMRI data are available.","The ROI-consistent selectivity of SAE units could be turned into a generative stimulus-search tool, using units tied to face, scene, or word regions to synthesize images and probe cortical selectivity directly.","Applying the same framework to recurrent or multi-scale architectures might reveal whether the bidirectional low- and high-level information flow observed in CLIP ViT-B/16 is specific to transformer residual streams or a general property of deep visual processing."],"forward_implications":["SAE features learned from images alone can be matched to individual fMRI voxels without any supervised brain encoder, providing a training-free voxel-wise alignment procedure.","The voxel dictionary inherits known ROI structure, so model features can be probed for face, body, scene, word, and food selectivity in a data-driven way.","Layer-to-cortex mapping reproduces the ventral-stream hierarchy, with early layers near V1 and later layers near high-level visual areas, across several architectures and training objectives.","For CLIP ViT-B/16, early layers convert low-level information into abstract semantic content while later layers re-integrate low-level details, a pattern the paper links to the residual stream.","The framework extends to any vision model with per-layer activations, making it a general tool for comparing model internals against human visual cortex."],"supporting_citations":[{"why":"Supplies the sparse autoencoder architecture, tied decoder weights, and L1 reconstruction loss that the paper adopts for each model layer.","marker":"[18]"},{"why":"Provides the fMRI dataset with z-scored voxel responses and shared natural image stimuli used for all similarity computations.","marker":"[20]"},{"why":"Gives the brain-decoding layer-alignment method and hierarchy slope metric used as comparison baselines for the proposed alignment.","marker":"[2]"},{"why":"Defines the brain encoder whose weights are treated as ground-truth cortical functional structure and supplies Max R2 and FactorTopy baselines.","marker":"[14]"},{"why":"Contributes the bilinear-interpolation activation heatmap procedure used to display unit selectivity for ROI-associated features.","marker":"[24]"},{"why":"Provides the CLIP ViT-B/16 visual backbone that yields the highest similarity scores and the main information-dynamics analysis.","marker":"[53]"}],"fun_headline_variants":["SAEs bridge deep learning and brain with 0.76 match","Sparse autoencoders map AI features to brain voxels","AI neurons correlate with fMRI at 0.76 similarity","SAE units mirror brain activity across visual hierarchy","Layer-wise SAEs align with human visual cortex"],"cache_read_input_tokens":32256,"weakest_assumption_plain":"The load-bearing premise is that cosine similarity between activation profiles in response to the same images measures functional correspondence; the paper reports maxima and averages without a null distribution, so chance alignment due to shared image statistics is not ruled out.","fun_headline_variants_meta":{"raw":{"variants":["SAEs bridge deep learning and brain with 0.76 match","Sparse autoencoders map AI features to brain voxels","AI neurons correlate with fMRI at 0.76 similarity","SAE units mirror brain activity across visual hierarchy","Layer-wise SAEs align with human visual cortex"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000759,"raw_usage":{"total_tokens":3359,"prompt_tokens":920,"completion_tokens":2439,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":536,"completion_tokens_details":{"reasoning_tokens":2359}},"tokens_in":536,"tokens_out":2439,"duration_ms":21577,"temperature":1.0,"reasoning_tokens":2359,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T05:09:39.918641+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A permutation test that shuffles the voxel-unit correspondence, or replaces trained SAE features with random features, and recomputes the maximum cosine similarities; if random features still yield maxima around 0.7, the claimed strong correspondence would reflect shared stimulus statistics rather than learned representations.","supporting_citations":[{"cited_title":"Sparse Autoencoders Find Highly Interpretable Features in Language Models, October 2023","cited_arxiv_id":null,"evidence_quote":"Supplies the sparse autoencoder architecture, tied decoder weights, and L1 reconstruction loss that the paper adopts for each model layer."},{"cited_title":"Brain Decodes Deep Nets, March 2024","cited_arxiv_id":null,"evidence_quote":"Gives the brain-decoding layer-alignment method and hierarchy slope metric used as comparison baselines for the proposed alignment."},{"cited_title":"Wang, Kendrick Kay, Thomas Naselaris, Michael J","cited_arxiv_id":null,"evidence_quote":"Defines the brain encoder whose weights are treated as ground-truth cortical functional structure and supplies Max R2 and FactorTopy baselines."},{"cited_title":"Prince, Matthew Kowal, Victor Boutin, Isabel Papadimitriou, Binxu Wang, Martin Wattenberg, Demba Ba, and Talia Konkle","cited_arxiv_id":null,"evidence_quote":"Contributes the bilinear-interpolation activation heatmap procedure used to display unit selectivity for ROI-associated features."}],"review_version":1}