Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

Shifting Attention to You: Personalized Brain-Inspired AI Models

T0 review · 3 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fine-tuning a CLIP model on human behavioral embeddings and millisecond-scale MEG recordings more than doubles its agreement with human similarity judgments (0.78 vs 0.32) and lets per-participant models capture individual neural dynamics.

desk verdict The behavioral fine-tuning result is credible and worth referee time; the personalization claim needs a group-level or permutation baseline before it can be accepted. read the letter →

arxiv 2502.04658 v2 pith:ABWC65QS submitted 2025-02-07 q-bio.NC cs.AI

Stephen Chong Zhao , Yang Hu , Jason Lee , Andrew Bender , Trisha Mazumdar , Mark Wallace , David A. Tovar This is my paper · ORCID
classification q-bio.NCcs.AI
keywords personalizedAICLIPfine-tuninghumanperceptualalignmentMEGneuraldynamicsrepresentationalsimilarityanalysisSPoSEembeddingsbrain-inspiredindividualdifferences
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that an off-the-shelf CLIP vision-language model can be reshaped into a human-like perceiver by fine-tuning it on two kinds of human measurements: sparse behavioral similarity dimensions derived from large-scale odd-one-out judgments, and millisecond-resolution magnetoencephalography (MEG) decoding RDMs. The behaviorally tuned model, CLIP-HBA-Behavior, is reported to reach a Spearman correlation of 0.78 with held-out human similarity judgments on 48 objects, more than double the 0.32 of the unmodified CLIP-ViT-L/14 baseline, and to improve scores on the NIGHTS human-similarity triplet benchmark. The neurally tuned variant, CLIP-HBA-MEG, learns a time-varying visual representation whose alignment with MEG data peaks around 300–400 ms after stimulus onset and generalizes across external datasets with different participants and image degradations. The same pipeline, applied participant by participant, is reported to capture individual neural signatures, with a Spearman correlation of 0.659 between personalized model RDMs and participant MEG RDMs on held-out stimuli. If these findings hold, they imply that human inductive biases can be injected into large pretrained models cheaply and adaptively, making them better tools for cognitive neuroscience, personalized medicine, and human-computer interaction.

What carries the argument

The load-bearing mechanism for the behavioral results is the binding of visual features to 66 SPoSE text dimensions via a dot-product projection, trained with mean squared error loss against behavioral SPoSE embeddings using DoRA parameter-efficient fine-tuning on the last text and vision attention layers. For the neural results, the central object is a learned feature-reweighting matrix W in $R^{{T x L}}$ (T MEG timepoints, L = 24 ViT layers), initialized to the last layer and then optimized in two stages; it recombines all ViT layer activations into a time-varying embedding. Temporal scalers alpha_T and beta_T modulate feature magnitude and semantic binding, dimension-wise Gaussian noise mimics neural variability, and a three-term loss combining Pearson correlation, MSE, and time-generalization aligns model RDMs to MEG decoding RDMs. This machinery converts a static CLIP representation into a dynamic one whose similarity geometry can be compared slice-by-slice to the brain's temporal response profile.

What would settle it

Train the group-level or individual CLIP-HBA-MEG pipeline on MEG decoding RDMs whose stimulus labels have been randomly permuted, or on RDMs generated from shuffled classifier outputs, then measure the model's temporal alignment with genuine held-out MEG data. If the permuted-target model still shows the reported peak alignment around 300–400 ms or Spearman correlations near the real model's, the objective was fitting classifier artifacts rather than neural structure; if alignment collapses to chance, the dynamic fine-tuning is genuinely reading representational content.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the representational geometry of a pretrained CLIP model is highly malleable: aligning it to human cognitive measurements, rather than only to image-text statistics, is enough to reorganize its embedding space so it tracks both average human similarity judgments and the temporal evolution of individual neural responses. Concretely, the paper reports that CLIP-HBA-Behavior achieves a Spearman correlation of 0.78 (95% CI [0.75, 0.80]) against fully sampled behavioral RDMs, versus 0.32 for baseline CLIP-ViT-L/14, and that CLIP-HBA-MEG, trained on MEG decoding RDMs, outperforms the static baseline in neural alignment across THINGS and three external datasets while peaking at 300–400 ms after stimulus onset. For personalization, 15 models fine-tuned on single participants' MEG data yield a global Spearman correlation of 0.659 (p < 1e-14) between model-embedding distances and participant-neural distances on 18 held-out stimuli. The paper interprets this as evidence that training methodology, not architecture alone, determines how human-like a network's representations become, and that individual cognitive styles can be encoded in model weights.

Load-bearing premise

The neural-alignment claims stand or fall on treating the MEG decoding RDMs, computed by linear-discriminant and support-vector classifiers and averaged over just three participants for group-level training, as faithful, genuinely time-resolved measurements of object representations in the brain; if those RDMs mostly reflect classifier bias or averaging artifacts, the dynamic fine-tuning objective would be fitting noise, and the claim that the model tracks the temporal evolution of individual neural responses would not follow.

Editorial extensions

If this is right

  • CLIP-HBA-Behavior's Spearman correlation of 0.78 on held-out similarity judgments (vs 0.32 baseline) means fine-tuning on 66 interpretable behavioral dimensions can make a general vision model approximate human pairwise similarity judgments far better than its original embedding space.
  • The NIGHTS benchmark gains (validation/test 0.88 vs 0.81 for 768-d features; 0.85/0.84 vs 0.80/0.79 for SPoSE dimensions) show that the behavioral alignment transfers to a large, independently collected triplet-similarity benchmark.
  • CLIP-HBA-MEG's neural alignment, peaking around 300–400 ms and generalizing to external participants and degraded images, implies the model has learned a time-resolved representation that tracks the late semantic phase of visual processing rather than only early image features.
  • Personalized models trained on individual MEG data reach a Spearman correlation of 0.659 between model and participant dissimilarity structure on held-out stimuli, implying that stable individual differences in neural dynamics can be encoded in model weights.
  • Because only DoRA adapter parameters and the feature-reweighting matrix are updated, the personalization pipeline is cheap enough to run participant-by-participant, supporting sequential or on-device adaptation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: because behavioral fine-tuning saturates with roughly 100 training stimuli, the same recipe could be used to align models to small, hard-to-collect behavioral datasets from special populations such as infants, patients, or non-verbal individuals, where large-scale judgments are impractical.
  • Editorial extension: the dynamic saliency maps are claimed to show where a person attends at each millisecond, but the paper does not compare them to eye-tracking or fixation data; such a comparison would be a direct, testable way to validate whether the model's dynamic attention is perceptual attention or merely a correlate of RDM geometry.
  • Editorial extension: the loss terms and reweighting mechanism are architecture-agnostic, so a natural next test is to apply the same MEG-targeted fine-tuning to a non-CLIP transformer and check whether the same 300–400 ms semantic peak appears; if it does, the effect is driven by the training objective rather than by CLIP's text-binding component.
  • Editorial extension: the paper frames personalized models as cognitive digital twins, but a stronger test of that framing would be to see whether a personalized model trained on one recording session can predict that same participant's neural RDMs from a later session, or distinguish them from other participants' data; the current 18-stimulus held-out validation is a single-session check.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The manuscript introduces CLIP-HBA, a family of CLIP-ViT-L/14 models fine-tuned with human-derived targets. CLIP-HBA-Behavior is trained with an MSE loss to predict 66 SPoSE behavioral embeddings from THINGS images, and is reported to reach Spearman rho=0.78 (95% CI [0.75,0.80]) against a fully sampled behavioral RDM for 48 held-out objects, versus 0.32 for the CLIP baseline, with improved NIGHTS triplet scores (Table 1). CLIP-HBA-MEG extends the method with a dynamic feature-reweighting matrix and a three-part loss (Eq. 10) aligned to MEG decoding RDMs, and is evaluated on THINGS MEG data and three external datasets (Tables 2-3). Finally, 15 participant-specific models are trained on individual MEG RDMs; a global Spearman correlation of rho=0.659 (p<1e-14) between model-pair distances and participant-pair distances on 18 held-out stimuli is presented as evidence of personalized neural alignment. The paper also reports dynamic saliency maps and discusses applications to personalized medicine and human-AI interaction.

Significance. If the main claims hold, the paper constitutes a useful contribution to human-aligned representation learning: the design excludes held-out stimuli from training, the NIGHTS and external MEG validations are appropriate generalization checks, and the SPoSE-based embedding space offers a degree of interpretability. However, the central novelty, personalized neural fine-tuning, is supported by a single correlational statistic without group-level or permutation controls, and the neural targets themselves are decoding RDMs whose fidelity to true neural geometry is not established. The behavioral gains are large relative to the CLIP baseline but lack comparison to competing human-aligned models and to a noise ceiling. The paper also contains no code or data availability statement. The strengths and weaknesses are unbalanced enough that the conclusions should be revised rather than accepted as stated.

major comments (3)
  1. [§2.7, §4.4–4.5] The individual-level claim that personalized models capture participant-specific neural dynamics rests on a single global Spearman correlation (rho=0.659, p<1e-14) between pairwise distances among 15 models and pairwise distances among participants' MEG RDMs, computed on 18 held-out stimuli. The analysis lacks a group-level baseline (e.g., one model trained on averaged MEG RDMs), a permutation test with shuffled participant–model assignments, and a cross-participant evaluation (train on A, test on B). Since each model is trained on its own participant's MEG RDMs and evaluated on the same participant's held-out neural RDMs, the correlation could be driven by shared object representational structure or by participant-specific decoding artifacts rather than by genuine individual dynamics. Moreover, the description of this analysis in §4.5 is internally inconsistent: it first describes a per-pair Spearman correlation over time, then a between-pair distance correlation, and the reported value changes from 0.659 to 0.65 without explanation. Please add a group-average model comparison and a shuffled-assignment permutation test, and clarify the exact evaluation protocol.
  2. [§4.3.4, Tables 2–3] The neural targets are 'MEG decoding RDMs' produced by LDA/SVM classifiers, but the manuscript does not specify how these RDMs are constructed, how many trials support each RDM, or whether the decoding RDMs are cross-validated. Because the model is optimized to match these RDMs (Eqs. 5–10), the reported neural-alignments could reflect the model reproducing classifier decision boundaries rather than the geometry of neural population responses. At least one central claim, such as the group-level temporal alignment in Figure 2B or the individual alignment in §2.7, should be validated against raw sensor- or source-space RDMs, or the authors should provide evidence that the decoding RDMs are stable across trials, are not dominated by classifier bias, and capture stimulus-specific representational structure.
  3. [§2.1–2.2] The headline behavioral improvement (rho=0.78 versus 0.32) is measured on 48 held-out objects from the THINGS benchmark, and the model is fine-tuned on SPoSE embeddings that are themselves fitted to human similarity judgments for the same object set. The external NIGHTS benchmark in Table 1 is a welcome generalization test, but the reported gains are modest (0.88 vs 0.81 for 768-d features; 0.85/0.84 vs 0.80/0.79 for 66-d features) and are reported without error bars or significance tests. The phrase 'over doubles behavioral performance' relies on comparison to a single baseline CLIP-ViT-L/14 and does not establish where the method stands relative to other human-aligned models (e.g., DreamSim, DINOv2) or to a noise ceiling. Please add such comparisons or temper the generalization claims.
minor comments (6)
  1. [Table 4 and §4.5] The number of held-out stimuli is inconsistent: §2.7 and §4.5 state 18 left-out stimuli, while Table 4 lists a Train/Test Split of 80/20 on 100 stimuli, implying 20 validation stimuli; please reconcile these numbers.
  2. [General] The manuscript provides no code or data availability statement, which substantially limits reproducibility of the fine-tuning pipeline and the personalized-model analysis.
  3. [§2.1] The p-value for the behavioral correlation (p<10^-229) is reported without specifying the statistical test or the number of elements in the vectorized RDMs used for the Spearman correlation; please state the test and sample size.
  4. [Eq. (4)] Equation (4) uses the element-wise product symbol \odot without defining it; please add a definition or explain the notation in the text.
  5. [§2.7, Figure 6A] The 'lower-bound noise ceiling' is mentioned in the text and shown in Figure 6A but is never defined or estimated in the Methods; please specify how this noise ceiling was computed.
  6. [References] References [22] and [49] are the same work (Kucyi et al., Network Neuroscience, 2024) and should be merged or cross-referenced.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the main behavioral and neural claims are tested on held-out or external benchmarks; the personalization analysis lacks a control baseline but is not circular by construction.

full rationale

The paper's derivation chain is largely self-contained and externally grounded. CLIP-HBA-Behavior is trained to predict SPoSE embeddings via Eq. (2), and its behavioral evaluation uses 48 objects explicitly excluded from training plus the independent NIGHTS triplet benchmark. Although both the training target and the behavioral RDM derive from human similarity judgments within the THINGS paradigm, the held-out object generalization and the external NIGHTS results mean the behavioral improvement is not forced by construction. The same holds for CLIP-HBA-MEG: it is trained against MEG decoding RDMs through Eq. (10), but its neural generalization claims are supported by three external MEG datasets with different participants, stimuli, and image degradations, so the alignment is not merely a refit of training data. The personalized-model analysis is the only place where the evidence is weaker: each model is fine-tuned on a participant's MEG RDMs and then correlated with the same participant's held-out neural RDMs. This is a legitimate held-out-stimulus evaluation, not a circular reduction, but the reported rho = 0.659 lacks a group-level model, cross-participant evaluation, or permutation test, so it cannot by itself establish that participant-specific dynamics rather than shared representational structure drive the correlation. That is a missing experimental control, which belongs in the correctness/validity assessment rather than the circularity score. Self-citations in the paper (e.g., references [43], [57], [58]) support specific design choices such as temporal scalers and multisensory integration; they are not load-bearing for the central claims. Overall, the central predictions are not equivalent to their inputs by definition, and the paper is appropriately benchmarked against external data, so no significant circularity is present.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claims rest on a few fitted components: the loss weights, the feature reweighting matrix W, and the under-specified temporal scalars alpha_T and beta_T. The most consequential domain assumptions are that SPoSE dimensions and MEG decoding RDMs faithfully represent human perceptual and neural structure. No new physical or conceptual entities are introduced.

free parameters (4)
  • Loss weights w1, w2, w3 = w1=1, w2=0.1 or 0.15, w3=0.15 or 0.1 depending on dataset
    The normalized loss weights are adjusted per dataset (Table 4); they control the balance between Pearson, MSE, and time generalization objectives and affect the dynamic alignment result.
  • Temporal scalars alpha_T and beta_T = not specified
    alpha_T (visual feature magnitude) and beta_T (semantic binding strength) are said to be controlled by neural response richness and time generalization patterns, but no formulas or values are given. They modulate the dynamic embeddings central to CLIP-HBA-MEG.
  • Feature reweighting matrix W = learned T x 24 matrix, initialized as a one-hot at the last layer
    W is fit to neural RDMs in Stage 1 and Stage 2; it determines which CLIP layers drive the temporal alignment and is central to the dynamic claim.
  • DoRA rank, dropout, and learning rates = r=32 group-level, r=6 individual-level; dropout 0.1; learning rates 3e-4, 3e-5, 3e-3
    Hyperparameters were selected for each fine-tuning stage; no sensitivity analysis is provided, so the central results may depend on these choices.
assumptions (5)
  • domain assumption SPoSE 66-dimensional embeddings capture the psychologically relevant structure of human similarity judgments
    The behavioral targets and evaluation are built on SPoSE; if these dimensions omit or distort perceptual semantics, the fine-tuning objective is misaligned with the claimed human alignment.
  • domain assumption MEG decoding RDMs are valid proxies for dynamic neural representations of object vision
    The neural targets are classifier-based dissimilarity matrices (LDA/SVM decoding) rather than raw signals; the claim of tracking neural dynamics inherits the validity of this decoding measure.
  • domain assumption Off-the-shelf CLIP-ViT-L/14 last-layer features are an appropriate baseline for human alignment
    The doubling claim compares against a single baseline representation; different CLIP layers or other baselines could change the magnitude of improvement.
  • domain assumption THINGS MEG group data from 3 participants is sufficient to train a generalizable neural alignment model
    Group-level CLIP-HBA-MEG is trained on averaged RDMs from only 3 participants; generalizability across participants is then tested on external datasets.
  • standard math Standard CLIP pretrained weights and transformer training dynamics are accepted as background
    The method relies on pretrained CLIP weights and standard backpropagation, AdamW, and DoRA as unproved background.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Shifting Attention to You: Personalized Brain-Inspired AI Models." pith.science (2026). https://pith.science/paper/ABWC65QS

@misc{pith2026250204658,
  author       = {Pith},
  title        = {Pith review of: Shifting Attention to You: Personalized Brain-Inspired AI Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ABWC65QS}},
  note         = {Machine review of arXiv:2502.04658}
}
read the original abstract

The integration of human and artificial intelligence offers a powerful avenue for advancing our understanding of information processing, as each system provides unique computational insights. However, despite the promise of human-AI integration, current AI models are largely trained on massive datasets, optimized for population-level performance, lacking mechanisms to align their computations with individual users' perceptual semantics and neural dynamics. Here we show that integrating human behavioral insights and millisecond scale neural data within a fine tuned CLIP based model not only captures generalized and individualized aspects of perception but also over doubles behavioral performance compared to the unmodified CLIP baseline. By embedding human inductive biases and mirroring dynamic neural processes during training, personalized neural fine tuning improves predictions of human similarity judgments and tracks the temporal evolution of individual neural responses. Our work establishes a novel, interpretable framework for designing adaptive AI systems, with broad implications for neuroscience, personalized medicine, and human-computer interaction.

Figures

Figures reproduced from arXiv: 2502.04658 by the authors.

Figure 1
Figure 1. Schematic of the CLIP-HBA-Behavior fine-tuning process using behavioral data. The 66 SPoSE text dimensions are fed into the text encoder, producing 66 text representations D1 . . . D66. Concurrently, visual stimuli from the THINGS dataset are input into the vision encoder, generating their corresponding visual representations V . These features from two modalities are bound via a dot product projection, mapping the … view at source ↗
Figure 2
Figure 2. Behavioral and neural alignment of fine-tuned CLIP-HBA-Behavior. (A) Behavioral results: Rep￾resentational dissimilarity matrices (RDMs) for 48 objects, predicted by CLIP-HBA-Behavior (top) and CLIP-ViT (bottom), with Spearman rank correlations (ρ) of 0.78 and 0.32, respectively. (B) Neural results: Temporal correlations between model-predicted RDMs and MEG RDMs from THINGS (top left) and external datasets under var… view at source ↗
Figure 3
Figure 3. Schematic of the CLIP-HBA-MEG fine-tuning process using neural signals. A Feature Reweighting Matrix, pre-optimized at initialization, dynamically computes weighted combinations of vision encoder layer activations to align with neural decoding RDMs. Temporal scalers, αT and βT , respectively, modulate the magnitude of visual feature aggregation and the binding of visual-semantic features. Dimension-wise Gaussian noi… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Behavioral and neural alignment of fine-tuned CLIP-HBA-MEG. (A) Behavioral validation: Comparison of the dynamic embedding space of the CLIP-HBA-MEG model across all timepoints (purple bars) with the THINGS behavioral data of 48 sample objects. The static behavioral al…
Figure 5
Figure 5. Figure 5: Example of dynamic saliency map of neurally fine-tuned CLIP-HBA-MEG As shown in [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Individualized model schematic and results. (A) Fifteen individualized models were trained on MEG RDMs from 15 participants viewing 100 visual object stimuli. Each model achieved enhanced neural alignment specific to its corresponding participant, demonstrating sustain…
Figure 7
Figure 7. Figure 7: Individual Visual and Embedding Differences. A. Example of individual differences in dynamic attention: Personalized separate CLIP-HBA-MEG models fine-tuned on 2 individual subjects. B. Example of Temporal Embedding Individual Differences: Pearson correlation distance …
Figure 8
Figure 8. Figure 8: Example SPoSE Embeddings of Image Stimuli: Behaviorally Fine-tuned CLIP-HBA-Behavior v.s. Baseline CLIP-ViT 22 [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]
Figure 9
Figure 9. Figure 9: THINGS Object Shifting within the 66-d SPoSE Mental Embedding Space after Behavioral Fine-Tuning; visualized using t-SNE [PITH_FULL_IMAGE:figures/full_fig_p023_9.png]
Figure 10
Figure 10. Figure 10: Evaluation of Training Effectiveness based on the Size of the Behavioral Dataset. CLIP-HBA-Behavior models are trained using random subsamples of the 1854-object THINGS behavioral dataset, and evaluated against the 48 excluded validation samples’ triple-out behavior g…
Figure 11
Figure 11. Figure 11: Schema of converting dynamic RDMs into a 1-D vector for distance and correlation comparisons.. Each 3D RDM object represents dynamic representations, with each slice corresponding to a specific millisecond time point. For each time slice, we extract and flatten the up…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Beginning with You: Perceptual-Initialization Improves Vision-Language Representation and Alignment

    cs.CV 2025-05 conditional novelty 6.0 of 10

    Perceptually initializing a CLIP vision encoder with NIGHTS triplet judgments before YFCC15M contrastive training improves zero-shot accuracy and retrieval over an identical random-start baseline.

Reference graph

Works this paper leans on

64 extracted references · 32 canonical work pages · cited by 1 Pith paper

  1. [1]

    Deep learning

    Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT Press, 2016. URL http://www. deeplearningbook.org/. 17 Personalized Brain-Inspired AI Models

    work page 2016

  2. [2]

    If deep learning is the answer, what is the question? Nature Reviews Neuroscience, 22:55–67, 2021

    Andrew Saxe, Sandro Nelli, and Christopher Summerfield. If deep learning is the answer, what is the question? Nature Reviews Neuroscience, 22:55–67, 2021. doi:10.1038/s41583-020-00395-8

    doi:10.1038/s41583-020-00395-8 2021

  3. [3]

    Towards neural foundation models for vision: Aligning eeg, meg, and fmri representations for decoding, encoding, and modality conversion, 2024

    Matteo Ferrante, Tommaso Boccato, Grigorii Rashkov, and Nicola Toschi. Towards neural foundation models for vision: Aligning eeg, meg, and fmri representations for decoding, encoding, and modality conversion, 2024. URL https://arxiv.org/abs/2411.09723

    arXiv 2024

  4. [4]

    Achieving more human brain-like vision via human eeg representa- tional alignment

    Zitong Lu, Yile Wang, and Julie D Golomb. Achieving more human brain-like vision via human eeg representa- tional alignment. arXiv preprint arXiv:2401.17231, 2024

    arXiv 2024

  5. [5]

    Bruna, Ilia Sucholutsky, Christopher Kello, and Thomas L

    Sunayana Rane, Polyphony J. Bruna, Ilia Sucholutsky, Christopher Kello, and Thomas L. Griffiths. Concept alignment, 2024. URL https://arxiv.org/abs/2401.08672

    arXiv 2024

  6. [6]

    Mahner, Lukas Muttenthaler, Umut Güçlü, and Martin N

    Florian P. Mahner, Lukas Muttenthaler, Umut Güçlü, and Martin N. Hebart. Dimensions underlying the representa- tional alignment of deep neural networks with humans, June 2024. URL http://arxiv.org/abs/2406.19087. arXiv:2406.19087 [cs, q-bio]

    arXiv 2024

  7. [7]

    Goldstone

    Robert L. Goldstone. The role of similarity in categorization: providing a groundwork. Cognition, 52(2):125–157, August 1994. ISSN 00100277. doi:10.1016/0010-0277(94)90065-5. URL https://linkinghub.elsevier. com/retrieve/pii/0010027794900655

    arXiv 1994

  8. [8]

    Distributional measures of semantic abstraction

    Sabine Schulte Im Walde and Diego Frassinelli. Distributional measures of semantic abstraction. Frontiers in Artificial Intelligence, 4:796756, 2022. doi:10.3389/frai.2021.796756. URL https://doi.org/10.3389/frai. 2021.796756

    work page arXiv 2022

Show all 64 references
  1. [9]

    Lampinen, Klaus-Robert Müller, and Michael C

    Frieda Born, Lukas Muttenthaler, Klaus Greff, Thomas Unterthiner, Andrew K. Lampinen, Klaus-Robert Müller, and Michael C. Mozer. Evaluating and supervising vision models with multi-level similarity judgments. Cognitive Computational Neuroscience Conference (CCN), 2024. URL htt...

    2024

  2. [10]

    McDermott

    Jenelle Feather, Guillaume Leclerc, Aleksander M ˛ adry, and Josh H. McDermott. Model metamers reveal divergent invariances between biological and artificial neural networks. Nature Neuroscience, 26(11):2017–2034, November

    2017

  3. [11]

    Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence

    Radoslaw Martin Cichy, Aditya Khosla, Dimitrios Pantazis, Antonio Torralba, and Aude Oliva. Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence. Scientific Reports, 6(1):27755, June 2016...

    2016 doi

  4. [12]

    Vandermeulen, and Simon Kornblith

    Lukas Muttenthaler, Jonas Dippel, Lorenz Linhardt, Robert A. Vandermeulen, and Simon Kornblith. Hu- man alignment of neural network representations, April 2023. URL http://arxiv.org/abs/2211.01201. arXiv:2211.01201 [cs, q-bio]

    2023 arXiv

  5. [13]

    Dickerson, Krishna P

    Vedant Nanda, Ayan Majumdar, Camila Kolling, John P. Dickerson, Krishna P. Gummadi, Bradley C. Love, and Adrian Weller. Do Invariances in Deep Neural Networks Align with Human Perception? Proceedings of the AAAI Conference on Artificial Intelligence, 37(8):9277–9285, June 2023...

    2023 doi

  6. [14]

    Sinz, Xaq Pitkow, Jacob Reimer, Matthias Bethge, and Andreas S

    Fabian H. Sinz, Xaq Pitkow, Jacob Reimer, Matthias Bethge, and Andreas S. Tolias. Engineering a less artificial intelligence. Neuron, 103(6):967–979, 2019. doi:10.1016/j.neuron.2019.08.034. URL https://www.cell. com/neuron/fulltext/S0896-6273(19)30740-8

    2019 doi

  7. [15]

    Peterson, and Thomas L

    Raja Marjieh, Nori Jacoby, Joshua C. Peterson, and Thomas L. Griffiths. The Universal Law of Generalization Holds for Naturalistic Stimuli, June 2023. URL http://arxiv.org/abs/2306.08564. arXiv:2306.08564 [cs, q-bio, stat]

    2023 arXiv

  8. [16]

    Griffiths

    Raja Marjieh, Sreejan Kumar, Declan Campbell, Liyi Zhang, Gianluca Bencomo, Jake Snell, and Thomas L. Griffiths. Using Contrastive Learning with Generative Similarity to Learn Spaces that Capture Human Inductive Biases, May 2024. URL http://arxiv.org/abs/2405.19420. arXiv:2405...

    2024 arXiv

  9. [17]

    Learning Transferable Visual Models From Natural Language Supervision, February 2021

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning Transferable Visual Models From Natural Language Supervision, February 2021. URL h...

    2021 arXiv

  10. [18]

    Hebart, Adam H

    Martin N. Hebart, Adam H. Dickter, Alexis Kidder, Wan Y . Kwok, Anna Corriveau, Caitlin Van Wicklin, and Chris I. Baker. THINGS: A database of 1,854 object concepts and more than 26,000 naturalistic object images. PLOS ONE, 14(10):e0223792, October 2019. ISSN 1932-6203. doi:10...

    2019 doi

  11. [19]

    THINGS-data, a multimodal collection of large-scale datasets for investigating object representations in human brain and behavior

    Martin N Hebart, Oliver Contier, Lina Teichmann, Adam H Rockter, Charles Y Zheng, Alexis Kidder, Anna Corriveau, Maryam Vaziri-Pashkam, and Chris I Baker. THINGS-data, a multimodal collection of large-scale datasets for investigating object representations in human brain and b...

    2023 doi

  12. [20]

    Dreamsim: Learning new dimensions of human visual similarity using synthetic data, 2023

    Stephanie Fu, Netanel Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. Dreamsim: Learning new dimensions of human visual similarity using synthetic data, 2023. URL https: //arxiv.org/abs/2306.09344

    2023 arXiv

  13. [21]

    Ruffle, Robert J Gray, Samia Mohinta, Guilherme Pombo, Chaitanya Kaul, Harpreet Hyare, Geraint Rees, and Parashkev Nachev

    James K. Ruffle, Robert J Gray, Samia Mohinta, Guilherme Pombo, Chaitanya Kaul, Harpreet Hyare, Geraint Rees, and Parashkev Nachev. Computational limits to the legibility of the imaged human brain. NeuroImage, 291: 120600, May 2024. ISSN 1053-8119. doi:10.1016/j.neuroimage.202...

    2024

  14. [22]

    Braga, Po-Jang Hsieh, and Shao-Min Hung

    Aaron Kucyi, Nathan Anderson, Tiara Bounyarith, David Braun, Lotus Shareef-Trudeau, Isaac Treves, Rodrigo M. Braga, Po-Jang Hsieh, and Shao-Min Hung. Individual variability in neural representations of mind-wandering. Network Neuroscience, 8(3):808–836, 10 2024. ISSN 2472-1751...

    2024 doi

  15. [23]

    Inductive biases for deep learning of higher-level cognition

    Anirudh Goyal and Yoshua Bengio. Inductive biases for deep learning of higher-level cognition. Proceedings of the Royal Society A, 478(20210068), 2022. doi:10.1098/rspa.2021.0068. URL http://doi.org/10.1098/ rspa.2021.0068

    2022

  16. [24]

    Zheng, Francisco Pereira, Chris I

    Charles Y . Zheng, Francisco Pereira, Chris I. Baker, and Martin N. Hebart. Revealing interpretable object representations from human behavior, 2019. URL https://arxiv.org/abs/1901.02915

    2019 arXiv

  17. [25]

    Hebart, Charles Y

    Martin N. Hebart, Charles Y . Zheng, Francisco Pereira, and Chris I. Baker. Revealing the multidimensional mental representations of natural objects underlying human similarity judgements. Nature Human Behaviour, 4(11): 1173–1185, October 2020. ISSN 2397-3374. doi:10.1038/s415...

    2020 doi

  18. [26]

    Representational similarity analysis – connecting the branches of systems neuroscience

    Nikolaus Kriegeskorte. Representational similarity analysis – connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience, 2008. ISSN 16625137. doi:10.3389/neuro.06.004.2008. URL http:// journal.frontiersin.org/article/10.3389/neuro.06.004.2008/abstract

    2008 doi

  19. [27]

    Hyperalignment of dynamic responses using meg

    Tijl Grootswagers, Emma Contini, and Thomas Carlson. Hyperalignment of dynamic responses using meg. In Proceedings of the Organization for Human Brain Mapping Annual Meeting, page 3548, Vancouver, Canada,

  20. [28]

    Wardle, and Thomas A

    Tijl Grootswagers, Susan G. Wardle, and Thomas A. Carlson. Decoding dynamic brain patterns from evoked responses: A tutorial on multivariate pattern analysis applied to time series neuroimaging data. Journal of Cognitive Neuroscience, 29(4):677–697, 04 2017. ISSN 0898-929X. do...

    2017 doi

  21. [29]

    Representation learning for neural population activity with neural data transform- ers

    Joel Ye and Chethan Pandarinath. Representation learning for neural population activity with neural data transform- ers. Neurons, Behavior, Data analysis, and Theory, 5(3), August 2021. ISSN 2690-2664. doi:10.51628/001c.27358. URL http://dx.doi.org/10.51628/001c.27358

    2021 doi

  22. [30]

    Tang, Mikio C

    Marino Pagan, Vincent D. Tang, Mikio C. Aoi, Jonathan W. Pillow, Valerio Mante, David Sussillo, and Carlos D. Brody. Individual variability of neural computations underlying flexible decisions. Nature, 2024. doi:10.1038/s41586-024-08433-6. URL https://doi.org/10.1038/s41586-02...

    2024 doi

  23. [31]

    RISE: randomized input sampling for explanation of black-box models

    Vitali Petsiuk, Abir Das, and Kate Saenko. RISE: randomized input sampling for explanation of black-box models. CoRR, abs/1806.07421, 2018. URL http://arxiv.org/abs/1806.07421

    2018 arXiv

  24. [32]

    Mahner, Jonas Perkuhn, and Martin N

    Philipp Kaniuth, Florian P. Mahner, Jonas Perkuhn, and Martin N. Hebart. A high-throughput approach for the efficient prediction of perceived similarity of natural objects, July 2024. URL http://biorxiv.org/lookup/ doi/10.1101/2024.06.28.601184

    2024 doi

  25. [33]

    A high-throughput approach for the efficient prediction of perceived similarity of natural objects

    Philipp Kaniuth, Florian P Mahner, Jonas Perkuhn, and Martin N Hebart. A high-throughput approach for the efficient prediction of perceived similarity of natural objects. bioRxiv, pages 2024–06, 2024

    2024

  26. [34]

    Improving neural network representations using human similarity judgments

    Lukas Muttenthaler, Lorenz Linhardt, Jonas Dippel, Robert A Vandermeulen, Katherine Hermann, Andrew K Lampinen, and Simon Kornblith. Improving neural network representations using human similarity judgments. arXiv preprint arXiv:2306.04507, 2023

    2023 arXiv

  27. [35]

    Variability in neural activity and behavior

    Alfonso Renart and Christian K Machens. Variability in neural activity and behavior. Current Opinion in Neurobiology, 25:211–220, 2014. ISSN 0959-4388. doi:https://doi.org/10.1016/j.conb.2014.02.013. URL https: //www.sciencedirect.com/science/article/pii/S0959438814000488. The...

    2014 doi

  28. [36]

    Garcia, Nina Lauharatanahirun, Sarah F

    Kanika Bansal, Javier O. Garcia, Nina Lauharatanahirun, Sarah F. Muldoon, Paul Sajda, and Jean M. Vettel. Scale-specific dynamics of high-amplitude bursts in eeg capture behaviorally meaningful variability. NeuroImage, 241:118425, November 2021. ISSN 1053-8119. doi:10.1016/j.n...

    2021

  29. [37]

    Towards a rigorous science of interpretable machine learning

    Finale Doshi-Velez and Been Kim. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608, 2017

    2017 arXiv

  30. [38]

    Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead

    Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. nat mach intell 1 (5): 206–215, 2019

    2019

  31. [39]

    Tolias, and Doris Tsao

    Anthony Zador, Sean Escola, Blake Richards, Bence Ölveczky, Yoshua Bengio, Kwabena Boahen, Matthew Botvinick, Dmitri Chklovskii, Anne Churchland, Claudia Clopath, James DiCarlo, Surya Ganguli, Jeff Hawkins, Konrad Kording, Alexei Koulakov, Yann LeCun, Timothy Lillicrap, Adam M...

    2023

  32. [40]

    Philipp Kaniuth and Martin N. Hebart. Feature-reweighted representational similarity analysis: A method for improving the fit between computational models, brains, and behavior. NeuroImage, 257:119294, August

  33. [41]

    Multimodal machine learning: A survey and taxonomy

    Tadas Baltrušaitis, Chaitanya Ahuja, and Louis-Philippe Morency. Multimodal machine learning: A survey and taxonomy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(2):423–443, 2018

    2018

  34. [42]

    The neural bases of multisensory processes

    Micah M Murray and Mark T Wallace. The neural bases of multisensory processes. CRC Press, 2011

    2011

  35. [43]

    Selective enhancement of object representations through multisensory integration

    David A Tovar, Micah M Murray, and Mark T Wallace. Selective enhancement of object representations through multisensory integration. Journal of Neuroscience, 40(29):5604–5615, 2020

    2020

  36. [44]

    The neuroconnectionist research programme

    Adrien Doerig, Rowan P Sommers, Katja Seeliger, Blake Richards, Jenann Ismael, Grace W Lindsay, Konrad P Kording, Talia Konkle, Marcel AJ Van Gerven, Nikolaus Kriegeskorte, et al. The neuroconnectionist research programme. Nature Reviews Neuroscience, 24(7):431–450, 2023

    2023

  37. [45]

    Industrial applications of digital twins.Philosophi- cal Transactions of the Royal Society A, 379(2207):20200360, 2021

    Yuchen Jiang, Shen Yin, Kuan Li, Hao Luo, and Okyay Kaynak. Industrial applications of digital twins.Philosophi- cal Transactions of the Royal Society A, 379(2207):20200360, 2021. URL https://royalsocietypublishing. org/doi/10.1098/rsta.2020.0360

    2021

  38. [46]

    Enhancing personalized learning: Ai-driven identification of learning styles and content modification strategies

    Md Kabin Hasan Kanchon, Mahir Sadman, Kaniz Fatema Nabila, Ramisa Tarannum, and Riasat Khan. Enhancing personalized learning: Ai-driven identification of learning styles and content modification strategies. International Journal of Cognitive Computing in Engineering, 5:269–278, 2024

    2024

  39. [47]

    Dynamic functional connectivity analysis reveals transient states of dysconnectivity in schizophrenia

    Eswar Damaraju, Elena A Allen, Aysenil Belger, Judith M Ford, S McEwen, DH Mathalon, BA Mueller, GD Pearlson, SG Potkin, A Preda, et al. Dynamic functional connectivity analysis reveals transient states of dysconnectivity in schizophrenia. NeuroImage: Clinical, 5:298–308, 2014

    2014

  40. [48]

    The dynamic functional connectome: State-of- the-art and perspectives

    Maria Giulia Preti, Thomas AW Bolton, and Dimitri Van De Ville. The dynamic functional connectome: State-of- the-art and perspectives. Neuroimage, 160:41–54, 2017

    2017

  41. [49]

    Individual variability in neural representations of mind-wandering

    Aaron Kucyi, Nathan Anderson, Tiara Bounyarith, David Braun, Lotus Shareef-Trudeau, Isaac Treves, Rodrigo M Braga, Po-Jang Hsieh, and Shao-Min Hung. Individual variability in neural representations of mind-wandering. Network Neuroscience, pages 1–66, 2024

    2024

  42. [50]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at...

    2010 arXiv

  43. [51]

    Dora: Weight-decomposed low-rank adaptation, 2024

    Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. Dora: Weight-decomposed low-rank adaptation, 2024. URL https://arxiv.org/abs/ 2402.09353

    2024 arXiv

  44. [52]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-Rank Adaptation of Large Language Models, October 2021. URL http://arxiv.org/abs/ 2106.09685. arXiv:2106.09685 [cs]

    2021 arXiv

  45. [53]

    Fine-tuning clip’s last visual projector: A few-shot cornucopia, 2024

    Mohammad Fahes, Tuan-Hung Vu, Andrei Bursuc, Patrick Pérez, and Raoul de Charette. Fine-tuning clip’s last visual projector: A few-shot cornucopia, 2024. URL https://arxiv.org/abs/2410.05270. 20 Personalized Brain-Inspired AI Models

    2024

  46. [54]

    Jerrold H. Zar. Significance testing of the spearman rank correlation coefficient.Journal of the American Statistical Association, 67(339):578–580, 1972. doi:10.2307/2284441

    1972 doi

  47. [55]

    Decoupled weight decay regularization, 2019

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019. URL https://arxiv.org/ abs/1711.05101

    2019 arXiv

  48. [56]

    High temporal resolution decoding of object position and category

    Thomas A Carlson, Hinze Hogendoorn, Ryota Kanai, Juraj Mesik, and Jeremy Turret. High temporal resolution decoding of object position and category. Journal of Vision , 11(10):9, 2011. doi:10.1167/11.10.9. URL https://pubmed.ncbi.nlm.nih.gov/21920851/

    2011

  49. [57]

    Representational dynamics of object vision: the first 1000 ms

    Thomas Carlson, David A Tovar, Arjen Alink, and Nikolaus Kriegeskorte. Representational dynamics of object vision: the first 1000 ms. Journal of Vision, 13(10):1, 2013. doi:10.1167/13.10.1. URL https://pubmed.ncbi. nlm.nih.gov/23908380/

    2013

  50. [58]

    Stimulus feature-specific information flow along the columnar cortical microcir- cuit revealed by multivariate laminar spiking analysis

    David A Tovar, Jacob A Westerberg, Michele A Cox, Kacie Dougherty, Thomas A Carlson, Mark T Wal- lace, and Alexander Maier. Stimulus feature-specific information flow along the columnar cortical microcir- cuit revealed by multivariate laminar spiking analysis. Frontiers in Sys...

    2020

  51. [59]

    King and S

    J-R. King and S. Dehaene. Characterizing the dynamics of mental representations: the tempo- ral generalization method. Trends in Cognitive Sciences , 18(4):203–210, 2014. ISSN 1364-6613. doi:https://doi.org/10.1016/j.tics.2014.01.002. URL https://www.sciencedirect.com/science/...

    2014 doi

  52. [60]

    Pearson’s Correlation Coefficient , pages 1090–1091

    Wilhelm Kirch. Pearson’s Correlation Coefficient , pages 1090–1091. Springer Netherlands, Dordrecht,

  53. [2008]

    doi:10.1007/978-1-4020-5614-7_2569

    ISBN 978-1-4020-5614-7. doi:10.1007/978-1-4020-5614-7_2569. URL https://doi.org/10.1007/ 978-1-4020-5614-7_2569 . 21 Personalized Brain-Inspired AI Models 5 Supplemental Figures Figure 8: Example SPoSE Embeddings of Image Stimuli:Behaviorally Fine-tuned CLIP-HBA-Behavior v.s. ...

    doi

  54. [2017]

    Poster Session presented on June 28

    Organization for Human Brain Mapping. Poster Session presented on June 28

  55. [2022]

    doi:10.1016/j.neuroimage.2022.119294

    ISSN 10538119. doi:10.1016/j.neuroimage.2022.119294. URL https://linkinghub.elsevier.com/ retrieve/pii/S105381192200413X

    2022

  56. [2023]

    doi:10.1038/s41593-023-01442-0

    ISSN 1097-6256, 1546-1726. doi:10.1038/s41593-023-01442-0. URL https://www.nature.com/ articles/s41593-023-01442-0

    doi

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.