Explaining the Human Visual Brain Challenge 2019 -- receptive fields and surrogate features

Romuald A. Janik

arxiv: 1907.00950 · v1 · pith:XVZW3HVHnew · submitted 2019-07-01 · 🧬 q-bio.NC

Explaining the Human Visual Brain Challenge 2019 -- receptive fields and surrogate features

Romuald A. Janik This is my paper

Pith reviewed 2026-05-25 11:09 UTC · model grok-4.3

classification 🧬 q-bio.NC

keywords representational dissimilarity matrixfMRIMEGneural networksreceptive fieldsmultidimensional scalingvisual cortexbrain decoding

0 comments

The pith

Intermediate surrogate features from multidimensional scaling, modeled by neural networks with tuned receptive field sizes, generate the representational dissimilarity matrices most similar to those from human fMRI and MEG recordings of the

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper reviews entries in the 2019 challenge to explain visual brain responses by building neural network features whose representational dissimilarity matrices best match those measured in fMRI and MEG. It examines how the granularity of receptive fields affects the match and shows that constructing intermediate surrogate features via multidimensional scaling allows neural networks to approximate brain-like dissimilarity structures more closely. The review also flags specific choices in how RDMs are computed that can distort comparisons. A sympathetic reader would see this as a practical route to reverse-engineer visual cortex representations from image sets. The work treats RDM similarity as the direct test of whether the artificial features capture what the brain does.

Core claim

By varying receptive field granularity and inserting multidimensional scaling to create intermediate surrogate features that neural networks then model, the submitted methods produce RDMs closer to those extracted from brain data than direct use of network layers alone.

What carries the argument

Representational dissimilarity matrices (RDMs) derived from neural network features after multidimensional scaling produces surrogate targets at chosen receptive field scales.

If this is right

Tuned receptive field sizes improve RDM alignment with brain data.
Multidimensional scaling supplies useful intermediate targets for neural network training.
RDM construction details such as distance metrics and normalization must be controlled to avoid spurious matches.
Neural network layers can be selected or combined to approximate brain dissimilarity structures at multiple scales.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If RDM matching is retained as the criterion, the same pipeline could be tested for predicting responses to entirely new image categories or under attention manipulations.
The approach leaves open whether the surrogate features correspond to any identifiable biological receptive field properties.
Success on this metric does not yet address whether the models generalize to dynamic or natural viewing conditions beyond the challenge image set.

Load-bearing premise

That closeness between a neural network RDM and a brain-derived RDM means the network explains how the human visual brain processes images.

What would settle it

A set of new images where the best RDM-matching network features still fail to predict measured brain responses above chance or baseline models.

read the original abstract

In this paper I review the submission to the Explaining the Human Visual Brain Challenge 2019 in both the fMRI and MEG tracks. The goal was to construct neural network features which generate the so-called representational dissimilarity matrix (RDM) which is most similar to the one extracted from fMRI and MEG data upon viewing a set of images. I review exploring the optimal granularity of the receptive field, a construction of intermediate surrogate features using Multidimensional Scaling and modelling them using neural network features. I also point out some peculiarities of the RDM construction which have to be taken into account.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

This is a descriptive review of 2019 challenge submissions on NN-to-brain RDM matching, with no new results or methods of its own.

read the letter

This paper reviews submissions to the 2019 Explaining the Human Visual Brain Challenge in the fMRI and MEG tracks. It covers how entrants explored receptive field granularity in neural nets and built intermediate surrogate features via multidimensional scaling to produce RDMs closer to those from brain data. It also flags some construction details in the RDMs themselves that affect comparisons. That last point is the most concrete takeaway and could save someone time if they are implementing similar pipelines. Beyond the recap, the paper adds no new experiments, no quantitative results from the submissions, and no fresh analysis of why certain approaches worked or failed. The methods it describes are already standard in representational similarity work. The assumption that RDM match is the right metric for model-brain alignment is stated but not examined for its limits or alternatives. The text stays at a high level with no equations, tables, or error breakdowns visible in the abstract or described structure. This makes it hard to judge how well the reviewed approaches actually performed. The paper is mainly useful to someone who wants a short orientation to that specific challenge without reading every entry. It does not have the depth or novelty to interest a broader audience in computational neuroscience. I would not bring it to a reading group or cite it. It does not look like it needs or deserves peer review; a desk reject fits what it actually delivers.

Referee Report

1 major / 0 minor

Summary. The manuscript reviews the author's submission to the Explaining the Human Visual Brain Challenge 2019 in both the fMRI and MEG tracks. The goal is to construct neural network features which generate representational dissimilarity matrices (RDMs) most similar to those extracted from fMRI and MEG data. It reviews exploring the optimal granularity of the receptive field, construction of intermediate surrogate features using Multidimensional Scaling (MDS), and modelling them using neural network features. It also points out some peculiarities of the RDM construction which have to be taken into account.

Significance. If the described construction of NN features via receptive-field granularity and MDS surrogates produces measurably closer RDM matches, the work could contribute a practical route for aligning artificial and biological visual representations. The discussion of RDM-construction peculiarities supplies a useful methodological note for the challenge community.

major comments (1)

Abstract: the manuscript describes the approaches at a high level without presenting specific results, error analyses, or validations, making it difficult to assess if the methods support the goal of generating RDMs most similar to brain data.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their review of our manuscript on the 2019 Explaining the Human Visual Brain Challenge submission. We address the single major comment below.

read point-by-point responses

Referee: Abstract: the manuscript describes the approaches at a high level without presenting specific results, error analyses, or validations, making it difficult to assess if the methods support the goal of generating RDMs most similar to brain data.

Authors: We agree the abstract is high-level. The manuscript focuses on methodological review of receptive-field optimization, MDS surrogate construction, and NN modeling for RDM alignment, with discussion of RDM construction peculiarities. To improve clarity, we will revise the abstract to include key quantitative outcomes from the challenge submission (e.g., achieved RDM correlations for fMRI and MEG tracks) and note any available validation metrics. This will better demonstrate how the methods contribute to closer RDM matches. revision: yes

Circularity Check

0 steps flagged

No significant circularity; methodological review only

full rationale

The paper is a review of a challenge submission focused on empirical construction of surrogate features (via receptive-field granularity and MDS) to produce RDMs matching brain data. No mathematical derivations, first-principles predictions, fitted parameters renamed as predictions, or self-citation chains appear in the abstract or described content. The central activity is data-driven matching rather than any claimed derivation that reduces to its inputs by construction. This is the expected honest non-finding for a purely methodological paper without equations or uniqueness theorems.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

As a review paper summarizing challenge participation, no new free parameters, axioms, or invented entities are introduced.

pith-pipeline@v0.9.0 · 5620 in / 1085 out tokens · 33560 ms · 2026-05-25T11:09:39.265772+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

Foundation/AlexanderDuality alexander_duality_circle_linking unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

optimal granularity of the receptive field... 5×5 grid for EVC/EARLY/LATE, 2×2 for IT

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.