{"id":"9be4b290-042c-4074-8fb5-f3b7ca500601","arxiv_id":"2502.00412","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"TROI learns a sparse brain-voxel mask from one hour of a new subject's fMRI data and improves several retrieval metrics over a MindEye2 baseline that uses manual ROIs.","lead":"This paper trains a model to automatically select the most informative brain locations from fMRI scans for reconstructing images a person saw, replacing hand-drawn brain regions. The method is aimed at new subjects where only one hour of brain data is available.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The evaluation split is the load-bearing uncertainty: fine-tuning on the first hour and evaluating on the full NSD dataset, where every image repeats three times, can inflate retrieval and reconstruction scores via image memorization unless the evaluation images are guaranteed disjoint from…","rationale":"The reader's identified weakest assumption is that the Stage-1 binary mask may discard informative voxels that Stage 2 cannot recover. That is a plausible mechanism-level risk, but the empirical headline claim does not depend on it alone: even an optimal mask cannot establish superiority over MindEye2 if the evaluation set includes training images. The paper's own text creates this risk: fine-tuning on the first 1-hour data and evaluating on the full target dataset, with NSD's three repeated viewings per image (§3.1, §3.2). The absence of any stated image-disjoint split or overlap check makes the central comparison unverifiable as written. I therefore do not think the reader's weakest assumption is the most load-bearing issue; the evaluation protocol is. The verdict remains CONDITIONAL rather than REJECT because the overlap may not actually exist and the paper may have followed the standard MindEye2 split without stating it; the concrete test above settles the question. If the test shows overlap, the central claim is retracted; if not, the conditional acceptance can proceed, though code and error bars would still be needed.","tokens_in":7424,"tokens_out":9342,"duration_ms":101831,"concrete_test":"Obtain NSD trial metadata for one target subject (e.g., Subject 1) and construct the exact training set: the first 1 hour of trials used in the paper. Build an evaluation set whose image IDs are guaranteed disjoint from the training image IDs, preferably the standard MindEye2 held-out image test set (982 images); if that is not available, exclude all repeated viewings of any training image from the full dataset. Recompute Table 1 metrics (image retrieval, brain retrieval, PixCorr, SSIM, Incep, SwAV) for both TROI and the MindEye2-with-ROI baseline on this disjoint set, and also report the overlap count for the current protocol. If the retrieval or Incep gains shrink or reverse, the paper's central claim is not supported; if they persist, the split concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is a quantitative superiority comparison against MindEye2 on small-sample decoding. Section 3.1 says the model is fine-tuned using the first 1-hour data of the target subject and 'finally evaluating on the full dataset of the target subject.' Section 3.2 states that in NSD 'each image is viewed three times during data collection, with the viewings spaced apart.' If the 'full dataset' evaluation includes later repeats of the first-hour training images, those exact images appear in the retrieval candidate pool and as reconstruction targets. That can inflate retrieval and high-level reconstruction metrics through memorization, and TROI's smaller mask may memorize the tiny training set differently from MindEye2's annotated-ROI mask. The paper does not report how the train/evaluation image split was made, whether image IDs overlap, or how the standard MindEye2 held-out test protocol was adapted. Without a disjoint-image evaluation, the headline advantage in Table 1 is not attributable to voxel selection; it could be an artifact of the evaluation split. The mask-irreversibility concern in the reader's verdict is secondary: even a perfect Stage-1 mask cannot validate the claim if the evaluation set is contaminated.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TROI, a two-stage data-driven ROI selection method for fMRI visual decoding. In Stage 1, a sparse 0-1 voxel mask is learned on a target subject's small sample (the first hour of NSD data) by optimizing a task loss with LASSO regularization, thresholding, and Gaussian low-pass filtering under a voxel budget. In Stage 2, the TROI module is retrained from scratch on the fixed mask using a learning-rate rewinding strategy. The backbone and MLP projector are pretrained on cross-subject data (subjects 2, 5, and 7 in the main experiments, with one passage also listing subject 3). Evaluation is reported for subjects 1, 2, 5, and 7 on image/brain retrieval and reconstruction metrics, comparing against MindEye2 with annotated ROI masks. The paper claims that TROI surpasses MindEye2 on the same small-sample dataset.","tokens_in":7703,"tokens_out":4011,"duration_ms":40277,"significance":"If the claims hold, TROI would be a practically useful contribution: it removes the need for manually annotated ROIs in fMRI decoding, reduces the input voxel count dramatically (to 3,000 from tens of thousands), and demonstrates that cross-subject pretraining plus sparse mask learning can transfer to a new subject with only one hour of data. The paper is honest in reporting all metrics in Table 1, and the ablations in Table 2 isolate the contributions of pretraining, low-pass filtering, and learning-rate rewinding. These are genuine strengths. However, the headline claim of surpassing MindEye2 is not supported by the current evaluation protocol: the paper does not establish that the evaluation images are disjoint from the fine-tuning images, and the metric-level results are mixed, with TROI consistently worse on PixCorr and SSIM. The four-subject comparison also lacks any error bars or significance testing, so several reported gains are within plausible noise. The core idea is defensible, but the central quantitative claim needs substantially stronger experimental support.","major_comments":[{"comment":"The evaluation split is the load-bearing issue. Section 3.1 states that the model is fine-tuned using the first 1-hour data of the target subject and then 'finally evaluating on the full dataset of the target subject.' Section 3.2 states that in NSD every image is viewed three times during data collection. If the full-dataset evaluation includes the later repetitions of images that appeared in the first-hour fine-tuning set, those exact images are present in the retrieval candidate pool and as reconstruction targets, which can inflate retrieval and reconstruction metrics through memorization. The paper does not report whether the evaluation images are disjoint from the fine-tuning images by image ID, how the standard MindEye2 held-out protocol was adapted, or the degree of overlap. This must be fixed: the comparison should be repeated on an evaluation set that is disjoint from the fine-tuning set at the image level, and the overlap statistics should be reported.","section":"Section 3.1-3.2"},{"comment":"The abstract's claim that 'our voxel selection method surpasses the state-of-the-art method MindEye2' overstates Table 1. TROI is worse than the annotated-ROI baseline on PixCorr and SSIM for all four subjects (e.g., subject 1 PixCorr 0.206 vs. 0.235; SSIM 0.382 vs. 0.428) and worse on image retrieval for subject 1 (93.6% vs. 94.0%). The improvements are concentrated in brain retrieval and Incep. The textual claims in Section 3.3.2 and the Conclusion should be qualified to reflect the mixed metric-level results rather than claiming overall superiority.","section":"Abstract and Section 3.3.2, Table 1"},{"comment":"There are no error bars, confidence intervals, or significance tests, and only four subjects are evaluated. Several reported differences are very small, for example subject 5 image retrieval 67.3% vs. 66.9% and brain retrieval 47.7% vs. 47.0%. Without repeated runs or a statistical test, these differences are not distinguishable from noise. The paper should at least report variability across multiple seeds or a paired significance test over subjects, and should temper conclusions for metrics whose differences are within a plausible noise range.","section":"Table 1"},{"comment":"Stage 2 fixes the binary mask M′ produced by Stage 1 and retrains the TROI module from scratch, but the manuscript is ambiguous about whether this retraining updates only the input layer or the whole backbone. If only the input layer is trained, any informative voxel discarded by the Stage-1 mask is unrecoverable, and the method's success depends entirely on the quality of a mask learned from roughly 15 epochs on one hour of data. The paper should clarify the scope of Stage-2 retraining and provide evidence on mask stability, for example by comparing masks learned with different training durations or showing that discarded voxels do not contain unique information needed for reconstruction.","section":"Section 2.3 and Algorithm 1"}],"minor_comments":[{"comment":"The pretraining subject list is inconsistent: Section 2.1 says the cross-subject backbone is trained using data from Subjects 2, 3, 5, and 7, while Section 3.1 says Subjects 2, 5, and 7. Please correct the discrepancy.","section":"Section 2.1 vs. Section 3.1"},{"comment":"The line 'Keywords First keyword · Second keyword · More' is a placeholder and should be replaced with actual keywords.","section":"Abstract"},{"comment":"The sentence 'we slightly outperform MindEye2 in small sample scenarios' is contradicted by the PixCorr and SSIM columns of Table 1; the discussion should acknowledge these metrics explicitly.","section":"Section 3.3.2"},{"comment":"The hyperparameters of the Gaussian low-pass filter (kernel size and standard deviation), the threshold th, and the voxel budget V are not specified in the experiments section; please report them for reproducibility.","section":"Section 2.2 and Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The evaluation-split contamination is the decisive issue. If the authors can demonstrate that the evaluation images are disjoint from the fine-tuning images and re-run the comparison under the standard held-out protocol, the manuscript could become acceptable after revision. In its current form, the central quantitative claim is not yet supported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this paper has a genuine idea, and it reports it honestly in the tables, but it overstates the conclusion and leaves a potentially load-bearing evaluation question unanswered.\n\nWhat's actually new: TROI combines cross-subject backbone pretraining with a trainable sparse input mask, a spatial low-pass filter before binarization, and learning-rate rewinding for the input layer. That specific pipeline is not in MindEye2 or in the pruning literature, and the ablations show each piece contributes something. The voxel budget experiment is also useful: going from 13k annotated ROI voxels to 3k selected voxels keeps retrieval competitive and even improves brain retrieval. If the mask transfer works for new subjects with one hour of data, that is a practical contribution for anyone working outside well-annotated datasets.\n\nThe paper is honest in its tables: PixCorr and SSIM are worse for all four subjects, image retrieval is worse for subject 1, and the abstract's \"surpasses\" language is too strong. Brain retrieval improves for all four subjects, which is a real effect but not the same as winning overall. The lack of error bars or significance tests is a flaw, but with four subjects and no seeds, it is the kind of flaw you expect in a first pass, not a fatal one.\n\nThe evaluation split is the bigger problem. Section 3.1 says fine-tuning uses the first hour of the target subject's data and evaluation runs on the full dataset. NSD images repeat three times. If the full dataset includes later repeats of the training images, then retrieval and reconstruction have already seen the exact target images, and a smaller mask can memorize the tiny training set differently than MindEye2's fixed ROI mask. The paper never states whether evaluation image IDs are disjoint from training image IDs. On this point the stress-test note is right: the discrepancy between strong retrieval gains and consistent reconstruction losses could be an artifact of the split, and the mask-transfer idea cannot be validated until this is ruled out.\n\nThe mask-irreversibility concern from the reader is secondary. Stage 2 only trains the input layer, so a bad Stage-1 mask cannot be fixed, but the voxel budget experiment (Table 3) suggests the mask is not catastrophic. Still, the paper should report failure cases and show what happens with masks of different quality.\n\nFor whom: this is a subfield paper for fMRI decoding researchers, particularly people who want to reduce ROI annotation burden. It deserves a serious referee, but the referee should require a disjoint-image evaluation, more subjects or seeds, and a qualified abstract before acceptance. I would cite it if the split issue is resolved and code is released.\n\nRecommendation: send to peer review, but treat it as conditional and ask for the evaluation protocol to be fixed and for the claims to be scaled back to what the data actually show.","headline":"TROI's cross-subject pretraining with a trainable sparse voxel mask is a plausible and honest small-sample fMRI decoding contribution, but the headline superiority claim over MindEye2 is undercut by a possibly contaminated evaluation split and by reconstruction metrics that get worse across all subjects.","tokens_in":8227,"tokens_out":711,"would_cite":true,"duration_ms":9755,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TROI's learned voxel masks beat manually annotated ROIs in fMRI decoding","keywords":["fMRI","visual decoding","cross-subject pretraining","voxel selection","sparse mask training","learning rate rewinding","image retrieval","image reconstruction"],"falsifier":"If a random 3,000-voxel subset of the same coarse brain region, under identical pretraining and Stage-2 rewinding, matches TROI's reported brain-retrieval accuracy on a held-out subject, then the learned voxel mask is not what carries the improvement over annotated ROIs.","tokens_in":7220,"feed_emoji":"🧠","tokens_out":6591,"duration_ms":62732,"temperature":0.7,"pith_summary":"This paper tries to show that fMRI visual decoding for a new subject can work without expert-drawn regions of interest (ROIs) and with very little data. It proposes TROI, a two-stage pipeline that first pretrains a decoding backbone on several other subjects using their annotated ROIs, then learns a sparse 0-1 mask over a coarsely segmented brain region to select roughly 3,000 informative voxels, and finally fine-tunes only the input layer on the new subject's first hour of data. The reported experiments on the Natural Scenes Dataset compare TROI against the MindEye2 baseline running on annotated ROIs and find that the learned mask improves brain-retrieval accuracy and high-level reconstruction metrics while using far fewer voxels. A sympathetic reader would care because, if this is right, it removes a major practical bottleneck: each new subject would no longer need manual anatomical labeling, only a coarse brain scan and an hour of data.","feed_headline":"Data-driven voxel masks beat manual ROIs in fMRI decoding","feed_subtitle":"Cross-subject pretraining plus sparse voxel selection decodes seen images from one hour of new-subject fMRI data.","key_machinery":"The central mechanism is the trainable region-of-interest mask: a 0-1 vector $M'$ applied to flattened fMRI voxels by a Hadamard product, learned in Stage 1 by optimizing the decoding loss plus an L1 penalty on a continuous weight mask $M$, then thresholding, Gaussian low-pass filtering, and binarizing under a voxel budget $V$. The mask does the work of the manual ROI by carving a coarse occipital segmentation down to about 3,000 voxels; the low-pass filter injects spatial locality as a prior, and Stage 2's learning-rate rewinding retrains the input layer on that fixed mask so the sparse structure, not the fine-tuned weights of a pretrained model, governs the input representation.","core_discovery":"On its own terms, the paper's central claim is that a data-driven voxel mask learned with sparse-mask training plus low-pass filtering can replace a manually annotated ROI for decoding visual stimuli from fMRI, and that cross-subject pretraining makes this feasible when the new subject contributes only one hour of data. The system first trains a shared residual MLP backbone and a diffusion-prior reconstruction head on multiple subjects using annotated ROIs; for the held-out subject it then trains a weighted mask under an L1 penalty, zeroes out small weights, smooths the mask with a Gaussian low-pass filter, and binarizes it to a fixed voxel budget of 3,000. Stage 2 retrains the subject-specific input layer from scratch at a rewound learning rate on the fixed mask. On subjects 1, 2, 5, and 7, the paper reports that this procedure beats annotated-ROI MindEye2 on brain retrieval (for example, subject 1: 85.3% versus 77.6%) and on two-way identification in reconstruction (85.9% versus 83.6%), while pixel-level similarity metrics stay mixed.","pith_inferences":["The paper's strongest reported result is on brain retrieval; a natural next test is whether the same mask-learning recipe transfers to other fMRI tasks or other coarse segmentations, since the method makes no visual-cortex-specific assumption beyond the initial coarse crop.","One implication the authors do not develop is that the learned mask could serve as a fast functional atlas for new subjects, potentially reducing the need for anatomical alignment when building brain-computer interfaces or clinical fMRI tools.","A testable extension would be to run the Stage 1 mask learner on data from a different scanner or resolution and check whether a fixed voxel budget still yields comparable decoding, which would probe the method's dependence on preprocessing and spatial smoothness assumptions.","The mixed pixel-level results leave open whether a mask tuned for retrieval and high-level semantics necessarily preserves the fine-grained spatial detail that pixel-correlation measures reward."],"forward_implications":["For a new subject with no ROI labels and only one hour of fMRI data, decoding pipelines can start from a coarse brain segmentation and learn the voxel subset automatically, instead of waiting for expert annotation.","The subject-specific input layer is the only part retrained for a new subject, so per-subject adaptation cost is low and overfitting risk is reduced in small-sample settings.","With a voxel budget around 3,000, the method is reported to reach or exceed annotated-ROI performance on brain retrieval and high-level reconstruction, implying that annotated ROIs contain a large fraction of task-irrelevant voxels.","The mask can be recomputed for each subject, so individual differences in brain organization are handled by the data rather than by a fixed atlas.","Different voxel budgets are available (1,000 to 3,000), giving a practical knob between input dimensionality and decoding accuracy."],"supporting_citations":[{"why":"Supplies the MindEye2 baseline and its shared-subject architecture, which TROI uses as the backbone and outperforms on annotated-ROI runs.","marker":"[3]"},{"why":"Provides the Natural Scenes Dataset, including the fMRI scans, images, and the manually annotated ROI masks used for pretraining and as the comparison baseline.","marker":"[11]"},{"why":"Establishes the LASSO/L1 penalty that drives the weighted voxel mask toward sparsity in Stage 1.","marker":"[14]"},{"why":"Defines the MixCo mix-up contrastive loss used to train the fMRI-image embedding space during cross-subject pretraining.","marker":"[15]"},{"why":"Supplies the latent diffusion prior and unCLIP decoder used to map fMRI embeddings onto image embeddings for reconstruction.","marker":"[17]"},{"why":"Motivates the learning-rate rewinding retraining strategy that Stage 2 applies to the input layer on the fixed mask.","marker":"[21]"}],"fun_headline_variants":["TROI's learned voxel masks outdo manual ROIs in fMRI decoding","Data-driven voxel selection beats manual ROIs for fMRI visual decode","Cross-subject pretraining plus learned masks boost fMRI visual decoding","TROI replaces manual ROI labels with data-driven masks for fMRI","Learned masks outperform hand-picked ROIs in brain reading"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the voxel mask learned from one hour of a new subject's data, after thresholding and smoothing, keeps every voxel that matters for decoding, because Stage 2 only refines the input layer and cannot recover information from voxels the mask discarded.","fun_headline_variants_meta":{"raw":{"variants":["TROI's learned voxel masks outdo manual ROIs in fMRI decoding","Data-driven voxel selection beats manual ROIs for fMRI visual decode","Cross-subject pretraining plus learned masks boost fMRI visual decoding","TROI replaces manual ROI labels with data-driven masks for fMRI","Learned masks outperform hand-picked ROIs in brain reading"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00058,"raw_usage":{"total_tokens":2774,"prompt_tokens":1027,"completion_tokens":1747,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":643,"completion_tokens_details":{"reasoning_tokens":1655}},"tokens_in":643,"tokens_out":1747,"duration_ms":13019,"temperature":1.0,"reasoning_tokens":1655,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T19:07:20.301848+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"If a random 3,000-voxel subset of the same coarse brain region, under identical pretraining and Stage-2 rewinding, matches TROI's reported brain-retrieval accuracy on a held-out subject, then the learned voxel mask is not what carries the improvement over annotated ROIs.","supporting_citations":[],"review_version":1}