{"id":"caf7a496-f7f2-4e7e-b25b-7697ca456b53","arxiv_id":"2411.14392","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A CvT architecture gives lower RMSE on Omega_m and sigma_8 than a ViT or a CNN on QUIJOTE field-level inference, with dark-matter pretraining helping CvT transfer to halo fields.","lead":"The paper tests a hybrid convolutional vision transformer (CvT) against a plain vision transformer and a CNN for estimating two cosmological parameters from simulated dark matter and halo maps. It reports that CvT gives tighter constraints than the other two networks and that pretraining on dark matter fields helps when fine-tuning on halo maps.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The architecture-level ranking is not yet supported because CvT is compared against much smaller baselines from single runs with no seed averaging; the transfer-learning claim is also contradicted for ViT by the paper's own numbers.","rationale":"Read in good faith, the paper is a clear, reproducible empirical study: it uses the public QUIJOTE suite, splits at simulation level to avoid leakage, makes code available, and describes architecture and training choices. My concern is not about fraud or internal math; it is about the inferential step from three single-run model comparisons to an architecture-level conclusion. The reader's weakest_assumption points at exactly this step, and I agree. The concrete test is a standard multi-seed, capacity-matched ablation with bootstrap uncertainty. Because no formal verification or parameter-free derivation protects the ranking, the conditional verdict is appropriate; I recommend no change to the reader's verdict. The transfer-learning contradiction in the abstract should be corrected regardless, but it is secondary to the capacity/fairness issue.","tokens_in":11481,"tokens_out":5576,"duration_ms":55020,"concrete_test":"Retrain CvT, ViT, and CNN under the identical pretrain/finetune/from-scratch protocol with at least 5 random seeds, and include a ViT matched to CvT's parameter count (e.g., increased depth/width) and a CNN of comparable capacity. Report per-simulation RMSE distributions and bootstrap 95% intervals over the 200 test simulations. If CvT's RMSE advantage over matched-capacity baselines is within seed scatter or flips sign, the §4 ranking is not established; if it persists, the central claim survives. In the same runs, record the ViT transfer gain to settle the contradictory abstract sentence.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in §4 is that CvT constrains Ωm and σ8 better than ViT and CNN when using DM or halo fields. For this to be a statement about architectures, the comparison must isolate architecture choice, but §2 reports 'No specific hyperparameter tuning has been performed' and each model is trained once. CvT-13 has 17.6M parameters, while the ViT baseline has 1.6M and the CNN has only five convolutional layers. Reported RMSE gaps (e.g., DM pretraining Ωm RMSE 0.059 vs 0.066 vs 0.073; σ8 RMSE 0.005 vs 0.1 vs 0.035) are therefore as consistent with capacity or initialization effects as with the hybrid design. The single-run protocol also makes the ranking vulnerable to run-to-run scatter; no confidence interval or seed average is given. A second, more local problem is that the abstract's transfer-learning claim is internally inconsistent: ViT's halo results improve with DM pretraining (Ωm RMSE 0.074→0.068, σ8 RMSE 0.112→0.106), so 'ViT and CNN do not show these benefits' is wrong for ViT as reported in §3. Both issues must be addressed before the headline claims are accepted.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper applies the Convolutional vision Transformer (CvT) to likelihood-free inference of the cosmological parameters Ωm and σ8 from 2D projected dark-matter and halo maps of the public QUIJOTE simulations. The authors train a moment-network regression head that predicts marginal posterior means and variances, pretrain CvT on dark-matter maps, and then finetune on halo maps, comparing against a vision transformer (ViT) and a small CNN. The headline claims are that CvT constrains both parameters better than ViT and CNN on both field types, that dark-matter pretraining helps CvT for halo-field inference, and that CvT is more computationally efficient than ViT. The paper reports RMSE and averaged error-bar metrics on a held-out test set and provides public code.","tokens_in":11739,"tokens_out":5444,"duration_ms":49139,"significance":"If the empirical claims survive robustness checks, the result would be practically useful: a hybrid convolutional-transformer architecture may combine the local feature extraction of CNNs with the global context of attention for field-level cosmological inference, and the transfer-learning finding could reduce the cost of training on expensive halo or galaxy simulations. The paper has clear strengths: it uses the public QUIJOTE simulations, a simulation-level train/validation/test split, a held-out test set, and released code, and there is no circularity in the evaluation because all numbers are computed on data not used in training. However, the current evidence base is a set of single-run point estimates from models of very different capacities, which is not yet sufficient to establish an architecture-level ranking or a robust transfer-learning conclusion.","major_comments":[{"comment":"The headline claim that CvT constrains Ωm and σ8 better than ViT and CNN is not yet supported as an architecture-level statement, because the comparison does not isolate architecture: CvT-13 has 17.6M parameters, the ViT baseline has 1.6M parameters, and the CNN has only five convolutional layers, and each model is trained once with no hyperparameter tuning and no seed averaging. Reported RMSE differences (e.g., dark-matter pretraining Ωm RMSE 0.059 for CvT versus 0.066 for ViT versus 0.073 for CNN) are therefore as consistent with capacity or run-to-run scatter as with the hybrid design. Please provide capacity-matched baselines or, at minimum, multiple seeded runs with error bars on every reported metric, and temper the abstract and conclusions if such a comparison is not performed.","section":"§3 'Comparison with traditional ViT' and 'Comparison with CNN'; §2 'Training details'"},{"comment":"The transfer-learning claim is internally inconsistent. The abstract and conclusion state that ViT and CNN do not benefit from dark-matter pretraining, but the ViT numbers in §3 show the opposite: halo training from scratch has Ωm RMSE 0.074 and σ8 RMSE 0.112, while pretraining then finetuning gives 0.068 and 0.106, which is an improvement for ViT. Only the CNN results fail to benefit. The abstract and conclusion should be corrected to state that the pretraining benefit was not found for CNN and that ViT did show an improvement in these single runs, or the conclusion should be recast with appropriate uncertainty.","section":"Abstract; §4 Conclusion; §3 'Comparison with traditional ViT'"},{"comment":"The CNN transfer-learning result appears pathological: for halo after dark-matter pretraining, Ωm RMSE is 0.21 and σ8 RMSE is 0.151 with ¯σ=0 for both parameters. The paper does not report whether the same head-reinitialization and unfrozen finetuning protocol was used for the CNN and ViT baselines, or whether the CNN failed to converge. As presented, this result is not interpretable as a property of CNN transfer learning; please specify the exact protocol for all models and either diagnose the CNN failure or remove it from the transfer-learning comparison.","section":"§3 'Comparison with CNN'"}],"minor_comments":[{"comment":"The preprocessing sentence 'The overdensities are first calculated (ρ/ρ)' is incomplete or typographically wrong; it should read δ = ρ/ρ̄ − 1, and the subsequent transformation should be written consistently as log10(1 + δ).","section":"§2 'Data'"},{"comment":"The sentence 'The constraints in (c) are worse than in (b) as shown by the lower values of RMSE and ¯σ' appears to say the opposite of what is meant; worse constraints should be reflected by higher RMSE values, so the phrasing should be corrected.","section":"§3 'Results'"},{"comment":"The abstract uses 'Convolution vision Transformer' while the body uses 'Convolutional vision Transformer'; please unify the terminology.","section":"Abstract"},{"comment":"Please clarify whether the scatter points represent per-simulation averages over the 30 2D maps and whether the reported RMSE and ¯σ are computed on those averaged predictions or on all individual maps; the definitions in §3 refer to N test examples without specifying the unit.","section":"Fig. 2 caption and §3 'Experimental details and evaluation'"},{"comment":"The statement 'We chose not to freeze any weights, as doing so only provided marginally better constraints...' is ambiguous about which model and finetuning configuration are being described; please make the subject explicit.","section":"§3 'Experimental details and evaluation'"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a workshop-style empirical study. The internal inconsistency in the transfer-learning claim is straightforward to fix, but the more substantial issue is that the architecture-ranking claim rests on single-run comparisons of models with very different parameter counts. Strengthening the evidence with seeded runs and capacity-matched baselines, or substantially tempering the claims, will be necessary before the paper meets the journal's evidentiary standard."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a straightforward, readable benchmark of CvT versus ViT and CNN for field-level Ωm/σ8 inference on QUIJOTE, with public code. The new thing is the first application of CvT to this task and a three-architecture transfer-learning comparison. The paper is honest about its own limitations, but two of its headline claims outrun the evidence.\n\nWhat works: clean setup, simulation-level train/test split, moment-network loss, held-out test metrics, and a free GitHub repo. The efficiency observation (17.6M-parameter CvT trains in about the same wall-clock time as a 1.6M ViT) is worth reporting even if the mechanism is not deeply explored. The writing is clear and the related-work placement is fine.\n\nWhere it gets shaky. First, the central claim that CvT 'constrains both parameters better than CNN and ViT' is not yet a claim about architectures. CvT has 17.6M parameters; the ViT baseline has 1.6M; the CNN has five conv layers. With no capacity matching and no seed averaging, the RMSE gaps could come from model size or initialization luck. Second, every metric is a single run, so there is no sense of run-to-run scatter; a couple of seeds would substantially change how much weight I give the ranking. Third, the abstract and conclusion say ViT and CNN do not benefit from dark-matter pretraining, but the paper's own numbers in §3 show ViT halo constraints improving with DM pretraining (Ωm RMSE 0.074→0.068; σ8 0.112→0.106). That makes the transfer-learning claim internally inconsistent as written. CNN indeed gets worse with pretraining, so the correct statement is architecture-dependent.\n\nThe paper also notes σ8 predictions for ViT are 'near-flat' and CNN gives σ̄=0 in one setting; those are flags that some baselines are poorly calibrated, which is useful context but not developed.\n\nBottom line: as a workshop-level empirical note, this is a reasonable contribution. It deserves a serious referee only if the authors are willing to address the capacity mismatch, run repeated seeds, and fix the transfer-learning wording. If those conditions are met, it is a useful data point for the field.\n\nWho it's for: someone shopping for a field-level inference architecture or studying transfer learning between biased and unbiased tracers. I wouldn't cite it yet in its current form, but I would read the revision.","headline":"Clean, honest first CvT benchmark for cosmology parameter inference, but the architecture-level claims rest on capacity-mismatched single runs and the transfer-learning claim is contradicted by the paper's own ViT numbers.","tokens_in":12228,"tokens_out":2879,"would_cite":false,"duration_ms":26578,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A hybrid convolutional vision transformer yields tighter constraints on matter density and clustering amplitude than pure CNN or plain transformer baselines on simulated cosmic fields.","keywords":["cosmological parameter inference","field-level inference","convolutional vision transformer","vision transformer","convolutional neural network","transfer learning","simulation-based inference","large-scale structure"],"falsifier":"Retrain CvT, ViT, and the CNN with matched numbers of parameters, several random initializations, and hyperparameter optimization on the same dark matter and halo maps; if CvT's RMSE advantage disappears or reverses, the claim that the hybrid architecture itself is better is refuted.","tokens_in":11274,"feed_emoji":"🌌","tokens_out":9232,"duration_ms":82649,"temperature":0.7,"pith_summary":"This paper tries to establish that a hybrid architecture, the convolutional vision transformer (CvT), estimates the matter density parameter $\\Omega_m$ and the clustering amplitude $\\sigma_8$ more accurately from two-dimensional maps of the dark matter distribution and of dark matter halos than either a plain vision transformer or a convolutional neural network does. The authors train all three models on the same simulated maps and report lower root-mean-squared errors and more representative error bars for CvT in nearly every comparison. They also find that pretraining CvT on dark matter fields and then finetuning on halo fields improves halo-field constraints relative to training from scratch, a transfer benefit they do not see for the two baselines. If these results hold, hybrid models would give cosmologists a practical way to extract more information from the full nonlinear matter distribution than traditional summary statistics allow.","feed_headline":"Hybrid transformer beats plain CNN and ViT at cosmic inference","feed_subtitle":"On simulated dark matter and halo fields, dark-matter pretraining also helps the hybrid model.","key_machinery":"The load-bearing object is the CvT architecture: a multi-stage vision transformer in which convolutional token embedding layers progressively reduce spatial resolution while increasing feature width, and depth-wise separable convolutions replace the linear query, key, and value projections of a standard transformer. This design captures local structure through convolutions and global relationships through self-attention in every block, and it eliminates the need for positional encoding. The paper's argument attributes CvT's tighter constraints and its successful dark-matter-to-halo transfer to this combined inductive bias. A supporting mechanism is the moment-matching loss, which trains the network to output both a posterior mean and a standard deviation for each parameter, allowing accuracy and uncertainty to be evaluated on the test set.","core_discovery":"In the paper's own terms, the central discovery is that the CvT design, replacing a vision transformer's linear projections with convolutional projections and convolutional token embeddings, gives tighter likelihood-free constraints on $\\Omega_m$ and $\\sigma_8$ from both dark matter and halo density maps. On the test set, CvT achieves lower RMSE than ViT for both parameters in the dark-matter pretraining and halo-transfer settings, and it achieves lower RMSE than the CNN in every setting except $\\sigma_8$ from dark matter, where the CNN is closer but CvT is still best. The paper also reports that initializing from a dark-matter pretrained model before finetuning on halo maps lowers RMSE and produces more representative error bars than training on halo maps from scratch, and that this benefit appears for CvT but not for ViT or the CNN. The authors interpret this as CvT leveraging the shared large-scale structure of dark matter and halos, while noting that more detailed tests are needed to confirm that mechanism.","pith_inferences":["Inference: the architecture comparison is not capacity-matched, since CvT has 17.6 million parameters versus 1.6 million for the plain ViT and a five-layer CNN, and no hyperparameter tuning was performed, so part of the reported gap may reflect model size and training budget rather than the CvT design itself.","Inference: a direct test of the transfer claim would give ViT and CNN the same pretraining data and budget as CvT; the paper reports no transfer benefit for them, but their smaller capacity may be the reason, and a capacity-matched comparison would clarify whether the benefit is specific to the hybrid design.","Inference: since the paper reports underestimated error bars on halo fields, especially for $\\Omega_m$, a practical next step is to recalibrate predicted variances on a validation set before quoting $1\\sigma$ uncertainties."],"forward_implications":["If robust, the comparison suggests hybrid convolution-transformer architectures should be the default choice for simulation-based cosmological parameter inference.","Dark-matter pretraining followed by halo finetuning gives a recipe for working with expensive, sparsely sampled tracers such as galaxies, where large simulation suites are often unavailable.","Because CvT does not require positional encoding, the same trained architecture can be applied to maps of different resolutions without modification, which is relevant for survey data with varying depth.","The transfer-learning result indicates that features learned from dark matter simulations carry over to biased tracers, so pretrained models could in principle be reused across multiple observables."],"supporting_citations":[{"why":"It supplies the CvT architecture with convolutional token embedding layers and convolutional projections that are the main subject of this work.","marker":"[26]"},{"why":"It defines the original vision transformer architecture that CvT modifies.","marker":"[8]"},{"why":"It provides the simplified ViT configuration that the paper adopts as its ViT baseline.","marker":"[2]"},{"why":"It supplies the N-body dark matter and halo simulations used for training, validation, and testing.","marker":"[23]"},{"why":"It gives the moment-matching loss that trains the network to output posterior means and variances.","marker":"[12]"},{"why":"It demonstrates CvT outperforming CNN and ViT on galaxy morphology classification, motivating the same comparison for cosmological inference.","marker":"[4]"}],"fun_headline_variants":["CvT hybrid outlearns ViT and CNN on cosmic parameters","Dark-matter pretraining only helps the hybrid transformer","Convolutional transformer tightens Omega_m and sigma_8 from simulated maps","Hybrid model: better cosmic constraints, faster than ViT","Pretrained CvT tops ViT and CNN on halo maps"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison assumes that the performance gap reflects architectural merit even though CvT is far larger than the baselines and no hyperparameter tuning was performed.","fun_headline_variants_meta":{"raw":{"variants":["CvT hybrid outlearns ViT and CNN on cosmic parameters","Dark-matter pretraining only helps the hybrid transformer","Convolutional transformer tightens Omega_m and sigma_8 from simulated maps","Hybrid model: better cosmic constraints, faster than ViT","Pretrained CvT tops ViT and CNN on halo maps"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001344,"raw_usage":{"total_tokens":5464,"prompt_tokens":950,"completion_tokens":4514,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":566,"completion_tokens_details":{"reasoning_tokens":4424}},"tokens_in":566,"tokens_out":4514,"duration_ms":28965,"temperature":1.0,"reasoning_tokens":4424,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:14:00.452551+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain CvT, ViT, and the CNN with matched numbers of parameters, several random initializations, and hyperparameter optimization on the same dark matter and halo maps; if CvT's RMSE advantage disappears or reverses, the claim that the hybrid architecture itself is better is refuted.","supporting_citations":[],"review_version":1}