{"id":"5d195349-b495-461f-a380-bd37c153d62f","arxiv_id":"2508.10034","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"An ensemble of ResNet50 and InceptionV3 convolutional networks classifies JetNet jet images with about 75% multi-class accuracy, slightly better than each network alone.","lead":"Researchers turned collider jets into images and used two pre-trained neural networks together to classify jet origins, achieving about 75% accuracy on five classes and slightly better results than either network alone. The study is a practical test of using well-known image recognition models for particle physics, though the gains are small compared to more advanced jet-tagging models.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Claimed ensemble superiority lacks statistically valid support: the only significance test is mis-specified and binary gains have no error bars.","rationale":"The reader's stated weakest assumption was the jet-image cropping in Section 2. That preprocessing choice affects absolute accuracy, but it does not directly threaten the ensemble-versus-components comparison because all models receive the same images. The load-bearing point for the central claim is instead statistical: the ensemble advantage is small and the only significance test provided is mis-specified for the paired cross-validation design. The paper has real strengths: a clear architecture, an ablation study, a held-out test split, and a public code repository, and the EM is numerically ahead on every table. Those make the claim plausible but not yet rigorously supported. I therefore agree with the reader's CONDITIONAL verdict, but for a different primary reason. The concrete test above would settle whether the observed superiority is real or an artifact of noise and inappropriate testing.","tokens_in":13635,"tokens_out":8268,"duration_ms":95364,"concrete_test":"Re-run all four binary tasks and the multi-class task with the same 5-fold splits, saving per-fold test-set predictions for EM, ResNet50, and InceptionV3. For each fold compute paired differences (EM minus component) in accuracy and AUC, then build a 95% confidence interval using a paired t-test or bootstrap. Also recompute the Section 4.2.2 t-tests as paired across folds (or use McNemar's test on concatenated per-jet predictions). If any confidence interval for a binary task includes zero, the claim of consistent superiority is unsupported; if all intervals exclude zero, the claim survives this statistical check.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the Ensemble Model is superior to ResNet50 and InceptionV3 individually. Tables 2-6 show the EM ahead on point estimates, but the differences are small for binary tasks (accuracy gains ~0.5-1.6%, AUC gains ~0.004-0.009) and no error bars or significance tests are provided for those four comparisons. The only formal significance evidence is in Section 4.2.2, and it is described as 'unpaired two-sample t-tests between the model accuracies.' This is miscalibrated for a 5-fold cross-validation design: the accuracy values for EM and for each component come from the same folds, so the samples are paired, not independent. An unpaired test discards this pairing and inflates the apparent significance; moreover, with only five folds the normality assumption is fragile. The report also gives the unusual pattern of validation accuracy consistently below test accuracy (Tables 2-6), which further suggests the evaluation pipeline may not be producing conventional, directly interpretable point estimates. Unless the significance computation is corrected and applied to the binary tasks as well, the observed EM advantage could be within fold-to-fold noise, so the paper's core comparative claim is not established by the reported statistics.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies jet tagging on the JetNet dataset by converting point-cloud jets into 299x299 images: relative transverse momenta are binned in the (eta_rel, phi_rel) plane, cropped to [-0.4,0.4], with at most 30 particles per jet. The proposed Ensemble Model (EM) concatenates the 1024-d feature vector from ResNet50 and the 2048-d vector from InceptionV3, projects the concatenation through a 512-d fusion layer, and ends in a softmax classifier. Binary tasks (gluon vs light-quark, top, W, Z) and a five-class task are evaluated with 5-fold cross-validation, 15 epochs per fold, reporting accuracy, AUC, and ROC curves, together with a component-wise ablation, t-tests, and Grad-CAM analysis. The central claim is that the EM outperforms either constituent network on these tasks.","tokens_in":13929,"tokens_out":5633,"duration_ms":54235,"significance":"The paper is a modest but clearly scoped application of standard CNN ensembles to jet images. Its strengths are the component-wise ablation, the inclusion of a no-pretraining comparison, the use of 5-fold CV, and the public code repository. If the statistical evidence were repaired, the result would be a useful data point that feature-level fusion of two ImageNet-pretrained CNNs can improve JetNet tagging over either network alone. The main weakness is that the central comparative claim currently rests on small point-estimate differences without uncertainty quantification, and the only formal significance test is mis-specified and limited to the multi-class case.","major_comments":[{"comment":"The tests are described as 'unpaired two-sample t-tests between the model accuracies.' Under a 5-fold cross-validation design, the three models are evaluated on the same folds, so the samples are paired, not independent; an unpaired test discards this pairing and can overstate significance. In addition, significance testing is reported only for the multi-class task, while the binary comparisons in Tables 2-5 are the core comparative claim. Please replace this with a paired test (e.g., paired t-test or Wilcoxon signed-rank) on per-fold metrics, report the number of paired observations and effect sizes, and apply the same procedure to all four binary tasks. As written, the extremely small p-values (3.57e-8, 2.2e-9, 2.76e-6) cannot be checked because no per-fold data are shown.","section":"Section 4.2.2, t-test paragraph"},{"comment":"Only mean accuracy and AUC point estimates are reported; there are no per-fold values or standard deviations. The EM gains in the binary tasks are small (accuracy +0.004 to +0.016, AUC +0.004 to +0.013), so without error bars they are consistent with fold-to-fold noise. The statement in Section 4.1 that the EM 'exhibits narrower performance fluctuations' is not supported by any reported dispersion measure. Please report per-fold metrics or confidence intervals for all models and tasks; this is necessary to support the central claim of ensemble superiority.","section":"Tables 2-6"},{"comment":"Validation accuracy is consistently lower than testing accuracy (e.g., Table 2: 0.7632 vs 0.7940; Table 6: 0.7298 vs 0.7508). This pattern is atypical and requires explanation. The text describes an 80/20 training-validation/testing split and then a 5-fold cross-validation protocol, but it is not clear how the folds relate to the 80% subset or how final test metrics are computed. Please clarify the exact evaluation pipeline: what the validation folds are used for, whether early stopping or model selection is applied, and how the reported test numbers are averaged. Without this, the reported point estimates are not directly interpretable.","section":"Tables 2-6 and Section 3.2"}],"minor_comments":[{"comment":"The text states that the EM achieved testing accuracy up to 0.91 and AUC of 0.97 for 'both the g-jet vs. W-jet as well as the g-jet vs. t-jet cases.' Table 3 gives t-jet AUC 0.950, while Table 4 (W) gives 0.973 and Table 5 (Z) gives 0.974. The 'both' claim is therefore incorrect unless it refers to W and Z.","section":"Section 5, first paragraph"},{"comment":"The sentence 'InceptionV3 (when InceptionV3 was removed)' should read 'InceptionV3 (when ResNet50 was removed).'","section":"Section 4.2, text above Table 6"},{"comment":"Typo: 'JeTNet' should be 'JetNet.' Also, the relationship between the 30-particle cutoff and the later image-based analysis is not discussed; a one-sentence justification would help.","section":"Section 2, first paragraph"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope and I do not see grounds for rejection. The idea is simple but the ablation and open code are valuable. The statistical concerns are fixable: if the authors add per-fold results, switch to paired tests, and clarify the cross-validation/test protocol, the central claim could be established. One point to watch is the unusual validation-below-test pattern; if it reflects a data handling issue rather than a reporting artifact, the paper would need a more substantive revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Paul,\n\nQuick take: this is a plainly written, competent empirical study that does what it says—puts two pretrained CNNs together in a late-fusion ensemble, applies them to JetNet jet images, and shows the ensemble edges out either network on point estimates. Nothing conceptually new: ensembling and jet images are both established, and the authors cite the relevant work. The contribution is the specific configuration plus an honest ablation and a public GitHub repo. Worth a look, but not a major step for the field.\n\nWhat it does well: preprocessing is described carefully (crop, binning, max 30 particles), the ablation is a real ablation (each network independently with the same fusion layer), the multi-class comparison includes repeated runs across seeds, and the Grad-CAM check is a reasonable sanity test. The authors also explicitly warn that a direct numeric comparison to JetClass-trained models is inappropriate—right call.\n\nWhere it gets soft is the statistics. The main tables give averaged test accuracies and AUCs with no standard deviations or error bars, and the binary gains are small (0.5–1.6% accuracy, 0.004–0.009 AUC). The only formal significance test appears in Section 4.2.2, described as an unpaired two-sample t-test between model accuracies. That is mis-specified for 5-fold cross-validation: the folds are paired, and an unpaired test ignores that pairing and inflates significance. With only five folds, the normality assumption is also fragile. Separately, validation accuracy is systematically lower than test accuracy in every table—unusual and unexplained, making me suspect the evaluation pipeline does something non-standard (e.g., validation measured at an earlier epoch, or a test split that is accidentally easier). None of this means the ensemble advantage is false; the direction is consistent across all tasks. But the paper's claim of 'superior performance' is not as firmly supported as the text suggests.\n\nThe crop choice (eta/phi within ±0.4) is plausible given the jet radius but untested; wide subjets from tops and W/Z bosons could lose signal. Minor.\n\nWho this is for: someone assembling a baseline pipeline for jet-image tagging on JetNet, or teaching an applied-DL course. It deserves a serious referee—the empirical work is real and reproducible—but the referee should ask for corrected significance testing, error bars on the main tables, and an explanation of the validation/test inversion. If those come back clean, the paper is fine as a modest application note.\n\nMy verdict: send to peer review, expect heavy revision. I would not cite it in the next year.\n\n—","headline":"Competent, honest empirical study of a known idea—ensemble of two pretrained CNNs for jet images—with real but small gains, and statistics that need fixing.","tokens_in":14391,"tokens_out":2617,"would_cite":false,"duration_ms":28923,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An ensemble that concatenates ResNet50 and InceptionV3 image features tags jets more accurately than either network alone on the JetNet dataset.","keywords":["jet tagging","ensemble learning","convolutional neural networks","ResNet50","InceptionV3","JetNet dataset","jet images","deep learning"],"falsifier":"Train the same Ensemble Model on the same JetNet classes but with the full available ranges $\\eta_{\\mathrm{rel}}\\in[-1.6,1.0]$ and $\\phi_{\\mathrm{rel}}\\in[-0.5,0.5]$, or with a different binning, keeping the same folds and epochs. If the ensemble no longer beats both ResNet50 and InceptionV3 on held-out accuracy and AUC, the claimed superiority is an artifact of the $[-0.4,0.4]$ crop rather than a property of feature concatenation.","tokens_in":13558,"feed_emoji":"⚛️","tokens_out":9511,"duration_ms":85226,"temperature":0.7,"pith_summary":"The paper sets out to establish that jet tagging improves when two convolutional networks with different architectural strengths are run as a single ensemble. On the JetNet dataset, each jet is converted into a $299\\times299$ image whose pixel intensity is the relative transverse momentum $p_T^{\\mathrm{rel}}$ in the $(\\eta_{\\mathrm{rel}},\\phi_{\\mathrm{rel}})$ plane, cropped to $[-0.4,0.4]$ in both directions. The same image is fed to ResNet50 and InceptionV3, both pretrained on ImageNet, and their final feature vectors are concatenated before a fusion layer and softmax classifier. Across binary gluon-versus-X tasks and a five-class task, the ensemble reaches testing accuracies up to 0.91 and AUC up to 0.97, beating either component network. If true, this offers a computationally modest route to competitive jet classification without the very large training sets used by particle-level transformer models.","feed_headline":"0.97 AUC: two-CNN ensemble beats either network on jets","feed_subtitle":"Concatenating 1024- and 2048-dimensional features pushes JetNet tagging accuracy to 0.91 and AUC to 0.97.","key_machinery":"The load-bearing object is the fusion layer at the top of the ensemble: it concatenates the 1024-dimensional global-average-pooled feature vector from ResNet50 with the 2048-dimensional average-pooled feature vector from InceptionV3, forming a 3072-dimensional representation that is projected to 512 units and then to a softmax classifier. This concatenation is what lets the network exploit two complementary inductive biases: residual depth that captures compact, localized energy patterns, and multi-scale inception convolutions that capture spatially distributed substructure. The images themselves are the input representation: $299\\times299$ histograms of relative transverse momentum $p_T^{\\m","core_discovery":"The central claim is that the Ensemble Model—ResNet50 and InceptionV3 fused by concatenating their post-pooling feature vectors (1024 plus 2048 dimensions), projecting to 512, and applying softmax—achieves better jet classification than either network alone. On balanced, stratified JetNet data (170,000 images per class, 80/20 train-test split, 5-fold cross-validation, 15 epochs per fold), the ensemble outperforms both components in every binary gluon-versus-class task and in the five-class task. The strongest binary results are testing accuracy 0.9175 with AUC 0.973 for gluon versus W and 0.9122 with AUC 0.974 for gluon versus Z; multi-class testing accuracy is 0.7508 with average AUC 0.935.","pith_inferences":["A natural extension, left untested in the paper, is whether the same feature-concatenation recipe transfers to richer datasets like JetClass or to transformer-based vision backbones; the paper lists both as future work.","The absolute accuracies are tied to the $[-0.4,0.4]$ crop and $299\\times299$ binning; no alternative preprocessing was tested, so the ensemble's advantage is established only within this representation.","The Grad-CAM evidence of complementarity suggests a concrete next experiment: compare concatenated deep features against late fusion of per-network class probabilities, or add a third diverse backbone, to see whether the gain scales with architectural diversity.","Because ImageNet-pretrained weights already help on physics images, part of the reported accuracy may be inherited from generic low-level edges rather than physics-specific features; freezing early layers or training from scratch for more epochs would separate those contributions."],"forward_implications":["The ensemble beats both single networks in all four binary gluon-versus-X tasks and in the five-class task, with the largest AUC reaching 0.974 for gluon versus Z.","The performance edge is stable: mean AUC across random seeds is 0.9342 with standard deviation 0.0002, and t-tests give p-values below $10^{-8}$ against each component.","Pretrained ImageNet initialization matters: with random initialization, multi-class testing accuracy drops from 0.7508 to 0.7007 and AUC from 0.935 to 0.902.","The ensemble adds only a modest computational overhead (roughly 3–6% more training time than the slower single network) while improving accuracy.","The approach reaches about 75% multi-class accuracy using fewer than one million training jets, which the authors contrast with particle-level models trained on 100 million jets, while cautioning that direct numerical comparison across datasets is inappropriate."],"supporting_citations":[{"why":"Supplies the JetNet dataset used for training and evaluation, defined in the paper as the benchmark with point-cloud jet representations.","marker":"[12]"},{"why":"Establishes the jet-to-image conversion that maps relative transverse momentum into pixels on the (eta_rel, phi_rel) plane, which the Ensemble Model inherits.","marker":"[13]"},{"why":"Defines ResNet50, one of the two pretrained backbone architectures whose residual features feed the ensemble.","marker":"[19]"},{"why":"Defines InceptionV3, the other pretrained backbone whose multi-scale features are concatenated with ResNet50's.","marker":"[20]"},{"why":"Motivates the ensemble approach by describing how combining classifiers leverages complementary strengths.","marker":"[21]"},{"why":"Supplies ImageNet pretrained weights used to initialize both networks; the paper's ablation shows pretraining gives a large performance advantage.","marker":"[28]"},{"why":"Provides Grad-CAM, the attribution method used to show that ResNet50 captures localized features and InceptionV3 captures broader patterns, supporting the complementarity argument.","marker":"[36]"}],"fun_headline_variants":["Fused ResNet50 + InceptionV3 hit 0.97 AUC on jets","Ensemble of two CNNs beats either alone on jet tags","Jet tagging: combined CNN ensemble reaches 0.97 AUC","Two-network fusion tops single models for jet classes"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The central claim depends on the choice to crop each jet to $\\eta_{\\mathrm{rel}}, \\phi_{\\mathrm{rel}} \\in [-0.4, 0.4]$ and bin it into a $299\\times299$ image whose pixel intensity is the relative transverse momentum; if particles outside this crop carry discriminative signal, the reported accuracies are tied to this preprocessing rather than to the ensemble itself.","fun_headline_variants_meta":{"raw":{"variants":["Fused ResNet50 + InceptionV3 hit 0.97 AUC on jets","Ensemble of two CNNs beats either alone on jet tags","Jet tagging: combined CNN ensemble reaches 0.97 AUC","Two-network fusion tops single models for jet classes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000168,"raw_usage":{"total_tokens":1099,"prompt_tokens":745,"completion_tokens":354,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":489,"completion_tokens_details":{"reasoning_tokens":280}},"tokens_in":489,"tokens_out":354,"duration_ms":3955,"temperature":1.0,"reasoning_tokens":280,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T22:20:39.801282+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same Ensemble Model on the same JetNet classes but with the full available ranges $\\eta_{\\mathrm{rel}}\\in[-1.6,1.0]$ and $\\phi_{\\mathrm{rel}}\\in[-0.5,0.5]$, or with a different binning, keeping the same folds and epochs. If the ensemble no longer beats both ResNet50 and InceptionV3 on held-out accuracy and AUC, the claimed superiority is an artifact of the $[-0.4,0.4]$ crop rather than a property of feature concatenation.","supporting_citations":[{"cited_title":"Particle cloud generation with message passing generative adversarial networks","cited_arxiv_id":null,"evidence_quote":"Supplies the JetNet dataset used for training and evaluation, defined in the paper as the benchmark with point-cloud jet representations."},{"cited_title":"Particle cloud generation with message passing generative adversarial networks.Advances in Neural Information Processing Systems, 34:23858–23871, 2021","cited_arxiv_id":null,"evidence_quote":"Establishes the jet-to-image conversion that maps relative transverse momentum into pixels on the (eta_rel, phi_rel) plane, which the Ensemble Model inherits."},{"cited_title":"Rethinking the inception architecture for computer vision","cited_arxiv_id":null,"evidence_quote":"Defines InceptionV3, the other pretrained backbone whose multi-scale features are concatenated with ResNet50's."},{"cited_title":"Ensemble-based classifiers.Artificial intelligence review, 33:1–39, 2010","cited_arxiv_id":null,"evidence_quote":"Motivates the ensemble approach by describing how combining classifiers leverages complementary strengths."},{"cited_title":"Imagenet: A large-scale hierarchical image database.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 248–255, 2009","cited_arxiv_id":null,"evidence_quote":"Supplies ImageNet pretrained weights used to initialize both networks; the paper's ablation shows pretraining gives a large performance advantage."},{"cited_title":"Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, andDhruv Batra","cited_arxiv_id":null,"evidence_quote":"Provides Grad-CAM, the attribution method used to show that ResNet50 captures localized features and InceptionV3 captures broader patterns, supporting the complementarity argument."}],"review_version":1}