{"id":"c671f0ca-8261-4b38-99a7-2cd9c576272c","arxiv_id":"1908.08256","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Jet charge as an input channel improves deep-learning W+/W-/Z classification, and a dual-CNN architecture gives the largest gains for Z versus W discrimination.","lead":"This paper shows that adding jet charge information to deep neural networks noticeably improves computer-vision tagging of boosted W and Z bosons at the LHC. It introduces a two-branch network that separately reads momentum and charge images, and reports large gains over simple cuts and boosted decision trees.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"A concrete issue: kappa and CNN2 depth appear to be selected on the same test set used for final metrics, which can inflate the reported R50 gains and the CNN2-vs-CNN advantage.","rationale":"The reader's weakest_assumption included the kappa/architecture selection issue as a secondary point, framed as unstated. The paper's own wording shows all ROC/SIC curves, including the kappa scans used to fix hyperparameters, are evaluated on the testing set, so this is not merely an unstated possibility: the reported final metrics are likely optimistically biased by selection on the test set. This is the single most load-bearing concern because it directly affects the central quantitative claims (30-40% improvement from deep learning, and the specific CNN2-vs-CNN advantage) and can be checked from within the manuscript without external data. The MC-faithfulness concern about transfer to real LHC data is real but less immediately checkable from the paper alone. Since the issue is addressable by re-running with a clean validation/test protocol, the appropriate verdict remains CONDITIONAL rather than ACCEPT or REJECT; the reader's conditional verdict is therefore unchanged, though the supporting reason shifts toward this more concrete methodological flaw.","tokens_in":19671,"tokens_out":4126,"duration_ms":44675,"concrete_test":"Re-run the analysis with a clean split: use only the 9/10 training subset and 1/10 validation subset to choose kappa and the CNN2 depth (e.g., number of convolutional layers in the pT branch), then freeze the models and evaluate once on the held-out test set. If the validation-selected CNN2 R50 for Z/W+ is not substantially above the CNN's, or if the gap shrinks by more than about 20%, the headline 'CNN2 further boosts performance' is an artifact of test-set selection. As a minimal check, report validation R50 alongside test R50 for Tables 5 and 6.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2 states 'All the ROC and SIC curves shown in this paper are evaluated using the testing sets described here.' Sections 4.1 and 5.1 set kappa=0.3 for the BDT and kappa=0.15 for the CNNs based on Figure 8, which plots AUC/ACC/R50 versus kappa on those test-set curves. Section 3.2 says the CNN2 architecture was chosen 'after investigations on the model structure and seeing performance trends in the different classification problems,' and Section 7.2 evaluates checkpoints 'on the testing jet samples.' Thus the final R50/AUC/ACC numbers in Tables 4-6 are not independent of model selection. This is especially damaging for the central claim that CNN2 substantially beats CNN for Z/W+ (R50 52.6 vs 40.4), because CNN2 has many more hyperparameters and its depth was selected to maximize those very test metrics. The paper does not report validation metrics or a separate final test set, so the magnitude of the optimism bias is unknown. The Monte-Carlo-faithfulness question remains, but this is the more immediate, checkable threat to the numerical claims.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies boosted hadronic W/Z tagging using jet charge and deep learning. The authors generate VBF-like events with heavy Higgs decays to WW and ZZ, shower them with Pythia, simulate the detector with Delphes, and reconstruct anti-kT jets with R=0.7 in the pT range 350-450 GeV. They compare cut-based and BDT taggers built from (M, Q_kappa) with CNN taggers trained on two-channel jet images (pT, Q_kappa), including a composite architecture (CNN2) with separate branches for the pT and Q_kappa channels. Binary tasks (W-/W+ and Z/W+) and a ternary task (W+/W-/Z) are evaluated through AUC, accuracy, and R50. The central claim is that deep learning with jet charge improves background rejection by about 30-40% over the cut/BDT baselines, and that CNN2 yields further gains, especially for Z versus W discrimination. An appendix repeats the CNN/CNN2 analysis with Herwig showering as a cross-check.","tokens_in":19892,"tokens_out":3132,"duration_ms":34570,"significance":"If the numerical claims survive scrutiny, the paper provides a useful benchmark for charge-sensitive weak boson tagging at the LHC and demonstrates a concrete architectural idea (separate network depths for pT and charge channels) that could transfer to other jet classification problems. The explicit comparison across multiple methods, the projection consistency check between ternary and binary taggers, the saliency-map visualization, and the Herwig cross-check are all valuable and are executed with care. The main limitation is that the performance numbers are obtained in a Monte Carlo simulation without data validation, and, more immediately, the model-selection procedure appears to use the test set, which can inflate the reported gains.","major_comments":[{"comment":"The hyperparameters and architecture appear to be tuned on the same test sets used for the headline metrics, which is a load-bearing issue for the quantitative claims. Table 2 states that all ROC and SIC curves are evaluated using the testing sets; Figures 8 and 10 plot AUC/ACC/R50 versus kappa, and the text in Sec. 4.1 fixes kappa=0.3 for the BDT and kappa=0.15 for the CNNs based on those test-set curves. Section 3.2 says the CNN2 architecture was chosen 'after investigations on the model structure and seeing performance trends in the different classification problems,' and Sec. 7.2 evaluates checkpoints on the testing jet samples. Consequently, the R50 values in Tables 4-6, in particular the CNN2-vs-CNN advantage in the Z/W+ row (52.60 vs 40.42), may be optimistically biased. The authors should either use a strict train/validation/test split for all tuning decisions, or report validation-based curves and a final untouched test set, or perform nested cross-validation. Without this, the central 'significant boost' claim is not yet established.","section":"Secs. 2, 4.1, 5.1, 3.2, 7.2"},{"comment":"No statistical uncertainties are reported for any of the performance metrics. Given that the test samples contain only ~35k-40k jets per class, the differences between, for example, CNN R50=40.4205 and CNN2 R50=52.6028 in Table 5 need confidence intervals, bootstrap errors, or at least a repeated-initialization run. The absence of uncertainties makes it impossible to judge whether the CNN2 improvement is real or within training noise, especially since CNN2 has many more trainable parameters and a more complex architecture.","section":"Tables 4-6 and Appendix A"},{"comment":"The Herwig cross-check as written does not test the robustness of the Pythia-trained taggers to a different shower model. The appendix trains CNN and CNN2 on Herwig-showered samples and compares their performance to the Pythia-trained results, but the robustness question that matters for the MC-faithfulness concern is whether a tagger trained on Pythia and then applied to Herwig retains its performance. The current statement that 'the tagging abilities of our CNN and CNN2 taggers are independent of showering and hadronization models' is stronger than what the presented test shows and should be either clarified or supplemented with a direct transfer test.","section":"Appendix A"},{"comment":"The central 30-40% improvement claim is framed as a benefit from deep learning and jet charge, but the baseline BDT uses only (M, Q_kappa) while the CNNs see full jet images with substructure information. The paper itself acknowledges in Sec. 5.2 that the cut-based and BDT methods do not include any jet substructure variables, so part of the gain is expected from additional low-level information rather than from deep learning per se. The abstract and conclusions should qualify the claim accordingly, or the authors should add a BDT baseline with substructure variables, so that the gain attributable to deep learning is not conflated with the gain due to extra input information.","section":"Secs. 5.2 and Abstract"}],"minor_comments":[{"comment":"The text around Eq. (2) says the sum runs over jet constituents with pT > 500 MeV, but the jet-charge images in Sec. 2.2 are defined by summing over constituents in each pixel; it would be helpful to state explicitly whether the same pT threshold is applied in the pixelated version.","section":"Sec. 2.1 and Eq. (2)"},{"comment":"The architecture strings in Table 3 are difficult to parse; a short textual description or a reference to a public model definition (for example, a Keras summary) would improve reproducibility.","section":"Table 3"},{"comment":"The caption says 'The dashed curves are for binary classifications, and the solid curves for the projected ternary results,' but the text in Sec. 6.1 refers to 'dotted curves'; the figure/legend terminology should be made consistent.","section":"Fig. 15 caption"},{"comment":"There is a typo in the first paragraph: 'parton showing' should be 'parton showering.' Also, the table captions use 'T able' instead of 'Table' in several places.","section":"Appendix A"},{"comment":"The conclusion mentions the pT~1 TeV scenario and says results are qualitatively the same, but no corresponding figures or tables are shown; a reference to a supporting plot or a quantitative statement would help the reader assess this claim.","section":"Sec. 8"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical study and the central architectural idea is worth publishing, but the test-set-based model selection is a serious concern that affects the numerical headline. I would require either a clean train/validation/test separation with final metrics on an untouched test set, or nested cross-validation, before accepting. Also, the absence of uncertainties and the conflation of deep-learning gains with input-information gains should be addressed. I do not see grounds for rejection: the issues are fixable within the scope of the manuscript if the authors can re-run the evaluation properly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Honest take: this is a solid ML-for-jets benchmark with a genuinely new twist. The ternary W+/W-/Z task plus CNN2, a dual-branch network that gives the pT and jet-charge channels different depths, is new as a combination, and the paper shows it pays off for Z/W separation beyond a plain CNN. The Herwig cross-check is a real plus; it doesn't prove the numbers transfer to data, but it does show the tagger isn't memorizing one shower model. Credit also where due: the comparisons with ATLAS and with Fraser-Schwartz are honest, and the citation pattern looks fine. Self-citation is not an issue here; the related work is genuinely related.\n\nThe soft spots are in proportion. First, no statistical uncertainties and no release of code or data. Second, the baselines are slightly unfair: the cut-based and BDT taggers get only M and Qκ, while the CNNs get full jet images, so the “deep learning gain” mixes algorithm and input representation. That is common in this literature, but it should be stated more carefully. Third, and this is the one I would push on in a report, the test set appears to be doing double duty as a model-selection set. Section 2 says all ROC and SIC curves are evaluated on the testing sets. Figure 8 is used to pick κ=0.3 for the BDT and κ=0.15 for the CNNs, and the CNN2 depth was chosen after “seeing performance trends in the different classification problems.” No independent final test set is reported. That does not kill the qualitative conclusion; the gains are large and likely robust. But it means the R50 numbers, and especially the CNN2-vs-CNN margin (52.6 vs 40.4 for Z/W+), are optimistic to an unknown degree. The paper should say exactly which choices were made on the validation split and confirm the final numbers come from a held-out set.\n\nWho is the paper for? LHC phenomenologists and jet-tagging practitioners who care about charge-sensitive W/Z identification, W charge asymmetries, and same-sign diboson searches. A serious referee can fix the reporting issues. I would send it to review, and I would ask for a clean validation/test protocol and uncertainty estimates before publication.","headline":"Worth engaging: a genuinely new ternary W+/W-/Z tagger with a dual-branch CNN, but the headline gains are partly selected on the same test set used for the final metrics, so treat the numbers as optimistic until a clean validation/test protocol is reported.","tokens_in":20419,"tokens_out":2340,"would_cite":true,"duration_ms":24828,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Including jet charge as a deep-learning input materially improves boosted W and Z tagging, with a composite CNN more than tripling background rejection for Z versus W compared with a boosted decision tree.","keywords":["boosted W/Z tagging","jet charge","deep learning","convolutional neural network","jet substructure","ternary classification","LHC phenomenology"],"falsifier":"Retrain the same taggers on jets tagged from LHC collision data through semileptonic decays and compare R50 for Z versus W; if the CNN2 advantage over a boosted decision tree shrinks toward zero, or the charge channel stops helping, the central claim would be falsified. An alternative shower model alone cannot settle this because it shares the same hard-process and detector model.","tokens_in":19490,"feed_emoji":"⚛️","tokens_out":8605,"duration_ms":81569,"temperature":0.7,"pith_summary":"This paper aims to show that boosted, hadronically decaying W and Z bosons can be distinguished much more accurately by feeding the jet charge, alongside the jet's image, into deep neural networks than by traditional cuts or boosted decision trees. It constructs binary taggers for W+ versus W− and Z versus W, plus a single three-way classifier, and reports roughly 30–40% higher background rejection at fixed signal efficiency from the deep networks. The largest gain is in Z versus W separation, where the composite CNN2 reaches a background rejection of R50 = 52.60, compared with 14.16 for the best single-kappa BDT and 40.42 for a plain CNN. If the simulation-based ranking survives real collider data, charge-sensitive weak-boson identification becomes practical for asymmetry measurements and searches for new charged particles.","feed_headline":"Jet charge plus deep learning improves W/Z tagging by 30-40%","feed_subtitle":"Composite CNN hits R50 = 52.6 for Z vs W, up from 14.2 for a BDT.","key_machinery":"The engine is the jet charge $$Q_\\kappa = \\frac{1}{p_{T,J}^\\kappa}\\sum_{i\\in J} q_i (p_{iT})^\\kappa$$, a pT-weighted signed sum of constituent charges, rendered as a second image channel alongside per-pixel pT. On top of it sits CNN2, a composite convolutional network with two parallel branches: a deep stack for the pT image and a shallow stack for the charge image, merged before the final dense layers. The depth asymmetry lets the network exploit fine spatial substructure to identify Z bosons while avoiding overfitting on the charge-only W+ versus W− distinction.","core_discovery":"The paper's central claim is that combining jet charge with convolutional networks trained on collider images substantially improves W+/W−/Z classification. Adding a jet-charge channel lifts the plain CNN's background rejection at 50% signal efficiency from about 14 to 40 for Z versus W, and the composite CNN2, which gives the pT channel a deeper stack of convolutional layers than the charge channel, reaches 52.60. The paper further claims that a single ternary classifier, when projected onto any pair of classes, recovers or exceeds the performance of dedicated binary taggers, so one network can serve several physics tasks.","pith_inferences":["A natural extension is a network fed raw per-track charges instead of a fixed-kappa weighted sum, which could remove the kappa tuning the paper required and further improve W+/W− separation.","The same two-channel architecture should transfer to other charged resonances, such as distinguishing W± from H±, whenever the charge asymmetry of the decay products is informative.","The reported gains would directly strengthen charge-asymmetry measurements and searches for doubly charged scalars, although the paper does not quantify the projected physics reach.","Because the taggers are trained on leading-order simulated samples, deployment at the LHC would need a check of stability against higher-order corrections and pileup conditions."],"forward_implications":["A single three-way tagger can replace separate binary taggers: its pairwise projections match or beat the dedicated binary classifiers.","Adding jet charge as an input channel raises background rejection by roughly 30–40% at fixed signal efficiency relative to cut-based and BDT baselines.","The composite CNN2, with unequal depth per channel, is the strongest Z versus W tagger in the paper, with R50 = 52.60 compared with 40.42 for the plain CNN and 14.16 for the single-kappa BDT.","The ordering of taggers persists, with some degradation, when an alternative parton-shower model is used for hadronization, indicating the result is not tied to one shower generator.","Charge-sensitive W+/W− identification opens the door to LHC measurements of charge asymmetries and same-sign diboson production."],"supporting_citations":[{"why":"Introduces the jet charge observable Q_kappa that the taggers are built on.","marker":"[25]"},{"why":"Derives the theoretical behavior of jet charge at the LHC, motivating its use for charge discrimination.","marker":"[26]"},{"why":"Constructs an earlier likelihood tagger using mass and jet charge for Z versus W, the baseline this paper claims to surpass.","marker":"[29]"},{"why":"Demonstrates that combining jet charge with machine learning improves quark/gluon tagging, supplying the direct methodological comparison point.","marker":"[31]"},{"why":"Describes the jet-image preprocessing steps (centralization, rotation, flipping) that the CNN inputs depend on.","marker":"[16]"},{"why":"Provides the showering and hadronization model used to produce the training and test jets.","marker":"[34]"},{"why":"Defines the anti-kt jet clustering algorithm used to reconstruct the jets with radius R=0.7.","marker":"[37]"}],"fun_headline_variants":["Composite CNN with jet charge lifts W/Z tagging 3x","Jet charge channel boosts QCD rejection in W/Z tags","Deep learning on jet charge triples W/Z discrimination","Ternary CNN matches dedicated binary W/Z taggers"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported gains depend on the Monte Carlo simulation reproducing the real joint distribution of jet charge and jet substructure at the LHC closely enough that the tagger rankings and R50 values survive on collision data; the paper validates only against an alternative shower model, not against observed events.","fun_headline_variants_meta":{"raw":{"variants":["Composite CNN with jet charge lifts W/Z tagging 3x","Jet charge channel boosts QCD rejection in W/Z tags","Deep learning on jet charge triples W/Z discrimination","Ternary CNN matches dedicated binary W/Z taggers"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000229,"raw_usage":{"total_tokens":1416,"prompt_tokens":822,"completion_tokens":594,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":438,"completion_tokens_details":{"reasoning_tokens":529}},"tokens_in":438,"tokens_out":594,"duration_ms":5507,"temperature":1.0,"reasoning_tokens":529,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:44:56.804674+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the same taggers on jets tagged from LHC collision data through semileptonic decays and compare R50 for Z versus W; if the CNN2 advantage over a boosted decision tree shrinks toward zero, or the charge channel stops helping, the central claim would be falsified. An alternative shower model alone cannot settle this because it shares the same hard-process and detector model.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the jet charge observable Q_kappa that the taggers are built on."}],"review_version":1}