{"id":"1cdcacca-b651-4d10-9920-d75b375d7a4f","arxiv_id":"2509.20116","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"K4GeneratorsConfig automates the configuration and benchmarking of e+e- Monte Carlo generators, showing sub-permil agreement among three generators at LO.","lead":"This paper presents K4GeneratorsConfig, a Python package that creates configuration files for several Monte Carlo event generators from one physics input, automating cross-generator benchmarking for future electron-positron colliders. It matters because next-generation Higgs factories need reproducible and consistent simulation tools to control systematic uncertainties.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Universal-input translation fails for PYTHIA at leading order: the EW scheme mismatch forces its exclusion from the Section 3.3 comparison, so the abstract's 'ensuring consistency' overstates the tool's guarantee.","rationale":"I read the paper as a software tool paper whose central claim is methodological: that a universal, generator-independent physics input can be mechanically translated into consistent configurations for multiple MC generators, thereby enabling reliable benchmarking. The strongest evidence is the sub-permil LO agreement among Sherpa, Whizard, and Madgraph in Section 3.3. The load-bearing condition is that the translation layer is faithful for every supported generator; if one generator's output is not comparable due to a scheme mismatch, the claimed consistency is only partial. The paper itself provides the counterexample: PYTHIA is excluded from the LO comparison because the GF-mW-mZ to effective sin^2(theta) translation leads to a percent-level discrepancy. This is not a mere implementation detail; it is a physics-level mismatch between input schemes that the tool does not fully reconcile. The tool's EWParamDevThreshold check only validates the internal LO consistency of the parameter set, not the cross-generator comparability of derived parameters like the effective weak mixing angle. The paper honestly acknowledges the PYTHIA limitation in Section 3.3, but the abstract and conclusions present the tool as ensuring consistency without this caveat. The concern is therefore load-bearing: it limits the central claim to a subset of generators and input schemes. I agree with the reader's verdict of CONDITIONAL. The proposed concrete test would settle whether the PYTHIA discrepancy is an inherent output of the tool's default card generation or an artifact of a particular parameter choice; if the former, the abstract should be revised to state that consistency is scheme-dependent. I do not see a stronger concern: the code is public, CI is mentioned, and the benchmark is reproducible in principle via Key4hep version pinning, though the paper would benefit from reporting exact versions and commit hashes.","tokens_in":12323,"tokens_out":7903,"duration_ms":57149,"concrete_test":"Use K4GeneratorsConfig to produce the PYTHIA datacard for e+e- -> mu+mu- at sqrt(s)=250 GeV from the example YAML (with default ParameterSets). Inspect the generated PYTHIA runcard and record the electroweak mixing angle value written (tree-level ~0.22277 vs effective ~0.23155). Run the generator with that card and compare the resulting LO cross section to the Sherpa/Whizard/Madgraph average. If the cross section deviates by more than 0.1%, the tool's default output reproduces the inconsistency and the abstract's 'ensuring consistency' claim is not supported for PYTHIA. If it agrees, then the Section 3.3 discrepancy is due to a non-default user override, and the central claim is not directly affected.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the tool translates universal physics inputs into generator-specific configurations, ensuring consistency, reproducibility, and reduced human error. Section 3.3 demonstrates sub-permil LO agreement among Sherpa, Whizard, and Madgraph, but PYTHIA is explicitly excluded because 'a true leading order comparison cannot include PYTHIA.' The stated reason is an electroweak input scheme mismatch: the tool uses the GF-mW-mZ scheme, while PYTHIA expects the effective sin^2(theta), and the translated value leads to a percent-level deviation. This directly undercuts the abstract's claim of ensuring consistency. Section 2.2 describes extending the parameter set with sin2theta and sin2thetaEff, but that does not make the universal input faithful: the same YAML input produces a PYTHIA configuration whose cross section deviates at the percent level. The tool's internal consistency check (EWParamDevThreshold) verifies the LO relation among GF, mW, mZ, sin^2(theta)_LO, alpha_EM_LO, and VEV; it does not verify that the effective sin^2(theta) used by PYTHIA is consistent with the GF-mW-mZ scheme. Therefore a user relying on the advertised 'universal physics inputs' could unknowingly generate a PYTHIA configuration that is not comparable at LO to the other generators. This is not a fatal flaw in the software, but it is a load-bearing gap in the central claim: the claimed consistency is only demonstrated for generators that accept the same EW input scheme, and the paper does not qualify this in the abstract or conclusions.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents K4GeneratorsConfig, a Python package in the Key4hep ecosystem that accepts a single YAML file containing 'universal physics inputs' (processes, center-of-mass energies, cuts, model parameters) and automatically produces run cards, execution scripts, and output conversion for several e+e- Monte Carlo generators (Sherpa, Whizard, Madgraph, Pythia, KKMCee, Babayaga). The package also provides interfaces to Rivet and Key4hep-based analyses for cross-section and differential-distribution comparisons. Benchmarking results are reported: at leading order, Sherpa, Whizard, and Madgraph agree at the sub-permil level for e+e- -> mu+mu- and e+e- -> ZH, with differential distributions consistent within statistical errors. PYTHIA is excluded from the LO comparison because the translation of the electroweak input scheme leads to a percent-level discrepancy. The authors also report several generator bugs identified using the tool.","tokens_in":12689,"tokens_out":6023,"duration_ms":45309,"significance":"If the claims hold, the package addresses a genuine and timely need for reproducible, low-effort benchmarking of MC generators in the FCC-ee/CEPC/ILC era. The tool's modular architecture and CI integration are sensible, and the reported sub-permil agreement among three independent generators is a useful, falsifiable check. The authors are honest about the PYTHIA limitation and about untested generator-specific overrides. However, the central claim of the abstract -- that the tool 'ensures consistency, reproducibility, and reduced human error' -- is stronger than what the evidence supports, because the one generator (PYTHIA) that does not accept the GF-mW-mZ scheme is not consistently translated and is excluded from the main comparison. The paper would be significantly improved by qualifying the claim and by adding a runtime warning or validation for the PYTHIA effective-angle input.","major_comments":[{"comment":"The abstract claims that the tool translates universal physics inputs into generator-specific configurations, 'ensuring consistency, reproducibility, and reduced human error.' Yet the only demonstrated consistency is among Sherpa, Whizard, and Madgraph, which accept the same GF-mW-mZ electroweak scheme. PYTHIA is a supported generator but is excluded from the LO comparison because the translated EW input leads to a percent-level deviation. The internal consistency check using EWParamDevThreshold validates tree-level relations among GF, mW, mZ, sin^2theta_LO, alpha_EM_LO, and VEV; it does not validate the effective sin^2theta used by PYTHIA. Consequently, a user who lists PYTHIA together with Sherpa in one YAML input can unknowingly generate an inconsistent configuration. This undercuts the advertised guarantee. The paper should either restrict the consistency claim to generators whose EW","section":"Sec. 3.3 and Fig. 3; Sec. 2.2"},{"comment":"The 'universal' input is not fully universal: the generator-specific override block (Generator:Key:Value) takes precedence over the main YAML keys, and the paper itself warns that this 'could potentially lead to incorrect set-ups for keys that have not been tested.' This is a caveat that directly affects the 'reduced human error' claim, because a user can silently override a physics parameter with an unvalidated value. The authors should mark generator-specific overrides as 'expert-only' in the documentation and ensure that any key used in an override is included in the CI validation. As written, the tool provides a mechanism for consistency but does not itself guarantee it when these overrides are used.","section":"Sec. 2.1 and Sec. 2.4; footnote 1"},{"comment":"The benchmarking evidence is limited to LO cross-sections for two processes (e+e- -> mu+mu- and e+e- -> ZH) and one differential distribution. The NLO flag is listed in Table 1 but no NLO benchmark is shown. In the differential comparison, the chi2/dof values are described as 'similar and below unity' without giving the actual numbers. For a paper with 'Benchmarking' in its title, this is a rather thin validation. The authors should either expand the benchmark suite (e.g., at least one NLO example, a few more final states) or explicitly state that the results are proof-of-concept demonstrations of the tool's mechanics rather than a comprehensive benchmarking study.","section":"Sec. 3.3 and Table 1"}],"minor_comments":[{"comment":"Grammar: 'simulating physical processes' should be 'simulate physical processes' to match the parallel structure.","section":"Abstract"},{"comment":"Typo: 'provide a valuable diagnostic insights' should be 'provide valuable diagnostic insights'.","section":"Sec. 1"},{"comment":"Typo: 'polariazation' should be 'polarization'.","section":"Sec. 2.1"},{"comment":"The naming of alphaEMMZM1 and alphaEMMZ is confusing: both rows show '1/alphaEM(mZ)' in the default-value column, but one should represent alpha_EM(mZ) and the other its inverse. Please clarify the notation.","section":"Sec. 2.2, Table 2"},{"comment":"The sentence 'Translation of the EW scheme to the PYTHIA input leads to a rejection of the calculated input which is required to be higher as PYTHIA expects a sin^2 theta including radiative corrections' is grammatically garbled and hard to parse. Rewrite for clarity, since this sentence is the entire explanation for the PYTHIA exclusion.","section":"Sec. 3.3"},{"comment":"Please provide the actual chi2/dof values for the polar-angle comparison rather than stating only that they are 'below unity'.","section":"Sec. 3.3"},{"comment":"Typo: 'a robust d automated framework' should be 'a robust and automated framework'.","section":"Conclusions"},{"comment":"The name 'Key4hep' is spelled inconsistently (Key4Hep, Key4HEP, Key4hep). Please standardize.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is honest in the body about the PYTHIA limitation and the untested overrides, but the abstract overstates the guarantee. A revision that qualifies the central claim and adds a runtime safeguard for the PYTHIA effective-angle issue would make the paper acceptable. I would also suggest asking the authors to provide a release DOI or a specific version tag for the repository so that the benchmark results are reproducible from a fixed codebase."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis is a straightforward software/infrastructure paper, and a decent one. K4GeneratorsConfig takes a single YAML description of a process and generates runcards for Sherpa, Whizard, Madgraph, Pythia, KKMCee, and Babayaga inside the Key4hep stack, plus scripts for event generation and rudimentary differential analysis. That integration is genuinely useful: it is the kind of glue work that has to exist for FCC/CEPC/ILC benchmarking to be reproducible, and as far as I can tell it didn't exist before.\n\nThe paper earns its keep in two ways. First, it ships code: Apache-2.0, GitHub, CI tests with actual Key4hep builds, and a documented list of issues found. Second, the benchmark section shows what the tool is for: Sherpa, Whizard, and Madgraph agree at LO to better than a per-mille, and the tool has already caught real generator bugs. That is reproducible evidence, not a promise.\n\nThe soft spot, as you'll have seen, is PYTHIA. The same 'universal' input produces a Pythia configuration that is off by a percent at LO, which the paper traces to an effective sin^2(theta) input scheme mismatch and then excludes Pythia from the LO comparison. The stress-test note is right that the abstract's 'ensuring consistency' is too strong: the consistency is only as good as the electroweak scheme translation, and that translation is demonstrably not faithful for Pythia. But the paper does not hide this. Section 3.3 states it plainly, and Section 2.2 explains the scheme decision. I'd call it a caveat that should be qualified in the abstract and conclusions, not a fatal flaw. The consistency check tool even has a warning threshold, and the footnote about untested generator-specific keys shows the authors know the limits.\n\nOther minor things: no commit hash or exact generator versions in the text, which would help reproducibility; the benchmark is limited to two process classes, though that's acceptable for an alpha release. I would not call this a physics breakthrough, but it doesn't pretend to be one. It's infrastructure, and it's the kind of infrastructure that needs refereeing to keep it honest.\n\nI'd send it to peer review. A competent referee can verify the claims in an afternoon, and the tool deserves to be part of the Key4hep record. I wouldn't cite it in my own phenomenology papers, but I'd definitely use it if I were doing generator validation for a future collider.","headline":"A solid, honest software paper: K4GeneratorsConfig is a genuinely useful YAML-to-datacard tool for e+e- generator benchmarking in Key4hep, with a real caveat about PYTHIA's EW scheme that the paper itself already flags.","tokens_in":13123,"tokens_out":3659,"would_cite":false,"duration_ms":47007,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"One universal physics input, auto-translated per generator, yields sub-permil leading-order agreement between Sherpa, Whizard, and Madgraph for e+e− collisions.","keywords":["Monte Carlo event generators","e+e− collisions","generator benchmarking","Key4hep","YAML configuration","electroweak input scheme","leading-order cross sections","Higgs factories"],"falsifier":"Run a single YAML input for the e+e− → μ+μ− process with Pythia configured using sin²θEff that includes radiative corrections (rather than the LO value derived from GF–mW–mZ), and check whether the percent-level offset in the ZH cross-section scan disappears. If it does, the translation layer is faithful and the framework delivers the claimed consistency; if the offset persists, or reappears for other process/scheme combinations, the central claim of faithful universal translation is refuted.","tokens_in":12250,"feed_emoji":"⚛️","tokens_out":9967,"duration_ms":72554,"temperature":0.7,"pith_summary":"Future e+e− Higgs and electroweak factories will need physics samples from several independent Monte Carlo generators, compared against each other to expose bugs and to bound the systematic error of the event-generation step. Each generator has its own input syntax, process keywords, and parameter conventions, so writing runcards by hand is slow and error-prone. This paper argues the whole task can be automated: K4GeneratorsConfig takes a single YAML file describing the physics—processes, energies, cuts, particle data, and a coherent electroweak parameter set—and generates the datacards, run scripts, and analysis hooks for Sherpa, Whizard, Madgraph, KKMCee, Pythia, and Babayaga inside the Key4hep stack. The payoff is demonstrated in leading-order comparisons where Sherpa, Whizard, and Madgraph agree to better than one permil (0.1%) on e+e− → ZH and difermion cross sections, while Pythia is set aside because its effective weak-mixing-angle convention does not line up with the common input scheme. The same automated comparisons, the paper reports, have already caught real generator defects: missing cross-section output, a rounding-driven event rejection, and ISR/boost inconsistencies.","feed_headline":"Sub-permil generator agreement from a single physics input","feed_subtitle":"A single YAML description generates runcards for six event generators and surfaces their real bugs.","key_machinery":"Central is the translation layer: a generator-independent YAML input (processes, √s, events, seed, selectors, polarization, beamstrahlung) that dynamically loaded per-generator Python modules convert into native datacards, with an inheritance scheme so new generators are added by writing one module. Two mechanisms carry the argument: the ParameterSets module, which fixes a coherent electroweak scheme in GF–mW–mZ, derives αEM, sin²θLO, the VEV, and Yukawa couplings, and warns if leading-order relations deviate by more than EWParamDevThreshold (default 10⁻³); and the per-generator process databases (GeneratorProcDB) that supply each generator's idiosyncratic process keywords, with precedence r","core_discovery":"One YAML physics input, translated automatically, can make independent e+e− generators agree at the sub-permil level at leading order and expose their internal bugs. The paper's demonstration: for ZH and difermion production built from a common parameter set anchored on GF, mW, and mZ, Sherpa, Whizard, and Madgraph agree to better than one permil (Whizard and Madgraph to 0.1 permil), with differential distributions compatible within statistics; Pythia is set aside because it demands an effective mixing angle rather than the LO value. The same workflow, the paper reports, found real generator defects—missing cross-section output, rounding-driven event rejection, ISR/boost inconsistencies—supp","pith_inferences":["The demonstrated sub-permil baseline gives future Higgs-factory analyses a concrete number: cross-generator spread at LO could be packaged as a quoted systematic uncertainty of the event-generation step, something the paper motivates but does not formalize.","The same harness could be pointed at processes where generators are not expected to agree, such as Bhabha scattering for luminosity or four-fermion backgrounds, turning the tool from a technical-consistency checker into a physics-validation probe; the NLO flag in the schema points in this direction.","The Pythia/effective-angle episode generalizes: every generator's parameter convention (on-shell versus effective, GF-based versus α(mZ)-based) is a latent failure mode of any translation layer; an automated on-the-fly scheme translation with radiative corrections, which the paper mentions but does not implement, would remove the remaining user burden."],"forward_implications":["Sub-permil leading-order agreement among Sherpa, Whizard, and Madgraph can serve as a regression benchmark in continuous integration, so generator updates that break technical consistency are caught automatically.","The bugs the workflow uncovered—missing cross-section metadata, rounding-induced event rejection, ISR/boost inconsistencies—show that automated inter-generator comparison works as a diagnostic complement to generators' own validation.","Version pinning by Key4hep release date makes event samples reproducible: the same YAML input regenerates the same physics with the exact same software stack later.","Pythia's percent-level offset is traced to the electroweak input convention, meaning Pythia requires sin²θEff including radiative corrections rather than the LO-derived value; extending the parameter set, not changing the physics, is the documented path to include it."],"fun_headline_variants":["One YAML physics input drives sub-permil generator agreement","Automated config tool exposes bugs, achieves sub-permil agreement","Single physics input, six generators, sub-permil parity","K4GeneratorsConfig: reproducible benchmarks, hidden bugs revealed","Sub-permil e+e− generator parity from one config file"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that one universal physics input maps faithfully onto every generator's parameter and syntax conventions—the paper itself shows Pythia's effective sin²θ convention breaks that mapping at the percent level, and it explicitly warns that user-supplied generator-specific override settings are potentially untested.","fun_headline_variants_meta":{"raw":{"variants":["One YAML physics input drives sub-permil generator agreement","Automated config tool exposes bugs, achieves sub-permil agreement","Single physics input, six generators, sub-permil parity","K4GeneratorsConfig: reproducible benchmarks, hidden bugs revealed","Sub-permil e+e− generator parity from one config file"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00021,"raw_usage":{"total_tokens":1210,"prompt_tokens":670,"completion_tokens":540,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":414,"completion_tokens_details":{"reasoning_tokens":456}},"tokens_in":414,"tokens_out":540,"duration_ms":4523,"temperature":1.0,"reasoning_tokens":456,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T15:12:19.875669+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a single YAML input for the e+e− → μ+μ− process with Pythia configured using sin²θEff that includes radiative corrections (rather than the LO value derived from GF–mW–mZ), and check whether the percent-level offset in the ZH cross-section scan disappears. If it does, the translation layer is faithful and the framework delivers the claimed consistency; if the offset persists, or reappears for other process/scheme combinations, the central claim of faithful universal translation is refuted.","supporting_citations":[],"review_version":1}