{"id":"da346e0a-d26e-4323-ad1a-3f0c93c026b9","arxiv_id":"2506.06838","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A new open-source GPU code, AthenaK, reproduces the GW150914 merger: remnant mass within 0.01%, spin within 0.02%, and waveform phase within about 0.35 radians of established simulations.","lead":"Researchers simulated the 2015 black-hole merger GW150914 with AthenaK, a new open-source, GPU-accelerated numerical relativity code. The computed remnant mass, spin, recoil, and gravitational waveform match the established SpEC and LazEV simulations closely, and a LIGO re-analysis using the new waveform recovers the event's known chirp mass and distance.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The N=128 run used for the quoted agreement numbers is not demonstrably converged: Appendix A shows a 0.38 rad (2,2) phase shift between N=128 and N=192, the same size as the claimed AthenaK-SXS dephasing, so the headline accuracy figures lack a resolution error bar.","rationale":"The paper is a valuable cross-validation: external independent comparisons, a public tutorial, and a self-contained higher-resolution appendix are real positive evidence. The reader's conditional verdict is appropriate. The single most load-bearing vulnerability is not the comparison itself but the resolution status of the run from which the headline numbers are taken. Appendix A provides the internal control: at N=192, the (2,2) phase at merger differs from N=128 by 0.38 rad, essentially equal to the quoted AthenaK-SXS dephasing of 0.35 rad. Since the N=192 run moves closer to SXS/RIT (0.12/0.17 rad), the N=128 values are not converged; they carry a discretization uncertainty that is not quoted. The same is true for remnant properties, which are only reported for N=128. This does not refute the central claim—the higher-resolution run suggests AthenaK works—but it means the paper's headline precision is not established. The contradiction in Sec. 4 should be fixed; more importantly, the quoted numbers should either come from a convergence-extrapolated result or carry resolution error bars. The concrete check is feasible because the N=192 run and pipeline are already in hand.","tokens_in":14101,"tokens_out":4664,"duration_ms":46724,"concrete_test":"Re-run the Sec. 3.2 comparison and the isolated-horizon remnant measurement with the N=192 simulation already described in Appendix A, using the same alignment window and same post-processing pipeline. Report Delta_phi, Delta_A/A, remnant mass, spin, and recoil relative to SXS:BBH:0305 (and RIT:BBH:0062), with a Richardson extrapolation from N=128 and N=192 (ideally plus N=256). If the N=192 values differ from the N=128 values by more than the quoted agreement (e.g., phase shifts by >0.12 rad or remnant spin by >2e-4), the published accuracy figures must be replaced by resolution-corrected values with error bars; if the shift is smaller than the quoted tolerances, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (abstract, Sec. 3.2) is that AthenaK reproduces the established GW150914 results—remnant mass and spin agree with the highest-resolution SpEC run to 9e-5 and 2e-4, and the (2,2) mode shows at most 0.35 rad dephasing and 0.4% amplitude difference against SXS:BBH:0305 and RIT:BBH:0062. For that claim to be a statement about AthenaK's accuracy, the run it is measured on must be converged to better than those tolerances. The paper's own Appendix A shows this is not established: the (2,2) phase at merger changes by 0.38 rad between the N=128 baseline and the N=192 higher-resolution run, with amplitude changing by 0.2%. These are the same size as, or larger than, the quoted cross-code differences (0.35 rad phase; 0.2-0.4% amplitude). Moreover, the N=192 run agrees with SXS to 0.12 rad and 0.01% amplitude, so the N=128 agreement numbers are not a converged measurement of AthenaK accuracy but a mixture of physics and discretization error. Section 4 states 'we have only considered a single grid resolution,' contradicting Appendix A. Because the quoted remnant mass/spin (Sec. 3.1) are also taken from the N=128 run and no N=192 remnant values are given, the same concern applies to the 9e-5/2e-4 figures.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents targeted numerical-relativity simulations of the GW150914 binary black hole merger performed with the GPU-accelerated code AthenaK, using puncture initial data inherited from the LazEV setup of Lovelace et al. (2016) and waveforms extracted at future null infinity through Cauchy-characteristic extraction with SpECTRE. The isolated-horizon measurements give remnant mass M = 0.951948 and spin chi = 0.691914, quoted as agreeing with the highest-resolution SpEC results to relative differences of 9e-5 and 2e-4, and a recoil velocity within 3% of SpEC. The dominant (2,2) mode is compared with SXS:BBH:0305 and RIT:BBH:0062 after time/phase alignment, with claimed maximum dephasing of about 0.35 rad and amplitude difference of about 0.4%. The waveform is then used in a bilby reanalysis of the GW150914 data, yielding chirp mass, luminosity distance, and inclination posteriors broadly consistent with IMRPhenomXPHM. The central claim is that these results validate AthenaK and its CCE pipeline as a state-of-the-art, open-source, GPU-native capability for many-orbit binary black hole simulations.","tokens_in":14306,"tokens_out":17292,"duration_ms":145736,"significance":"If the accuracy claims hold, this is a valuable code-validation and application paper: it benchmarks a new open-source, GPU-native numerical-relativity code against two independent codes (SpEC and LazEV) and against real LIGO data, and it demonstrates a complete pipeline from puncture initial data through CCE to parameter estimation, with a public step-by-step tutorial and reproduction scripts that are a genuine community asset. The inclusion of memory effects in the CCE modes and the very tight remnant mass and spin agreement are additional strengths. The significance of the waveform-accuracy claims, however, is currently limited by the resolution issue described in the major comments: the headline dephasing and amplitude figures are comparable in size to the resolution error, so they do not yet support the 'excellent agreement' statement at the quoted precision. Restating the analysis around the higher-resolution run, or with explicit error bars, would make the paper's conclusions well supported.","major_comments":[{"comment":"Section 4 states that 'the main limitation of this work is that we have only considered a single grid resolution with AthenaK,' but Appendix A presents a complete second simulation with 192 points in the coarsest refinement level and uses it to validate the N=128 baseline (Fig. A1). This is a direct internal contradiction on a load-bearing point: the reader cannot tell whether the quoted results are meant to come from a single-resolution study or from the two-resolution set described in the appendix. Please correct Section 4 (for example, by describing the N=192 run as a convergence check performed after the main analysis) and state explicitly which run underlies each quoted number.","section":"Sec. 4 and Appendix A"},{"comment":"The headline agreement figures — maximum dephasing Delta phi ~ 0.35 rad and amplitude difference Delta A/A ~ 0.4% — are quoted from the N=128 baseline run in the abstract and Sec. 3.2 (bottom panel of Fig. 4) without a resolution error bar. Appendix A reports that the (2,2) phase at merger changes by 0.38 rad and the relative amplitude by 0.2% between N=128 and N=192, so the resolution effect is equal to or larger than the quoted AthenaK-SXS dephasing, and the headline numbers do not cleanly characterize AthenaK's physical accuracy. The same appendix shows that the N=192 run agrees with SXS to 0.12 rad and 0.01% amplitude, so the paper's own higher-resolution data actually support a stronger statement than the abstract makes. The abstract, Sec. 3.2, and Fig. 4 should be revised to quote the converged (N=192-based) values or to attach explicit resolution error bars; the Sec. 3.3 parameter-estimation results, which use the N=128 waveform, inherit the same caveat and should be qualified accordingly.","section":"Abstract; Sec. 3.2, Fig. 4"},{"comment":"The remnant mass and spin, M = 0.951948 and chi = 0.691914, together with the claimed relative differences of 9e-5 and 2e-4 from the highest-resolution SpEC results, are taken from the N=128 run, and no N=192 remnant values are reported anywhere in the paper. There is therefore no resolution estimate for these quantities, and the quoted precision is unsupported even though the qualitative agreement is probably robust. Please report the N=192 remnant mass and spin (or a Richardson-extrapolated estimate) and quote the resolution uncertainty alongside the SpEC comparison.","section":"Sec. 3.1"}],"minor_comments":[{"comment":"There is a typo: 'broadely consistent' should read 'broadly consistent.'","section":"Abstract"},{"comment":"The sentence 'The parameters from the initial data are taken from Ref. [80] to match that used for the LazEV code' should read '...to match those used for the LazEV code.' In addition, the mass ratio and dimensionless spin magnitudes of the initial black holes are never stated; given the paper's reproducibility emphasis and the role these parameters play in the Sec. 3.3 analysis, they should be listed explicitly.","section":"Sec. 2"},{"comment":"The alignment window [t_i, t_f] used in Eq. (1) and indicated by the dotted lines in Fig. 4 is not specified numerically; stating the window would make the comparison reproducible.","section":"Eq. (1), Sec. 3.2"},{"comment":"The table caption contains a typo: 'extrisinc' should read 'extrinsic.'","section":"Table 1"},{"comment":"The phrase 'around the time GPS of event t0' should read 'around the GPS time of the event t0.'","section":"Sec. 3.3"},{"comment":"The caption 'Real (or imaginary) parts of the waveform multipoles' is ambiguous; please specify which part is plotted for each mode.","section":"Fig. A1"},{"comment":"Because the initial data are inherited from Ref. [80], the comparisons with SpEC and LazEV largely validate the evolution and extraction pipeline rather than the choice of initial parameters; drawing this implication explicitly in Sec. 3 would help readers interpret the agreement.","section":"Sec. 2 and Sec. 3.2"}],"recommendation":"major_revision","confidential_remarks":"This is a code-validation and application paper that fits the journal's scope. The main issue is internal to the manuscript: the appendix's N=192 run contradicts the 'single resolution' statement in the conclusions, and the headline waveform numbers are not the converged ones. I want to emphasize that fixing this would strengthen the paper, since the N=192 comparison is more favorable to AthenaK than the quoted N=128 comparison. I have no concerns about citation practices or novelty disclosure."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take on the AthenaK GW150914 paper. The genuinely new thing is that a new, open-source, GPU-native NR code has produced a targeted BBH simulation through merger, extracted waveforms at null infinity via CCE including memory, and released a tutorial and scripts that reproduce the run. That is real and useful. The cross-code comparison targets are independent — SpEC and LazEV results from Ref. [80] — and the remnant mass/spin agreement at 9e-5/2e-4 and recoil within 3% are credible in direction. The bilby PE exercise is a nice application, and the posteriors are consistent with established results.\n\nNow the soft spots. The quantitative waveform claims — 0.35 rad dephasing, 0.4% amplitude — are quoted from the N=128 baseline run, and the paper does not give a resolution error bar on those figures. Appendix A shows the (2,2) phase changes by 0.38 rad between N=128 and N=192, essentially the same size as the AthenaK-SXS dephasing. The N=192 run actually agrees with SXS to 0.12 rad, which is better, so the direction is fine — but the stated numbers are a mixture of physics and truncation error, not a clean accuracy statement. Worse, the conclusions say 'we have only considered a single grid resolution,' which directly contradicts the appendix. That needs fixing, and the higher-res results should either be promoted to the headline or folded into a proper error bar. Also, nowhere in the paper do they state the mass ratio and spins actually used in the simulation; Table 1 labels M as total mass while the text and Fig. 6 treat it as chirp mass (the 30.7 solar masses has to be chirp mass). The comparison also never quantifies the eccentricity mismatch between runs, though the estimated eccentricity of 8e-4 is small. These are minor but real clarity problems.\n\nOverall, the central validation is very likely correct; the higher-resolution run confirms the trend, and the code is worth the community's attention. But as written, the precision claims outrun the evidence on resolution. This deserves serious peer review and a request to harmonize the numbers with the appendix. I'd read it again after revision.","headline":"First AthenaK GW150914 run with CCE and memory is a real capability demonstration, but the headline agreement numbers come from a run whose own convergence test differs at the same size as the quoted dephasing.","tokens_in":15084,"tokens_out":2753,"would_cite":false,"duration_ms":30050,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["83C57","83C35","83-08"],"pacs":["04.25.D-","04.30.-w","04.70.Bw"],"model":"deepseek-v4-flash","headline":"A GPU-accelerated numerical relativity code reproduces the GW150914 black-hole merger waveform to within 0.35 radians of phase and 0.4% in amplitude, and recovers consistent source parameters from the observed signal.","keywords":["numerical relativity","binary black hole","GW150914","Cauchy characteristic extraction","AthenaK","gravitational waves","parameter estimation","GPU-accelerated simulation"],"falsifier":"Run the same GW150914 setup at $N=192$ with otherwise identical parameters, align the $(2,2)$ modes against the reference waveforms as in Sec. 3.2, and compare the merger-phase difference to the $N=128$ value. If the higher-resolution run changes the AthenaK-minus-reference dephasing by more than about $0.2$ rad, or if the amplitude difference moves outside the $0.2\\%$--$0.4\\%$ band, the headline agreement is resolution-limited rather than a measure of AthenaK's accuracy.","tokens_in":13678,"feed_emoji":"🕳️","tokens_out":11425,"duration_ms":95947,"temperature":0.7,"pith_summary":"This paper reports a new numerical-relativity simulation of the binary black hole merger that produced the gravitational-wave event GW150914, run with the GPU-accelerated open-source code AthenaK. The authors claim that AthenaK reproduces the results of established reference codes: the remnant mass and spin agree with the highest-resolution reference run to relative differences of $9\\times10^{-5}$ and $2\\times10^{-4}$, the recoil velocity is within 3%, and the dominant $(2,2)$ waveform mode shows at most about 0.35 radians of phase difference and 0.4% amplitude difference. They also re-analyze the gravitational-wave data with their new waveform and recover chirp mass, luminosity distance, and inclination consistent with semi-analytic models. If correct, these results validate AthenaK as an open, GPU-native capability for many-orbit binary black hole simulations and for producing waveforms at future null infinity via Cauchy characteristic extraction.","feed_headline":"GPU code reruns GW150914, matches merger to 0.4%","feed_subtitle":"Remnant mass, spin, and recoil agree with reference simulations, and the new waveform recovers GW150914's source parameters.","key_machinery":"The load-bearing machinery is the Cauchy-characteristic extraction (CCE) pipeline that converts AthenaK's near-zone metric data on a world tube at $49M \\le r \\le 51M$ into gravitational-wave strain at future null infinity $\\mathscr{I}^+$, using a null evolution code [104]; this yields waveforms that include memory effects, which are visible in the $m=0$ multipoles. Remnant mass and spin come from the isolated horizon formalism applied to horizon data recorded in Cartesian boxes around the punctures. The comparison against reference waveforms minimizes a $\\chi^2$ time-and-phase alignment over an inspiral interval, and the underlying evolution uses the Z4c formulation of Einstein's equations with sixth-order finite differencing, adaptive mesh refinement down to $\\Delta = 0.0078125\\,M$, and a fourth-order Runge-Kutta integrator.","core_discovery":"The central claim is that a single AthenaK run, at a base resolution of 128 points per direction with 12 adaptive mesh refinement levels, reproduces the GW150914 remnant and waveform of the reference simulations closely enough that the code can be trusted for this class of problems. Concretely, the remnant mass is $M=0.951948$ and the dimensionless spin is $\\chi=0.691914$ in units where the initial binary mass is 1, matching the highest-resolution reference calculation to $9\\times10^{-5}$ and $2\\times10^{-4}$; the recoil velocity is $138.68$ km/s, within 3% of the reference; and the $(2,2)$ mode shows dephasing of $\\Delta\\phi\\simeq0.35$ with relative amplitude difference $\\Delta A/A\\simeq0.4\\%$ at merger. The paper further claims that Bayesian re-analysis of GW150914 data with this waveform yields chirp mass $30.7^{+0.6}_{-0.5}\\,M_\\odot$, luminosity distance $460^{+140}_{-140}$ Mpc, and inclination $2.7^{+0.3}_{-0.4}$ rad, consistent within 90% credibility with the semi-analytic model analysis used for comparison.","pith_inferences":["The paper's appendix gives a partial resolution check: at $N=192$ the $(2,2)$ phase difference against the reference simulation drops from $0.35$ to $0.12$ rad, suggesting the headline $N=128$ numbers are conservative and that a formal convergence study would show AthenaK's intrinsic accuracy is better than the abstract states.","The memory and spin-memory features seen in the $m=0$ modes are not used in the parameter estimation; a natural next step is to include those modes in the likelihood and test whether current or next-generation detectors are sensitive to them.","The same pipeline could be applied to other observed events with different mass ratios, spins, or precession, where CCE waveforms are rarer; the single-resolution caveat would need to be re-examined for each event."],"forward_implications":["AthenaK can evolve a roughly ten-orbit binary black hole merger on GPU hardware in about 130 hours on 192 GPUs, lowering the computational barrier for long-inspiral simulations.","The CCE pipeline produces waveforms at infinity that carry memory effects, so memory can be included from the start in waveform modeling and parameter estimation.","The $(2,2)$-mode agreement at the level of $0.35$ rad and $0.4\\%$ across the full evolution makes AthenaK waveforms usable as cross-checks for semi-analytic inspiral-merger-ringdown models in the GW150914-like region.","The open tutorial, input files, and analysis scripts allow other groups to reproduce the simulation and results, turning the pipeline into a community resource."],"supporting_citations":[{"why":"Supplies the targeted reference simulation results (remnant mass, spin, recoil, and waveforms) that AthenaK is compared against.","marker":"[80]"},{"why":"Provides the null-characteristic evolution code used for Cauchy-characteristic extraction of waveforms to future null infinity.","marker":"[104]"},{"why":"The code paper describing AthenaK's formulation and numerical methods that this simulation uses.","marker":"[91]"},{"why":"Describes the CCE pipeline with world-tube metric data that this work follows and extends to a binary black hole inspiral.","marker":"[34]"},{"why":"The frame-transformation tool used to move waveforms to the binary super rest frame before comparison.","marker":"[105]"},{"why":"The isolated-horizon formalism defining the remnant mass and spin reported as the primary remnant results.","marker":"[119]"}],"fun_headline_variants":["AthenaK matches GW150914 to 0.4% amplitude","GPU code AthenaK reproduces GW150914 to 0.4%","AthenaK reruns GW150914: waveform matches to 0.4%","AthenaK simulation matches GW150914 to 0.4%","GPU-accelerated AthenaK reruns GW150914, 0.4% match"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The numbers quoted as AthenaK's accuracy come from a single simulation at the baseline $N=128$ resolution, and that run is assumed to be converged enough that resolution error is smaller than the reported differences; the paper itself notes in Sec. 4 that only one grid resolution was used, while its Appendix shows the $(2,2)$ phase at merger changes by $0.38$ rad when resolution is raised to $N=192$, a shift comparable to the reported $0.35$ rad dephasing against the reference codes.","fun_headline_variants_meta":{"raw":{"variants":["AthenaK matches GW150914 to 0.4% amplitude","GPU code AthenaK reproduces GW150914 to 0.4%","AthenaK reruns GW150914: waveform matches to 0.4%","AthenaK simulation matches GW150914 to 0.4%","GPU-accelerated AthenaK reruns GW150914, 0.4% match"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001061,"raw_usage":{"total_tokens":4496,"prompt_tokens":1037,"completion_tokens":3459,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":653,"completion_tokens_details":{"reasoning_tokens":3352}},"tokens_in":653,"tokens_out":3459,"duration_ms":23896,"temperature":1.0,"reasoning_tokens":3352,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T05:51:45.070997+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same GW150914 setup at $N=192$ with otherwise identical parameters, align the $(2,2)$ modes against the reference waveforms as in Sec. 3.2, and compare the merger-phase difference to the $N=128$ value. If the higher-resolution run changes the AthenaK-minus-reference dephasing by more than about $0.2$ rad, or if the amplitude difference moves outside the $0.2\\%$--$0.4\\%$ band, the headline agreement is resolution-limited rather than a measure of AthenaK's accuracy.","supporting_citations":[],"review_version":1}