{"id":"8be2a8cd-9df9-4a90-bf82-286fdbf3dac9","arxiv_id":"2412.00815","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A unified masked Transformer with absolute and relative location embeddings outperforms specialized baselines on ground-motion forecasting, interpolation, and earthquake early warning in California.","lead":"QuakeFormer is a single Transformer model that predicts where earthquake shaking will be strong, whether it is used before an earthquake with only source information, during one with early seismic signals, or after one with station recordings filling in gaps. It reports better accuracy than several specialized methods on California data and shows that pretraining on forecasting and interpolation helps the early-warning version.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The comparative claim is tested against modified baselines—ASK14 is refit, MVN is fed by QuakeFormer, EPS is given catalog hypocenters—so the unqualified claim that QuakeFormer outperforms specialized architectures remains conditional.","rationale":"I read the paper in good faith and identify the same load-bearing point the reader flagged: the comparative claim depends on the representativeness of the baselines. However, I do not regard this as a decisive flaw because most modifications make the baselines stronger rather than weaker. Refitting ASK14 to the local training distribution, giving EPS catalog source parameters, and (to a lesser extent) feeding QuakeFormer into MVN all stack the deck against QuakeFormer, so a failure of the unmodified-baseline check is not the most likely outcome. The concern is instead about the precision and scope of the central claim: the paper currently demonstrates superiority over modified implementations, not over the exact published ASK14, the operational ShakeMap pipeline, or a self-locating EPS. This is exactly why the paper should remain CONDITIONAL rather than being accepted as an unqualified state-of-the-art comparison. The held-out test-on-2023-events design, the explicit spatial split, and the residual-decomposition evidence give real independent support to the core methodology. I would not move the verdict to REJECT because there is no internal inconsistency in the training or evaluation logic that would invalidate the method; the issue is external and empirically checkable. The proposed test—rerunning the three comparisons with unmodified baselines—would settle the concern directly. If the margins survive, the unqualified headline becomes justified; if they change materially, the conclusions must be narrowed to the modified baselines or the specific implementations tested.","tokens_in":24508,"tokens_out":11016,"duration_ms":112483,"concrete_test":"Rerun the Section 3 and Section 3.2 comparisons with unmodified operational baselines: (1) ASK14 using the original published coefficients instead of the Appendix C.1 regression; (2) ShakeMap-style MVN with an independent GMPE (e.g., ASK14 or the standard regional GMPE) as the forecasting mean instead of QuakeFormer; (3) an EPS that estimates epicenter and magnitude from the first 4 s of P-wave data instead of using catalog hypocenters and ShakeAlert reports; and optionally (4) PLUM with station/site corrections as in Kodera et al. Recompute R2 on TR+SM and TR+SR for all three tasks, including the EEW time-evolution curves in Figure 7. If QuakeFormer retains nonnegative margins everywhere, the unqualified comparative claim stands; if margins shrink or reverse, the paper should restrict its claim to the modified baselines or quantify the gap against each unmodified system.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is explicitly comparative: QuakeFormer 'outperforms specialized architectures' on forecasting, interpolation, and early warning. The evidence for that claim comes from baselines defined in Appendix C, and several are not the operational systems they are said to represent. ASK14 is a simplified equation refit to the training set (C.1); the MVN interpolation baseline uses QuakeFormer itself as its forecasting model (Section 3 and C.2); the EPS baseline is given catalog hypocenters and ShakeAlert magnitude reports instead of estimating source parameters (C.3); and PLUM omits site corrections (C.4). The direction of bias is mixed: refitting ASK14 to the local small-magnitude dataset and giving EPS exact source information should make those baselines harder to beat, whereas the QuakeFormer-fed MVN makes the interpolation comparison an internal backboned pipeline rather than the full ShakeMap algorithm it stands for. As written, the paper establishes performance against these modified implementations, not against the unmodified published ASK14, an independent ShakeMap-MVN, and a self-locating EPS. The held-out 2023 event split, the masking formulation, and the non-ergodic residual analysis are sound and internally consistent; the load-bearing weakness is in the external comparison, not in the method itself.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces QuakeFormer, an encoder-only masked Transformer for ground-motion prediction that jointly handles three tasks: forecasting PGA from event and site metadata, interpolating PGA from partial intensity and waveform observations, and earthquake early warning from real-time partial waveforms. A unified model is first pretrained on forecasting and interpolation via a randomized station/wave masking strategy, then fine-tuned for early warning. The evaluation uses 2023 events as a time-held-out set and spatially held-out stations, with comparisons to ASK14, MVN, EPS, and PLUM baselines. The paper also reports position-embedding ablations, a non-ergodic residual decomposition, and claims that the unified model outperforms specialized architectures across all three tasks.","tokens_in":24820,"tokens_out":7677,"duration_ms":71566,"significance":"The core idea of a single masked-Transformer architecture that unifies three ground-motion tasks is timely and potentially impactful for operational seismic hazard workflows. The held-out event and held-out station split is a genuine strength, and the non-ergodic residual analysis is a constructive way to inspect what the model learns. The masking formulation and the use of both absolute and relative spatial position encodings are well motivated. However, the headline comparative claim is not yet fully supported: several baselines are modified or hybridized relative to their published operational form, and the reported R2 values lack confidence intervals or significance tests. If the comparisons are run against unmodified published baselines, or clearly recast as comparisons against simplified implementations, the central claim would be much stronger.","major_comments":[{"comment":"The forecasting baseline is not the published ASK14 model but a simplified one-dimensional equation whose coefficients are refit by least squares to the training set TM+SM. Since the paper's central claim is that QuakeFormer outperforms state-of-the-art forecasting models, the comparison in Fig. 3(a) must either use the original published ASK14 coefficients (with an explicit discussion of small-magnitude extrapolation) or the claim must be limited to \"outperforms a simplified refit ASK14\". This is load-bearing for the forecasting result.","section":"Section 3, Appendix C.1, Eq. (S4)"},{"comment":"The MVN interpolation baseline uses QuakeFormer itself as the forecasting model for the prior mean, so Fig. 3(a) does not compare against an independent ShakeMap/MVN system. The paper's stated rationale of isolating the interpolation algorithm does not justify the abstract's unqualified claim of outperforming state-of-the-art interpolation. Moreover, the uncertainty parameters required by the MVN are unspecified: the text reads \"we assume phi= and tau=\" without values. Please supply the assumed values and, ideally, a parallel comparison with an MVN using an independent GMPE prior.","section":"Section 3, Appendix C.2, Eqs. (S5)-(S7)"},{"comment":"The EPS baseline is given catalog hypocenters and ShakeAlert magnitude reports, so it does not perform real-time source estimation. This removes the main source-estimation error from the EPS baseline. The paper acknowledges this in C.3, but the abstract and Section 3.2 present QuakeFormer's end-to-end superiority without this caveat. A fair comparison should include an EPS that estimates its own source, or the claims must be explicitly limited to \"QuakeFormer outperforms an EPS provided with perfect locations.\" The comparison of QuakeFormer's magnitude error to a literature value for ElarmS-3 is also not a concurrent run and should be labeled as such.","section":"Section 3, Appendix C.3"},{"comment":"All R2 comparisons are reported as point estimates without confidence intervals or significance tests. Some differences are numerically small (e.g., Fig. 8(a) M1 vs M2 on SR; Fig. 7 finetune vs single), and the reader cannot tell whether they are meaningful beyond the large sample size. Please report event-based or bootstrap confidence intervals, and where possible paired comparisons on the same held-out events, for the headline forecasting, interpolation, and early-warning results.","section":"Section 3, Figs. 3, 7, 8 and Table S2"},{"comment":"The residual standard deviations (sigma, tau, phi, tauL, phiS) are compared directly with values from Baltay et al. (2017) and Villani & Abrahamson (2015), but the datasets, magnitude ranges, and residual-decomposition methods differ substantially. For example, this study is dominated by small-magnitude events, while the cited studies include larger earthquakes and use different decomposition approaches. The claim of \"significantly smaller values\" for tauL and phiS should be supported by matching magnitude/distance distributions, or restated as an indicative comparison.","section":"Section 4.2, Table 3"}],"minor_comments":[{"comment":"\"Sourth California\" should be \"Southern California\"; the data availability statement saying the implementation \"will be available after paper reception\" should be replaced with a concrete repository URL or a clear statement of how code will be shared.","section":"Section 6"},{"comment":"\"Conventional Neural Network (CNN)\" should read \"Convolutional Neural Network (CNN)\".","section":"Section 2.3.1"},{"comment":"\"if the total residual distributions\" should be \"The total residual distributions\".","section":"Figure 3 caption"},{"comment":"There are typos in Table 2 (\"atfer OT\", \"atrer OT\") and in the Figure 7 caption (\"finuetune\"); these should be corrected.","section":"Table 2 and Figure 7 caption"},{"comment":"The text states 88 relatively large events in 2023 (M > 3.5), while Figure S3 says 95 large earthquakes; the numbers should be reconciled.","section":"Section 3.2 and Figure S3"},{"comment":"Use \"R2\" consistently; the caption \"R2-squared\" is redundant.","section":"Figure 8 caption"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's central method is sound, but the external comparison is the main obstacle. The authors should be asked to rerun or clearly reinterpret the baselines (ASK14 with original coefficients, an independent MVN prior, and a self-locating EPS) or to qualify the abstract and conclusion accordingly. Also, the incomplete sentence in Appendix C.2 should be fixed, and the paper would benefit from a code/data availability statement that points to an actual repository. The held-out split and large dataset are strong assets, and the architecture is worth publishing once the comparative claims are made precise."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a serious attempt at a unified deep-learning model for three ground-motion tasks, and it deserves refereeing. The masking formulation, the continuous-coordinate RoPE, and the multi-scale absolute location embeddings are a sensible combination, and the held-out event/station split is a genuine strength. The non-ergodic residual decomposition, including the Cluster 12 example and the site-term comparison, goes beyond a pure metrics dump and shows real geological thinking.\n\nThe stress-test note is on target: the headline claim can only be read as 'outperforms the modified baselines in Appendix C.' ASK14 is a simplified form refit to the training set; MVN uses QuakeFormer itself as its forecasting model; EPS is handed catalog hypocenters and magnitude reports; PLUM omits site corrections. The directions of bias are mixed, so I don't suspect cherry-picking, but the abstract's unqualified wording goes past what the experiments establish. The paper deserves some credit for documenting these modifications, yet a referee should ask for at least one comparison against unmodified operational baselines.\n\nOther soft spots: all R2 comparisons are point estimates with no confidence intervals or significance tests, and some of the reported margins could easily shrink. There is also no code or data released, which is a real limitation for an architecture-centric paper. And there is an internal inconsistency: Section 2.3.2 sets N=d/4 with d=384, which would give a 384-dimensional embedding, while Appendix B.1 says N=24 with output dimension 96 'equal to our model's dimension D.' Those cannot both describe the same model. Likely a typo, but it matters for reimplementation.\n\nNone of this kills the paper. The architecture is coherent, the masking strategy is flexible, and the pretraining/fine-tuning benefit for EEW is interesting. The authors also openly acknowledge the main limitations—SR generalization, near-source underestimation, and larger epicenter error than ElarmS-3—which makes me trust the reporting. This is for the GMP/EEW community, both seismologists and ML researchers, and for anyone thinking about how to evaluate neural surrogates against operational baselines. I would send it to peer review with requests for code, confidence intervals, and a cleaner baseline comparison.","headline":"A promising unified transformer for ground-motion tasks with a credible held-out split, but the 'outperforms specialized architectures' claim rests on modified baselines and still needs code and error bars before it can be taken at face value.","tokens_in":25305,"tokens_out":2832,"would_cite":true,"duration_ms":27827,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single masked Transformer architecture outperforms specialized models on forecasting, interpolation, and early-warning ground motion.","keywords":["ground motion prediction","masked transformer","earthquake early warning","spatial interpolation","location embedding","nonergodic ground motion","ShakeMap","pretraining"],"falsifier":"Rerun the three comparisons against the original published ASK14 coefficients, an MVN interpolator conditioned on an independent ground-motion mean, and an EPS that estimates its own epicenter from P-wave arrivals; if QuakeFormer no longer leads on R2 on the 2023 California test events, the claimed superiority fails.","tokens_in":24294,"feed_emoji":"🌊","tokens_out":5584,"duration_ms":48944,"temperature":0.7,"pith_summary":"QuakeFormer aims to show that one masked-Transformer architecture can replace the three specialized families of ground-motion prediction models used before, during, and after earthquakes. The paper trains a single network on forecasting and interpolation as one continuous masking task, then fine-tunes it for early warning, and reports that it outperforms the specialized ASK14, MVN, EPS, and PLUM baselines. The claim matters because current systems chain separate models—source estimation, ground-motion equations, and interpolation—so errors cascade; a single data-driven model could make shaking estimates faster and more consistent.","feed_headline":"One transformer beats specialized quake models in all three tasks","feed_subtitle":"QuakeFormer unifies forecasting, interpolation, and early warning of shaking in one masked-Transformer model.","key_machinery":"The load-bearing mechanism is a masking schedule over station intensity tokens and waveform tokens. Masking all intensity tokens ($r_s=1$) reproduces forecasting, masking a fraction ($r_s<1$) produces interpolation, and a time mask on waveforms produces early warning; the Transformer predicts the masked tokens, so one architecture covers all three tasks. Spatial reasoning is carried by two location encodings: a multi-scale sinusoidal encoder of absolute coordinates plus a learned network, and Rotary Position Embedding applied to continuous geographic coordinates inside self-attention, which encodes relative separation between stations.","core_discovery":"The paper's central claim is that a uniform transformer-based architecture, QuakeFormer, outperforms task-specialized ground-motion prediction models on forecasting, interpolation, and earthquake early warning. The same network predicts the distribution of peak ground acceleration at any station from whatever observations are available: no observations for forecasting, partial station intensities for interpolation, and partial early-arrival waveforms for early warning. Pretraining the forecasting–interpolation model before fine-tuning on the early-warning task improves early-warning accuracy relative to training early warning from scratch, which the paper attributes to the model having already absorbed site and regional effects.","pith_inferences":["If the reported margins survive stronger baselines, QuakeFormer-type models could become the core of regional ShakeMap and EEW pipelines, with forecast, alert, and post-event products generated by one checkpoint.","The masking formulation suggests a natural continuum: the same checkpoint could serve any observational completeness, including sparse networks or degraded telemetry, by adjusting mask ratios at inference.","A direct test is to probe the architecture on a different tectonic regime (for example, Japan or New Zealand) without retraining the location encoder, to see whether the learned relative-dependency structure transfers.","Because site terms shrink for seen stations but remain large at unseen sites, the paper's own limitation points to a promising direction: conditioning on 3D velocity models or geological surveys should yield the largest further gains."],"forward_implications":["A single QuakeFormer model can serve hazard forecasting, ShakeMap-style interpolation, and early warning, removing the need to maintain separate specialized models.","Because early-warning intensity is predicted end-to-end, the cascaded errors of source estimation followed by a ground-motion prediction equation are reduced.","Adding even a small number of observed stations (mask ratio 0.8) sharply improves shaking prediction, and full waveforms add further gains.","Pretraining on the full historical California dataset transfers useful site and regional knowledge to the early-warning task, improving magnitude and PGA estimates within seconds of P arrival.","Residual decomposition shows smaller location-based event and site terms than GMPE-based studies, suggesting much of what is treated as aleatory randomness is actually modeled."],"supporting_citations":[{"why":"Supplies the ASK14 ground-motion prediction equation used as the forecasting baseline.","marker":"Abrahamson et al. (2014)"},{"why":"Defines the conditional multivariate-normal interpolation algorithm (ShakeMap's MVN) used as the interpolation baseline.","marker":"Worden et al. (2018)"},{"why":"Provides the magnitude-from-P-wave scaling relation used in the EPS early-warning baseline.","marker":"Kuyuk & Allen (2013)"},{"why":"Defines the PLUM wavefield-based early-warning baseline.","marker":"Kodera et al. (2018)"},{"why":"A comparable transformer-based early-warning model that QuakeFormer's EEW-single variant is measured against.","marker":"Münchmeyer et al. (2021)"},{"why":"Introduces Rotary Position Embedding, adapted here to continuous geographic coordinates for relative spatial dependency.","marker":"Su et al. (2024)"},{"why":"Motivates the multi-scale sinusoidal position encoding used for absolute location embeddings.","marker":"Mildenhall et al. (2020)"},{"why":"Provides the residual decomposition into event, location-based event, site, and single-station components used for nonergodic analysis.","marker":"Baltay et al. (2017)"}],"fun_headline_variants":["One transformer tops specialized models in all quake tasks","QuakeFormer unifies forecasting, interpolation, early warning","Masked transformer masters three ground-motion tasks at once","Single model beats task-specific quake predictors with masking"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline comparison assumes the baselines represent state-of-the-art specialized practice, yet ASK14 is refit to the training data, the MVN interpolator uses QuakeFormer as its mean model, and the EPS system is handed catalog hypocenters.","fun_headline_variants_meta":{"raw":{"variants":["One transformer tops specialized models in all quake tasks","QuakeFormer unifies forecasting, interpolation, early warning","Masked transformer masters three ground-motion tasks at once","Single model beats task-specific quake predictors with masking"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000168,"raw_usage":{"total_tokens":1236,"prompt_tokens":893,"completion_tokens":343,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":509,"completion_tokens_details":{"reasoning_tokens":278}},"tokens_in":509,"tokens_out":343,"duration_ms":3941,"temperature":1.0,"reasoning_tokens":278,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T04:59:13.369497+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun the three comparisons against the original published ASK14 coefficients, an MVN interpolator conditioned on an independent ground-motion mean, and an EPS that estimates its own epicenter from P-wave arrivals; if QuakeFormer no longer leads on R2 on the 2023 California test events, the claimed superiority fails.","supporting_citations":[{"cited_title":", Thompson, E M","cited_arxiv_id":null,"evidence_quote":"Defines the conditional multivariate-normal interpolation algorithm (ShakeMap's MVN) used as the interpolation baseline."},{"cited_title":"\\ Allen, R M","cited_arxiv_id":null,"evidence_quote":"Provides the magnitude-from-P-wave scaling relation used in the EPS early-warning baseline."},{"cited_title":", Yamada, Y","cited_arxiv_id":null,"evidence_quote":"Defines the PLUM wavefield-based early-warning baseline."},{"cited_title":", Hanks, T C","cited_arxiv_id":null,"evidence_quote":"Provides the residual decomposition into event, location-based event, site, and single-station components used for nonergodic analysis."}],"review_version":1}