{"id":"a2b75605-6988-438a-8624-6daa2a0b080d","arxiv_id":"2506.06380","paper_version":1,"verdict":"ACCEPT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"A review of synthetic data generation for extreme events that compiles methods, datasets, and an evaluation framework focused on extremeness rather than privacy.","lead":"This paper surveys how synthetic data is generated for rare, extreme events such as market crashes, floods, and pandemics. It organizes existing methods, datasets, and evaluation metrics, and argues that extreme-event synthesis needs a different evaluation framework than privacy-focused synthetic data.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation (11) defines AKE as a 1-Wasserstein distance on raw sorted values while claiming it measures Kendall dependence; this mis-specification sits at the center of the survey's evaluation-framework contribution and needs correction.","rationale":"I read the paper as a survey whose central claim has two parts: it is the first comprehensive overview of synthetic data generation for extreme events, and its distinctive contribution is an in-depth evaluation framework for extremeness. The first part is hard to falsify from the text alone and is not internally inconsistent, so I do not rest a verdict on it. The reader's weakest assumption concerned whether extreme-event synthesis needs its own framework; I agree that this premise is asserted rather than empirically defended, but a survey may reasonably organize around a conceptual distinction, and the framework does contain genuinely extreme-specific elements such as extremal dependence, extreme coverage, and tail-adapted MSLE. The sharper, load-bearing problem is internal to the framework. In Section 6.2.1, Equation (11) defines AKE in a way that, taken literally, is the marginal 1-Wasserstein distance between raw samples, not a measure of dependence. Because the paper explicitly promises detailed mathematical definitions and because AKE is the metric the framework relies on for assessing dependence preservation in extremes, this mis-specification directly weakens the central contribution. The issue is concrete, localized, and testable; it does not indicate any problem with the paper's broader survey content. A conditional acceptance is therefore the right outcome: the evaluation framework should be accepted only after Eq. (11) and its supporting text are corrected or verified against [2] and [21]. If the primary sources confirm that a rank/Kendall transform is intended, the printed formula should be amended; the concern then disappears and the verdict can return to a straightforward accept.","tokens_in":29896,"tokens_out":11041,"duration_ms":117671,"concrete_test":"Re-derive AKE from the primary sources EV-GAN [2] and HTGAN [21]. If their definition uses a rank/Kendall transform before sorting, amend Eq. (11) to include that transform; the concern is resolved. If the definition is raw sorted observations, run a simple experiment with two bivariate samples having identical margins but opposite dependence structures (e.g., comonotone vs. independent); the formula as printed returns the same value, proving it is not a dependence metric. Either outcome settles whether Section 6.2.1 supports the framework's claims.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The survey's central contribution is the evaluation framework in Section 6, and Section 6.2.1 introduces Absolute Kendall Error (AKE) as a dependence-preservation metric. As printed, Eq. (11) defines AKE = (1/n) * sum |Z_{i,n} - Ztilde_{i,n}|, where Z_{1,n} <= ... <= Z_{n,n} are 'the sorted values from real and synthetic data.' For raw data values, this expression is exactly the 1-Wasserstein distance between the two univariate empirical marginal distributions; it is invariant to any permutation of the samples and therefore cannot measure concordance, rank dependence, or tail co-movement. The claim that it 'corresponds to the 1-Wasserstein distance between the Kendall's dependence functions' requires Z_i to be values of an empirical Kendall function (e.g., max of normalized marginal ranks), but that transformation is absent from the definition. The same under-specification affects the cited uses in EV-GAN [2] and HTGAN [21], and it undermines the paper's stated promise of 'detailed mathematical definitions' and 'in-depth insights into each metric's applicability in extremeness.' This is an internal correctness issue in the central contribution, not a dispute about field consensus.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This survey reviews synthetic data generation for extreme events, covering EVT-enhanced GANs and VAEs, importance sampling, diffusion models, hybrid architectures, domain-constrained generative models, and LLMs. It catalogs benchmark datasets by data type, proposes an evaluation framework organized into eight metric categories (distributional similarity, dependence preservation, extreme coverage, extremal dependence, reconstruction loss, extreme magnitude accuracy, visualization diagnostics, and downstream performance), and discusses application domains and open challenges. The paper claims to be the first comprehensive overview of synthetic data generation for extreme events, with a central contribution being the in-depth analysis of each metric's applicability to extremeness and its domain-specific adaptations.","tokens_in":30168,"tokens_out":5600,"duration_ms":45283,"significance":"If appropriately revised, this survey could serve as a useful structured entry point for researchers working on rare-event synthesis, consolidating a fragmented literature and providing a practical evaluation taxonomy. The dataset catalog (Tables 2 and 3) and the hands-on guidance in Section 6 are valuable, and the inclusion of a benchmark repository link supports reproducibility. The mathematical definitions of standard metrics (KS, KL, Wasserstein, FID, extremal coefficient) are mostly correct. However, the mis-specification of AKE in Eq. (11) undermines one component of the central evaluation framework and must be corrected before publication; this is an internal correctness issue, not a matter of field consensus.","major_comments":[{"comment":"The definition of AKE is mis-specified. As printed, Eq. (11) computes the average absolute difference between sorted raw values of the real and synthetic datasets. That quantity is exactly the univariate 1-Wasserstein distance between the two empirical marginal distributions; it is invariant to any permutation of the samples and therefore cannot measure concordance, rank dependence, or tail co-movement. The claim that it 'corresponds to the 1-Wasserstein distance between the Kendall's dependence functions' requires the Z_i to be values of an empirical Kendall function (e.g., maxima of normalized ranks), but that transformation is absent from the definition. The same under-specification affects the cited uses in EV-GAN [2] and HTGAN [21]. Because the evaluation framework is the paper's central contribution and AKE is a listed dependence-preservation metric in Table 4, this error is load-bearing. The authors should provide a correct definition (e.g., based on the empirical Kendall distribution function) or, if the primary sources use a different metric, report that faithfully.","section":"6.2.1, Eq. (11)"},{"comment":"The statement that \"all tabular datasets in extreme data modeling are synthetically generated rather than derived from real-world sources\" is contradicted by the paper's own dataset catalog. Table 2 lists the Market Supervision dataset [38] as a real-world dataset with over 20 features including transaction volume, market volatility, financial health, and regulatory disclosures, which appears to be tabular rather than a time series (it is placed under 'Finance Time Series' in the table). Either reclassify this dataset or qualify the claim about tabular data, since as written it is factually incorrect.","section":"5.4, Table 2"}],"minor_comments":[{"comment":"In the sentence after Eq. (1), the shape parameter is referred to as 'ε' but is denoted 'ξ' in the CDF; please use consistent notation.","section":"3.1"},{"comment":"The t-statistic formula is missing the bars on the sample means; the text defines \\(\\bar{x}\\) and \\(\\bar{\\tilde{x}}\\), so the equation should read \\((\\bar{\\tilde{x}} - \\bar{x}) / \\sqrt{s^2/n + \\tilde{s}^2/m}\\).","section":"6.1.2, Eq. (7)"},{"comment":"Eq. (18) defines MSLE with \\(\\log(1+x_i)\\) terms, while Eq. (19) uses \\(\\log x_i^{(j)}\\) without the offset; clarify whether the \"log(1+)\" transformation is part of the definition or a separate tail-adapted variant.","section":"6.6.3, Eqs. (18) and (19)"},{"comment":"The claim that synthetic data generation for extreme events 'differs fundamentally' from conventional approaches is asserted rather than demonstrated. Several metrics in Section 6 (KS, KL, Wasserstein, FID) are standard in general synthetic-data evaluation; the distinctiveness lies in tail-focused adaptations, not the metrics themselves. A more nuanced framing would strengthen the motivation.","section":"1"},{"comment":"Even after correcting Eq. (11), the statement that AKE is 'robust to outliers and invariant to marginal transformations' will need support: this holds for rank-based versions but not for raw-value versions.","section":"6.2.1"},{"comment":"There is a duplicated phrase in the text: 'supporting for stress testing stress testing of grid reliability'.","section":"5.2.3"},{"comment":"The text says 'as shown in Table 3' when referring to underexplored areas such as behavioral anomalies and DeFi disruptions; the relevant material appears in Table 5, not Table 3.","section":"7.1"}],"recommendation":"major_revision","confidential_remarks":"The survey is useful but needs a careful technical pass. The AKE mis-definition in Section 6.2.1 is the kind of error that can erode reader trust in the mathematical details, even though it is fixable. The authors should also consider trimming the number of self-citations; roughly eight of the 78 references are from the same group and some are tangential to the survey's core. The repeated 'first comprehensive' claim is plausible but hard to verify; a more modest phrasing would be safer. Overall, I recommend major revision rather than reject because the central content is otherwise sound and the load-bearing issues are localized."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The survey is a competent map of a young field, but the centerpiece evaluation framework has a definitional bug that needs fixing before the paper can be trusted as a reference.\n\nWhat is actually new: it is the first survey I know of that organizes synthetic-data generation specifically around extreme events, and the organization is useful. The method taxonomy (EVT-enhanced GANs/VAEs, importance sampling, diffusion, hybrids, physics-constrained, LLMs) is sensible, and the dataset catalog with sources, resolutions, and extreme-event definitions is a practical contribution. The section-by-section \"extremeness applicability\" commentary on metrics is genuinely helpful for practitioners choosing evaluation protocols.\n\nWhat it does well: most of the mathematical background is correct. The GPD definition, KS test, Wasserstein distance, FID, extremal coefficient, and MSLE are stated accurately enough to be usable. The authors are also honest about metric limitations (e.g., KS tail sensitivity, FID with ImageNet features on weather maps). The application coverage, especially underexplored domains like behavioral finance, wildfires, and infectious outbreaks, is a reasonable reading of the field.\n\nThe soft spot is real, and it is more than a typo. Equation (11) defines AKE as the L1 distance between sorted raw data values. That is exactly the 1-Wasserstein distance between the two univariate empirical marginals; it is invariant to permutations and cannot capture concordance, rank dependence, or tail co-movement. The text claims it is the 1-Wasserstein distance between Kendall's dependence functions, but the definition never constructs those functions. For AKE to mean what the paper says, the Z_i need to be values of an empirical Kendall process (e.g., some function of normalized ranks), not raw observations. This is an internal correctness issue in the paper's stated central contribution. The same under-specification affects the reported uses in EV-GAN and HTGAN, so the correction matters for anyone relying on the survey to understand those methods.\n\nOther soft spots are minor. The t-test statistic in Eq. (7) drops the bars over sample means, and the claim that all tabular datasets in extreme modeling are synthetic (Section 5.4) is overbroad. The premise that extremeness-focused evaluation is necessary, rather than general fidelity plus tail diagnostics, is asserted rather than empirically defended; I do not think that sinks the survey, since the framework's value is organizational.\n\nBottom line: this paper is for practitioners and newcomers who want a structured entry point into synthetic extreme-event data. It deserves a serious referee, but the referee should insist on correcting the AKE definition and re-checking the cited uses before publication. I would cite it after that fix; for now, it is a useful but not fully reliable map.","headline":"Useful map of a young field, but the AKE metric in the central evaluation framework is mis-specified and needs correction before the survey can be cited as a reliable reference.","tokens_in":30604,"tokens_out":3756,"would_cite":false,"duration_ms":33128,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This survey argues that synthetic data generation for extreme events is a distinct field requiring its own extremeness-centered evaluation framework, not the privacy-oriented one used for general synthetic data.","keywords":["synthetic data","rare events","extreme value theory","generative adversarial networks","evaluation metrics","heavy-tailed distributions","large language models","benchmark datasets"],"falsifier":"Compare a generic high-fidelity generator (for example, a standard GAN or diffusion model) against an EVT-enhanced generator on the same benchmark datasets using both global metrics (Wasserstein distance, FID) and extremeness-specific metrics (tail-restricted MSLE, extremal coefficient, coverage rate); if the generic model matches or beats the EVT-enhanced model on extremeness-specific metrics across multiple domains, the survey's central claim that extreme-event synthesis needs specialized methods and evaluation would be refuted.","tokens_in":29742,"feed_emoji":"🌪️","tokens_out":7839,"duration_ms":65634,"temperature":0.7,"pith_summary":"This survey argues that generating synthetic data for extreme events—market crashes, floods, pandemics—should be treated as its own field, with methods, benchmarks, and evaluation metrics designed for rarity and impact rather than for privacy or general fidelity. It reviews generative models, particularly GANs, VAEs, and diffusion models enhanced by extreme value theory and importance sampling, together with large language models for unstructured or text-based rare-event data. Its stated central contribution is an evaluation framework that organizes metrics into eight categories and, for each metric, analyzes how well it measures extreme behavior and how it should be adapted to specific domains. If the framing is correct, then anyone building or judging a rare-event generator should look at tail accuracy, extreme coverage, and downstream crisis performance, not just at how realistic the bulk of the samples look.","feed_headline":"Measure synthetic rare events by their extremes, not privacy","feed_subtitle":"First survey of synthetic rare-event data says evaluation should center on tail accuracy and coverage, not privacy.","key_machinery":"Two devices carry the argument. The first is the pairing of extreme value theory (EVT) with generative models: the Generalized Pareto Distribution from the Peaks Over Threshold method supplies tail-guided sampling, tail-aware losses, heavy-tailed latent priors, and spatial tail dependence structures that ordinary Gaussian-prior generators lack. The second is the evaluation framework itself, which separates general distributional similarity from extremeness-specific checks such as extreme coverage, extremal coefficient, extremal correlation, and tail-restricted mean squared logarithmic error. Together these devices turn 'does the synthetic data look real?' into 'does the synthetic data get the extremes right?'.","core_discovery":"The paper's central claim is that synthetic extreme-event data differs fundamentally from ordinary synthetic data because its purpose is not to enable privacy-preserving sharing but to train models on rare, high-impact scenarios. It documents a body of methods that inject extreme value theory into generative models—through tail-guided sampling, tail-aware training objectives, heavy-tailed latent priors, and spatial tail dependence modeling—and organizes the field's benchmark datasets by data type, from precipitation grids and limit order books to river discharge and keystroke intervals. The distinctive contribution is the evaluation framework: eight categories (distributional similarity, dependence preservation, extreme coverage, extreme dependence, reconstruction loss, extreme magnitude accuracy, visualization diagnostics, downstream performance validation) with per-metric guidance on extremeness applicability and domain-specific adaptations. The survey also maps application domains and identifies underexplored areas such as behavioral finance, wildfires, windstorms, earthquakes, and infectious outbreaks.","pith_inferences":["A testable consequence of the framework is that generic high-fidelity generators will look strong on global distributional metrics yet underperform EVT-enhanced models on tail-restricted metrics and downstream extreme-event tasks; this can be checked directly on the surveyed benchmarks.","The survey's clean split between privacy-oriented and extremeness-oriented synthesis is probably too clean: in healthcare and finance the two goals interact, since rare disease records and fraud cases are both sensitive and extreme, so future evaluation may need metrics that weigh both at once.","The catalogued underexplored domains amount to a concrete research agenda: behavioral finance, wildfires, windstorms, earthquakes, and wide infectious outbreaks are the places where the next benchmark datasets and method comparisons will likely form.","The eight-category evaluation framework could be turned into a practical scoring checklist or public leaderboard for rare-event generators, but the survey itself does not specify a consolidated scoring procedure."],"forward_implications":["Evaluation of rare-event generators should center on extremeness-specific metrics such as tail-restricted MSLE, extremal coefficient, extreme coverage, and domain-adapted FID rather than on global fidelity or privacy measures.","Benchmark suites for this field should include heavy-tailed, multivariate, spatially dependent datasets with explicit extreme definitions, as the surveyed precipitation, financial, hydrological, and energy datasets do.","EVT-enhanced generative models provide a reusable template—tail-guided sampling, tail-aware losses, heavy-tailed latent priors—that can be carried into domains the survey flags as underexplored, including earthquakes, wildfires, and infectious outbreaks.","Downstream validation becomes a standard evaluation step: augmenting real data with synthetic extremes and measuring improvements in prediction or risk detection, as done in flood forecasting and financial market supervision.","Domain-specific metric adaptations are necessary before generic metrics transfer, for example training an autoencoder on target data before computing FID for weather maps."],"supporting_citations":[{"why":"Introduces the extremeness measure and GPD conditioning that define the EVT-guided sampling line of methods.","marker":"[7]"},{"why":"Extends GPD-based generation to multivariate tails with adaptive extremeness levels.","marker":"[50]"},{"why":"Shows how EVT-based training and loss modifications stabilize GANs on heavy-tailed data.","marker":"[37]"},{"why":"Demonstrates tail-risk simulation via a loss exploiting joint elicitability of VaR and ES.","marker":"[12]"},{"why":"Provides the bounded tail-index parametrization and the sorted tail MSLE evaluation.","marker":"[2]"},{"why":"Brings EVT into VAEs through radius–angle decomposition of multivariate extremes.","marker":"[46]"},{"why":"Supplies the GMM-based importance sampling approach for biasing generation toward extremes.","marker":"[77]"},{"why":"Models spatial extremes by combining multivariate EVT with GANs.","marker":"[8]"},{"why":"Uses extremal coefficient matrices to evaluate tail dependence in synthetic hazards.","marker":"[59]"},{"why":"Provides the downstream-task validation template for synthetic financial extreme data.","marker":"[38]"}],"fun_headline_variants":["Rare-event AI needs synthetic data built for tails","Synthetic data for rare events: judge by the extremes","First survey: synthetic rare-event data must target tails","Tail-aware synthetic data: the missing piece for rare events","Synthetic rare-event data: measure extremes, not privacy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that synthetic data generation for extreme events is a distinct subfield whose evaluation must center on extremeness rather than privacy or general fidelity; if ordinary metrics already capture tail quality, the survey's framework loses its reason for being.","fun_headline_variants_meta":{"raw":{"variants":["Rare-event AI needs synthetic data built for tails","Synthetic data for rare events: judge by the extremes","First survey: synthetic rare-event data must target tails","Tail-aware synthetic data: the missing piece for rare events","Synthetic rare-event data: measure extremes, not privacy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000197,"raw_usage":{"total_tokens":1361,"prompt_tokens":942,"completion_tokens":419,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":558,"completion_tokens_details":{"reasoning_tokens":340}},"tokens_in":558,"tokens_out":419,"duration_ms":4647,"temperature":1.0,"reasoning_tokens":340,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T10:41:35.209857+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare a generic high-fidelity generator (for example, a standard GAN or diffusion model) against an EVT-enhanced generator on the same benchmark datasets using both global metrics (Wasserstein distance, FID) and extremeness-specific metrics (tail-restricted MSLE, extremal coefficient, coverage rate); if the generic model matches or beats the EVT-enhanced model on extremeness-specific metrics across multiple domains, the survey's central claim that extreme-event synthesis needs specialized methods and evaluation would be refuted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows how EVT-based training and loss modifications stabilize GANs on heavy-tailed data."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the extremeness measure and GPD conditioning that define the EVT-guided sampling line of methods."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Extends GPD-based generation to multivariate tails with adaptive extremeness levels."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the bounded tail-index parametrization and the sorted tail MSLE evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the GMM-based importance sampling approach for biasing generation toward extremes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Models spatial extremes by combining multivariate EVT with GANs."},{"cited_title":"Combining deep generative models with extreme value theory for synthetic hazard simulation: a multivariate and spatially coherent approach","cited_arxiv_id":"2311.18521","evidence_quote":"Uses extremal coefficient matrices to evaluate tail dependence in synthetic hazards."}],"review_version":1}