{"id":"41a0186c-ecaa-4f2c-b079-5c6e85b61005","arxiv_id":"2606.21038","paper_version":1,"verdict":"ACCEPT","confidence":"LOW","novelty_score":4.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"Recommends median, median absolute deviation, and empirical confidence interval coverage as better general summaries than means and standard deviations for simulation studies of asymptotic approximations.","lead":"The paper argues that simulation studies evaluating asymptotic approximations should report quantile-based summaries such as the median and median absolute deviation rather than means and standard deviations. A smart generalist might read it to see why common reporting practices can mislead when distributions have outliers or moments fail to converge.","discovery_kind":"extension","skeptic_critique":{"model":"grok-4.3","headline":"No significant objection identified","rationale":"The reader's weakest_assumption matches the paper's own scoping language; once that scope is granted, the argument contains no internal gap that would alter the ACCEPT verdict.","tokens_in":1572,"tokens_out":220,"duration_ms":7746,"concrete_test":"Extract the exact wording in §2 or §3 that defines the intended scope of the simulation study; verify that the quantile recommendation is conditioned on that scope and does not assert superiority when moments are the estimand of interest.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is scoped to simulation studies whose goal is assessing accuracy of a distributional approximation (rather than direct estimation of moments). Under that scope the theoretical point (convergence in distribution does not entail moment convergence) is standard and correctly applied; the practical point about outlier sensitivity of means/SDs for near-normal distributions is also standard. The manuscript explicitly carves out the case where moments are the substantive target, so the recommendation does not over-claim.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript argues that simulation studies evaluating asymptotic approximations should prefer quantile-based summaries (median, median absolute deviation, and empirical coverage) over means and standard deviations. The theoretical rationale is that convergence in distribution does not imply convergence or even existence of moments, rendering sample moments unsuitable for assessing distributional accuracy. Practically, means and variances are sensitive to occasional outliers even when the underlying distribution is approximately normal. Moments are to be reserved for settings where they are the direct object of substantive interest.","tokens_in":1620,"tokens_out":295,"duration_ms":22603,"significance":"The note identifies a widespread but theoretically unsupported reporting convention in simulation studies. Adoption would align reporting practices more closely with the actual inferential target (distributional approximation) and reduce sensitivity to tail behavior. The argument relies entirely on standard probability theory with no new parameters, derivations, or self-referential constructs.","major_comments":[],"minor_comments":[{"comment":"The manuscript is a short note; a single illustrative numerical example (even a small Monte Carlo illustration of mean vs. median behavior under a heavy-tailed approximation) would make the practical claim more concrete without altering the central argument.","section":null},{"comment":"The title 'De-meaning Simulation Studies' is concise but may not immediately signal the content to readers scanning tables of contents; a subtitle or clearer phrasing could improve discoverability.","section":null}],"recommendation":"accept","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for their supportive review and recommendation to accept the manuscript. The summary accurately captures our central arguments on the theoretical and practical limitations of moment-based summaries in simulation studies of asymptotic approximations.","responses":[],"tokens_in":1088,"tokens_out":58,"duration_ms":7403,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main thing to know is that the authors argue quantile summaries (median, MAD, coverage) are better than means and SDs for simulation studies whose goal is to assess how well an asymptotic distribution approximates the finite-sample behavior. The theoretical reason is standard: convergence in distribution does not imply convergence of moments, so sample moments can be poor indicators even when the approximation is good in the distributional sense. The practical reason is that means remain sensitive to occasional outliers in near-normal data.\n\nThe paper does this cleanly. It states the scope explicitly, notes that moments should still be reported when they are the substantive target, and avoids any over-claim. The logic follows directly from basic probability without new derivations or fitting.\n\nThe soft spots are small. No numerical example is given showing how much the choice actually changes conclusions in a typical simulation, which would have made the practical point more concrete. The recommendation to add coverage is already routine in some subfields, so the main addition is the quantile preference for location and scale. These are not load-bearing weaknesses.\n\nThe note is aimed at people who run or referee simulation studies in statistical methodology. A reader already attentive to reporting standards will find it a useful reminder. It deserves peer review because the point is sound, the change is low-cost, and the argument is internally consistent.","headline":"The paper makes a straightforward case for switching to quantiles when simulations check distributional approximations rather than moments themselves.","tokens_in":2086,"tokens_out":334,"would_cite":false,"duration_ms":12535,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Simulation studies evaluating distributional approximations should report medians and coverage rates instead of means and standard deviations.","keywords":["simulation studies","asymptotic approximations","quantile summaries","median","median absolute deviation","confidence interval coverage","statistical methodology","Monte Carlo methods"],"falsifier":"A concrete simulation in which the target distribution converges in law but the sample mean and variance across replications diverge or become unstable, while the median, MAD, and coverage remain stable and correctly indicate good approximation quality.","tokens_in":2458,"feed_emoji":"📊","tokens_out":679,"duration_ms":13929,"temperature":0.7,"pith_summary":"The paper contends that simulation studies assessing asymptotic approximations commonly rely on averages and standard deviations, but these are theoretically and practically inferior to quantile-based summaries. Convergence in distribution does not ensure that moments exist or converge, making sample means unreliable indicators of how well a limiting distribution approximates the finite-sample behavior. In practice, occasional outliers in near-normal simulation results distort means and variances, whereas the median, median absolute deviation, and empirical confidence-interval coverage give more stable and relevant information. A reader would care because this changes how methodological papers demonstrate the reliability of their asymptotic results.","feed_headline":"Medians and coverage rates beat means for checking distributional approximations","feed_subtitle":"Convergence in distribution need not imply moment convergence, so quantile summaries give a more reliable picture of approximation quality.","key_machinery":"The distinction between convergence in distribution and convergence of moments, together with the practical fragility of means in the presence of outliers; this distinction motivates the shift to median, median absolute deviation, and empirical coverage as the default reporting tools.","core_discovery":"Quantile-based summaries are more appropriate than moment-based ones for assessing the accuracy of distributional approximations in simulation studies. Theoretically, convergence in distribution does not imply convergence of moments or even their existence, so sample moments are not guaranteed to reflect the quality of the approximation. Practically, means and variances are sensitive to outliers even when the distribution is approximately normal. The paper therefore recommends the median and median absolute deviation as general summaries, together with empirical confidence-interval coverage, and reserves moments for cases where they are the direct object of substantive interest.","pith_inferences":["Re-analysis of existing simulation studies that used means could change conclusions about how well certain approximations perform.","Software packages for Monte Carlo simulation could adopt quantile summaries as the default output format.","The same logic extends to simulation studies in other fields that validate limiting distributions, such as bootstrap or MCMC diagnostics."],"forward_implications":["Papers claiming asymptotic normality would report median absolute deviation rather than standard deviation to describe variability across simulations.","Empirical coverage of nominal confidence intervals would become a standard reported quantity in simulation tables.","Moments would appear in simulation results only when the study is explicitly investigating expected values or variances as quantities of interest.","Outlier-resistant summaries would reduce the influence of rare simulation failures on reported performance."],"fun_headline_variants":["Medians better than means for checking distributional approximations","Quantiles more reliable than moments for approximation quality","Use medians and coverage to assess asymptotic approximations","Moments unreliable for confirming simulation distributional approximations"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The main goal of the simulation study is to judge how well a distributional approximation works, rather than to study moments that have direct substantive meaning.","fun_headline_variants_meta":{"raw":{"variants":["Medians better than means for checking distributional approximations","Quantiles more reliable than moments for approximation quality","Use medians and coverage to assess asymptotic approximations","Moments unreliable for confirming simulation distributional approximations"]},"model":"grok-4.3","cost_usd":0.007957,"raw_usage":{"total_tokens":3579,"prompt_tokens":577,"num_sources_used":0,"completion_tokens":56,"cost_in_usd_ticks":79574500,"prompt_tokens_details":{"text_tokens":577,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2946,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":577,"tokens_out":56,"duration_ms":23163,"temperature":1.0,"reasoning_tokens":2946,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-26T14:04:48.194234+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A concrete simulation in which the target distribution converges in law but the sample mean and variance across replications diverge or become unstable, while the median, MAD, and coverage remain stable and correctly indicate good approximation quality.","supporting_citations":[],"review_version":1}