{"id":"29fe9d60-ca0d-4456-bd81-007ceb7a0155","arxiv_id":"2411.13696","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A statistical analysis of IFSC speed climbing data finds the Tomoa Skip is associated with about 14.5 percent faster best times, with no significant effect on fall rates.","lead":"This paper uses competition records and video review to estimate whether the Tomoa Skip, a bouldering-style move introduced into speed climbing, made climbers faster. It finds the technique is associated with roughly 14 percent faster best times, while falling risk did not change significantly.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"TS effect likely conflates adoption with individual improvement trends: M3 controls only a global time trend, no climber-level time slope, despite claiming to model differing progression rates; refitting M3 with climber-specific time slopes is the decisive test.","rationale":"I read the reader's verdict as CONDITIONAL, and I agree that the paper needs robustness work before the headline claim can be accepted. However, I do not think the persistence assumption is the single most load-bearing concern. Misclassification of the TS indicator would generally shrink the estimated association toward zero, making the reported benefit conservative rather than spurious. The more serious threat is that the model's within-climber before/after comparison is not adjusted for individual-specific improvement trajectories. Section 3.1 explicitly promises random effects for 'rate of progression of individual climbers,' but the M3 specification contains no random slope for time progression; the only time control is a common fixed effect. Since TS adoption was a choice made by athletes who were often in a period of intense speed-specific training, the adoption indicator is likely correlated with unobserved individual trends. The -0.1568 estimate could therefore reflect improvement that would have happened without the TS. This is an internal inconsistency between the prose and the model, and it is directly testable. Adding a climber-level random slope for time is a minimal, standard extension that would either confirm the estimate or reveal its fragility. If the estimate survives that extension, the central claim is substantially stronger; if not, the paper should be reframed as descriptive association rather than effect estimation. I keep the verdict CONDITIONAL because the needed test is well-defined and the existing evidence does not decisively reject a real benefit.","tokens_in":8384,"tokens_out":6346,"duration_ms":61742,"concrete_test":"Refit M3 adding a climber-level random slope for time progression: log(y_ij) = γ00 + γ01 x1_ij + γ02 x2_j + γ03 x3_ij + (γ04 + v_j) x4_i + μ0i + μ1i x1_ij + υ0j + υ1j x1_ij + ε_ij, with v_j ~ N(0, ψ^2). If the TS fixed effect remains near -0.157 with a CI excluding zero, the concern is mitigated. If it attenuates by more than 30% toward zero, or if a placebo regression assigning next-event TS status to the current event yields a nontrivial negative coefficient, the headline effect is not identified. This test is feasible with the paper's data and directly targets the missing control.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The main coefficient γ01 is identified from within-climber before/after variation in TS use, after adjusting for gender, age, a single event-level time-progression term, and random intercepts/slopes for TS. This does not control for individual-specific improvement trajectories. Section 3.1 states that the random effects 'allow us to consider variations in both the initial skill level and the rate of progression of individual climbers,' but the model written there includes random effects only for intercepts and for the TS slope (μ0i, μ1i, υ0j, υ1j); there is no random slope for time progression x4i. The fixed γ04 is common to every climber at an event. Adoption of the TS was voluntary and concentrated among elite, Olympic-focused athletes who were simultaneously intensifying speed training, so adoption timing is likely correlated with unobserved athlete-level improvement slopes. The estimator then assigns part of each adopter's personal improvement to the TS indicator, inflating -0.1568. The persistence-assumption misclassification identified by the reader would tend to attenuate the estimate, so it is less threatening than this omitted-trend confounding.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper investigates the effect of the Tomoa Skip on IFSC speed climbing times using scraping of competition results from 2012 to 2022 and manual coding of Tomoa Skip usage from 54 YouTube broadcasts of finals (2018-2022). TS usage is imputed forward and backward under a persistence assumption. The authors fit mixed effects models with log best time as the outcome, fixed effects for TS, gender, age, and time progression, and random intercepts and random TS slopes for climbers and events. Model M3 is selected by BIC and ANOVA, yielding a fixed TS coefficient of -0.1568 (95% CI: -0.1898, -0.1238), corresponding to a multiplicative 0.8549 change in average best time. The paper also examines variability of times and a generalized mixed model for falls, finding no significant fall effect.","tokens_in":8620,"tokens_out":6072,"duration_ms":82445,"significance":"If the central estimate is accepted, the paper provides a quantitative measure of a novel technique's impact in a sport with limited quantitative analysis, and the confidence interval is reasonably tight. The manual video coding of 108 hours of footage is a substantial data collection effort, and the model is a sensible starting point for this question. However, the causal interpretation is fragile: TS adoption is observational and likely correlates with individual training trajectories, and the paper's own limitations section acknowledges the imputation and the finalist-only sample. The contribution is of moderate significance for sports analytics rather than a methodological advance.","major_comments":[{"comment":"The text states that the random effects 'allow us to consider variations in both the initial skill level and the rate of progression of individual climbers.' However, the model as written in Section 3.1 includes random intercepts (μ0i, υ0j) and random slopes for TS usage (μ1i, υ1j) only; there is no random slope for the time progression variable x4i. The fixed coefficient γ04 is common to all climbers. Because adoption of the Tomoa Skip was voluntary and concentrated among elite, Olympic-focused athletes who were concurrently intensifying speed training, adoption timing is plausibly correlated with unobserved athlete-level improvement slopes. Under this scenario, γ01 will absorb part of the adopters' personal improvement, biasing the estimate away from zero. I consider a refit of M3 with climber-specific time slopes (e.g., a random slope for x4i, or a within-climber time trend) to be the decisive robustness check; the authors should report how γ01 and its CI change.","section":"Section 3.1 (M3 specification)"},{"comment":"The paper acknowledges that TS usage is only observed for final-round broadcasts and imputes usage to unobserved rounds via the persistence assumption. This creates two threats to the main estimate. First, if climbers switch per round or per event without being observed (the paper documents one switch back), the predictor x1ij is misclassified; the direction of the resulting bias is not established. Second, restricting TS labels to finalists conditions the sample on reaching finals, so the estimate may not generalize to the broader population of speed climbers and may be subject to selection bias. The paper should add a sensitivity analysis, for example, estimating the model only on observations where TS use is directly video-confirmed, or treating unobserved rounds as missing rather than imputed.","section":"Section 2 / Section 4 (TS assignment and sample selection)"},{"comment":"The units of the age and time progression variables are not clearly defined in the model output. The text says time progression x4i is 'the number of days since the first observed competition,' but the reported estimate γ04 = 0.0057 would then imply that each additional day increases log best time by 0.0057 (about 0.57% per day), which is implausibly large and inconsistent with the record-breaking trend described in Section 4. Similarly, γ03 = -0.0931 for age in years would imply about 9% faster times per additional year of age, which is also implausible over adult athletes' careers. The authors should state the scaling (e.g., days, years, or standardized values) and, if the variables are not on their natural scales, explain. This clarification matters for interpreting the fixed effects and for assessing whether the model is correctly specified.","section":"Table 3 (fixed effects)"}],"minor_comments":[{"comment":"The first paragraph refers to 'the model used to analyze the effect of the Tomoa Skip on speed climbing times in Section 3.2'; this should be Section 3.1.","section":"Section 3.2"},{"comment":"The axis label 'Worst - Best Times (s)' appears to be reversed relative to the text, which defines the range as 'best time minus their worst time'; please clarify the intended direction.","section":"Figure 5"},{"comment":"The random effect correlations η01 and τ01 are -0.98 and -0.99, respectively; such near-perfect correlations can indicate non-identifiability or overparameterization and deserve a brief comment.","section":"Table 3"},{"comment":"The sentence 'In the world of speed climbing, where a fraction of a second can mean the difference between success and failure, this seemingly small improvement constitutes a significant and impactful advancement' could be strengthened by quantifying the effect relative to within-climber variability or the historical rate of record progression.","section":"Section 3.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for stat.AP. The central estimate is plausible, but the omitted individual-specific time trend is a load-bearing concern that can be addressed with a straightforward robustness check. I do not see circularity or self-citation issues. If the refit confirms the stability of γ01, the paper could become acceptable after revision."},"author_rebuttal":null,"desk_editor":null,"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Using the Tomoa Skip in speed climbing is associated with roughly a 15 percent reduction in a climber's best time, after accounting for gender, age, and time trends in a mixed-effects model of IFSC results from 2012 to 2022.","keywords":["speed climbing","Tomoa Skip","mixed effects model","sport statistics","Olympic climbing","performance analysis","technique innovation","IFSC"],"falsifier":"Obtain full per-attempt Tomoa Skip usage for all rounds (not just finals) of IFSC speed events from 2018-2022, re-estimate the paper's mixed model with the actual usage indicator instead of the forward-filled one, and check whether the Tomoa Skip coefficient's 95% confidence interval still excludes 1. If it includes 1 (no multiplicative change), the central claim is falsified.","tokens_in":8088,"feed_emoji":"🧗","tokens_out":7016,"duration_ms":66912,"temperature":0.7,"pith_summary":"The paper sets out to measure whether the Tomoa Skip, a dynamic step-up move that bypasses the second handhold of the 15-meter speed wall, made climbers faster and whether it made them less reliable. Watching 54 broadcast finals and combining that with IFSC results, the authors fit mixed-effects models to each climber's best time per event. They find that using the skip is associated with a multiplicative time change of 0.8549 (95% CI 0.8271, 0.8836), i.e., about 14.5% faster, after controlling for gender, age, and the general historical improvement in times. They do not find a statistically significant relationship between skip usage and falling in finals. If the estimate is right, the move that boulderer Tomoa Narasaki introduced into speed climbing offers a large competitive edge and helps explain the rapid succession of world records after 2018.","feed_headline":"Tomoa Skip cuts speed-climb times by about 15%","feed_subtitle":"Analysis of 2012-2022 IFSC results links the move to faster records and no more falls.","key_machinery":"The argument is carried by a two-level mixed-effects model fitted to log-transformed best times, with random intercepts and random slopes for Tomoa Skip usage varying by both climber and event. Exponentiating the fixed coefficient for skip usage yields the multiplicative effect on time, and the log link is what turns the model into a statement about percentage change. The predictor itself was built by manually labeling skip use from 54 IFSC broadcast finals and then forward-filling the binary indicator: a climber observed doing the Tomoa Skip in one final is assumed to use it in all later events (and in unobserved qualifying rounds), with the reverse applied for the one climber documented switching away from it.","core_discovery":"On the paper's own terms, the central discovery is that the Tomoa Skip confers a substantial speed advantage: the estimated fixed effect of using the move is -0.1568 on the log-time scale, which back-transforms to a 0.8549-times multiplicative change in a climber's expected best time (95% CI 0.8271 to 0.8836). Interpreted concretely, a 7-second climb without the skip is predicted to become roughly 5.98 seconds with it. This estimate comes from a model that includes random intercepts and random skip slopes for both climbers and events, together with fixed effects for gender, age, and a time-progression variable, and the outcome is the logarithm of the climber's fastest time in any round of each event. The paper also reports that in a binomial generalized mixed model, skip usage has no statistically significant association with the probability of falling in a final round, although descriptive plots show wider within-event time ranges for early skip adopters.","pith_inferences":["The forward-filling assumption is the main vulnerability: if climbers sometimes try the skip in finals but abandon it in qualifying or later events, the predictor is misclassified and the estimated effect could be biased in either direction; a testable extension is to label every attempt from full-event video and re-estimate with a time-varying skip indicator.","Because skip adoption is voluntary, the model does not fully rule out selection bias—stronger, more adaptive climbers may have adopted the move earlier—so a within-climber pre/post design with event fixed effects would be a sharper causal test.","The paper's consistency analysis conflates within-event range with consistency; a variance-model extension (e.g., a mixed-effects location-scale model) could formally test whether skip users have higher performance variance, not just wider ranges in early events.","The record-breaking narrative is descriptive; a changepoint or intervention analysis on world-record times could statistically separate the skip's introduction from the general training and hold-technology improvements."],"forward_implications":["If the association is causal, climbers who have not adopted the skip are racing with an effective handicap of roughly 15% on their best time, far larger than the margins that decide finals.","Because adoption was still incomplete at the end of 2022, the average effect estimated on early adopters implies the sport's baseline speed should keep shifting downward as the move becomes universal.","The absence of a significant fall effect suggests the skip is not a riskier strategy on average, which supports its use as a standard technique rather than a gamble.","The finding implies the Olympic combined format, by forcing boulderers into speed climbing, produced a lasting technique transfer that likely contributed to the post-2018 wave of world records."],"supporting_citations":[{"why":"Supplies the mixed-models-in-sport method the paper adopts for longitudinal performance data.","marker":"Newans et al. 2022"},{"why":"Demonstrates mixed linear modeling for monitoring acute effects on athletic performance, the analytical template for the paper's models.","marker":"Vandenbogaerde & Hopkins 2010"},{"why":"Provides the model specification and notation reference for the mixed-effects framework used throughout.","marker":"Clark 2022"}],"fun_headline_variants":["Tomoa Skip shaves 15% off speed-climbing times","Speed climbing's Tomoa Skip cuts times by 15%","The Tomoa Skip: a 15% faster speed climb","How one move made speed climbers 15% faster","Tomoa Skip's 15% edge in speed climbing"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central assumption is that a climber documented using the Tomoa Skip in one broadcast final is using it in every other round and future competition; if usage actually varies, the main predictor is misclassified and the estimated 0.8549 time multiplier could be biased.","fun_headline_variants_meta":{"raw":{"variants":["Tomoa Skip shaves 15% off speed-climbing times","Speed climbing's Tomoa Skip cuts times by 15%","The Tomoa Skip: a 15% faster speed climb","How one move made speed climbers 15% faster","Tomoa Skip's 15% edge in speed climbing"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000208,"raw_usage":{"total_tokens":1453,"prompt_tokens":1044,"completion_tokens":409,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":660,"completion_tokens_details":{"reasoning_tokens":324}},"tokens_in":660,"tokens_out":409,"duration_ms":4358,"temperature":1.0,"reasoning_tokens":324,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:59:24.641172+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Obtain full per-attempt Tomoa Skip usage for all rounds (not just finals) of IFSC speed events from 2018-2022, re-estimate the paper's mixed model with the actual usage indicator instead of the forward-filled one, and check whether the Tomoa Skip coefficient's 95% confidence interval still excludes 1. If it includes 1 (no multiplicative change), the central claim is falsified.","supporting_citations":[],"review_version":1}