REVIEW 3 major objections 5 minor 2 references
Analysis of points outcome in ATP Grand Slam Tennis using big data and machine learning
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read ML point prediction in tennis can't beat a serve-win average
desk verdict Honest negative benchmark undermined by target leakage in the feature set; the accuracy comparisons are salvageable, the strategic-factor conclusions are not. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is a point-winner classification model built separately for first and second serves. Each row is meant to describe the match state up to the serve, with previously played points accumulated into features and the serving player normalized to player 1. The baseline is the share of serves won by the server in the training data, and the machine-learnt models are judged against that baseline. Feature contribution is measured by XGBoost gain, which ranks how much each feature reduces loss across trees. This machinery lets the paper separate the marginal value of predictive features from the strong class prior in tennis points.
What would settle it
Retrain the same tuned XGBoost pipeline on the same data with every post-point feature removed (return depth, rally count, distances run, winner shot type, and similar columns), then compare accuracy and feature gains; a large change in either would show the paper's placement-feature conclusion is an artifact of outcome leakage.
Extended reading notes
Core claim
The central claim is that, with public point-level data, point-outcome prediction offers no practical gain over the serve-win average and cannot identify strategic factors. On first serves, where the server wins 73.2% of points, the tuned XGBoost (extreme gradient boosting) model scores 73.4% while almost never predicting the returner; on second serves, where the server wins 57.2%, the tuned model scores 53.1%, below baseline. Four models (logistic regression, random forest, AdaBoost, and XGBoost) perform nearly identically in cross-validation. The paper concludes that accumulated score features and player rankings carry whatever predictive signal exists, that serve and return placement features do not, and that the approach is not optimal when opponents are at the same ranking position.
Load-bearing premise
The paper's feature-importance conclusions assume that each row contains only information known before the serve, even though table features such as return depth, rally count, distance run, and winning shot type can only be known after the point is played.
Editorial extensions
If this is right
- If public point-level data cannot beat a serve-win average, any credible point-outcome or tactics model needs finer-grained data, such as higher-resolution tracking data.
- Serve or return placement, as encoded in this dataset, contributes no measurable predictive signal for who wins a point, so strategic claims drawn from placement would not be supported by these data.
- For first-serve points, the practical prediction is essentially the server wins, and the class imbalance makes accuracy a weak measure of model skill.
- For second-serve points, the more balanced classes reveal that the models actually underperform the average, meaning public data adds no value for this harder question.
- Feature-importance rankings pointing to sets won, break points, and rankings suggest match context, not shot placement, drives whatever signal exists.
Reading between the lines
- A natural extension is to retrain the same pipeline using only features truly known before the serve; if accuracy and feature-importance rankings change, the paper's conclusion about placement features would reflect data leakage rather than tennis.
- A testable next step is to apply the same first-serve/second-serve split and baseline comparison to a spatiotemporal dataset; a model that beats the baseline on second serves would indicate the limiting factor is data, not method.
- The accuracy metric on first serves hides that the model almost never predicts the returner; future comparisons of point-prediction models should also report recall, F1, or ROC-AUC.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies point-level outcomes in 709 men's Grand Slam singles matches (Wimbledon and US Open, 2016-2020) using the public Sackmann point-by-point data. It builds separate classification models for points starting with a first serve and a second serve, using logistic regression, random forest, AdaBoost, and XGBoost, and compares them to a baseline defined as the empirical server-win proportion. The tuned XGBoost first-serve model achieves 73.4% accuracy against a 73.2% baseline; the second-serve model achieves 53.1% against a 57.2% baseline. The authors also use XGBoost gain-based feature importance to conclude that serve and return placement features have no significant influence on point outcomes, and that the public data are insufficient for point-outcome prediction beyond serve-win averages.
Significance. The paper's strengths include its use of publicly available data, the comparison of four modeling approaches, and the honest reporting of negative results, including the fact that the second-serve model underperforms its baseline and that the first-serve model rarely predicts the returner. The GitHub repository makes the code available. If the leakage concerns below are resolved, the finding that public point-level data do not beat simple serve-win averages is a useful negative result for the sports-analytics community. However, the central interpretive claim about strategic factors rests on a feature-importance analysis that is invalidated by potential target leakage and by the degenerate behavior of the first-serve model; as written, the paper's main conclusions are not reliable.
major comments (3)
- [§3.1.1, Table 1] The paper does not establish that the models are trained only on information available before the point is played. Table 1 lists ReturnDepth, RallyCount, P1DistanceRun, P2DistanceRun, and WinnerShotType as model features, yet these are realized only after the serve is made; the text only states that columns 'containing information regarding the outcome of a point' are shifted one row up, and it explicitly excludes only direct outcome indicators such as P1UnfErr. If any of these five features enter as current-point values, the model receives target-proximal information, so the feature-importance analyses in Figures 4 and 5 and the conclusion in §5 that serve and return placement features had no significant influence are unsupported.
- [§4.1, Figures 4-5] For the first-serve model, the tuned XGBoost has recall of 1.2%, meaning it almost always predicts the majority class; feature-importance (Gain) values computed from such a degenerate model cannot support claims about which pre-point variables do or do not matter. Low gain is not evidence of absence of influence when features are correlated and the model has collapsed to the majority class, so the discussion in §4.1.1 and §5 over-interprets the importance plots.
- [§2.1, Tables 3 and 5] The baseline is the empirical serve-win proportion computed on the same data used to fit the models, and the reported differences (73.4% versus 73.2% for the first serve, and 53.1% versus 57.2% for the second serve) are presented without confidence intervals or any significance test. The claim that the first-serve model is 'a fraction better than the average' is therefore not established; the observed difference is well within plausible sampling noise, and the comparison should be accompanied by a paired or matched uncertainty estimate.
minor comments (5)
- [Throughout] There are numerous typographical and grammatical errors (e.g., 'Rafeal Nadal', 'Altough', 'inform of', 'when when player two is serving', 'Paramter') that should be corrected before resubmission.
- [§3.2, Figures 1-2] The figure captions and text do not make it possible to verify the claimed percentages of serves 'close to the sidelines' because the court diagrams are not labeled with the underlying coordinate mapping; a clear description of how ServeWidth and ServeDepth were converted to court positions is needed.
- [§3.3, Table 1] The hyperparameters found by random search for the tuned XGBoost models are not reported in the paper; the GitHub repository is cited, but the paper should state the final hyperparameter values so the results are reproducible without external code.
- [§4, Feature importance] The feature-importance figures (Figures 4 and 5) show only features with gain above a threshold of 0.01, and the text does not define how gain is computed or why this threshold was chosen; this should be stated explicitly.
- [§3, Data preparation] The paper should report the number of points, after data cleaning, for the first-serve and second-serve datasets, and the exact composition of the train/validation/test split by match, so that the reader can assess the effective sample size.
Circularity Check
No circular derivation: the point-winner models are empirical classifiers tested on held-out matches against a constant baseline; the only self-citation is non-load-bearing.
full rationale
The claimed derivation is an empirical pipeline: public point-level data are restructured so that each row contains information known before the serve (§3.1.1), direct outcome columns such as P1UnfErr are excluded, accumulated and match features are used as inputs, models are trained on 90% of matches with 10-fold cross-validation, tuned on validation data, and finally evaluated on a held-out 10% of matches (Tables 2–5). The baseline is a constant class proportion ('number of times the player wins the serve divided by the total amount of serves'), not a parameter fitted to the prediction target, so comparing model accuracy to it is not an example of a fitted input being renamed a prediction. The target PointWinner is not used as a feature; outcome-related columns are said to be shifted one row up so that the row contains information up until the serve. The concern that post-point features such as ReturnDepth, RallyCount, P1DistanceRun, P2DistanceRun, or WinnerShotType might leak the label is a data-validity and correctness risk rather than a circularity in the derivation sense, and the text asserts that outcome columns are lagged. The only self-citation is Illum (2022), the authors' GitHub repository, used for the random-search hyperparameter procedure; it is not load-bearing for any theoretical or empirical claim, and the code is externally inspectable. No self-definitional identity, uniqueness-theorem import, or ansatz-via-citation chain was found. The paper's negative conclusions about predictive power and feature importance are honest empirical outcomes, not results forced by construction.
Assumptions & free parameters
free parameters (4)
- XGBoost hyperparameters (first serve) =
scale_pos_weight = 1.3; other values from random search not reported
- XGBoost hyperparameters (second serve) =
not reported
- Baseline serve-win percentages =
73.2% first serve, 57.2% second serve
- Train/validation/test split proportions =
90% train+validation, 10% test; within train+validation, 80/20 split
assumptions (4)
- domain assumption The Sackmann point-by-point data are accurate and complete for the selected matches, and the serve placement categories (ServeWidth, ServeDepth) correctly encode actual placement.
- domain assumption All features used to predict a point are known before the point is played.
- domain assumption Matches with missing serve placement data are missing at random and can be excluded without biasing the results.
- domain assumption The 10-fold cross-validation respects match boundaries, so points from the same match do not appear in both training and validation folds.
Cite this review
Pith. "Pith review of Analysis of points outcome in ATP Grand Slam Tennis using big data and machine learning." pith.science (2026). https://pith.science/paper/TZGAXWY7
@misc{pith2026250605866,
author = {Pith},
title = {Pith review of: Analysis of points outcome in ATP Grand Slam Tennis using big data and machine learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/TZGAXWY7}},
note = {Machine review of arXiv:2506.05866}
}
read the original abstract
Tennis is one of the world's biggest and most popular sports. Multiple researchers have, with limited success, modeled the outcome of matches using probability modelling or machine learning approaches. The approach presented here predicts the outcomes of points in tennis matches. This is based on given a probability of winning a point, based on the prior history of matches, the current match, the player rankings and if the points are started with a first or second. The use of historical public data from the matches and the players' ranking has made this study possible. In addition, we interpret the models in order to reveal important strategic factors for winning points. The historical data are from the years 2016 to 2020 in the two Grand Slam tournaments, Wimbledon and US Open, resulting in a total of 709 matches. Different machine learning methods are applied for this work such as, e.g. logistic regression, Random forest, ADABoost, and XGBoost. These models are compared to a baseline model, namely a traditional statistics measure, in this case the average. An evaluation of the results showed that the models for points proved to be a fraction better than the average. However, with the applied public data and the information level of the data, the approach presented here is not optimal for predicting who wins when the opponents are on the same position on the ranking. This methodology is interesting with respect to examining which factors are important for the outcomes of who wins points in tennis matches. Other higher quality data sets exists from e.g. Hawk Eye, although these data sets are not available for the public.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
Application of the Logistic Function to Bio-Assay
Berkson, Joseph (1944). “Application of the Logistic Function to Bio-Assay”. In: Journal of the American Statistical Association 39.227, pp. 357–365. ISSN : 01621459. URL: http://www.jstor.org/stable/2280041 (visited on 08/13/2022). Breiman, Leo (2001). Random Forests. Tech. rep. none, pp. 5–32. Carboch, Jan (Mar. 2017). “Comparison of game characteristic...
-
[2021]
A Statistical Model of Serve Return Impact Patterns in Professional Tennis
DOI: 10.1162/99608f92.b665c0f4. URL: https://hdsr.mitpress.mit.edu/pub/uy0zl4i1. Kovalchik, Stephanie A. and Jim Albert (2022). “A Statistical Model of Serve Return Impact Patterns in Professional Tennis”. In: none. DOI: 10.48550/ARXIV.2202.00583. URL: https://arxiv.org/abs/2202.00583. Kovalchik, Stephanie Ann (2016). “Searching for the GOAT of tennis win...
work page Pith review arXiv doi:10.48550/arxiv.2202.00583 2022
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.