Partially Regularized Ordinal Regression to Adjust Teams' Scoring for Strength of Schedule and Complementary Unit Performance in American Football

Andrey Skripnikov; Sujit Sivadanam

REVIEW 2 major objections 7 minor 22 references

Partially Regularized Ordinal Regression to Adjust Teams' Scoring for Strength of Schedule and Complementary Unit Performance in American Football

T0 review · 2 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Complementary football is measurable: turnovers and field position shift next-drive scoring.

desk verdict A solid applied paper with a genuine method tweak and a plausible headline finding, but the causal reading of the complementary-football effects needs to answer an endogeneity challenge before I'd trust the projections. read the letter →

arxiv 2506.03057 v1 pith:JHVJO5NZ submitted 2025-06-03 stat.AP

Andrey Skripnikov , Sujit Sivadanam This is my paper

classification stat.AP

keywords complementaryfootballordinalregressionelasticnetstrengthofscheduledrive-by-driveanalysisAmericananalyticsnon-scoringturnoversfieldposition

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

American football's offense and defense rarely share the field, so analysts often rate them separately. This paper argues that the two units are linked through the previous drive: when a defense forces a turnover or leaves the offense near midfield, the next drive becomes easier to score. Using drive-by-drive data from nine NFL seasons and seven FBS college seasons, it fits a partially regularized ordinal regression to five scoring outcomes, separating strength-of-schedule team effects from penalized complementary-football effects. Non-scoring turnovers and post-turnover starting field position emerge as the features that consistently matter, and adding them to the model improves out-of-sample prediction of points per drive. The practical payoff is a rating system that projects each offense and defense onto a league-average complementary unit, changing some team rankings.

What carries the argument

The machinery is a cumulative-logit ordinal regression with partial regularization: drive outcomes are ordered as defensive touchdown, safety, no score, field goal, and offensive touchdown, and the log cumulative odds are modeled through unpenalized team offense and defense margins, home-field and game-context terms, plus complementary statistics from the preceding drive such as a turnover indicator and post-turnover starting position. An elastic-net penalty with a partially relaxed proportional-odds assumption selects which complementary features matter, while a binarized surrogate residual check verifies fit for each cumulative-odds equation separately.

What would settle it

Refit the main model with each previous-drive complementary statistic instrumented by a source of variation the offense cannot control, such as weather, referee crew, or the opponent's health, and check whether the turnover and field-position coefficients collapse toward zero.

Watch

Extended reading notes

Core claim

The central claim is that complementary football is measurable and consequential: after accounting for opponent strength, home field, and game context, the defense's non-scoring turnovers and the field position they create for the next drive are the stable features that raise the offense's probability of scoring, especially the chance of scoring at least a field goal. The paper shows this with a flexible ordinal model in which team-strength coefficients stay unpenalized, so strength of schedule is fully adjusted, while complementary-feature coefficients are selected under a near-LASSO elastic-net penalty that also lets some effects vary by scoring category. When team performances are projected onto a league-average complementary unit, offenses backed by good defenses are downgraded and offenses backed by poor defenses are upgraded, with larger single-game shifts than season-long shifts. Out of sample, the model with complementary features beats models using only game context and strength of schedule on mean absolute error of expected points per drive.

Load-bearing premise

The model assumes the previous drive's complementary statistics, especially turnovers and starting field position, are exogenous to the current drive's scoring once team, home-field, and game-context effects are controlled, even though those statistics are produced by the opponent's offense against the team's own defense.

Editorial extensions

If this is right

Team offensive and defensive ratings become conditional statements: they describe scoring ability given a league-average complementary unit, not scoring ability in isolation.
Rankings can shift when complementary-unit quality is accounted for, as with the 2012 Giants dropping from third to eighth and the 2011 Texans from seventh to thirteenth in NFL offensive points per drive.
The value of turnovers comes largely from starting field position near field-goal range, so models that count turnovers without field position miss most of the effect.
Complementary adjustments are larger for single games than for full seasons, making them especially relevant for game-level evaluation.
Strength-of-schedule adjustments remain the dominant season-long correction, especially in college football, but complementary features add predictive value on top.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the exogeneity assumption fails, meaning a team's own previous-drive performance correlates with the complementary statistics it faces, the estimated turnover and field-position effects would partly reflect team quality and the projected ranking shifts would need re-interpretation.
The same modeling template could transfer to other sequential team sports where a defensive stop creates possession, such as rugby, though the two-way-player caveat makes the complementary-unit adjustment conceptually messier.
Because point values are assigned only after the ordinal model is fit, an ordinal likelihood that incorporates actual point values could sharpen estimates at the extremes such as safeties and two-point conversions.
Comparing these complementary-unit-adjusted ratings against sportsbook or market point spreads would show whether the adjustment is already priced into public evaluations.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Worth a look if you care about ordinal regression in sports or about adjusting team ratings for context. The methodological contribution is real but modest: a partial elastic-net ordinal regression that lets you keep proportional odds for some covariates while relaxing it for others, built on the ordinalNet framework. That extension is sensible and clearly explained, and the binarized surrogate diagnostics are a reasonable adaptation of Liu and Zhang. The empirical finding also holds up as a descriptive matter: non-scoring turnovers and post-turnover starting position are the most consistently selected complementary features across both NFL and FBS, and adding them improves out-of-sample prediction of drive points, albeit modestly. The projections onto a league-average complementary unit are transparent, with bootstrap intervals, and the code and data are public. Credit where due: the authors did the work, show the robustness, and don't oversell the magnitude of the complementary adjustment relative to strength of schedule.

The soft spots are real. The main one, which the stress-test note lands, is that the lagged complementary statistics in Eq. (1) are generated by the opponent's offense against the team's own defense, but the model only includes team i's offensive strength and team j's defensive strength. If a team's offensive and defensive qualities are correlated—or the opponent's offensive and defensive qualities are—the gamma coefficients for turnovers and field position will soak up team-quality correlation, not just the sequential complementary effect. The paper's language ("impacting the complementary unit" and the league-average projection in Section 2.7) leans causal, and that reading needs more support than the cross-validation provides. This isn't fatal to the predictive finding, but it is load-bearing for the projections. A robustness check that controls for team i's defensive strength (or uses a more plausibly exogenous event like turnovers on downs) would go a long way. The 35% college-half missingness is a secondary concern; the authors check that it is consistent across seasons and conferences, but not whether it relates to scoring outcomes. Also, the numeric CV results are only plotted, not tabulated, and I did not execute the code, so I can't independently confirm the magnitudes.

Who is this for? Sports-analytics practitioners and anyone fitting ordinal models with many nuisance parameters. It deserves serious peer review—the methodology is clearly presented, the data work is careful, and the empirical finding is interesting enough to warrant referee time. I would recommend acceptance contingent on addressing the exogeneity concern and reporting the missingness analysis.

Referee Report

2 major / 7 minor

Summary. The paper develops a partially regularized ordinal regression model for drive-level scoring in American football. Using play-by-play data from the NFL (2009–2017) and FBS college football (2014–2020), the model includes unpenalized team offensive and defensive ratings for strength-of-schedule adjustment, home-field and game-context controls, and elastic-net-penalized lagged "complementary football" statistics from the preceding drive, with non-proportional odds allowed for the penalized features. The authors select non-scoring turnovers and post-turnover starting position as the most consistent complementary features, project each team's expected points per drive onto a league-average opponent, complementary unit, and home/away split, and compare out-of-sample mean absolute error of models with and without complementary features. They also extend the ordinalNet package and the surrogate residual approach to the partial non-proportional odds setting.

Significance. If the identification concerns are addressed, the paper would offer a practical adjustment for complementary football in team ratings and a useful extension of partial regularization for ordinal outcomes with many unpenalized fixed effects. Strengths include the use of drive-level sequential data, two-league replication, selection-stability analysis across CV replicates, out-of-sample evaluation, bootstrap confidence intervals for projections, and public data and source code on GitHub. The methodological modifications to ordinalNet and the surrogate residual diagnostics are clearly described and could benefit other applications.

major comments (2)

[Sections 2.5, 2.6, and 2.8] The exogeneity assumption for the lagged complementary statistics is load-bearing but untested. In Eq. (1), x*_ji(k-1)l is produced by team j's offense against team i's defense, yet the model includes only team i's offensive fixed effect α_i and team j's defensive fixed effect β_j. Team i's defensive quality and team j's offensive quality are omitted. Consequently, the γ and γ_s coefficients may absorb persistent correlation between a team's offensive and defensive strengths, or game-script effects, rather than a purely sequential complementary effect. The paper's causal language in the Discussion ("the football statistics that proved to be most important in impacting the complementary unit") and the projection of teams onto a league-average complementary unit in Section 2.7 require that x* be exogenous conditional on the included controls. This is not established. A concrete robustness check would be to refit the model adding team j's offensive fixed effects and team i's defensive fixed effects, or to run a placebo using future (lead) drives instead of the lag, and to report whether the selected features and projection shifts survive.
[Sections 2.5, 2.6, and 2.8] The feature selection and the out-of-sample evaluation are not nested. In Section 2.5, the complementary features are selected using 10-fold cross-validation within each season; Section 2.6 then refits the model on the same data using only the selected features; Section 2.8 evaluates a fresh 10-fold CV of models that include that pre-selected feature set. Because the feature choice was made using information from all folds in the season, the mean absolute error comparison in Figure 5 and the bootstrap intervals in Tables 2 and 3 do not account for selection uncertainty and may overstate the predictive gain from complementary features. The authors should either use nested CV for the final comparison or hold out entire seasons, and they should report the sensitivity of the bootstrap intervals to the selection step.

minor comments (7)

[Eq. (2)] The ridge penalty term contains a misplaced parenthesis: it should read ½(1−α)||γ||₂², not ½(1−α||γ||₂²).
[Section 2.5] The robustness of variable selection to the choice of α is only cited to the supplementary materials; a small table or figure in the main text showing selection stability across the α grid [0.8, 0.9, 0.95, 0.99, 0.999, 0.9999] would strengthen the claim.
[Section 2.1] The exclusion of 35% of college halves is a substantial data loss; although the authors note the proportion is consistent across seasons and conferences, they should state explicitly in Section 4 that the college-specific projections may be affected by non-random missingness and discuss what sensitivity analyses, if any, were considered.
[Figure 5 caption] The caption uses "GS" for "game context with strength of schedule" while the text and figure key use "GC"; please unify the abbreviations.
[Section 2.4] The statement that game-context variables were "statistically significant at the 0.05 level ... for most NFL and college football seasons" is vague; report the exact number of seasons out of nine (NFL) and seven (college) for which each variable was significant.
[Section 2.2 and Table 1] Standardizing all offensive and defensive touchdowns to 7 points is a simplifying approximation; the manuscript should note explicitly that this collapses the rare 6- and 8-point outcomes and could slightly affect the expected-points weighting and the proportional-odds interpretation.
[References] Several references, especially Oliver (2011) and Horowitz (2018), lack full retrieval information (URLs or access dates); please complete them for reproducibility.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the turnover/field-position findings and out-of-sample gains are genuine CV quantities, and the only self-citation is a non-load-bearing replicability note.

full rationale

The derivation chain is self-contained. Equation (1) is a standard cumulative-logit model in which the current-drive scoring outcome Y_{ijkl} is regressed on team offensive/defensive fixed effects (alpha_i, beta_j), home-field, game-context controls, and centered statistics from the immediately preceding drive (x*_{ji(k-1)l}); these features are predetermined covariates, not functions of the response, so no construction imposes the turnover or field-position result. Feature importance is measured by selection stability across three replicated 10-fold CV partitions per season, and the headline conclusion (non-scoring turnovers and post-turnover starting position are most consistently selected) is a genuine hold-out quantity. The out-of-sample gain in Figure 5 is likewise computed on held-out folds against baseline models that omit complementary features; this is an empirical comparison, not a fitted value relabeled as a prediction. The team-rating shifts in Tables 2-3 are explicitly described as projections of the fitted model onto league-average complementary-unit values, and the paper transparently states in the future-work paragraph that point-value weights are applied post hoc when converting fitted probabilities to expected points; this is model interpretation, not circularity. The only self-citation (Skripnikov 2023) appears in the introduction and discussion as context and as a replicability note comparing game-total versus drive-level findings; the present analysis does not use that prior work as a premise, and the data/code are public, so the citation is not load-bearing. Flagged caveats that keep the score slightly above zero: the TO.nonscor:start.pos interaction was engineered after exploratory evidence (Figure 1b) that starting position matters, so its 100% selection rate is partly confirmatory of the same data; the 10-fold CV in Section 3.5 evaluates a feature set chosen on the same seasons, so the reported improvement carries mild selection optimism; and the lagged complementary statistics x* are produced by the opponent against the current team's own defense, an endogeneity concern for any causal reading. These are correctness/identification limitations, not construction-level circularity: no equation reduces to its own input, and no fitted parameter is presented as an independent prediction.

Assumptions & free parameters 5 free parameters · 8 assumptions · 0 invented entities

The central empirical claim rests on the ordinal model specification, exogeneity of previous-drive features, missing-at-random college data, and the partial proportional odds restriction. No new physical or theoretical entities are postulated. The main hand-chosen tuning constants are alpha equal to 0.99, sparse-CV lambda, the 0.05 game-context threshold, touchdown standardization to 7 points, and the 5-game college threshold.

free parameters (5)

Elastic net mixing weight alpha = 0.99
Section 2.5. Chosen among a grid to keep the LASSO emphasis while guaranteeing a unique optimum; robustness across alpha values is asserted but not shown in the main text.
Sparse cross-validation lambda = Season-specific, not reported
Section 2.5. Largest lambda within one standard error of the best 10-fold cross-validation multinomial log-likelihood; directly controls which complementary features survive selection.
Minimum games threshold for college projections = 5 games
Section 3.4. Hand-set to guarantee at least 60 drives per team; changes which teams enter the college shift tables.
Touchdown point standardization = 7 points
Section 2.2. Offensive and defensive touchdowns are set to 7 points even though 6 or 8 points occur; affects expected points per drive calculations.
Game-context variable inclusion significance level = 0.05
Section 2.3. Game context variables are kept if significant at the 0.05 level in most seasons; this selection is made before complementary feature selection.

assumptions (8)

domain assumption Cumulative link ordinal regression is the correct generating model for drive scoring outcomes.
Section 2.2 and model (1): the five scoring outcomes are treated as ordered categories with a multinomial likelihood instead of linear regression.
domain assumption Previous-drive complementary statistics are exogenous to current-drive scoring conditional on team, home-field, and game-context controls.
Section 2.3, Equation (1). The regressor x* enters directly; the paper does not discuss endogeneity or instrument the complementary features.
domain assumption Excluded college halves are missing at random with respect to team strength and complementary effects.
Section 2.1: 65% of college halves are retained; the authors note no conference contributes disproportionately but do not test outcome-related missingness.
domain assumption Proportional odds holds for team, home-field, and game-context coefficients; only complementary features get non-proportional coefficients.
Section 2.5: partial relaxation is imposed for computational feasibility, not tested for the constrained coefficients.
standard math Centering complementary statistics by league means gives intercept and team coefficients an interpretation as projections onto a league-average complementary unit.
Section 2.3: algebraic consequence of mean-centering the complementary statistics.
domain assumption Selection by sparse cross-validation followed by unpenalized refit yields honest estimates of the selected effects.
Sections 2.5 and 2.6: standard practice, but inference on selected coefficients ignores selection uncertainty.
standard math The elastic net with alpha equal to 0.99 has a unique optimum.
Section 2.5: uniqueness result from Wurm et al. 2021 for alpha values strictly less than 1.
domain assumption Drive-level observations are conditionally independent.
Section 2.4: likelihood ratio test with a game random intercept found no evidence of within-game dependence; independence is assumed in the final model.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Partially Regularized Ordinal Regression to Adjust Teams' Scoring for Strength of Schedule and Complementary Unit Performance in American Football." pith.science (2026). https://pith.science/paper/JHVJO5NZ

@misc{pith2026250603057,
  author       = {Pith},
  title        = {Pith review of: Partially Regularized Ordinal Regression to Adjust Teams' Scoring for Strength of Schedule and Complementary Unit Performance in American Football},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JHVJO5NZ}},
  note         = {Machine review of arXiv:2506.03057}
}

read the original abstract

American football is unique in that offensive and defensive units typically consist of separate players who don't share the field simultaneously, which tempts one to evaluate them independently. However, a team's offensive and defensive performances often complement each other. For instance, turnovers forced by the defense can create easier scoring opportunities for the offense. Using drive-by-drive data from 2014-2020 Division-I college football (Football Bowl Subdivision, FBS) and 2009-2017 National Football League (NFL) seasons, we identify complementary football features that impact scoring the most. We employ regularized ordinal regression with an elastic penalty, enabling variable selection and partially relaxing the proportional odds assumption. Moreover, given the importance of accounting for strength of the opposition, we incorporate unpenalized components to ensure full adjustment for strength of schedule. For residual diagnostics of our ordinal regression models we apply the surrogate approach, creatively extending its use to non-proportional odds models. We then adjust each team's offensive (defensive) performance to project it onto a league-average complementary unit, showcasing the effects of these adjustments on team scoring. Lastly, we evaluate the out-of-sample prediction performance of our selected model, highlighting improvements gained from incorporating complementary football features alongside strength-of-schedule adjustments.

Figures

Figures reproduced from arXiv: 2506.03057 by the authors.

**Figure 2.** Binarized surrogate residuals-vs-fitted plots (top row) [PITH_FULL_IMAGE:figures/full_fig_p022_2.png] view at source ↗

**Figure 3.** (a): Relationship between the offense’s points per drive sc [PITH_FULL_IMAGE:figures/full_fig_p023_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: The magnitudes of shifts in points scored per drive based o [PITH_FULL_IMAGE:figures/full_fig_p029_4.png]

**Figure 5.** Figure 5: The 10-fold cross-validation results are presented to co [PITH_FULL_IMAGE:figures/full_fig_p030_5.png]

**Figure 6.** Figure 6: (a): Proportion of cross-validation replicates for 2009- [PITH_FULL_IMAGE:figures/full_fig_p037_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

22 extracted references · 21 canonical work pages

[1]

Kharrat, and I

Boshnakov, G., T. Kharrat, and I. G. McHale (2017). A bivariate weibull count model for forecasting association football scores. International Journal of Forecasting\/ 33\/ (2), 458--466

2017
[2]

Christensen, R. H. B. (2022). ordinal---regression models for ordinal data. R package version 2022.11-16. https://CRAN.R-project.org/package=ordinal

work page 2022
[3]

Edwards, J. and D. Archambault (1979). The home field advantage. Sports, games, and play: Social and psychological viewpoints\/ , 409--438

work page 1979
[4]

Fox, J. and G. Monette (1992). Generalized collinearity diagnostics. Journal of the American Statistical Association\/ 87\/ (417), 178--183

work page 1992
[5]

Easwaran, J

Gilani, S., A. Easwaran, J. Lee, and E. Hess (2022). cfbfastr: The sportsdataverse's r package for college football data. R package version 1.9.0

work page 2022
[6]

Govan, A. Y., A. N. Langville, and C. D. Meyer (2009). Offense-defense approach to ranking team sports. Journal of Quantitative Analysis in Sports\/ 5\/ (1)

work page 2009
[7]

Greenwell, B. M., A. J. McCarthy, B. C. Boehmke, and D. Liu (2018). Residuals and diagnostics for binary and ordinal regression models: an introduction to the sure package. R J.\/ 10\/ (1), 381

work page 2018
[8]

Harville, D. (1977). The use of linear-model methodology to rate high school or college football teams. Journal of the American Statistical Association\/ 72\/ (358), 278--289

work page 1977

Show all 22 references

[9]

Horowitz, M. (2018). Detailed nfl play-by-play data 2009-2018. https://www.kaggle.com/datasets/maxhorowitz/nflplaybyplay2009to2016

2018
[10]

Kennedy, and A

Kempton, T., N. Kennedy, and A. J. Coutts (2016). The expected value of possession in professional rugby league match-play. Journal of sports sciences\/ 34\/ (7), 645--650

2016
[11]

Liu, D. and H. Zhang (2018). Residuals and diagnostics for ordinal regression models: a surrogate approach. Journal of the American Statistical Association\/ 113\/ (522), 845--854

2018
[12]

McGovern, C. (2013). A closer look at touchdowns in the red zone. https://www.footballoutsiders.com/stat-analysis/2013/closer-look-touchdowns-red-zone

2013
[13]

Oliver, D. (2004). Basketball on paper: rules and tools for performance analysis . Potomac Books, Inc

2004
[14]

Oliver, D. (2011). Guide to the total quarterback rating. ESPN. com\/

2011
[15]

R: A language and environment for statistical computing

R Core Team (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing. Version 4.0.3

2020
[16]

Shea, S. M. (2014). Basketball analytics: Spatial tracking . CreateSpace Independent Publishing Platform

2014
[17]

Skripnikov, A. (2023). Partially constrained group variable selection to adjust for complementary unit performance in american college football. Journal of Applied Statistics\/ , 1--15

2023
[18]

Vergin, R. C. and J. J. Sosik (1999). No place like home: an examination of the home field advantage in gambling strategies in nfl football. Journal of Economics and Business\/ 51\/ (1), 21--31

1999
[19]

Wilson, R. L. (2020). College football overtime outcomes: implications for in-game decision-making. Frontiers in Artificial Intelligence\/ 3 , 61

2020
[20]

Wurm, M. J., P. J. Rathouz, and B. M. Hanlon (2021). Regularized ordinal regression and the ordinalNet R package. Journal of Statistical Software\/ 99\/ (6), 1--42

2021
[21]

Ventura, and M

Yurko, R., S. Ventura, and M. Horowitz (2019). nflwar: a reproducible method for offensive player evaluation in football. Journal of Quantitative Analysis in Sports\/ 15\/ (3), 163--183

2019
[22]

Zou, H. and T. Hastie (2005). Regularization and variable selection via the elastic net. Journal of the royal statistical society: series B (statistical methodology)\/ 67\/ (2), 301--320

2005

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Kharrat, and I

Boshnakov, G., T. Kharrat, and I. G. McHale (2017). A bivariate weibull count model for forecasting association football scores. International Journal of Forecasting\/ 33\/ (2), 458--466

2017

[2] [2]

Christensen, R. H. B. (2022). ordinal---regression models for ordinal data. R package version 2022.11-16. https://CRAN.R-project.org/package=ordinal

work page 2022

[3] [3]

Edwards, J. and D. Archambault (1979). The home field advantage. Sports, games, and play: Social and psychological viewpoints\/ , 409--438

work page 1979

[4] [4]

Fox, J. and G. Monette (1992). Generalized collinearity diagnostics. Journal of the American Statistical Association\/ 87\/ (417), 178--183

work page 1992

[5] [5]

Easwaran, J

Gilani, S., A. Easwaran, J. Lee, and E. Hess (2022). cfbfastr: The sportsdataverse's r package for college football data. R package version 1.9.0

work page 2022

[6] [6]

Govan, A. Y., A. N. Langville, and C. D. Meyer (2009). Offense-defense approach to ranking team sports. Journal of Quantitative Analysis in Sports\/ 5\/ (1)

work page 2009

[7] [7]

Greenwell, B. M., A. J. McCarthy, B. C. Boehmke, and D. Liu (2018). Residuals and diagnostics for binary and ordinal regression models: an introduction to the sure package. R J.\/ 10\/ (1), 381

work page 2018

[8] [8]

Harville, D. (1977). The use of linear-model methodology to rate high school or college football teams. Journal of the American Statistical Association\/ 72\/ (358), 278--289

work page 1977