A Bayesian circular mixed-effects model for explaining variability in directional movement in American football

Quang Nguyen; Ronald Yurko

REVIEW 3 major objections 5 minor 29 references

A Bayesian circular mixed-effects model for explaining variability in directional movement in American football

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A Bayesian circular model of turn angles provides the first public in-game ranking of NFL ball carriers by change-of-direction variability.

desk verdict Useful new public metric for NFL shiftiness, but the differentiation claim is weakened by ignored frame-level autocorrelation and an untested interpretation. read the letter →

arxiv 2507.06122 v1 pith:K3GTUOQQ submitted 2025-07-08 stat.AP

Quang Nguyen , Ronald Yurko This is my paper · ORCID

classification stat.AP

keywords Bayesianstatisticscircularmixed-effectsmodelturnanglechangeofdirectionplayertrackingdataNationalFootballLeagueheterogeneity

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to quantify how variably NFL ball carriers change direction during real plays, using the angle between successive movement steps — the turn angle — as the frame-level measure. It fits a Bayesian circular mixed-effects model to turn angles for running backs after the handoff and receivers after the catch, modeling both the average turning direction and the concentration of turns. The quantity that carries the analysis is a player-specific random effect on the concentration parameter, which estimates whether a carrier turns more erratically or more predictably than peers in the same position group. If the model works, it would give teams and analysts the first public, in-game measure of change-of-direction skill, complementing the fixed drills of the Scouting Combine.

What carries the argument

The load-bearing object is the von Mises distribution for the frame-level turn angle $\phi_{ijt}$, with both parameters modeled: the mean $\mu$ through a tan-half link and the concentration $\kappa$ through a log link. The concentration equation includes movement and contextual covariates plus a ball-carrier random intercept $u_j$ whose variance $\sigma^2_{p[j]}$ is allowed to differ by position group. Because $\kappa$ is the inverse of directional variability, the posterior of $u_j$ directly ranks players by how consistently or erratically they turn, which is the paper's main output.

What would settle it

Fit the same model to a full season or to multiple seasons and check whether the player concentration random effects reproduce from one half of the data to the other; additionally, regress the player-level estimates on outcome measures such as yards after contact or missed tackles forced. If low-variability players outperform high-variability players on these outcomes, or if the player rankings do not replicate, the central claim about measuring shiftiness collapses.

Watch

Extended reading notes

Core claim

The paper's central claim is that the player-specific concentration random effects reliably separate NFL ball carriers by their turning variability, providing discriminative power for measuring change of direction in the NFL. Running backs emerge as the most homogeneous position group, while wide receivers and tight ends show wider player-to-player differences, consistent with their more unstructured post-catch movements. The player rankings align with known playing styles: receivers known for straight-line speed rank low in variability, while players known as shifty rank high. The paper treats these rankings as evidence that turn-angle variability is a usable, objective in-game signature of a player's ability to make sudden, unpredictable directional adjustments.

Load-bearing premise

The rankings only mean what the paper says they mean if more variable turning is actually a beneficial skill, which is asserted as a hypothesis and never tested against game outcomes.

Editorial extensions

If this is right

If the model is right, teams can evaluate change-of-direction skill from in-game tracking data rather than from Combine drills alone, and can do so for every ball carrier in a play.
The position-group variance estimates imply that comparisons should be made within position: running backs are a more homogeneous group, so smaller differences among them can still be meaningful.
Combining turn-angle variability with 40-yard dash times yields movement profiles that separate straight-line speed specialists from shifty runners, a distinction no single metric captures.
The same modeling framework transfers directly to other ball-carrying contexts, such as punt and kick returners, quarterbacks on scrambles, and defensive players in coverage.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural next step the paper does not take is to validate the variability measure against play outcomes such as yards after contact, missed tackles forced, or expected points added; the rankings are only practically useful if higher variability actually predicts better performance.
Because the model is fit on only nine weeks of one season, an obvious reliability check is to refit on later weeks or another season and ask whether players' concentration random effects are stable across time; the paper's leaderboards assume this stability.
The weak correlation with 40-yard dash times suggests turn-angle variability captures a distinct athletic trait; one could test whether a player's variability in the open field matches his variability on scripted routes or in pre-catch movements, which would tell whether the trait is positional or personal.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's my take on 2507.06122. It's a solid, useful applied stats paper: it builds a Bayesian circular mixed-effects model for NFL ball-carrier turn angles and produces the first public in-game measure of change-of-direction variability for skill players. The statistical machinery is standard in animal movement, but the application to NFL tracking data is new, and the authors ship code and data. Good things: the positional results are plausible, the player rankings line up with reputations (Justice Hill shifty, Metcalf straight-line), and the posterior intervals are reported honestly. Citation pattern is fine; self-citations are for feature engineering, not the central result.

The soft spots are real. The main one is the Section 3.3 claim that the estimates 'reliably differentiate' players. The model treats every frame as conditionally independent given covariates and the player random effect, with only a lag-1 term in the mean. At 10 Hz, consecutive turn angles within a play are autocorrelated, so the posterior intervals for the player effects are probably too narrow. The non-overlap in Figure 4 is weaker evidence than it looks. The fix is a play-level random effect or an autoregressive structure. Second, the 'shifty is good' interpretation is hypothesized in Section 1.3 and then used as a conclusion in the Discussion, but it is never validated against on-field outcomes like missed tackles or yards after contact. Third, there's no comparison to a simpler baseline, so we don't know what the model adds over empirical per-player variance. And the speed-concentration finding could partly be a measurement artifact: positional noise generates smaller angular changes when the player is moving fast, which would inflate concentration. The stress-test note about autocorrelation holds up on reading; it's the most important technical concern.

Who this is for: football analytics people wanting an objective shiftiness metric, and applied statisticians interested in circular mixed models. I'd send it to peer review with a request for revision: the application is worth publishing, but the differentiation claim needs tempering and the independence assumption should be acknowledged and preferably fixed.

Referee Report

3 major / 5 minor

Summary. The paper proposes a Bayesian circular mixed-effects model for frame-level turn angles of NFL ball carriers, using tracking data from the first nine weeks of the 2022 NFL season. The turn angle is modeled with a von Mises distribution whose mean and concentration are both functions of spatiotemporal and contextual covariates, and player-level random intercepts on the concentration (nested within position) are the key parameters of interest. The authors report posterior estimates for fixed effects, position-specific variance parameters, player leaderboards of turn angle variability, and a joint analysis with 40-yard dash times. The central claims are that the player random effects reliably differentiate ball carriers (Section 3.3) and that the model offers a public, in-game measure of change-of-direction ability.

Significance. If the central claims are sustained, the paper would provide a novel public in-game metric for evaluating change-of-direction in the NFL, complementing Combine drills. The work uses publicly available data and provides reproducible code; Bayesian uncertainty quantification and convergence diagnostics are reported. However, the practical interpretation of the leaderboards as measuring 'shiftiness' rests on an untested hypothesis, and the reliability of the player-level intervals depends on strong conditional-independence assumptions that are not validated.

major comments (3)

[§2.4, §3.3, Figure 4] The claim in Section 3.3 that the estimates 'reliably differentiate between the players' is based on non-overlapping 95% credible intervals for the top and bottom ball carriers. The model in Section 2.4 assumes that frame-level turn angles are conditionally independent given the covariates and the player random effect, with only a lag-1 covariate in the mean and no play-level random effect or residual autocorrelation. Because tracking data are recorded at 10 Hz, successive turn angles within a play are strongly serially dependent, so the posterior variance of the player random effects is likely understated and the reported non-overlap may be overconfident. Please address this with posterior predictive checks, a model that includes a play-level random effect or an autoregressive error term, or a simulation study demonstrating that the differentiation intervals remain valid under realistic within-play dependence.
[§1.3, §3.3] The paper's practical output is the ranking of players by turn angle variability, but the interpretation that higher variability corresponds to beneficial shifty movement is introduced as a hypothesis (Section 1.3) and is never tested against on-field outcomes. The leaderboards, the discussion of positional differences, and the speed-and-turn profiles are only meaningful as performance evaluations if this interpretation holds. Please validate the measure against outcome-based metrics (e.g., yards after contact, broken tackles, success rate) or explicitly reframe the paper as a descriptive analysis of movement variability rather than an evaluation tool.
[§3.1, Table 3] The positive relationship between speed and turn angle concentration (ψ̂_s = 0.709, 95% CI [0.705, 0.713]) may be partly a geometric artifact of the turn angle definition. For a fixed path curvature, the angle between successive displacement vectors decreases as step length increases, and step length is proportional to speed at a fixed sampling rate. The reported coefficient therefore does not necessarily reflect biomechanically more consistent direction at high speed. Please control for step length or displacement magnitude, or demonstrate via simulation that the effect is not an artifact of the angle metric.

minor comments (5)

[§3.3] The play-count inclusion thresholds for the leaderboards (25 for running backs, 10 for tight ends, 15 for wide receivers) are presented without justification; a sensitivity analysis or a principled rationale would strengthen the results.
[Figure 4] Figure 4 is visually dense and hard to parse; consider separating the three position groups into individual panels with larger fonts or using a more legible display of posterior intervals.
[§2.4] The prior specification is only partially described; the text mentions vague half-t priors for σ_p but does not state the priors used for the fixed effects or the random effect u_j, which would be useful for reproducibility.
[§3.4] The correlations between the posterior mean random effect and 40-yard dash time are reported without uncertainty intervals; adding confidence intervals would help readers assess the strength of these relationships.
[Acknowledgments] The personal anecdote in the acknowledgments about naming the measure 'STRAIN' is out of place in a formal journal article; consider removing it or moving it to a footnote.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's player-level estimates are empirical posterior quantities from observed turn angles, and its self-citations are not load-bearing.

full rationale

The paper's central output—ball-carrier-specific concentration random effects u_j and position-specific variances—is estimated directly from observed frame-level turn angles through a von Mises mixed-effects model. There is no fitted parameter that is then relabeled as a prediction of a closely related quantity, and no input is defined in terms of the output. The feature-engineering self-citations (Yurko et al. 2020, 2024) are used only to motivate covariate construction, not to justify the model's conclusions. The 'reliably differentiate' claim in Section 3.3 is a posterior-interval comparison from the estimated model, not a result forced by construction. The untested hypothesis that higher turn-angle variability is beneficial is an interpretive assumption about the metric's meaning, not a circular derivation. The main potential weakness—frame-level dependence under a 10 Hz sampling rate—is a statistical misspecification concern about interval coverage, not a circularity. The derivation chain is therefore self-contained with respect to the data and model assumptions.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The model contains many regression coefficients estimated from data, which are not free parameters in the ad hoc sense. The genuinely hand-chosen inputs are the leaderboard inclusion thresholds and the interpretive assumption that variability equals shiftiness. The von Mises and independence assumptions are standard but load-bearing for the likelihood. No new entities are postulated.

free parameters (1)

Minimum-play inclusion thresholds for leaderboard = RB: 25; TE: 10; WR: 15
Hand-chosen cutoffs in Section 3.3 determine which players appear in the rankings; different thresholds would change the leaderboard, though not the fitted model parameters.

assumptions (4)

domain assumption Frame-level turn angles follow a von Mises distribution with player-specific concentration random effects.
Section 2.4: this distributional assumption is the core of the likelihood and is not tested against alternatives.
standard math Turn angles are conditionally independent across frames given covariates and random effects.
The likelihood in Section 2.4 treats each frame as an independent observation; serial correlation in movement is only partially captured by including the previous turn angle as a covariate.
domain assumption The turn angle between successive displacement vectors at 10 Hz captures change of direction ability.
Section 2.3 defines the metric; its validity as a measure of evasive skill is assumed, not demonstrated.
ad hoc to paper Higher variability in turn angle corresponds to better shifty movement.
Section 1.3 states this as a hypothesis used to interpret all player rankings; no outcome data support it.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A Bayesian circular mixed-effects model for explaining variability in directional movement in American football." pith.science (2026). https://pith.science/paper/K3GTUOQQ

@misc{pith2026250706122,
  author       = {Pith},
  title        = {Pith review of: A Bayesian circular mixed-effects model for explaining variability in directional movement in American football},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K3GTUOQQ}},
  note         = {Machine review of arXiv:2507.06122}
}

read the original abstract

Change of direction is a key element of player movement in American football, yet there remains a lack of objective approaches for in-game performance evaluation of this athletic trait. Using tracking data, we propose a Bayesian mixed-effects model with heterogeneous variances for assessing a player's ability to make variable directional adjustments while moving on the field. We model the turn angle (i.e., angle between successive displacement vectors) for NFL ball carriers on both passing and rushing plays, focusing on receivers after the catch and running backs after the handoff. In particular, we consider a von Mises distribution for the frame-level turn angle and explicitly model both the mean and concentration parameters with relevant spatiotemporal and contextual covariates. Of primary interest, we include player random effects that allow the turn angle concentration to vary by ball carrier nested within position groups. This offers practical insight into player evaluation, as it reveals the shiftiest ball carriers with great variability in turning behavior. We illustrate our approach with results from the first nine weeks of the 2022 NFL regular season and explore player-specific and positional differences in turn angle variability.

Figures

Figures reproduced from arXiv: 2507.06122 by the authors.

**Figure 2.** Geometric representation of the instantaneous turn angle [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

**Figure 3.** Distribution of turn angle for NFL ball carriers during the first nine weeks of the 2022 [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Posterior distributions of the ball carrier concentration random effect for NFL running [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]

**Figure 5.** Figure 5: Relationship between the posterior mean of the ball carrier concentration random effect [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 28 canonical work pages

[1]

S., Matthews, G

Baumer, B. S., Matthews, G. J., and Nguyen, Q. (2023). Big ideas in sports analytics and statistical tools for their investigation. WIREs Computational Statistics , 15(6):e1612

work page 2023
[2]

Brughelli, M., Cronin, J., Levin, G., and Chaouachi, A. (2008). Understanding change of direction ability in sport: A review of resistance training studies. Sports Medicine , 38(12):1045--1063

work page 2008
[3]

B\" u rkner, P.-C. (2017). brms: An R Package for Bayesian Multilevel Models Using Stan . Journal of Statistical Software , 80(1):1--28

work page 2017
[4]

D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., and Riddell, A

Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., and Riddell, A. (2017). Stan: A probabilistic programming language. Journal of Statistical Software , 76(1):1--32

work page 2017
[5]

Chu, D., Reyers, M., Thomson, J., and Wu, L. Y. (2020). Route identification in the National Football League . Journal of Quantitative Analysis in Sports , 16(2):121--132

work page 2020
[6]

Deshpande, S. K. and Evans, K. (2020). Expected hypothetical completion probability. Journal of Quantitative Analysis in Sports , 16(2):85--94

work page 2020
[7]

Durkin, D. (2019). On the clock: Justice Hill’s explosive speed makes him an intriguing prospect for the Bears . The Athletic. https://www.nytimes.com/athletic/931228/2019/04/18/on-the-clock-justice-hills-explosive-speed-makes-him-an-intriguing-prospect-for-the-bears

work page 2019
[8]

Dutta, R., Yurko, R., and Ventura, S. L. (2020). Unsupervised methods for identifying pass coverage among defensive backs with nfl player tracking data. Journal of Quantitative Analysis in Sports , 16(2):143--161

work page 2020

Show all 29 references

[9]

Gelman, A. (2006). Prior distributions for variance parameters in hierarchical models. Bayesian Analysis , 1(3):515--534

2006
[10]

B., Stern, H

Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., and Rubin, D. B. (2013). Bayesian Data Analysis , Third Edition . Chapman & Hall / CRC Texts in Statistical Science . Taylor & Francis

2013
[11]

and Rubin, D

Gelman, A. and Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science , 7(4):457--472

1992
[12]

Giles, B., Kovalchik, S., and Reid, M. (2019). A machine learning approach for automatic detection and classification of changes of direction from player tracking data in professional tennis. Journal of Sports Sciences , 38(1):106--113

2019
[13]

Giles, B., Peeling, P., and Reid, M. (2024). Quantifying change of direction movement demands in professional tennis matchplay: An analysis from the australian open grand slam. Journal of Strength and Conditioning Research , 38(3):517--525

2024
[14]

and Carl, S

Ho, T. and Carl, S. (2025). nflreadr: Download 'nflverse' Data . R package version 1.4.1.07

2025
[15]

Hoffman, M. D. and Gelman, A. (2014). The no-u-turn sampler: Adaptively setting path lengths in hamiltonian monte carlo. Journal of Machine Learning Research , 15(47):1593--1623

2014
[16]

B., Johnson, D

Hooten, M. B., Johnson, D. S., McClintock, B. T., and Morales, J. M. (2017). Animal Movement: Statistical Models for Telemetry Data . CRC Press

2017
[17]

Kassam, K. (2025). Minding the College Football Analytics Gap . Presented at the 2025 MIT Sloan Sports Analytics Conference. https://youtu.be/RZNLXU2O0WM

2025
[18]

Kovalchik, S. A. (2023). Player Tracking Data in Sports . Annual Review of Statistics and Its Application , 10(1):677--697

2023
[19]

Kramer, A. (2019). Jonathan Taylor Wants More, More, More . Bleacher Report. https://bleacherreport.com/articles/2854041-jonathan-taylor-wants-more-more-more

2019
[20]

Levy, L. (2019). Combine Conundrum: What To Make Of DK Metcalf's Agility Testing . Optimum Scouting. http://www.optimumscouting.com/news/dk-metcalf

2019
[21]

Lopez, M., Bliss, T., Blake, A., Mooney, P., and Howard, A. (2024). NFL Big Data Bowl 2025 . https://kaggle.com/competitions/nfl-big-data-bowl-2025

2024
[22]

Lopez, M. J. (2020). Bigger data, better questions, and a return to fourth down behavior: an introduction to a special issue on tracking datain the National Football League . Journal of Quantitative Analysis in Sports , 16(2):73--79

2020
[23]

Maclean, Q. (2024). Defensive Back Scouting: Using Pose Estimation to Measure Hip Fluidity . SumerSports. https://sumersports.com/the-zone/defensive-back-scouting-using-pose-estimation-to-measure-hip-fluidity

2024
[24]

Nguyen, Q., Jiang, R., Ellingwood, M., and Yurko, R. (2025). Fractional tackles: leveraging player tracking data for within-play tackling evaluation in A merican football. Scientific Reports , 15:2148

2025
[25]

and Yurko, R

Nguyen, Q. and Yurko, R. (2025). A multilevel model with heterogeneous variances for snap timing in the national football league. arXiv preprint arXiv:2502.16313 . https://arxiv.org/pdf/2502.16313

2025 arXiv
[26]

Nguyen, Q., Yurko, R., and Matthews, G. J. (2024). Here Comes the STRAIN: Analyzing Defensive Pass Rush in American Football with Player Tracking Data . The American Statistician , 78(2):199--208

2024
[27]

R: A Language and Environment for Statistical Computing

R Core Team (2025). R: A Language and Environment for Statistical Computing . R Foundation for Statistical Computing, Vienna, Austria

2025
[28]

F., Granered, N., Pospisil, T., Pelechrinis, K., and Ventura, S

Yurko, R., Matano, F., Richardson, L. F., Granered, N., Pospisil, T., Pelechrinis, K., and Ventura, S. L. (2020). Going deep: models for continuous-time within-play valuation of game outcomes in American football with tracking data . Journal of Quantitative Analysis in Sports ...

2020
[29]

Yurko, R., Nguyen, Q., and Pelechrinis, K. (2024). NFL G hosts: A framework for evaluating defender positioning with conditional density estimation. arXiv preprint arXiv:2406.17220 . https://arxiv.org/pdf/2406.17220

2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

S., Matthews, G

Baumer, B. S., Matthews, G. J., and Nguyen, Q. (2023). Big ideas in sports analytics and statistical tools for their investigation. WIREs Computational Statistics , 15(6):e1612

work page 2023

[2] [2]

Brughelli, M., Cronin, J., Levin, G., and Chaouachi, A. (2008). Understanding change of direction ability in sport: A review of resistance training studies. Sports Medicine , 38(12):1045--1063

work page 2008

[3] [3]

B\" u rkner, P.-C. (2017). brms: An R Package for Bayesian Multilevel Models Using Stan . Journal of Statistical Software , 80(1):1--28

work page 2017

[4] [4]

D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., and Riddell, A

Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., and Riddell, A. (2017). Stan: A probabilistic programming language. Journal of Statistical Software , 76(1):1--32

work page 2017

[5] [5]

Chu, D., Reyers, M., Thomson, J., and Wu, L. Y. (2020). Route identification in the National Football League . Journal of Quantitative Analysis in Sports , 16(2):121--132

work page 2020

[6] [6]

Deshpande, S. K. and Evans, K. (2020). Expected hypothetical completion probability. Journal of Quantitative Analysis in Sports , 16(2):85--94

work page 2020

[7] [7]

Durkin, D. (2019). On the clock: Justice Hill’s explosive speed makes him an intriguing prospect for the Bears . The Athletic. https://www.nytimes.com/athletic/931228/2019/04/18/on-the-clock-justice-hills-explosive-speed-makes-him-an-intriguing-prospect-for-the-bears

work page 2019

[8] [8]

Dutta, R., Yurko, R., and Ventura, S. L. (2020). Unsupervised methods for identifying pass coverage among defensive backs with nfl player tracking data. Journal of Quantitative Analysis in Sports , 16(2):143--161

work page 2020