The team behind Next Gen Stats (NGS) created the “Passing Score” statistic to assess whether a quarterback made the optimal decision after the play. The NFL said dozens of quarterback evaluation metrics exist but they all miss out on isolating the specific variables a quarterback must evaluate before a passing play. Josh Helmrich, NFL Director of strategy and business development, told ZDNet that AWS and the NFL worked together to combine 7 different machine learning models and several play variables to create the Next Gen Stats Passing Score. “With the NFL playoffs fast approaching, we want to deliver fans an increased depth of analysis, more context, and additional ways to understand one of the most important positions in sports – the NFL quarterback. This new metric will help fans better understand the quarterback’s passing performance, relative to his decision, on each play,” Helmrich said. “The quarterback position is much more complex than just the outcome of the play. To help understand the quarterback’s decision-making and all the variables that go into it, we spent over a year developing the components that make up the Next Gen Stats Passing Score.” In a blog post, AWS and the NFL explained that quarterbacks make hundreds of split-second decisions based on how the defense is lined up, the time remaining in the game, and the yards needed for a first down or touchdown. To illustrate the statistic’s accuracy, AWS ranked the performances of quarterbacks in 2021, finding that Aaron Rodgers, Joe Burrow, and Matthew Stafford were the only players with scores above 90. Ryan Tannehill, Patrick Mahomes, Josh Allen, and Tom Brady weren’t far behind with scores above 88. Dak Prescott, Kyler Murray, and Jimmy Garoppolo all had 87s. Jalen Hurts, Derek Carr, and Mac Jones had scores above 82. Ben Roethlisberger was 14th on the list with a 70 and is the only quarterback over the last four seasons to get to the playoffs with a score under 82. Despite hundreds of variables affecting the outcome of a play, the statistical credit or blame is always assigned to the quarterback. AWS ML Solutions Lab’s Priya Ponnapalli told ZDNet the machine learning technology and data analysis tools used to create the Passing Score will help viewers understand the game and the decisions made by quarterbacks. “NGS spent nearly a year solving a series of complex technology problems to create the Passing Score tool,” AWS explained. “Building on years of creating and deploying AWS-powered machine learning (ML) models, the NGS engineering and analytics team worked with the AWS Professional Services data science group to optimize and combine seven ML models, including a new model to predict the value of a pass before the ball is thrown. The combined ML models power the new NGS Passing Score. Each individual model provides compelling insights into what’s happening on the field, like predicted yards and completion probability. By combining the models, a score (a number between 50–99) better reflects the intuition used by fans, coaches, and players to assess a quarterback’s passing performance.” Ponnapalli and AWS explained that the new statistic isolates the contribution of the passer on the outcome of a pass play, agnostic of the receiver, and controls for the level of difficulty of every pass. Passing Score is calculated by using tracking data to predict the number of yards a receiver will gain if targeted, combined with an enhancement to the completion probability model, which can now also estimate the probability of an interception. AWS noted that in the future, the Passing Score could be expanding to factor in other things like rushers, sacks, and pressures. “Machine learning technology helps to deliver insights never before possible, and we believe the Next Gen Stats Passing Score is the next great example of that,” Helmrich said. “We know it will help fans better understand and engage with the game.”
