The Extra Point What the Top Predictions Looked Like in the NFL's Big Data Bowl

What the Top Predictions Looked Like in the NFL's Big Data Bowl

January 27, 2020

As part of the NFL’s second Big Data Bowl, more than 2,000 data scientists from all over the world competed on Kaggle to predict rushing play outcomes over the last five weeks of the 2019 regular season.

Data scientists Phillip Singer and Dmitry Gordeev from Austria captured the top prize of $50,000 with their highly technical approach. Nicknamed The Zoo on Kaggle, Singer and Gordeev’s team led throughout the contest, besting second place by the same scoring margin that separated second place and 24th place.

The Zoo posted their predictions on Kaggle, and today’s Extra Point assesses how closely their predictions matched what happened on the field. Note: each prediction was made only using run plays from the past. In statistical terms, these are considered out-of-sample predictions. 

What was being predicted?

The goal of the Kaggle contest was to predict where every ball carrier would end up on a handoff play. Participants were given several characteristics of the play, including the NFL’s Next Gen Stats, and game and play traits such as the down, distance, starting yard line and which players are on the field. 

For example, on this Cordarrelle Patterson carry for the Chicago Bears in Week 1 against the Green Bay Packers, Patterson barely had a chance of reaching the line of scrimmage. The Zoo assigned the following probabilities: 

Patterson:

• loses 3 yards or more: 9.4%
• loses 2 yards: 19.4%
• loses 1 yard: 19.9%
• gains 0 yards: 21.4%
• gains 1 yard: 11.5%
• gains 2 yards: 6.9%
• gains 3 yards or more: 7.2%

Patterson lost 2 yards on the play — one of the three most likely outcomes predicted by The Zoo’s model. 

While predictions on Patterson’s carry were grouped near zero yards, on this Raheem Mostert 41-yard touchdown run against the Carolina Panthers in Week 8, The Zoo gave the San Francisco 49ers back a 10.5% chance of reaching the end zone — making it more likely that he would score than Patterson was to pick up three yards. According to The Zoo, Mostert had a 61% chance of picking up 10 yards or more on his carry.

How accurate was The Zoo across all handoff plays?

Looking at all Big Data Bowl plays from Weeks 1–12 provides a sense of how precise these predictions were. The following chart groups each carry into a predicted number of yards, and averages the observed yardage gained within each group of predictions. The black dotted line corresponds to settings where predictions perfectly matched reality.

By and large, The Zoo’s predictions reflected what eventually happened on 2019 carries. For example, there were precisely five carries where The Zoo’s model predicted that the most likely yardage gained would be  20 yards — on those carries, the ball carriers averaged 19.2 yards. The Zoo’s most common yardage prediction was four yards, and of the 1,543 carries where four yards was the prediction, ball carriers averaged 3.98 yards. 

Predicting football outcomes is challenging, and the predictions made as part of the Big Data Bowl were no different. The Zoo’s most likely rushing distance on each play was an exact match on only 17.6% of plays, which means that more than four out of every five times, the offenses over or underperformed expectations. But being able to better understand what goes into successful plays — separating Patterson’s carry from Mostert’s, for example — gives us deeper insight into what drives ball carrier success. 

On February 26, The Zoo and other Kaggle top finishers will join finalists from the Big Data Bowl collegiate contest and present their results to NFL team analytics staff at the Scouting Combine in Indianapolis.