Authors: Andrew Luke, Trevor Hayward, Henry Douglass, Charles Berman

andrewluke2025@u.northwestern.edu, trevorhayward2024@u.northwestern.edu, henrydouglass2026@u.northwestern.edu, charlesberman2026@u.northwestern.edu

Written February 23rd, 2024

Week 18 of the 2022-23 NFL season: the 3-13 Chicago Bears faced their division rival the Minnesota Vikings, while the 2-13-1 Houston Texans faced the Indianapolis Colts with the number one overall draft pick on the line. The Bears went on to lose that game improving their chances to steal the number one pick from the Texans. Later that day, former Bears head coach Lovie Smith and his Texans found a way to upset the Colts on the last play of the game scoring a touchdown followed by a game-winning two point conversion to give the Texans a 1-point lead and officially give the Chicago Bears the number one overall pick. 

Following a very disappointing 2022-23 NFL season for the Bears where they ended the season on a 10 game losing streak, Ryan Poles (GM) and his Bears were met with a massive decision with what to do with the number one overall pick. While there were highly coveted QB prospects coming up in that draft with Bryce Young from Alabama and CJ Stroud from Ohio State, the Bears, an organization who have never had a franchise QB, finally looked like they had their ‘guy’ with current QB Justin Fields showing massive strides in his game accounting for 2242 passing yards and 113 rushing yards along with 25 total touchdowns. With the massive improvement in Justin’s game in that season, Poles made the decision to trade down in the draft to receive one of the most lopsided trades in NFL history, getting star wide receiver DJ Moore, 2023 9th overall pick, 2024 1st overall pick, and their second round pick in 2023 and 2025. 

Just like that, the Bears seemingly decided they wanted to stick with Justin and surround him with star players and talent, in order for him to make the proverbial third-year NFL QB jump. Chicago fans spent the offseason celebrating the fact that they might have finally found the franchise QB that they have been searching for. However, the beginning of the 2023-24 saw the same issues that the team had last year extending their losing-streak to 14 games. While the hope was that Justin Fields would make a massive leap this season, he still struggled to pass in the pocket, win games in the clutch, take care of the football, and throw with anticipation. While Justin finished the season very strong with a 5-3 record, making plays with his feet and arm, and showing that he can take care of the football in clutch time and lead a team to victory, the Bears have been given a once-in-a-lifetime opportunity.  

The 2024 draft class is absolutely stacked with talent including QB Caleb Williams from USC who is touted as the best QB prospect since Peyton Manning by former NFL GM Rick Speilman, followed by QB Drake Maye from North Carolina, and QB Jaylen Daniels from LSU, as well as WR talent like Marvin Harrison Jr. from Ohio State, Malik Nabers from LSU, and Rome Odunze from Washington. With this extremely talented pool of draft prospects and Justin Fields not making the massive third-year jump that is expected for NFL quarterbacks, the decision for Poles with what to do with the number one overall pick this year is a lot more difficult. Does he stick with Justin Fields who has shown consistent improvement over his tenure with the Bears and continue to add around him or does he take a gamble on ‘generational-talent’ Caleb Williams who has proven to be one of the best college QBs in modern history? What Poles ends up doing in this draft will determine the future trajectory of the Chicago Bears, making it arguably one of the most important decisions in Bears history. Using the data of past college QBs’ success in college and how they panned out in the NFL, we will determine what the Bears should do in this draft and who should be their QB for the 2024-25 season and beyond.    

To assist the Bears in their decision-making process, we developed a supervised machine learning model using R to forecast the NFL success of the 2024 quarterback draft class. Our model relies on historical college data spanning from 2016 to 2023, encompassing key passing statistics such as passer rating, touchdowns, interceptions, passing yards, and completion percentage. This data was sourced from ESPN.com, specifically the College Quarterback Passing Stats section. Click here for an in-depth look at the data.

In constructing our model, we introduced a binary variable named “Verdict,” which determines whether an NFL quarterback is deemed successful (1) or not successful (0). Success is defined as either maintaining a winning percentage above .500 or starting over 50 games. This metric accommodates both newer quarterbacks and those with longer tenures, ensuring a fair assessment of success across different career stages. For instance, quarterbacks like Patrick Mahomes and CJ Stroud are considered successful despite differences in the number of games played.

To predict the Verdict variable, we utilized a recipe that incorporates several predictor variables, including Passer Rating, Touchdowns, Yards, and whether the player has won a Heisman trophy. These visuals of the historical data helped us decide which variables to include in the recipe:

We evaluated the model’s performance by testing eight different algorithms on our training set, including boosted tree, elastic net, K nearest neighbors, random forest, neural network, mars, SVM poly, and SVM radial models. Given that this is a classification problem, we assessed model accuracy using ROC_AUC (Area under curve). Ultimately, we combined the SVM Radial and Neural Network models, which both had the best results, to create an ensemble model, which yielded the most accurate predictions and a ROC_AUC of .816.

The model demonstrated a solid level of accuracy, effectively predicting success for players such as Patrick Mahomes, Lamar Jackson, Joe Burrow, Jared Goff, Kyler Murray, Baker Mayfield, and Josh Allen, and the majority of the non successes. However, it encountered challenges in foreseeing the success of players like Dak Prescott, Brock Purdy, and Deshaun Watson. Moreover, there were instances of false positives, including Stetson Bennett, Nate Stanley, Sam Ehlinger, and Bryce Young. While these players may potentially prove successful in the future, their current verdicts do not align with our model’s predictions.

It’s important to acknowledge that while our model leverages college statistics as a predictive metric, the transition from college to professional play is multifaceted, and not all aspects of collegiate performance reliably translate to NFL success. Despite these challenges, with a ROC AUC of 0.816, our model demonstrates a commendable level of accuracy in predicting NFL success based on college performance.

Here are the results for the first bunch out of 78 QBs in terms of passing yards, but we are mainly focused on Caleb Williams, Jayden Daniels, and Drake Maye:

Based on our model’s predictions, only Jayden Daniels and Caleb Williams are forecasted to emerge as successful quarterbacks from this draft class. The model also suggests that Drake Maye will not achieve success at the NFL level. Consequently, other teams in need of a quarterback, such as Washington or New England, can leverage this insight to inform their draft selection. Overall, our model aligns fairly well with the general consensus of draft pre rankings. 

Comparing these predictions to Justin Fields, who our model correctly categorized as not successful, it becomes apparent that the Bears should strongly consider drafting Caleb Williams with the first overall pick in this year’s draft. While Justin Fields has demonstrated improvement in his NFL development, our data suggests that Caleb Williams holds a competitive edge in terms of both potential and performance. By selecting Caleb Williams as their starting quarterback for the future, the Bears can strategically shape their roster to complement his playing style. Furthermore, with the 9th overall pick in this draft and the significant trade value Justin Fields still holds, the Bears have the opportunity to make massive improvements to their team under Caleb Williams’ leadership. This decision would set the Bears for a huge step forward in their competitiveness in the next few years. We are excited to see what the Bears will do at the draft and what their future holds.