The NFL draft and recruiting, 2014-2024 (and a question)
First of all, congrats to all of the Michigan players that found a way to the NFL.
But now that the draft is over, I’m sure at least some of you are wondering how much Michigan’s 13-man draft class is going to impact its future recruiting efforts. Even if you weren’t wondering, there’s never anything wrong with looking at some data, is there?
Well, I’ve been working on a diary where I analyze the predictors of 247 team recruiting scores (see more below). And since I already had the data on past draft picks, I thought it might be interesting to visualize the effect of draft history on recruiting scores. The graph below shows the 247 Sports team recruiting scores as a function of the # of draft picks in the previous two years.
The figure should be interactive, so if you hover over any of the observations, you can see the team, recruiting year, recruiting score, and distance (in standard deviations) of the recruiting score from the linear prediction. I’m still trying to figure out how to embed after the update, so 🤞 that it works!
So what can we say from the graph? Well, the relationship is statistically significant (p < .001), and the regression coefficient suggests that a one-unit increase in draft picks will increase the 247 team recruiting score by 6.4 points. To put this into context, this simple model would predict that Michigan’s 2025 recruiting score would increase by 51.2 points over its 2024 score (8 more draft picks heading into the 2025 recruiting class (22) than in the 2024 class (14)). Only four teams have had 22 or more picks over a two year period—Alabama in 2019 & 2020 and Georgia in 2023 & 2024—and in each case, they signed recruiting classes with scores > 300 points. Michigan does not appear to be on track to meet this prediction in 2025.
But holy hell, just look at the dispersion. Getting players picked in the draft clearly matters, especially at the upper range of draft picks, but the distribution of observations around the fitted regression line doesn’t give me a whole lot of confidence. Obviously other things matter. So here’s my question:
What else to include in the model?
Here are the variables for which I have data so far. Is there anything else you would suggest to include in a model predicting team recruiting scores? Keep in mind that these have to be measurable.
- SP+ Ratings (previous 2-year average): teams that are more successful on the field should have greater recruiting success, right?
- # Draft picks (previous 2-year total): in the model, I break this down into 1st round, 2nd round, and 3rd+ round totals.
- # of CFP appearances (previous 2 years): again, on-field success should lead to recruiting success, right?
- # of in-state top-100 recruits (same recruiting year): teams in more fertile recruiting territory should have higher recruiting scores.
- New coach (same recruiting year): does a coaching change hurt recruiting? In the model, I also break new coaches down into those that have previous head coaching experience and those that don’t (i.e., first-time head coaches).
- AAU school: do academics matter for recruiting? (Hahahaha)
I also have data on:
- the number of AP top-10, 15, and 25 finishes in the previous two years, but this is too highly correlated with SP+ ratings to include in the same model.
- # of team wins (previous two years). Again, this is too highly correlated with SP+ to include in the same model.
- Athletic department revenue, but these data are missing for private schools.
What else do you recommend?
Two additional questions:
- I use 2-year totals/averages for previous on-field and draft results because it theoretically makes sense that a kid in, say, the 2025 recruiting class would pay attention to a school’s on-field and draft success in 2023 and 2024 (during his sophomore and junior seasons). Do you think I should extend these to 3-year averages/totals? Cut it to one year?
- I’m debating how I should code the “new coach” variable. Right now I have it coded for the same recruiting year. That is, Michigan is coded as having a new coach in 2024, so this coaching change would have an effect on the 2024 recruiting class. This makes sense to me because, well, Sherrone still had to hold the recruiting class together, even though Harbaugh built it. But would it make more sense to have the coaching change influence the next year’s (i.e., 2025’s) recruiting class? After all, this would be Sherrone’s first real class. Or maybe both (i.e., Michigan would be coded as having a new coach for the purpose of predicting the 2024 and 2025 recruiting classes).
Anything you find interesting?
Thanks for reading and Go Blue!
Very nice job on this. As much as I like correlation, I am interested in finding outliers and reasons for variation. An interesting question: What kinds of things explain the difference between aTm 2022 and Washington 2022? ASU 2022 and Texas 2018? On the surface, all those teams on the bottom are Pac12 teams. Maybe there's bias in the recruiting rankings based on geography?
Bias in the recruiting rankings? Unpossible! S-E-C! S-E-C!
An SEC offer embiggens the smallest recruit?
April 29th, 2024 at 11:25 AM ^
It just means more.
Also, good use of embiggens
Does the data really lead to another question? As a Michigan Engineer - I wonder if there is a correlation between the University / College academic ranking and football recruiting. Specifically, is Michigan’s College of Engineering falling behind other “leading” Engineering programs at schools like Bama, Georgia, LSU, Ole Miss, and South Carolina.
Dupe.. deleted.
Thanks!
That's kind of my goal in the end. I want to build the best predictive model using observables like on-field success (SP+, draft picks, CFP appearances, etc.), coaching relationship (using new/returning coach as a proxy), academics, recruiting territory base... Then once I've done that, I'll look at the residuals to see what hasn't been picked up by these observables to explain team recruiting over-/underachievement. In other words, the bag ($$$) is in the error term, but so also are other things like team culture.
There are almost certainly biases in recruiting rankings based on geography. I can't upload images, but I mapped the 247 top-200 previously and it is definitely biased toward the south/west.
April 29th, 2024 at 10:55 AM ^
There's clearly at least some regional bias in recruiting rankings; the question is how much it affects them. To give a trivial example, recruits in SEC territory are far more likely to be scouted than recruits in the northeast. Kwity Paye, for instance, coming out of Rhode Island was mostly a low three star--except for ESPN, which at the time Ace suspected was the only one of them that had done any in-depth scouting.
This may hold true for other areas, but it may also be negligible (depending on the area in question). Is there any other bias? I'm inclined to doubt it, but it's also possible that scouts in one area tend to over- or under-rank recruits enough to affect rankings.
Recruits that commit early often don't seem to get scouted as thoroughly as other players. Which will tend to affect rankings. I think that tends to be a small minority of players these days (although that's just my impression).
April 29th, 2024 at 10:03 AM ^
I would consider expanding the "new coach" indicator, into a coaching tenure.
April 29th, 2024 at 10:05 AM ^
You mean like a count of the number of years that they've been head coach at the school? Or maybe a "coach winning percentage" variable? That's excellent, thanks!
April 29th, 2024 at 10:06 AM ^
Have you seen this analysis that relates recruiting to revenue and location? You could use this model to anchor your data.
April 29th, 2024 at 10:14 AM ^
I haven't seen this, but I'm going to take a look at his stuff. Thanks!
April 29th, 2024 at 10:06 AM ^
The problem with trying to compare today's recruiting to earlier years is that we now have the Portal as a 2nd recruiting option.
But, of course, NIL will be the unknown in all of this. The schools with the capability to spend the most money will probably rule going forward. Perfect example is Oregon with the backing of Phil Knight-Nike money. They're going to be hard to beat
April 29th, 2024 at 10:07 AM ^
My first thought was that it would be interesting to compare the fully pooled model 1 (score~1+picks) with alternatives score~1+picks+(1|school) and score~1+picks+(1|school)+(1|picks). The Michigan points look like a flat line, which made me wonder if the picks fixed effect was really capturing school-specific variation (random intercept by school). [Edit, random effects not strictly germane, I guess--school-specific intercept/effects could be "fixed"].
April 29th, 2024 at 10:58 AM ^
Thanks! I hadn't thought about that. I'm going to look into this some more when I get a chance. But just a quick look at a model with team fixed-effects washed out any effect of draft picks. A large part of the variation is picked up by the simple team-level effects (which is consistent with your observation about Michigan's scores being a flat line).
Very cool, thanks!
April 29th, 2024 at 10:18 AM ^
Only four teams have had 22 or more picks over a two year period—Alabama in 2019 & 2020 and Georgia in 2023 & 2024—and in each case, they signed recruiting classes with scores > 300 points. Michigan does not appear to be on track to meet this prediction in 2025.
Isn't the possible implication being the inverse of this analysis? That recruiting rankings are driving draft picks and thus further reinforcing recruiting rankings? A key difference between Georgia/Alabama and Michigan is the consistency of 'elite recruiting'. So, to what extent is the draft success an independent driver of recruiting success vs. a component of a much more complicated picture?
April 29th, 2024 at 10:51 AM ^
Could be in 2026?
April 29th, 2024 at 11:14 AM ^
You're definitely right, for some teams anyway. Georgia and 'Bama certainly fit that pattern, but others like Texas A&M, Miami, Texas (this year excepted) don't seem to put out as much draft production as you would expect based on their consistently good recruiting scores.
April 29th, 2024 at 11:39 AM ^
Maybe it's too early on a Monday and I'm not looking at things like p values but I keep seeing the opposite as you. It seems recruiting drives draft picks. Your counter to that was that Texas A&M, Miami and Texas don't put out the NFL talent that the rankings would indicate but that doesn't support your predictions either. If draft picks drives future recruiting (and inversely lack of draft picks should hurt recruiting), you would predict those same three schools to recruit poorly... but they don't. A&M is probably not a good comparison because they had that monster NIL class that completely fell apart but Texas shouldn't recruit well if they don't have many NFL draft picks yet they do. I like what you're doing and think there is probably something here but to me it's more about outperforming recruiting rankings, not predicting future recruiting rankings. Do you consider transfers? If Team A recruits a kid and he transfer to Team B for 1 year and gets drafted are you giving credit to Team B for that draft pick? Does Iowa get credit for having Erick All get drafted even though he only played a couple games for them? Do we deserve much credit for getting Barner or Henderson drafted? It would be interesting to see a similar model where you remove a kid from this model if they have transferred. Neither the original or final school gets credit. That would show how well a team recruits, develops and puts players in the NFL without some of the high variability noise of the portal.
April 29th, 2024 at 11:57 AM ^
Apologies for sounding argumentative in my response. That certainly wasn't my intent and I do agree with you to a point.
I don't think I'm really making a prediction about whether draft picks matter for recruiting, but just testing a hypothesis. think my bigger point is that, though the model shows that there is a correlation between draft picks and recruiting success, I'm a bit more skeptical. There's just too much dispersion to give me much confidence in the results. There are a lot more (and more important) factors that go into recruiting success and I'm trying to get ideas on other factors that other people would think to include in the model as well.
As far as transfers go, I don't know either way. I guess I'm thinking that recruits wonder "does Michigan have a history of getting people drafted?".
I was thinking the same thing, but on second thought not sure those are the data you're showing? That would require looking at recruiting followed by the draft results, but here you have the draft results followed by recruiting.
April 29th, 2024 at 11:10 AM ^
How much did you account for transfers - one 4 or 5 star guy could come into say TAMU and then end up at Bama and get drafted?
April 29th, 2024 at 12:03 PM ^
That's a good point. I'm assuming that recruits attribute the draft picks to the schools they ended up at, not those they transferred from. Maybe I could weight the draft picks by the number of years they were at that school, with schools getting full credit for the draft pick if the player spent 3+ years at that school?
April 29th, 2024 at 11:24 AM ^
+1 for introducing the first (to my knowledge) interactive graph to the MGoBoard!
April 29th, 2024 at 12:06 PM ^
Thanks! The website https://app.datawrapper.de/ did most of the work for me. I lose some flexibility (i'd like to be able to put some confidence intervals around the prediction) but it's still a pretty cool tool.
April 29th, 2024 at 11:34 AM ^
I wonder if any effect would be left if you take out Georgia, Alabama, and Ohio State.
From the eyeball regression, it's apparent that they are strong contributors to the slope and significance. But is it really the number of draft picks from the prior two years that are the driver of their recruiting success?
April 29th, 2024 at 12:14 PM ^
Yep. They are driving the results to a certain extent. I'll have to give it a shot after taking them out.
April 29th, 2024 at 11:37 AM ^
The "real" question is - What is the average cost SEC schools and O$U spend - "per recruiting star" - for their classes? This includes both "above the table" - NIL contracts and includes "transportation costs" - such as a Lamborghini, as well as traditional "under the table" agreements shared via a Brown Bag delivery program.
Not just SEC and OSU, B1G newcomers Oregon and USC are also spending a ton. Take a look at the Oregon roster and look at the USC transfers.
I think 2023 was the last year of somewhat normal, traditional roster building and from here on in it will be the Wild, Wild West show when it comes to NIL spending. I agree that CF players needed some compensation going forward but this has blown up into insanity.
Since O$U was brought up, I read a few weeks back on Eleven Warriors that one of their NIL collectives - The Foundation - raised $1.5 million in a week just from donations from everyday, blue collar fans - no businesses or wealthy donors were involved. And watch PSU ramp it up - I believe they have the largest - or one of the largest - alumni groups in the country.
Like it or not, this is what we're up against.
April 29th, 2024 at 11:38 AM ^
As a guy who got an A in a 400-level UM Stats class, this is pretty darn impressive!
Curious if you had looked in the prior years too - specifically MSU - when Dicktonio was at his most obnoxious "we're selling results here" while we were floundering with good recruiting classes under Hoke.
As for new coaches, the challenge is "did the recruits consider the new coach an upgrade over the last one?" - Kevin Sumlin was perceived as an upgrade over Mike Sherman at TAMU, ditto for Fisher over Sumlin, but not sure Aggie fans (and recruits) considered Elko a great hire.
Texas 2018 and Ole Miss 2016 are very interesting. Texas was going through the post Mack Brown era with Charlie Strong and then Tom Herman, who could recruit but not coach worth a darn. Looks like a lot of recruits never developed to be drafted.
As for Ole Miss, that was the crooked (and sleazy, as in using university plane and cell phone to call for escort services) Hugh Freeze era. They were busted later for recruiting violations. Another batch of high stars who never developed.
April 29th, 2024 at 12:38 PM ^
Thanks! I haven't looked at earlier years, but (I think) I have all the data to do it. For some reason, I think the 247 recruiting data gets a bit spotty before 2010, so I could probably start there.
That's a good point about whether the new coach is considered an upgrade. The hard part would be finding a way to measure that. Maybe look at the coaches winning percentage over their entire tenure, so a new coach would come in with some history compared to the previous coach. But I'm not sure how to capture this with first-time head coaches (Sherrone or Herman at Houston, for example).
April 29th, 2024 at 12:05 PM ^
Does the model take into account where in the draft kids went vs. the next year's recruiting rankings? Intuitively, I'd think that saying 'we got 3 first rounders, and 17 overall' has more impact than saying 'we got 20 overall,' and then the kid digs a little and sees those are lumped into the later rounds.
April 29th, 2024 at 12:21 PM ^
Good point. I do that in a different model and yep, as you expect, round 1 and round 2 draft picks have a greater effect than later round picks.