Semi-Objective CFP Rankings System: 2016 Week 10 Results
At the conclusion of last season, I posted the results of a fun exercise that took a relatively objective look at ranking teams based on the quality of their wins and losses. It was inspired by Seth's post that proposed a point system to determine bowl eligibility. The premise was to create a ranking as a starting point from which to apply considerations such as the eye test, margin of victory, or head-to-head results.
The feedback I received from last year's post led me to adjust the model such that top 10 and top 25 wins/losses would refer to the model's own rankings and iteratively converge to a stable ranking. This was an improvement over using the end-of-season AP poll as the top 10 and top 25 rankings. In addition, wins over FCS teams are no longer considered to determine a team's "winning" status. Below you will find the point system description and the rankings resulting from this model as of the conclusion of week 10.
I would love to have feedback from the community again, especially with regards to:
- The weight of points for each category of win or loss.
- The weight of points for conference championship.
- Should there be a bonus for division winners?
- Should Notre Dame and/or BYU be considered power 5 teams? (They were not, here.)
- What's the best way to break a tie in points? (So teams do not have the same rank.)
- What to do when the system does not converge?
The rules for this proposed system are:
+3 points for a conference championship.
+4 points for a win over a top 10 team.
+3 points for a win over a top 25 team (not in top 10).
+2 points for a win over a winning P5 team (not in top 25).
+1 point for a win over a winning G5 or losing P5 team.
+0 points for a win over a losing G5 or any FCS team.
-1 point for a loss to a top 10 team.
-2 points for a loss to a top 25 team (not in top 10).
-3 points for a loss to a winning P5 team (not in top 25).
-4 points for a loss to a losing P5 or any G5/FCS team.
2016 Week 10 Results
Rank | Team | Points |
---|---|---|
1 | 'Alabama' | 18 |
2 | 'Michigan' | 17 |
3 | 'Clemson' | 14 |
3 | 'Ohio State' | 14 |
5 | 'Washington' | 10 |
6 | 'Penn State' | 9 |
7 | 'Texas A&M' | 8 |
8 | 'Auburn' | 7 |
8 | 'Louisville' | 7 |
8 | 'Wisconsin' | 7 |
11 | 'North Carolina' | 6 |
12 | 'Nebraska' | 5 |
12 | 'Tennessee' | 5 |
12 | 'West Virginia' | 5 |
15 | 'Arkansas' | 4 |
15 | 'Colorado' | 4 |
15 | 'Oklahoma State' | 4 |
15 | 'Virginia Tech' | 4 |
19 | 'Boise State' | 3 |
19 | 'Florida' | 3 |
19 | 'Florida State' | 3 |
19 | 'Minnesota' | 3 |
19 | 'Oklahoma' | 3 |
19 | 'Washington State' | 3 |
19 | 'Western Michigan' | 3 |
26 | 'Stanford' | 2 |
27 | 'LSU' | 1 |
27 | 'USC' | 1 |
27 | 'Troy' | 1 |
27 | 'Utah' | 1 |
31 | 'East Carolina' | 0 |
31 | 'Wyoming' | 0 |
33 | 'Baylor' | -1 |
34 | 'Pittsburgh' | -2 |
35 | 'Appalachian State' | -3 |
35 | 'Houston' | -3 |
35 | 'San Diego State' | -3 |
35 | 'South Florida' | -3 |
35 | 'Tulsa' | -3 |
40 | 'Georgia' | -4 |
40 | 'Iowa' | -4 |
40 | 'Maryland' | -4 |
40 | 'Northwestern' | -4 |
44 | 'Kentucky' | -5 |
44 | 'Miami (FL)' | -5 |
44 | 'Navy' | -5 |
44 | 'South Carolina' | -5 |
44 | 'Wake Forest' | -5 |
49 | 'Georgia Tech' | -6 |
49 | 'Indiana' | -6 |
49 | 'Kansas State' | -6 |
49 | 'Ole Miss' | -6 |
49 | 'TCU' | -6 |
49 | 'Toledo' | -6 |
55 | 'BYU' | -7 |
55 | 'Temple' | -7 |
55 | 'Texas' | -7 |
55 | 'Western Kentucky' | -7 |
59 | 'Arizona State' | -8 |
59 | 'Lousiana Tech' | -8 |
59 | 'New Mexico' | -8 |
59 | 'Ohio' | -8 |
59 | 'Vanderbilt' | -8 |
64 | 'Colorado State' | -9 |
64 | 'Middle Tennessee' | -9 |
64 | 'Mississippi State' | -9 |
64 | 'NC State' | -9 |
64 | 'Syracuse' | -9 |
64 | 'Texas Tech' | -9 |
70 | 'Air Force' | -10 |
70 | 'Boston College' | -10 |
70 | 'California' | -10 |
70 | 'Memphis' | -10 |
74 | 'UCF' | -11 |
74 | 'Idaho' | -11 |
74 | 'Illinois' | -11 |
74 | 'Old Dominion' | -11 |
74 | 'Oregon' | -11 |
79 | 'Army' | -12 |
79 | 'UCLA' | -12 |
79 | 'Eastern Michigan' | -12 |
79 | 'North Texas' | -12 |
79 | 'UTSA' | -12 |
84 | 'Arkansas State' | -13 |
84 | 'Southern Miss' | -13 |
86 | 'Hawaii' | -14 |
86 | 'Oregon State' | -14 |
86 | 'Purdue' | -14 |
86 | 'Rutgers' | -14 |
90 | 'Akron' | -15 |
90 | 'Central Michigan' | -15 |
92 | 'Charlotte' | -16 |
92 | 'Duke' | -16 |
92 | 'SMU' | -16 |
95 | 'Ball State' | -17 |
95 | 'Georgia Southern' | -17 |
97 | 'Cincinnati' | -18 |
97 | 'Kent State' | -18 |
97 | 'Louisiana-Lafayette' | -18 |
97 | 'Miami (OH)' | -18 |
97 | 'South Alabama' | -18 |
102 | 'Louisiana-Monroe' | -19 |
102 | 'Missouri' | -19 |
102 | 'New Mexico State' | -19 |
102 | 'Notre Dame' | -19 |
102 | 'Utah State' | -19 |
107 | 'Arizona' | -20 |
107 | 'Michigan State' | -20 |
107 | 'Virginia' | -20 |
110 | 'Texas State' | -21 |
111 | 'Northern Illinois' | -22 |
111 | 'UTEP' | -22 |
113 | 'Iowa State' | -23 |
113 | 'Tulane' | -23 |
115 | 'Georgia State' | -24 |
115 | 'Kansas' | -24 |
115 | 'Marshall' | -24 |
115 | 'Nevada' | -24 |
115 | 'UNLV' | -24 |
120 | 'San Jose State' | -25 |
121 | 'FIU' | -26 |
121 | 'Florida Atlantic' | -26 |
123 | 'Buffalo' | -27 |
123 | 'Connecticut' | -27 |
125 | 'Bowling Green' | -29 |
125 | 'Massachusetts' | -29 |
127 | 'Rice' | -31 |
128 | 'Fresno State' | -33 |
November 10th, 2016 at 1:15 PM ^
Are you consciously excluding margin of victory from your system?
November 10th, 2016 at 1:16 PM ^
November 10th, 2016 at 3:23 PM ^
CFP Selection committee states they don't take margin of victory into consideration. So assume that is why op didn't.
November 10th, 2016 at 1:24 PM ^
For the moment, yes. It's hard to treat margin of victory robustly. Does a 30+ point win over top 10 Penn State mean the same as a 30+ point win over a losing record Illinois? For now I'm leaving it off for that reason, and also because I'm too lazy to gather winning margin data... for now.
November 10th, 2016 at 2:49 PM ^
Sent from MGoBlog HD for iPhone & iPad
November 10th, 2016 at 3:16 PM ^
To account for margin of victory, you could add half of the points awarded for a win greater than 3 scores (>17 points). This would award blowout wins over good teams more than blowout wins over average teams.
For example: Michigan's win over Penn State would gain them 6 points (4 for a Top 10 win + 2 for a blowout win); Michigan's win over UCF would gain 1.5 points (1 for Group of 5 w/ winning record + 0.5 for blowout win); Michigan's win over Rutgers would still net 0 points :(
I like the rankings so far. They certainly pass the eye test.
EDIT: I misread your point system. The Rutgers win would gain 1.5 points. Hawaii would have been a better example of a 0 point blowout win.
November 10th, 2016 at 3:30 PM ^
I like this idea (even though I'm not totally convinced margin of victory should come into this ranking), and Kzoo's idea above it. However, I'm still going to leave margin of victory out for now because I don't want to scour games to add in margin of victory data to the model (we're talking nearly 600 games so far). If someone has an easy way to pull this data (each team's margin of victory/defeat against each of their opponents), I'd love to hear it.
November 10th, 2016 at 3:59 PM ^
go to this website
Copy and paste as text into excel.
Highlight column, go to Data/text to columns/ Fixed width --> then scroll to the right and add one more line after the home scores.
Boom - all results in excel ready to parse.
November 10th, 2016 at 4:19 PM ^
This is fantastic. Thanks!
November 14th, 2016 at 4:34 PM ^
Do you know if something like this exists for previous years?
November 16th, 2016 at 11:11 AM ^
He used to host a lot of years, but it doesn't look like they're still there.
But, go here.
Down at point 26, there are past seasons, and those links have season schedules, but I think they're only D1A v. D1A. You may have to go in a delete the dates before you do the text to columns thing though.
November 21st, 2016 at 4:45 PM ^
Thanks again! Unfortunately for the older ones, even after you delete the dates you still have different column widths so you can't easily separate scores from team names (especially because some team names have spaces so you can't use a space as a delineator). Could probably make a script to read it in but can't justify putting the time into it.
November 10th, 2016 at 9:56 PM ^
Major rugby tournaments integrate a margin-of-victory aspect into pool play. 4 points for a win, 2 for a draw, and 0 for a loss. However, a team can earn an extra point by scoring 4 tries (my Aussie/Kiwi friends still laugh at me when I say touchdown because they LITERALLY have to touch the ball down to the ground) and teams losing by less than 8 points get somewhat of a consolation point.
This wouldn't necessarily work for your hypothetical system because teams aren't attempting to hit these "bonus point thresholds," but this system seems to have worked in rugby. Yes, it was a change, but teams adjusted. You'll see a team up by 25 points working to score a try in the waning minutes or a team down 9 trying desperately to score a kick to get the extra point.
Biggest detraction to this is that it's in pool play where all teams play each other within the same pool, which isn't the case for NCAAF. Only works if everyone is playing the same slate of games.
November 10th, 2016 at 1:17 PM ^
Are the opponents judged based on where they're ranked now or where they were ranks when the game was played??
November 10th, 2016 at 1:20 PM ^
They are judged on their overall resume, not their resume when they played. The rankings are determined by this system, not the CFP or AP or other rankings.
November 10th, 2016 at 2:47 PM ^
So each week the teams are placed in a certain order. Then each team that beat one of the top 25 on that ranking system gets extra points? Won't that change the order again? Do the teams that beat that new list of top 25's get additional points now? When does it end? Am I over thinking this?
November 10th, 2016 at 3:00 PM ^
Yes, it does change the order and the calculations are done again. And again and again. It ends when the calculation is redone and there was no change to the rankings. This week it only took 6 iterations before the model converged on a constant ranking for each team.
November 10th, 2016 at 3:04 PM ^
Thank you for clearing that up.
November 10th, 2016 at 1:57 PM ^
Grades out better than MSU. Should be a fun Toilet Bowl this week in the B10. Is it bad that I'm rooting for the Cable Subscribers?
November 10th, 2016 at 2:23 PM ^
November 10th, 2016 at 2:22 PM ^
I saw someone else call MSU "Green Rutgers" and I have to say that it makes me laugh every time I think of it. Rutgers vs. Green Rutgers Saturday at noon on BTN, haha!
November 10th, 2016 at 3:45 PM ^
According to this, Staee is two losses to a winning P5 program WORSE that Rutgers. The schadenfreud is delicious.
November 10th, 2016 at 3:43 PM ^
What about doing using the S&P+ and doing an inverse rating for wins, so 128 pts for beating the #1 team and 1 point for beating the #128 team, and subtracting loses with the actual rank, minus 1 point for losing to the #1 team and minus 128 points for losing to the #128 team. Zero points for an FCS team, minus 128 points for a loss. Maybe add in a home/away point modifier. Maybe this is too simple or is something that is already taken into account by S&P+. I'm not much of a statistics guy.
I did the top 6 teams in the CFP poll in this manner. I didn't add a home/away or margin of victory modifier.
1. Alabama - 817pts
2. Clemson - 695pts
3. Michigan - 606pts
4. OSU - 576pts
5. Washington - 421pts
6. Louisville - 410pts
November 10th, 2016 at 11:15 PM ^
I like this because I don't trust the human polls. Even late into the season there's too much preseason bias and I do kind of think that West Coast teams get screwed because most of the voters are east.
November 10th, 2016 at 3:48 PM ^
I feel like the committee is harping on UM for not having any big road wins...
November 10th, 2016 at 7:49 PM ^
But it is hardly Michigan's fault that a bottomless sink hole opened up in East Lansing. If Sparty is 8-1 with their only loss to Michigan there would be no talk of Michigan's soft road schedule.
November 10th, 2016 at 4:30 PM ^
it's just the old bowl system with a plus-one game.
Committee decisions at the point it matters:
- P5 conf winners minus one -- who is the minus one?
- P5 conf winners minus two with one P5 conf #2
- P5 conf winners minus one with a G5 undefeated -- who is that deserving G5 undefeated?
- P5 conf winners minus two with one P5 conf #2 and one G5 undefeated.
I guess multiple multiple-loss conf winners would make this bollocks, but it's still hard for me to see more than two P5 conf winners getting locked out. Haven't looked at the history, but my admittedly poor memory doesn't bring a lot to mind.
The old bowl conference commitments are probably no worse than seeding 1-4, and adds the benefit of tradition/fan planning on locations.
The championship game? Well, that's special now isn't it?
All the lead-up is woo-woo-ESPN-makes-money.
Not a fan of the lead-up. You win your conference, you get a shot. You don't and you are among the hopers.
November 10th, 2016 at 5:32 PM ^
is there no way to tinker the system to make MSU even worse?
November 11th, 2016 at 8:53 AM ^
I'm sure there is, but they already have zero positive points here since I didn't count Notre Dame as a P5 school. Maybe I can add a losing streak penalty: -10 points for losing 7 in a row. Or maybe a -10 points for losing to Rutgers penalty, if that comes into play tomorrow.
November 10th, 2016 at 9:01 PM ^
November 10th, 2016 at 9:32 PM ^
November 10th, 2016 at 10:27 PM ^
November 11th, 2016 at 5:51 AM ^
November 11th, 2016 at 6:18 PM ^
November 14th, 2016 at 9:33 AM ^
First, remember that winning the conference championship means you also got an extra win over a probably top 25 or top 10 team if you're in a P5 and at least a winning G5 team if not (Big 12 excluded). Last year if I waited the conference championship more than this, it looked as though even bad division winners if they won the championship game would make the playoffs.
Second, I think you're right about Notre Dame, I'm going to make them count as P5.
November 11th, 2016 at 6:25 PM ^
It would be curious to see the results of your method for the last 10 years relative to the final rankings.
November 12th, 2016 at 7:19 AM ^
November 12th, 2016 at 3:24 PM ^
Got to test the null hypothesis with a larger N. It is the Michigan difference.
November 14th, 2016 at 9:38 AM ^
Okay, I will consider this. Right now it would be too onerous to enter the data but I'll make a script to automate it and then it should be relatively trivial.
Curious, would it be best to compare to results before the bowl games (i.e. last CFP/BCS rankings)? Or after bowl games (would have to use AP poll)?
Comments