NET replaces RPI, early returns give 'worst rankings ever'
Back in August, it was said a new system termed NET would replace RPI in NCAAM. See here. The NET ratings consider strength of schedule and margin of victory and will be the primary measurement the tournament selection committee will use to finalize the field of 68 in March.
Now however, the first rankings have come out and are, well, baffling. Article here. Frankly, though it may be way too early and more data will normalize the results. But man, it's going to look quite ridiculous until it does. Another reason to not have analytics until later in the season.
Here are some select quotes
Among the head-scratchers in the initial rankings released three weeks into the 2018-19 season: Kentucky is ranked 61st. The Wildcats have one, 34-point neutral site loss to Duke, the Associated Press poll's No. 1 team last week.
Ohio State, which has wins against four sub-200 squads in KenPom.com's rankings, is No. 1 in the NET ratings.
Michigan State is also four spots ahead of the Kansas team it lost to in the Champions Classic.
Among the critics of the new system Monday was statistician Nate Silver of FiveThirtyEight, who called the first list "the worst rankings I've ever seen in any sport, ever."
November 27th, 2018 at 7:37 AM ^
The rankings look terrible, but B1G teams are littered amongst the top 10 - so while it is laughable at this early stage, at least the early returns look favorable for us.
November 27th, 2018 at 7:43 AM ^
The Nate Silver quote sounds like a Trumpism, which is not a great sign if it comes out of a normal person's mouth.
It should also be noted that the NET ranks Loyola Marymount at 10, and they're at 120 in Kenpom. Stuff like this can be weird with small sample sizes but that's not a great sign.
November 27th, 2018 at 8:24 AM ^
You're the worst poster ever. Terrible poster. Sad!
November 27th, 2018 at 9:09 AM ^
Aside from the injection of politics (Trumpism), the poster makes a good point. Nate Silver knows small sample sizes produce weird results, and knows better than to call anything "the worst" before it has enough data to be appropriately judged.
November 27th, 2018 at 9:37 AM ^
And Sr. Trump does say "the worst" a lot. But that was probably going to go over like a lead balloon!
I wonder if the NCAA had just said "these rankings are going to be relatively meaningless until we're ten or so games in" they could have saved themselves a lot of heartache. I don't think the powers that be at NCAA HQ have any idea what disdain even the average fan holds for them. . .
November 27th, 2018 at 9:38 AM ^
exactly - why nate silver wouldn't say "let's wait and see" is beyond me.
November 27th, 2018 at 10:40 AM ^
Have you seen my posts? They're tremendous. Everyone thinks so. Really really great. Huge posts, very important content, believe me.
November 27th, 2018 at 2:03 PM ^
Dude, this isn't MLive. Take it easy with the "let's interject politics into everything."*
I like robots too.
*the funniest randomest example is a YouTube video I was watching about how to use electric arcs to clear a yellow jacket infestation. Middle of the guy's instructions, he exclaimed "they're like liberals", then just kept going. First 15 comments below the video were about comparing liberals to yellow jackets.
November 27th, 2018 at 7:47 AM ^
Just use Kenpom or Torvik.
November 27th, 2018 at 9:06 AM ^
Predictive models don't work for tournament selection, which is based on prior results, and (rightly so) heavily weighted towards wins and losses over points per possession.
November 27th, 2018 at 10:21 AM ^
Predictive models measure quality of opponent. It is a better model to use when you have to put wins buckets (1-50, 51-100, etc) and if you have 2 teams that have similar resumes it is a better model to use to say which team is better. After all, aren't we looking for the 68 best teams?
November 27th, 2018 at 11:14 AM ^
It's the eternal debate -- résumé vs. team strength. There's no universal answer.
November 27th, 2018 at 8:05 AM ^
Any time Ohio State is ranked #1 in a poll regarding athletics, I call BS too.
November 27th, 2018 at 8:44 AM ^
"That's BS."
-Mork D'antonio
November 27th, 2018 at 2:08 PM ^
Pretty sure Mark left this kind of stuff in gods' hands. I talked to Apollo about, he didn't care one way or the other
November 27th, 2018 at 8:14 AM ^
The reason these rankings look so far off is that they don't use preseason projections. They only use actual results from this season. At this stage, Kenpom ratings are mostly based on preseason expectations, not on the actual games.
I think they released their rankings too soon. The sample size is way too small to get reliable numbers no matter how good your system is. And knowing the NCAA, it's probably not very good at all. Why now? Why not after the first week? Those would have really been some crazy looking rankings.
For everyone complaining about this, remember all the complaints about the football polls. That there is far too much emphasis placed on the preseason polls and it's too hard to jump teams and unfair if you start low. It pretty much has to be done one way or the other.
November 27th, 2018 at 8:19 AM ^
This about nails it. KenPom, Sagarin, etc. all use preseason projections and carryover from last year. The NCAA does not and should not, since their need for neutrality is much greater.
November 27th, 2018 at 9:32 AM ^
Excellent point and one I had not thought of until you mentioned it. You would think the people running this rating service would make a MUCH bigger deal out of this fact given that it clearly exposes the gap between what the "experts" thought a team would play like going into the season and how they're actually playing.
November 27th, 2018 at 11:00 AM ^
Well, but it doesn't. What it really does is expose the difference between how a team is expected to play for the season and its résumé to date. Consider Kentucky: they're 5-1 with 5 wins -- Southern Illinois, North Dakota, VMI, Winthrop, and Tennessee State -- and the blowout loss to Duke. If their jerseys said "Belmont," nobody would blink about having that team at #61.
That doesn't mean they aren't playing well -- although, looking at the efficiency margins in these games, they're really not playing great -- it just means they don't have a tournament résumé yet.
November 27th, 2018 at 10:56 AM ^
The RPI always looked awful after three weeks too. This is much ado about nothing. I'm not sure NET will be a good ranking system, but it's too soon to tell.
What the NCAA really should have done is release the formula and underlying data to allow people to go back and recompute it for past years. Not only would that have given us more information than the currently daily snapshots will, making the data public allows people to catch and fix errors.
The only good thing about the RPI was that it could be replicated easily, so when there were discrepancies, they could be reported to the NCAA and addressed.
November 27th, 2018 at 8:18 AM ^
I don't see anything wrong with the rankings, except that people care about rankings so early in the season.
If a team loses by 35 points, it may not be a good team. If it is it will be ranked higher in subsequent weeks.
Anything that eliminates the bias of preseason rankings is good in my opinion.
November 27th, 2018 at 8:40 AM ^
though it may be way too early and more data will normalize the results
Yes... this. Small sample size.
November 27th, 2018 at 8:52 AM ^
Like many have said, this is a small sample size. I bet these rankings will be much more toward the norm at the end of December.
I would like to see a breakdown of each team's data in this metric. Does anyone know where to find that?
November 27th, 2018 at 9:17 AM ^
AFAIK, there is no "under the hood" data available and it doesn't appear that the NCAA intends to release it.
November 27th, 2018 at 9:07 AM ^
I’m sure some Ohio State homer is slobbering all over themselves
...and by “homer” I mean “their official twitter account”.
https://twitter.com/ohiostatehoops/status/1067108222572605440?s=21
November 27th, 2018 at 10:19 AM ^
At least until there is much more solid data (like others said, to give this a fair shake, they probably should have withheld releasing these until after the non-conference schedule had mostly played out), grabbing onto this particular stat is pretty much like celebrating being the #1 used underwear salesmen in a place like Anderson, Indiana.
November 27th, 2018 at 10:26 AM ^
Weird they only included the top 3 in that post.
November 27th, 2018 at 11:11 AM ^
For a state that pretends the letter M doesn't exist for a week per year, are you surprised?
They're the most insufferable lot.
November 27th, 2018 at 1:06 PM ^
Without a doubt.
November 27th, 2018 at 9:15 AM ^
Should have waited til January to release the initial rankings. Teams will have all played at least 10-12 games.
November 27th, 2018 at 10:26 AM ^
yes. yes, yes, yes...
this is akin to publishing the cfp rankings after week 1. remember when virginia tech whipped up on florida state in their opener, and shot up to #12 in the country? same thing.
(they lost to old dominion in game 3)
November 27th, 2018 at 9:59 AM ^
This is probably the worst ranking system ever....maybe it will work itself out down the road as it seems to put a huge emphasis on road wins, so any team with a few road wins even against trash teams is boosted.
When Radford & Belmont are in the Top 25 you know the poll is trash
November 27th, 2018 at 10:47 AM ^
It was a dumb system when they announced it, and it was fun seeing stats guys like Nate Silver and guys like Ken Pomeroy and Jeff Sagarin either light it on fire or just ignore it completely.
I get that models take some time to get fixed, but this felt poorly conceived and implemented from the jump.
November 27th, 2018 at 12:02 PM ^
My great hope for the NET rankings is that by season's end, they'll be pretty damned accurate and strongly demonstrate the silliness of preseason polls, thereby rendering them powerless.
November 27th, 2018 at 1:42 PM ^
Except that KenPom has found that the preseason polls are actually more predictive than the end-of-season polls, and not actually that bad of a predictor in the grand scheme of things: