RIP, RPI: The NCAA has a new ranking system for college basketball teams
The NCAA announced Wednesday that it has replaced the Rating Percentage Index as its primary tool for evaluating teams’ résumés for the NCAA Tournament. A new ranking system, the NCAA Evaluation Tool, will be used starting this coming season.
“What has been developed is a contemporary method of looking at teams analytically, using results-based and predictive metrics that will assist the Men’s Basketball Committee as it reviews games throughout the season,” said Dan Gavitt, senior vice president of basketball for the NCAA. “While no perfect rankings exist, using the results of past tournaments [as test sets] will help ensure that the rankings are built on an objective source of truth.”
RPI had been used to help select and seed teams in the men’s NCAA Tournament since 1981. The women’s tournament adopted the same method a year later. Unfortunately it was a highly inefficient means of gauging a team’s strength. For example, if one used RPI to fill out last year’s men’s bracket (minus the play-in games), one would have ended up with 39 of 63 correct selections, the second-worst result among 75 rating systems tracked by statistician Kenneth Massey. RPI ended up 36th out of 61 systems (38 correct) in 2016 and 40th out of 64 systems in 2015 (43 correct). Plus, RPI had too much influence in the seeding process, a concern magnified further when the committee altered the definition of a quality win, placing more emphasis on winning away from home.
The new ranking system, referred to as the NET, relies on “game results, strength of schedule, game location, scoring margin, net offensive and defensive efficiency and the quality of wins and losses.” The dates of games and the order in which they are played are not included in an effort to give equal importance to early and late season games. Also, margin of victory is capped at 10 points, preventing teams from getting a boost by running up the score against inferior opponents.
The quadrant system, adopted last season to better reward success in games played on the road, will remain as another tool to be used by the committee in its selection process, only now the NET rating will be used to classify
teams within the quadrant system rather than RPI.
Unfortunately, no specifics were given Wednesday as to how the new elements of the ranking are determined, or who would be calculating new statistical categories in the system, such as net offensive and defensive efficiency. RPI wasn’t perfect but it was transparent, with several websites offering an RPI ranking list based on public-facing data.
The NCAA’s news release did say the new NET model used team performance data to predict the outcome of games in test sets and optimized that until it was as accurate as possible. This statement raises the concern of overfitting the data, which simply means you “train” the data on known outcomes only to find out later its predictive ability is subpar — but there isn’t enough detail in the release to get into specifics. Time will tell how predictive, and rewarding, the new process will be.
The bright side is the NCAA’s willingness to expand its use of less-traditional metrics in the hopes of giving more deserving teams entry into the end-ofseason tournament, a move that hopefully adds more madness to March.