Predicting basketball RPI What is RPI? • Ratings Percentage Index • Based on win/loss percentage throughout the season. • Not necessarily a predictor of a stronger team. How is RPI Calculated? • • • • Weighted wins, losses Wins worth 1.4 away, 1 neutral, .6 at home Losses worth 1.4 at home, 1 neutral, .6 away Two parts: – Win pct (wins/(wins+losses) – Strength of schedule • Opponents unweighted win pct • Opponents’ opponents unweighted win pct What is RPI used for? • Estimator of team strength, as it factors in strength of schedule • Helps to seed the NCAA tournament. • Helps selection committee/analysts determine quality of wins. The Selection Committee • 10-person committee that determines who will receive an at-large bid and seeding for the tournament • 5 year Tenure • Use multitude of selection tools – – – – – – Win/loss Conference strength How a team won Voodoo Apparently, more random numbers than me. Decidedly NOT just RPI. How can RPI be predicted? • Predict outcomes of games • Run through season • Rinse, repeat (Monte-Carlo!) Kenpom statistics • Statistics on all division 1 basketball teams – Offensive Efficiency – Defensive Efficiency – Tempo • Average possesions per game: FGA-OR+TO+.42 FTA • • • • Meteorologist from Salt Lake City, Utah Basketball stats just a hobby, no background Stats referenced by ESPN, wall street journal Season averages Step 1: Predicting games • Generate scores: compare and mark • Home games: – xscore<round(x[8]/100*rnorm(1,1.05,.2)*(((x[4]+y[4])/2)* rnorm(1,1.1,.2))+((y[12]*rnorm(1,1.05,.2))/80)) – yscore<round(y[8]/100*rnorm(1,.95,.2)*(((y[4]+x[4])/2)*r norm(1,.9,.2))+((x[12]*rnorm(1,.95,.2))/80)) Step 2: Recording data • Compare scores • Higher score wins • Mark wins/losses in appropriate places Step 3: Run through season • 5284 division 1 vs division 1 games. • Import list of all games, which team is home, away, to be called and put into game function. • Run 1 of three game situations based on court (1 is home, 2 is away, 3 is neutral). • Each team plays approximately 30 games. Step 4: Compile RPI, rank, repeat! • After season is done, run through game list to grab opponent’s win-losses. • Next, re-run through game list to grab opponent’s opponent’s win-losses by. What do these results mean? • Interesting estimator, but cannot be taken too seriously. • True Top 25 RPI missed by average of 13 places. • My top 25 missed true RPI by average of 11. • Kansas, WVU right where they should be! • Villanova ranked 18, therefore project should be considered a success. Is the RPI reliable? • [252]Wake Forest(7.4%), [202]DePaul (9.9%) more likely to make tournament than nearly 200 other teams based on RPI alone. • Too much weight placed on who you play, not how you play. • Still only one factor in determining NCAA tournament. Interesting Oddities • Program took over 15 hours to run. • In 10,000 simulated seasons, 31 teams will not receive an at-large bid (will not be in the top 37 RPI at the end of the season) • Of those 31 teams, half of them would likely end up as a play-in team going to the final four. – VCU received at large bid with RPI rank of 49 (to fill 36th-38th at large bid), Harvard with RPI rank of 35 denied tournament bid Are these results reliable? • Maybe? • Only 2 teams predicted correctly in top 25. • Season averages inaccurate for day-of play, but might average out over whole season. • Effect of random variables should eventually absorb things like suspensions, injuries, team morale. • Possible that one of the seasons actually matches this season perfectly. Theoretical/Technical Issues • Unable to account for mid-season tournaments, changes in schedule, delayed games. • ‘Labor Intensive’ program – 5 trillion calculations. • Still near-impossible to seed mock tournament without just taking 68 highest RPI (which might not be a bad idea) March Madness • There are about 14,757,395,260,000,000,000 different brackets of the NCAA tournament (but only 1 winner!) • Over 6 million brackets were submitted to ESPN.com this march in competition. • Of those brackets, the best bracket, just 1 of 6 million, got 52, or 77.6% of their picks correct. • This year was the first time 2 11 seeds made the sweet 16, and the first time no 1 or 2 seeds made the final 4. • Most even field the tournament has ever had, no great teams My terrible bracket • My original bracket: – 33.6th percentile on ESPN at 480 pts (4-millionth place) Champion: Notre Dame – No final four team, only 2 elite 8 (UNC, Uconn) Basically, terrible. Last place in every pool I was in. Can I make my bracket any better? • This year, no, but next year? Maybe! • Goal: create a bracket based on Kenpom rankings, and see if it does any better. Results: • Worth twice as many points on ESPN, enough to put me in the 92nd percentile! – Beat 5,520,000 brackets! • Still had no final 4 team. Is it a reliable method? Votes for no: • Highest championship percentage was less than 6%, only 4% better than flipping a coin. • Many games were decided by less than a percentage point • Doesn’t take into account injuries, coaching, stage fright, ‘home field,’ streaks, incredible ability to lose the lead, or recruiting violations. Examples: – Georgetown and St. Johns both had their star players hurt going into the tournament and lost in the first round. – Tennessee head coach Bruce Pearl was hit with school and NCAA sanctions the day before the game and lost by 30 points. – George Mason entered the tournament on an 11 game wnstreak Votes for Yes: • Randomness exists to account for the issues previously mentioned. • Anything can happen, this season could have been that 1 in 10,000 chance for VCU, data could be reliable. • 55.2% of the bracket picked correctly, up from… well, zero-ish. Oddities and anecdotes • First trial of the tournament I ran (before looping) yielded Butler over Uconn, with Kentucky and Kansas in the final 4. • Defeated teams sometimes more likely to advance: Notre Dame has a higher chance of making the championship game over Wisconsin, 8.47% over 8.38%, but Wisconsin is more likely to win the championship, 4.90% over 4.76% • VCU had only a 51.5% chance of winning it’s first game, a 22% chance of advancing past Georgetown, and a 2.64% chance of advancing to the final 4. Conclusions: RPI • Can RPI (remember RPI?) be predicted for a season using Monte-Carlo methods? – Decent yardstick, but not perfect – Since RPI is just a yardstick anyway, should work okay. • Can it be used to seed a tournament? – Difficult but yes, would need to run through 31 conference tournaments and determine an Ivy league AQ as well. Conclusions: NCAA tournament • Can Monte-Carlo methods be used to predict the NCAA tournament? – Better predictor than me and better than a coin flip. – Good for calculating odds but not for absolute winner.