2014 NFL Draft Efficiency Rankings

The 2014 NFL concluded on Saturday evening.  The three-day event featured Johnny Manziel taking over the Twitterverse on Thursday night, and the St Louis Rams selecting Michael Sam near the end of the draft on Saturday.  A lot of the post-draft analysis was either based on total number of draft picks from a college or draft picks from a college adjusted for when they were picked in the draft.  Of course, there are also a plethora of inane draft grades where clairvoyant “experts” project how well the draft picks will perform on the team.

Our take on the draft is a bit different, as we will examine the process of taking high school talent and converting it into NFL draft picks. In other words, we want to understand how efficient are colleges at transforming their available high school human capital into NFL draft picks?

Our approach is fairly simple.  Each year, every FBS football program has an incoming class.  The players in the class have been evaluated by several national recruiting/ranking companies (e.g. Rivals, Scout, etc…).  In theory, these evaluations provide a measure of the player’s talent or quality*.  Each year, we also observe which players get drafted by the NFL.  Thus, we can measure conversion rates over time for each college.  Conversion rates may be indicative of the school’s ability to coach-up talent, to identify talent, or to invest in players.  These rates may also depend on the talent composition of all of the players on the team.  This last factor is particularly important from a recruiting standpoint.  Should players flock to places that other highly ranked players have selected?

How did you compute the conversion rate?

The conversion rate for each school is defined as (Sum of draft picks for the 2014 Draft)/(Weighted Recruiting Talent).  Weighted Recruiting Talent is determined by summing the recruiting “points” for the relevant eligible class for the 2014 NFL Draft for each program (this can include eligible juniors as well as fifth year seniors).  These “points” are computed by weighting each recruit by the overall population average probability of being drafted for recruits at that corresponding talent level over the last three years.  For example, a five-star recruit is much more likely to get drafted than a four or three-star recruit.  We are using ratings data from Rivals.com.

2014 Full NFL Draft Efficiency

The figure above shows the top ten schools in the FBS for converting high school talent into draft picks for the 2014 draft.  We have indexed the efficiency rating based on the leader, Boise State.  It is interesting to note that the team with the most draft picks in the 2014 NFL Draft, LSU, finished 11th in our rankings.

Do the results of one draft really matter?

A fair criticism of this ranking is that it only represents one draft year; what if this draft was an anomaly for Boise State and Wisconsin?  The rankings below consider the 2012, 2013, and 2014 NFL Drafts.  While Boise State and Wisconsin are still on top, schools such as Connecticut, Iowa, and Nevada are now also in the top ten.

2012-2014 Full NFL Draft Efficiency

How can you treat a first-round draft pick the same as a seventh rounder?

Our study is primarily considered with schools that give high school talent the opportunity to play in the NFL.  Thus, the rankings above do not discern between rounds of the draft.  Ostensibly, a player’s initial contract and status in the NFL seems tied to draft order (although Richard Sherman has done real well for a 5th round pick).  Let’s assume that being picked in the first three rounds of the draft is of importance to players.  We can conduct a similar type of analysis, but only consider picks in the first three rounds of the draft, and adjust the weighting to reflect population averages for being picked in the first three rounds.  The rankings below are based on an analysis of only the first three rounds over the last three years.  Boise State is still on top, but schools like LSU, Cincinnati, & North Carolina have moved up the list.

2012-2014 First 3 Rounds NFL Draft Efficiency

 Of course, there are many other ways for trying to understand or rate draft efficiency.  In the past we have also conducted regression-based analyses with additional data such as program investment to better understand the phenomenon of human capital development in both football & basketball.

Mike Lewis & Manish Tripathi, Emory University 2014.

  *We can already hear our friends at places like Alabama & USC explaining how players are rated more highly by services just because these schools are recruiting them.  We acknowledge that it is very difficult to get a true measure of a high school player’s ability.  However, we also believe that over the last few years, given all of the media exposure for high school athletes, this problem has attenuated. 

The “Best” Football Fan Bases in the Non-Automatic Qualifying Conferences

For our first look at the “best” college fans, we start with schools from Non-Automatic Qualifying (NAQ) conferences (as of the 2013 season affiliation).  As in our previous studies of fan/brand equity, we use a revenue premium model that measures fan support over the last ten years while controlling for team quality (click here for details).  The NAQs are an interesting group as there has been a significant scramble to join major conferences (e.g. TCU, Houston, Utah, etc…) over the past few seasons.  The current “NAQ” plan seems to be to try and build a strong brand in order to generate an invite to one of the more financially lucrative conferences.

This trend towards joining “stronger” conferences places NAQ programs in a curious position.  To get an invite, schools need to be successful and develop significant customer equity.  But as schools like Utah and TCU have found out, shifts to higher profile conferences can also result in less competitive teams.

The number one ranked school on our list was a surprise, as we identified San Diego State as having the most supportive fan base of the NAQ schools.  Surprise, we say?  Actually, this was more of a shocker, but this is the beauty of looking at the numbers.  The thing about San Diego State is that it receives fairly consistent support even when the team struggles.  The best and most illustrative comparison is between Boise State and San Diego State.  Over the last decade, Boise State has won about twice as many games as San Diego State, but only has a slight advantage in terms of attendance and revenues.  What does this mean?  It means that San Diego State has a very valuable asset in its customer base (and could likely benefit by investing more in the program).

The second place team BYU is a solid program both on the field and at the box office.  The ability to attract 60,000+ crowds makes BYU something of an outlier in the NAQ world.

Wyoming in third place was another surprise.  Again, we need to point out that we are controlling for team quality.  The key to Wyoming’s ranking is that revenues and attendance are solid despite some on-field struggles. On the plus side, this level of support for an often struggling Cowboy team suggests that Wyoming might benefit from investing into their program.  On the other hand, perhaps there are just fewer entertainment options in Laramie, and the quality of the team just doesn’t matter since people are looking for things to do.

The Idaho Vandals finished fourth in the rankings.  This is an easy entry to write.  Just replace Idaho wherever you see Wyoming in the paragraph above.  In the fifth position on the list we have the Marshall Thundering Herd.  Marshall is again a solid program that usually averages between 25,000 and 30,000 fans regardless of the team’s record.

As we computed our rankings for the NAQs and for the bigger conferences, the NAQs generated the least intuitive results (e.g. Where’s Boise State?).  As we drilled down, the story became clearer.  First, we are looking at the conferences where schools are currently, rather than where they have been.  This removes traditional powers like Utah and TCU.  The other eye opener came from looking at revenues and attendance figures.  Often the highest profile NAQs do not convert their success to revenues.  While Boise State is arguably one of the most successful programs at any level, the fan support is often not what one would suspect.  Boise State has 20,000+ students, a metro area population of more than 600,000, regularly wins more than 10 games and doesn’t sell out.

The Boise State story also says something about the economics of college sports.  In the absence of significant BCS revenue sharing and conference specific television deals it is hard to justify the investments needed to develop a high quality on-field product.  In other words, Boise fans should probably be grateful for the program they have, and should provide more support.

Mike Lewis & Manish Tripathi, Emory University 2013.



Who’s the “Best” College Football Coach in the Past Decade? We think he works for the Philadelphia Eagles.

As the title implies, we are about to go down a road that will inspire debate and we expect considerable hate.  As we suspected, and have since confirmed in our 3 months of publishing, there is no sport with more passionate fans than college football.  We know that as soon as we provide our ranking of college coaches that we will immediately be told that we are wrong (and in rare cases that we are right).

For our coaching analysis, the starting point is the idea that we should rank coaches’ performance relative to the resources that are at their disposal.  In other words, we can’t compare the coaches at the University of Illinois and the University of Florida simply based on win-loss records. For the analysis, we gathered data on results (winning percentage, major bowl participation), football expenditures, historical performance (won-lost records, major bowls, national championships, etc…), attendance and other factors.

We use this data to create models that predict team success based on financial resources, historical performance and market potential.  These factors can all contribute to on-field success.  For example, the logic of including historical performance is that a more storied program may be more attractive to both bowls and to potential recruits.  We use the models to predict the performance of each school for each of the last 10 years.  We then assess the contribution of the coach by comparing actual to predicted performance.

We analyzed coaches using the past 10 years of data in terms of two criteria: winning percentage and selection to play in a major bowl (Rose, Orange, Fiesta, Sugar, and National Championship).   As an aside, our performance models were both estimated using logistic regression.

In terms of incremental winning percentage effects, the top coach was Chris Petersen from Boise State. Coach Petersen has achieved a 91.3% winning percentage while at a school with only moderate football expenditures (even among non-BCS schools) and limited history.  In terms of specific numbers, we find that Petersen has achieved a winning percentage that is 37% higher than what a school of comparable resources and history achieves.  The top five also includes Urban Meyer, Brian Kelly, Bret Bielema and Bobby Petrino.  In positions six through ten we have Steve Spurrier, Bob Stoops, Gary Patterson and Frank Beamer.  Two other former coaches produced notable results on the winning percentage criterion.  Chip Kelly won 32% and Pete Carroll won 13.3% more games than expected.

Okay so who is missing?

We already anticipate complaints from Alabama fans along the lines of “The list is invalid because at Alabama we play for championships, not for winning an incremental game or two.”  Well, Coach Saban does finish 11th on the incremental winning list, and there is some merit to this argument.  It is more difficult to drive incremental wins at schools like Alabama than Boise State.

In part two of our analysis we looked at incremental participation rates in BCS bowl games (not just the BCS championship).  Our approach was similar as in the winning percentage analysis and used the same set of predictor variables.  On this metric, the top performer was Bret Bielema.  Bielema’s record includes taking Wisconsin to the last three Rose Bowls.  In terms of percentages, we find that with Bielema in charge, Wisconsin improved their rate of BCS bowl participation by 28%.  It looks like Arkansas made a great choice! In positions two through five we have Chris Petersen, Bob Stoops, Frank Beamer and Urban Meyer.  And where does Coach Saban fall?  Just behind Urban Meyer in 6th place.

If we also look at former college coaches one name really stands out.  Chip Kelly was by far the leader on the BCS bowl participation metric. Combined with the winning rate results, one can argue that Chip Kelly has been the most effective college coach over the past few years.

Returning to Coach Saban, first, it must be noted that he scores really well on our measures (11th for incremental wins and 6th for incremental BCS games).  And we definitely understand the argument that he should be number one.  In terms of winning championships, it is hard to argue that he is not the go-to coach.

Mike Lewis & Manish Tripathi, Emory University, 2013.

The Perils of Realignment and a Final Look at the “Fairness” of the BCS System: Would TCU fans rather play for “Major Bowls” or be an average Big Twelve Team?

As the 2013-2014 college football season approaches, we would like to reflect back on one of our favorite sources of college football controversy: The BCS system.  And while we have every confidence that the extension to a 4 game playoff will ALSO provide an annual source of controversy (How can two of the four teams be from the same conference?  How can they leave out the second place SEC team? Etc…) we wanted to say goodbye to the BCS by taking a look at how fair it was.  One way to look at the fairness of the system is to look at how likely it has been for different “types” of teams to qualify for, and be selected to play in one of the BCS bowls.  The other purpose of this post is related to the topic of conference realignment.  The past few years have seen a number of schools jump at opportunities to join more financially lucrative conferences.  While the financial benefits of switching to a major conference (or switching from a major conference to the SEC or the Big Ten) are obvious, these moves have a potential downside in that teams moving up the financial ladder may often become less competitive on the field.

To assess the “fairness” of the BCS system we created a model for the relationship between on-field success (as evidenced by the level of bowl game achieved in a season) and variables such as program expenditures, AQ (automatic qualifying conference membership) status, attendance and previous bowl participation.  For this analysis, we used an ordered logit model and defined 4 levels of post-season participation: no bowl game, minor bowl game, major bowl game, and national championship game.  An ordered logistic model is similar in spirit to a basic linear regression but it is designed to model the probability of categories of (ordered) outcomes.  The logic of each of the explanatory variables is as follows:  1. Expenditures are included to control for expected team quality level.  2. Attendance is used as a measure of the team’s fan base (an important factor since bowls prefer teams with large fan bases that are more likely to travel to the bowl site).  3. Previous bowl participation reflects the brand equity or television appeal of the team.  4. AQ status is included to capture the influence of the BCS structure on bowl participation.  Given the concerns expressed about limited access for non-AQs this term is directly related to the controversial nature of the BCS.

The model reveals that selection to play in a “major bowl” is more likely for teams that spend more, have higher attendance and have participated in more bowls in past seasons.  The most dramatic finding from the analysis is the significance and direction of the AQ term.  We find that when we control for the other team characteristics, that AQ conference membership reduces teams’ post season opportunities. The model’s implications are best illustrated graphically.  The figure below shows the relationship between expenditures and major bowl participation for an artificial AQ and Non-AQ school.  The figures are for a school that has participated in 12 minor bowls, 4 major bowls, has won a single national championship and has average attendance of 60,000.  The vertical axis is the probability of a team being selected for a BCS bowl and the horizontal axis is the team’s football expenditures relative to the average expenditures of FBS teams.

For the Non-AQ school, the probability of achieving a major bowl is given for expenditure levels ranging from 50% to 150% of the overall FBS.  For the AQ schools, we plot the probabilities for expenditures ranging from 100% of the average to 250%. When expenditures are controlled for, the probability of playing in a major bowl is significantly greater for Non-AQ schools.  At a spending level equal to the overall FBS average, the model predicts the Non-AQ school has a 14.4% chance at a major bowl, versus just 5% for the AQ school. When a non-AQ spends 150% (think TCU) of the average the probability of a major bowl is about 27%.  For the AQ school this level of spending yields a probability of just 12%.

While the finding that Non AQ schools actually have preferred access to the BCS games is at first surprising, upon reflection it is a very intuitive result.  Consider the case of TCU.  Prior to moving the Big Twelve, TCU was the biggest spending non-AQ program and played in bowl games every year from 2005 to 2012.  This run included Rose and Fiesta Bowl appearances.  In the Horned Frogs’ first season in the Big Twelve they finished with a 7-6 overall record and a 4-5 conference record.  This change from being competitive for BCS bowls to being a middle of the pack team is interesting because TCU’s spending on its program was very high in comparison to Non-AQ schools but only moderate for AQ conference schools.  However, we should note that TCU had a number of issues in the 2012 season and that the Horned Frogs are one of the Big twelve favorites this year.  The Utah Utes experience in the PAC-12 has been similar.  In their final season in the Mountain West, Utah was 7-1 in conference and 10-3 overall.  Since moving to the PAC 12, Utah has had conference records of 4-5 and 3-6.

As we near the 2013-14 season if we had to bet we would guess that it is more likely that we see Boise State in a major bowl than Florida.  Boise State’s toughest opponents include Washington and Fresno State.  In contrast, Florida’s schedule features Georgia, South Carolina, LSU, Miami and Florida State; not to mention a possible SEC championship game against Alabama.  Northern Illinois is another likely non-AQ candidate for a major bowl.

Earlier this month we did a post about conference realignment that focused on how the Big East crack-up and various conference shifts by other schools impacted the quality of each basketball conference. The preceding analysis of the BCS system highlights another aspect of realignment: the consequences for fans versus the incentives of athletic programs.  The cases of TCU and Utah provide examples of non-AQ schools trading off wins for the financial rewards of joining an AQ conference.  The University of Missouri shows how finances can even work across the big 6 conferences.  Missouri football was a competitive Big Twelve program winning 10 or more games 3 times from 2007 to 2011 (and 8 wins the other two years).  In their first year in the SEC, the Tigers went 5-7 and ended their 7 year streak of playing in bowl games.  While the SEC does have a richer set of contracts in place, it also seems likely that Missouri will struggle to be competitive.  The question for fans and for athletic departments is the tradeoff between winning and revenue.  At this stage it doesn’t appear (with the possible exception of Boise State) that winning is viewed as anywhere near as important as money.


2011-2013 NFL Draft Performance by the Non-BCS Conferences FBS: Nevada, Boise State, and Idaho Excel, Notre Dame Disappoints

We have spent the last few days examining the performance of BCS Conferences schools in the 2013 NFL Draft with respect to converting high school talent into NFL draft picks (SEC, Big 10, ACC, PAC 12, Big 12, & Big East).   In this study, we consider the talent conversion ability of Non-BCS Conferences schools over the last three NFL drafts.  We find that the University of Nevada did the best job of converting high school talent into draft picks.  It should be noted that Notre Dame finished near the bottom of the list of Non-BCS schools.  While the Fighting Irish produced only one more pick than Boise State and two more than Nevada, their recruiting classes were better by leaps and bounds.

The FCS schools are excluded from this study because there is very limited recruiting data available.  However, Appalachian State produced six draft picks in the 2011-2013 NFL drafts!  It is not surprising that Appalachian State is moving to the FBS.

(*ARP refers to the average recruiting points as given by Rivals.com for recruiting classes represented in the 2011-2013 NFL Drafts)









Mike Lewis & Manish Tripathi, Emory University 2013.