Fanalytics Podcast: Sports Sponsorships

Maybe the thing that sets sports apart from other industries is the incredible passion, loyalty and, in fact, fanaticism, shown by customers.  Sports fans proudly wear logos and describe themselves as brand loyalists.  The end result of this level of passion is that sports brands are incredibly powerful marketing assets.  In addition to figuring out the direct value of sports brands (how they influence customers to attend and spend), another important question for “Marketing Analytics” is the power of these brands to change behavior in other categories.  In other words, how can sports brands be used in sponsorship deals.

On the current episode of the Fanalytics Podcast episode, we have Nick Mentel of Vantedge and a discussion of sports sponsorships.  Nick provides insights into how these deals get done.

The discussion is wide ranging but we find ourselves focusing on the idea of using “comparables.”  This is a very common topic in the real world. It’s also an important concept for the analytics community and I think there are opportunities to add some analytics horsepower to enhance “comparables” based methods.

Nick Mentel is the Vice President of Sponsorship Insights at Vantedge, helping lead the company serve the needs of an array of Fortune 500 clients.  His marketing insights and analytics experience ranges from sponsorship valuation and targeting to social and digital metrics reporting and analysis.  Nick has experience delivering customized client solutions to clients in wide-ranging industries, having previously worked for Lehman Brothers and Deloitte Consulting.

At Vantedge, he has had an opportunity to leverage his quantitative skills learned from being a diehard sports fan to help maximize the earning potential of CSE Talent’s clientele.  Nick holds bachelor degrees in Business Administration and Spanish from the University of North Carolina and an MBA from the University of Pennsylvania’s Wharton School.  He also holds a Limited Certification as a Player Agent on behalf of Major League Baseball Players Association.

You can learn more about Vantedge by clicking here:


Click on the logo to listen to the podcast episode.

Fanalytics Podcast: The Analytics of Paying NFL Running Backs

The Le’Veon Bell contract situation was one of the more interesting NFL stories of the past month. In today’s Fanalytics podcast, economist Tom Smith and I talk about the story from a statistical and economics perspective.

NFL running backs are an incredibly interesting position for analytics and salary cap management.  The dilemma for teams and the frustration for players comes from the nature of the position.  Running backs are often at their peak during the early career years when the player’s salary is most constrained by the leagues collective bargaining agreement.

In the case of Le’Veon Bell, he is entering the region where past carries (and touches) combined with age start to build some uncertainty about future performance.  This future uncertainty combined with the fact that running backs are a relatively inexpensive position create an interesting situation for the Steelers.  Bell may be the best running back in the league but can they “replace” a significant amount of his production at a much lower cost?

This conversation was wide ranging and had some technical elements.  Probably our most technical episode yet as terms like “hedonic pricing model” and “constrained dynamic optimization” were thrown around.  That said – it was a great conversation.  The economist (Tom) combined with the Operations Researcher (Mike) offers a unique perspective of analytics and decision making.

Hope you all enjoy the podcast and please rate and subscribe on iTunes.


Click icon below to listen:

Fanalytics Podcast: Wrestling Fandom

One of my favorites category of sports is “combat sports.”  Combat sports have a few characteristics that make them different than traditional team sports.  Sports like MMA, Boxing and (even) professional wrestling are sold as pay-per-view events, sold as cards or bundles of events, and are usually very “star” focused.  One of the long-term plans for the podcast is to bring in people with a variety of experiences related to the combat sports industry.

In this episode we start with the fan perspective.  Really, I should call this, “the sophisticated fan” perspective.  Today’s guest is a former MBA student and current consultant named Hari Gopal.  Hari brings both business acumen and incredible passion for professional wrestling.  Our plan for the podcast was to have a wide ranging discussion of the fan side of the industry.  Our goal is to understand how fan interest is driven by “stories” and “stars”.  This “understanding” becomes the foundation for thinking about marketing and operating combat sport businesses.

To listen to the podcast episode, click the Fanalytics icon below.


NFL Fan and Brand Report 2018

Each year, I do an analysis of NFL fandom.  The analysis is grounded in economic and marketing theory, and uses statistical tools to shed light on the question of which teams have the most loyal or “best” fans.  The key point of differentiation is that this is a truly quantitative analysis.  It’s driven by data, not by emotion.

On a side note, I also regularly podcast on sports and sports analytics topics.  You can find the accompanying episode (and all sorts of other cool stuff) via the link below.

The fundamental question that guides the analysis is simple – Who has the best fans in the NFL?  For the business folks maybe we say this as – What are the best brands in the NFL? These are simple questions without simple answers.  First, we have to decide what we mean by “best”.  What makes for a great fan or brand?  Fans that show up even when the team is losing?  Fans that are willing to pay the highest prices?  Fans that are willing to follow a team on the road or social media?

Even after we agree on the question, answering it is also a challenge.  How do we adjust for the fact that one team might have gone on a miraculous run that filled the stadium?  Or perhaps another team suffered a slew of injuries?  How do we compare fan behavior in a market like New York with fans in a place like Green Bay?

My approach to evaluating fan bases is to use data to develop statistical models of fan interest (more details here).  The key is that these models are used to determine which city’s fans are more willing to spend or follow their teams after controlling for factors like market size and short-term changes in winning and losing.

I use three measures of fan engagement: Fan Equity, Social Equity and Road Equity.  Fan Equity focuses on home box office revenues (support via opening the wallet). Social Media Equity focuses on fan willingness to engage as part of a team’s online community (support exhibited by joining social media communities).  Road Equity focuses on how teams draw on the road after adjusting for team performance.   These metrics provide a balance – a measure of willingness to spend, a measure unconstrained by stadium size and a measure of national appeal.

To get at an overall ranking, I’m going to use the simplest possible method.  A simple average across the three metrics.  (similar analyses are available for the NBA and MLB).  The rankings are based on multiple years of data, use multiple performance measures and sophisticated statistical techniques.  But nothing is perfect and I’d be remiss if I didn’t discuss some of the issues and controversies surrounding the NFL.

Its an understatement, but it’s been a tumultuous last few years for the NFL.  Concussions, anthem protests and domestic abuse scandals have all “complicated” fan’s relationships with teams.  The analytics I present use historical data to provide insight into recent of fan interest.  The analytics provide a measurement of the “fandom” that has been built over decades.  So, what is the impact of the current controversies?  Its impossible to say.

The problem is that while we can measure current fandom with snapshots of spending and social media behavior, the impact of incidents or events such as the anthem protests or the concussion lawsuits may play out over years or decades.  These types of issues might have an immediate impact on some metrics but the salient question is how will they influence long-term preference levels.

There may be “signals” in the data such as changes in TV ratings or higher no-show rates, but it’s tough to tell if these are blips or trends.  In terms of the observed decline in TV ratings, there is no shortage of theories – the aforementioned controversies, key player retirements, the 2016 presidential election, too many games, and just too many entertainment options have all been mentioned as root causes.  The existence of so many theories means that an analytics based approach is going to be difficult if not impossible.  This is especially true because while fandom can dissipate faster than its built, fan loyalty and passion is more likely to fade over years rather than disappear over weeks.

My conjecture is that the concussion issue and the anthem protests are both very significant problems for teams and the NFL brand.  The issues related to concussions may lead to lawsuits and decreased youth participation.  The anthem protests are something about which I’m reluctant to write (given the unfortunate state of the modern university).  But to keep it simple – the anthem protests have inserted some ugly “politics” into what is fundamentally an entertainment category.  If the product becomes less fun, why would you expect fans to enjoy it as much?  And while the phrase “less fun” might seem to trivialize the issues, spending on sports entertainment is about as discretionary as it gets.

Nevertheless, while the NFL has challenges, it is still the preeminent US sports league.  How the league fares in the future is probably going to be based on the strength of its strongest brands.  Which brings us back to our fundamental question – What are the best brands or fan bases in the NFL?


The Winners

The top five fan bases (team brands if you prefer) are the Cowboys, Patriots, Eagles, Giants and Steelers.  This is unchanged from last year.  The first switch in the rankings is the number 6 and 7 positions with the Bears moving ahead of the Saints.

The Cowboys excel on all the metrics.  They win in terms of Fan Equity (a revenue premium measure of brand strength), Road Equity and finish second in social media.  The underlying data (I will spare everybody the statistical models) reveals why Dallas does so well.  The Cowboy’s average home attendance usually leads the league, fans are willing to pay high prices, and the team’s twitter following is exceptional.  The Cowboys are America’s team.

The similarity across rankings gives me faith in the results.  However, the fan in me still questions some of what I see.  In terms of full disclosure, I grew up a Steelers fan in the 1970s and lived in the Chicago during the Bear’s glory days.  As such, I bring my personal biases to the interpretation of the findings.  I can’t help but to think of the Patriots as having bandwagon fans, and the Eagles ranking above the Steelers just does not seem right.

The analyst in me understands that the value of using a statistical approach is that the data can help correct my biases.  A couple of comments.  Patriot fans may be bandwagon fans.  But they have been on the bandwagon a long time.  A couple of decades of success likely means that the Patriots will remain NFL royalty even after Tom Brady leaves the game.

The Eagles surprise me, and probably most of western Pennsylvania.  They do get a bump from playing in the NFC East in terms of the Road Equity metric.  The NFC East is a strong collection of brands that benefit each other.  The Giants also benefit.  It is not easy to disentangle these effects.  And perhaps we shouldn’t since we can make a case that the rivalries that benefit these teams are because of the interest in the individual team brands.


The Losers

At the bottom of the rankings, we have the Browns, Jaguars, Chiefs, Rams and Titans.  This is an interesting group.  We have the struggling Browns, but we also have some teams like the Titans, Jaguars and Chiefs that have had recent success.

The important fact is that the statistical model I use, evaluates each team’s results based on how the league works on average.  If a team wins but does not convert the wins to increased revenues or social following, then the team will suffer in the rankings.

The good news for these teams (Jags, Chiefs, Titans) is that on-field success is the best way to create brand equity and fan loyalty.  The bad news is that it takes a good amount of success to move the needle long-term.

For the Rams and the Chargers, we should probably include an asterisk. Moving markets and playing in temporary stadiums can lead to some questionable findings.


The List

The complete list follows.  In addition to the overall ranking of fan bases, I also report rankings on the fan equity, social equity and road equity measures.  Following the table, I provide a bit more detail regarding each of the metrics.

2018 NFL Brand Rankings


Further Explanations

Fan Equity

Winners: Cowboys, 49ers, Patriots

Losers: Rams*, Raiders, Jaguars

Fan Equity looks at home revenues relative to expected revenue based on team performance and market characteristics.  The goal of the metric is to measure over or under performance relative to other teams in the league.  In other words, statistical models are used to create an apples-to-apples type comparison to avoid distortions due to long-term differences in market size or short-term differences in winning rates.

Just like last year, the 49ers are the interesting winner on this metric. After the last couple of years, it is doubtful that people are thinking about the 49ers having a rabid fan base.  However, the 49ers are a prime example of how the approach works.  On the field, the 49ers have not performed well.  Despite the on-field struggles, the 49ers still pack in the fans and charge high prices.  This is evidence of a very strong brand because even while losing the 49ers fans still attend and spend.  In terms of the overall rankings the 49ers don’t do all that great because the team does not perform as well as a road or social media draw.

In terms of business concepts, the “Fan Equity” measure is similar to a “revenue premium” measure of brand equity.  It captures the differentials in fan’s willingness to financially support teams of similar quality.  From a business or marketing perspective this is a gold standard of metrics as it directly relates to how a strong brand translates to revenues and profits.

One important thing to note is that some teams may not be trying to maximize revenues.  Perhaps the team is trying to build a fan base by keeping prices low.  Or a team may price on the low side based on some notion of loyalty to its community.   In these cases, the Fan equity metric may understate the engagement of fans.  I suspect that this is the case for the Steelers.


Social Media Equity

Winners: Patriots, Cowboys, Steelers

Losers: Rams, Jaguars, Titans

Social Media Equity is also an example of a “premium” based measure of brand equity.  It differs from the Fan Equity in that it focuses on how many fans a team has online rather than fans’ willingness to pay higher prices.  Similar to Fan Equity, Social Media Equity is also constructed using statistical models that control for performance and market differences.

In terms of business application, the social media metric has several implications both on its own merits and in conjunction with the Fan Equity measure.  For example, the lack of local constraints, means that the Social Equity measure is more of a national level measure.  While the Fan Equity metric focuses on local box office revenues, the social metric provides insight into how a team’s fandom extends beyond a metro area.

Social Media Equity may also serve as a leading indicator of a team’s future fortunes.  For a team to grow revenues it is often necessary to implement controversial price increases.  Convincing fans to sign expensive contracts to buy season tickets can also be a challenge.  Increasing prices and acquiring season ticket holders can therefore take time, while social media communities can grow quickly.  Preliminary analysis suggests that vibrant social communities are positively correlated with future revenue growth.

A comparison of Fan Equity and Social Media can also be useful.  If Social Media equity exceeds Fan Equity it is evidence that the team has some marketing potential that is not being exploited.  For example, one issue that is common in sports is that it is difficult to estimate the price elasticity of demand because demand is often highest for the best teams and best seats.  The unconstrained nature of social media can provide an important data point for assessing whether a team has additional pricing flexibility.


Road Equity

Winners: The NFC East, Raiders, Patriots and Steelers

Losers: Texans, Titans and Browns

Another way to look at fan quality is how a team draws on the Road.  There was a famous case in Atlanta just a few years ago, when Steelers fans turned the Georgia dome Gold and Black.

The Road Equity measure can be interpreted in multiple ways.  If a team has great road attendance, is it because the fans are following the team or because they have a national following?  In other words, do the local fans travel or does a team with high road attendance have a national following. When the Steelers turned the Georgia Dome Gold and Black was it because Steelers fans came down from Pittsburgh or because Steelers fans are everywhere.

I suspect that we are capturing a measure of national following rather than a tendency to travel.  The Road Equity rankings are dominated by high profile teams such as the Cowboys, Patriots and Steelers.  These teams also do very well on the Social Equity measure (which also measures national following).  This correlation gives me a confidence that the Road Equity picks up a measure of national following.





How Mad is Too Mad?: Cinderellas, Blue Bloods and TV Ratings

Each Spring I teach a course on sports marketing analytics.  As part of this course I ask student to develop a research project focused on either a marketing or player analytics topic.  What follows is a project that looks at the relationship between upsets and TV ratings in the NCAA tournament.  This project is interesting in several respects.  It has an interesting foundation in consumer behavior theory as it is motivated by an open question of whether fans prefer a tournament dominated by Cinderellas (upsets) or Blue Bloods (high brand equity teams).  This underlying theory then drives the data collection and modeling efforts.  Finally, the results speak to what fans actually prefer.

I think this project was interesting as it could be the starting point for deeper analyses.  Additional data could be collected and we could develop different models.  This is a great lesson because this is the case with almost all analytics projects.  We also did a podcast episode where we talked through the analysis and possible extensions.


How Mad is too Mad?

by Katie Hoppenjans

“March Madness” is a fitting nickname for the NCAA Division I Men’s Basketball Tournament. Since its inception in 1939, the tournament has been characterized by Cinderella stories; in years that are particularly “mad” with upsets, the underdog winners seem to be on everyone’s mind. In a year of historic upsets like this one, however, I have to wonder whether the level of excitement in a tournament really has any impact on fans’ engagement. Do people really love an underdog, or would they prefer to watch the same old powerhouses? Is “madness” really what viewers want?

To examine the value of an “exciting” March Madness, I built a model examining the relationship between the number of upsets in a tournament and the number of viewers who watch the championship game. The model includes data from 2005 to 2017, and upsets are defined as games won by teams seeded 11 or lower. Since no team seeded 11 or lower has ever made it farther than the Final Four, only the first four rounds of competition were counted. Finally, since significant upsets in later rounds are arguably more unexpected than those in early rounds, I assigned more points to the later rounds in my analysis; 1 point was given for each upset in the Round of 64, 6 were given for the Round of 32, 12 were given for the Sweet Sixteen, and 24 were given for the Elite Eight.

Using this model, I found that there is actually a significant negative correlation between the number of upsets in the early rounds of a tournament and the number of viewers who watch the championship game. In other words, the more “exciting” the tournament, the fewer the viewers who stick around until the end. One possible explanation for this may be that many people only watch March Madness because they have filled out a bracket; if their bracket is “busted” by early upsets, they might tune out of the tournament entirely. It may also be true that historically strong teams (like Kentucky, Indiana, Kansas, etc.) have more fans than small, Cinderella-story schools. Since powerhouse teams win more often, their fans are also more likely to be engaged and loyal viewers than smaller teams’ fans. As a result, when a major team is taken out of the tournament by a smaller school, viewership may drop off as the larger school’s fans lose interest. This is only conjecture, and further analysis would be needed to determine the cause of the relationship between viewers and upsets. However, as demonstrated by the graph below, upsets certainly seem to have an impact on how many people watch the championship game.

As advertising spend for the March Madness championship game continues to climb (per the graph below), the continued volatility in viewership must be troubling to sponsors. Particularly in a year like this one, in which a 1-seed lost in the Round of 64 for the first time in history, things are not looking good for the championship game ratings; with a tournament this unpredictable, it may be more important than ever for advertisers to find reliable ways of predicting the impact of their championship game sponsorship. Early-round upsets may be one factor in determining viewership, but there are many more questions that need to be answered before championship game ratings can be accurately estimated. “Madness” in the NCAA tournament may sound exciting, but if its negative correlation with viewership is to be believed, it’s actually bad news for fans and sponsors alike.


“NCAA Men’s Final Four Ratings Hub.” Sports Media Watch,

“NCAA Records Books.” – The Official Site of the NCAA, 17 Jan. 2018,

MLB Fan Marketing Report 2018

As we enter the 2018 season, it’s time to take a look at MLB from a marketing perspective.  Specifically, the goal today is to evaluate MLB teams in terms of fan loyalty and engagement.  Who has the best fans in Major League Baseball?  What are the best brands in MLB? These are simple questions without simple answers.  What makes for a great fan or brand?  Fans that show up even when the team is losing?  Fans that are willing to pay the most?  Fans that are willing to follow a team on the road or via social media?

Even after we agree on the question(s), answering it is also a challenge.  How do we adjust for the fact that one team might have gone on a miraculous run that filled the stadium?  Or perhaps another team suffered a slew of injuries?   An analysis of fandom should account for short-term variations in on-field performance.  There is also the matter of differences across markets.  How do we compare fan behavior in a market like New York with fans in a place like Milwaukee?  What if a team just opened a new stadium?  Did the fans stream in to see the building or to see the team.

For the past few years, I have been studying fandom across professional and college sports.  My approach to evaluating fan bases is to use data to develop statistical models of fan interest (more details here).  The key is that these models are used to determine which cities fans are more willing to spend or follow their teams after controlling for factors like market size and short-term variations in performance.  The goal is to provide as much of an “apples” to “apples” comparison as possible.


The Best Fans?

It is possible to rank fans on many different dimensions. And different dimensions can have different meanings and nuances.  For today – I’m going to develop an “overall” ranking of fans based on three sub-rankings – Fan Equity, Social Equity and Road Equity.  Fan Equity is a revenue premium based metric that compares team’s box office results with league standards.  In other words, Fan Equity assesses how much fans are willing to spend relative to fans across the league.  I think of this metric as about “attend and spend.”  The KEY idea is that we measure “attend and spend” while controlling for team success and market characteristics like income and population.

  • Fan Equity is a great metric for assessing the CURRENT level of passion or engagement in a local fan base.

Social Equity is focused on the team’s social media followings (Facebook and Twitter).  Again, the rankings are based on how a team’s social media results compare across the league after controlling for team success and market.  Social Equity is also attractive in that the metric does not require fans to spend or to live in a local market.

  • The Social Equity metric provides insight into the team’s POTENTIAL fan passion.

The third metric is Road Equity.  This metric is based on a statistical model that looks at how teams draw incremental fans when on the road.  The KEY idea is that draw outside of the home market reveals something about a clubs national appeal. This passion can be positive (love the Cubs) or negative (hate the Yankees).

  • Road Equity provides a metric of passion beyond the local market.  A measure of NATIONAL brand equity.

I could go on.  In the past I have developed additional metrics related to win sensitivity or price sensitivity.  Willingness to attend even when the team loses probably says something about loyalty.  Fans that don’t watch a loser might be termed bandwagon fans.  Willingness to pay is a great marketing metric.  Willingness to pay to see a team that isn’t winning is another great indication of loyalty.  These metrics are available upon request (FYI, I don’t look at the comments so please email) but I want to keep this article brief.

So, we have three metrics with different pluses and minuses.  In the quest to find an overall winner – I’m going to take the simplest approach and average the rankings.  I don’t think this is the ideal approach, but it is simple. Simple is a great default.


The Winners

Overall, the group of clubs that comprise the Top 6 contains little in the way of surprises.  The Red Sox rank number one and are followed by the Yankees Giants, Dodgers, Cubs and Cardinals.  The Red Sox are perennially strong and finished third last year.  I sort of hate to say it, but Boston is probably the best sports town in America.


In general, the clubs at the top of the list share several traits.  They are all able to motivate fans to “attend and spend” as they all possess great attendance numbers and are able to charge relatively high prices.  More to the point, these teams are able to draw well and command price premiums when they are not winning.  The last few years excepted, Cubs are the best example of this.

The list of winners probably raises an issue of “large” market bias.  However, keep in mind that the methodology is designed to control for home market effects.  The method is explicitly designed to control for differences in market demographics (and team performance).  While the “winners” tend to come from the bigger and more lucrative markets, other major market teams do not fair particularly well (White Sox, Mets, A’s).  There is also a more subtle point.  The large market teams likely have the best fan bases because they often have significant histories of success and are often featured in the media. The topic of how these brands are built over time is another of my favorite things to talk about.  But, it’s a topic for another day.


The Laggards

The bottom of the list features the Marlins, Athletics and White Sox.  It is interesting that the bottom of the rankings includes teams from major markets such as the SF Bay Area, Chicago and Miami.  Being in a major market might be a double edged sword.  There are natural advantages in terms of building brand equity but there are also dangers.  Failing to succeed in a “large” market might be the worst possible situation (fan expectations?)

The Marlins finish is a reflection of how the team struggles on multiple dimensions. Attendance is often in the bottom 5 of the league despite being located in a major metro area.  Pricing is also below average for MLB.

From a branding perspective it is not surprising that we see one dominant brand in the cities with two clubs.  Being a sports fan is about being part of a community.  Many fans are drawn to the bigger and more dominant community – Yankees, Cubs, Giants or Dodgers rather than the Mets, White Sox, A’s or Angels.

There is also likely a story about consistency.  I chose an old school logo for the Sox.  I grew up in Chicago and the Cubs were always the same classic stadium and classic uniforms.  The Sox seemed to change things every season – new colors and logos.  I even have faint memories of the team wearing shorts on occasion.

The List

The complete list follows.  In addition to the overall ranking of fan bases, I also report rankings on the Fan Equity, Social Equity and Road Equity measures. Enjoy!

Listen to the full podcast episode here:

Player Analytics Fundamentals: Part 5 – Modeling 102

In Part 4 of the series we started talking about what should be in the analyst’s tool kit.  I advocated for linear regression to be the primary tool.  Linear regression is (relatively) easy to implement and produces equations that are (relatively) easy to understand.  I also made the point that linear regression is best suited for predicting continuous measures and used the example of predicting the number of touchdown passes thrown by a rookie QB.

But not everything we want to predict is going to be a continuous variable.  Since we are talking about predicting quarterback performance, maybe we prefer a metric that is more discrete such as whether a player becomes a starter.  Can we still use linear regression?  Maybe.

Let’s return to the example from last time.  The task was to predict professional (rookie year) success based on college level data.  We assumed that general managers can obtain data on the number of games won as a college player, whether the player graduated (or will graduate) and the player’s height.

Our initial measure of pro success was touchdown passes.  We then specified a regression model using the following equation.

But let’s say that we don’t like the TD passes metric.  Maybe we don’t like it because we think TD passes are more related to wide receiver talent than to the quality of the QB.  Rather than use TDs as our dependent variable we want to use whether a player becomes a starter.  This is also an interesting metric as it captures whether the player was selected by a coaching staff to be the primary quarterback.  This is a nice feature as the metric includes some measure of human expertise.  I’ll leave criticism to the readers as an exercise.

This leads us to the following equation:

One issue we have to address before we estimate this model is how we define the term starter.  In a statistical model we need to convert the word or category of “starter” into a number.  In this case, the easy solution is to treat players that became starters as 1’s and players that did not as 0’s.  As a second exercise – what would we do if we had three categories (did not play, reserve, starter)?

Let’s pretend we estimated the preceding model and obtained the following equation:

We can use the equation to “score” or “rate” our imaginary prospects from last time (Lewis Michaels and Manny Trips).  In terms of the input data, Lewis won 40 college games, graduated and is 5′ 10”.  Plugging Michael’s data into the equation gives us a score of .22.  The analysis that we have performed is commonly termed a linear probability model.  A simple interpretation of this result is that the expected probability of Michaels (or better said a prospect with Michaels statistics) becoming a starter is 22%.

So far so good.

Our second prospect is Manny Trips out of Stanford.  Manny won 10 games, failed to graduate and is 6’ tall.  For Manny the prediction would be -12.8%.  This is the big problem with using linear regression to predict binary (Yes/No) outcomes.  How do we interpret a negative probability?  Or a probability that is greater than 1?

So what do we do next?  I think we have two options.  We can ignore the problem.  If the goal is just to rank prospects then maybe we don’t care very much.  In this case, we just care about the relative scores not the actual prediction.  If we are just using analytics to screen QB prospects or to provide another data point then maybe our model is good enough.  The level of investment in a modeling project should be based on how the model is going to be used.  In many or most sports applications I would lean to simpler less complicated models.

Our second option is to move to a more complicated model.  There are a host of models available for categorical data.  We can use a binary logit or Probit model for the case of a binary system as above.  If the categories have a natural ordering to them (never played, reserve, starter) then we can use an ordered logit.  If there is no order to the categories, then we can use a multinomial logit.  I’m still debating on how much attention I should pay to these models.  Having a tool to deal with categorical variables can be invaluable but there is a cost.  The mathematics become more complex, estimation of the model requires specialized software and interpretation of the model becomes less intuitive.

I think I will discuss the binary logit next time.

Player Analytics Fundamentals: Part 4 – Statistical Models

Today’s post introduces the topic of statistical modeling.  This is, maybe, the trickiest part of the series to write.  The problem is that mastering the technical side of statistical analysis usually takes years of education.  And, more critically, developing the wisdom and intuition to use statistical tools effectively and creatively takes years of practice.  The goal of this segment is to point people in the right direction, more than to provide detailed instruction.  That said – I can adjust if there is a call for more technical material.  (If you want to start from the beginning parts 1, 2 and 3 are a click away.)

Let’s start with a simple point.  The primary tool for every analytics professional (sports or otherwise) should be linear regression.  Linear regression allows the analyst to quantify the relationship between some focal variable of interest (dependent measure or DV) and a set of variables that we think drive that variable (independent variables).  In other words, regression is a tool that can produce an equation that shows how some inputs produce an outcome of interest.  In the case of player analytics, this might be a prediction of future performance based on a player’s past statistics or physical attributes.

To make this more concrete, let’s say we want to do an analysis of rookie quarterback performance (we’ve been talking a bit about QB metrics so far in the series).  Selecting QBs involves significant uncertainty.  The transition from the college game to the pro game requires the QB to be able to deal with more complex offensive systems, more sophisticated defenses and more talented opposing players.  The task of the general manager is to identify prospects that can successfully make the transition.

Data and statistical analysis can potentially play a part in this type of decision.  The starting point would be the idea that observable data on college prospects can help predict rookie year performance.  As a starting point let’s assume that general managers can obtain data on the number of games won as a college player, whether the player graduated (or will graduate) and the player’s height.  (We just might be foreshadowing a famous set of rules for drafting quarterbacks).

The other key decision for a statistical analysis of rookie QB performance versus college career and physical data is a performance metric.  We could use the NFL passer rating formula that we have been discussing.  Or we could use something else.  For example, maybe the number of TD passes thrown as a rookie.  This metric is interesting as it captures something about playing time and ability to create scores.

Touchdowns are  also a metric that “fits” linear regression.  Linear regression is best suited to the analysis of quantitative variables that vary continuously.  The number of touchdowns we observe in data will range from zero to whatever the is the rookie TD record.  In contrast, other metrics such as whether the player becomes a starter or a pro bowler are categorical variables.  There are other techniques that are better for analyzing categorical variables.  (if you are a stats jockey and are objecting to the last couple of statements please see the note below).

The purpose of regression analysis is to create an equation of the following form:

This equation says that TD passes are a function of college wins, graduation and height.  The βs are the weights that are determined by the linear regression analysis.  Specifically, linear regression determines the βs that best fits the data.  This is the important point.  The weights or βs are determined from the data.  To illustrate how the equation works lets imagine that we estimated the regression model and obtained the following equation.

This equation says that we can predict rookie TD passes by plugging in each player’s data related to college wins, graduation and height.  It also says that a history of winning is positively related to TDs and graduation also is a positive.  The coefficient for height is zero.  This indicates that height is not a predictor of rookie TDs (I’m making these number up – height probably matters).  One benefit of developing a model is that we let the data speak.  Our “expert” judgment might be that height matters for quarterbacks.  The regression results can help identify decision biases if the coefficients don’t match the experts predictions.  I am neglecting the issue of significance for now – just to keep the focus on intuition.

Let’s say we have two prospects.  Lewis Michaels out of the University of Illinois who won 40 college games (hypothetical and unrealistic), graduated (in engineering) and is 5’10” (a Flutiesque prospect).  Our second prospect is Manny Trips out of Duke.  Manny won 10 games, failed to graduate and is 6’ tall.  Michaels would seem to be the better prospect based on the available data.  The statistical model allows us to predict how much better.

We make our predictions by simply plugging our player level data into the equation.  We would predict Lewis would throw 10 TDs in his rookie year (1+.1*40+5*1+0*70).  For Manny the prediction would be 2 TDs.  For now, I am just making up the coefficients (βs).  In a later entry I will estimate the model using some data on actual NFL rookie QB performance.

Regression has its shortcomings and many analysts love to object to regression analyses.  But for the most part, linear regression is a solid tool for analyzing patterns in data.  It’s also relatively easy to implement.  We can run regressions in Excel!  We shouldn’t underestimate how important it is to be able to do our analyses in standard tools like Excel.

I will extend our tool kit in a future entry.  I briefly mentioned categorical variables such as whether or not a player is a starter.  For these types of Yes/No (starter or not a starter) there is a tool called logistic regression that should be in our repertoire.

*One reason this note is tricky is that I’m trying to get the right balance and tone.  I can already hear the objections.  Lets save these for now.  For example, readers do not need to alert me to the fact that TDs are censored at zero.  Or that there is a mass point at zero because many rookies don’t play.  Or that TDs are counted in discrete units so maybe a Poisson model is more appropriate.  You get the idea.  There are many ways to object to any statistical model.  The real question isn’t whether a model is perfect.  The real question should be whether the model provides value.

Player Analytics Fundamentals: Part 3 – Metrics, Experts and Models

Last time I introduced the topic of player “metrics.” (If you want to get caught up you can start with Part 1 and Part 2 of the series.)  As I noted, determining the right metric is perhaps the most important task in player analytics.  It’s almost too obvious of a point to make – but the starting point for any analytics project should be deciding what to measure or manage.  It’s a non-trivial task because while the end goal (profit, wins) might be obvious, how this goal relates to an individual player (or strategy) may not be.

However, before I get too deep into metric development, I want to take a small detour and talk briefly about statistical models.  We won’t get to modeling in this entry – the goal is to motivate the need for statistical models!  If we are doing player analytics we need some type of tool kit to move us from mere opinion to fact based arguments.

To illustrate what I mean by “opinion” lets consider the example of rating quarterbacks.  In the previous entry, I presented the Passer Rating Formula used to rate NFL quarterbacks.  As a quick refresher let’s look at this beast one more time.The formula includes completion percentage (accuracy), yards per attempt (magnitude), touchdowns (ultimate success) and interceptions (failures).  Let’s pretend for a second that the formula only contained touchdowns and interceptions (just to make it simple).  The question then becomes how much should we weight touchdowns per attempt relative to interceptions per attempt?  The actual formula is hopelessly complex in some ways – we have fractional weights and statistics in different units – so let’s take a step back from the actual formula.

Imagine we have two experts proposing Passer Rating statistics that are based on touchdowns and interceptions only.  One expert might say that touchdowns per attempt are twice as important as interceptions.  We will label this “expert” created formula as ePR1 for expert 1 Passer rating.  The formula would be:

Maybe this judgment would be accompanied by some logic along the lines of “touchdowns are twice as important because the opposing team doesn’t always score as the result of an interception.”

However, the second expert suggests that the touchdowns and interceptions should be weighted equally.  Maybe the logic of the second expert is that interceptions have both direct negative consequences (loss of possession) and also negative psychological effects (loss of momentum), and should therefore be weighted more heavily.  The formula for expert 2 can be written as:

I suspect that many readers (or a high percentage of a few readers) are objecting to developing metrics using this approach.  The approach probably seems arbitrary.  It is.  I’ve intentionally presented things in a manner that highlights the subjective nature of the process.  I’ve reduced things down to just 2 stats and I’ve chosen very simple weights.  But the reality is that this is the basic process through which novices tend to develop “new” or “advanced” statistics.  In fact, it is still very much a standard practice.  The decision maker or supporting analysts gather multiple pieces of information and then use a system of weights to determine a final “grade” or evaluation.

The question then becomes which formula do we use?  Both formulas include multiple pieces of data and are based on a combination of logic and experience.  I am ignoring (for the moment) a critical element of this topic – the issue of decision biases.  In subsequent entries, I’m going to advocate for an approach that is based on data and statistical models.  Next time, we will start to talk more about statistical tools.

Player Analytics Fundamentals: Part 2 – Performance Metrics

I want to start the series with the topic of “Metric Development.”  I’m going to use the term “metric” but I could have just as easily used words like stats, measures or KPIs.  Metrics are the key to sports and other analytics functions since we need to be sure that we have the right performance standards in place before we try and optimize.  Let me say that one more time – METRIC DEVELOPMENT IS THE KEY.

The history of sports statistics has focused on so called “box score” statistics such as hits, runs or RBIs in baseball.  These simple statistics have utility but also significant limitations.  For example, in baseball a key statistic is batting average.  Batting average is intuitively useful as it shows a player’s ability to get on base and to move other runners forward.  However, batting average is also limited as it neglects the difference between types of hits.  In a batting average calculation, a double or home run is of no greater value than a single.  It also neglects the value of walks.

These short-comings motivated the development of statistics like OBPS (on base plus slugging).  Measures like OBPS that are constructed from multiple statistics are appealing because they begin to capture the multiple contributions made by a player.  On the downside these types of constructed statistics often have an arbitrary nature in terms of how component statistics are weighted.

The complexity of player contributions and the “arbitrary nature” of how simple statistics are weighted is illustrated by the formula for the NFL quarterback ratings.

This equation combines completion percentage (COMP/ATT), yards per attempt (YARDS/ATT), touchdown rate (TD/ATT) and interception rate (INT/ATT) to arrive at a single statistic for a quarterback.  On the plus side the metric includes data related to “accuracy” (completion percentage) to “scale” (yards per), to “conversion” (TDs), and to “failures” (interceptions).  We can debate if this is a sufficiently complete look at QBs (should we include sacks?) but it does cover multiple aspects of passing performance.   However, a common reaction to the formula is a question about where the weights come from.  Why is completion rate multiplied by 5 and touchdown rates multiplied by 20?

Is it a great statistic?  One way to evaluate is via a quick check of the historical record.  Does the historical ranking jive with our intuition?  Here is a link to historical rankings.

Every sport has examples of these kinds of “multi-attribute” constructed statistics.  Basketball has player efficiency metrics that involve weighting a player’s good events (points, rebounds, steals) and negative outcomes (turnovers, fouls, etc…).  The OBPS metric involves an implicit assumption that “on base percentage” and “slugging” are of equal value.

One area I want to explore is how we should construct these types of performance metrics.  This is a discussion that involves some philosophy and some statistics.  We will take this piece by piece and also show a couple of applications along the way.