Debunking a Debunking: Our Response to the NetsDaily

If the true purpose of the Emory Sports Marketing Analytics blog is to generate conversation, then our NBA brand equity study has been a success.  The reality is that we are a couple of sports fans that happen to be university professors.  I (Mike) spend most of my time developing dynamic models that predict customer behavior over time.  This blog is an effort to combine our jobs and hobbies.  An example of this is a paper on competitive balance in MLB.

Our blog is intended to present more quick hitting analyses of current issues in the world of sports.  With these studies we rely on publicly available data, and we tend to use relatively straight-forward statistical methods.  We really don’t begin with any agenda; we just let the numbers speak.

In the case of the NBA fan study, the numbers spoke and then the NetsDaily answered.  Given our love of debate, we can’t help but answer back.  Though all kidding aside, to the Nets/NetsDaily guys: we really do like what you guys have done.  You are obviously an exciting franchise with a lot of great marketing.

In what follows we reprint the NetsDaily notes in blue and then give our comments in bold:

Final Note: That Emory University study

We’re not going to devote a whole lot of effort to debunking the silly Emory College attempt at defining passionate fans by analytic means.  It found that the Nets were the worst home fans in the NBA and the Knicks the best.

Okay, we get the idea, the NetsDaily is not happy with the results.  A couple of things come to mind.  One, Emory is a University.  Second, debunking is probably the wrong way to start a discussion.  With almost any study like ours, there will need to be assumptions made.  As we noted in our original post,we are using a revenue premium brand equity model.  One could definitely argue that passion and revenue are different concepts.  It is probably more useful to understand what the study is saying than to claim it is wrong.

We would just like to point out some serious flaws in the study. The original study was so devoid of data that the authors were asked to provide some, which they did in a self-congratulatory addendum.

We are trying to strike a balance in the blog.  We could report all statistical models, but we are trying to keep things interesting so we emphasize intuition and report the interesting findings.  We are more than happy to share additional details.  And as academics, self-congratulation is probably our best hope for some positive feedback.

The authors note that key data they used to derive their conclusions is something called “home revenue.” They attempt to estimate “home revenue,” which is a finite, known but proprietary figure not available to them. Shouldn’t they note that, explain its relevance?

“The analysis begins with a model of box office revenue based on variables that correspond to market potential (capacity and market population), team quality (winning percentage) and entertainment value (number of all stars, payroll). The insight or theory that drives the analysis is that this model can be used to predict the revenue that is due to quality and market potential. Any difference between this predicted value and actual value is due to ‘fan loyalty’.”

So the reality is that they don’t have the finite, known but proprietary information that is the core of the theory so project it based on other data, including things as spurious as number of all-Stars, but ignore other data that might be important, like say MERCHANDISE SALES. Need we go there? The Nets now rank fourth in NBA merchandise sales. In the first several months after the merchandise was introduced, they ranked first.

The NetsDaily does not seem to properly understand the analysis.  We use a really simple estimate of home box office revenue using the popular Fan Cost Index and attendance reported by ESPN.  Is this the ideal way to determine home revenue?  Of course not!  Is it a reasonable way given that teams are private and do not report detailed financials?  Reasonable might even be too strong.  In fact, let’s say that it is a crude way to compute box office revenues. (Point for the Nets)

We then use this revenue measure as a dependent variable in a regression model that uses the previously mentioned factors.  We are not estimating revenue in terms of things like number of all-stars, we are developing a model that explains revenues by these factors that indicate team quality, market size and entertainment value (fans come out to see all-stars).  We then compare the difference between the predicted and the (crude) estimate of actual revenue.

The idea is to look at attendance AFTER controlling for how well the team did.  Does Miami selling out mean much given the quality of the team on the court?  Our goal is to really get at the true core support for a team.

On comparing the attendance between the Knicks and Nets, they use gross numbers of attendance.

“The teams share the largest population metropolitan areas but the Knicks achieve a 10.7% advantage in terms of attendance DESPITE charging much greater prices. It is this greater pricing power that pushes the two teams to opposite ends of the ranking.”

Suppose instead of gross numbers, they used capacity percentage. The original analysis appears to rely on ESPN attendance percentages, that is, the percentage of arena capacity sold out on average each game. We say “appears” because the original article notes, ” A quick look at attendance data from ESPN shows that the Trail Blazers regularly exceed capacity for entire seasons.” Two points: the Trail Blazers attendance last season was 95.4 percent, which did not exceed capacity (or it would have been in excess of 100 percent.)

This is an example of why it’s probably better to ask questions and have a discussion than to go on the attack.  We control for stadium capacity in the revenue prediction model to account for differences in stadium capacity.

But they do make a good point here.  When we talk about “best” fans there really is no obvious metric.  We choose revenue, the Nets suggest capacity utilization.  Also, the Trail Blazers attendance percentage was 102.6 percent of capacity in 2012, 102.7 percent in 2011, and 102.6 percent in 2010.  We believe that this is very consistent with our wording.

However, using ESPN data presents problems. It is inaccurate regarding the Nets. ESPN uses an NBA capacity of 18,000 for Barclays Center. That was the original number for NBA games. As the arena was completed, capacity was reduced to 17,732 (apparently to accommodate loge seating added late in construction.) The number can be found on the Nets website. So the actual percentage of seats sold this season is 96.9 percent, not 94.9. That would put the Nets at tenth (not 16th) in the NBA, just ahead of … drum roll … the Knicks at 96.3 percent and the Blazers.

Again, some good points are raised here by the NetsDaily.  As statisticians, we love to have very accurate data.  In reality, data almost always contains some noise or error.  The Nets would have a valid complaint if the publicly available data was somehow consistently biased against the Nets.  Does Team Marketing Report systematically underestimate the Nets’ prices, and does ESPN systematically underestimate the Nets’ attendance?  We don’t know.  If so, we apologize.

We leave it to the readers to decide whether our measure which includes quantity and prices is preferred to straight capacity utilization.

But let’s put aside the methodology and data and look at the final product, which suggests below average work.  Is there ANYONE in the NBA who believes that fan loyalty to the Dallas Mavericks, Phoenix Suns, and Orlando Magic is on the rise … or that their fans had greater loyalty than the teams that follow them in the Emory rankings: the Miami Heat and San Antonio Spurs??? You want an example of analytics gone wild???

The comment above seems to be a common misinterpretation.  The example of the Orlando Magic is really what is at the core of our study.  We are not saying that the Orlando Magic has more fan support than the Miami Heat this season.  We are saying that after you control for the difference in the quality of the teams it appears that the Orlando Magic have a more devoted fan base.  Over the past season, the Magic had an average home attendance of 17,595, while only winning 24% of their games.  The key question (and what we use the statistical models to get at) is what would Miami have drawn if they didn’t win 80% of their games and have 3 all-stars in the lineup?

If we were petty people we would also point out that the ESPN attendance figures for last season report that the Magic drew 721,414 fans while the Nets operating in the largest metropolitan area and winning about 60% of their games drew 704,702.  We usually aren’t petty, but they did call our study “silly”, and called us “C students” (Manish says thanks for the passing grade) on Twitter.

The study is also a rare, rare instance where Barry Baum, chief communications officer of the Nets, and Norman Oder, the leading critic and chronicler of Atlantic Yards find common ground.

Says Baum, after reading the original article, “With all due respect to Emory University, that is a seriously flawed study.”

Thank you Mr. Baum for getting the school name correct.

Says Oder, after reviewing the article and supporting data, “The Knicks’ attendance edge is magnified by an arena with greater capacity, and the willingness of Knicks fans to pay more. [It] has less to do with passion than a longstanding monopoly position in a large market.”

This is also a good point, and one that we acknowledged.  It seems likely that the Knicks may benefit from locational advantages that translate into greater pricing power.

If there is a continuing dispute on this, we suggest a review by Nate Silver, the New York Times stats guru … and Nets fan. We are sure he will get to the bottom of it.

We would of course very happy to extend the discussion.  We were in fact surprised when we ran the numbers, and found the Nets on the bottom.  And remember we did point out that the Nets were rapidly improving.

Mike Lewis and Manish Tripathi, Emory University 2013.

Follow Up to the NBA Fan Equity Study & Why We Do This

Our post about which NBA teams have the best fan bases has generated a good deal of response.  This response has included insightful questions about the models and variables.  We wanted to use this post to provide some more detail and examples.

Before we get into the NBA study, it is probably useful to give folks a bit more background on what we are trying to accomplish.  Our unofficial mission is to use marketing concepts and statistical methods to understand the behavior of players, teams, and leagues.  We are both sports fans and academics, so our goal is to go beyond opinion, and use data to generate new insights into the world of sports.

When we start an analysis like the NBA fan equity study we really don’t start with an agenda (though Professor Lewis does acknowledge a personal bias against Duke Basketball).  We start with a bunch of data and some concepts (theory) that guide the way we approach the analysis.  In the case of the fan equity study, our guiding theory is that team revenue is based on the loyalty of fans, the size of the team’s market, the quality of the product and the entertainment value of the team.

The analysis begins with a model of box office revenue based on variables that correspond to market potential (capacity and market population), team quality (winning percentage) and entertainment value (number of all stars, payroll).  The insight or theory that drives the analysis is that this model can be used to predict the revenue that is due to quality and market potential.  Any difference between this predicted value and actual value is due to “fan loyalty.”

In their responses to us, readers tended to ask about a few specific teams.  For instance, there was a great deal of interest in comparisons between the Knicks and the Nets.  The table below shows several differences between the two teams that are drivers of the differential fan equity.  The teams share the largest population metropolitan areas but the Knicks achieve a 10.7% advantage in terms of attendance DESPITE charging much greater prices.  It is this greater pricing power that pushes the two teams to opposite ends of the ranking.

Another illuminating comparison could be made between Orlando and Golden State.  Given the excitement surrounding the Warriors the casual fan would likely assume that Golden State enjoyed a stronger fan base.  However, when we look at the numbers we see that Orlando generates almost the same attendance (and charges slightly higher prices) while operating in a market that is half the size and winning fewer than half as many games.  Our analytics driven approach accounts for these differences.  It also makes sense when you consider that Orlando fans provide about the same amount of box office revenue as GSW fans, while the team draws from a smaller market and wins only 24% of their games.

This last point is really the key to our analysis.  As a further example, while Miami is currently a great revenue driving team, we need to realize that their fans are attracted to a team that won 80% of their regular season games and has three all- stars and the likely MVP in the lineup.  The true test of fan loyalty is what happens when a team slumps.  This is why teams like Orlando and Dallas do so well in our rankings.

Mike Lewis & Manish Tripathi, Emory 2013.

Does Seattle deserve an NBA franchise? Yes!

Yesterday, the NBA owners voted to keep the Sacramento Kings from being sold to a group that would have brought the franchise to Seattle.  Yesterday, we also posted an analysis of fan base quality for the NBA.  We reported results for each Team’s “Fan equity” for the past season (and the change from 2012).  We found that the franchises with the best fan bases included the Knicks, Bulls, Celtics and Trail Blazers.  We also found that the worst fan bases were those of the Nets, Hawks and Pistons.

While we focused on this most recent season, our methodology can also be applied to previous seasons.  This morning we have run the numbers for seasons from 2002 to 2006 to see how the Sonics fan base compared to other cities (we excluded the last two years to avoid having the numbers biased by a negative fan reaction due to the planned move to Oklahoma City).

We found that the Seattle market ranked 20th over this time period.  While this is not a great performance it does place the Sonics far from the bottom.  The Sonics fan base was more supportive than fan bases in Denver, Minnesota, Milwaukee, Atlanta, Detroit, Philadelphia and the Clipper side of LA.

The conclusion seems to be that while Seattle isn’t a truly great NBA market it is at least a decent one.  And one that does have something of a history.  Now that the opportunity to grab the Kings is gone, the next question should be what about expansion.  This recent article in Forbes describes an analysis of markets such as Louisville and Seattle as candidates for expansion.  While we don’t have access to the underlying data, we would like to point out that Louisville already has a very strong basketball market.

Mike Lewis & Manish Tripathi, Emory University 2013


Which NBA Team has the Best Home Fans? And Who has the Worst? Hint: It’s New York!

Note: We have received a lot of responses to this study.  For more about the specifics of the study and answers to common questions, click here.

One of the core concepts we work with at Emory Sports Marketing Analytics is brand equity.  Brand equity is basically the advantage that a firm has over its competitors due to their brand being better known and having more loyal followers.  In the realm of sports, brand equity can be thought of as capturing the size and intensity of a team’s fan base.  As the NBA playoffs proceed to their climax this year, we decided to examine the brand equity of all 30 NBA franchises (for a similar analysis of NCAA basketball click here).

A quick Google search shows several other rankings of fan base quality (links below).  These rankings are largely based on consumer surveys or opinion.  In contrast, our method uses statistical models of team revenue results to measure which fan base best votes with their wallets.  Basically, what we do is estimate a statistical model of team box office revenues as a function of the team’s winning percentage, team payroll, market population, arena capacity, number of all-stars, and other factors that capture the quality of the team’s product and revenue potential in a given year.

Home Revenue = f(win%, Payroll, Market Population, etc…)

We then compare team’s actual home revenue with predictions from our model to discern teams that out- or under-perform.*  We call this quantity “Fan Equity.”

Fan Equity = Reported Home Revenue – Predicted Home Revenue

For the 2013 regular season, we find that the New York Knicks have the top ranked fan base.  The Knicks are followed by Chicago, Boston, Portland and Dallas.  Of these, Portland is probably the most interesting case.  A quick look at attendance data from ESPN shows that the Trail Blazers regularly exceed capacity for entire seasons.  Portland is a small market, but a market with passionate and supportive fans.  Portland also likely does well because they are the only “pro” game in town.

At the other extreme, we find that the Brooklyn Nets, the Atlanta Hawks and the Detroit Pistons are the greatest underperformers.  To reiterate, our method basically suggests that these teams should be making more revenues based on their markets and on-court performance.  The Brooklyn Nets are a fascinating example, given the hype that surrounded the move to Brooklyn, and Jay-Zs “ownership.”  While the Nets finished dead last in our rankings, if we look at year over year changes we do see signs of life.  In terms of year-to-year changes, the Nets had the 5th greatest improvement from 2012 to 2013 (even though their overall ranking did not change).

On a local level, we find that the Atlanta Hawks have very little fan support.  This comes as little surprise to folks from the South (aka SEC territory).  Atlanta as a city has the reputation of a place where everyone is from somewhere else.  This is probably a critical factor as a great deal of fan loyalty is built as fans grow up watching the home team.

*As with any analysis of this type, it is possible to quibble with assumptions.  For example, our method does not consider television revenues or that some cities have a greater corporate presence.

Links to other rankings of fan base quality: on Teams with the Best Crowds (Golden State #1)

Forbes Study of Loyal Fans (Miami #1)

Bleacher Report on Best Fan Bases (Chicago #1)

Mike Lewis & Manish Tripathi, Emory University 2013.