Last week, I mentioned that my paper with my PhD cohort Chris was accepted for the poster session at the MIT Sloan Sports Analytics Conference. I’ll give the summary and some pictures today (you can find the full paper on my academic website). The project looks at the age old subject of tanking for position in the NBA draft lottery. We answer two questions:
- Should teams tank for a better draft position?
- How much do teams actually tank?
For the first question, we head right to the lottery. We are interested in the causal effect of obtaining the top pick in the draft. If the first pick is truly valuable, then teams should be willing to lose intentionally to get it. LeBron James, Tim Duncan, and Shaquille O’Neal were all first picks, but so were Greg Oden and Michael Olowokandi. We want the average value of all the first picks since the draft lottery took its current form in 1990.
Since there is some randomness in who wins the lottery, it’s tempting to just compare lottery winners and losers. However, the lottery isn’t a simple coin flip. The worst teams have a stronger chance of winning. If we compare winners and losers with no adjustments, winners may come out looking worse than they should, since they were worse to begin with. This will generate estimates of first pick value that are too small.
To eliminate this bias, we use a statistical trick to adjust for team quality. We regress future attendance and winning percentage on a dummy variable equal to one if the lottery team won the first pick. In the regressions, we control for each team’s probability of winning the lottery — this is called their “propensity score” in statistics. Intuitively, this holds the chance of winning the lottery constant, which fixes our problem that worse teams are more likely to win; it’s as if the lottery winners were chosen from an even odds lottery.
We run regressions for attendance and winning percentage for each of the five years after the draft lotteries. Here are the effects of winning the first pick on future attendance:
Winning the lottery (the blue line) increases attendance in each of the following five years. We plot attendance in logs — winning the lottery is worth about 1,200 extra fans per game the first year and even more in later years. That’s a lot of money, providing a strong incentive to tank.
What about winning percentage?
In most years following a lottery win, getting the first pick increases winning percentage about 3 or 4 percentage points (this increase is not statistically significant). The exception is the fourth year after the first pick, when winning percentage is about 10 points higher. Still, it’s fair to say that, rather than team improvement, general excitement and high expectations for the first pick drive attendance gains.
For teams interested in the bottom line, tanking is worthwhile. Do teams actually tank? This has been studied before; most researchers have compared winning percentages for playoff eliminated teams and playoff eligible teams. Some studies compare the same teams pre/post elimination. The idea is that teams don’t think about tanking before they are eliminated from the playoffs but do respond to draft incentives as soon as the playoffs are out of reach. While possibly true for fringe playoff teams, this argument seems wrong for teams like the Bobcats, Wizards, and Hornets this year. These teams knew quite early that they would not make the playoffs. Their incentive to tank is unfettered and ever present. Even when fringe teams are eliminated from the playoffs, they lose the incentive to work for a championship. Comparing their performance before and after will capture this change as well as a possibly new focus on tanking.
Instead, we decided to look at a smaller window of games where we can isolate the effect of tanking incentives. In the last few games of each season, some playoff eliminated teams reach a position where they can no longer change their lottery rank. In playoff seeding lingo, they have clinched all they can clinch, so they have nothing left to play for. In particular, they have no incentive to tank, while other lottery teams still do. We can compare these two sets of teams to figure out whether teams truly respond to draft incentives.
Here’s the comparison of winning percentages for lottery teams that can change their lottery position and teams that can’t (restricted to home teams, to avoid double counting games):
We adjusted these numbers for team quality by controlling for team by season fixed effects in a regression. We also included fixed effects for the number of games remaining in the season in case teams generally tried harder in the last couple games. This means that our estimates capture a “difference in differences.” In plain terms, we compare how performance changes when teams “clinch” their lottery spot with how performance changes for teams that haven’t yet clinched as the season winds down.
Before the 1985 draft (when there was just a coin flip between the worst two teams for the top pick), winning percentage barely changed when teams clinched their lottery spot. From 1985 to 1989, all teams had even odds of winning the lottery, so no one had an incentive to tank. After 1989, under the current (or similar) lottery system, tanking is rampant. Teams have a lower winning percentage by about 25 points when they can change their lottery spot (red bars) versus when they can’t (blue bars). If we restrict the sample to games where the opposing team still has a chance at the playoffs (and hence will not tank themselves), the difference is even larger.
Tanking is primarily driven by teams in the first or second lottery position:
The size of the bubbles corresponds to the number of games in each category. Teams ranked first and second clinch their spot more often than other positions (biggest blue bubble), and they must feel like they have the most to gain from losing, though in fact the return to improving your lottery rank is only slightly higher at the highest ranks (6% versus 3% improvement in first pick odds). Perhaps fans are more understanding for these teams as well, since they are quite bad.
We can also see that our tanking measurements are taken primarily from the last two games:
The small size of the blue bubbles for three, four, and five games remaining means that teams generally don’t clinch their lottery position until the last two games. Surprisingly, after much losing, clinched lottery teams win over half their home games in the last two games of the year.
The results for the current time period (since 1990) are quite robust. Whether we control for team fixed effects, the number of games remaining in the season, or nothing at all, clinched teams win much more than teams whose position can still change. Despite even stronger incentives to tank in the early period (pre-1985), teams didn’t do it. Fans don’t blame teams for doing it now — they reward them with higher attendance if they win the first pick and seem to understand why they tank. And, we’re not accusing teams of putting in low effort on the court. “Tanking” could mean experimenting with different lineups or shutting down an injured star player. Still, it’s not your dad’s NBA anymore.