To tank or not to tank

Last week, I mentioned that my paper with my PhD cohort Chris was accepted for the poster session at the MIT Sloan Sports Analytics Conference. I’ll give the summary and some pictures today (you can find the full paper on my academic website). The project looks at the age old subject of tanking for position in the NBA draft lottery. We answer two questions:

  1. Should teams tank for a better draft position?
  2. How much do teams actually tank?

For the first question, we head right to the lottery. We are interested in the causal effect of obtaining the top pick in the draft. If the first pick is truly valuable, then teams should be willing to lose intentionally to get it. LeBron James, Tim Duncan, and Shaquille O’Neal were all first picks, but so were Greg Oden and Michael Olowokandi. We want the average value of all the first picks since the draft lottery took its current form in 1990.

Since there is some randomness in who wins the lottery, it’s tempting to just compare lottery winners and losers. However, the lottery isn’t a simple coin flip. The worst teams have a stronger chance of winning. If we compare winners and losers with no adjustments, winners may come out looking worse than they should, since they were worse to begin with. This will generate estimates of first pick value that are too small.

To eliminate this bias, we use a statistical trick to adjust for team quality. We regress future attendance and winning percentage on a dummy variable equal to one if the lottery team won the first pick. In the regressions, we control for each team’s probability of winning the lottery — this is called their “propensity score” in statistics. Intuitively, this holds the chance of winning the lottery constant, which fixes our problem that worse teams are more likely to win; it’s as if the lottery winners were chosen from an even odds lottery.

We run regressions for attendance and winning percentage for each of the five years after the draft lotteries. Here are the effects of winning the first pick on future attendance:

Winning the lottery (the blue line) increases attendance in each of the following five years. We plot attendance in logs — winning the lottery is worth about 1,200 extra fans per game the first year and even more in later years. That’s a lot of money, providing a strong incentive to tank.

What about winning percentage?

In most years following a lottery win, getting the first pick increases winning percentage about 3 or 4 percentage points (this increase is not statistically significant). The exception is the fourth year after the first pick, when winning percentage is about 10 points higher. Still, it’s fair to say that, rather than team improvement, general excitement and high expectations for the first pick drive attendance gains.

For teams interested in the bottom line, tanking is worthwhile. Do teams actually tank? This has been studied before; most researchers have compared winning percentages for playoff eliminated teams and playoff eligible teams. Some studies compare the same teams pre/post elimination. The idea is that teams don’t think about tanking before they are eliminated from the playoffs but do respond to draft incentives as soon as the playoffs are out of reach. While possibly true for fringe playoff teams, this argument seems wrong for teams like the Bobcats, Wizards, and Hornets this year. These teams knew quite early that they would not make the playoffs. Their incentive to tank is unfettered and ever present. Even when fringe teams are eliminated from the playoffs, they lose the incentive to work for a championship. Comparing their performance before and after will capture this change as well as a possibly new focus on tanking.

Instead, we decided to look at a smaller window of games where we can isolate the effect of tanking incentives. In the last few games of each season, some playoff eliminated teams reach a position where they can no longer change their lottery rank. In playoff seeding lingo, they have clinched all they can clinch, so they have nothing left to play for. In particular, they have no incentive to tank, while other lottery teams still do. We can compare these two sets of teams to figure out whether teams truly respond to draft incentives.

Here’s the comparison of winning percentages for lottery teams that can change their lottery position and teams that can’t (restricted to home teams, to avoid double counting games):

We adjusted these numbers for team quality by controlling for team by season fixed effects in a regression. We also included fixed effects for the number of games remaining in the season in case teams generally tried harder in the last couple games. This means that our estimates capture a “difference in differences.” In plain terms, we compare how performance changes when teams “clinch” their lottery spot with how performance changes for teams that haven’t yet clinched as the season winds down.

Before the 1985 draft (when there was just a coin flip between the worst two teams for the top pick), winning percentage barely changed when teams clinched their lottery spot. From 1985 to 1989, all teams had even odds of winning the lottery, so no one had an incentive to tank. After 1989, under the current (or similar) lottery system, tanking is rampant. Teams have a lower winning percentage by about 25 points when they can change their lottery spot (red bars) versus when they can’t (blue bars). If we restrict the sample to games where the opposing team still has a chance at the playoffs (and hence will not tank themselves), the difference is even larger.

Tanking is primarily driven by teams in the first or second lottery position:

The size of the bubbles corresponds to the number of games in each category. Teams ranked first and second clinch their spot more often than other positions (biggest blue bubble), and they must feel like they have the most to gain from losing, though in fact the return to improving your lottery rank is only slightly higher at the highest ranks (6% versus 3% improvement in first pick odds). Perhaps fans are more understanding for these teams as well, since they are quite bad.

We can also see that our tanking measurements are taken primarily from the last two games:

The small size of the blue bubbles for three, four, and five games remaining means that teams generally don’t clinch their lottery position until the last two games. Surprisingly, after much losing, clinched lottery teams win over half their home games in the last two games of the year.

The results for the current time period (since 1990) are quite robust. Whether we control for team fixed effects, the number of games remaining in the season, or nothing at all, clinched teams win much more than teams whose position can still change. Despite even stronger incentives to tank in the early period (pre-1985), teams didn’t do it. Fans don’t blame teams for doing it now — they reward them with higher attendance if they win the first pick and seem to understand why they tank. And, we’re not accusing teams of putting in low effort on the court. “Tanking” could mean experimenting with different lineups or shutting down an injured star player. Still, it’s not your dad’s NBA anymore.

21 responses to “To tank or not to tank

  1. Does this consider lottery teams that were trying for playoff spots and didn’t make it? For example, a #9 or 10 seed might have been trying to do the opposite and win to get into the playoffs…

    Also, John Lucas pretty much admitted the Cavs tanked in 03:

    • All the figures are restricted to teams that have been eliminated from the playoffs, even if they haven’t quite clinched their lottery spot. So, for example, a 10 seed that could still catch the 9 seed but not the 8 seed would be in there as a team that could still change its lottery position (if they could still catch they 8 seed, they would be excluded). As soon as they clinch the 10 seed for sure (can’t catch the 9 and can’t be caught by the 11), they switch to the clinched category (can’t change their lottery position).

      • Nice– thanks for the clarification!

        I also found it interesting in your full paper that only 4 out of the 25 1st picks since the lottery have actually won a championship… Seems that getting the #1 pick might help put fans in the seats, but not rings on fingers.

      • Yeah that’s one of my favorite stats from this project. I was surprised it was that low. Also, regarding your earlier comment about the Cavs, I’m always surprised when people like Simmons complain about tanking. If you want fairness/redistribution generated through the draft, you have to have incentives for tanking. There’s no way around it. I think the fans of most bad teams understand that and don’t fault their teams for it.

  2. Thanks for the “casual” analysis Tyler…

  3. Shit….. Causal….

    This whole time I thought the title was Casual…

  4. Have you done any prelim work to see if NFL results would be similar ?

  5. Not yet, but I should. It’s harder to value the draft picks in the NFL, since there’s no lottery procedure to introduce randomness. I could do the exact same tanking analysis, though.

  6. Hadn’t realized ’05, ’06 and ’07 first round draft picks were such massive draft busts. 2011 NFL Eagles example makes for more difficult analysis. While the 4-8 Eagles had every incentive to tank the season, at 4-12 they would have had the 4th pick in the draft and at 8-8 now pick at 15. However overriding self interest took over, Head Coach Andy Reid likely would have been fired, and his friend Def. Coordinator Juan C. certainly would have been fired. Instead both keep their jobs.

  7. Indeed, the Eagles don’t follow the pattern. Every coach/management team has similar incentives, but what made the Eagles different from most teams with poor records is that they actually have a good team. In their case, fans would have been pretty pissed off if they tanked for a pick, because they don’t really need a high pick to be good next year. I think they are the exception, not the rule, however. If the Eagles had a less talented group, management might have gotten together with Andy Reid and said, look, you’re getting fired either way, so just tank it for us. Coaches are often sacrificed along with a tanked season, but they generally deserve it. In Reid’s case, he probably didn’t (see my earlier posts on their bad luck

  8. Pingback: Sloan Sports Conference publicity | Causal Sports Fan

  9. Pingback: Who tanks in the NBA? | Causal Sports Fan

  10. Pingback: The Tank Watch (4/17/2012) | Causal Sports Fan

  11. Pingback: Here come the tanks | Causal Sports Fan

  12. Pingback: Cavaliers, what have you done? | Causal Sports Fan

  13. Pingback: The Tank Watch (4/27/2012, final edition) | Causal Sports Fan

  14. Pingback: “Fairness” in sports | Causal Sports Fan

  15. Pingback: Is Joe Flacco Elite? Barnwell strikes again! | Causal Sports Fan

  16. Pingback: Is Joe Flacco Elite?

  17. Pingback: Is Joe Flacco Elite?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s