Getting in the zone: A Q&A with the Predators’ analytics consultant
Posted by Dean Holden at November 10th, 2013
by J.R. Lind, 27 September 27 2013
During a radio interview on 102.5 The Game earlier this week, Hockey Abstract’s Rob Vollman mentioned that the Predators had consulted with a hockey analytics man named Eric Tulsky. (Indeed, Tulsky’s Twitter bio alludes to the relationship.)
By trade, Tulsky is an inorganic chemist and nanotechnology researcher and he holds a bachelor’s in chemistry and physics from Harvard and a PhD in chemistry from Berkeley. He also blogs about hockey at Philadelphia Flyers blog Broad Street Hockey. At MIT’s Sloan Sports Analytics Conference, he presented research on the importance of zone entries to a hockey team — basically, his research explores how a team brings the puck into (or out of) the defensive, neutral and offensive thirds of the ice and how that can lead to success.
In essence, Tulsky says his research “finally gets our stats into a form that’s useful to a coach and a player — we can pull the video of the plays where you dumped the puck in and figure out what needs to change to give you a better chance to carry the puck in. … The coach can go to the video, find the correct adaptation, and work with his players to improve their shot differential — something that’s hard to do with just a big-picture metric like Corsi.”
Throughout the offseason and into training camp, Predators Coach Barry Trotz has said he wants his team to get better at transitioning the puck, to focus on carrying the puck into the attack (rather than dumping it into the corner and chasing it down). During free agency, General Manager David Poile lauded the amount of shots (read our explanation of Corsi and its relationship to possession and success) his free-agent acquisitions produced and, in some cases, how well those new players carried the puck. It’s an easy leap to make between Tulsky’s research, his relationship with the team and this new emphasis from the Predators decision-makers.
Here’s a Q&A with Tulsky on his research, his findings and the world of hockey analytics in general — and he provides a clue as to why the Predators see Matt Cullen as a first-line center.
PostSports: It was — and in lots of circles still is — challenging to get the hockey community to embrace analytics. But it seems there’s been a sea change certainly in the last year. People like TSN hockey insider Bob McKenzie and Hockey Night In Canada‘s Elliote Friedman cite Corsi, a handful of teams are open about employing dedicated analytics people in their hockey operations departments with a few discussing openly how they use analytics.
As training camps have opened, coaches like Minnesota’s Mike Yeo have explicitly pointed to analytics as reasons for a change and here in Nashville, Barry Trotz has dropped buzzwords into his comments — talking about “puck possession” and “zone entries” and so on. Was there a specific point where you said, “Hey, this is really something that’s become part of the conversation”? If there has been a sea change, what can it be ascribed to?
Tulsky: The change doesn’t seem as abrupt to me as you’re describing. Two years ago, James Mirtle was writing about how Gabriel Desjardins was working for several teams and two others had hired Sports Analytics Institute. Six months later, (Boston Bruins’ general manager) Peter Chiarelli said half the teams in the league were doing analytics work of some form. There’s no doubt that things have moved along farther since then, but I think if there’s been a sea change in the last year, it’s been more in the media than in team front offices.
PS: It’s generally accepted that shot-based metrics — Corsi and Fenwick — are seen as both accurate indicators of possession and, by extension, measurements of success and indicators of future performance. With that having taken hold, you and others have started looking at what drives those numbers.
In other words, how can a team address a deficiency. Your research focuses a lot on zone entries. Explain what we’re talking about when we say “zone entries.”
ET: In zone entry tracking, each time a team sends the puck into the offensive zone, we note which player sent it in and whether they carried it in or played dump and chase. With those little pieces of information, we can calculate a bunch of interesting metrics for each player:
• How involved is he in gaining the zone: if the team enters the zone with him on the ice, what are the odds he’s the one sending it in?
• How successful is he at gaining the zone: when he brings the puck in, how often does he carry it to possession rather than dump and chase?
• How good is he with the puck in the offensive zone: when he carries the puck in, how many shots does the team get?
• How good is he off the puck in the neutral zone: what is the team’s carry-in percentage when he’s on the ice?
• How good is he off the puck in the offensive zone: how many shots per carry-in does the team get when he’s on the ice?
• How good is he at puck retrieval: how many shots per dump-in does the team get when he’s on the ice?
• How good is his neutral zone defense: what is the opponent’s carry-in percentage when he’s on the ice?
• How good is his defensive zone defense: how many shots per carry-in do the opponents get when he’s on the ice?
And that’s just a start; there are literally dozens of interesting things we can calculate from just this one piece of information and the data in the NHL’s play-by-play sheet. It’s an incredibly powerful tool.
PS: How did you decide to examine zone entries relationship to possession? Was it a trend you perceived before you dove into the numbers or was it more a matter of looking at a bunch of different things then discovering zone entry was the keystone?
ET: Honestly, when I started the project, I thought we were headed in a different direction altogether.
One thing that troubled me is that we didn’t really have statistics that separate offensive skill from defensive skill. Shot differential metrics like Corsi obviously involve both, but even if you look just at shots for, it’s convolved with defense — maybe the team gets a lot of shots because the player’s defensive skill keeps the opponent’s possessions short and so they get back into the offensive end quickly. To try to address this, I wanted to move from looking at shots per minute to shots per possession. All I needed to do that was the information in the official play-by-play plus a list of zone entries. That lets me divide the game into a series of possessions that each begin with either a zone entry or an offensive zone faceoff and each end with either a stoppage or a clear. Then I can calculate how many shots come from each possession and look for players who have a systematic influence.
It was only after we collected the data that I found how important a driver of possession the zone entry was.
PS: Sort of counter-intuitively perhaps, you found that neutral-zone play is the key — that it matters how a team gets into and out of the center third of the ice more than it matters what they do in their attacking or defending ends. Was that a surprise to you? When you talk to teams, is it a surprise to them? Are you turning the world on its ear?
ET: I touched on this above, but that definitely wasn’t what I was expecting. It turned out that the difference between players in how often they establish possession in the offensive zone was much larger than the difference in how many shots they generate from each possession. The neutral zone appears to largely determine a team’s shot differential, and the offensive and defensive zone play is often more related to the quality of those shots.
Teams seem surprised and interested when I share the data with them, but not so shocked as to be skeptical. It’s obvious that carrying the puck into the offensive zone is better for the offense than dumping it in, but it’s a bit surprising how large that difference is — and how much less damaging the failed attempt to carry is than we might imagine. A bad entry certainly can turn into an odd man rush the other way, but we’ve tracked failed entries and find that those plays aren’t nearly as frequent as people think.
PS: When someone just looks at Corsi, for example, it’s tempting to say “Well, they’d be better if they only shot more and increased their Corsi.” But it’s not that easy, right? If Corsi is a reflection of possession, then isn’t the bigger question “How do we get, keep and maintain possession?” Is Corsi just the tip of the iceberg?
ET: As you note, shot differential correlates very well with puck possession and scoring chances. So it’s safe to guess that if your team consistently outshoots the opponents whenever you’re on the ice, you’re probably doing something right — but it doesn’t say what. It’s useful information for a GM deciding whether you’re someone he wants on his team, but it’s not much help to you or your coach.
Breaking things down to analyze exactly what drives possession makes that form of player evaluation even better; it lets us suss out exactly how responsible you were individually for the good results your team had with you on the ice. But just as important, it finally gets our stats into a form that’s useful to a coach and a player — we can pull the video of the plays where you dumped the puck in and figure out what needs to change to give you a better chance to carry the puck in. Maybe you need to get the puck earlier, or maybe you need to be positioned differently, or maybe a teammate needs to be positioned differently, or maybe you had a lane that you didn’t see, or maybe you just need to be more aggressive in challenging the defense. The coach can go to the video, find the correct adaptation, and work with his players to improve their shot differential — something that’s hard to do with just a big-picture metric like Corsi.
PS: Describe, if you can, the big takeaway of your research. Is it as simple as “carrying the puck in the zone is better than passing it in and both are far better than dumping and chasing”? To what measurable degree does carrying the puck in improve a team’s chances over a dump in or a pass in?
ET: I’ve touched on some of this in my answers above, and I presented a fairly thorough review [pdf] at last year’s MIT Sloan Sports Analytics Conference. The short version is this:
• Carrying the puck (or passing it; they’re roughly equivalent) in generates more than twice as many shots and goals as dump-and-chase, even after you weed out odd-man rushes and plays where the team with the puck dumps the puck in and goes for a line change.
• At least across a few hundred games, differences between players on shots per carry-in (or shots per dump-in) were small, so the majority of shot differential was determined in the neutral zone.
• Failed attempts to carry the puck in don’t appear to be nearly as damaging as people might think.
With the upside of carrying the puck in being bigger than people realize and the downside of failing smaller than they realize, the net result is that teams should be much more aggressive at the blue line than they currently are.
PS: When you consult with a team, what do you say? How do you approach these grizzled old hockey guys? Do you see a light go on in their eyes and what makes that happen?
In a lot of ways, an outside consultant in hockey is no different than an outside consultant in any field. They are bringing you in because you have an idea of how to do something better, but inertia is a powerful force in all businesses.
ET: In general, I try to be really honest about what I can and can’t do with the data. If I go in with the attitude of trying to sell something, I might land some business in the short run, but if I’m overpromising and can’t deliver, it won’t help me in the long run.
I think one of the big things I have going for me is an ability to bridge the gap between rigorous analytics and plain language, to explain in fairly simple terms exactly what the data says (and doesn’t say). So my goal is to bring teams along with me, to show them exactly why I believe something and how confident in it I can be. Doing that, and building their confidence and trust, is the only way to really have an impact.
Some people try to sell black box ranking systems with a proprietary methodology. I can’t see how that would ever really work well — if someone comes to me with a system like that, one of two things happens. Either:
1. The ranking system matches what I already believe, and then it doesn’t have much value for me, or
2. The ranking system is different from what I believe, and I’m left wondering why I should trust this black box over my own intuition.
These guys are absolute experts in what they do, and they’re right to be skeptical of a random fan who says he can help them do better. It’s my job to show them exactly why I think that, and to help them see what my analytics can do.
PS: Is there a player-level way to look at entries or is it more at a team level? Is there a way for GMs to leverage what you’ve learned when they are looking at free agents, for example, or is this more about what coaches can do tactically? Is it a mixture of both? And if there is a way to look at it on individual level, what, if anything, can you tell us about the Predators? Who is “the best” at this stuff? Some of your research used data from the Minnesota Wild, so is a guy like Matt Cullen someone who stands out?
ET: I think I’ve touched on this above, and the SSAC paper has a bit more about how it can be used to evaluate players and systems. (Cullen was indeed the best on the Wild at carrying the puck in in 2011-12.) It’s easy to imagine using zone entry data to figure out whether a player with a good Corsi was driving his line’s results or just benefiting from playing with good teammates, and making sure to target the free agents who really drive the play. And at the system level, it’s clear that teams can push harder to try to carry the puck in, to encourage their players to take risks and put them in situations where they have every chance of gaining the zone.
Another interesting example: in 2011-12, Danny Briere and Wayne Simmonds were on a line together almost all year [for the Flyers]. Briere was great at gaining the zone, carrying the puck in on about 2/3 of his entries. Simmonds wasn’t very good at it at all, dumping the puck in more than half the time. So as a coach, you want Briere to be the one with the puck as they enter the zone, but in practice Simmonds actually had a lot more entries than Briere did. If I were the Flyers, I’d be going to the video to figure out whether Briere is putting Simmonds in bad situations or whether Simmonds is missing some opportunities, and working to help the two of them adjust their approach to entering the zone to get the most out of their abilities.