Sunday, 13 April 2014

Quick Hits: Shanahan Is Full Of Crap

I'm sure that most of these are going to start like this, but I was reading tonight's Provies when something caught my eye. It was this quote, from Brendan Shanahan, new overlord of the Toronto Maple Leafs:
I think it’s a complete cop-out that you can’t learn or be taught how to score at the NHL level. I hear coaches say all the time that you can teach defence, but you can’t teach offence. I don’t buy that. I’m an example of the opposite.
I bolded the important bit. This stuck out to me because I have this running theory (you know the one if you've read anything I've written about any Canucks prospect ever) that guys that turn into NHL scorers, in general, dominate their junior leagues. Shanahan scored nearly 1400 NHL points, so to hear him say "I had to be taught how to score" with the implication that he didn't know how to before hand is a bit curious. So I went to HockeyDB and looked at how a draft-year Brendan Shanahan performed in the OHL relative to his fellow 17 year olds. Here's how he did:

That is a list of every 17-year old forward who played at least 20 OHL games in Shanahan's draft year. As you can see, he wrecked them. He's on an entirely different level from the rest of those guys. So it's not as if Shanahan was a talentless hack coming in to the NHL. He dominated the OHL, so he dominated the NHL. For him to say that he had to be taught how to be a scorer is a crock of shit. He was always just way more talented than everyone else. The power was in you all along, Brendan!

Wednesday, 9 April 2014

Quick Hits: The Boston Model

Jason Botchford wrote a bonus episode of the Provies tonight, and I tweeted some things about it. I'd like to elaborate on these things a bit, but first here's what I said:







And now, in the style of Elliotte Friedman, here are 12 Thoughts:

  1. I like toughness. I really do. Despite fawning over Jamie Benn over Twitter, my favourite NHL player is Milan Lucic. I saw him lead the Vancouver Giants to the 2006-07 Memorial Cup, and my lasting memory from that run was this shift in which he flattened two Medicine Hat Tigers and facepunched a third. At the same time, his toughness kinda distracts people from what makes him an effective hockey player. He moves the puck extremely effectively, sees the ice well, makes smart passes, and is able to possess the puck in the attacking zone. There are plenty of other guys that can punch faces in the NHL and minor leagues, and there are plenty of guys who can blow people up with hits. The thing that separates Milan Lucic is that he's really good at hockey.
  2. Full disclosure: the moment that Jamie Benn entered my heart as a hockey fan was when he fought Jarome Iginla in 2010 and split Iginla open pretty good. Hockey fights are fun, but they're not necessary for success.
  3. Back to the Bruins. When the "Boston model" is differentiated with the "Detroit model," it's pretty safe to assume that people are contrasting on the basis of perceived team identity. Detroit has the perception of a small, skillful team, whereas Boston is seen as a bit of a Broad Street Bullies throwback. The reality, however, is that both teams are/were remarkably similar in their heydays. When Detroit was at it's post-lockout peak, they had a generational talent on defense who had the puck all the time, one of the best 1-2 punches at centre in the league, and were able to fill in the gaps with exceptional drafting that allowed them to win in the margins. What's really important is that first and foremost, both teams were really good at hockey.
  4. It's funny to me that Patrice Bergeron is brought up as an example of a player that analytics doesn't really appreciate enough, because I think the reality is that analytics appreciate both him and David Krejci as hockey players a whole hell of a lot more than the MSM seems to.
  5. To me, singling out the "Boston model" indicates that ownership isn't looking at the "really good at hockey" part of the Bruins though. This is what worries me. If they were intent on building a strong feeder system for the Canucks, re-vamping the draft process, and becoming a team that was really good at hockey, it wouldn't matter if they followed the "Detroit model" or the "Boston model" since both are essentially the same in terms of stuff that matters. 
  6. I see Botch's comments (particularly the "from bullied to bully" one) as hinting that ownership wants a "big and tough" team. Well, Toronto wanted to get tougher over the offseason so they went out and got Bolland and Clarkson. Buffalo didn't want to be bullied anymore so they traded for Steve Ott and signed John Scott. Washington thought they needed to play a more grinding-type game so they made Bruce Boudreau play the trap before firing him.
  7. San Jose, conversely, has stuck to their guns and are a cup contender once again. Yeah they haven't won the cup, but the others are a steaming pile of garbage and bottom feeders in a worse conference. I'd rather be a contender than just suck. "Let's go get TOUGHER" has yet to work.
  8. I'm not sure what to think of the Bruins. On the one hand, you can't argue with a Stanley Cup. On the other, you totally can. They have managed to trade away Joe Thornton, Phil Kessel, and Tyler Seguin all in the past decade and remain one of the top-3 highest scoring teams at 5v5 for the last 3 full seasons and 4 of the last 5. Basically since David Krejci became a 1C, Patrice Bergeron fully recovered from his concussion issues, and Claude Julien became the coach. Those guys they traded are two top-5 NHL scorers and another guy who would still lead their team in points this season. How many other teams could have withstood that?
  9. Offensive talent is the hardest thing to acquire in this NHL. This is why guys who score a lot get the biggest contracts. I don't think you ever "win" a deal in which you give up a star, and the Bruins have done that three times (although only twice under Chiarelli). In that sense, they're incredibly fortunate to be where they are, and also incredibly fortunate that guys like Bergeron, Lucic, Krejci, and Marchand have all over-performed their draft position. With the benefit of hindsight, those guys all should have been top-10 or top-15 draft picks in their years.
  10. I'm hesitant to call this "good scouting" or "good drafting" though because after Marchand in 2006, the Bruins scouts have yet to find a single legitimate NHLer outside of the first round, and only Tyler Seguin and Dougie Hamilton have made the NHL at all. They dug up 5 core pieces between '03 and '06, and found nothing in the 5 years before or the 7 since. On the whole, that's not a stellar record.
  11. I've tossed around the idea of GM PDO a few times on Twitter, and it may apply to Boston. Have the Bruins figured out exactly the right thing to value in a hockey player, or did they get lucky because a small handful of guys with stuff they valued turned out to be really good at hockey too? I mean, Milan Lucic had 19 points in his draft year and could hardly skate at a CHL level. I honestly doubt that the Bruins really foresaw him turning into a legitimate NHL 1st line power forward.
  12. If this is the case, is it even possible to model your franchise after the Bruins? Many of the teams who have set out in search of toughness have failed spectacularly in becoming good. What made the Bruins different? Well, they signed the best defenseman post-Lidstrom and Pronger and got hit on a handful of mid-round draft picks in quick succession. I'm not sure that this is really a viable long-term strategy.
I'm not saying that Bruins management are just lucky and not smart (they have made some very shrewd pickups), but they are where they are because they have found good hockey players, not ones that are just tough and rugged and big. Emulating just toughness is just setting yourself up for failure. Vancouver's emphasis should be on finding good hockey players, regardless of size, facepunching ability, or perceived character flaws, and ss long as the Canucks' focus is not on getting really good at hockey, I fear for the direction of this franchise.

Quick Hits: First Impressions on the Second Linden Era

As much as I've panned hiring Trevor Linden as the president of hockey operations on Twitter, I think it's important to distinguish between criticism of Linden and criticism of the hiring. The fact of the matter is that Trevor Linden has no experience making hockey decisions at any level, and we don't know if he's going to be anything more than a well-informed figurehead. This means that no one can accurately say whether or not he's capable of fulfilling this role if he is making decisions on the hockey ops. side of things. The important thing that I heard in today's presser, and really the only bit of stuff worth chewing on, is that Linden seems to understand the need to surround himself with quality support staff and an excellent team of decision makers.

He brought up the example of Steve Yzerman when asked about his lack of experience, but the big difference is that Yzerman spent 4 years as an assistant GM under Ken Holland, Jim Nill, and the Detroit Red Wings, before moving on to Tampa Bay. There, he's surrounded himself with quality people like Julien BriseBois and Pat Verbeek, and a full-time statistical analyst among other support staff. Even though Linden doesn't have the experience of Yzerman, he can still make this work by building a team that does have experience in making hockey decisions. He can probably make this work quite well. We'll see.

What I think we can criticize is the fact that Francesco Aquilini hired someone like Linden though. From the outside, it looks like ownership's priorities lay more with corporate gladhanding and pandering to the fans still enthralled with the '94 run, rather than conducting an exhaustive search for the guy best suited to putting an effective hockey team on the ice. As Taj so lovingly transcribed for us, Elliotte Friedman suspects that this is a pretty transparent PR move more than anything. I'm of the mind that you get fans back by putting an entertaining and successful product on the ice rather than distracting them with shiny objects. Rather than looking for a guy who's determined to play an entertaining and successful style of hockey, ownership went and got a shiny object. "Look! It's Trevor! Go buy tickets! Do it for Trevor!"

I mean, I think Harrison Mooney pretty well nails it here:
Maybe it's because I'm admittedly a younger fan so '94 and by extension Trevor Linden the person don't resonate with me, but I don't see this as a move that will lead Vancouver to the promise land. Whether this move is successful really depends on who Linden (and ownership) pick to form the hockey operations team that will surely make the bulk of decisions, because leaning on Linden seems like a losing proposition.

Tuesday, 8 April 2014

Quick Hits: Crowdsourcing An NHL Team

Note: I often have things I want to say that I usually just tweet, but aren't really conducive to a medium that limits thoughts to a series of 140-character fragments. I figure I have this old blog so why the hell not use it. "Quick Hits" will just be my various thoughts and musings that aren't really Canucks-related, but I figure I should write about anyways. Hope you enjoy, and feel free to call me an asshole over on Twitter if you disagree: @Thats_Offside.

I've said this before, and I'm quite confident that it's still true: I really think that there is enough appropriate talent and brainpower floating around in the public that if we were to get together and form a front office, I really think that we could run a legitimately above-average to good NHL franchise. I say this for a couple of reasons: first of all, I'm confident in the quality of work that's done on the internet and I'm confident in the abilities of the people that do it, and second of all, I don't think that there's really a mystical innate understanding of the game of hockey that "hockey guys" are blessed with.

I'll tweeted about this first thing earlier:
The validity of our current best available measures of hockey have been criticized in large part to the fact that they aren't things developed by either reputable academics or, more importantly, "hockey people." You've all heard the tired "basement bloggers" thrown around, I'm sure. What's largely discounted is that stuff developed in the eye of the public is subject to ridiculously intense peer review. Every day, people who believe in fancystats are forced to read commentary from people critical of their insights, defend their arguments, and consistently refine their viewpoints as more and more other smart people challenge the current paradigm.

What's resulted is a way of thinking about and analyzing hockey that is as accurate as anything out there, and much more significantly, works and is successful. The very best coaches in the NHL, guys who have very recently won Jack Adams trophies and Stanley Cups, seem to believe in the very same things that Corsi and Fenwick tell us is true. Hell, I wrote an article for Shnarped Hockey on Monday that shows this. Whether or not the NHL has quietly been at the forefront of this great learning, the fact still remains that "basement bloggers" have found the very same "magic formula" (even though there is no magic formula) to building a successful hockey team as successful hockey teams have.

This brings me to my second point: I don't really think that being a "hockey guy" equates to a unique understanding of the game that the general public can never hope to achieve. I'm a believer in "deep practice,' meaning that anyone can become good at a specific skill, given enough time and effort. Give the people I interact with on Twitter enough games to watch and specific stuff to watch for, and I don't think that most would do worse than an average NHL scout.

This is even more applicable to the "leading voices" of the online analytics community. They're smart people. Eric Tulsky has a Ph.D. from UC-Berkeley and a B.A. in chemistry and physics from Harvard. Tyler Dellow has a J.D., a B.A. in poli sci, and a B.Comm. There are many others on hockey Twitter that are really bright, well-educated people too, so it's not as if the "basement bloggers" stereotype is applicable or even appropriate. Given that these people have a track record of being able to think critically and solve problems, I don't believe for a second that not a single one of these guys is incapable of seeing the same thing as some guy who has a high school diploma but is a "hockey lifer" can.

Of course, there's a ton more that an NHL front office has to handle than just player acquisition, but I still don't believe it's as if no one that's a part of hockey Twitter that can't be found to handle each role. The hockey fans whose work I read and who I interact with are a diverse and intelligent bunch, and I'm confident that a good number of them would be an asset to any front office, even if they were just to be a part of an organizational think tank and challenge what established hockey minds think.

If all else fails, we'll just appoint Kyle Dubas as our head. He still counts as hockey Twitter, right?

Saturday, 4 January 2014

CHL Goalies, Decision Making, and Hockey Canada's Failings

Despite a ton of hype and a high draft selection, Zach Fucale is an extremely risky prospect.
Soon after Hockey Canada announced the pre-tournament rosters for this year's World Juniors, I was pretty critical of their selections, most notably the decision to bring Zachary Fucale and Jake Paterson in goal over Edmonton's Tristan Jarry and Tri-City's Eric Comrie. I wrote this article on why I thought this was a massive, inexcusable mistake. If you haven't read that article, basically my argument was this: a goalie's ability to stop the puck is the absolute core of their job, so you want to bring the goalies that are the best at stopping the puck. Relative to their respective junior leagues, Eric Comrie and Tristan Jarry proved to be elite puck stoppers, while Fucale and Paterson were just barely above average. It's Hockey Canada's job to supply the World U-20 team with elite talent, and they bypassed the elite goaltending talent that Canada has at its disposal in favour of two average guys for essentially no good reason.

I received a lot of feedback on the post, most of it positive, but some understandably critical. However, most of the feedback I saw wasn't addressing the Fucale/Paterson vs. Comrie/Jarry debate that the whole piece centred around, but the small add-on at the end of the article where I implied that both Fucale and Paterson had extremely poor chances at becoming regular NHL goalies, let alone star starters. Wading through the usual "watch the games, nerd!" bullshit, a couple of consistent criticisms emerged:

  1. The intangible, cerebral aspects of goaltending were too easily dismissed.
  2. The WHL, OHL and QMJHL may play very different styles, so the effects of those styles on save percentage made comparison across leagues difficult.
I'll address the first point before delving into the massive pile of data I looked at to come to a conclusion about the second. Eric Tulsky did a really good job here showing that "clutch playoff performance" isn't really a consistent, magical, unexpected thing at the NHL level. Basically, any deviation between "clutch" to "choker" can be explained by simple random variance over small sample sizes. For example, "clutch" Zach Fucale had an abysmal 0.884 save% over the WJCs and pre-tournament games, after a disappointing 5-1 loss to Finland in the semi-finals. Did Fucale lose his ability to win big games after performing poorly in these ones? Nope, sometimes guys have tough stretches. That's just the way goaltending works.

As a result, it's far more important to analyze a goaltender's body of work as a whole in an effort to determine how many pucks you can reasonably expect a guy to stop. A goalie who has stopped more pucks in the past is a better bet to stop more pucks in the future and is therefore a better bet to win you more games down the road - it's pretty simple. Therefore, bringing a goalie with a consistently higher save percentage is a smarter bet than a guy with an average one. Jake Paterson has a career 0.902 save%, and Zach Fucale sports a career save% of 0.900. How do these numbers compare to their peers and are they good bets to become NHL players? I looked at that question in detail here:

Purpose of Study & Hypothesis

As I indicated above, the purpose of this analysis was not only to determine whether Hockey Canada's goaltending choices for the World Juniors were reasonable bets to perform at a comparable level to their WHL counterparts that were left off the team, but to also see if these players were likely to become NHL players based on the measurable attributes of their peers that made the NHL. 

My hypothesis was largely the same as the one I held for the series of defensemen posts I wrote back at the NHL Entry Draft (you can read those here, here and here): I believe that the NHL is really, really good super-concentrated collection of the very best hockey talent on the planet. As a result, you have to be an elite performer in your pre-NHL days to even make the league, and this elite performance is reflected one way or another on the scoresheet. As this applies to goalies, we should expect the guys who make the NHL as regulars to stop pucks at an elite level in junior.

Procedure

First, I defined a "regular NHL goalie" as a goalie who has:

  • 20 or more games of NHL experience.
  • Played in a minimum of 10% of available games played between their draft+3 and draft+13 seasons (basically, 8 games per year for 10 years).
  • Appeared in a minimum of 8 or more games 5 separate seasons if they are no longer in the NHL.
I think this criteria is pretty generous, since it basically includes guys that were rarely-used journeyman backups. Then, data was collected on every goalie who faced at least 1000 shots in the CHL, and their career save percentages were determined. Unfortunately, a lack of data limited the analysis to looking at goalies drafted only as far back as the 1998 NHL entry draft, so Andrew Raycroft was the earliest available NHL goalie who had complete and accurate data to work with.


(Side note: in the next couple of days, I'm going to write a post outlining how and why the data was collected the way it was, but that's much less interesting for most of you I'm sure. However, Josh Weissbock's work was absolutely invaluable in compiling this data, and I couldn't have put this post together without him, so a big thank you to you Josh #statthuglife)

After this was done, I compared each future NHL goalie's CHL performance with all of the goalies that played in his league and the CHL over the time frame when he was active. For example, Carey Price played four seasons in the WHL between 2003-04 and 2006-07, so his career save percentage was first compared to all WHL goalies who were active sometime within one or more of the seasons between and including 2003-04 and 2006-07, and then to all CHL goalies who played during that same period. To level the playing field and provide an accurate comparison across all leagues and all years, a standard score (z-score) was calculated for each goalie, then converted into a percentile for ease of comparison. If a goalie scored 85%, it means that he was better than 85% of his peers. Here are the results:

OHL

Save Percentage of all OHL goalies drafted since 1998 that made the NHL. Jake Paterson included for comparison.
Percentile of OHL goaltender career save percentage.
Percentile difference (percentile minus 50%) from OHL average.
The bottom two graphs are essentially the same, with the middle one organized in chronological order left to right, and the second demonstrating the difference between goalies studied and the average save percentage of their peers. This format is the same for the other leagues as well.

You'll notice that only Michael Leighton performed worse compared to his league peers than Jake Paterson has, and only Alex Auld is within + or - 10%. Given that there were a whole ton of goalies that performed at about the same slightly above average level as Auld and Paterson (~85 in the sample I studied), this would seem to indicate that Alex Auld is the huge outlying data point here, and probably the absolute best-case scenario for Jake Paterson in terms of NHL success.

QMJHL

The Q is interesting, because average goaltending is just currently so much worse than the other two Canadian junior leagues:



As shown by the graph, Quebec goaltending has, on average, been in the tank since roughly 2003-2004. This means that goalies with lower save percentages than in the OHL would score higher in percentile since their competition is so poor. However, if the QMJHL talent still holds its own relative to the rest of the CHL we can theoretically discount at least some of the proposed systemic effects that could depress save percentage. As it turns out, M-A Fleury and Corey Crawford (and by extension, Ondrej Pavelec and Jaroslav Halak) are still elite performers when comparing them to the rest of the CHL, and Jonathan Bernier is strong as well. If there were legitimate style-of-play reasons why QMJHL goalies were seeing lower save percentages, we would expect to see this effect across the board on elite goalies too. Since this is not the case, I'm led to believe that there isn't a quantifiable negative effect that's exclusive to the QMJHL.

Still, some decent talent has come out of the QMJHL in recent years. I also included the two European import goalies that made the NHL out of the Q in just the career save% graph as a small comparison:

Save Percentage of all QMJHL goalies drafted since 1998 that made the NHL. Ondrej Pavelec, Jaroslav Halak and Zach Fucale included for comparison.
Percentile of QMJHL goaltender career save percentage.
Percentile difference (percentile minus 50%) from QMJHL average.
Pavelec and Halak scored similar to Crawford and Fleury, which is to say they were elite QMJHL goalies. Zach Fucale performed at a level similar to but slightly below Kevin Poulin and Pascal Leclaire, neither of whom are or were anywhere close to a legitimate starting goalie, let alone an NHL star. It's also worth noting that Poulin and Leclaire, like Alex Auld in the OHL, have beat monumental odds to qualify as a "legitimate NHL goalie." 85 goalies scored similarly across the CHL, and only these three made the NHL in any capacity. That's a success rate of about 3.5%. Fucale scored closer to league average than Leclaire and Poulin did too, and only Poulin's career save% numbers were significantly inflated by a really strong draft+2 season in the QMJHL.

WHL

The strongest goalies drafted in the past 10 years have all arguably come out of the WHL, which is interesting because the 'dub was awful back in 1998. Then out of nowhere, goaltending improved dramatically and six legitimate NHL starting goalies emerged:

Save Percentage of all WHL goalies drafted since 1998 that made the NHL. Tristan Jarry and Eric Comrie included for comparison.
Percentile of WHL goaltender career save percentage.
Percentile difference (percentile minus 50%) from WHL average.
4 of the 6 goalies that made the NHL out of the WHL all scored in the 90th percentile or better, and a 5th (James Reimer) was just shy. Braden Holtby a bit of a surprise and an outlier for sure, but even his .905 career save% puts him more than 0.5 standard deviations above the mean in the entire data set. While he was better than average among his peers, he also played in a super-competitive WHL and was compared to guys like Price, Reimer and Dubnyk. 

The two WHL guys that were passed over for the opportunity to represent Canada also score exceptionally well here. Jarry's career save percentage is above 94% of his peers, while Eric Comrie is better than 90% of other WHLers. This also comes at a period where WHL goaltending is as strong as it's been since the 04-05 lockout period, so they're not dominating a bunch of scrubs. Just as it did before the tournament, Hockey Canada's decision to bypass these two entirely completely mystifies me.

CHL

Now that we've seen each future NHL goalie against their league in their era, here is each future NHL goalie against the whole CHL in their era:

Percentile of CHL goaltender career save percentage.
Percentile difference (percentile minus 50%) from CHL average.
While Tristan Jarry and Eric Comrie compare admirably to a group comprised mostly of NHL starters including Carey Price, Craig Anderson, Marc-Andre Fleury, and James Reimer, Paterson and Fucale most closely compare to guys like Poulin, Leclaire, and Auld. Michael Leighton still remains a total mystery but then again, it's Michael Leighton who's most generously described as a "replacement-level journeyman backup" at best, so it's not as if a legitimate NHL starter slipped through the cracks. It's still bizarre though.

Discussion

I would be remiss if I didn't hammer this point home at least once in this article: CHL success is no guarantee of NHL success. Even if a given goalie was an elite performer in junior, there was no guarantee that they'd make the NHL as even guys who posted career save percentages in the 84th percentile or above (more than 1 standard deviation above the mean) only had about a 20% chance of becoming an NHL'er, as shown by this table:

69th percentile represents ~0.5 standard deviations
But here's the takeaway from that: a draft pick is a bet. When drafting a player, you're betting that the player you pick will outperform any other player you have the opportunity to spend that pick on. A goalie who has an elite save percentage such as Tristan Jarry or Eric Comrie is historically 5-6 times more likely to develop into an NHL player than a goalie who performs similarly to Jake Paterson or Zach Fucale, so using an early draft pick on a goalie with mediocre numbers is generally a terrible, terrible bet.

And if you look at what is happening to NHL goaltending, the odds of a non-elite junior making the NHL appear to only be getting worse. You'll notice that the majority of guys who made the NHL with non-elite CHL save percentages did so about a decade ago, and basically only Braden Holtby has done so since. I believe this could be due to two related factors: the improvement of NHL goaltending as a whole, and the increasing presence of European netminders in the NHL. Here are those trends in graph form:

NHL ES Sv% by year since 2000.
Courtesy of www.QuantHockey.com.
This growth in NHL average save percentage seems to correlate with the explosion of the presence of Swedish and Finnish goaltenders in the NHL. Keep in mind that every time a Henrik Lundqvist or a Pekka Rinne or a Kari Lehtonen or a Tuukka Rask comes from over seas, they're not just taking a job - they're taking a job for 5-10 seasons. This means that goalie turnover is extremely slow. Maybe you'll only get one or two new faces per season and those one or two new faces have to be better than the guys they're overtaking. As a result, NHL jobs that would previously go to Canadian backup goalies (like Alex Auld, Michael Leighton, and Andrew Raycroft) are now going to top prospects and more talented Europeans - see Robin Lehner, Eddie Lack, Viktor Fasth, Antti Raanta, etc.

The improvement in NHL goaltending and the influx of European goaltenders conceivably means that as a proportion of total goalies that play, fewer and fewer CHL goalies will end up making the NHL as regulars, and those that do will be the absolute cream-of-the-crop elite performers. Given that only ~5% of CHL goalies turn into NHL'ers and there are roughly 60-70 legitimate average or better goalies currently in the CHL, there are probably only 3 or 4 future NHL goalies in the entire CHL right now. So who are they? Aside from Eric Comrie and Tristan Jarry who I've talked about already, future NHL'ers could be Flyers prospect Anthony Stolarz or Columbus prospect Oscar Dansk. They could be overage import stars like undrafted Eetu Laurikainen or Los Angeles' Patrik Bartosak too. Not all of those guys are going to pan out, and every single one has performed better in their CHL careers than both goaltenders Hockey Canada selected to represent the country at this year's World Juniors.

Perhaps the most damning thing about goaltending is that scouts and "hockey people" who claim to be experts on goaltending and forecasting performance are simply completely wrong the vast majority of the time. The following two graphs are courtesy of Matt Pfeffer, who works as a scout/analyst for the OHL's Ottawa 67's. They measure Goals Versus Threshold (GVT), which is essentially a hockey equivalent of baseball's Wins Above Replacement (WAR), against the draft position of a player. The first graph for skaters is essentially what you'd expect: the best players are taken early, then the value of skaters falls off considerably after the first few selections:


By contrast, goalies are all over the map. There's no real relationship between what their value turns out to be and what scouts think of them:


Add to this the fact that 84% of all goalies drafted out of the CHL between 1995 and 2009 failed to become NHL regulars, and you start to get the sense that scouts are, at best, really bad at judging goaltender talent level.

Conclusion

Odds are that any given goalie in the CHL will not make it to the NHL no matter who they are. That's not a slight against any goalie, that's just the hard truth. The NHL is the absolute best league in the world, getting in is ultra competitive and really, really difficult. The vast majority of prospects out of the CHL fail to get in, and those that do have tended to be elite performers when they were in junior. But, some guys stand a better chance to make the league than others and those guys tend to be the Tristan Jarrys and the Eric Comries of the world, not the Zach Fucales and Jake Patersons. Right now, based on what I've found, I would bet against Zachary Fucale becoming the #1 starting goalie that scouts forecast him to be, since goalies that have performed at the same level as he has have missed the NHL entirely 95% of the time. This is not to say Fucale almost guaranteed to fail. I don't know what he'll develop into. But it's far more likely he develops into someone playing in the Swiss A League than he does the next Henrik Lundqvist.

Of course, this whole thing underpins a much larger and broader discussion about Hockey Canada and player evaluation at the highest levels - mainly that the people that are in charge right now are not very good at it. While the goaltenders selected to represent Canada at this year's World Juniors were unquestionably the wrong ones, Zach Fucale and Jake Paterson were absolutely not the reason why Canada will fail to win the gold medal for the fifth consecutive year. 

Canada's roster was filled with "tough" and "gritty" and "responsible" players like Bo Horvat and Scott Laughton and Josh Anderson and Frederik Gauthier and Adam Pelech and Chris Bigras, while ultra-talented but "risky" stars like Max Domi and Darnell Nurse were left at home. In the end, Canada couldn't score and couldn't even carry the play against a far-inferior Finnish team. Canada had 11 1st round picks on their roster, Finland had 11 undrafted players. Player quality and player development clearly aren't the problems, so what else could it be? Well if the players aren't the problem, then scouting, evaluation, coaching, and management are. Canada is still the best country in the world at hockey, so it's time we start giving ourselves the best chances at winning again, and that starts with accurately evaluating who our best players really are.

Monday, 2 December 2013

Hockey Canada Screws Up The WJC's. Again.

No no no no no no no no no no no no no.
This morning, Hockey Canada officially announced the selection camp roster for the 2014 World Junior Hockey Championships. There were some glaring omissions for sure, as Darnell Nurse and Max Domi headlined the players that weren't even invited to attend camp. The full roster can be found here.

Now, if you follow me on Twitter and saw what I had to say during the CHL Super Series whenever the topic of goaltending was brought up, you'll know that I'm no Craig Button when it comes to Zachary Fucale. I don't particularly think he's the best U20 goalie in Canada, and certainly shouldn't garner consideration for the World Junior starting gig. I've listed a few of the reasons why I believe this before, most notably this:
Basically, a goalie's only job is to stop the puck. Zachary Fucale is bad at stopping the puck. Therefore, Zachary Fucale is bad at his job and shouldn't be on the World Junior team. It's not difficult logic to follow, but for some reason, logic seems to have completely abandoned Ryan Jankowski and the rest of the Hockey Canada decision makers, as they're citing stuff like Fucale's "ability to win."
Now, a couple of very reasonable people who I know understand my point have put forth some arguments in support of Fucale as a WJC goalie. They are as follows:


Basically the statistical argument in favour of Fucale is this: the QMJHL has a worse save percentage on average, so Fucale is just as good relative to the rest of his league as Eric Comrie, Tristan Jarry and Jake Paterson have in theirs. I'll explore this in a minute, but first I'll quickly go over why I ignore wins and GAA when evaluating goalies.

Goals against average is the average number of goals scored on a goalie per 60 minutes of play. It can be found by this formula:

(Total Goals Against/Total Time On Ice)*60

Goals against, however is a function of shots against and save percentage, so GAA can also be calculated in this way:

([Total Shots Against*1 - Save Percentage]/TOI)*60

As you can see, a goalie doesn't have any control over half of GAA, and the other half is just save percentage. Tracking GAA then only serves to add useless and irrelevant information when looking solely at goalies.

Wins are much the same, except now goals for are added to the picture. Bill James' expected win formula as it applies to hockey is Goals For^2/([Goals For^2]+[Goals Against^2]). Again, a goalie can't control the vast majority of what goes on in a win. The only part he can control is the number of shots he stops, which is his save percentage. It's for these reasons why I don't put a lot of stock into "knowing how to win games." It's especially relevant in Fucale's case because last year's Halifax Mooseheads probably could have won with 2013 John Garrett in goal. They basically crushed the rest of the QMJHL last year in score close possession, as illustrated by this data dug up by Josh Weissbock:


Having a team with two super-elite finishers in MacKinnon and Drouin, and that can tow nearly 60% of possession is ridiculously dominant. Having a huge season under those circumstances is less "knowing how to win games" than it is "having four functional limbs and a pulse."

Anyways, on to more relevant stuff. The prevailing argument in favour of Fucale is that he's as good relative to the QMJHL as Eric Comrie and Tristan Jarry are to the WHL and Jake Paterson is to the OHL. To see if this is true, I took the average save percentage over the last 3 years in each league, and compared each goalie's career CHL save percentage to this baseline. Here are the results:


Even with a league average save percentage more than 10 percentage points lower than both the WHL and OHL, Fucale only performs 97 basis points (0.97%) of save percentage better than the rest of his league. By contrast, both Comrie and Jarry perform far better compared to their peers, while Paterson lags behind the OHL.

I'll touch on this next bit a little more later, but I suspect the reason for a depressed QMJHL save percentage doesn't have anything to do with the quality of shooters or coaching systems or anything, but just fewer good goalies. For some reason, the QMJHL only developed 3 legitimate NHL goalies in the decade between 1999 and 2008, and none of those guys are really above NHL average for a starting goalie. Anyways, if you assume all leagues are of equal player talent, you can take a Canada-wide average of save percentages and compare all goalies against that baseline. Here are the results of that:


Okay, this is a little math-y now so bear with me. The Z-scores in the above table are a standard score. They measure how many standard deviations above the mean a given number is. A Z-score of 0 means average, below 0 is below average, and above 0 is above average. A score of 1 here would mean that a goalie's save percentage is better than 84% of those save percentages posted in the three year sample of games I looked at.

Long in short, Eric Comrie and Tristan Jarry score better than 90-95% of all CHL goalies. Jake Paterson is better than about 60% of all CHL goalies. Zachary Fucale is just barely above CHL average. In a cap-less system with no constraints on players, Hockey Canada passed on two elite performers to take two average ones.

So let's review the findings here:
1) Eric Comrie and Tristan Jarry are better than their respective league more so than both Fucale and Paterson.
2) Fucale's baseline comparison was easier, yet he still couldn't outperform it as much as his competitors.
3) Using a constant baseline for all goalies, Comrie and Jarry were found to be elite, while Paterson and Fucale are just slightly above average.

But wait, there's more. 

I feel like I'm piling on Zach Fucale at this point. I really don't mean to, but I think it's important to get this point across: he probably isn't good enough to be an NHL goalie. Similar to what I did with those defenseman posts back around the draft, I looked at the 10 year window between the 1999 and 2008 NHL entry drafts to see which goalies that came from the CHL turned out to be NHL-calibre players. In all, 120 goalies were drafted out of the CHL; 39 from the WHL, 47 from the OHL, and 34 from the QMJHL. Of this sample, just 15 goalies turned out to be what I consider an NHL regular. Here's the whole list:

The takeaway here is that guys who went on to be anyone of consequence in the NHL were all extremely good to elite junior performers. Zachary Fucale is neither extremely good or elite. He is average. And average CHL'ers don't make it to the NHL in any kind of significant capacity, despite what scouting reports like this will tell you.

To sum this all up, leaving one of Tristan Jarry or Eric Comrie off the World Junior team would be a poor decision. Leaving both off in favour of Fucale and Paterson is indefensible. Despite this, Canada should still win the gold medal because the talent level in this country is so much better compared to every single nation in the world that it should be nearly impossible to screw up. Yet Hockey Canada somehow has for five years now, and it's decisions like these that prioritize "heart" and "grit" and "winningness" over "ability to play hockey" that play a major role in those failures.

Sunday, 13 October 2013

Dan Hamhuis Has a 57% Corsi; May Be Struggling

Pictured: despair. Image via Chris Peters
One of the quickly developing storylines of the still infantile season is the play of Dan Hamhuis. If you're reading this, you probably know that he's failing the eye test in ways that are either hilarious or depressing, depending how much of your life hinges on a game that takes place before the season is even 10% done. Most recently he, Jason Garrison and Roberto Luongo did a thing. They did this thing:


I, uhh, yeah. That happened. Remember, you, valued season ticket holder, Francesco Aquilini paid those three guys a combined $212,976 tonight for that blooper, which really means nothing but I'm sure it'll get someone somewhere fired up if you tell them.

What isn't meaningless is that despite all his perceived struggles, Dan Hamhuis is killing it on possession. At the time of this post, he's sitting at a Corsi% of 56.8%, good for 3rd on the team behind only the Sedins. Yes, his PDO has been awful and he's due for an uptick in that. But, saying "oh you're just perceiving his poor play because Luongo isn't stopping pucks when he's on the ice" isn't really a valid response to criticisms like "he took a miserable interference penalty" or "he turned the puck over and fell down" because it doesn't address the aspects of his game that are being complained about.

So is there some crucial context that we're missing when looking at his Corsi numbers in isolation? Why is Hamhuis crushing puck possession but failing the eye test so mightily? Well, there are a couple of possibilities that I'll sum up in three main points:

  • Hamhuis' most common forward linemates are the Sedins, who are the Canucks' best possession players.
  • When Hamhuis starts a shift, it is more likely to be in the offensive zone with talented offensive players, relative to all other Vancouver Canucks defenders.
  • Consequently, Hamhuis' possession numbers are being inflated by John Tortorella's coaching decisions.

We know that starting a shift in the offensive zone leads to a boost in Corsi% immediately proceeding the faceoff, and we also know that playing with the Sedins will drag up a Corsi% too. Hamhuis leads all Canuck d-men in OZone start%, and is 4th behind Edler, Garrison and Bieksa in quality of competition. He has also started the lowest percentage of his shifts in the defensive zone out of all Canucks defensemen. While he's not being sheltered, he's the closest to it right now.

As a result, it's possible that Hamhuis' possession numbers are simply being inflated by Tortorella's usage of him. Tortorella has put Hamhuis in the best position of any Canuck defender to have strong possession numbers, and his deployment is working. This also would help to explain why Hamhuis' WOWYs look so damn good, as someone playing with Hamhuis is already more likely to be in a position where the Canucks get the next Corsi event, and someone playing apart from Hamhuis is more likely to be buried in the defensive zone. Good WOWYs here would have less to do with Hamhuis and more to do with coaching.

Of course, there's always an element of good old human error when using the eye-test. Since we're all in a frenzy just waiting for a Hamhuis gaffe, we'll definitely notice when he makes a small mistake just like every other NHL player makes from time to time. As a result of our attention, we're more likely to recall times in which Hamhuis struggled in order to better fit our hypothesis of him playing poorly. This is called confirmation bias.

I do suspect that there's a large element of confirmation bias going on in regards to Hamhuis' play, but quoting his Corsi% as evidence that he's not playing poorly misses a ton of context. The bottom line is that Dan Hamhuis should have the best possession numbers of any Canucks defenseman because he's been put in a position to have the best possession numbers of any Canucks defenseman. He may not be struggling as much as you think, but his possession numbers definitely don't tell the whole story.