Introducing hbGAR, Part 1: A New Catch-All Player Evaluation Tool
The ins, outs, and everything in between on my new and original hockey statistic
Note: This article is an updated piece that is based off the original version of this model, first published on June 1. GAR totals you see in this article are slightly out of date
One of the main goals I have for my hockey and baseball visuals is to provide as much context on a player as I possibly can. After all, my hockey visuals specifically are fairly comprehensible, but they aren’t exactly simplistic, with the amount of information I try to jam into them.
Over the last few days, I’ve tried to add a whole new level of context to player evaluation in hockey with my own take on Goals Above Replacement, a catch-all statistic popularized by Evolving-Hockey.com. My model is primarily based on many metrics available on their site, so allow me to explain everything there is to know about my take on this stat, which I call hbGAR: HB Analytics’s Goals Above Replacement.
*A reminder that the term “goals”, in the connotation of GAR, means hypothetical goals, not actual ones. “Replacement-level” is a term given to a player who is often on the shuttle between the AHL and NHL, or a common healthy scratch used as an injury replacement*
The basic framework for this stat isn’t too different from the baseball statistic I created around two weeks ago called HBV, which you can read about here. The 6 categories Evolving-Hockey uses in their GAR model are Even-Strength Offense (EVO), Even-Strength Defense (EVD), Powerplay Offense (PPO), Shorthanded Defense (SHD), Taking penalties, and drawing penalties. In my model, there are the same 6, plus another: EVO, EVD, PPO, SHD, Take, Draw, and A3Z (all-three-zones). The A3Z, Take, and Draw categories are weighted much less than the other 4 categories, because the stats I use in those categories (which include offensive zone entry rate, defensive zone exit rate, total taken penalties, and total drawn penalties) are either hand-tracked and fall onto the “microstat” side of the scale of hockey stats, or in the case of the penalty categories, I deemed a player’s ability to draw penalties and stay out of the box to not be as important as his impact in the offensive and defensive zones.
I have two slightly different models for forwards and defensemen at even-strength. The model for forwards focuses more on offense, including more offensive statistics than defensive ones; vice versa for defensemen. In terms of the powerplay, penalty kill, and A3Z categories, I used the same stats for both forwards and defensemen, because when attacking on the powerplay, all 5 players on the ice are trying to create offense, regardless of whether they’re forwards or defensemen. The same goes for defending on the penalty kill.
The way I came up with actual GAR figures is by taking the average of players’ Z-scores in various statistical categories. For those who don’t know, a Z-score in this case is how far above (or below) average a player ranks in a certain stat. The higher the Z-score, the greater a player is in a certain area compared to league average. At this point you may be asking, why isn’t this called Goals Above Average instead of Goals Above Replacement? Well, I did some digging, and there’s actually an extremely acute difference between average in most of the statistics that Evolving-Hockey uses, and their GAR model’s interpretation of replacement-level. A large portion of my GAR model comes from the stats in Evolving-Hockey’s RAPM section, so calling this my own version of Goals Above Average would simply be incorrect. Essentially, if the bars on a player's RAPM chart fall even slightly on the wrong side of zero in any given statistic, not only does that player rank below-average in that statistic, they almost always rank below replacement-level too.
Next, I’ll go over which stats I decided to use, and why I decided to use them:
Even-Strength Offense (Forwards):
=(Average of Z-scores of GF, xGF, CF, and the difference between FSh% and xFsH%, times 4) times (TOI/GP divided by 1000)
Even-Strength Offense (Defensemen):
=(Average of Z-scores of GF, xGF, and CF, times 3) times (TOI/GP divided by 1000)
Even-Strength Defense (Forwards):
=(Average of Z-scores of xGA and CA, times 3) times (TOI/GP divided by 1000)
Even-Strength Defense (Defensemen):
=(Average of Z-scores of xGA and CA, and FA, times 4) times (TOI/GP divided by 1000)
=Average of Z-scores of GF, xGF, CF, and the difference between FSh% and xFsH%, times 1.75
=Average of Z-scores of xGA and CA, and FA, times 1.25
=(PossessionEntry% - 0.5) + (PossessionExit% - 0.5), times 1.5
G (results), xG (shot quality), C (shot quantity) in RAPM form; FSh% means Fenwick Shooting Percentage (shooting percentage on unblocked shot attempts)
=Z-score of total penalties taken / 10, divided by 1.75
=Z-score of total penalties drawn / 10, divided by 1.75
=EVO + EVD + PPO + SHD + A3Z+Take+Draw
Now, you may notice that all of the shot-based metrics I used are not on a per-60-minutes, or rate, basis. The reason for this is that GAR isn’t traditionally measured on a rate basis. Per-60 stats tend to favour players who aren’t deployed in a large role. To adjust for this factor even more, the TOI portion of the model acts like a weight in itself so as to further ensure that players deployed in smaller roles won’t have skewed GAR ratings. The logic behind the A3Z section is simply to see how far above or below the 50th percentile a player is in zone entry rate and zone exit rate. The reason I divide 50 by 100 to get to 0.5 is to weigh this section lighter than the other sections - I’m obviously not going to value how good a player is in transition on the same level as the actual impact he has on the game. To put that into context, Roman Josi had the highest A3Z GAR of any NHLer in 2019-20 according to my model, with a total that rounded up to 1.5.
Some other pieces of info that should be worthy of note: A cutoff of 100 minutes of TOI (time on ice) was used for even-strength, for both forwards and defensemen. A cutoff of 50 minutes of TOI was used on the powerplay and the penalty kill. A cutoff of 500 minutes of TOI was used for taking and drawing penalties. Any player who doesn’t meet the TOI cutoff in any of these categories will have their GAR total for that category regressed to zero. There was no linear regression or code used to determine the weights; I experimented with many different numerical figures for that part. In the end, those specific weights were used solely to base the model a lot more off even-strength play as opposed to weighing all strength states evenly, which was the biggest flaw of an earlier version of this model I previously experimented with.
At this point, I could have simply just posted a spreadsheet for you all to download so you could have access to this data, and that would be that. But you know what? That’s boring. Instead, I created a GAR visual, that not only shows a player’s GAR totals, but exactly where his GAR total in each category comes from.
*Note: GAR totals are slightly out of date due to more updates to the model.
Golden Knights winger Max Pacioretty is our first examplar. As you can see, scored 9.4 in the EVO category, 4.6 in the EVD category, 2.3 in the PPO category, -0.3 in the A3Z category, -0.7 in the Take category, and 0.5 in the Draw category to total a Goals Above Replacement of 15.8. According to my model, he is worth 15.8 “goals” above a replacement-level player. The bar charts on the right show exactly which areas Pacioretty achieved his GAR in each category from, mostly for the purpose of predicting future success. The supplementary charts are essentially an alternative to creating and xGAR model to complement this one: If we know how the player’s GAR total came to be, then we should theoretically be able to estimate if his success (or lack thereof) is sustainable.
Here’s the thing about Max Pacioretty, and this should magnify just how great of a comeback season he had in 2019-20: He was a positive, or above replacement-level, in every single macro-level statistic that makes up my GAR model. His EVO score of 9.4 was the fourth-highest among NHL forwards in 2019-20, and looking at his EVO chart on the right, we can see that he generated a lot of shots, and a lot of quality ones too. He saw results on par with the expected outcome of his shot volume, which means his finishing ability wasn’t that high. Essentially what this means is, Pacioretty’s offensive results this season weren’t uncharacteristic for someone who generated as many shots of quality as he did. If he was better at finishing on his chances, he’d likely have seen Artemi-Panarin-level results this season, which is quite crazy when you think about it. Could you imagine how much more recognition Pacioretty would be getting for his performance this year if he had Draisaitl’s, or Laine’s, or Pastrnak’s finishing ability? He’s already regarded as a good sniper to begin with.
Pacioretty was a positive contributor on defense as well. He was better at limiting shot quantity as opposed to quality against this year, and on the powerplay, his impact was very similar to how he profiled at even-strength, which is a lot less common than you would think: Lots of shot attempts, and his results didn’t stray too far from what was expected. The one area in which he was a negative was in the neutral zone: He didn’t enter the offensive zone with possession very much, and profiled as below-average in terms of exiting the defensive zone too. Still, no NHL forward combined offense and defense better than Pacioretty in 2019-20.
As I said, though, the model is different for defensemen. So, let me show you that one, using the defensemen who led the NHL in hbGAR in 2019-20 as our examplar. That player is none other than Pacioretty’s Vegas teammate: Shea Theodore.
Anaheim is definitely kicking themselves for this one. Theodore is an offensive monster, and was on pace to break his career high in points before the shutdown. Defensively, though, he’s barely above replacement level, and he owes all of his defensive ability to preventing a high quantity of shots to go against the Golden Knights. The shots that do get by Theodore are of high quality. On the powerplay, he’s quite the force, and he’s very good in the neutral zone too.
As I previously stated, I don’t go as in-depth into a defenseman’s shooting talent at even-strength, and I include unblocked shot attempts against in the defense section in its place (not in RAPM form, like the other two stats that make up a player’s EVD). This is simply because to me, having a great shot isn’t as important for defensemen as actual defense is. I’m not here to tell you what is and what isn’t important when it comes to evaluating players, though. When creating visuals and statistics, deciding which stats to use is entirely up to the creator; and thus, that process is more-or-less entirely subjective.
So, how does my model differentiate from Evolving-Hockey’s model? Well, their GAR model focuses more on goals for and goals against when it comes to offense and defense, and their xGAR model, as you probably could have inferred, focuses more on expected goals for and expected goals against, or quality shot rates instead. Seeing as my model includes a bit of both, let’s do a comparison between the two models, by looking at Max Pacioretty and Shea Theodore’s hbGAR totals by category, and looking at how they’re evaluated under Evolving-Hockey’s GAR and xGAR models:
Max Pacioretty, hbGAR: 9.4 EVO, 4.6 EVD, 2.3 PPO, 0 SHD, -0.7 Take, 0.5 Draw
Max Pacioretty, E-H’s GAR/xGAR averages: 9.6 EVO, 2.2 EVD, 2.2 PPO, 0 SHD, 0.5 Take, -0.7 Draw
Shea Theodore, hbGAR: 8.2 EVO, 0.4 EVD, 1.9 PPO, 0 SHD, 0.1 Take, -0.4 Draw
Shea Theodore, E-H’s GAR/xGAR averages: 7.5 EVO, 1.1 EVD, 0.9 PPO, 0 SHD, 1.5 Take, -0.6 Draw
So, we can see here that the models aren’t very different from each other at even-strength and on special teams, but they give quite different figures when it comes to penalty differential. I don’t know the specifics behind Evolving-Hockey’s model, but my model only takes into account the amount of penalties a player takes and draws into the penalty side of things - nothing else. Still, there wasn’t a difference of more than 1 GAR between my model and their models’ interpretation of EVO, EVD, PPO, and SHD for either player outside of Pacioretty’s defense. Neither player was used on the penalty kill this year, hence the blank slate for them in that category.
The Spicy-Hot Takes
Now that the information-heavy part of this is done (reach out to my Twitter if you have any further questions), let’s have some fun. Here’s how Patrick Kane stacks up in my GAR model.
I can hear the traditionalists calling for my head on a stake already, so here’s why Patrick Kane shows up very poorly in my model, in as small of a nutshell as I can pack it before the entire hockey sub-Reddit abducts me and takes me hostage: Kane has never been good at defense, but it was a lot more tolerable as recently as last season when he was good at offense. However, he has dipped to below-replacement-level in terms of shot rates at even-strength, with only good results and a good finishing ability keeping him as a serviceable offensive player. To be clear, I said a few sentences ago that he was never good at defense, but he has now become the worst defensive forward in the NHL and it isn’t even close. Nobody allowed more quality shot attempts to come against their team than Kane, and as you can see, he’s almost fallen off the chart entirely in that regard. He’s still a weapon on the powerplay, however, and remains one of the greatest neutral-zone talents of the generation. I’m not going to say Kane is a bad hockey player because of one visual, and it’s up to you as to how you want to value offense, defense, and transition when it comes to player evaluation, but hockey is a two-way game and there’s no getting around that. Kane’s inability to play defense is going to be a lot harder for the Blackhawks to tolerate if his offense keeps slipping like it did this season. The averages of Kane’s GAR and xGAR, per Evolving-Hockey, are as follows: 8.6 EVO, -6.2 EVD, 1.5 PPO, 0 SHD, 1 Take, 0.2 Draw.
Next up: Connor McDavid.
You may notice that McDavid is worse defensively than he is good offensively. That’s right: In terms of offense and defense combined, Connor McDavid was a “negative” at even-strength this season. He wasn’t always this way - between 2016 and 2019, Connor McDavid was making a case for himself as the best offensive player of the 21st century. However, he dropped to below replacement-level in terms of shot quantity at 5v5, but generated a ton of quality, and the product of that is a ton of results. His FSh% was actually lower than his xFsH%, so results-wise, he actually didn’t meet his full potential there. Defensively, he still has a long way to go, but he had the second-highest PPO hbGAR in the league, behind only Leon Draisaitl. As you probably assumed, he’s a transition god.
So, we know that Max Pacioretty had the highest total even-strength hbGAR at even-strength of any NHL forward this season. In terms of EVO and EVD combined, no forward was better. But, who led defensemen in that regard?
You love to see it.
I’ve been on the Fox-for-Calder train for a while now, so this certainly helps my case. He’s an offensive stud and he’s pretty good defensively too, better than his rookie counterparts. He can do it all offensively at even-strength, as a matter of fact, and doesn’t stray too far from average on the powerplay. Defensively, his strength is preventing a high quantity of shots to go against his team (why the Rangers didn’t give him PK time is beyond me), and he’s a force in transition as well. Fox ranked 3rd among NHL defensemen in EVO, and 13th in EVD.
Overall, Goals Above Replacement provides a very different way of looking at hockey. Catch-all statistics are sure to not discount the fact that hockey is a two-way game, and that defense is important, but like all statistical models out there, this one is far from perfect, and there is a human element to hockey that none of us can logistically quantify. I hope you decide to take my interpretation of GAR into consideration when evaluating players in the future (and that you support Adam Fox in the Calder race), and you consider supporting Evolving-Hockey so you can see their own GAR model, and the numerous statistics that went into my own.
Again, my DMs are open on my Twitter @HBAnalytica if you have any further questions about this stat.