Evaluating Closers- Part 1

Tuesday, January 23 2007 @ 07:12 AM EST

Contributed by: Mike Green

The recent induction into the Hall of Fame of Dennis Eckersley and Bruce Sutter, and the rising vote totals for Goose Gossage, have me thinking about standards for greatness in a closer.

Isn't it enough that a closer is among the very best at his craft? No. The problem is role. Not all roles are created equal. Pinch-hitters might come to bat 70 times in a season, but can play a decisive role in the game. Strangely, here is no hue and cry to put Manny Mota or Smoky Burgess in the Hall of Fame. Obviously, the closer plays a decisive role in the game more frequently than a pinch-hitter, and has more influence than a pinch-hitter by far on the results of games over the course of a season. It is possible that closer effectiveness could be so important to game results that they could be considered to be among the greatest players to ever play the game, but it is not immediately obvious that it is so. How then do we measure the contribution of a closer compared with other pitchers?


A closer's contribution is potentially different from a starter's in the following ways:

Reliever vs. Starter Effectiveness

When I looked at John Smoltz last February, I pointed out that both he and Eckersley were both much more effective when working as closers than as starters. In July, Steve Treder of the Hardball Times took a thorough approach, looking at how pitchers who had performed in starting and relief roles for a minimum of 75 appearances after 1957 had fared in each. He found that they performed much better as relievers- 14% fewer homers, 15% more strikeouts, 7% more unintentional walks and 8% lower ERAs. For closers, the data was even more extreme- 18% fewer homers, 13% more strikeouts, 8% fewer walks and 16% lower ERAs. Treder's conclusion is that we ought to apply an effectiveness equalizer of 5 to 15% to rate stats including ERA when comparing relievers with starters. I agree with this conclusion especially in relation to closers (for whom the improvement was across the board), and will apply an equalizer of 10% for comparison purposes. This is a conservative estimate of the effect.

As an aside, why is it that closers improve on their performance in the "unintentional walk" department as compared with their work as starters, whereas other relievers deteriorate in this respect? I would suggest one possible answer. Non-closer relievers enter the game more often with runner(s) on and first base open, making the "pitch-around" walk a more common element of their games.

Leverage

We do know that closers come into games at different stages of the game, depending on the era and depending on the manager. We have easily accessible statistics for 2002-06, courtesy of fangraphs.com. Here is the card for Mariano Rivera. The pLI is the average leverage of opponent's plate appearance and ranges from 1.82 to 2.23 over the years with an average of 1.99. In Joe Nathan's time with the Twins, his pLI has ranged between 1.62 and 1.92. Baseball Prospectus uses a somewhat different definition of leverage, and has slightly different numbers. For instance, BP has Joe Nathan's leverage index in 2004 as 2.06. The average fangraphs leverage for closers in 2006 and 2005 was 1.96 and 1.93 respectively; using the BP definition, it ws 1.86 and 1.83. For comparison, starters typically have a leverage index of about 1.

To put it together, have a look at John Smoltz' fangraphs card. While working as a closer during 2002-04, his average leverage was 1.91; when he returned to the starting role in 2005-06, his average leverage was 1.05. If we assess closers as having double the leverage of starters in general, we are being generous to them.

Chaining

The loss of a starter impacts a team differently than the loss of a closer. If a starter is lost, the team must simply replace the starter. If a closer is lost, the replacement pitcher will not be placed in the closing role. This means that the loss of performance cannot simply be measured by the difference in performance between the closer and the replacement multiplied by the leverage. It is time for an example. Rather than choosing potential Hall of Famers, I thought that it might be better to stay closer to home, by taking a closer look at Tom Henke and Jimmy Key , during their primes in Toronto from 1985-1992. Besides, I like both of them. Cue Glory Days, Mr. D.J.

The 1985 club provides a good illustration of chaining. The pen that year, aside from Henke, consisted of Bill Caudill, Gary Lavelle, Jim Acker and Dennis Lamp. Caudill had struggled in the closing role, when Henke came up, but was a pretty good pitcher as were Lavelle, Acker and Lamp. Henke's leverage index, according to Baseball Prospectus in 1985 was 1.57. If Henke had been lost, Lavelle or Caudill would have reassumed the closing role, the other 3 pitchers would have moved up, and Ron Musselman or Steve Davis or a young John Cerutti would have moved into the low leverage mop-up role. The loss of Henke would have been equal to the difference between Henke and Lavelle say (large, but Lavelle was substantially above replacement level) at a leverage of 1.57, and the difference between Dennis Lamp and Ron Musselman at a lower leverage (Lamp's leverage was 1.15 that year).

So, when we assess what Henke's value, we cannot simply multiply the difference between his performance and replacement performance by his leverage. The actual difference will depend on the remaining members of the pen, but squaring things off at between 75% and 85% of leverage seems to me to be fair.

The situation is different for starters. Roy Halladay and Gustavo Chacin have essentially the same leverage when they pitch (as starters). The same was true for Jimmy Key, Jim Clancy and Luis Leal. If Key was lost in 1985, it would have meant more starts for Luis Leal, or some other replacement level pitcher.

Post-season and the wonder that is Mariano Rivera

Bob Gibson is considered by many to be the greatest post-season pitcher ever. He was superb indeed, making 9 starts and throwing 8 complete games and 81 innings, while posting a sweet 7-2 record with a 1.89 ERA. Still, it's only 81 innings in a career of almost 4000 regular season innings. Sandy Koufax was spectatcular in the post-season with a 0.95 ERA in 57 innings, but still, 57 innings in a career of 2300 innings doesn't amount to much. Of course, with the advent of the multiple playoff rounds, starters are throwing more innings in the playoffs, Greg Maddux and Roger Clemens have each thrown 200 post-season innings in careers of over 4500 innings.

And then there's Mariano Rivera. With 889 regular season innings and 112 post-season innings (and a 0.80 ERA) on his resume, one simply cannot ignore the importance of the post-season in evaluating him. With closers, there is not only "game leverage", there is "season leverage". The object is to help one's team win a World Series. If a team is mathematically eliminated or has clinched a division title, and the closer comes on, the leverage may be high vis a vis the game, but it is nil for the purpose of the season. All post-season games have high "season leverage". In this respect, Rivera's 112 post-season innings are not only a significant proportion of his workload, but also disportionately important.

Rivera is, at this point, one of a kind. In the future, though, it does seem likely that post-season performance will be more important in evaluating closers than it is for other players.

A sample reliever's adjusted line

Shall we go back to Tom Henke again. For variety, we will look at his 1986 line, and compare it with Jimmy Key's:

Pitcher IP W W/9 K K/9 HR HR/9 ERA LEV.
Henke-act. 91.3 32 3.15 118 11.63 6 0.59 3.35 1.57 (1.33 adjusted)
Henke-adj. 91.3 35 3.45 107 10.55 7 0.69 3.69 1.57 (1.33 adjusted)
Key 232.0 74 2.87 141 5.47 24 0.93 3.57 1.00(assumed)

I have made the 10% adjustment to Henke's walk, K, HR and ERA totals, as described, and I am using 85% of his leverage in recognition of the strength of the pen overall and Henke's modest leverage. Even with the adjustments, Henke was a better pitcher during the season overall, save for the ERA, but over less than half the innings. Let's see how they fare over their career.

Pitcher IP W W/9 K K/9 HR HR/9 ERA LEV.
Henke-act. 789.7 255 2.91 861 9.81 64 0.73 2.67 1.44 (1.22 adjusted)
Henke-adj. 789.7 281 3.20 775 8.83 70 0.80 2.93 1.44 (1.22 adjusted)
Key 2591.7 668 2.32 1538 5.34 254 0.88 3.51 1.00 (assumed)

Henke was clearly the better pitcher, even with the adjustments. over his career. But, he pitched 1/3 the innings that Key did, and even accounting for the increased leverage (which ought to be adjusted to account for the chaining impact), it is challenging to argue that Henke contributed more than Key did. Henke's ERA+ falls using the adjustments from 156 to 141 (Key's was 122), and applying the leverage factor to his innings results in adjusted innings of about 1,000.  In other words, Henke's career in total had less value than the first 6 years of Dwight Gooden's.

In Part 2, we will compare Tom Henke with the relievers currently inducted in the Hall of Fame, using these measures.

12 comments


https://www.battersbox.ca/article.php?story=20070117201223826