Nobody’s been that glad to get out of Denver since Neal Cassady.

I had planned to give Jays fans a nice scouting report of some Rockies players and a recap of how Toronto hitters looked at Coors Field with today’s series report. However, the Sportsnet Gods conspired against me by not showing this series on Sportsnet and left me unable to watch any of the three games. Subsequently, I only got to listen to portions of the three games on the radio, including only the last three innings of yesterday’s match. All in all, perhaps I should be thankful for that considering the way Toronto played in Denver.

So, let’s turn our attention to win probability added (WPA), which is being tracked daily this season by the wonderful Fangraphs website. Let’s look at what WPA has to tell us about the Blue Jays bullpen this season.

Before we begin, you should familiarise yourself with WPA. WPA is like any other statistic. It has its uses and its limitations. Just as one wouldn’t to use saves to determine which of two ballplayers is likely to be better the next season (unless your information was extremely limited), one wouldn’t want to use WPA to predict that, either. Dave Studeman is a leading proponent of the statistic and has published several articles over at The Hardball Times on WPA and its uses. Particularly illuminating for the beginner is this primer that Studeman wrote in 2004. As he stresses, WPA has its uses and its shortcomings. It is not a predicative tool. Just because Vinnie Chulk is the worst pitcher on the Jays right now, it doesn’t mean he is necessarily likely to be the worst pitcher over the course of the season.

However, one area WPA is very illuminating is in evaluating bullpen effectiveness and strategy. As Studeman writes, “Using the [Win Expectancy] Finder again, if a pitcher gives up a bases-empty home run in the first inning of a tie game, his team's Win Probability decreases about 10%. If he does the same thing in the eighth inning, it decreases about 25%, because his team has less time to come back. In this context, the eighth inning is about 2.5 times more important than the first inning. And if you apply this sort of analysis to every appearance made by a relief pitcher, you can quantify the importance of all of his innings pitched.”

After the ballgame is over it doesn’t really matter when the team scored a run, because the run counts the same. However, during the game a run in the second inning is very different from a run in the eighth inning. Think about how differently yesterday’s game would have been managed if Rios had not hit his 3-run homer in the fifth inning, but instead had hit it with two outs in the ninth inning. Not only would the Jays had only one out to work with when they needed two runs, but it would have affected how both coaching staffs managed the game and we likely would have seen different pitchers on the mound, different strategies employed and so forth.

WPA can work well as a measure of a reliever’s effectiveness because many traditional statistics, such as ERA, are inadequate as tools of evaluation. Relievers pitch so few innings that a disastrous inning can have a disproportion effect on their ERA. I care little if Scott Downs allows 6 runs in a game Toronto is losing 8-2 to the Indians, but this could raise his ERA by a whole run at the end of the year. Similarly, ERA doesn’t reflect inherited runners at all, which is one of a reliever’s most important duties. A reliever with a 2.40 ERA who allows 75% of inherited runners to score isn’t doing a very good job, despite his low ERA. Just like Leverage Index, WPA reflects the fact that some relieves pitch when the game is on the line, and their statistics can’t be treated identically to those of starters. WPA allows analysts (including amateurs like myself) to actually quantify what a reliever contributed (or deducted) to a team’s chance of winning the game.

(Note: These stats are as of before Sunday’s game.)

This table reveals a lot of information that we knew or could have guessed, but it also reveals some interesting statistics. To begin with, one number dwarves all the others, and that’s BJ Ryan’s 136.9. That number is the largest positive contribution on this year’s Jays team, although Josh Towers’ -178.3 is the biggest absolute number. It’s no small feat that Ryan has, according to WPA, won Toronto one and one-third games by himself so far this season. Ryan also has the very impressive statistic of being the only Jay to have not had an appearance where he posted a negative WPA score. That is, every game he’s pitched in he has contributed positively to Toronto’s chances of winning the game. He’s been the third-best reliever in the AL so far this season, behind only Jonathan Papelbon (191.8) and Chris Ray (147.0). Even with Mariano Rivera’s struggles so far this season, a team doesn’t want to enter the ninth inning losing to a team in the AL East. The odds for a comeback are slim, at best.

Now, that’s not a surprise. BJ Ryan is a beast. However, some of you may be wondering what the story is with Brian Tallet? How can a 9.00 ERA in 7 innings have made him fourth-best in the Jays bullpen in WPA? Simply put, in the only game he pitched where it really mattered he did well. Tallet had three positive appearances and three negative ones. To start with the best, on April 8th Tallet pitched 2.2 scoreless innings against Tampa Bay in an 8-4 win. He gained +15.7 for the outing, as Toronto came back to win after falling behind in a forgettable Ted Lilly start. Tallet got the win as Toronto rallied while he was pitching and Tallet worked very effectively, striking out three while allowing two hits. Tallet also added positive contribution on April 14th when he pitched a 2-run inning against the White Sox in a 13-7 win. He gained +0.1 for the appearance, because the odds of Toronto winning were so large that allowing 2 runs in one inning didn’t matter. On April 30th Tallet pitched a scoreless inning against the Yankees in a 4-1 loss. He gained +0.7 for the appearance.

Tallet had three negative outings, but these also came in games where he was pitching very low-leverage innings. On April 5th Tallet pitched 1.1 two-run innings against Minnesota in an 13-4 win. He ended up with -1.5 for the outing. He also gave up two runs on April 13th, where he was only penalised -0.1. Toronto beat Boston 8-6, but Tallet entered the game in the top of the eighth inning, when it was 8-1. This was the game where Frasor gave up 3 in the ninth, before Ryan shut the door.

Tallet gave up a homer to Miguel Tejada on May 1 against the Orioles that scored two runs (one his, one Schoeneweis’s), but this was in the ninth inning of a game that the Jays were winning by four. The Jays held onto win by 2 and Tallet was penalised with only a -1.1. MGL spoke about a similar situation in a thread over at BTF yesterday, discussing Wagner’s Saturday implosion, as he argued against using Wagner in that situation, unless there was a specific purpose to pitching him that particular day. He argued that the Mets stood about a 98.1% chance of winning the game with a really bad pitcher pitching (say, Jose Lima), as compared to 99.2% with Wagner. I’m unsure if that takes into account the quality of the opposition, which WPA doesn’t, but it does demonstrate that this homer was relatively meaningless. Even if he had stayed in to finish it, Tallet would almost certainly have won the game. And thus concludes the story of how Brian Tallet has been the fourth-best reliever on the Jays so far this year.

So now the mystery of Brian Tallet is solved, onto other things. Justin Speier is second to B.J. Ryan, which is not a surprise, although I didn’t think that Jason Frasor would be this close to him. This is a reflection of both the usage pattern of Speier and Frasor, as well as how they have pitched. Both are being used in high-leverage situations and, more often than not they succeed. Seeing Chulk with a negative score is also not surprising, as he’s been at the centre of a few collapses in high-leverage situations, which was bound to outweigh his positive contributions. But, let’s investigate the difference between Chulk and Frasor in more detail.

Frasor’s made 12 appearances, of which he’s given up runs in half of them. He allowed four runs in the second game of the season, the aforementioned 13-4 loss to the Twins. However, given the insignificance of those runs, he was only docked -3.7 for that game. In another previously discussed game, Frasor gave up 3 runs to the Red Sox following a strong Lilly start and the Jays held on to win 8-6. This one only cost him -0.9, as three runs in the ninth inning with a sizable lead costs you very little when you also get two outs at the same time. Frasor gave up 2 runs to the O’s on April 27th, when the Jays lost to Baltimore 8-6, which cost him -13.2. That was his most costly outing of the season, as he turned a 4-2 Baltimore lead into a 7-2 margin, by allowing an inherited runner and two of his own to score. Frasor gave up one run thrice, but actually ended up with positive scores in two of the games. Most of these outings, besides the April 7th and 27th cost him less then -4 points. Frasor’s had only one noticeable poor appearance.

Most of his positive appearances balance that out, with the highlights being 8.3 on April 8th and 8.2 on April 26th; the latter where he pitched the sixth and part of the seventh inning in a game Toronto was leading 3-1 at the time. His most recent outing is what accounts for nearly his entire positive score, as he received 35.3 points for his great job against Anaheim on Thursday evening. In the bottom of the ninth of a tied game Frasor managed to escape from a situation where a runner was on third with one out (the out having been recorded on the previous batter by Scott Downs) and managed to force a 10th inning. He then retired the side 1-2-3 in the 10th, although Toronto’s had a 4-run lead at that point.

That one game is currently the defining moment of Frasor’s season. So what performance defines Chulk’s 2006? Vinnie’s had four games where he’s amassed WPA scores higher than 10 or lower than -10. In his one positive performance he entered the April 14th contest against the White Sox in the sixth inning, with Toronto leading 7-5, and pitched three scoreless innings as the Jays added four runs. For those three innings he had a 10.2 WPA score, which was the highest score amongst Jays’ pitchers for that game.

Chulk’s three negative performances demonstrate some of the strengths of WPA. On April 5th Chulk received -10.7 in a game Toronto would eventually lose 13-4. The game was only 5-4 for the Twins when Chulk entered, although Tallet had left runners at first and third. Chulk allowed both runners and one of his own to score in the inning, and turned the game into one where Toronto had little chance of coming back. No other statistic can recognise the fact that Chulk allowed these runners to score quite like WPA, as even inherited runner percentages still can’t take into account outs and game situation. On April 23rd Chulk entered the game in the top of the eighth with the Red Sox leading 4-3, and promptly surrendered a walk and two singles. He left the now 5-3 ballgame with runners on first and second with none out and Ryan came in and didn’t allow Youkilis or Loretta to score. Again, WPA captures exactly how costly Chulk’s performance was, as Ryan really saved his stat line that game, but that does not ignore the fact that Chulk created this situation and only Ryan’s exemplary performance kept the Jays in the game.

In Chulk’s most costly performance – the one that may define his season so far – he was penalised -27.5 WPA on the April 7th game against Tampa. He was summoned by John Gibbons in the sixth to preserve the 6-3 lead and Scott Downs’s victory and he allowed singles to Johnny Gomes and Aubrey Huff before Ty Wigginton hit a three-run homer to tie the game. Although Chulk escaped the inning without any more runs, the damage was done. Pete Walker would allow a 2-run homer in the seventh to give Tampa a lead they would not relinquish. In fact, Chulk and Walker’s appearances in this game are tied for the most costly relief appearance by a Blue Jays reliever so far this season.

In conclusion, what we can learn from WPA is that Ryan is incredibly dominant; Tallet’s been the Jays fourth-best reliever; McGowan is incredibly costly on a WPA/inning basis; Frasor’s ERA is mostly the result of a few bad performances in games that really didn’t matter much and that Chulk’s poor outings are much more costly.

Now, this doesn’t mean very much going forward, as one good performance by McGowan next time he’s in Toronto could erase his whole deficit. What is likely to continue is that Ryan, Speier and Frasor will continue to get the high-leverage innings, with Schoeneweis facing a left-handed batter or two. Not only have the three been used that way so far this year, but they’ve performed well and have given Gibbons no reason to change his usage of the bullpen so far. Gibbons has employed some “unorthodox” usage of Ryan, which I support as long as he continues to be careful about it. However, as progressive as Gibbons may be with Ryan, he’s still cognisant of the save statistic, as the chart below demonstrates. Nevertheless, this may be the best approach, if it’s what makes Ryan the most comfortable, as long as Ryan is willing to pitch more than one inning on some occasions and as long as Speier and Frasor continue to pitch well in the middle innings.

Stay tuned until the next WPA bullpen update….

Now, for your reading pleasure:

Top 5 Starts

1. Casey Janssen, 48.7, vs. Angels, May 17
2. Casey Janssen, 46.5, vs. Angels, May 7
3. Roy Halladay, 36.9, vs. Angels, May 8
4. Josh Towers, 33.1, vs. Devil Rays, May 14
5. Roy Halladay, 31.7, vs. Yankees, April 28

Top 5 Relief Appearances

1. Jason Frasor, 35.3, vs. Angels, May 18
2. Justin Speier, 29.8, vs. Red Sox, April 21
3. BJ Ryan, 29.8, vs. Red Sox, April 21
4. Justin Speier, 27.0, vs. A’s, May 10
5. BJ Ryan, 21.9, vs. Red Sox, May 3