Some more scores:

Shawn Chacon, Colorado, 52.0 IP, 42 R, 5 ISt = -44 RP
Rocky Biddle, Montreal (as reliever only) 35.1 IP, 37 R, 3 Ist = -73 RP
Terry Adams, Tor/Bos 49.0 IP, 25 R, 16 Ist = 54 RP
Mike Nakamura, Tor 25.2 IP, 23 R, 10 ISt = -18 RP

Great idea, Kieran! This is definitely a large hole in current stat lines, a good way to measure relievers. A big part of the problem is that Inherited Runners and Inherited Runners Scored are not commonly reported.

RP as applied to current bullpen usage could make set-up men look more valuable than closers, as setup men face more inherited runners than closers do. This is a good thing, as optimal bullpen usage has the best pitcher pitching in the highest leverage situations, and a situation involving inherited runners is usually high-leverage. However, you may want to add a penalty for inherited runners scored, or you could end up giving to much credit to a mediocre setup man on a bad team. For lack of better options, this guy is called on to deal with a lot of inherited runners, and the current formula will give him credit for that even if he's stranding them at a sub-par rate. It's probably as simple as adding Inherited Runners Scored to Runs Allowed.

A big part of the problem is that Inherited Runners and Inherited Runners Scored are not commonly reported

Thankfully, they are part of the ESPN extended statlines in the player reports, which is where I get my data from. I agree that the fact that IR/IS/ISt (ISt is my acronym for Inherited Runners Stranded) are not generally reported makes this more difficult.

Let me add that I'm very open to other suggestions for the weighting numbers; mine are relatively simple and easy but there are other good possibilities, I'm sure.

http://www.baseballprospectus.com/statistics/rrereport04.html?PHPSESSID=bf473b00d63360e78d90a810cb8ef835
This wouldn't fall in 'easy and simple to calculate' category, but BP ranks relievers by adjusted runs prevented (COMN, stats as of 8/6) which I believe incorporates all the relieving factors one would be interested in.

Pistol, ARP is fine, but as you say it isn't simple to calculate; you'd need a spreadsheet the size of a... well, I don't know. A big one. :)

Also, unlike RP it isn't scaleable down to a game or a week or what have you. RP are like Game Scores, it actually works well on a game-by-game basis.

I agree with Craig, in that Relief Points is meant to be a relatively simple-to-calculate metric. The analogy to game scores is a good one, in that RPs can be measured on a game-by-game basis.

There certainly is a weighting towards set-up men who strand runners, but then again, shouldn't there be? Isn't this proof that managers are not using their best relievers at the most appropriate time? Perhaps if something like RP caught on, managers would once again be influenced by a stat.

Re: the weightings, Craig chose a 5.40 ERA as the "average". For a second, I thought 4.50 might be closer to the truth, but this will likely cause too many relievers to end up with negative RP scores, so 5.40 might be better for optics.

I think you need another counting stat: something about runners left for the next pitcher.

Example:

Pitcher A: comes into the game at the start of the 8th inning. Strikes out two guys and then leaves for a LOOGY.
Relief Points: 2 (for the outs)

Pitcher B: comes into the game at the start of the 8th inning.
Strikes out two guys, but then walks one and allows a single. Leaves for a LOOGY with two men on base.
Relief Points: 2 (for the outs)

But obviously Pitcher B was not as effective as Pitcher A, but would get the same RP score.

Graham...good point...I suppose there is a chance the runners left on base could be captured if the next reliever allowed them to score...but really, that is reflective of his teammate rather than his ability.

So, what do you propose?

Are runners left on base easily found on ESPN.com or another stat service?

A fantastic idea. Great work Kieran. It may be wishful thinking, but how cool would it be to introduce an original (not to mention useful) Batters Box statistic into the world?

If a stranded runner is worth two points, then is an 'allowed' runner worth negative two?

I would think they should be worth something... the pitcher is introducing risk into the game.

Following up on Jonny's idea, I would subtract two points for every inherited runner that scores in addition to the original formula. I would also consider giving a little extra weight to recording outs in high leverage situations. Say, 1 point per out recorded, 2 points per high leverage out recorded. Generally, these situational outs would be easy to calculate; determine the number of IP during Holds and Saves, though it would be more accurate if Holds weren't dependant on Saves. A set-up guy that holds a slim lead should be given credit even if the closer blows it.

Thinking about it some more, there are just too many problems with Holds to use them. A guy that pitches a scoreless eighth and gets the win gets no credit, etc... I don't guess there's an easy way to distinguish high leverage outs, so might as well scrap that part of my formula until we get some better stats.

Kieran, a 5.40 RA (remember, we're talking runs allowed, not earned runs) would equal about a 4.90 ERA.

I like Graham's way of explaining things, so I'll use the same method to extend miVulgar's thoughts:

Pitcher A: Inherits 1 runner. Allows triple, then strikes out three.
Result: 3 points (3 outs)

Pitcher B: Inherits 1 runner. Strikes out three.
Result: 3 points (3 outs)

In this sense, the inherited runner that scored is equally as damaging as the pitcher allowing a run to score on his own behalf (which would result in -5 points). So would this RP equation make sense?:

RP = 3*IP - 5*(R + IS) + 2*(IR - IS)
   = 3*IP - 7*IS - 5*R + 2*IR

Test cases:

Marte: 57.2 IP, 31 IR, 8 IS, 21 R, 74 RP
Hasegawa: 52.1 IP, 39 IR, 11 IS, 38 R, -51 RP

A set-up guy that holds a slim lead should be given credit even if the closer blows it.

They do get credit. For example, last night Ayala got a hold, but Cordero got a blown save after relieving Ayala.

Does runs allowed count only runs the pitcher gives up himself, or does that include inherited runners?

For example, if a pitcher comes into the game with a runner on 2nd and gives up a HR is that 1 or 2 runs allowed?

Nice idea. Finally, a stat with my initials.

I looked up the Jays' current RP stats, using IP*3 - R*5 + ISt*2, as well as Brent's formula, which confused me to no end until I realized that 2*(IR - IS) is the same as 2*ISt. I know, I know, stop laughing and pointing at me.

Here they are, with Craig's weighting on the left and Brent's on the right.

Name        RP(CB)  RP(BS)   
Frasor      111     76         
Ligtenburg   11     59
Chulk        33      3
Speier       62     12
x-Adams      18     54      
Peterson     17     32

I calculated Ligtenburg's three times, because I couldn't believe it was so bad. I did those all by hand, so please point out any errors.
Adams is the only pitcher to improve one IR are taken into effect. Interesting.

Pistol: Under Brent's formula, it's minus 5 for the batter who hit the homerun, and minus 7 for the inherited runner on base who scored.

So it counts both runs, but not equally.

Rob,

The weight of the -59 RP(b) score was caused by the fact that Ligtenberg has allowed 48% of his inherited runners to score.

Plus, the runs. Those hurt too. ;)

Gents,

This is exactly what I was hoping for...my fellow Bauxites running with the idea and flushing it out.

Hopefully we can nail down the weightings and begin using RPs!

Actually, one thing before lunch:

Adams' points actually decreased from the original formula to the new one. Adam's RP(b) = -31.

Yeah, I figured that was wrong. Adams didn't seem that good, after all.

You know, Relief Points is a good name for the stat, but we need a name for the award given each year (well, each year from now on) to the best reliever based on RP.

Reliever of the Year Award?
Kieran Roy Award?
Or name it after the pitcher who had the best season in MLB history, according to RPs.
Or just call it the Kerry Ligtenburg Award, as a joke.

You know, Relief Points is a good name for the stat, but we need a name for the award given each year (well, each year from now on) to the best reliever based on RP.

It'd be pretty cool if it was the R.I.P. award.

Reliever's Individual Points?

Is there a difference in performance here?

Pitcher A: starts inning, gives up a walk, strikes out 3 and ends the inning with one man on base.

Pitcher B: starts inning, gives up a walk, gets two "fielder's choice" plays and line drive out and ends the inning with one man on base.

Macro result for both pitchers:
3 outs
1 walk
1 man left on base (but should count as zero since the next pitcher wouldn't have to worry about him)

Micro result for both pitchers:
Pitcher A: never a threat to give up a run, and defence made no difference on whether one would score
Pitcher B: defence relied on heavily to stop run from scoring

I guess if we are looking for a quick and dirty single-game reliever score, it shouldn't make a difference. But over the long haul, I think Pitcher A should get more credit as a "better" reliever.

But over the long haul, I think Pitcher A should get more credit as a "better" reliever.

I see RP as the rear view, looking to measure past performance, and in that sense they're equal. Pitcher A was more dominant, but Pitcher B was equally effective. If Pitcher A is truly better, it will show up in future performances, when Pitcher B is betrayed by his defence. Or perhaps it'll turn out that Pitcher B is better, as Pitcher A tires out from the effort and pitch count involved in striking batters out. If I'm a GM deciding between two free agents with similar RP scores, I look at things like K/BB, K/9, G/F to decide which one is more likely to have future success.

Is there a difference in performance here?

Pitcher A: starts inning, gives up a walk, strikes out 3 and ends the inning with one man on base.

Pitcher B: starts inning, gives up a walk, gets two "fielder's choice" plays and line drive out and ends the inning with one man on base.

Graham, I have to agree with Jonny here. We're not trying to evaluate how good a players is; this is a performance metric, and those two performances are equal.

Want to see something weird?

Mota: 74.1 IP, 16 R, 19 IR, 4 IS
RP(b): 153

Gagne: 61 IP, 20 R, 15 IR, 0 IS
RP(b): 113

Interesting.

Graham, I have to agree with Jonny here. We're not trying to evaluate how good a players is; this is a performance metric, and those two performances are equal.

Okay. Now I understand.

You know I say that, but now I'm not so sure. The Game Scores system gives double points for strikeouts. Should we do the same here? I still think probably not, but it's an interesting question. Should the pitcher get extra credit for doing the whole job himself?

But the assumption you're making, if you give extra credit for strikeouts, is that the pitcher intends to strike the batter out. That may sound obvious, but it's not always the case. Take a guy like Roy Halladay - his game plan is quick resolutions, not strikeouts. Or, generally, take a double-play situation; do you give the pitcher extra credit for inducing a ground ball, or is it too much to assume that he intended to induce a ground ball and that his fielders didn't have to do anything spectacular to turn it?

Not quite. You don't have to reward a pitcher for only doing what he intends to do; if a batter intends to move the runner over with a grounder the other way, but instead hits a flyball that gets caught up in the updraft and clears the fence, we still give him credit for the home run. (OK, not the best analogy!)

What you'd be giving him credit for, is either (1) the dominance of his performance; or (2) credit for accomplishing everythig himself without reliance on his defense - who should get some of the credit for recording non-K outs.

I think I prefer the simpler method of no "K bonus", because I'm not convinced by these reasons. But they should be considered.

I'm not convinced either about K bonuses, but I think there should be some kind of Penalty to the relief pitcher incurred from amazing defense that possibly saves run(s).

Also if a Releiver comes on w/ men on base and gets the batter to ground into a DP (getting 2 outs but facing one batter), he should get some credit for that too if you are going to hand out credit for Ks.

Getting the Perfect groundball can be just as hard as it is to get a strikeout too.

Great article! My only suggestion is that you might want to adjust for the situation, i.e. the inning and the score.

I have toyed with the idea of Relief Points in the past, but haven't gotten very far with it. Keeping it simple and assigning the correct weights have been sticklers.

My idea was to assign a certain number of points for every out recorded. When the pitcher enters the game, the point total is predetermined by a formula that combines the number of outs in the inning, the runners on base, the score and the outs remaining in the game.

Regardless of what happens in the inning, the pitcher gets the same number of points for each out; he doesn't get extra points for putting men on base. Of course, he also loses points for allowing runs to score, either inherited or his own.

The four factors would then be

1) - number of outs in the inning. He gets more points if he starts the inning, rather than coming in with two outs.

2) - number of inherited runners. The more he inherits, the greater number of points he gets for each out that inning.

3) - the score. The closer the game, the more points he gets.

4) - the number of batting outs left that his own team has. If he is pitching in the top of 8th, his team has 6 batting outs left; if it's the bottom of the 9th, they have zero.

So the formula would be something like

(Outs remaining in inning) * X

plus

(Runner on 3rd) * A + (Runner on 2nd) * B + (Runner on 3rd) * C

minus

(Remaining batting outs) * Y

minus

(Difference in score) * Z

After the inning is complete, we subtract:

(Inherited runners scored * M + Own runners scored * N)

If Jason Frasor enters the top of the ninth with a two run lead tonight, and allows 1 run, then it would be...

(3x + 0 - 3y - 2z)

...Relief Points for each out recorded in the inning, minus n for allowing his own run to score.

If Vinnie Chulk enters the top of the 8th with two out, a 1-run lead and a runner on third, and strands the runner, then he gets...

(x + a - 6y - z)

...Relief Points for the one out he records, with no extra demerit.

Like I said, I have no idea what the weights should be, and it's on the verge of being too complicated for public consumption. I'm also not sure if extra points should be given for working with a lead instead of a deficit. But it's a framework for a possible system.

Catching up on the thread...

1) Re: P. Smith's comments - I think there is validity to weighting situations, but I just don't think it's practical. We want the stat to be as easy to calculate as possible if it's going to catch on

2) Re: naming of the award. My narcisistc self likes the idea of the Kieran Roy Award for best relief pitcher...but I'd be happy with RIP, too.

3) Have we arrived at a final weighting/formula? Which is best:

IP*3 - R*5 + ISt*2
or
RP = 3*IP - 5*(R + IS) + 2*(IR - IS)
= 3*IP - 7*IS - 5*R + 2*IR

I think I prefer the simpler method of no "K bonus", because I'm not convinced by these reasons. But they should be considered.

I am in this boat too. I was also tossing around the idea of adding in a way to account for not only if a relief pitcher left a runner on base, but where he left him on base. Because leaving a runner on 1st or a runner on 3rd has different consequences. However, this is both tough to measure and account for, and thus should be left out of our little system.

Simplistic is the key word.