Superb stuff.

I remember seeing something similar Keith Woolner wrote when he was supporting EqR.

The bottom line is that most of these formulas agive results that are close to each other. Some want to debate endlessly about which is the best. However for most of us, any one of them will give us good enough information. Because OPS is so close to mainstream now, being published in some statistical tables, it will be the most used in the short term.

Gerry,

The fact that almost all of these formulas will work for team-seasons about equally well does not mean that they will work equally well when they are used to estimated how many runs Albert Pujols or Barry Bonds creates. But that's how all of them are used.

Bill James uses Runs Created to estimate how many runs each player on the team creates. Clay Davenport uses EqR to do the same thing, etc. These formulas do not possess equal logical soundness and not all of them work well in contexts other than team-season run totals.

Tangotigre's excellent series on how runs are really scored points to conceptual problems some of the formulas suffer from, and shows how they break down in some conditions.

In terms of the "multiplicative" equations, you have BaseRuns which
- models reality the best
- is as precise, and according to Patriot, more precise than any other equation
- as easy to use as the others

In terms of "linear" equations, anything out there is simply an offshoot of Linear Weights.

So, the debate is between using BaseRuns in some form (like the "theoretical team concept"), or Linear Weights.

EqR, RC, and everything else doesn't enter the picture. The only reason that they exist is because they have existed. Baseball Prospectus and Bill James do a disservice by continuing their use, and justifying their use because "well, what's the point of getting that last little piece of accuracy?". (That quote is my paraphrase of James who talked about when having a pencil, do you really need a better pencil?)

The debate would end if those who support their equations would hold them up to the 3 standards that we probably all look for: accuracy of model, precision in results, and ease of use.

http://economics.about.com
What I'd find *really* helpful when looking at all the different systems is the actual results they generate. I can go to Baseball Prospectus and see all the EqR stas. Where can I find any stats generated by the other metrics?

The one thing EqR has that (to the best of my knowledge) the other ones do not is up to today's date EqR stats, as well as historical EqR stats. If I want to find out how many "runs" Brad Wilkerson is worth, I'd much rather point my browser to a website rather than downloading raw data and fiddling with Excel. I suppose this goes to your "Ease Of Use" criterion.

Mike

As I mentioned, I agree that EqR exists because it exists. The "ease of use" is with the equation itself, and not its accessibility.

Heck, I look at Woolner's stuff just because it's there. And I look at the basic RC stuff at baseball-reference.com just because it's there. Again, these things exist because BP chooses to make it exist, and Sean Forman finds it easy to just give the most basic thing out there, rather than try to figure out what equations to use.

So, I *applaud* that these people are putting out these results. And as I've mentioned in the past, whether you look at these things or OPS+, you'll basically get what you want within say 3 or 5 or 8 runs or whatever.

What you *can't* do is then start using the results of these well-publicized but inferior methods and point to that as the basis for your conclusion (Bonds is better than Pujols cause RC says he's up by 3 runs, etc). And what you can't do is decide that EqR is "better", because it's not. Rob Neyer once said that Win Shares "works", whatever that means. Don't say it works, and don't say it's better, unless you can show it works, or you can show it's better.

The only thing you are left with is to use EqR and RC and OPS+ and Win Shares because the work has already been done for you, and you will get reasonably accurate, though not the most accurate, measures available.

All these measures are fine, within limits. It's when people go outside these limits that problems occur. The "debate" exists because of the poor presentation of the limits of these measures.

http://economics.about.com
So, I *applaud* that these people are putting out these results. And as I've mentioned in the past, whether you look at these things or OPS+, you'll basically get what you want within say 3 or 5 or 8 runs or whatever.

I suppose that's true. Any estimator is going to have a margin of error, though. At what point do you take two players run estimates and say "It's too close to tell"? How much different would that point be for BaseRuns than for EqR?

If these new estimators (and there seems to be more every year) that people come up with are supposed to be so good, then why do the authors of those metrics rarely generate results with those metrics? Are these new-fangled run estimators supposed to have any applied value, or are they just an intellectual curiousity?

Mike

And as I've mentioned in the past, whether you look at these things or OPS+, you'll basically get what you want within say 3 or 5 or 8 runs or whatever...What you *can't* do is then start using the results of these well-publicized but inferior methods and point to that as the basis for your conclusion

My own feeling is that after this point, once you get down to a pretty accurate reflection of player worth, the investment/reward ratio skyrockets. There's little value in picking out who was best with precision, aside from discussion fodder for websites like this one. If you need to absolute conclude that Player A's year was better than Player B's then by all means, go for it. That's really interesting to some people and to each his own. But is there a significant amount of value in that conclusion?

Where the true, applicable-to-life, value lies is in figuring out what combination of player skills, determined through statistical analysis, will generate x amount of runs. And then we can use past performance of individual players to best decide who will perform at what level in the future. Are more precise metrics of previous years going to help us see trends more clearly or able to more accurately forecast future worth? Yes, but only to a small degree. If I'm building a roto team or if I'm JP Ricciardi trying to build a major league roster, things like age, injury history, attitude, salary are going to have a much greater influence on my decision than a possible swing in the results of 1 or 2 runs generated.

http://www.baseballstuff.com/fraser/articles/basenew.html
At the above link, when David introduced the world to BaseRuns, he did just that. When I presented David's idea, I was more interested in the underpinnings to the whole thing.

At Baseballprimer, I had 2 articles on OPS which did show the differences in runs between OPS, BaseRuns and Linear Weights. (i.e., if you have a .300/.500 player or a .350/.450 player, what's the real difference between them?, etc)

I think the big roadblock is that BP and ESPN and Forman have a system in place, and a devotion to, publishing the metrics they so choose. The weight of their work shouldn't necessarily override the existence of better metrics.

Just like the existence of the Toronto Star doesn't preclude the Batters Box as being a more enjoyable source of information. But if you live in Toronto, you'll find it easier to pick up the paper, than to log in from a cafeteria.

The investment/reward benefits the GM more than anyone else. Once you figure out that the price of a stock is the present discounted value of its future earnings, are you going to use P/E:G?

When looking at exteme cases, that is those cases where EqR and xR and RC are limited to and fail at, you might unearth say an extra 10 runs in there. 10 runs = 1 win. And a team will pay 2 million$ / win.

So, in the case of say Delgado, you really need the absolute best most precise measure you can get. Even BsR by itself is not enough. You'd need to create a sim model that will show exactly how Delgado will interact, or expect to interact, with his teammates, and the impact of Delgado specifically. You might also find that putting him in the #2 spot will further add more runs and wins to the team.

Sign him for 5 years, and you now need an "aging" model, similar to the (unverified but seemingly sharp) PECOTA. Add all that up, and you can find say 10 million$ of value in a player. You can make a case say that while the market may value Delgado at 68 million over the next 4 years, you might figure out that he's really worth 52 million$. (Say like what the A's did with Giambi.)

Use the right tool for the right job. And, for the casual fan, OPS is fine. For the more devoted fans, EqR, xR, RC, and Win Shares are fine too. But, don't try selling these things as more than they are. Limits. That's what it's about.

I thought it would be good to point out that me and Brandon are the same person and I shamefully link my own article:)

The other thing to point out(which I explain in one of the linked articles) is that EQR basically is LW. In fact, if you ignore SB and CS, it is a 100% linear formula. Why Davenport chose to shove SB and CS in the denominator I do not pretend to know.

Of course, BP will never tell you that EQR is LW with non-linearity thrown in from SB and CS. IMO it would be more intellectually honest to do so, becuase EQR is just about the most misunderstood stat out there it seems.

When Bill James wrote about Runs Created in the 1983 and 1984 Abstracts, he exuded satisfaction from what he perceived to be the inherent beauty of his formula: A*B/C - what could be a more aesthetcally pure expression of the way offence works?

He was fairly close, but he didn't quite find paydirt, as it turned out. Base Runs provides an aesthetcally pleasing formula that makes logical sense and works very well at the micro-levels of offensive production. The "mainstream" sources would be well advised to promote it because it really is better.

Sadly, they won't. As we have learned from typwriters and Microsoft,standards - whether well thought out or not - tend to be hard to displace once they become well-established.

EQR is a relatively recent stat, right? I mean, it does not have the excuse of RC that it is 20 yrs old and therefore should not be held to the same standards of analysis as nowadays. But as far as I recall, I came up with BsR in 1996-97 (before I had ever looked at a computer). EQR can't be much, if any, older than that, and it is probably newer. So why didn't Clay expand on what had been done prior, which is what I did in trying to improve on the shortcomings of RC at the extremes? Maybe because he didn't want a stat which looks kind of like RC, as BsR does, for proprietary/ambition sorts of reasons. As far as I can tell, ERQ is just a hodgepodge stat, and perhaps put together in that way only to be "unique" in the "marketplace". If that is not at least partly the case, I will be surprised, and would be happy to apologise to Clay.

I agree with Tango. It is clear that there are several, or even quite a few, math formulations which can do a similar job for regular teams. That is the nature of math, and of the spread of performance in MLB. So how to distinguish among these? You have to look at game level data and inning level data. Tango did that in his articles on Primer. It would have been fun to see it also for EQR and a few other respected stats.

Eventually, it is inevitable that the best formulas and methods will get the proper recognition from the sabermetric community. The quality of the minds we are dealing with all but guarantees it.

http://economics.about.com
I'm going to make a few comments before bowing out of this conversation for good. This probably isn't a topic I should be posting in. :)

EQR is a relatively recent stat, right? I mean, it does not have the excuse of RC that it is 20 yrs old and therefore should not be held to the same standards of analysis as nowadays. But as far as I recall, I came up with BsR in 1996-97 (before I had ever looked at a computer). EQR can't be much, if any, older than that, and it is probably newer.

EqR was developed a heck of a lot longer time ago than 1997. [ Reply to This ]

http://economics.about.com

uggh.. I give up. Anyone want to get some Tim Hortons?

Mike

*** There's nothing at all preventing you from doing what Baseball Prospectus does.

Yes, my interest and time in doing so! BP does a great job at what they do. I'm not sure of the value-added I can give, and I'd rather do research than provide stats.

I don't know what motivation Clay and James et al have with their metrics. I'm sure they are giving an honest and large effort. But I don't think they want to engage in any kind of debate on this issue. Not that they should anyway.

I like the Michael Lewis line that James prefers to leave an honest mess than a tidy lie. It's not always fun to be the janitor.

One of the big problems with Baseruns, and I'm as guilty of it as the next is that there are about 37 different baserun formula's. I know, Tango, Pat, David, and I all use different ones and have for a few years.

If we could all pick one, and just stick to it we'd be better off. Or one technical version, and one basic version like XR.

I'm of the opinion that you take the full technical equation that I last published.

Anything that you don't know, estimate based on league average.

James' many tech versions haven't stopped people from using it.

"Honestly, I think you owe Clay an apology"

For what? Because I didn't know the date of the beginning of EQR? Fine. But B Prospectus and B James continue to use stats (EQR and RC) which structurally make liitle sense. Why do they do that? Because it's their *own* stat, and they have a proprietary interest in maintaining the image that it is *state of the art*.

I realize that such is not a criminal offense, but it is also not a criminal offense for me to point out the obvious. Look, I'm a capitalist. Without such motivations, there may have been no Prospectus books or website, both of which I have paid for willingly. And there may also have been no Win Shares book or New Historical Abstract book. Or a company called STATS, Inc.

But when you merge honest sabermetrics with proprietary motivations, you are walking a tight line. Too much of the former, and maybe your market share goes down. Too much of the latter, and maybe your credibility goes down.

One way to try to prevent your credibility from going down is simply not to respond, as though you are above it all, or don't have enough time, etc. B James has taken that approach a number of times. And, as Tango has mentioned, B Prospectus did not respond to his inquiry about their replacement formula construction. Surely they know who Tango is, and that an inquiry from him is not just one of the hundreds of emails they get in a week, etc.

I could go on, but you get the idea.

And just to complete that, several years ago the Big Bad Baseball Annual had a few articles about the problems with the B James Offensive Winning Percentage, as applied to individuals. I sent them a fairly long letter with the essential solution, and they never responded. They speculated on possible solutions in the book. I sent them a proper solution, and they ignored it. Either they didn't understand it (which I doubt), or they didn't want an outsider stealing their thunder. I'll never know for sure, but I have had a bad taste in my mouth ever since, from all of these organized proprietary sabermetric groups.

http://economics.about.com
Jeez, I said I wouldn't respond. I won't after this. Honest! :)

"Honestly, I think you owe Clay an apology"

For what?

Because you said you would if you were mistaken. It doesn't really influence me one way or the other. I don't have shares in Prospectus.

Treating Prospectus as a "Organized Proprietary Sabermetric Group" is kind of silly, when it was formed by a bunch of rec.sport.baseball guys who had been posting stuff on there for years. It'd be no different than you and tangotiger and Dudek publishing a book of your findings. If your ideas are so good and you are a capitalist, why don't you publish a book. I'd buy it. Of course, that will never happen because there's a conspiracy of ESPN, Baseball Prospectus, and the "Liberal Media" preventing that from taking place. :)

The Prospectus guys were the ultimate outsiders who decided to produce the book they always wanted to read. I guess you had to be there (rec.sport.baseball in the mid 1990's) to understand.

Mike

I will accept your explanation of how Prospectus began. I will not allow you to use that to whitewash all of the things I have mentioned which are relevant to the *present*.

And as far as their being a "conspiracy" to prevent present outsiders from coming in and taking over part of the market share, don't delude yourself that this is not reality. Bill James was not hesitant to discuss that, regarding the Elias Bureau, in his books. Why would you think that all is rosy now? I will say again, I am a capitalist. I do not begrudge anyone who fills a need and makes a profit off of sabermetrics. I benefit from such a dynamic.

But all that doesn't mean that EQR make much sense, and it doesn't mean that Clay is unaware that such is the case, which is all I'm trying to say.

http://economics.about.com
RE: Taking my word for it. It's not necessary. Google Groups has about a decade's worth of rec.sport.baseball material. You can see it for yourself.

I will accept your explanation of how Prospectus began. I will not allow you to use that to whitewash all of the things I have mentioned which are relevant to the *present*.

That's all fine, but your beef with Prospectus, ESPN, or the phone company really doesn't effect me. You'll have to take it up with them.

My position on it is this. If I e-mailed Milton Friedman with a proof of why one of his theories is flawed, chances are he wouldn't get back to me. The Republicans probably don't spend a lot of time considering policy proposals made by the Libertarians. It's not that the Libertarians ideas (or my ideas) are flawed, just that these people and groups probably get hundreds of proposals thrown at them in a year. I can't see it being in the best interests of these groups to spend the time considering the merits of each one.

If you want people to accept your ideas, you'll have to convince people of why they should. Maybe you could write your own book or present at an SABR conference. How about providing a website where people could get daily stat updates using your metrics? If your ideas start to generate interest then you'll see the "big" guys get interested. First you have to generate quite a bit interest at the grass roots level. The work you've presented on Primer is an excellent start.

Even if Prospectus never takes an interest, there's no reason why your ideas can't become more accepted than the ones hosted on that site. You guys do have good ideas, and with your collective brainpower I'm sure you could convince the majority of statheads that your metrics are better if you're willing to do the legwork.

Mike

Here's another example. In around 1992 or 1993, in response to their annual book, I sent a long letter to the Big Bad Baseball Annual folks, outling a method I had come up with. It was called "Outs Saved", and was the forerunner to the Base Wins concept. Both the Outs Saved method and Base Wins are *mathematically equivalent* to Win Shares. They are all the same essential formula, in different guises.

And of course, no response from the BBBA, even though it was way better than what they were doing at that time.

That stuff about the Republicans and Libertarians doesn't hold water. And to the extent that it does, it just illustrates my point, that the GOP (and Democrats and now Libertarians) are more about maintaning/increasing their power (Market share) than about actual changes or improvements in their positions. If you have read some of the more recent Libertarian literature (such as statements by former Libertarian candidate Harry Browne, whom I respect based on his older books), you will see that they are slowly turning into a typical political party that is willing to compromise in order to increase their chance of winning. Such seems to be the same pattern which has befallen the old rsbb guys who started Prospectus.

Mike, I wouldn't take the position that you "won't reply again", because who knows what interesting thing will come out in this thread.

***

As for BP responses, most of them are pretty good at replying all the time. Clay was very open the last time I corresponded with him (on a topic to my benefit), even though my last email was not replied to (to a topic of his benefit). I think I reply to all my emails, but maybe every now and then one slips through. I really don't owe anyone anything, and neither does Clay (unless you are a subscriber of his, at which point I suppose he owes you something).

As for writing a book and all that, I'm trying, but my family life takes a huge chunk of my time. I'm writing this post because my wife is breastfeeding this very second! As soon as she's done, we finally get to talk to each other alone for the first time today. And it's like this every day!

I do have the outline all drawn up with MGL. We've got the data, we have the methodology on proceeding. It's a slow and long process. And it's a book that should fill a void.

http://economics.about.com
Mike, I wouldn't take the position that you "won't reply again", because who knows what interesting thing will come out in this thread.

Yeah, I probably shouldn't. For whatever reason discussions of metrics often get really heated for no particular reason, and I always end up regretting that I was a part of the fray.

I do have the outline all drawn up with MGL. We've got the data, we have the methodology on proceeding. It's a slow and long process. And it's a book that should fill a void.

I'm really glad to hear it. If you need a proofreader, I'd love to get my hands on a draft copy. I realize it's probably not going to be anytime soon.

Cheers,

Mike

Metrics debates, like all other debates, if they don't have an agenda or process to follow, end up as pointless. Debates where we can't even agree on the question won't give you any answers. How could they?

If two people wanted to participate in a structured debate on run metrics, we could probably stop all future debates on this. Why? Because you'd have two sides present all arguments, and the reader will be left to choose his position. The reader wouldn't need to justify his position, because all the justifications will have been made in the debate.

It seems to me as the debate here is not really about metrics at all, but is about the "ethics" of sabermetric research, ie propriety v accuracy.

Just assuming for the moment that the BP folks are purposefully ignoring criticisms of their methods, I can still see where they're coming from, especially based on what Mike was saying about their origins. You see, they're just a bunch of guys who were posting about baseball on the internet and decided to write a book and they made a success of it eventually. They have no special status that anyone here doesn't have, save maybe buisness drive and skills. So if they start citing other people who are just independent sabermetricians and saying "they are right, we were wrong" what does that do for their status? Why should I listen to what Clay says about the Red Sox if he's not even right about EQR which is what he's supposed to know about? I see they're predicament, if in fact it is purposeful, which I am just assuming for the sake of argument.

It's easy to sit on the outside and criticize those on the inside. It's just like politicians who run for the House advocating term limits and then after 4 terms, decide to run again themselves. It's a different situation once you get there then it was before you were there. If I was a Representitive, I'd enjoy my publically funded pension and nice salary, and if I was a BP writer, I'd enjoy being able to make some money for doing something I like to do, and being quoted sometimes on ESPN.com, and in both cases I can see myself doing whatever I could to maintain my position. Of course, the BP admitting that they screwed up a little bit on defining replacement level shouldn't really hurt their credibility, but if they percieve that it might, then perception is reality.

"I'm of the opinion that you take the full technical equation that I last published."

Well considering I read just about every word you write and I have no idea where to find the full technical version, it explains why baseruns doesn't have more of a following than the 2 dozen of us who use it.

http:///www.geocities.com/tmasc
Go to my site above, and right after the 3 links for the "How Runs Are Created", I have a "BaseRuns addendum" or something. That's the one you want.

The "technical version" is right there on Tango's site.

And you know what? If you start from there and modify to a more basic version, you do not end up with the same event values (such as .463 for a 1b), and you do not end up with the same event ratios (in the full formula a HR is worth a bit over 3 times a 1b, but in a basic version derived from the same formula, a HR is only worth 2.9 * a 1b.

IOW, when you limit the input data to a more basic level, on-base increases in importance vs advancement. And we see the same thing in the regression formulas, such as that in Curve Ball.

So, someone asked for a BsR basic and advanced versions. This really depends on which outcomes you want to include, and why...

But here is one basic version, which is pretty decent:

A = H + BB - HR
B = .8*(1b+SB) + 2.2*2b + 3.5*3b + 2* HR
C = (AB-H)
D = HR