Tangotiger...great work. It was a big project and an excellent idea.

Unfortunately, the list hasn't ended up as reasonable as I thought it would be. Neifi Perez is the 3rd worst fielder in all of baseball? Worse than Mo Vaughn? Worse than Ben Grieve? Methinks people are having a hard time differentiating between what they think of Neifi the fielder and what they think of Neifi the hitter.

I don't think it's a good idea to lump all the players together regardless of position. I'd opt for outfielders, middle infielders, corner infielders and catchers. 4 categories.

I'd agree with Robert. Comparing a corner OF's speed and arm with a 2nd baseman is unrealistic. Still, it's a fun project.

Baseball commentators are often criticized for evaluating defense based on what they see...."I see Jeter every day and I say he is a good shortstop." The answer is to objectively measure defense to take our eyes out of it.

So this project is back to the eyes. Tango, are you trying to see if our eyes are better than the media's?

I don't see why it's "unrealistic". Soriano's speed is his speed, regardless of whether it's at 2B or RF. Certainly, they won't be used to the same degree (that'll be another article).

One of the results (which will be part 2) is that Derek Jeter's best comp is Bobby Abreu. Looking at the 7 traits that the fans evaluated on, independent of position, and they say that Jeter is similar to Abreu. I couldn't say that if I asked the fans to look at it with respect to position.

However, I will accept that it is *difficult* for a fan to try to evaluate the arm strength of Jeter and of Abreu outside of their position. But, this is what a scout does anyway. He'll see Jeter playing SS in college, and he'll evaluate his traits independent of position, and then try to figure out his best position based on those traits.

The 7 categories of mine should really be 20, but then I wouldn't have had 500 ballots cast.

***

In a future article, I was going to regress 2002 UZR and the Fans' Scouting Report against 2003 UZR. Again, I did ask the fans to NOT look at the stats, so I'm hoping that they were not influenced by the stats or others' presentations of stats.

In part 3, I was going to do breakdowns by position. Please remember that this first article is only a prelimary set of results. I'll be giving you enough data in enough ways that you'll be able to pick whatever you think is most useful to you. I envision at least 5 articles, and possibly more.

Part 2, by the way, will be similarity scores for ALL 406 fielders, all linked to each other.

Tango, if you haven't already, you should make this a clutch hit. It would get a lot of eyeballs over there.

I have it as a "Primate Studies".

This is very interesting stuff, but somebody with an agenda is skewing the data -- Neifi Perez is in no way anything less than a phenomenal fielder.

Okay, let's see what's happening with Neifi. I translated everyone's stats into a zero to 100 point scale, with 50 as average. His totals:
Instincts: 1
First Few Steps: 21
Speed: 32
Hands: 1
Release: 4
Strength: 17
Accuracy: 0

Those numbers are god-awful.

Let's look at the individual ballots:

Hmmm... only 4 legitimate ballots. It seems that there were 5 total ballots cast for him, but one was tainted. So, Neifi shouldn't have even appeared on this list. Out of the 4 people, he was ranked as average or below by all in all, except by 1 person who marked him as above average in arm strength. If there is a conspiracy, all 4 fans had it in for him.

To be fair, I should in the future list the total ballots for this player, so that you can take that into account.

Ok, I see what I did. Instead of using a cutoff of at least 5 ballots, I used 3 ballots. I kept bouncing between the two, and I thought I had settled on 5, but I have 3. Like I said, I can keep the 3 as the minimum, but also show total ballots cast, and let the reader decide what cutoff is appropriate.

What reason is there to believe that people are really doing as they are supposed to--evaluating the players by observation. I strongly suspect that many or most participants are biased by all of the stuff they have seen on defensive analysis, and other sources. If a good fielder is +10 UZR and a bad fielder is -10, that's only about 1 extra play every 6.5 games that the good fielder makes over the stiff. The whole point of needing the PBP to evaluate defense is that even a regular observer cannot be expected to "see" this level of difference. So when I see the "usual suspects" on these lists, I can't help but wonder if Tango is getting tons of noise in this study. I don't really have any confidence that we are really seeing any sort of "scouting" reports.

For example, take A Ramirez. He has a low Fld% and a low UZR, but I watched him the last half of 2003 on the Cubs. I tried to watch him closely, being aware of his poor def. stats. I saw a guy with good reflexes, a good arm--essentially all of the qualities that good 3Bmen possess--except he makes too many errors. So my feedback on Ramirez was, I believe, pretty favorable overall. It would be interesting to see the overall results for Aramis. If they are very bad, then I question the ability of the average voter to observe, independent of the printed stuff they have seen, especially since nobody knew that Tango's study was on the horizon.

As I've said before, I've watched baseball for over 20 years, I watch over 100 games a year, and I don't know that I could tell an A. Ramirez from a Scotty Rolen or a Jeter from a Sanchez by watching them. That doesn't mean that others can't and even if it is diffcult, there still may be lots of "collective scouting wisdom" out there. It's a great project that may yield some interesting and useful results. It would have been nice to have lots more ballots, and it would have been nice (but probably not possible) to have had some kind of "voire dire" such that only those fans who were not familiar with UZR or other defensive metrics would have been allowed to participate...

Aramis did very poorly:

24/31/22/18/13/75/3

That 75 is for his throwing strength.

***

As I said, I think it's rather easy to tell the difference between Tim Wallach and Hubie Brooks, or Brooks and Larry Walker, without any fielding stats.

***

I agree more participation would be better, especially from those that criticize the results, but don't partake in the balloting!

I also have no way of saying how much bias there is in the study.

***

I also agree with MGL that this may yeild some valuable results. For example, one of the problems with UZR is regressing everyone to the same population mean. But, getting the results, now we can regress Cesar Izutris upwards because of what the fans say, and not towards 0. We'll see how the 2002 UZR + Scouting = UZR 2003 will give us.

Thanks so much to Tangotiger for joining our merry little band and posting this article here. Hopefully there will be many more to come!

As for me, I'm most interested in the Part 3 breakdowns by position.
Looking forward to it. There is simply too much good research going on in the community these days, that it's impossible to stay on top of it all.

Did Orlando Hudson qualify? I'm surprised to not see his name in the best group for several of those categories.

---"I agree more participation would be better, especially from those that criticize the results, but don't partake in the balloting!"

If you are referring to me, I did participate, but under an alias (for some reason I don't recall).

And those results for Aramis were what I expected. I may be wrong (I've never tried to rank arms), but Ramirez' arm strength seemed good to me.

David, no I was referring generally. You made a point to say that you did vote.

Hudson, Orlando 73,79,73,59,74,52,60

An excellent fielder, who just missed the cutoffs.

His most similar comps:
Cintron, Alex
Matos, Luis
Griffey Jr., Ken

The reason I think that you need to look at these measures on a position by position basis is that certain positions do not offer you the same ability to judge. Take for example arm strength and accuracy. How do you judge first basemen relative to third basemen or rightfielders? I watched over 120 Jays games last year. If I saw Delgado make more than 20 throws in earnest last year I don't remember it. The same comments could be made with respect to speed for catchers or third basemen. These things really don't matter at certain positions and can't be measured subjectively with any reliability against a shortstop who I saw make 3 or 4 throws a game. Not only that, but my experience with the game and the attributes attributable to various positions will inherently bias some of the results (i.e. players tend to play first because they lack the speed or arm strength to play a corner outfield spot; players tend to play second because their arm is not strong enough to stay at short). I think experienced baseball fans will naturally make those biased assessments.

Having said all of that, I think the basic premise is a good one. If you can accumulate enough responses, I think it will provide a relatively accurate measure of a few key traits, but I think they will be more accurate measured against position.

Nigel, I don't disagree with any of your points of fact.

The first issue I have is what to do with Craig Biggio, Jose Cruz, and the many many other multi-position players. I can't put Biggio at 2B and CF, and then have the readers give different responses for his speed or arm strength. The other issue is that I want to know what position best suits Jeter and Soriano. I can't do that if I consider them in a universe of their position. The point is that a scout will look at Biggio speed, and give it a number.

Now, you are right that the reliability of Delgado's speed won't be the same as that of Wells or Hudson. But, we can try to establish that reliability as well. How? Look at all players who are primary 1B and primary 2B and primary CF, and see what kind of ranges the fans gave them. What if all the ranges are the same? Then, we can probably say they have the same reliability. But, you can also say that there is a systematic bias, as you mentioned.

However, we shouldn't expect this study to be the end-all and be-all. It has certain promises, and it has certain payoffs. I for one have never seen Mark Ellis really play. UZR thinks he's great, and the fans think he's great. I have to believe that Mark Ellis is a great 2B. Derek Jeter's best comp is Bobby Abreu. Aramis Ramirez has a great arm, and not so good everything else.

I would treat this as giving the collective wisdom of the fans SOME degree of reliability. How much is yet to be determined. But, I'd rather have this information to a player's range factor. And maybe I prefer this info to ZR. I don't think I would prefer it to UZR.

Tangotiger, I recognize the problem with players that switch positions or play multiple positions and there is really no easy answer to that other than creating another category or class of players. I also agree that the concept of a "range" for a position may be useful or may be the result of the survey. My only problem with that concept is that I think some people may vote unintentionally with the thought process that they are measuring the attribute against others of the same position. For example, I know that when I completed the survey I said to myself that Kevin Cash has pretty good speed for a catcher. Now, relatively speaking Kevin Cash has average speed at best. I don't know whether I caught all those biases.

Anyway, none of this invalidates the study. I think its an ambitious and interesting study. I think my only point is to say I think the results should be interpreted with an understanding of a few of these inherent biases.

Thanks for your clarification. I agree that we should be careful to note and to try to account for the biases.

In the ballots I have received, some people ignored my requests for position-neutral evaluation, and even said things like "For player x, I based him as a 1B, and compared him to the avg 1B", etc, etc. I agree, the biases should not be ignored.

tango,

As far as I know, Jose Cruz only plays the outfield. Outfielders should definitely be in one group - they all use the same skills.

Thanks, Tango. The cumulative subjective evaluation adds something to the picture.

It's obviously not perfect (how could it be?), but it is an ingenious use of the power of the internet. Here are some suggestions to improve it for next time:

1. I agree with Robert that it would be helpful to break it down by primary position into the 4 defensive position types, and
2. it would be helpful to have a "double play pivot" for second basemen.

Cruz plays CF and RF. The CF speed is on display alot more than it is in RF. For LF/RF, how often do you see the accuracy of the throw from LF compared to RF?

I don't agree that you can merge 2B and SS into one position either. Again, how often are you going to see the 2B arm strength and accuracy in action. His 2B throw is less than 90 feet. SS throws are also alot of the times "on the run". He also has more time to release the ball at 2B, or even throw it sidearm, etc.

Essentially, all positions are different. Keeping them as 8, or 4, or 1, you can make a case for any of them. This is why I will do both ends, showing it by position, and by no position. The reader can take from that whatever he can.

I was thinking about DP pivot, framing the pitches, blocking the plate, hitting the cutoff, relays to home, backing up the play, etc. I haven't decided how position-specific I wanted to make it. For this first pass, I wanted something that would be common to all positions.

I'm afraid that too many options = too little participation. Even 500 ballots doesn't seem to be enough for some players.

Well, you could group them differently when different skills are being assessed. Arm strength could be C/SS/3B in one group and OF in another (all others ignored). Range could be middle infielders in one group and all outfielders in another. Hands could be catchers and first basemen in one group, and the other 3 infielders in another (no outfielders).

A leftfielder has to throw home (strength) and has the able to hit the cut-off man (accuracy) and has to throw to second to prevent the batter/runner taking it while a runner goes first to third (strength and accuracy). There are lots of opportunities to watch a leftfielder throw. The only throw that the rightfielder has to make that's significantly different is to third base - one can adjust for this).

IMO, grouping them altogether, and then by 8 positions both miss the mark. I don't think the first type is a sensible comparison and the latter is too narrow when dealing with tools.

I think your trait/position breakdown is an excellent suggestion.

Break up the traits by how often they are leveraged. Throwing strength would be SS/3B/RF/C. Hands might be the 4 IF spots, etc, etc.

http://www.baseballstuff.com/tangotiger/scouting/scout_Results1B.html
I have more results here, if you are interested.