The results are in. Blue Jay fans considered John McDonald and Vernon Wells as the best defenders on the club. Aaron Hill did very well on my ballot, but I probably sneaked in some consideration of his superior pivot on the double play into the "instincts" and "release" departments. Surprises?
No surprises; these match up well with the fans' general perception of the players' defensive abilities. I admire what Tom is trying to do with these but it appears to me that the analytical method is generally misunderstood by those filling the surveys; as a result, the output doesn't have the consistency or objectivity necessary for a scouting report.
I'm all for making it better. Please offer your suggestions.
As for some of the byproducts of the scouting report: in 20 years, not a soul outside of McDonald's family will remember him. But, we'll have something here. Rios? Who knows about him, outside of Toronto? Heck, Adam Everett gets terrific sabermetric play, but without the GG, who'd really believe it in 20 years? Well, the Houston fans are all over him.
You can also possibly track aging patterns.
You can compare players inter-positionally.
You can look for fan bias.
That just scratches the surface. The data itself is simply that... data. It needs to be analyzed properly, and understand biases (just this year was the first year that Junior's fielding fell below-average, according to Fans).
The good thing with the survey is that it takes a bit of effort, so that knocks out alot of lurkers. And, I'm able to knock out another 1 or 2% of junk ballots. It leaves you with a representative sample of hardcore fans. Whether they know what they are talking about is another story. But, it at least reflects the fan's opinion.
Oh, I'm all for the data. The data is great. I would rather have one advance scouting report than all this data, but this is unquestionably good stuff. I wish we had it for the last 100 years, believe me.
I just don't think that this data has enough reliability to employ it to make reliable judgments about the quality of players' tools and skills. But I've used it before for lots of things - it's good to use it to confirm something you read in the press, or something you have inferred from the statistical record.
What I need to do someday, Tango, is to sit down and help design a short video-based guide to scouting to help people fill these out, enlisting the help of some scouts and/or guys who've gone through MLB's scout school. I think an introduction to scouting methods would help bolster the reliability of this stuff significantly. I've been wanting to pick Joe Dimino's brain about what he learned at scout school for a while now, I just haven't found the time to do it.
That would be wonderful! Looking forward to whatever you produce.