Looking at defensive stats

The past two years have shown us that the latest market inefficiency in baseball is clearly defense. Actually, that’s incorrect. The last market inefficiency was defense, because now everyone is on to it and soon enough something else will be undervalued. Jack Zduriencik’s Seattle Mariners are the poster child for improving defense (even though Billy Beane beat him to it), as we watched his team improve by 24 wins in 2009 despite scoring 31 fewer runs than 2008. They went from a team that allowed 5.01 runs per game in 2008 to a team that allowed 4.27 runs per game in 2009, improving their run differential by a net of 88 runs.

Evaluating defense has come a long way from the days of fielding percentage and errors, as more advanced statistics can more precisely measure the difficulty of a play based on where and how hard the ball was hit. In a piece for MLB.com, Doug Miller chronicles all of the newfangled defensive stats being used today, speaking to both the developers of various defensive statistics as well as team officials. Allow me to excerpt at length:

One of the pioneers of these stats, “The Fielding Bible” author John Dewan, says it all seems complex, but it isn’t. Dewan’s main stats, the DRS metric and Plus/Minus, are the result of logical data culled from comprehensive, painstaking attention to detail throughout a Major League season.

Simply put, Dewan’s company, Baseball Info Solutions, has upwards of 2,000 “scouts” who pore over video of every game played in the course of a 162-game MLB season and track each batted ball, analyzing how hard the balls are hit, how close or far they are from each fielder deemed to be responsible for making the play, and the result of what said defender does.

Many factors go into the point totals, including adjustments for things like stadium dimensions, wall height and even the occasional bonus points for home-run-saving catches.

Successful plays are awarded with a positive point total, points are subtracted for perceived failures, and the scores are added up and equated to “runs saved” throughout a year. Dewan and most other defensive-stat purveyors tend to agree that 10 runs saved equals one win over the course of a season.

“For Boston last year at third base, for example, Mike Lowell, who was unable to move well because of injury, cost them 20 runs, and now they have Adrian Beltre, and he added about 20 runs,” Dewan explains. “Right there, the Red Sox have added four wins. Plus they’ve added three wins at short with Marco Scutaro and a couple more in the outfield with Mike Cameron. It’s a huge improvement.”

UZR, developed by Mitchel Lichtman, is similar to DRS in its variables such as park adjustment, and to Dewan’s Plus/Minus in the sense that its scores are based on how often each defensive player is better than average on balls hit into their specific “zones” on the field.

Gutierrez, for example, led baseball with a UZR score of 29.1, while Aaron Rowand of the San Francisco Giants was one of the lowest-ranked center fielders in the game with a UZR of 1.3.

“Gutierrez had as much to do with our success as anybody last year,” Mariners manager Don Wakamatsu said. “He made our outfield better, he made our pitching staff better, he made our whole club better.”

For Dewan, Lichtman and David Pinto, who came up with the similar PMR metric, watching the Mariners improve by 24 games gave strong evidence that these stats are legit and the old methods of ranking defense, fielding percentage and range factor, are becoming antiquated.

Of course, defensive stats are far from perfect, just like offensive and pitching stats aren’t perfect either. Moshe Mandel at TYU pointed out the uncertainty of UZR given the naturally small sample sizes of defensive chances (think about how many balls a given fielder actually makes a play on in a game), and suggests a weighted system based on about three years of data. Jeff Zimmerman at Beyond The Box Score used a similar system and four years data to create UZR projections for 2010, which project the Yanks’ to be a below average defensive team next year (disclaimer: this was long before any major moves were made this offseason).

The more information used to make an evaluation, the better. By no means should statistics replace scouts, because there’s far too much information stats can’t measure. A spreadsheet won’t tell you if a hitter is losing bat speed (though they could suggest it), nor will they tell you that the guy throwing 97 is at risk for injury because his mechanics are deeply flawed. However, at the same time a scout’s eye won’t tell you that Nick Swisher’s down year in 2008 was a function of bad luck more so than declining skills.

The statistics born out of the game of baseball, just like the game itself, continue to evolve. What we have in UZR, +/-, RZR, PMR and the like are the most advanced defensive metrics ever available. They’re not perfect and they suffer from the same sample size issues as do the more traditional stats, but we’d be foolish to ignore them just because the don’t agree with what our eyes tell us. Like the dude from Memento said, “Memory can change the shape of a room; it can change the color of a car. And memories can be distorted. They’re just an interpretation, they’re not a record, and they’re irrelevant if you have the facts.” Statistics are facts.

Frankly, we’ve only seen the tip of the defensive metric iceberg, just wait until HitFX and this monster get fully implemented.

Photo Credit: David J. Phillip, AP