Stats Sunday – UZR and Defensive Metrics
The following article was posted in 2012 as part of WGN’s Stats Sunday. Thanks to Joe Posnanski for his work.
Bill James said this once, and he’s right: Defense is much harder to quantify with statistics than offense. This is true in every single sport, if you think about it. What are the defensive stats in football? Tackles? Sacks? Interceptions? They tell such a small part of the story. Same goes with basketball. Steals? Blocked shots? How does that get even close to the heart of everything that goes into basketball defense?
The problem is particularly stark in baseball. As you already know — and as the fine writers in this series have demonstrated — every offensive play in baseball is pretty easily catalogued. There are only so many things a hitter can do at the plate and on the bases, and great baseball fans and statisticians have found ways to measure all of them. How many hits per at-bat? How many times on base per plate appearance? How many total bases? How many runs did you knock in? How many runs did you score? How should a bunt that moves over a runner be scored? How should a fly ball that scores a man from third be scored? How should you score a hitter reaching first base after a strikeout? What if someone gets on board because a defender blew an easy play? On and on and on, the plays are indexed and catalogued, credits and debits are calculated, and then some very smart people figured out new ways to integrate those stats, to adjust them by ballpark and era, to divide them up by situation in order to give us a pretty powerful look at what makes offense.
Defense? Um. How about we just count up how many times they make obvious blunders?
That’s how defense has been judged for more than a century, Truth is, for years the credit for baseball “defense” was given almost entirely to the pitcher. This, I think, is why baseball pitchers have been granted more statistical power than any player in any sport. Think about it: In what other sport does a single person get statistical credit for a win or blame for a loss? Even an NFL quarterback, who might be the most influential player in any sport, doesn’t get a “W” or “L” at the end of every game. No quarterback has ever said, “It’s nice to get my 10th win of the season.”
But in baseball, that the pitcher “wins” and “loses” games is just taken for granted … and I think the biggest reason is that, because defense is so hard to inventory, everyone decided it would just be for the best to give the pitcher all the credit or all the blame. The pitcher gets the wins or losses. The pitcher gives up runs or prevents them. The only thing through the years that anyone did not credit or blame the pitcher for was when a fielder made a ghastly mistake. That, we called, an “error,” and that is ALL the fielder’s fault. If that error leads to a run, even hypothetically, then the pitcher cannot be blamed for it. Those runs, of course, are called “unearned runs.”
Really, it seems a goofy way to think about baseball. But, goofy or not, nobody really knew how else to evaluate defense. How can you measure a defensive player’s instincts, his range, his placement, his arm, his focus, his consistency, his reaction time? Instead, we counted errors — the obvious mistakes as judged by an official scorer up in the booth. Instead, we cheered amazing plays — when a player dived for a ball, or when a took away a home run, or when a player made a fantastic throw. We counted on our eyes to tell us who played great defense and who didn’t. And in many ways, we still do.
Well, as we get more and more information — more sense where balls are hit, more ways to measure those balls, technological advancements — there have been some efforts to get closer to the core of defense. Years go, Bill James came up with range factor, which tells you how many outs a player was involved with (both assists and putouts). There are some problems with this too, as Bill will tell you. There are ground ball pitchers and fly ball pitchers, lefties tend to force more balls to the left side and righties more to the right side and so on. So simply counting the plays made can be a deceiving way to judge a defensive player’s ability. But, it was a new start.
STATS Inc came up with something called “zone rating.” This is an effort to break up the field into different zones and then see how often a defender makes a play hit in that zone. Again, there are limitations to zone rating, but it’s another effort to quantify players’ defensive skills and productivity.
Ultimate Zone Rating — UZR for short — is the defensive statistic of choice for the excellent Fangraphs Web site. It takes zone rating up several notches and attempts to come up, through various means, how many runs a player saves his team with his defense.
The great website Baseball Reference uses Sean Smith’s Total Zone rating, which is a big more difficult to explain, but through various means — depending on how much information available — Total Zone charges fielders something for each hit allowed and then compares those numbers to others. It’s an interesting system that gives very interesting results. By this system, the five best defensive players in baseball history are:
1. Brooks Robinson
2. Mark Belanger
3. Ozzie Smith
4. Andruw Jones
5. Roberto Clemente
That’s not a bad list. I mean, you could argue with it, of course, but it’s not bad. Here are the five best Cubs defenders of the last 50 years by the same system:
1. Sammy Sosa
2. Mark Grace
3. Ryne Sandberg
4. Ron Santo
5. Randy Hundley
The defensive statistic that has intrigued me a lot over the last year is John Dewan’s Plus-Minus system. Generally it works like this: Dewan and his staff watch every single play on their computers. They try to, with as much precision as they can, mark where the ball is hit, the speed with which it is hit, the trajectory of line drives and fly balls and so on.
Then, they try to determine, with their database of all the balls hit, how often an average player makes that play. Let’s take an example: Say a ground ball with good speed is hit seven feet to the left of second base. How often does a shortstop make that play? Let’s say the shortstop makes that play 24% of the time.
OK, so if the shortstop they are grading makes the play, he gets credit for .76 of a play — that is 1 (a full play) minus .24 (the percentage of times someone makes the play).
If the shortstop FAILS to make the play, he gets docked .24 of a play — that’s the percentage of shortstops who make it.
It’s a hugely time-consuming process to figure out the Plus-Minus, of course, and again, there are limitations. But it does offer a pretty good feel for how a player compares to the average. How much better a shortstop is Starlin Castro from his first couple of years? Take a quick look at his plus-minus:
2010: +4 (4 plays better than average)
2011: -11 (11 plays worse than average)
So, he is getting better. According to the system, he’s really improved on balls hit to his right. Last year, he was minus-7 on those plays. This year, he’s plus-17. Huge, huge difference.
Of course, there are plenty of complaints about all these systems. I’ve heard from a lot of different people in baseball that they don’t need statistics to tell them who is doing the job defensively and who isn’t. I appreciate their viewpoint, but I disagree. Statistics are not everything, of course, but they do show us many things that the mind misses. If you see a guy make an incredible defensive play, that will stay with you. You can’t help it. I remember once seeing Nick Punto make a play at third base that still boggles my mind … he dived into foul territory, somehow got to his feet, somehow threw out the runner, it was as good as any Brooks Robinson or Graig Nettles or Adrian Beltre play I had ever seen. Maybe better.
And, no matter what, when I see Nick Punto, that plays over and over in my head. Is he a good defensive player? No idea. But he was amazing the day I saw him play. And so, in my mind, he will always be at least a little bit amazing. If you see someone make FIVE great plays or TEN great plays — say Torii Hunter — I would argue that you will never be convinced that player isn’t a great defensive player.
Same goes with seeing a few botched plays. I remember a friend in baseball coming back from a Philadelphia series telling me that Chase Utley had to be the worst second defensive baseman in baseball. This is a smart guy, too, a good baseball fan. He had seen Utley have a bad defensive weekend. But it was just one weekend. I looked up Utley’s numbers — they showed Utley to be absolutely THE BEST defensive second baseman in baseball. So who am I going to believe? My friend who saw him for a weekend? Or these statistical efforts that, flawed as they might be, attempt to break down every element of a player’s defense?
The eyes can deceive you. This is true in offense too. If we counted on our eyes to tell us who are the good hitters, well, think about this: On Friday night, Player A went two-for-for with a homer and a triple and Player B went 0-for-4 with two strikeouts and a double play ground ball. So OBVIOUSLY the eyes will tell you that Player A is better than Player B … except that we know he isn’t because of the stats. Player A is Scott Hairston, a useful enough player with a lifetime .247 average. Player B is Robinson Cano, one of the best in the game.
Defensive stats will keep getting better, no question. But they are pretty good already, if you use them in conjunction with each other. It’s easy to get bogged down by their limitations. But let’s face it … for too long we judged players defense by their best plays and worst mistakes. There has to be a better way. There is a better way.
Joe Posnanski writes about sports at SportsOnEarth.com. You can find him on Twitter too: @JPosnanski.