Breaking Defensive Metrics and the Large Hole in Public Facing Defensive Ratings

I started writing a piece about J.P. Crawford at third base when I stopped to dig into his defensive metrics, in particular the math behind the positional adjustment. Right now defensive metrics think Crawford is a god at third, but they also are at a sample size that is near useless. This brought me to thinking about Freddy Galvis playing second base as a rookie and being good enough defensively to offset the fact that he couldn’t hit, and this logically continued to thoughts about great up the middle defenders playing lesser defensive positions and what it does to their defensive metrics.

The following musings are taken from a set of run on Twitter threads from earlier today.

With the proliferation of advanced stats it has become common to just use WAR as the value of a player. I have some philosophical problems with WAR and how components are measured, but what I want to really talk about is how we relate players performing different skills on the field into a single relateable number. The core of this is the positional adjustment. It is a run factor that is supposed to translate the difference between the defensive requirements of positions and how that relates to their expected offensive outcome. Here are the run differentials that Fangraphs puts into their formula for WAR.

PositionFull Season Adjustment
 Per 1,458 innings (162 defensive games)

This creates the defensive spectrum that we tend to understand. This chart has a difference of 10 runs between a center fielder and a right fielder. A study by Jeff Zimmerman in 2015 postulated the gap might be lower with the actual gap between center and a corner outfield position might actually be only 6 runs.

Final Defensive Adjustments
Position(s)Runs/162 Games
CF, 3B, 2B1.75
RF, LF-4.25
1B, DH-9.25

What I have noticed is that when a player from the top of the spectrum is moved for reasons other than their physical abilities (a better defender, deference to a vet, etc.) the numbers tend to get weird. We have seen a few huge outlier seasons in corner spots by players moving from center to a corner, such as Mookie Betts experiencing a 15 run career gap between center and right, a number that may actually be made smaller by a bad 97 game sample in 2015. Outliers like this could indicate one of two possibilities.

  • We are measuring defense wrong: This could be a whole different post, but the top and upper bounds of defensive value start to strain against what is physically possible for a player to accomplish given a finite number of chance.
  • Our positional adjustments are off: This would cause us to over or under value players in relation to players at other positions in our final WAR calculation.

If both of these are not true, which is the current data output of public WAR calculations, then the conclusion is that it is in a team’s best interest to play their best defenders at weaker positions as long as the defender at the premium position is competent. Now we know this isn’t true because teams aren’t doing this intentionally, and given that this whole construction is just our model that means it doesn’t logically fit either. That means at least one of the other two possibilities is true.

The actual numbers we use are based on some calculations that were done about a decade ago that used the performance of players who moved positions. It’s certainly reasonable to suggest that those numbers have changed as the game has changed, so use the adjustments as guides more than as firm rules. – Fangaphs

When one quantifies these differences and also looks at the changes in fielding performance when players move to different positions, we can estimate the average differences between positions. – Baseball Reference

Now this is a major problem. The entire sample size of players for the positional adjustment study is limited to players who have played at least two different positions over the course of the study. In the original adjustments, Tom Tango tried to account for the experience difference that might come from the positional change, but what I am more interested in is what kind of players these measurements do and do not come from.

In this case any player who is good enough to not move down the defensive spectrum. This is significant to me because we aren’t comparing the elite of the elite to the other players involved. If we go out to a large enough sample size to include potentially later career moves down the defensive spectrum we encounter another problem. If Player A is moved from center to right field late in their career, we are comparing right field minus a step to center field, not a center fielder at their height in a right field corner. In this case the UZR spike from moving down the defensive spectrum would be less than we expected. Right now we don’t have a meaningful sample size of players playing multiple positions competently while being in the same physical state of being.

Is this difference significant? I don’t know. I also don’t know what the correct way to calculate the positional adjustment should be. What I do is that the current positional adjustments seem to favor in their prime plus defenders playing lesser defensive positions. The conclusion of this would create a logical problem with how the game is actually played and how teams are constructed.

Author: Matt Winkelman

Matt Winkelman

Matt is originally from Mt. Holly, NJ, but after a 4 year side track to Cleveland for college he now resides in Madison, WI. His work has appeared on Phuture Phillies, The Good Phight, and TheDynastyGuru.