Written by: Jeff Adams
About a week ago, a group out of Boston University released a report on the efficacy of umpires in Major League Baseball. After looking at nearly 4 million pitches from 2008-2018, they concluded that umpires missed about 20 percent of ball-strike calls. Today, I wanted to look at the same type of data, but with a slightly different focus. For starters, I don’t have 4 million pitches on hand to analyze. I am limited somewhat by time and my own personal hardware so I’ll just be analyzing last season’s pitches pulled from the Statcast database.
Before we get started, I have a few assumptions about the data that I need to put out there. According to Statcast, ball placements are accurate to within an inch, while I considered the ball placements to be accurate. Additionally, I assume that the recorded point for each pitch location is representative of the exact middle of the baseball such that half the diameter exists to either side of the point.
In my 2018 dataset, Major League umpires had to make a call on 591,763 pitches. This represents approximately 82 percent of all pitches thrown, quite a bit more than I had originally thought. The plot below shows a sampling of these pitches to highlight where umpire misses generally occur.
I tagged all called pitches as one of three options: Correct, Strike called Ball, and Ball called Strike. Since only the edge of the ball must touch the edge of the zone for it to be called a strike, I consider the inner and outer boundaries as set by the plate width (17 in.) plus the diameter of a ball (~2.86 in.) on either side. The upper and lower boundaries are set by the batter and can vary from pitch to pitch.
As you can see the edges of the strike zone are relatively consistent with high concentrations of missed calls. The white outline is based on the correct width of the zone and the average top and bottom of the zone determined by Statcast. The most striking result of this plot is how many wild strike calls occurred well out of the zone. Partially, I believe this is because the Statcast database does not track check swings and simply tracked them as call strikes (if I am wrong, please correct me). In my dataset, I found that the pros missed 16.6% of the calls they made behind the plate which is less than BU report. This is probably due to differing definitions of the strike zone, but I’ve been unable to access their definitions.
As you can see, there are some pretty consistent trends in how umpires miss calls. If you’ve ever seen a player (batter or pitcher) get upset following an at-bat, an umpire’s likelihood to call an incorrect pitch jumps nearly 10 percent in all two strike counts; however, the common thinking that umpires “hunt strikes” in late counts does not appear to be true. In 2018, umpires were about eight times more likely to incorrectly call a ball with two strikes. That rate easily outstrips the normal occurrence and shows that the umpires could be on the batter’s side considerably more than you might think. In three ball counts, the distribution of bad calls swings in favor of balls yet again. Overall, umpires incorrectly called a ball three and half times than a strike. With nearly one-in-six pitches getting called incorrectly, maybe robot umpires don’t sound that bad.
Since Major League arms are (mostly) all talented at locating pitches off the corner of the plate, I wanted to see if the edges of the strike zone account for a disproportionate amount of missed calls. In the plot below, pitches that fell within one ball diameter of the strike zone edge were included. Keep in mind that due to my definition of the strike zone, the “outside” zone reflects pitches that are actually two balls off the edge of the plate and the “inside” zone shows pitches one ball off the plate. Pitches in this area made up just over 33% of all pitches thrown and 54.6% of all incorrect calls. The pitches that fell in the “outside” zone accounted for 41.1% of the misses with strikes being incorrectly called 4.26 times more than balls.
The “inside” pitches only accounted for 13.6% of incorrect calls which does make sense as pitched in this area cross the plate rather that painting the edge. Umpires have a slightly better handle on the width of the zone and are more likely to make a wrong call on a pitch that is either too high or too low with miss rates of about 45% and 42%, respectively.
Overall, it is clear that umpires miss quite a few calls, but I’ve been able to show that a plurality of missed calls occurs on the edge of the zone. With the increased velocity and movement in today’s game, I believe it’s becoming harder for umpires to discern the corners of the plate. That being said, without some serious improvement behind the plate the arguments against some sort of electronic strike calling are eroding.