The Closer You Get …

One question has fascinated me for a while now: How much have modern closers changed the game? I mean, sure, we know they have changed many things SURROUNDING the game in obvious ways. Closers have helped change the salary structure of the game. They have changed the way managers direct a game. They have given us indelible memories — the stomping of Al Hrabosky, the high heat of Goose Gossage, the mustache of Rollie Fingers, the Dan Quisenberry sinker ball, the unhittable pitches of Craig Kimbrel, the wonder of Mariano.

My question is just a little bit different and more focused on results: How much more often do baseball teams win games they lead going into the ninth inning now that closers rule the ninth inning?

I’ve written some about this before, so first I’ll review a bit and then get to some relatively new stuff. We start with a surprisingly simple fact:

When teams lead the game going into the ninth inning, they win 95% of the time.

No, the number is not all that surprising — I suspect all of us would probably have guessed that teams leading going into the ninth win somewhere around 95%. What’s surprising is how constant that statistic has been through the years — teams winning 95% of the time they lead going into the ninth is pretty close to a universal truth. It was true in the 1950s. It was true in the 1960s. It was true in the 1970s and 1980s and 1990s and 2000s.

Last year, 2,206 teams led going into the ninth inning, and 2095 of them won — that’s 95%.

Of course, it’s never EXACTLY 95% — last year, for instance, it was 94.968% — so there are small fluctuations which we will talk about in a minute. But do those fluctuations mean anything? If you fairly flip a coin 2,000 times, it almost certainly will not land on heads exactly 1,000 times and tails exactly 1,000 times. We still know that it’s a 50% chance of heads or tails. And it’s a 95% chance for teams to hold on to their ninth inning leads — the consistency of this number is staggering.

An example: In 1945, baseball was a different game. Almost all the baseball stars of the time were fighting World War II. The game was affected. Nobody in baseball hit 30 home runs that year. Guys like Stuffy Stirnweiss and Nels Porter and Steve Gromek were stars. It was disorienting.

In 1945, teams that led going into the ninth inning won 95% of the time.

The star players came back in 1946. Ted Williams led the league in on-base percentage, slugging percentage, total bases. Stan Musial hit .365. Hank Greenberg was back and he led baseball with 44 homers. Bob Feller pitched 371 innings (THREE HUNDRED SEVENTY ONE! It looks even more amazing in word form) with 36 complete games (THIRTY-SIX!) and he struck out 348 batters. Baseball was back.

In 1946, teams that led going into the ninth inning won 95% of the time.

The sheer stubbornness of this statistic is pretty remarkable. Baseball teams, through the years, have tried many different strategies to hold on to their ninth inning leads — some interesting, some provocative, some seemingly stupid. And while, in the short term, they might cause a few ripples, the long term percentage never changes. It stays at 95%.

So, you might ask: If the percentage is so constant (and so high), why do teams try all these new strategies? Why do they spend so much money on closers? It’s a provocative question. I do think that the idea of winning every single game you lead going into the ninth is SO tantalizing — it seems SO achievable — that teams just can’t help themselves. And it’s something easily documented. In 2011, the Baltimore Orioles lost four games they were leading going into the ninth. In 2012, they lost only one. It’s easy to say this made a huge difference between 93-loss 2011 Orioles and the 93-win 2012 Orioles.*

*I suspect the bigger difference was that 2011 Orioles led going into the ninth inning 63 times, the 2012 Orioles led going into the ninth inning 76 times. Well, that and the Orioles freakish 16-2 record in extra innings.

There’s something else. I think, that drives teams’ constant effort to cut into that one time in 20 that they blow a lead in the ninth inning: Emotion. When teams DO blow a game in the ninth, it hurts like a monster. Everybody takes these kinds of losses much harder than the garden variety 6-2 loss. I think teams overcompensate  because of that.

So, how much can new strategies affect the game? Well, if you look at the big picture, you have to go to the next decimal to find the differences:

1950s: .948
1960s: .946
1970s: .948
1980s: .951
1990s: .949
2000s: .954
2010s: .952

You can see that the last dozen or so years, the win percentages HAVE gone up slightly … the closer might deserve some of the credit. We’ll get into that in a second. But, how much of a difference are we talking about? In the 2000s, teams held on to 95.4% of their leads against, say, 94.8% in the 1970s. That’s roughly 135 extra wins in the 2000s. That’s 13.5 per season. That’s fewer than half a win per team. It’s not nothing. But you might argue that it’s not worth the many, many, many millions teams spent to get it.

This is how far I got last time … looking at this thing through a wide-angle lens. But, as many of you pointed out, looking at “ninth inning leads” as one entity is a very incomplete way of looking at things. Obviously a five-run lead going into the ninth is very different from a one-run lead going into the ninth. Before, I had no idea how to break down the leads by runs — Baseball Reference doesn’t yet give that option (though Sean Forman says it’s something they might try to do in the future) and I just don’t have the dexterity to manipulate the amazing Retrosheet database that way.

Well, Tom Tango and Baseball Prospectus to the rescue. Tom pointed out that by using the Baseball Prospectus expected win matrix, you can go back to the 1960s to find what a team’s win percentage would be when leading with 0 outs and 0 base runners in the ninth inning. Great, great information. Now, finally, I would see just how much closers have affected the game. Right?

Well, first thing I found is something obvious: Teams up five runs or more going into the ninth inning win just about 100% of the time. There’s a fluke comeback every now and again, but it’s pretty close to 100%.  Teams up four runs going into the ninth win 99% of the time. So we’ll throw those out for now.

How about three runs? Well, Goose Gossage said one time that if he got a save for pitching one inning with his team up three runs, he would be “embarrassed.” He’s not wrong there. Teams up three going into the ninth almost always win.

Winning percentages when team leads by three runs going into the ninth inning: 

1960s: .974
1970s: .977
1980s: .975
1990s: .963
2000s: .976

You will note that the lowest win percentage is in the 1990s. This is a big theme. Yes, teams obviously were using closers in the 1990s, but teams were also scoring runs at a historic rate.

Winning percentages when team leads by two runs going into the ninth inning:

1960s: .930
1970s: .925
1980s: .941
1990s: .936
2000s: .931

The numbers are kind of all over the place — but as you can see the winning percentage in the 2000s, with closers and setup-men and all that, almost precisely matches the winning percentage of the 1960s, when runs were hard to come by and starters often finished what they started. I’m not sure what you can learn from this. Now, to the big one.

WInning percentages when team leads by one run going into the ninth inning:

1960s: .844
1970s: .850
1980s: .852
1990s: .846
2000s: .848

And … yeah, the stat kind of pops like wet firecrackers. Not a lot to see here. Apparently, the win percentage when teams are up one entering the ninth leading doesn’t change much no matter what managers do. It was .850 in the 1970s. It was .848 in the 2000s.

Sure, yes, there are many variables here, and if you wanted to do an in-depth study of comebacks you would, as Tom Tango points out, take into consideration the run scoring environment. You would also consider ballparks and many other things. But I wasn’t really interested in that. I was really interested in knowing if closers have made it more likely that teams will win games they lead going into the ninth. And the answer, I believe, is no.

Now, wait a minute: You could argue that the game is constantly evolving and that teams need to use closers JUST TO MAINTAIN the status quo. That is to say, if teams tried to stretch their starters like they did in the 1970s or use their bullpen the way managers did in the 1960s, teams might come back in the ninth inning a much higher percentage of the time. Maybe the comeback rate would be 10% instead of 5%. I don’t know. It’s a great topic of conversation and somewhat beyond my own meager analytical skills.

But I just find it fascinating that no matter how much everyone tries to fiddle with the last inning of a game — closers, match-ups, specialists, pinch-hitters, whatever else — those overall ninth-inning win percentages just do not move. I would guess that teams with great closers having great years might help a team squeeze an extra win or two in a season. Maybe. But I do wonder if all of the ninth inning tactics are about as useful as rearranging furniture.

36 thoughts on “The Closer You Get …

  1. Justin

    Maybe instead of looking at league averages, you could go team-by-team, year-by-year and see whether the teams that have invested heavily in the back end of the bullpen have gotten a higher 9th inning winning percentage than teams that haven’t.

    Reply
    1. Sean

      Exactly. League averages are not that helpful for this exercise. If the Yankees with Mariano Rivera won 98% of games with 1-run lead in the ninth and the league avg was 85%, then right there is your value.

      Reply
    2. Dodger300

      And even if you do find that a Mariano Rivera makes all the difference, that would be ironclad evidence that unless you have him on your team, you would be even more foolish to spend big money on a proven closer, wouldn’t it?

      So when it comes to spending big money on closers, the numbers show that it is heads, you lose, and tails, you lose.

      Reply
    3. merle-thepearl

      I have no idea why I did this, but I went back to Rivera’s 2011 season and looked at each of his games. My hand tally comes up with a 14-3 record for the Yankees in games where they led by 1 run in the 9th inning and Rivera pitched in the 9th. That comes out to a win rate of .823, a bit below the league-wide average. That is a terribly small sample size, but it does suggest that there is not a ton of “value” (or variation really) against the league average.

      Reply
  2. richarder

    I’d be interested in seeing the breakdown by inning. How many teams were still ahead at the end of an inning after being up at the beginning of the inning. See if the percentages are different from the 9th innning.

    Reply
  3. First Baptist Church, Stephens

    This information seems to be whether they won in the end. It might be interesting to see how often they won in 9. It’s not a very important distinction, but the closer often doesn’t keep pitching in the 10th if he gives up the tying run in the 9th.

    Reply
  4. Zak

    A related question is how much has the atomization of relief-pitcher roles overall changed the game? Take a look at, say, what the chance of winning is from having a SEVENTH inning lead over time.

    Reply
    1. merle-thepearl

      My hand tally for Rivera’s 2011 games comes up with a team record of 11-0 when up by 3 runs, 18-0 when up by 2 runs and 14-3 when up by 1 run. That’s a tiny sample size, but within it, he is above league average with a 2-3 run lead and below league average with a 1 run lead. Overall that would come in below league average with a .938 winning percentage. I’m almost certain that would even out if you looked at more years and more games, but it does suggest Joe may be on to something.

      Reply
  5. Robert Rittner

    Justin makes a good point.

    I wonder also whether it is possible to analyze whether teams would more likely have leads in the 9th inning if they used their “best reliever” (i.e. their Rivera or Hoffman) earlier in the game when the win probability hangs more in the balance.

    Is it possible to speculate with any confidence, for example, what the difference in games won/lost would have been by looking at every game played while Rivera has been on the Yankees and put him in different situations rather than always holding him out until the 9th inning?

    Reply
  6. Pizza Cutter

    Joe, a few years ago, I ran some numbers and found that if teams used a strategy where they went to their closer in the 8th inning with a one-run lead, but left the three-run-lead ninth inning saves to the 8th inning guy, it might net them a sliver of a win. It would also mean that they would have to overhaul their closer development programs to produce guys who could regularly throw two innings. That’s certainly not impossible, but the system has settled into an equilibrium where closers expect to be one-and-done. Why overhaul your whole system for a sliver of a win?

    Reply
    1. Dodger300

      I think Joe’s point is that teams have alreayd overhauled their entire systems for just a sliver of a win, if that.

      They have eliminated a roster spot or two that used to go to utility players and platoon partners in order to carry more arms in a bullpen. I would like to see a comparison run to see if a good batting platoon at a position can contribute more than a half a win a year that a closer does.

      Please enormous amounts of money have been allocated, poorly, based on the premise that teams need a proven closer. The opportunity cost this creates is huge.

      Try spending a little less on a proven closer, and instead sign three sixteen-year-olds from Venezuela or the Dominican Republic.

      Let’s learn which approach helps a team more in the long run.

      Reply
    2. Mark A

      Have to disagree, Dodger360. The roster spots aren’t eliminated because of the closer role. They are eliminated because starters AND relievers of all roles are pitching fewer innings as they “protect” arms. Do closer by committee and you’d still have a big bullpen.

      Joe’s point is much more about how celebrated and well paid closers are relative to their small overall impact on wins.

      Reply
    3. Dodger300

      Closers are not limited to one inning in order to protect their arms, but rather just to conform with LaRussa’s model for the one inning proven closer.

      If any manager users his pitchers differently he is then subject to questioning whenever they fail. But if he follows the herd and designates a seventh inning guy, an eight inning guy, a lefty specialist and a proven closer, than the rigid manager becomes faultless, no matter how many times the bullpen implodes. It is only the pitchers who take the blame for non-performance.

      I stand by my assertion that it is the modern structure of the bullpen roles that has eliminated a position player from each team’s 25 man roster.

      Reply
    4. Mark A

      The one inning for closers isn’t to protect arms, no. It is so that you can have the closer available in the next game (the ones that can work more than an inning often have a day off after).

      But innings for starters are reduced to protect arms. And even long relievers either work fewer innings or get more rest between appearances. Goose Gossage relieved 8 innings a few times. Even a converted starter working long relief will almost never work that long now. And the fact that all these players work so few innings is why you need so many of them. Teams will really good rotations often have one less guy in the bullpen, because there are fewer innings to go around.

      You could have your better relievers working 2-3 innings instead of 1, but then they’d be getting days off. Which means you need another guy to be the reliever the next day.

      Reply
  7. Mark A

    It certainly is an interesting observation, but I’m not sure its as significant as you think.

    The closer may be a modern invention, but that doesn’t mean teams used to have their least reliable pitchers pitching out those leads.

    I don’t have numbers on this, and if you can find a way to study this (9th inning with a lead ERA vs overall league ERA) it would be cool. But it seems to me that up until maybe the late seventies, starters were far more likely to stay in throughout a game and close it out themselves. And a starter was generally one of the best 4 arms in your organization. And if he’s in there in the 9th with a lead, he’s having a pretty good day.

    Even on the occasions where its not the starter, even without a dedicated closer, I’m sure with a lead a manager was putting in his most reliable reliever, even if for more than one inning.

    So to me, the modern closer is less about increasing the % of leads you can hold onto, and more about the rarity of starters working a complete game anymore. If your starter isn’t finishing their own game, you are into the bullpen. And most of the bullpen is failed starters and lefty specialists. So having one guy in there who can just shut it down makes sense, in theory.

    So maybe the real question isn’t “why aren’t teams holding a higher % of leads than they did in the 50′s”. Maybe the question is “would teams be holding a much lower % of leads than in the 50s if they didn’t have closers?”

    I don’t know that to be the case, but its worth looking into. Maybe look at all the teams that have done closer by committee in the last 20 years, and see if they get to 95%.

    It still doesn’t justify the whole closer mythology, which is over the top. But I think the mythology is way more about the fans and media. And a big part of the big closer paycheque (and it seems to me only a few guys in the league get that, btw) is that the fans buy in. Rivera jerseys sell. Yankees fans believe Rivera greatly helps their chances.

    But the “mythology” aside, you can certainly make the case why having one top arm in your bullpen makes sense in an era where your other top arms won’t be pitching the 9th, and where you may even have a shaky arm or two in the rotation.

    Reply
  8. Kevin

    It astonishes me that teams don’t get this. It is more important to be leading going into the 9th than to have a closer who can close it. That is why I ALWAYS want to have my best reliever come in during the highest leverage inning. Let me bring up the example from the WBC and break it down into why Canada made a bonehead move and why USA did as well (even though theirs turned out well).

    In the top of the 8th. Canada is winning 3-2. Joe Mauer singles against Phillippe Aumont and then David Wright walks. Now at that point, you probably believe in Aumont but you have a 1-run lead. You have an ace relief pitcher in Axford who has amazing stuff. His K per 9 ratio is 12.1. You can only use him for 1 inning. Why not bring him in right then and there with men on the base. He is your best strike out reliever. Instead you stay with Aumont who has like 18 games of Major League experience. Let’s see, a guy who finished 9th in Cy Young voting (2011) or a rookie who gave up 10 hits in just 14.2 innings of work the year before? It seems like a no brainer to me. Bring in the ace reliever.

    The same situation happened in the bottom of the 8th. And it was even a worse decision. I say worse because Axford actually wasn’t that great last year, but he was better than a rookie. So in the bottom of the 8th USA now leading 5-3. Votto singles, the Saunders bunts to get on safely. Chris Robinson singles to load the bases. David Hernandez obviously doesn’t have it going. He has faced 4 guys and only got one of them out. You have the go-ahead run on base. You have the tying run in scoring position with 1 out. WHAT do you need? I DP or strike out to keep the Canadians for scoring. Now David Hernandez isn’t that bad of a pitcher. He has good stuff. His K/9 is 12.9. He seems like a logical choice to stay in the game. But he has already faced 4 batters. He has given up 3 hits. You need to bring in someone to close the door. The tying run is on 2nd. Why not bring in your strikeout machine. Craig Kimbrel is an absolute unhittable stud. He has a K/9 of 16.7. He pitched in 62.2 innings last year and struck out 116 people. That isn’t a misprint. HE is unhittable (a fact he prove in the meaningless 9th when he struck out 2 of the 3 people he faced).

    BUT let’s just say you want to trust Hernandez. Fine. When the next person drives in the 4th run for Canada, then you go to Kimbrel. You need 1 out. The tying run is at 3rd. Who would you rather have than the unhittable reliever you are saving for the 9th.

    Turns out the 8th was the most important inning of the game, because in the top of the 9th the USA broke it open. It was meaningless for both teams to save their star reliever for it. And it is something I said while watching the game.

    Reply
  9. Rhapsodizing Eclecticly

    Here’s a statistical question. Is the SPREAD of team’s winning percentages in the ninth inning similar across the decades? In other words, if teams are all imitating the same strategy now, you may see the CENTER stay the same, but the standard deviation decrease throughout the decades. Back in the 50s, 60s and maybe 70s, perhaps some managers were ahead of the curve and getting a big advantage out of closers or other choices (and vice-versa, far more “traditional” managers getting killed for not doing things), versus now, where if you do anything outside of the box, you can get creamed in the media for not doing the “right” move.

    Lots of confounding variables. But I agree that a reasonable conclusion would be for teams to not stress about innovating in terms of the ninth inning, and instead be concerned about the bigger picture, unless more studies can justify doing more.

    Thanks for this! Nice to see some heavy baseball stuff from you, even with all the new NBC work you’re doing.

    Reply
  10. Rick Rodstrom

    A baseball game is normally divided into 9 innings, or more precisely, 18 half innings. The deciding run in a game can be scored in any of these half innings. Over the course of a season, one would figure that the half inning in which the deciding run was scored would be more or less evenly distributed among all 18 half innings.

    1/18 is 5.56 percent. Of course, some games go extra innings, so the figure should be lowered a bit to accommodate the extra frames. Let’s say to 5%. So the fact that 5% of the deciding runs are perennially scored in the last half inning should be no more surprising than the 5% that are scored in the first half inning.

    Of the 5% of all deciding runs that are scored in the final turn at bat, one would expect some teams to win more and some teams to win less. 5% is approximately 8 games, so Team A could win 12 games in their last at bat and Team B could win 4 and the average would still be 8. Last year’s Orioles come to mind as a team with a lot of walk off wins. With pitchers, of course, it’s about losing games versus holding the lead, but the same logic applies. Some teams have lots of blown saves, some teams have few blown saves, even if the overall average of holding a ninth inning lead is 95%. In a sport where the best teams win 60% of the time and the worst teams 40%, and the pennant is often decided by a game or two, those handful of wins and losses have a big impact. And of course, the emotional highs and lows of winning/losing in dramatic 9th inning fashion are incalculable.

    Reply
    1. Trent Phloog

      This comment is brilliant! I would love to see a breakdown of how often the deciding run of a game is scored in each half-inning.

      My off-the-cuff guess would be that it isn’t actually 5% uniformly. Maybe the 5th or 6th inning (when starters begin to tire and managers may leave them in too long) are when a greater percentage of games are decided. Or maybe the 7th or 8th innings (when subpar relievers are called upon more often) are more critical?

      This would provide a rough approximation of which innings are, on average, “higher leverage,” and should thus have higher-quality pitchers assigned to them.

      Reply
  11. Patrick Ritter

    One mistake we don’t want to make when interpreting the 95% number, or any of these results from close games, is assuming that these are a product of league-average pitching.

    Can we assume that managers have, since at least the advent of the relief pitcher, played their best relievers in close games and put their worst relievers in during blowouts?

    If we assume that–or if the data bear that out–then any closer value above and beyond the “league average” is actually value above and beyond a fairly high-level reliever, not value as measured above replacement or above league average.

    Reply
  12. Mark Daniel

    Interesting. The problem, obviously, is that baseball is played one game at a time, and some games are ultra-important. Thus, if it’s the 7th game of the World Serie, bottom of the 9th, and you are up 1 run, would you trust someone else over Mariano Rivera because the statistics say there is only a 1 in 20 chance of blowing this game no matter who you pitch?

    Reply
  13. Dodger300

    Absolutely yes, if the seventh or eigth was a high-leverage inning and I needed my best reliever to save the game right then and there. Otherwise, of course you would bring Rivera in if he had not been needed earlier in the game.

    A key point is that the ninth inning is not played in a vacuum. Sometimes the earlier innings do not generate much of a threat to the lead and it is fine to let your best releiver ride until the ninth.

    But if a serious threat is mounted earlier in the game, do you trust someone else over a Mariano Rivera to save the game, by pooh-poohing the fact that it is only the seventh or eigth inning? I think not.

    Reply
  14. Bruce Watson

    Fantastic debate, but I wonder if there is a simpler explanation to it all.

    Although pitching tactics have changed over time – mainly the emergence of several bullpen specialist roles, including the closer – I think managers have always used the best pitcher they have available for the situation. That’s it. That’s why starters are starters pitch more innings than relievers: they’re simply better.

    As far as I can see, the only thing that’s really changed over time is that we now have about twice as many teams in baseball as we did in the 1960s. Back then, there were enough comparatively elite starting pitchers that they were the best guy to have in the game, whether it was the 1st inning or the 9th. Yes, we have more international talent in baseball now, but that fact is offset by the fact that more American kids than ever are going into other sports (basketball, football, even soccer) than ever before.

    So I see the emergence of closers and other bullpen specialist roles as resulting more or less from the overall dilution of pitching talent, combined with an increase in hitting prowess.

    (FYI my last statement there is based on the well-documented hypothesis that the rise of strength training has benefited hitters far more than pitchers. Also, I don’t think there’s any question that hitting with a maple or ash bat is generally a lot easier than those old hickory sticks.)

    Reply
  15. Pogoschtick

    At the end of every season, any league’s won-loss percentage average is .500 ( skipping interleague play, for now). Using your flawed and ridiculous logic that fueled this completely misguided article, Poz, one could argue that having .300 hitters and good pitching is worthless, since the league average will always take you back to .500. Year after year after year. Catch my drift?

    Applying averaged conclusions from data collected from a networked group to specific entities across the board in that survey group only gives you an useful average when applied on a interpersonal level. But it doesn’t extrapolate down to the intrapersonal level. For that, you have to isolate each specific entity over time and average only taht entitie’s data. If you’re gonna play with sabermetrics, you gotta know the rules. Do this article over, and do it right. Do it on a team by team basis.

    Reply
  16. Jon W.

    What you need to do is look at transitions, times where some teams used one strategy and some used another. If everyone is using a 9th inning closer, or everyone is throwing complete games, or everyone is using a ’70s fireman, then the percentage of held 9th inning leads is 95%.

    But what if one team is using a 9th-inning-only closer and another team is letting their starters go 9?

    One thing I do know is that the 1980 Oakland A’s famously had twice as many complete games as any other team in the league, had a league-leading 3.31 ERA. Billy Martin intentionally used a pitching strategy that would have been more typical from 30 or 50 years prior. And those A’s were dead last in the majors in 9th inning opponent’s OPS.

    Reply
    1. dodger300

      Please do provide a link to the web site where you got that information regarding the A’s opponent’s 9th inning OPS in 1980, okay Jon?

      That must be a VERY specific web site. Especially since no one even talked about the concept of OPS back in 1980, let alone, breoke it down for just the ninth inning.

      But I am sure you must have the data because no one here would be tempted to make a statement like that up out of the thin blue air, right?

      Reply
  17. Pingback: Opening the HoME Doors to Closers? | the Hall of Miller and Eric

  18. Pingback: O’s Continued Struggles Closing Games » The Natty Dugout

  19. Pingback: Orioles Continued Struggles Closing Games - The Natty Dugout

Comment: