By In Stuff

Vanguard after the revolution

My latest on SportsWorld is on Bill James and his never ending quest to stamp out bullshit.

Print Friendly

24 Responses to Vanguard after the revolution

  1. VM says:

    I hope you challenged Bill a bit on the WAR issue. I know from reading him that he can be, ah, stubborn.

    Anyway, Tango responded here:

    I’d hope that Tango and Bill could discuss the issue and that Bill could understand more fully where Tango is coming from. The irony is that Tango DOESN’T get lost in the data – his conclusions are also fully grounded in reason, logic, etc.

    • Blue says:

      Tango’s not without his own issues.

      To begin with, his post ignores that the heart of the problem with WAR is that many of the people who invoke confuse the useful analytical concept of “replacement level” with a description of the actual world–the common fallacy of reification. One only needs to look at the number of sub-zero WAR seasons trotted out by players every year to realize that one simply cannot assume a replacement level player is available to every team at every point in the season. Even more egregious is the complete pull-out-of-one’s butt positional adjustment factor that is used to compare performance across positions.

      Beyond the chasm of problems that exists with WAR (or at least treating WAR as anything more than a broadly descriptive statistic of relative performance) sabermetricians constantly present a simple measure of central tendency as a point estimate without error bands. Here Tango is directly responsible for perhaps one of the worst offenses: his win expectancy matrix and everything based on it. Yes, it is a correct measure of central tendency–but it is shot through with a ridiculous level of false precision because it does not take into account variations in true talent level across teams and lineup positions. To take win expectancies seriously, you have to believe that the 2008 Royals tied 2 and 2 in the 8th with their number four hitter (Reggie Sanders) have the exact same chance to win as the 2001 Giants with Barry Bonds in the same position. That two on and two out with a one run lead in the bottom of the ninth has the same win expectancy is Mariano Rivera is on the mound of Ambrixos Burgos. This is clearly nonsense, and yet people are perfectly willing to treat this nonsense as gospel when, say, analyzing decisions to bunt or not.

  2. Guest says:

    I am glad to see this. I have never followed a post season as closely as this one, but this one has become almost unbearable, with writers — particularly those who normally say to ignore small sample sizes — focusing on arbitrary little decisions that were made or not made by the manager, and acting if the flip side of the decision had been made or not made, the outcome wold be the complete opposite. Bumgarner throws a gem and yet Ned Yost playing the infield in or not seems to be a major focal point for the media. Such a strange focus. It’s merely outcome based analysis masked as deep, analytical thinking. And, as Bill James would say, it’s bullshit.

  3. Mark Daniel says:

    Great column. James’ notion of statistics breeding their own form of bullshit was something that was experienced in medicine, back in the 90s when “evidence-based medicine” first came about. Doctors started using research evidence, as opposed to the opinion of an authority figure or expert, to guide decision making. There soon came a backlash because it was found that much evidence was flawed, and even the strong evidence was inapplicable in many cases. Thus, “evidence-based medicine’ has evolved to being the use of the best available evidence integrated with clinical expertise and patient values and preferences. Thus, the evidence is one part of a larger process. Baseball stats are probably similar. Some of the evidence is flawed, and otherwise the evidence must be integrated with expertise (managers, coaches) and player skill.

    • With medicine, you’re also facing a conflict of interest. When your research funding is coming from a major drug manufacturer, that would make a lot of money with a specific finding…. and you’d sure like to be used again for another multi million dollar research project…. well, guess what? Data can be twisted, especially when there are nuances to your findings. There are protocols to prevent it, but how do we explain the extensive use of statins when the evidence is that statistically they don’t help prevent first heart attacks? And with side effects factored in, they may not be advantageous for anyone who doesn’t already have a serious heart problem.

      • Mark Daniel says:

        Yes, you are right. I think that’s a bit issue right now. Statins is one, but TamiFlu is another drug. So you have drug companies overemphasizing small differences, and obtaining statistical significance by way of huge sample sizes, and you have drug companies not publishing the results of trials where the drug in question is shown to be ineffective.

  4. EnzoHernandez11 says:

    I suppose it was inevitable, given his contrarian nature, that Bill James would ultimately turn on his younger self. But, still, is Bill Freaking James actually saying, when speaking of leadership/clutch performance, that absence of evidence is not evidence of absence, which just so happens to be the rallying cry of science-deniers everywhere? I mean, there may be a point to that, but the idea that watching David Ortiz over a reasonably small sample of opportunities has altered his thinking on clutch hitting…well…I just thought, “That guy wouldn’t get away with comments like that if Bill James were still alive.”

    Bill James made me see baseball in new and exciting ways. He made me love the game even more that I already did, and I’ll always be grateful for that. But he’s also slung his fair share of bullshit over the years. Just read the various political musings spread throughout the Historical Abstract: almost all speculative, untethered bullshit. It’s his book, of course, and he can smuggle whatever socio-political commentary he wants into a book on baseball. I still enjoy the Abstract despite these silly sidebars, but there does seem to be a bit of a pot-kettle problem here.

    • Enzo: it comes down to the philosophy that if you can’t see it with your own eyes, that still doesn’t mean it’s not true. You might not be looking in the right place. We have a habit of gathering data, looking at it and making decisions about it. When, in reality, if we were honest, the data we have is woefully imperfect. So, we always need to be mindful that our seemingly brilliant observations, based on data, are still sometimes 100% wrong. In my work, we absolutely use data these days to solve problems. Using data, we’ve come up with some brilliant changes that have yielded great results. We’ve also used data to come up with brilliant changes that we were sure would be great…. and they weren’t. We missed something, often related to human behavior. Even “human behavior” is tough to predict because there are so many possible variations of human behavior. We watch as Bobby Cox contructs a perfect clubhouse and rides it to a bunch of World Series games (though only one championship) and all agree how important a good clubhouse is. Then remember how Billy Martin bred chaos in those Yankee clubhouses, but still won. Maybe clubhouses don’t matter. Or, maybe they don’t matter when you’ve got enough talent to overcome a bad clubhouse. Or, maybe if you have a bunch of talented meglomaniacs, it doesn’t much matter because they are used to being in conflict…. and maybe revel in it. There are so many possible variations. I’ll go with Bill’s thought that, if we’re honest, we have to realize that we know a lot less than we think…. and data just makes us more sure of our wrong opinions.

    • MikeN says:

      What are you talking about? ‘Absence of evidence is not evidence of absence’ is a perfectly valid scientific principle. And if you are talking about evolution, this is an argument that is used against the ID people, not by them. Just because science does not have an explanation, doesn’t mean one doesn’t exist.

      • EnzoHernandez11 says:

        Bellweather and Mike: Sorry, I should have made myself a bit clearer. Of course absence of evidence is not evidence of absence. But that’s why we study these issues systematically. We may not be able to prove a negative, but we can study an issue from enough angles that we are able to say, sorry, there’s just no persuasive evidence that clutch hitting makes that much a difference.

        Here’s what actually set me off: “He has been a consultant for the Red Sox for more than a decade, and he has watched David Ortiz deliver so many big hits in so many big moments, and he finds himself unwilling to deny that Big Papi does have an ability in those situations others don’t have.” Once we lose Bill James to confirmation bias, what hope is there for the rest of us? Nate Silver, who actually agrees that clutch hitting may exist, says this about Ortiz (spoiler: Ortiz’s clutch ability is questionable at best):

        FWIW, I agree that we almost certainly know less than we think, and that quantitative evidence can’t answer all our questions. And I share James’s concerns about WAR. But anecdotal evidence gets us nowhere–never has, never will.

  5. Phaedrus says:

    I’m not a sabermetrics guy, but it seems like the hard-core guys can’t enjoy baseball anymore. For example, Jeremy Guthrie will give up 1 run in 8 innings, and all the saber guys will say, “Guthrie didn’t strike anyone out last night. He was just lucky. Wait for the regression.”

    Baseball reminds me of the stock market now. You’ve got all these physics phds analyzing the market and coming up with all these fancy formulas, but the formulas all depend on assumptions that are bullshit, so the phds blowup sooner or later.

    Meanwhile, you’ve got an 80 year old grandma that majored in psychology, bought good companies when the market panicked, reinvested her dividends, and dies a multi-millionaire.

    Just because you can measure everything, doesn’t mean you can explain everything. Sometimes it’s best to use common sense and just enjoy the ride.

  6. lionoah says:

    As far as the leadership/clutch argument is concerned…I think the only way anyone could have ever missed that is precisely the evidence needed to prove it. Bill James says something, explains it fairly well, and everybody is buying beachfront property. Of course, anyone who can THINK for themselves (that is, relatively objectively) can clearly see that there is SOMETHING THERE.

    Joe, you and your readers have been bashing anyone who could possibly have a few words about clutch with this ad that without bothering to really think about what you were really saying. It seems to me that NOW folks are going to start thinking, but how could you miss it in the first place? I mean, straight up there are so many factors that we cannot measure and to flat out say that because we cannot measure something that it cannot exist is just the height of arrogance and pure mularkey.

    To underscore what James said about the sea and the bucket…at some point we’re going to have to factor in if a guys shoes were too tight one day while he was walking to school in the 9th grade on the 3rd of November, his breakfast, and whether he kissed his mom on that day among other things as the reason he missed that jump shot from his spot with no one guarding him in the 3rd quarter of a (seemingly) meaningless game in early February.

    • EnzoHernandez11 says:

      I don’t think anyone is saying that if “we cannot measure something that it cannot exist”. We’re just saying that if people are going to assert that leadership or clutch hitting or great managers are associated with winning baseball, they need to bring some good, systematic evidence to the table. If they can’t, then their assertions don’t carry much weight. And until they can provide such evidence, they truly–in the literal, and not the insulting, sense of the term–don’t know what they’re talking about.

      Also, clutch hitting and leadership are, from an analytical point of view, very different concepts. Clutch hitting IS easily measurable, or at least it is once we agree on how to define a “clutch situation”. Many people have studied this issue from all sorts of angles, and nobody has yet uncovered systematic evidence that certain players perform far above their skill level when the game is on the line. Sure, we can–and do–search for better definitions of clutch performance, which we can then test, but we’ve been doing this for a long time now, and so far, nothing…or, let’s say, next to nothing. So unless we build our definitions to suit our biases (clutch hitting is the ability to go long in the 12th inning of a playoff series when your team trails 3 games to 0), I think we have every right to be highly skeptical of clutch hitting.

      Leadership, on the other hand, is tougher. How do you measure it, without working the maze backwards (i.e., players whose teams consistently win must be leaders)? Probably, if it does exist, it’s so idiosyncratic that it’s impossible to define. Pete Rose leadership seems like a very different animal than Derek Jeter leadership. So, yeah, I’ll concede that leadership may affect winning baseball as long as you concede that we’re likely never to know how or when, and that assertions of leadership will always be post hoc and therefore at least somewhat suspect.

  7. No Comment says:

    I was always under the impression that the argument was not whether leadership and clutch performance were actual things, but whether we could reliably identify them.

  8. whatacrocker says:

    Awesome. The two biggest purveyors of Joe Paterno-related bullshit are congratulating one another on each other’s finely-tuned bullshit detectors.

  9. Michael Green says:

    As The Vin says–and he didn’t originate it–too many use statistics the way a drunk uses a lamp-post: for support rather than illumination. We need to remember that.

  10. John Leavy says:

    If Bill James really hates “bullshit,” he may want to re-read some of his apologies for Pete Rose and send that long-overdue apology to John Dowd.

    But I’m not holding my breath.

    • Gesge says:

      The thing to remember about James is that whenever he departs from the fairly narrow world of statistical analysis, he is not all that especially bright, and in fact can be quite crazy and/or stupid. His apologias for and passionate defense of Pete Rose is a good example; obviously the apology to John Dowd is years overdue but James, with his arrogance, will never give one. He wrote a terrible true crime book in which, among other things, he argues that Lizzie Borden didn’t do it (short answer: OF COURSE SHE FREAKING DID). And of late he has latched on to some idiotic JFK conspiracy theory that claims the fatal headshot was actually friendly fire from a Secret Serviceman.

      So don’t think that just because it’s Bill James, he has any idea what the hell he’s talking about. And that probably applies to the mystical concept of “clutch”.

  11. Joe R. says:

    What he wrote about The Dowd Report was not bullshit; he merely examined it and deemed it contained insufficient evidence of Rose’s (probable) guilt. James did not proclaim Rose to be innocent nor ever “apologize” for him.

    I agree you should not hold your breath.

    • John Leavy says:

      Bill James did a lot more than question the strength of John Dowd’s evidence. He was persistently rude, snarky and insulting. To re-use one of my favorite movie lines (Spencer Tracy in “Bad Day at Black Rock”), Bill James was not only wrong, he was wrong at the top of his voice.

      I don’t fault him for that- I’ve been wrong hundreds, maybe thousands of times in my life. We’re human, and we’re entitled to be make mistakes sometimes. But when state false/incorrect things loudly and publicly, we have a duty to apologize just as loudly and publicly.

      I neither hate nor worship Bill James- I think he’s a very useful thinker. He’s infuriated me often, but he’s also taught me many interesting things. His problem is that BECUASE many of his insights have been contrary to the conventional wisdom, he’s become REFLEXIVELY contrarian far too often.

  12. thitchner says:

    He called the guy “Doofus Jon Dowd,” among other things. He was a lot less dispassionate than you’re making him out to be.

Leave a Reply

Your email address will not be published. Required fields are marked *