We all live in a sea of ​​data, but three very public professions struggle in it: baseball, poker and election prediction. These three illustrate the benefits and pitfalls of a data-driven approach to decision-making. The key challenge in each is knowing when it makes sense to trust the data and when it makes sense to interpret it through your instincts—instincts born of experience that simply can’t be captured in numbers.

Let’s start with baseball. Freddy Freeman’s grand slam in the first game of this year’s World Series brought to mind a classic managerial move that still lives on 25 years later. It was the 1988 World Series, pitting the Dodgers against the Oakland Athletics. The Dodgers trailed by one run with two outs in the bottom of the ninth when Dodger manager Tommy Lasorda sent Kirk Gibson, a badly injured power hitter who could barely walk, to pinch He would face one of the best pitchers in the game, Oakland A’s shortstop Dennis Eckersley.

The announcers were puzzled. Gibson was injured in both legs and was sitting in the clubhouse. One said Gibson would need a double just to get to first base.

And then the magic happened. Gibson hit a home run to end the game. Lasorda had trusted his gut—that Gibson’s grit and determination would overcome his injuries and at least create some drama. The longtime Dodger manager was known for his decision making and was often criticized for it. But considering the entire context of the game, including his other options, he made an unexpected move. Modern data analysts would have scorned the decision, but fans loved it, even before Gibson hit his home run.

Less than 10 years after the Dodgers won that World Series against the Athletics, Billy Beane became the A’s general manager. He began using sophisticated data and statistics to build a successful team on a much smaller budget than of its major competitor on the market. And it worked! With one of the smallest payrolls in Major League Baseball, the A’s went to the playoffs six times in the 2000s. The story is chronicled in Michael Lewis’ book, Moneyball: The Art of Winning an Unfair Game.

Since then, baseball has been transformed—there’s no other word for it—by art and science Sabermetricsthe data analysis behind Beane’s approach. But teams have moved far away from its original use, which was to build a successful team by buying or trading players who were undervalued by the market and selling those who were overvalued by the market. Data analysis is now used for a number of everyday decisions, such as when to pull a pitcher, often confusing fans. Analysts at MIT claim that their model for pitching can significantly increase the winning rate, for example, and managers, including the current Dodger boss, are increasingly pulling pitchers deep into a no-hittera once unheard of practice.

Poker has a similar history. The best poker players used to be intuitive in their approach, but the best poker players in the modern era play more by data than guts. In his book, On the sidelines, The art of risking everythingNate Silver calls this game GTO (for Game Theoretic Optimal). Players have the stats down and can assess the likelihood that a two-pair hand will beat a hypothetical hand for an opponent that includes open cards. The hypothetical hand is deduced from the betting patterns of the opponents, which include bluffing.

However, statistics cannot really help detect a bluff. Sensing deception requires human skill and incorporates unquantifiable things such as the tension in the air, a knowledge of a particular opponent’s past bluffing behavior, and subtle cues (which may be subconsciously sensed rather than explicitly identified). To win at world-class poker, a player needs both extraordinary analytical skills and the ability to bluff and bluff.

Nate Silver, of course, is best known these days not for poker, but for his detailed analyzes of polling data, which he updates daily at natesilver.netand for his commentary on the modeling game. One New York Times article titled “Here’s what my gut says about the election, but don’t trust anyone’s guts, not even mine.” Silver encourages readers to stick to the dates, as frustrating as that might be in a very tight race. Data, at least at the meta level where Silver plays, is carefully processed before presentation. Statisticians do their best to eliminate or compensate for various forms of polling bias. They accommodate for survey affiliations that take surveys; consistent biases one way or another in past survey results; and an assessment of survey methodology, which includes adjustments, weights and treatments for anomalies such as Covid-19. When you’re done with all the massages and collecting lots of surveys, Silver suggests it’s best to go with the data, even if you feel momentum or energy or some other ineffable force at work. If you do otherwise, one is susceptible to desires and internal mind games. (But he still has a gut feeling about the choices.)

So what should you be doing in your industry? The key is to discern where the data has reached its useful limits. Make the most of the data, but reserve your final decision to a combination of analysis and human judgment.

How is this inn practice done? Stories about baseball, poker, and polls provide some pointers:

• Immerse yourself in the data; know your data inside out
• Look for contradictions in the data and actively seek to understand them; many of Silver’s model notes focus on understanding anomalies
• Access your gut feelings based on data and other factors not captured in data; try to explain what the other factors are and why you believe what you do with them
• Look for disconfirming data for any hypothesis you develop; get out of your bubble and find people with different assumptions who can express why they disagree

Today, for better or for worse, you can bet on your gut in each of the above arenas. Poker, of course, is a betting game, but there are also markets for baseball (and other sports) and picks (see, for example, Polymarket). Winning and losing in these markets is the ultimate test of your overall analytical acumen in a given area. It also highlights another reality: you can’t be an expert at everything. Doing a good analysis takes time.

In an ideal world, we will all become cyborgs: real-life breathing, thinking and feeling people with valuable experience, sitting at the controls of data sources and analytical tools that allow us to work with analytics to make the right decisions in our areas of interest. At the moment, the statistics are very good, but the ultimate winners are unlikely to be those who throw the judgment of people in the know out of the loop.