Advanced Search

Leaders should have some understanding of variation, including appreciation of a stable system, [and] some understanding of special causes of variation and common causes.

A number is a number, but a trend is a thing of beauty.

Torture numbers, and they will confess to anything.

Statistics are like a bikini. What they reveal is suggestive; what they conceal is vital.

Averages hide more than they disclose.

We must keep in mind the story of the statistician who drowned while trying to wade across a river with an average depth of four feet. That is to say, in a culture that reveres statistics, we can never be sure what sort of nonsense will lodge in people's heads.

The field of statistics is based on something called the law of large numbers: as you increase your sample size, no single observation is going to hurt you. Sometimes that works. But the rules are based on classes of distribution that don’t always hold in our world.

All statistics come from games. But our world doesn’t resemble games. We don’t have dice that can deliver. Instead of dice with one through six, the real world can have one through five—and then a trillion. The real world can do that.

That’s why portfolio theory simply doesn’t work. It uses metrics like variance to describe risk, while most real risk comes from a single observation, so variance is a volatility that doesn’t really describe the risk. It’s very foolish to use variance.

Studying even the right tail of a distribution doesn’t tell you how to break free of the distribution. In short, if you want to use inferential methods to get outside the box, you have to look at someone who is outside the box!

To see the importance of this step in the analysis, ask yourself this question: If two firms in the same industry had differences in shareholder returns (or sales growth or return on assets…it doesn’t really matter) of 0.1 percent over one year, would you think they were behaving in fundamentally different ways? What about 1 percent? 5 percent? 10 percent? Over two years? Or ten? How much of a difference do you need, and for how long, before the performance differences are marked enough that you’re willing to believe that the two firms behaved differently? In every success study we’ve reviewed, what constitutes great performance is simply asserted. Wouldn’t it be useful instead to know – really know – which companies have delivered truly great performances?

Torture numbers, and they will confess to anything.

Statistics are like a bikini. What they reveal is suggestive; what they conceal is vital.