Too Big to Ignore: When Does Big Data Provide Big Value?

Much of the language surrounding big data conveys a muddled conception of what data, “big” or otherwise, means to the majority of organizations pursuing analytics strategies. Big data is shrouded in hyperbole and confusion, which can be a breeding ground for strategic errors. Big data is a big deal, but it is time to separate the signal from the noise.

James Guszcza, David Steier, John Lucker, Vivekanand Gopalkrishnan, Harvey Lewis

The same body of psychological research that underpins behavioral economics also suggests that we are very poor natural statisticians. We are naturally prone to find spurious information in data where none exists, latch on to causal narratives that are unsupported by sketchy statistical evidence, ignore population base rates when estimating probabilities for individual cases, be overconfident in our judgments, and generally be “fooled by randomness.” … [ Read more ]