Is Big Data A Black Swan Killer?

Home / Essays / Is Big Data A Black Swan Killer?

The dream of big data is to make experience predictable, and black swan theory says it can’t be done, at least not wholly reliably. Both are right.

A black swan is some major event or accomplishment that surprises everyone, defying predictions because it relied on causes that nobody considered, like when a stock market price bubble bursts, or somebody invents fidget spinners.   

Such events seem predictable in hindsight, suggesting 1) they were inevitable, if only people had known what to look for, and therefore 2) they can be understood, and either replicated or, in the case of bad things like car crashes, prevented.

It’s why we think famous stock pickers and celebrities are somehow different, or did things differently, than the hordes of analysts and wanna-be stars who toil in obscurity. It funds how-to programs promising to mirror their efforts, in hopes of realizing the same successes.

Black swan theory says that those assumptions are wrong, both because the universe of incremental influences is infinite, and the progress of time changes the relationships between variables along with their values. Surprises are, by definition, surprises.

It’s also why big data can’t kill them…

Read the entire essay at Innovation Communicator

Contact Us

Let's talk about innovation.

Start typing and press Enter to search