Black swans

I just read Nassim Nicholas Taleb’s The Black Swan— a remarkable book, though one feels it is not quite as remarkable as the author thinks it is.

With a characteristic breeziness, Taleb redefines Black Swans to mean highly improbable yet highly consequential events.  (The traditional idea was a reversal of expected ideas— before Australia was discovered people thought all swans were white— but without the factor of importance; the color of swans is pretty trivial.)  Taleb points out Black Swans are all around us: 9/11, the Great Depression, most inventions and bestsellers.

Can I save you $17?  Maybe.  The main points are these:

  • Black Swans make a hash of predictability in history and especially economics.  All those financial experts with their carefully tuned risk management programs are charlatans.  Any prediction of the medium future will be made ludicrous by Black Swans.
  • They can hardly be dismissed as “aberrations”.  E.g., in the last fifty years, half the gain in value in U.S. stocks is due to just ten days of trading.
  • Many varying factors, especially biological— e.g. height or age— can be understood with a simple bell curve, a normal distribution.  Taleb calls this world Mediocristan.  But other variables, such as income or sales or the death counts of wars, are highly non-normal, and expectations based on the bell curve will produce serious errors; this is Extremistan.  And increasingly we all live in Extremistan.
  • More technically: in a normal distribution, the tails rapidly fall off.  You will occasionally see a 7-foot man; you will never see a 50-foot man.  The distribution of something like income or book sales or market changes, is non-normal, with a fat tail: you simply can’t put an upper limit on them based on statistics alone.  You’ll occasionally see a $70K income; you will fairly often see a $500K one; you can’t rule out $500,000K.  To put it another way, in a normal distribution instances more than three standard deviations (sigmas) from the mean are vanishingly rare; in a non-normal one you can easily find instances six or ten sigmas out.
  • People are just not made to wrap their minds around Black Swans.  We try to rationalize them after the fact, but we are also prone to building systems that exclude them.

He enumerates a few common errors; my favorite is the ludic fallacy: generalizing from toy models (especially games) to complex human systems.  Sometimes naming something is a powerful move; I think this one is a useful addition to the lexicon of skepticism.  Games of chance, in particular, are lousy metaphors (because they largely follow normal distributions).

Taleb is a financial trader and amateur philosopher, highly opinionated and often witty.  He reserves a particular scorn for quants (financial analysts with mathematical models that are apparently all based on normal distributions), economists, CEOs, central bankers, and the French.  In the first edition statisticians are on the list too, but by now he’s learned not to alienate them.

Did Mr. Whiz Kid make money during the present crisis?  Oh sure; he doesn’t even consider the 2008 crash a Black Swan; he’d expected a meltdown for years.  (As he points out, Black Swans are relative to the observer.)

I’d say I go along with most of what he says; like most skeptics he gets a little overexcited about what we don’t know.  Occasionally there’s no one more naïve than a habitual rebel: they’re so used to dismissing authority that they’ll accept some nonconventional idea just because a friend brought it up.  (An example is a throwaway line claiming that there’s nothing wrong with high-fat diets; he offers no backing except a friend’s say-so.  Similarly, he mentions that his newest strategy is a hedge against hyperinflation.  Hyperinflation??)

I don’t think history or even economics are as hopeless as he says.  Indeed, his favorite investing strategy (put 90% of your money in ultra-safe investments like government bonds; put the rest, what you can afford to lose, in diverse ventures that have a chance of great success) assumes that there are safe investments, that not everyone is an idiot.  Or to put it another way, our skepticism should be specific.  Alan Greenspan’s claims that the markets had risk under control, or febrile 1990s claims that the new economy validated dot-com business models with no income, were not only foolish, but dead wrong.  That doesn’t mean that everything in economics is wrong.

My other reservation is that I kept expecting Taleb to get to the real meat, and it felt like he never did.  There’s plenty more in the book, but he spends most of it making the basic points over and over in different ways rather than getting into details.  E.g. at one point he deigns to explain that non-normal distributions can be identified by their kurtosis.  That I understand; I used to work for SPSS.  But he doesn’t bother even to tell us what are the telling values for the kurtosis.

Similarly, he has an interesting section on scalable and non-scalable jobs; the first, like book-writing, software sales, and  financial trading, more clients can be added without additional work: the same book can be sold to 100 or 1 million readers; it takes no more effort to trade 100 than 100,000 shares.  In non-scalable jobs, like dentistry, farming, or prostitution, more clients mean more work.  Scalable jobs are in Extremistan; they have an all-or-nothing reward structure.  And more and more jobs are scalable— e.g. ‘musician’ used to be a non-scalable job, as there was a limit to how many people could hear a given musician; now it’s scalable— a small group of artists can get 90% of the market and the rewards, and they don’t even have to be alive.  That’s a useful distinction to make.  But wouldn’t it be interesting to quantify how many jobs are scalable, and how this has changed over time?  Not to Taleb, apparently.

On the other hand, if your reaction to all this is “Yeah, but…” or “Fine, there are aberrations, who cares”, you probably need to read the whole book to get the ideas farther into your brain.  We’re rationalizing creatures, but there is no valid rationalization that turns non-normal into normal distributions.