The Security Guard

He works in a very boring building. It basically never gets robbed. He sits in his security guard booth doing the crossword. Every so often, there’s a noise, and he checks to see if it’s robbers, or just the wind.

It’s the wind. It is always the wind. It’s never robbers. Nobody wants to rob the Pillow Mart in Topeka, Ohio. If a building on average gets robbed once every decade or two, he might go his entire career without ever encountering a real robber.

At some point, he develops a useful heuristic: it he hears a noise, he might as well ignore it and keep on crossing words: it’s just the wind, bro.

This heuristic is right 99.9% of the time, which is pretty good as heuristics go. It saves him a lot of trouble.

The only problem is: he now provides literally no value. He’s excluded by fiat the possibility of ever being useful in any way. He could be losslessly replaced by a rock with the words “THERE ARE NO ROBBERS” on it.

The Doctor

She is a primary care doctor. Every day, patients come to her and says “My back hurts” or “My stomach feels weird”. She inspects, palpates, percusses and auscultates various body parts, does some tests, and says “It’s nothing, take two aspirin and call me in a week if it doesn’t improve”. It always improves; no one ever calls her.

Eventually, she gets sloppy. She inspects but does not palpate. She does not do the tests. She just says “It’s nothing, it’ll get better on its own”. And she is always right.

She will do this for her entire career. If she is very lucky, nothing bad will happen. More likely, two or three of her patients will have cancer or something else terrible, and she will miss it. But those people will die, and everyone else will remember that she was such a nice doctor, such a caring doctor. Always so reassuring, never poked and prodded them with needles like everyone else.

Her heuristic is right 99.9% of the time, but she provides literally no value. There is no point to her existence. She could be profitably replaced with a rock saying “IT’S NOTHING, TAKE TWO ASPIRIN AND WAIT FOR IT TO GO AWAY”.

The Futurist

He comments on the latest breathless press releases from tech companies. This will change everything! say the press releases. “No it won’t”, he comments. This is the greatest invention ever to exist! say the press releases. “It’s a scam,” he says.

Whatever upheaval is predicted, he denies it. Soon we’ll all have flying cars! “Our cars will remain earthbound as always”. Soon we’ll all use cryptocurrency! “We’ll continue using dollars and Visa cards, just like before.” We’re collapsing into dictatorship! “No, we’ll be the same boring oligarchic pseudo-democracy we are now” A new utopian age of citizen governance will flourish. “You’re drunk, go back to bed.”

When all the Brier scores are calculated and all the Bayes points added up, he is the best futurist of all. Everyone else occasionally gets bamboozled by some scam or hype train, but he never does. His heuristic is truly superb.

But - say it with me - he could be profitably replaced with a rock. “NOTHING EVER CHANGES OR IS INTERESTING”, says the rock, in letters chiseled into its surface. Why hire a squishy drooling human being, when this beautiful glittering rock is right there?

The Skeptic

She debunks everything. Telepathy? She has a debunking for it. Bigfoot? A debunking. Anti-vaxxers? Five debunkings, plus an extra, just for you.

When she started out, she researched each phenomenon carefully, found it smoke and mirrors, and then viciously insulted the rubes who believed it and the con men who spread it. After doing this a hundred times, she skipped steps one and two. Now her algorithm is “if anyone says something that sounds weird, or that contradicts popular wisdom, insult them viciously.”

She’s always right! When the hydroxychloroquine people came along, she was the first person to condemn them, while everyone else was busy researching stuff. When the ivermectin people came along, she was the first person to condemn them too! A flawless record

(shame about the time she condemned fluvoxamine equally viciously, though)

Fast, fun to read, and a 99.9% success rate. Pretty good, especially compared to everyone who “does their own research” and sometimes gets it wrong. Still, she takes up lots of oxygen and water and food. You know what doesn’t need oxygen or water or food? A rock with the phrase “YOUR RIDICULOUS-SOUNDING CONTRARIAN IDEA IS WRONG” written on it.

This is a great rock. You should cherish this rock. If you are often tempted to believe ridiculous-sounding contrarian ideas, the rock is your god. But it is a Protestant god. It does not need priests. If someone sets themselves up as a priest of the rock, you should politely tell them that they are not adding any value, and you prefer your rocks un-intermediated. If they make a bid to be some sort of thought leader, tell them you want your thought led by the rock directly.

The Interviewer

He assesses candidates for a big company. He chooses whoever went to the best college and has the longest experience.

Other interviewers will sometimes choose a diamond in the rough, or take a chance on someone with a less-polished resume who seems like a good culture fit. Not him. Anyone who went to an Ivy is better than anyone who went to State U is better than anyone who went to community college. Anyone with ten years’ experience is better than anyone with five is better than anyone with one. You can tell him about all your cool extracurricular projects and out-of-the-box accomplishments, and he will remain unswayed.

It cannot be denied that the employees he hires are very good. But when he dies, the coroner discovers that his head has a rock saying “HIRE PEOPLE FROM GOOD COLLEGES WITH LOTS OF EXPERIENCE” where his brain should be.

The Queen

She rules over a volcanic island. Everyone worries about when the volcano will erupt. The wisest men of the kingdom research the problem and decide that the volcano has a straight 1/1000 chance of erupting any given year, uncorrelated with whether it erupted the year before. There are some telltale signs legible to the wise - a slight change in the color of the lava, an imperceptible shift in the smell of the sulfur - but nothing obvious until it’s too late.

The queen founded a Learned Society Of Vulcanologists and charged them with predicting when the volcano will erupt. Unbeknownst to her, there were two kinds of vulcanologists. Honest vulcanologists, who genuinely tried to read the signs as best they could. And The Cult Of The Rock, an evil sect who gained diabolical knowledge by communing in secret with a rock containing the words “THE VOLCANO IS NOT ERUPTING”.

Every so often an honest vulcanologist felt like the lava was starting to look little weird and told the Queen. The Queen panicked and ask everyone for advice. The Honest vulcanologists said “look, it’s a hard question, the lava seems kind of weird today but it’s always weird in some way or other, this volcano rarely erupts but for all we know this time might be the exception”. The rock cultists secretly checked their rock and said “No, don’t worry, the volcano is not erupting”. Then the volcano didn’t erupt. The Queen punished the trigger-happy vulcanologist who sounded the false alarm, grumbled at the useless vulcanologists who weren’t sure either way, and promoted the confident cultists who correctly predicted everything was okay.

Time passed. With each passing year, the cultists and the institutions and methods of thought that produced them gained more and more status relative to the honest vulcanologists and their institutions and methods. The Queen died, her successor succeeded, and the island kept going along the same lines for let’s say five hundred years.

After five hundred years, the lava looked a bit weird, and the new Queen consulted her advisors. By this time they were 100% cultists, so they all consulted the rock and said “No, the volcano is not erupting”. The sulfur started to smell different, and the Queen asked “Are you sure?” and they double-checked the rock and said “Yeah, we’re sure”. The earth started to shake, and the Queen asked them one last time, so they got tiny magnifying glassses and looked at the rock as closely as they could, but it still said “THE VOLCANO IS NOT ERUPTING”. Then the volcano erupted and everyone died. The end.

The Weatherman

He lives in a port town and predicts hurricanes. Hurricanes are very rare, but whenever they happen all the ships sink, so weathermen get paid very well.

If you’ve read your Lovecraft, you know that various sinister death cults survived the fall of Atlantis, and none are more sinister than the Cult Of The Rock. This weatherman was an adept among them and secretly communed with a rock that said “THERE WON’T BE A HURRICANE”.

For many years, there was no hurricane, and he gained great renown. Other, lesser weathermen would sometimes worry about hurricanes, but he never did. The businessmen loved him because he never told them to cancel their sea voyages. The journalists loved him because he always gave a clear and confident answer to their inquiries. The politicians loved him because he brought their town fame and prosperity.

Then one month, a hurricane came. It was totally unexpected and lots of people died. The weatherman hastily said “Well, yes, sometimes there are outliers that even I can’t predict, I don’t think this detracts from my vast expertise and many years of success, and have you noticed some of the people criticizing me have business connections with foreign towns that probably plot our ruin?” An investigation was launched, but the businessmen and journalists and politicians all took his side, and he was exonerated and restored to his former place of honor.

Heuristics That Almost Always Work

Sometimes there’s a Heuristic That Almost Always Works, like “this technology won’t change everything” or “there won’t be a hurricane tomorrow”.

And sometimes the rare exceptions are so important to spot that we charge experts with the task. But the heuristics are so hard to beat that the experts themselves might be tempted to secretly rely on them, while publicly pretending to use more subtle forms of expertise. “My statistical model, accounting for chaos theory, barometric pressure, and the price of tea in China, says there won’t be a hurricane tomorrow. Rejoice!”

Maybe this is because the experts are stupid and lazy. Or maybe it’s social pressure: failure because you didn’t follow a well-known heuristic that even a rock can get right is more humiliating than failure because you didn’t predict a subtle phenomenon that nobody else predicted either. Or maybe it’s because false positives are more common (albeit less important) than false negatives, and so over any “reasonable” timescale the people who never give false positives look more accurate and get selected for.

This is bad for several reasons.

First, because it means everyone is wasting their time and money having experts at all.

But second, because it builds false confidence. Maybe the heuristic produces a prior of 99.9% that the thing won’t happen in general. But then you consult a bunch of experts, who all claim they have additional evidence that the thing won’t happen, and you raise your probability to 99.999%. But actually the experts were just using the same heuristic you were, and you should have stayed at 99.9%. False consensus via information cascade!

This new invention won’t change everything. This emerging disease won’t become a global pandemic. This conspiracy theory is dumb. This outsider hasn’t disproven the experts. This new drug won’t work. This dark horse candidate won’t win the election. This potential threat won’t destroy the world.

All these things are almost always true. But Heuristics That Almost Always Work tempt us to be more certain than we should of each.

[EDIT: Some people are asking if this is just the same thing as black swans. I agree black swans are great examples, but I think I’m talking about something slightly different, which includes heuristics like “you should hire the person from the top college” or “you should believe experts”. If you want you can think of a high school dropout outperforming a top college student as a “black swan”, but it doesn’t seem typical. And the point isn’t just “sometimes black swans happen”, but that the existence of experts using heuristics causes predictable over-updates towards those heuristics.]

Whenever someone pooh-poohs rationality as unnecessary, or makes fun of rationalists for spending zillions of brain cycles on “obvious” questions, check how they’re making their decisions. 99.9% of the time, it’s Heuristics That Almost Always Works.

(but make sure to watch for the other 0.1%; those are the people you learn from!)