Robin Hanson

Many thinkers believe that the next transformational change in human organization will be the onset of human-level artificial intelligence (the 'singularity'), and that the most likely method of achieving this will come through brain emulations or "ems": the ability to scan human brains and program their connections into ever faster computers. Taking this as his starting point, Hanson describes what a world dominated by these ems will be like.

Work, Love, and Life when Robots Rule the Earth

The Age of Em

Gavin

Believe me that it's remarkable; it's easily in the top 5 most insightful books out of the 500 I have reviewed here. I called Superintelligence the most rigorous exploration of the nonreal I had ever read: this beats it by a lot. You will find yourself reading pages on the properties of coolant pipes and be utterly engrossed. It is imaginary sociology, imaginary economics, real fiction. 

(But it lacks an ethnography entirely: no em speaks to us themselves.)

People tend to wrap Age of Em in ulterior motives and esoteric intentions, because they love it but see futurism as an unworthy goal for such an achievement. I am no different: this is the greatest compendium of real social science I have ever found.

No review can do much justice, but here's one particularly hair-raising point in it: Hanson surveys the whole course of human history, and notes the many ways our culture is unprecedented and, in the evolutionary sense, nonadaptive:


  we live in the brief but important “dreamtime” when delusions [drive] history. Our descendants will remember our era as the one where the human capacity to sincerely believe crazy non-adaptive things, and act on those beliefs, was dialed to the max.

Why is our era so delusory?

  
 Our knowledge has been growing so fast, and bringing such radical changes, that many of us see anything as possible, so that nothing can really be labeled delusion. 

  Rich folks like us have larger buffers of wealth to cushion our mistakes; we can live happily and long even while acting on crazy beliefs. 

  We humans evolved to signal various features of ourselves to one another via delusions; we usually think that the various things we do to signal are done for other reasons. For example, we think we pay for docs to help our loved ones get well, rather than to show that we care. We think we do politics because we want to help our nation, rather than to signal our character and loyalty. We are overconfident in our abilities in order to convince others to have confidence in us, and so on. But while our ancestors’ delusions were well adapted to their situations, and so didn’t hurt them much, the same delusions are not nearly as adapted to our rapidly changing world; our signaling induced delusions hurt us more. 

  Humans seem to have evolved to emphasize signaling more in good times than in bad. Since very few physical investments last very long, the main investments one can make in good times that last until bad times are allies and reputation. So we are built to, in good times, spend more time and energy on leisure, medicine, charity, morals, patriotism, and so on. Relative to our ancestors’ world, our whole era is one big very good time. 

  Our minds were built with a near mode designed more for practical concrete reasoning about things up close, and a far mode designed more for presenting a good image to others via our abstract reasoning about things far away. But our minds must now deal with a much larger world where many relevant things are much further away, and abstract reasoning is more useful. So we rely more than did our ancestors on that abstract far mode capability. But since that far mode was tuned more for presenting a good image, it is much more tolerant of good-looking delusions. 

  Tech now enables more exposure to mood-altering drugs and arts, and specialists make them into especially potent “super-stimuli.”... today drugs are cheap, we can hear music all the time, most surfaces are covered by art, and we spend much of our day with stories from TV, video games, etc. And all that art is made by organized groups of specialists far better than the typical ancestral artist. 

  We were built to be influenced by the rhetoric, eloquence, difficulty, drama, and repetition of arguments, not just their logic. Perhaps this once helped us to ally us with high status folks. And we were built to show our ideals via the stories we like, and also to like well-crafted stories. But today we are exposed to arguments and stories by folks far more expert than found in ancestral tribes. Since we are built to be quite awed and persuaded by such displays, our beliefs and ideals are highly influenced by our writers and story-tellers. And these folks in turn tell us what we want to hear, or what their patrons want us to hear, neither of which need have much to do with reality.
  

These factors combine to make our era the most consistently and consequentially deluded and unadaptive of any era ever. When they remember us, our distant descendants will be shake their heads at the demographic transition, where we each took far less than full advantage of the reproductive opportunities our wealth offered. They will note how we instead spent our wealth to buy products we saw in ads that talked mostly about the sort of folks who buy them. They will lament our obsession with super-stimuli that highjacked our evolved heuristics to give us taste without nutrition. They will note we spent vast sums on things that didn’t actually help on the margin, such as on medicine that didn’t make us healthier, or education that didn’t make us more productive.

Our descendants will also remember our adolescent and extreme mating patterns, our extreme gender personalities, and our unprecedentedly fierce warriors. They will be amazed at the strange religious, political, and social beliefs we acted on, and how we preferred a political system, democracy, designed to emphasize the hardly-considered fleeting delusory thoughts of the median voter rather than the considered opinions of our best experts.

Perhaps most important, our descendants may remember how history hung by a precarious thread on a few crucial coordination choices that our highly integrated rapidly changing world did or might have allowed us to achieve, and the strange delusions that influenced such choices. These choices might have been about global warming, rampaging robots, nuclear weapons, bioterror, etc. Our delusions may have led us to do something quite wonderful, or quite horrible, that permanently changed the options available to our descendants. This would be the most lasting legacy of this, our explosively growing dream time, when what was once adaptive behavior with mostly harmless delusions become strange and dreamy unadaptive behavior, before adaptation again reasserted a clear-headed relation between behavior and reality.

Our dreamtime will be a time of legend, a favorite setting for grand fiction, when low-delusion heroes and the strange rich clowns around them could most plausibly have changed the course of history. Perhaps most dramatic will be tragedies about dreamtime advocates who could foresee and were horrified by the coming slow stable adaptive eons, and tried passionately, but unsuccessfully, to prevent them.


It's easy to read a radical critique of our liberal values in there, but I believe him when he says that he doesn't dislike dreamtime; he just predicts it cannot last, because we are fighting an old and inexorable tide.

There are several thoughts this large, and a thousand other small insights in Age of Em.

The Age of Em Work, Love, and Life when Robots Rule the Earth

Reviews