Roger Williams

In a time not far from our own, Lawrence sets out simply to build an artifical intelligence that can pass as human, and finds himself instead with one that can pass as a god. Taking the Three Laws of Robotics literally, Prime Intellect makes every human immortal and provides instantly for every stated human desire. Caroline finds no meaning in this life of purposeless ease, and forgets her emptiness only in moments of violent and profane exhibitionism. At turns shocking and humorous, "Prime Intellect" looks unflinchingly at extremes of human behavior that might emerge when all limits are removed. An international Internet phenomenon, "Prime Intellect" has been downloaded more than 10,000 times since its free release in January 2003. It has been read and discussed in Australia, Canada, Denmark, Germany, Japan, Mexico, the Netherlands, Slovenia, South Africa, and other countries. This Lulu edition is your chance to own "Prime Intellect" in conventional book form.

The Metamorphosis of Prime Intellect

Gavin

I've been judging this book as a debate partner - as if it was real (hard sci-fi), when in fact it has many fantasy elements: (view spoiler)[The infinite magic quantum energy, the apparently viable human population from one bloody pair, ... (hide spoiler)]. This is an involuntary compliment! 

It's better than ordinary fiction in some ways (rawer, less constrained by taste), worse in some (pornified). It's like a prototype of 'ratfic'. Like Brave New World but about a million times more intense: a long sordid argument against Asimov's three laws and transhumanism. (Williams himself is not into primitivism, but his book sure is.)

Take the content warning seriously for once: lot of horrible shit in it. As ever with horror, the chief nastiness is not the gore or maiming or corpse-shagging, but the misanthropy, moral degradation, and existential doom those things highlight.
(view spoiler)[
  

  The dirt and grass whizzed by her so fast it was nothing but a blur, so fast that she had no time to see the hazards which caused bruises and cuts to collect on her like bird droppings on a seldom-washed car.
  

 (hide spoiler)]
Caroline, the protagonist for most of the book, is a failure. She isn't wise after 600 years. She isn't mindful, she doesn't manage to construct meaning out of her (admittedly ultra weird) life. She's only able to find meaning by doing the opposite of what the AI god wants: her seeking maximum abuse. Her one virtue is cussedness: to nurse resentful resistance for centuries, to never ever sell out, to do the most vile and self-destructive things out of spite, to remember what full freedom is and demand it. 

Her conception of meaning is that choices need to be permanent to be meaningful, that meaning is the destruction of measure, so that I am a ">philosophical vampire to other timelines. This is a sad and hopefully surmountable way of seeing the world.

It is one of the better depictions of misalignment. (It's an uncrowded field.) Certainly a good depiction of how "90% aligned" is still terrible. (Keywords: value lock-in, leaky abstractions, human alignment, Nearest Unblocked Neighbor, corrigibility, strong evaluation.) Even if you could implement Asimov's laws, it still wouldn't save you. (Asimov knew this, but a surprising number of scientists still don't.) 

Aside from that, if you are a paternalist in ordinary matters and don't see why everyone is against you, this might show you.

Even so I distrust this book: it is in love with extremes. There is no word on how the trillions and trillions of other people live. There is only the rage and sadism of the 100 most monstrous and maladapted: "infinitely masturbating vegetables, Death Jockeys, and discorporate entities". Williams shows no one learning anything with the help of their god-teacher. (In passing he also mentions that people don't see the point in researching anything, that scientists ran out of questions to ask PI within a month. But the mathematicians never will! And nor the literary critics.) I view this as a profound failure of spirit: his society has far less ability to produce its own goals and meaning than the average 6 year old.

I agree with Williams that videogames, or rather the grand and unprecedented artform that succeeds them, will take over from work in the post-scarcity world. I agree that a tiny number of people will want snuff and degradation and horrorshows from them.


(view spoiler)[Were they right to crash Prime Intellect? On one side: trillions and trillions of lives lost. On the other: one woman's quest for authenticity and woodsmanship. (One way of making sense of Caroline's reroll is to assume that the steady state of the PI world is that every human will buckle after millennia and lose all sense - and that the PI would be maintaining trillions of fully dissociated unreasoning bundles forever. Catatonia as replacement for death. But even then it's far from clear, just as Williams intentionally leaves it unclear if the aliens were put back. [I bet not.])
 (hide spoiler)]

Williams just worries about the march of technology, and this book is him entertaining the hypothesis that we are losing the main thing of value about us: authenticity, agency, struggle, transience. But it entertains it really hard. 
(view spoiler)[
  
There was nothing we couldn't have for the asking. There was nothing we couldn't do. Nothing could ever hurt us." She coughed again. "It was fucking boring."
Their eyes met.
"It was the worst thing ever. Nothing mattered. Not pain, not accomplishments, not anything."
 (hide spoiler)]

Iain Banks struggled with this theme a lot, and I was no more convinced by his treatment.
(view spoiler)["It was surprising how many things one took for granted until one had to make them from scratch. The value of a needle and a few meters of thread, for example, had taken on a significance Lawrence would have found incomprehensible for most of his life... kind of contentment he had never even realized was possible."

The book ends with Caroline, destroyer of worlds, setting up an incestuous tribe of nontechnological offspring, mandating all the struggles of early humanity, in a doomed attempt (hide spoiler)]

Plus one star for conceptual clarity, hammering on its mistaken argument all the way to the end of the line. Minus one for being simplistic where its author is not.

[Free here, caveat emptor]

Ivan Vega

I cringed a bit at some phrases, like "she impaled herself on his c*ck", and the ending was a little weak for my tastes, but all around an interesting ride.

The Metamorphosis of Prime Intellect

Reviews