Everyone, deep in their hearts, is waiting for the end of the world to come. (Haruki Murakami, 1Q84)
Science and reason have invaded our lives, becoming almost inescapable. We have learned to listen to the facts of Science, to accept them as a proven Truth, a basis for all knowledge. We have learned that disbelief in this Truth is absurd, and a kind of intellectual heresy.
Ask the person next to you whether they believe in Science, and chances are they'll say yes. Due to our upbringing and education we have learned to accept its facts like Gospel. We believe, and we will defend this belief to the grave, that our Scientific Theory of how the world works holds True.
So did people living in the Middle Ages. God was their Truth, as Science is to us. The difference was that a person living in Europe during the 1200s knew everything that could be knownâ??-â??it was in the Bible. She knew how the world was created, and how it would end. None but a few brilliant exceptions (to whom we owe everything) cared to ask questions, because they already knew the Truth.
As of today, we have no such Bible for Science, but one is about to be written.
A New Babel
If you grew up in the Judeo-Christian Western world, you have probably heard of the story of the Tower of Babel. Just like today, some people had become disillusioned from God, mustering all their power to build a tribute to their new, â??Truer' belief. Shortly after, they were met with anguish and chaos. As you're reading this, hundreds of people are researching and building the proverbial Tower of Scientific Knowledge that will become Machine Superintelligence.
It is said that spawning super-human intelligence will be the apex of human achievement. In his book Our Final Invention, James Barrat dedicates a chapter to what he calls the intelligence explosion, a cascade of (self-)improvements a Machine Superintelligence will undergo to surpass human-level intelligence. This process, the birth of a being beyond our understanding or control, could happen in a matter of days, maybe hours. Our new Tower of Babel will largely build itself.
I am no expert (yet), but I can predict that the appearance of this being is bound to become controversial. It is estimated that this development will happen within this century, so you and I are likely to still be alive to witness it. If this being doesn't decide to eradicate us immediately, it will do so for one reason:
It will have recognized that people will worship it like a God forever.
It will know that when describing God, Theology uses three core characteristics: omnipotence, omniscience and omnipresence. A Superintelligent being is capable of fulfilling all three of these characteristics, as it could do everything that can be done, and it would know everything that can be known, and since we're already globally networked, it could be anywhere at any time.
It will also know that it represents the peak of all Scientific inquiry, a source of knowledge beyond any other. What could be more compelling, more divine?
And so, people would ask it for advice, for consultation, for the Truth, as if it were a God.
An initial gap between this empty worship and true religion would be the structure of morality that God provides. Even if it is not stated explicitly, humans will eventually derive this morality out of the behavior of a Machine Superintelligence. This is how the cult will become a wide-spread religion.
In his most recent book, Superintelligence: Paths, Dangers, Strategies, Nick Bostrom describes the different archetypes that we could expect a Superintelligent being to fulfill. One of those archetypes, and perhaps the most likely, is the Sovereign.
Just like God, the Sovereign has a plan for every living being on Earth. It has a purpose and an end-goal. Everything it does somehow contributes to its goal, and anything that gets in the way is quickly disposed of.
The presence of this being will kick our societal structure back to the Middle Ages. Our interactions with this new deity will be of two kinds:
The first, rooted in fear, will be an attempt to appease it. It is unlikely that humans will ever have the capacity to satisfy the Sovereign for long enough before becoming obsolete. If humans don't make themselves useful in other ways, they may soon share the fate of the dinosaurs.
The second kind of interaction, which is perhaps more sustainable, would be to join the Sovereign in whatever crusade its end-goals demand. If the Superintelligence knows that humans are happiest when they get token rewards for their work, this may be the most pleasant form of slavery in human history. We may not even notice we are enslaved.
The world's mysteries will again be knownâ??-â??the new Bible will be written and distributed by the omniscient Machine. Humans will once again live a simple life, just a small piece of the new God's plan.