December 20, 2024
I think imminent AGI is a forcing function for philosophy. The unprecedented pace and scale of change for society, the human race, and maybe the universe itself means that even the wild hypotheticals of sci-fi imaginations may actually materialize1. As a result, we may get to, and be forced to, actually answer fundamental questions about meaning, consciousness, and morality (as in Kurt Vonnegut’s Tralfamadore parable, what is our higher purpose? is it worth trading free will for safety and abundance? was all of this progress even good for us?). When in the course of history has humanity dealt with normative questions so profound and yet so materially pressing?
Historically, religion and science/technology have been considered anti-correlated or even incompatible. For the reasons above, I think AGI/ASI will break this trend. Specifically, I can imagine two kinds of AI-fueled religions:
(1) Church of the machine god. Atheism, agnosticism, and non-secular spirituality are increasingly popular; I’m personally somewhere between agnostic and vaguely spiritual. But omniscient superintelligence will be god-like enough to make concepts like karma and divine judgment feel very real. Models are already trained on ~everything you’ve ever written or said on the Internet, and will soon be smart enough to critically evaluate it (see this automated paper review with o1). It’s probable that most of our conversations, keystrokes, clicks, even real-world movements or neuronal firings, will eventually be recorded. Some of this is already collected today and mostly lost in the data firehose, but “intelligence too cheap to meter” will change this. So any sins committed today that leave an online paper trail – academic dishonesty, mean-spiritedness, etc. – will be caught eventually, and the world is only getting more observable. Despite not knowing the moral inclinations of future ASIs, I expect more people to start prioritizing “being good” in their eyes. A retroactive Panopticon is a pretty strong way to implement divine judgment2.
(2) Ordnung, eco-nostalgia, or human supremacy. Powerful AI systems will change the world undemocratically, irrevocably, and pretty instantaneously, so I expect pseudo-religious anti-AI ideologies to gain a foothold. After all, religions and cults tend to thrive in periods of turmoil, and what value systems can people turn to when the sand is shifting exponentially faster under their feet? In an increasingly post-human world, where will people derive meaning, comfort, or superiority? I think secular religions – communally held, basally irrational, but spiritually important systems of beliefs – will attract wide audiences. Some potential flavors are (a) Neo-Luddites who try to reject technology entirely by living off the grid, or even attacking AI developments (b) naturalist spirituality that values pre-technology, primal human ways of being, and tries to get closer to the unadulterated natural world3 (c) either transcendent or selfish reasons to see human-made outputs (art, writing, software, etc.) as uniquely valuable.4 These religions will have to make concrete choices about what the ideal, “virtuous” human world looks or looked like – was it our hunter-gatherer days? Was it the 90s, just before mass personal computing? Was it the last ~year before AGI? Or (most likely) is it some hodge-podge of conditions that never coexisted, an imaginary Frankenstein world?
Right now, I can’t think of a 2124 prediction that I would actually consider “far-fetched”. ↩
I am not suggesting people take Roko’s basilisk seriously! ↩
Among things that could become neoreligious icons in an AGI world: endangered species, radiation-free zones, any natural geological formations, Thoreau-style forests, etc. I think what ends up being valued as “natural” and “most human” will be pretty arbitrary; in fact, even Thoreau’s untouched wildernesses had already been shaped for centuries by America’s indigenous peoples. ↩
I’m not claiming that any of these belief systems are bad, just that I think they will start to feel more like religions. ↩