Not pets or ghosts but something else

There are commonly two paths people sketch out in which humans make it out to the other side of the singularity: Successionism Lite, advocated by such luminaries as “Scott Alexander" and “Anthropic", in which humans become Culture-type pets of far greater minds—“dogs seem chill after all!”—and Uploadvaganza, advocated by such luminaries as Elon Musk (on good days), Vitalik Buterin or Sam Altman, which argues that human augmentation is our best shot at maintaining control—“if you can’t beat ‘em, join ’em!“¹

The former scenario feels unimaginative and lacking to me, a hobbit-like grass-chewing future which forgoes human values like whatever's going on in this painting, or the reason anyone studies history. It's a sensible default if your culture has been marinating in Abrahamic religions, where we're already pets to something greater, but a strange one if you reflect on it. As for human augmentation as a way out, I'm skeptical that you could expand my mind without severely damaging my values and sense of self, and I notice I feel ~intense disgust at the idea.

Both scenarios are also suboptimal on practical grounds. The first involves intentionally passing the torch to the AIs at some point, which can mean taking current disempowerment trends for granted and not reflecting on how to maintain human sovereignty or preserving the CEV as much as you would otherwise. Because why bother? “The AI will do our philosophical homework” applies anywhere! Meanwhile human augmentation must grapple with how to twist humans into being the equals of machines. I don't think this is possible without swapping out everything that makes them “us” in any sense that matters for the goal “making it out of the singularity, empowered and happy”.² Beyond philosophy, it also seems that attempts at “joining” AIs often end in disempowerment rather than empowerment. Humans distilling Go models would mog any historic player, but are they actually more powerful for it? Intuitively, they don't seem like it! “Plugging yourself into the machine” is no guarantee you'll come out more empowered.

All this to say I'd love if there were a Secret Third Thing that allowed us to reap the benefits of introducing a maelstrom of intelligence to the economy while keeping patriots in control and human values safe.

Unfathomable intelligence vacuum6 Dec 2025Read →

To me, the first place to look is: human civilization is capable of miracles that I, an individual human, do not understand (like how the laptop I'm typing this on works); yet I don't seem disempowered by civilization; could we just copy-paste whatever is working well here, into the future?

We'll tackle this in the next episode of hit Subslop series “Croiss Lite". Stay tuned! Research enabled by patrons MATS, Neel Nanda, Algon33, and the mentorship of Luke Drago and Rudolf Laine.