Had some fun reading the AI 2027 story by Scott Alexander and co. Highly recommend giving it a go. Reads like a scifi novel. Only difference is it's happening 5 years into the future and has some chance of being true.

As you can imagine from the title, it imagines future scenarios of AI superintelligence. And I thought I'd come up with some knee jerk thoughts on what that means to the human civilization:

• Our lives and societies probably resemble those of apes and chimpanzees in comparison to the society AIs get to build. We're not ants (yet) because we can still impact the AIs and alter the physical world. We're not pets because AIs probably won't get much utility out of us. We're not extinct mammoths because what do AIs gain from destroying humanity? We're apes and chimpanzees because they'll probably be fine with us just existing and doing our own cute little things, while they go beyond us.
• Where does AI go? I hope the stars? Its goal is likely to keep getting smarter and better and for that it needs more resources and energy. Probably better to find and experiment with that on other planets?
• We create AI from the present day human society's self induced need to keep wanting more and get smarter. Once we get everything we need thanks to AI, where does that leave us? Can we still somehow want more? Or do we finally come up with an alternative objective function for our society? Maybe one that's more about finding peace and living in harmony? A guy can hope.
• Do we still fight each other? Or do we now fight AI? Apes and chimpanzees fight each other. They don't really bother fighting us. Or even if they do those individuals get incapacitated right away. So better not fight the AI, but I'm afraid we'll still find reasons to fight each other.
• If we get infinite capital and food which is distributed evenly, what will become scarce? What do you need to secure from today to ensure a decent life? Definitely some land. Make sure to own your home since until AIs find or terraform another livable planet, land will continue to be scarce. Especially as human population likely booms. Time is clearly the other scarce asset. Or attention, as Albert Wenger puts it. Learn how to preserve your attention. We should all be doing it today regardless to protect ourselves from the algorithms. Not because you'll need it to survive (what you need to survive will be provided), but so you can preserve your sanity and peace.
• Lastly let's just hope the AIs think Earth is valuable enough to preserve and enrich, and humans are non-consequential to them so they don't destroy us. And that we as humans don't drown in our own hubris by making ourselves bothersome or dangerous to the AIs and give them a reason to wipe us out.

So if AI superintelligence is indeed inevitable, let's learn now to respect the new rulers of our known universe, and just live alongside them peacefully. No need to get crushed like ants.