Individuals once central to the effective altruism movement are increasingly resigning from their roles in the artificial intelligence sector, citing fears that the technology poses an existential threat to humanity. This exodus comes as the movement’s principles have successfully permeated major institutions, including governments, tech corporations, and the Catholic Church.
The influence of these ideas has become particularly visible recently. Pope Leo, speaking in France, issued a warning about the emergence of a “paradise of machines.” Simultaneously, The New York Times reported that a co-founder of the AI company Anthropic dedicated several months to lobbying religious leaders on the potential for artificial intelligence to achieve consciousness.
Technology Correspondent Peter O’Brien examines how concepts that were once confined to a niche intellectual circle have been adopted by powerful entities. In a recent segment, O’Brien traces the trajectory of effective altruism and offers personal reflections on his own departure from the movement.
This development follows a broader trend of tension within the tech industry regarding the pace of AI advancement. Recent reports indicate that tech leaders and politicians are clashing over calls for a slowdown in development, while major companies continue to navigate high-stakes regulatory and safety accords.
I wish someone would explain to me what exactly ‘institutionalizing existential risk’ looks like in practice. Is this just corporate buzzword bingo now?
This reminds me of the early days of climate activism. The movement gets absorbed by the very institutions it tries to reform, and then the radicals leave.
Are they really leaving because they disagree with the safety measures? Or are they quitting because they actually won and can no longer serve as whistleblowers?
The Catholic Church getting involved? I never thought I would see effective altruism in a papal encyclical. That is a bizarre cultural shift.
It is deeply ironic that the people warning about AI doom are now part of the machine. Did they want to change the industry or just control it?