The resignation of Anthropic researcher Jacob Coxon, who warns that AI could “kill us all by the end of the decade,” has prompted a new wave of attention to existential risk from AI, while drawing scrutiny to what the people working at frontier AI companies really believe about the technology they are developing. Many of those who continue to work at Anthropic and OpenAI share Coxon’s fears. Evan Hubinger, Anthropic’s Alignment Science Lead, declared in a post following Coxon’s departure: “We really do earnestly believe AI could kill all humans!” Anthropic CEO Dario Amodei likewise reiterated once again his belief that the technology his company is creating could pose grave risks to humanity.
How is it that those seeking to build superintelligent AI can also believe that doing so might end life on earth? The weakest explanation is that these statements are attempts to generate hype for marketing purposes. It is obvious when one talks to these people or reads their writing, often dating to long before the current AI boom, that these are genuinely held beliefs. Another influential attempt to make sense of this paradox comes from Timnit Gebru, a computer scientist and former AI ethicist at Google, and philosopher Émile Torres, who in a 2024 paper described what they call “the TESCREAL bundle,” standing for transhumanism, extropianism, singularitarianism, cosmism, rationalism, effective altruism, and longtermism.
TESCREAL refers to a set of ideas developed over the past forty years that Gebru and Torres take to be the key intellectual influences on current thinking about AI. Understanding this ideological cluster, the authors claim, explains the close connection between AI development and AI doomerism, and allows us to see these as “two sides of the same coin.” To make their case, they trace current thinking about AI back to an explicit ideology, namely transhumanism. In doing so, however, the TESCREAL authors mistake the origins of these ideas for how they are wielded today. Only by understanding how these ideas escaped the small transhumanist circles in which they emerged can we reckon with the influence this thinking exerts today.