If worries about AI safety and the end of the world entered your life recently, it’s probably because of a resignation letter.
On September 8, 2026, a researcher named Jacob Coxon quit Anthropic, the one of the two premier AI labs in the U.S. whose brand has tried to cultivate the image of being the careful one relative to its competitors. He had spent the previous three years doing pretraining research at OpenAI and then at Anthropic, the engine-room work that makes large models capable, and his goodbye note said, among other things: “The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt.” He left two months before his equity vested and walked away from it . The post was viewed more than a hundred million times within a day.
His colleagues, surprisingly did not rush to correct him. Evan Hubinger , who leads alignment science at Anthropic, posted within hours that Coxon was right, that they really do earnestly believe AI could kill all humans, and that his personal odds were somewhere north of one in ten inside a decade. More than two dozen members of Congress said something needed to be done. Elon Musk, who is building a frontier lab of his own, called the whole thing a psy-op , and mainstream news outlets predictably gave breathless coverage to the frenzy that followed.
In order to realistically evaluate the gravity of this warning, we need to understand the people who are making it. The loudest voices warning about the end of the world come out of a very specific twenty-five-year-old internet subculture, one that understands itself as a community of people who think more clearly than everyone else. Outside Silicon Valley the subculture looks obscure, a fandom with unusually dorky literary preferences. Inside the technology industry it is anything but. Its writers make or break reputations. Its alumni staff the labs, the safety institutes, and the philanthropic organizations that decide what “AI safety” means. Its arguments, marriages, feuds, and sexual proclivities are heavily interwoven into the structure of the organizations now asking for the public’s trust about the future of the species.
Ten months before Coxon resigned, in December 2025, nearly five hundred people gathered in a Berkeley music hall for a ceremony about the possible end of humanity . There was a live band and a twenty-eight-person choir. There were liturgical readings. As songs and speeches carried the room from humanity’s distant past toward its uncertain future, the chandeliers dimmed one by one. Heavily in attendance were employees of organizations trying to build, or contain, artificial intelligence. At the darkest point, the organizer sat at the edge of the stage, his voice breaking, and told the room: “Guys… I don’t think we’re gonna make it.”
The organizer was Raymond Arnold, a web developer, game designer, musician, and longtime builder of rationalist community institutions. Raised between Catholicism and atheism, Arnold discovered the rationalist movement as a young adult and found that it “radically expanded” his capacity for sacred feeling. In 2011 he invited nineteen friends into a New York apartment furnished for the evening with oil lamps, plasma balls, and lightsabers to beta-test a holiday for their new worldview. They sang about the first winter campfires and the technologies shaping humanity’s future, extinguished the final candle, and sat together in the dark. Two years later Arnold turned the experiment into a public production called Brighter Than Today . Versions soon spread from New York and the Bay Area to cities across North America, Europe, and beyond.
Arnold had initially imagined Secular Solstice as a broad humanist holiday, but it became the rationalist calendar’s principal feast day. This was not entirely accidental. He has organized the community since 2011, now helps run both LessWrong and Lighthaven—the Berkeley conference campus that serves as its physical center—and has described his function, earnestly, as that of a “village priest.” Solstice was his attempt to construct the cultural technology a durable movement needs: a canon of songs and readings, a ritual progression from light into darkness and back again, and a yearly occasion for geographically scattered believers to gather in one room.
By 2025, Arnold estimated the probability of an AI catastrophe at greater than fifty percent and worried that his own ritual was no longer equipped to contemplate extinction in a psychologically healthy way. He had largely ceded Solstice production to other organizers, but returned to shape what he thought might be “the last Solstice,” complete with an aftercare gathering around a firepit. The evening supplied everything religions have traditionally supplied: fellowship, ritual, a shared story about humanity’s place in the universe, and somewhere to sit with the knowledge that you are going to die. Possibly soon. Possibly because of something your friends built.
Follow the connections out from that room and you pass through some genuinely strange territory. A Harry Potter fanfic used as a recruiting pipeline. Psychological workshops that former participants describe as coercive. A program for breeding smarter children, promoted by an AI institute. A sex worker from Idaho who became one of the subculture’s most-read writers and now runs an AI-doom propaganda residency with Grimes, Elon Musk’s ex-girlfriend. A splinter sect that fled to tugboats and whose members have been linked to six deaths. And a blogger who wanted to abolish democracy, found the ear of several billionaire patrons, and watched his ideas come to fruition in the vice president’s office.
I am aware, that if you are not lurking on the internet as much as I do, of how utterly insane that paragraph sounds. However, every bit of it is heavily documented by numerous sources. The people in this story are not a secret society coordinating over dinner. They are a social scene: overlapping friendship groups, funding relationships, and sexual and romantic relationships, all of it threaded through the institutions asking you to trust them about AI regulation.
One of many unfortunate shortcomings of mainstream media is that credibly reporting on these things is so absurd that you can’t expect readers to take it seriously. However, I’ll endeavor to provide a thorough account of how our current slide into neo-fascism, theocratic dystopia, and the forecasted end of the world is deeply rooted in the history of this subculture.
So, let’s go back to the yeeeeaaaar two thousannnnd.
The prophet
We’ll start with Eliezer Yudkowsky , who had developed a formidable sense of his historical importance years before any contemporary AI company existed.
Yudkowsky was born in 1979 and grew up in a Modern Orthodox Jewish family. By his own account he wrote off the entire institution of public education in second grade, the year he discovered his math teacher did not know what a logarithm was. He read Feynman’s QED at nine and says he understood most of it. He counts his time in school as ten lost years, dropping out around eighth grade, partly for health reasons. No high school, no college, no formal training of any kind in the field of AI.
At eleven a grand-uncle handed him Great Mambo Chicken and the Transhuman Condition , a 1990 chronicle of cryonicists, space eccentrics, and people planning to upload their minds into computers. His own summary of the encounter is brief: after that he knew what was important, and that his career would touch it. In his late teens he found the transhumanists online. By 2000, at twenty, he had co-founded the Singularity Institute for Artificial Intelligence, and his website was already offering the complete package: an essay about the Singularity, a document sketching how you might actually code a transhuman AI, a plan for getting humanity from here to there, and an autobiography explaining the author’s singular qualifications.
His autobiography even opens with a parable: as a small child he threw his favorite squirtgun off a third-floor balcony rather than accept that his mother could lend out what was his, and felt complete when he heard it hit the pavement. He goes on to describes himself as a specialist in the technological transformation of the human species, one of the few people alive with the standing to steer it. He identifies with Buffy the Vampire Slayer, the one person who sees the danger clearly and has to act while everyone else gets on with ordinary life.
While his technical ideas would change a great deal over the next twenty-five years. His feeling of being the one person who to see the coming doom of AI did not.
The Singularity Institute still exists, since renamed the Machine Intelligence Research Institute, or MIRI, and its founding worry has aged better than most founding worries: a sufficiently capable artificial intelligence might pursue objectives incompatible with human survival, and the job is to make it safe before somebody makes it powerful. While reasonable on the surface, the trouble was everything that grew on top of it. In 2025 the book he co-wrote with MIRI’s executive director, Nate Soares, about why superhuman AI would kill us all, made the New York Times bestseller list .
During the late 2000s Yudkowsky developed a second project: teaching people how to think. His prolific writing on Overcoming Bias, the blog he shared with economist Robin Hanson, became the raw material for a body of essays his readers call the Sequences. In 2009, LessWrong launched as the community’s home base. The institute’s described the website as a successful recruiting channel .
You can see the appeal. Here was a unified explanation for why people argue badly, believe comforting nonsense, procrastinate, and confuse social approval with truth. And here was a program: notice your biases, revise your beliefs, do better. If you are clever, dissatisfied, and accustomed to being told you ask too many questions, this is a very persuasive invitation. It was for a lot of people.
And notice who the invitation is calibrated for, because the community has told on itself, at length, in its own surveys. The LessWrong census found a population 87 percent male, with autism-spectrum diagnoses or self-diagnoses in nearly one respondent in five and depression in nearly half; a later, related community called the Slate Star Codex, issued its own surveys , drawing thousands of respondents, finding more than half working in programming, engineering, math, or physics. The biography it sketches is consistent: a bunch of boys identified as exceptional at eight, told they were was smarter than everyone else, left alone with a computer upon struggling with social interaction, and delivered to adulthood precocious, socially misread, and never once in a room where everyone else understood them. Silicon Valley inevitably draws these folks in. Rationalism is where they find their meaning and community.
What the movement offers that person is belonging paired with steady reinforcement and affirmation. A peer group of similarly gifted individuals. A philosophy and dialect that finally renders the social world legible. And, most intoxicating, a promotion: you were not misanthropic, the masses genuinely are running corrupted firmware; you were not arrogant, you genuinely are a hero in the fight for the species. It is the gifted-and-talented program perfected: special sessions for special children, workbooks (the Sequences, Harry Potter and the Methods of Rationality), year-end ceremonies (Solstice), and an adult who finally believes in you. Main character syndrome stops being a character flaw the day you join a religion whose plot has you in the lead.
It also contains a trap– a community organized around overcoming human irrationality starts to treat membership as evidence that its members have overcome it. Being a rationalist transforms into a credential for superior judgment, and superior judgment is the community’s principal social currency.
The gospel according to Harry Potter
In 2010, Yudkowsky began publishing Harry Potter and the Methods of Rationality , a sprawling alternate-universe fanfic in which Harry is a precocious scientific rationalist confronting magic with curiosity and a highly developed appetite for explaining why everyone else is wrong. It ran for five years and became enormously popular outside the community as well as inside it. The Sequences were too dry for many, but HPMOR became an easy on-ramp to rationalist thought. Moreover, it captured the target audience by holding up a mirror: Harry is the target demographic’s self-portrait at eleven– identified as gifted, right about everything, and continually surrounded by inferiors.
The novel’s most consequential idea is “heroic responsibility” . Harry explains that whatever happens, it is ultimately your fault if disaster occurs, because you could have prevented it. You cannot hide behind following the rules, or behind the expectation that someone else was handling it. The world is your responsibility, all of it, personally. Hermione, the closest thing the story has to a normally decent person, objects that this sounds like it leaves very little room for everyone else’s agency. The story never quite answers her objections in a satisfying way.
Shockingly, this fanfic’s popularity far exceeded the readership you’d expect for such a niche artifact. By its completion on Pi Day 2015, the fanfic ran 122 chapters and 661,619 words , longer than War and Peace . It had drawn more than thirty-seven thousand reviews on the main fanfiction site, a wild number for the category, tens of thousands of favorites, a fan-made audiobook, a TV Tropes page the size of a small suburb, and translations into a long list of languages, Russian foremost. A Washington Post legal blogger called it , straight-faced, one of his favorite books written this millennium. The respectable press barely registered any of this, because the respectable press does not read fanfiction. However, a generation of future engineers did.
The book’s real function announced itself at the end. For the finale Yudkowsky posted a chapter titled “Final Exam” , in which Harry is trapped in a seemingly hopeless situation, and the chapter itself issued terms and a deadline: if readers posted a viable solution by March 3, 2015, the story would continue; otherwise, in the author’s words, you get a shorter and sadder ending. The fandom submitted more than 700,000 words of proposed solutions in two and a half days, then organized a task force of its own to sort the submissions and deduplicate the ideas, and the resolution that shipped came out of the crowd. Reread that sequence as an organizational exercise. The founder of an AI-safety research institute recruited a distributed volunteer workforce of thousands, assigned it an optimization problem, imposed a deadline with a threatened adverse outcome, and harvested the output into his book. It worked. The readers had not merely read about the method; they had performed it, at scale, on command. Whatever else the fanfic was, it was a working demonstration of what its author could induce a crowd of smart strangers to do by structuring incentives correctly. Remarkably that sentence also describes the next decade of the AI industry, perhaps not by coincedence.
Outside the story, Yudkowsky’s the movement-building was explicit. Yudkowsky’s author’s notes promoted fundraising for the Center for Applied Rationality, or CFAR, a nonprofit that would soon teach the Sequences’ techniques at live workshops. The fanfic’s finale in 2015 was the occasion for organized celebration parties in dozens of cities. Fiction, fundraising, training, and an international community, all wrapped up together as one. A VICE profile from 2015 already noticed that the whole apparatus was starting to resemble a belief system rather than a popular piece of fiction.
While calling the novel a holy book would overstate it, we’d also miss its function in calling it entertainment. It supplied a shared vocabulary, exemplary characters, and a dramatic theory of what a smart person owes the world in order to convert agreement with its arguments into identification with a mission. Much in the same way that L. Ron Hubbard navigated the transition from science fiction writer to cult leader, Yudkowsky charted a similar path.
Speaking of which: some rationalists deliberately built rituals for the burgeoning cult too. A 2011 account of a New York Solstice describes communal songs, readings, and liturgy adapted from the Sequences, with Yudkowsky commenting approvingly. Years later, Arnold wrote openly about how to run a Solstice when he personally assigns more than even odds to an AI catastrophe.
A movement can reject God and still develop scripture, ceremony, and a demanding personal relationship with the apocalypse. Secularization, it turns out, does not automatically remove the human instincts for religion and liturgy, it just repurposes their worship.
Yudkowsky’s Apostles
Yudkowsky was, by temperament, a prophet, and prophets are demanding company. The community’s real growth engine was someone much easier to like: a psychiatrist and blogger named Scott Alexander.
Alexander was not an outsider who wandered in after getting famous. By his own account he spent years immersed in Overcoming Bias and LessWrong and found the material revelatory despite already having studied philosophy. His blog Slate Star Codex , launched in 2013 , grew directly out of that participation. And it was a very different instrument. Where Yudkowsky was imperious, Alexander was funny, discursive, sympathetic, and apparently scrupulous about entertaining objections. He wrote about medicine, statistics, science fiction, and the baffling behavior of human beings, and his readership came to include Sam Altman, Paul Graham, and a large share of Silicon Valley’s professional class. The New York Times ended up calling the blog a safe space for the tech world .
This mattered because Alexander was the interpreter. You did not have to enter the world of rationalism through a prediction of extinction or Harry Potter fanfiction. You could arrive through an interesting essay about antidepressants, discover a community whose arguments seemed unusually careful and whose members seemed unusually smart, and only later find out what the careful arguments were carefully arguing toward, and slowly integrate into the community.
You could also arrive through Curtis Yarvin.
Writing as Mencius Moldbug, Yarvin began publishing the blog Unqualified Reservations in 2007. The capsule description is a programmer writing long political essays. The project was far more sinister than that description suggests: Yarvin became a founding intellectual of the authoritarian far right, and unlike most of that scene he wrote in complete sentences, which turned out to be a genuine strategic asset.
His program, stated plainly in 2008 , was “the liquidation of democracy, the Constitution and the rule of law.” The new order he sketched in the same extended essay would eliminate the press, dismantle the universities, privatize public schools, and confine designated populations in facilities involving compulsory labor. He also defended, in the same document, the legality of people selling themselves and their children into slavery.
His Patchwork series imagined future sovereign territories operated as businesses. Residents would not govern; their principal recourse would be exit, the customer’s remedy of taking their business elsewhere. The premise converts a political question, ho