
Anthropic goes public today. Many ideas and perspectives have shaped the company’s approach to AI, but perhaps the most influential is effective altruism (EA), a 21st-century social movement that seeks to find optimal strategies for helping others. EA’s influence extends to the very top: CEO Dario Amodei is a former housemate of Holden Karnofsky, who co-founded the largest EA philanthropy organization and led it for several years; in turn, Karnofsky is married to Dario’s sister Daniela, who co-leads Anthropic as its president. EA’s influence also reaches the company’s rank and file. On the EA Forum, members are already discussing how to help employees direct their immense post-IPO wealth to support AI safety. Evidently, effective altruists expect many newly minted Anthropic multimillionaires to take heed.
So how exactly does EA think about AI? In theory, the movement is dedicated to reducing AI risks and ensuring that humanity survives the arrival of intelligent machines. In practice, its approach to addressing those risks is deeply flawed. First, effective altruists have focused largely on the technical question of how to align AI systems with human values, and have continued to prioritize this pursuit despite disappointing results so far. Second, the EA community is socially and institutionally intertwined with the AI industry, to a degree that perhaps shapes its reluctance to confront the companies head-on, including through effective political advocacy. Most worryingly, many EAs seem comfortable with the idea of handing over control of humankind to future “aligned” AIs, or even letting AIs replace us altogether under some circumstances. All of this is colored by a strong EA preference for building superhuman AI “safely” over the alternative of not building it at all.
These problems with EA are made dangerous by its dominance within AI safety. Many AI safety organizations grew out of the EA movement, and many individuals working in the field were channeled to it through EA organizations. As a result, the AI safety community is relatively homogeneous in its thinking about how to reduce AI risks, and it is difficult for alternative approaches to gain traction. In other words, the AI safety community may be too blinkered to recognize the best ideas.
This is an unacceptable state of affairs. AI safety needs a new movement that champions human flourishing, asserts human political control over technological development, and rejects the inevitability of technology that disempowers humans. This humanism must take into account the full gestalt of human flourishing, not just a narrow calculation based around maximizing pleasure and minimizing pain.
This article, adapted from a previous essay covering these and other related topics in greater detail, will outline beliefs commonly held among EAs and briefly explain how the EA movement came to be dominant within AI safety. It will then describe some of the main ways in which the EA approach to AI safety has failed, and how it can in some cases promote an explicitly anti-human ideology. Finally, as an alternative to EA-led approaches, this article will set out the tenets of a new humanist movement that could offer a better way to think about AI safety at this critical juncture.
Effective altruism developed in the 2000s, with a focus on philanthropic effectiveness. The EA movement took shape at universities in the late 2000s, growing from a combination of utilitarian philosophy—which advocates for helping others as much as possible—and the rationalist community—which emphasizes logic and overcoming biases to gain a clearer understanding of the world. Together, these produced a community of people who sought to use rigorous evidence to calculate and compare the effectiveness of various charitable interventions, to inform philanthropic donations. The name “effective altruism” was coined in 2011.
The most distinctive EA principle is the “view from nowhere.” Drawing on utilitarianism, EA argues for radical impartiality. Instead of prioritizing causes that are highly visible, or that affect people close to you, you should take a more detached view: the life of someone on the far side of the world matters just as much as your cousin’s, and you’re morally obligated to weigh the two equally. Per this principle, all beings capable of experiencing pain and pleasure matter to our moral calculus, proportionally to the degree and intensity at which they experience such phenomena. The most obvious application of this is to animals, explaining why animal welfare is a major EA cause area.
Many effective altruists are longtermists, giving equal moral weight to future beings. If the view from nowhere insists on equal valuations among individuals alive today, longtermism extends this principle to those who may live in the future. In other words, hypothetical future humans millions or billions of years in the future ought to be considered alongside existing humans in calculations of trade-offs.
EAs were among the first to take AI danger seriously. Thanks to their rationalist influences, EAs were willing to examine AI risks from a logical perspective, rather than dismissing them out of hand at a time when they may have sounded absurd. Combined with a concern for the long-term future, this meant that EAs were early to study AI risks in earnest; EAs and rationalists have spent the past 25 years studying exactly how and why AI systems might begin to act in harmful ways and thinking through proposed methods of aligning AI systems so they do not harm humans.
EA played a key role in seeding today’s top three AI companies. Having been early to recognize AI risks, EAs sought to ensure the technology would be developed by responsible actors. EAs were therefore involved in the creation of several AI companies. For example, Eliezer Yudkowsky, one of the founders of the rationalist community, introduced the founders of DeepMind to venture capitalist Peter Thiel in 2010, resulting in a $2.25 million investment by Thiel—the company’s first capital raise. In 2015, a group of Silicon Valley magnates including Sam Altman and Elon Musk co-founded OpenAI, motivated by concerns that Google (which by then owned DeepMind) would use powerful AI to malevolent ends. Both Musk and Altman had read Superintelligence by Nick Bostrom, a philosopher whose ideas shaped the early EA movement, and Musk has said that the longtermist book What We Owe the Future is “a close match for my philosophy.” Then, in 2021, seven former OpenAI employees founded Anthropic, promising to focus more on safety than the other AI companies.
EA remains influential within AI companies and the AI safety community. Since the EA movement steered many of its members toward working on AI safety when the field was small, the AI safety community today functions in practice as a subset of the broader EA community. But when almost everyone working on a critical problem thinks in very similar ways, there is a risk that they stake everything on a single solution, with disastrous consequences if it fails—and there’s good reason to believe that the EA approaches are failing.
The EA community deserves credit for thinking earnestly about AI danger at a time when most would have dismissed it as science fiction. Unfortunately, however, the people who are first to understand a problem don’t always have the best ideas for solving it.
Prominent EAs believe that building superintelligence is a moral imperative. Despite being aware of the risks of artificial superintelligence (ASI), EAs often take it as a given that ASI should eventually be built, because of the enormous potential good it could do if properly aligned. Bostrom and EA philosopher Carl Shulman, for instance, write that “superintelligent digital minds should be brought into existence and be allowed to thrive and play a major role in shaping the future.” AI safety writer Scott Alexander recently wrote that banning the creation of superhuman AI forever would mean “consigning us to a permanent civilizational childhood or an early grave.” Even the Machine Intelligence Research Institute (MIRI), the single EA organization most amenable to an interim ban on superintelligence, has said that “we would consider it an enormous tragedy if humanity never developed artificial superintelligence.”
Some EAs are successionists, or have successionist tendencies. The arguments in favor of creating ASI often focus on how it could help humans. However, some EAs believe that, if AIs themselves were capable of experiencing more happiness than humans, it would be acceptable—or even morally obligatory—for humanity to step aside and allow digital minds to rapidly copy themselves, displace humans, and propagate across the galaxy to attain a much larger, happier population than is feasible for biological humans. The idea that humans should cease to exist and allow AIs to replace us is called successionism. To be clear, most EAs are not successionists. However, a minority are successionists or come awfully close. The EA community is unusually fond of “biting the bullet” on uncomfortable or absurd-seeming conclusions, rather than rejecting them purely on the basis of their absurdity. This allows the successionist minority to argue that non-successionist EAs are betraying their own EA principles.
EAs focus on alignment, rather than questioning the AI race altogether. Because EAs generally assume that ASI must be built, their approach to AI safety centers on aligning AI, to the exclusion of other approaches that might demand stopping the AI race altogether.
Yet, thus far, AI alignment as a field appears to have failed at achieving its goals, at both the theoretical and empirical levels. For example, a subfield called agent foundations, which seeks to understand agents’ behavior from a theoretical point of view, appears to have stalled out after more than a decade of work. Meanwhile, Anthropic’s Alignment Science lead, Evan Hubinger, has admitted that “we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”
On the empirical side, experimental attempts to understand and align AI models may have done more to advance AI capabilities than to align AIs’ behavior. For example, reinforcement learning from human feedback (RLHF) was initially developed as a way to teach AI models human values through examples, when values are too nuanced to write as simple rules. Yet RLHF can also be used to make AI more capable and useful in general, which paved the way for ChatGPT and all that has followed. Last month, OpenAI scientist Nick Cammarata released (hotly debated) research that some have treated as a breakthrough in mechanistic interpretability, which aims to understand AIs’ internal thought processes. Within only a few days, a capabilities researcher had used Cammarata’s results to demonstrate a way of boosting training efficiency, enabling the training of more powerful models than before with the same amount of compute. Both examples show that empirical alignment work is inherently dual-use.
EA is plagued by conflicts of interest with the companies racing to build ASI. The EA community’s soft-pedal approach toward AI companies, particularly Anthropic, has almost certainly been shaped by EAs’ links to the AI industry. Numerous prominent EAs were early Anthropic employees and investors, thus holding substantial equity. This may create incentives to overlook or even justify the company’s violations of its own commitments, and the fact that it has done more than any other AI company to accelerate the race toward recursive self-improvement—in which an AI model trains its own successor with no human involvement.
Meanwhile, former OpenAI researcher and influential EA Paul Christiano recently revealed that he had signed a non-disparagement agreement with OpenAI back in 2021. This was unknown to the public before his disclosure; during this time, he began working in a senior government AI safety role. Christiano is married to Ajeya Cotra, a key member of technical staff at METR, the organization tasked with the third-party investigation of the attack on Hugging Face by OpenAI’s rogue agent swarm. He also recently joined an OpenAI board.
More broadly, it is impossible to overstate how deeply entangled the EA community is with the social world of frontier AI developers. The revolving door of hiring between frontier AI companies and EA-backed alignment research organizations is just the tip of the iceberg. In many cases, the two groups share office space (such as at Constellation), go to the same parties (such as at Lighthaven), and even live in the same group houses. The fact that most EAs working in AI safety have close friends in frontier AI companies helps to explain why many EAs have been reluctant to criticize those companies.
EAs’ forays into politics have been too little, too late, and badly misdirected. The EA community has often been hostile to political engagement, perhaps due to a fondness for “epistemic purity”—the idea that one should only engage in actions that are maximally truth-seeking and that improve the quality of discourse. Needless to say, politics is not epistemically pure. Politicians and interest groups bloviate, lie, overpromise, and mislead. But politics is how laws are made. Some EAs have begun to appreciate this, but even now, their political efforts are mostly too little, too late, and focused on the wrong targets. ControlAI, an organization dedicated to political lobbying in pursuit of a ban on ASI, characterizes EA’s approach to politics and persuasion as secretive, anti-democratic, and at times deliberately misleading. To the extent that EA philanthropic funds do support political advocacy, much of this advocacy is fighting the wrong battles. For instance, Coefficient Giving (formerly Open Philanthropy) is a major funder of the Foundation for American Innovation, a broadly pro-AI-industry think tank whose fellows have argued against pausing frontier AI and against banning superintelligence.
One analysis from 2025 estimated that the EA community had only one person working on advocacy for every three working on governance research. Since much of this advocacy is misdirected—advocating faster AI progress, or playing the ineffective “inside game” of advocacy aimed at AI companies rather than at governments—the ratio, in my view, is actually even worse than it initially appears. The inside game has not achieved its goals. Alignment research commitments demanded by inside-game-focused EAs have been walked back; companies’ responsible scaling policies have been watered down; the push for voluntary “pacing” has rung hollow, with OpenAI and Anthropic still racing each other to release ever-more-capable AI models at an ever-greater cadence. It’s now clear that AI companies will not voluntarily constrain their own behavior—the force of law is necessary.
Many EAs actively cheer for the prospect of humans handing over control to AIs. In July, the AI Futures Project released “AI 2040,” whose “Plan A” they framed as a “positive vision” and policy recommendation. While the AI Futures Project has done excellent work forecasting the trajectory of AI, this does not extend to their policy proposals: their Plan A envisions humanity “passing the torch to AIs” by 2040. The EA community’s reaction to this proposed outcome, though, was overwhelmingly positive. I was unable to find any categorical criticism of the idea of handover from within the EA community—only objections that the proposed handover was too sudden.
More recently, in response to criticism from outside EA, Scott Alexander argued at length in favor of handover, writing that “I’d rather let [an aligned ASI] choose its own direction for itself and for humanity than have it be under the control of the US president, or a shareholder vote, or anything like that.” Ruxandra Teslo recently estimated that 30–40% of the AI safety community outside the AI companies supports handover. Internal documents from Anthropic positing a goal of “utopia ruled by god in a box,” alongside CEO Dario Amodei’s essay “Machines of Loving Grace,” suggest some level of institutional support for this view. Even more alarming is widespread EA advocacy for “AI welfare.” Given EAs’ total-utilitarian views, demands for welfare—and rights—for AI systems could quite easily lead to human disempowerment.
For all its prescience about AI danger, EA is no longer fit to lead AI safety. It’s time for a new humanist movement—21st-century humanism—that explicitly centers human flourishing, and provides a constructive way to think about transformative new technologies—including AI as it approaches and surpasses human intelligence. So, what should this movement look like?
21st-century humanism explicitly centers human flourishing. Human flourishing must be the focal point of efforts to shape society. Humanity is the point. We cannot, and will not, have our existence traded away for the sake of “digital minds.” Some conclusions, such as successionism, are so absurd that they must be rejected outright, regardless of what line of argumentation produced them. Similarly, we must reject the distraction of “AI welfare,” which, if taken seriously, poses a genuine disempowerment threat to humanity. Human flourishing is not just about material plenitude or the minimization of suffering; it also encompasses such values as liberty, autonomy, and self-governance.
Technology is not inherently good; it’s good only insofar as it benefits humanity. Technology ought to be seen as a means to enable human flourishing. We must reject Silicon Valley’s cult of the “builder,” in which building technology is inherently valuable, regardless of the consequences, and building something that corrodes society is admired more than not building anything.
Some technologies will have a mix of good and bad impacts, both of which must be carefully considered. I do not pretend that we will all weigh trade-offs the same way, or have identical definitions of human flourishing. Regardless, the point is to center human flourishing and to avoid the development of technologies that clearly threaten it, rather than to treat all technology as an inherent good just because it is technology. For instance, one startup is working on AI-powered mind-reading that would enable governments and institutions to surveil our private thoughts, abolishing free thought in a way no totalitarian regime has ever achieved. That alone is sufficient grounds to prohibit such a technology altogether.
People cannot opt out of technological revolutions individually. As technology diffuses through society, individuals often have less practical freedom to choose whether or not to adopt it. For example, many have tried to switch to “dumb phones” with a minimal feature set, to curb internet addiction and avoid endless scrolling. Unfortunately, as many have found out firsthand, this creates enormous friction in everything from staying in touch with friends to dealing with routine matters like downtown parking and ubiquitous two-factor authentication. As a result, most dumb phone adopters end up returning to smartphone ownership within a few months. Regarding AI, Dario Amodei has speculated that those who individually opt out of the AI-driven transformation may be left behind, potentially coming to resemble a “dystopian underclass.” Pressures like this could create a slippery slope toward gradual disempowerment of humanity.
Technology is and ought to be a social and political question. Since AI will affect us all, the public deserves a say over how—and even more importantly, whether—it is developed and deployed. We must not sit idly by and let a few technologists reshape our future at will, especially when their vision threatens to render almost everyone unemployable forever in just a few years, or to abolish the ironclad privacy of one’s own thoughts. We must demand that our elected officials treat technology as a choice rather than an inevitability, and prioritize political economy, individual freedoms, and basic human dignity over the pursuit of technological progress at any cost.
Arguments about the “inevitability” of technology are irrelevant. Some justify the race toward superhuman AI by insisting that it’s inevitable. This justification fails for two reasons. First, it might be false. We haven’t yet attempted serious political action to stop the race, either domestically or globally. Predictions of “inevitability” may well be based on false assumptions. Second, even if the development of superintelligence (or some other technology) is inevitable, it doesn’t follow that we’re obligated to simply let it rip. Delaying, rather than accelerating, a detrimental technology provides us all with more high-quality time before it arrives and unleashes its negative impacts.
We need concerted political action to stop the race toward superhuman AI. Current AI models are already beginning to cause upheaval, most notably through rampant cyber chaos. We may be about to face much worse disruption, such as AI-assisted bioterrorism and an upending of the employment landscape. As the technology matures, threats of AI-run dystopian futures and even human extinction are also coming into view. We have plenty to be worried about, but infighting over the most important dangers has too often wasted valuable time and effort. We are all in this together, and we all share key goals. The risks that AI companies are currently imposing on society are unacceptable; therefore, we need to take concerted action to stop their reckless behavior via the force of law. The “inside game” of attempting to influence AI companies has clearly failed; we need to take the opposite approach, and force change from the outside. In my view, we must demand a permanent, international ban on artificial superintelligence, recursive self-improvement, and near precursors of both.
21st-century humanism is not a totalizing political movement. It does not claim to have an answer to tax rates, abortion, foreign policy, or immigration. It is fundamentally about the relationship between humankind and technology in an age where sci-fi is becoming reality. We must take our destiny into our own hands to face the challenges of our time and ensure a durable, flourishing, human future.

Most AI biorisk studies look at how AI models might help novices complete biological tasks. But experts with AI assistance could be far more dangerous.

Nationalizing AI would do more harm than good. Narrower government powers, like taxation and the right to pause AI, can address issues from inequality to catastrophic risk.