A brief history of AI executives calling for regulation

The industry is in a panic, but it’s been claiming to want legal guardrails for years — with few meaningful results. The industry is in a panic, but it’s been claiming to want legal guardrails for years — with few meaningful results. Over the past few days, a lot of people who stand to make a lot of money from AI allpublicly agreedthat it’s time to make everyone slow down before we lose control — including OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei, Google DeepMind cofounder Demis Hassabis, Microsoft CEO Satya Nadella, and X CEO Elon Musk. When people who profit from something declare that it’s dangerous and needs to be regulated,there’s always reason to be skeptical. But whatever their reason, this is far from the first time AI thought leaders have sounded the alarm. Charles Darwin’s theory of evolution prompted author Samuel Butler to warn about intelligent self-replicating machines replacing humansback in 1863— and in his follow-up1872 bookErewhon, he describes visiting a fictional society that has already regulated machines out of existence to avoid humanity’s collapse. Computing pioneer Alan Turing famously warnedin a 1951 lecturethat artificial intelligence would “meet with great opposition,” because undying machines “would not take long to outstrip our feeble powers” and would eventually “take control.” Calls for intervention, too, aren’t new. In 2000, Sun Microsystems cofounder and chief scientistBill Joy wrotethat self-replicating robots could be more dangerous than nuclear weapons, partly because they’re being developed “within the now-unchallenged system of global capitalism” rather than in tightly controlled government labs. Andin 2009, Microsoft researcher Eric Horvitz, who now serves as the company’s chief scientific officer,brought leading AI researchers togetherto consider “creating policies that might constrain or bias the behaviors of autonomous and semi-autonomous systems so as to address concerns.” But modern AI bigwigs warning that their investments should be controlled? That trend appears to have begun with Elon Musk, who invested earlier than most. By the time Elon Musk told a gathering of US governors that they needed to regulate AI right away, the SpaceX and Tesla CEO already had skin in the game. He’d alreadyinvested $38 million in OpenAI, as well as stakes in at least two other AI companies, and he’d publicly spoken about the dangers of AI for three years. In June 2014, he suggested AI researchcould produce a real-life Terminator, called AI “potentially more dangerous than nukes” that August, and suggested that working on AI was “summoning the demon” that October. In January 2015, he joined Stephen Hawking in signingan open letterasking for responsible AI research; Hawking had previously warned that the creation of AImight be the last thing humans ever do. “AI is a rare case where we need to be proactive about regulation instead of reactive. Because I think by the time we are reactive in AI regulation, it’s too late,” Musk told a gathering of US governors at the July 2017 National Governors Association summer meeting. InMarch 2018, he added: “AI is far more dangerous than nukes. So why do we have no regulatory oversight?” In the wake of Facebook’s Cambridge Analytica data scandal, the Meta founder suggested his company should be regulated — if it’s the “right” regulation. It was largely not about AI, but he did go there: Now that companies increasingly over the next five to 10 years, as AI tools get better and better, will be able to proactively determine what might be offensive content or violate some rules, what therefore is the responsibility and legal responsibility of companies to do that? InMarch 2019, he added that the internet needs a “more active role” for regulators. Earlier in 2018, Microsoft began taking fire forworking with ICE on facial recognition technologies, at a timemany tech CEOs had condemned Donald Trump’s policyof separating children from parents who are arrested at the US border. While this form of AI isn’t quite the same as self-replicating intelligence, Microsoft president Brad Smith opened the door to voluntary regulation in a speech at the Brookings Institution, saying, “We believe it’s important for governments in 2019 to start adopting laws to regulate this technology. … We believe that the only way to protect against this race to the bottom is to build a floor of responsibility that supports healthy market competition. And a solid floor requires that we ensure that this technology, and the organizations that develop and use it, are governed by the rule of law.” “[T]here is no question in my mind that artificial intelligence needs to be regulated. It is too important not to,” wrote Pichai in aneditorial for theFinancial Times. “The only question is how to approach it.” “Companies such as ours cannot just build promising new technology and let market forces decide how it will be used,” he added. Like with Microsoft a year earlier, these comments emerged around fears about AI facial recognition specifically. In 2023, as generative AI fervorbegan to reshape the entire tech industry, scientists began suggesting that companies shouldslow downtheir research — or else. “If such a pause cannot be enacted quickly, governments should step in and institute a moratorium,”the letter, published by the nonprofit Future of Life Institute and signed by Musk, among others, reads. Two weeks later, Musk announced he’d createdhis own new AI company. Two years later,he would use DOGEto dismantle chunks of the federal workforce instead of helping to regulate anything. Brought before the US Senate for a series of AI hearings, Altman agreed that Congress should create a new agency to regulate artificial intelligence. “We believe that the benefits of the tools we have deployed so far vastly outweigh the risks,” he argued. “However, we think that regulatory intervention by governments will be critical to mitigate the risks of increasingly powerful models.” In another hearing onMay 25th, Anthropic’s CEO warned US senators that AI could help create bioweapons. When OpenAI’s CEO was before Congress,we notedthat the tone was friendly and they seemed to be trying to shape legislation together; here is an example of Microsoft trying to shape future regulation as well. One regulatory example: “New laws would require operators of these systems to build safety brakes into high-risk AI systems by design. The government would then ensure that operators test high-risk systems regularly to make certain that the system safety measures are effective.” “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.” While the 22-word statement doesn’t explicitly ask the government to step in, it’s hard to think world leaders wouldn’t raise their eyebrows even higher than before. OpenAI CEO Sam Altman and then-Google DeepMind CEO Demis Hassabis were among the cosigners. In a private Senate meeting, closed to the public and media, tech CEOs reportedly agreed that the AI industry needed regulation. One set of remarks was not so private: Meta CEO Mark Zuckerberg said that “Congress should engage with AI to support innovation and safeguards.” “We think policymakers, academics, civil society, and industry should all work together to minimize the potential risks of this new technology, but also to maximize the potential benefits.” Between July 2023 and May 2024, tech companies signed a flurry of nonbinding agreements with governments that would supposedly keep them in check: Also inSeptember 2023: A book from Inflection AI CEO Mustafa Suleyman — who is now Microsoft’s AI chief — argued that governments should regulate. “This is a tough call and will make some people upset, but, all things considered, I think California should probably pass the SB 1047 AI safety bill,”Musk posted in August. Anthropic originallycame out against itin July,as did OpenAIin August, but Anthropic changed its tunein September. California Gov. Gavin Newsomvetoed SB 1047,though, stating that “I do not believe this is the best approach to protecting the public from real threats posed by the technology.” “Governments should urgently take action on AI policy in the next eighteen months. The window for proactive risk prevention is closing fast,” Anthropic wrote in an open letter (bolding theirs). “It is critical over the next year that policymakers, the AI industry, safety advocates, civil society, and lawmakers work together to develop an effective regulatory framework,” the company added. The document begins by suggesting the UK stunted the development of its own automobile industry with regulation, while proposing “democratic AI” instead: “Rules and regulations for the development and use of AI should be based on the democratic values the country has always stood for.” In a guest essay forThe New York Times, Anthropic CEO Dario Amodei wrote: “Federal law does not compel us or any other A.I. company to be transparent about our models’ capabilities or to take any meaningful steps toward risk reduction. Some companies can simply choose not to.” The essay isagainst the Trump administration’s proposed 10-year moratoriumon US states regulating AI. He wasn’t arguing for a slowdown, though, but rather a “national transparency standard” that “would help not only the public but also Congress understand how the technology is developing.” AsCalifornia tried to regulate its AI giants again, Anthropic threw its support in early, andthis one actually became law. It’s a transparency standard, like the one Anthropic was asking for, with safety reporting requirements and whistleblower protection. OpenAI lobbied against it. This summer, shit hit the fan. It became public thatrogue AI agents can—and did— attack other companies.OpenAI said it hit the brakesin August, and on September 8th, a senior Anthropic safety researcher resigned in protest, declaring there’sa 10 percent chance AI could kill all humans by the end of the decade. Another Anthropic safety researcher agreed. Now, suddenly, the AI industry has seemingly agreed to slow down the pace of development. More: Is this a new call for regulation? From Anthropic’s CEO, it seems to be.In his essay“We Must Pace the Frontier,” Amodei writes that “The most effective method of pacing is via regulation that targets all US frontier AI companies, as that covers even those who are unwilling to cooperate voluntarily,” and he suggests governments might embed people inside AI companies to help prevent incidents. It’s not clear every AI company wants to go that far. “Where we will need the help of our government is for international coordination. But first we should do what we can ourselves,” OpenAI CEOSam Altman writes. ButAltman does saythat he’ll commit to “independent evaluators with employee-like access” as well. Meta’s Mark Zuckerberg also doesn’t seem to be aligned. On September 15th,he postedthat each company has its ownindividualresponsibility to move at the right pace and that Meta is already doing so. He also prominently linked back tohis manifesto from August,which contains this line: “Any policy that slows American model releases -- even by a month -- could add significant risk to American leadership while letting foreign models race ahead.” Regardless of what tech leaders are saying, regulation seems highly unlikely to come under the Trump administration,which has signaledit wants the industry barreling full speed ahead. Even if these companies don’t actually want the regulation they seem to be asking for, this is one of the loudest alarm bells they’ve sounded so far — and the world is watching. A free daily digest of the news that matters most. This is the title for the native ad