Is the AI Safety Movement, Said to be Dead, Coming Back?
A March 2023 petition calling for a pause was said to be dead a year later, but then nine current and former OpenAI employees issued an open letter challenging the company’s AI safety practices
By John P. Desmond, Editor, AI in Business
In March 2023, many luminaries in the AI field signed a petition calling for a six-month pause in AI development. This may have been the peak of the AI safety movement.
On the day the letter was released, researcher Eliezer Yudkowsky published an essay in Time magazine explaining his support for the six-month moratorium. Yudkowsky leads research at the Machine Intelligence Research Institute; he has worked on artificial general intelligence since 2001 and is considered a founder of the field.
His essay stated in part, “It took more than 60 years between when the notion of Artificial Intelligence was first proposed and studied, and for us to reach today’s capabilities. Solving safety of superhuman intelligence—not perfect safety, safety in the sense of “not killing literally everyone”—could very reasonably take at least half that long.”
In a comparison to bridge engineers, he stated, “If we held anything in the nascent field of Artificial General Intelligence to the lesser standards of engineering rigor that apply to a bridge meant to carry a couple of thousand cars, the entire field would be shut down tomorrow.”
He suggested, “We are not prepared. We are not on course to be prepared in any reasonable time window. There is no plan. Progress in AI capabilities is running vastly, vastly ahead of progress in AI alignment or even progress in understanding what the hell is going on inside those systems.”
About a year later, safety-minded employees at OpenAI saw the resignation last month of two senior AI researchers at OpenAI, Ilya Sutskever and Jan Leike, as a setback. Dr. Sutskever as a member of OpenAI’s board had voted to fire Altman, who was later hired back. Dr. Sutskever had helped to create the Super Alignment team inside OpenAI to explore ways to prevent future versions of its technology from doing harm.
In mid-May of this year, economist Tyler Cowen wrote an account in Bloomberg with excerpts on his blog Marginal Revolution that declared the AI safety movement to be dead. The professor at George Mason University stated, “It was a bad idea, and it got nowhere.”
He is more optimistic about the future of AI than Yudkowsky was in 2023. “The influence of the AI safety movement may have waned, but the opportunity to make AI more safe is only just beginning,” he stated.
Cowen sees the greatest obstacles to AI development to be “the hundreds of pending AI regulatory bills in scores of US states. Many of those bills would, intentionally or not, significantly restrict AI development.”
He sees as a good sign that in mid-May, Senate Majority Leader Chuck Schumer (D-NY) released guidance for a federal policy on AI, calling for support of research and development of AI, and a recognition of its national security implications, especially in the competition with China with its goal to achieve superiority in AI.
He noted that OpenAI, Anthropic, Google and Meta all released major service upgrades this spring, offering services that are “smarter, faster, more flexible and more capable. Competition has heated up, and that will spur further innovation.”
Safety Movement Comeback?
This week, the AI safety movement may have been resuscitated to a degree with the publication of a letter from nine current and former OpenAI employees saying that the company is home to a “culture of recklessness” and is not doing enough to prevent AI systems from becoming dangerous, according to an account in The New York Times.
As OpenAI pursues profit and growth, the company has tried to prevent workers from voicing their concerns about the technology, including through restrictions in “nondisparagement agreements” that departing employees were asked to sign, according to the account. One area of concern has been the pursuit of artificial general intelligence (AGI), similar to concerns voiced by the researcher Yudkowsky, that the technology may not be able to be controlled.
“OpenAI is really excited about building AGI, and they are recklessly racing to be the first there,” stated Daniel Kokotajlo, a former researcher in the governance division of OpenAI and one of the dissident group’s organizers. He joined OpenAI in 2022 from a background in AI safety. He had predicted then that AGI might arrive in 2050, but its quick development causes him to now predict with 50 percent certainty that AGI will arrive by 2027.
He refused to sign the nondisparagement agreement on his way out, putting $1.7 million in his vested equity at risk, the Times reported. In their open letter, Kokotajlo and other former OpenAI employees called for an end to the nondisparagement and nondisclosure agreements. They call for AI companies to support “a culture of open criticism” and to support a way to raise safety-related concerns anonymously. They have hired an attorney, Lawrence Lessig, a prominent legal scholar, on a pro bono basis, the Times reported.
The latest developments followed the resignation last month of two senior AI researchers at OpenAI, Ilya Sutskever and Jan Leike. Dr. Sutskever, a cofounder and former chief scientist at OpenAI, as a member of its board had voted to fire Altman, who was later hired back. The departure of Dr. Sutskever was seen as a setback by safety-minded employees, the Times reported. He had helped to create the Super Alignment team inside OpenAI to explore ways to prevent future versions of its technology from doing harm.
UN’s AI for Good Summit Cannot Escape AI Safety Issues
On a potentially more optimistic note, the AI for Good Summit, under the purview of the United Nations and organized by The International Telecommunications Union, was held in Geneva last weekend. The overall theme was to explore how AI can be used to meet the UN’s sustainable development goals.
MIT Technology Review sent a writer to the event. “I didn’t leave the conference feeling confident AI was going to play a meaningful role in advancing any of the UN goals,” stated the writer, Melissa Heikkilä, in her account. The UN’s sustainable development goals include eradicating poverty and hunger, achieving gender equality, promoting clean energy and climate action.
She found the most interesting talks to be around AI safety themes, with climate activist Sage Lenier talking about preventing AI from accelerating environmental destruction and Tristan Harris, the cofounder of the Center for Humane Technology, talking about connecting the dots between addiction to social media and the tech sector’s financial incentives. Mia Shah-Dand, the founder of Women in AI Ethics, reminded her audience about still deeply ingrained gender biases in tech.
“So while the conference itself was about using AI for ‘good,’ I would have liked to see more talk about how increased transparency, accountability, and inclusion could make AI itself good from development to deployment,” Heikkilä stated.
OpenAI’s CEO Sam Altman was interviewed remotely. Heikkilä characterized it as a “vague, high-level discussion about safety, leaving the audience none the wiser about what exactly OpenAI is doing to make their systems safer.”
She referred to a recent interview with Helen Toner, a researcher at the Georgetown Center for Security and Emerging Technology and a former member of OpenAI’s board, in which Toner stated that Altman “had on multiple occasions given the board inaccurate information about the board’s formal safety processes.”
Toner also argued that it is a bad idea to let AI firms govern themselves, because the immense profit will “always” prevail.
Read the source articles and information in Time, in the Marginal Revolution blog, in The New York Times and in MIT Technology Review.