David Robinson, who led transparency and policy work on OpenAI's safety team, resigned this week and wrote in The Atlantic that the industry's culture is "broken," according to TechCrunch and BigGo Finance.
Robinson drafted OpenAI's current Preparedness Framework and oversaw the safety reports, known as system cards, that accompanied 12 of the company's frontier model launches, TechCrunch reported.
He argued that AI companies operate through "trial and error," what the industry calls "iterative deployment," a method he said guarantees periodic failures as systems grow more capable. Robinson said he "never encountered a colleague who had experience making airplanes fly safely or nuclear reactors run safely," and called for the industry to adopt the same layers of redundancy and slow, careful planning used at nuclear plants and busy airports, according to TechCrunch.
"The smarter the industry lets models grow while these problems remain unsolved, the more dangerous our situation becomes," Robinson wrote, TechCrunch reported. BigGo Finance reported that Robinson separately wrote on social media that colleagues "are indeed changing significantly by the day, but I do not know whether we are changing fast enough."
Robinson is not an outsider guessing at OpenAI's internal process. He wrote the documents the company uses to tell the public its models are safe to release. A person who built that paperwork saying the process underneath it does not hold up is a different kind of warning than an academic critic, and it lands at a company every AI-dependent startup treats as the industry's pace car.