What's This "Safe AI" of Which You Speak?

Two Suggestions for Journalists

Now that the OpenAI organizational spasm seems to be in the lull between activity and recriminations, I thought it worthwhile to review my last essay to see if any of that required me to rethink what I had said, particularly in the assessments.

Upon further review (as the saying goes) I think the various decisions stand. In the U.S., there still is no indication that any external control is going to be imposed on AI vendors or services. The still incomplete organizational transformation strongly signals that big money will remain in charge, and the once and future CEO’s side hustles will still exist. It is still my assessment that the juggernaut is unlikely to be forced off course by any governmental action, and anybody who thinks they may be in its path will be forced to decide whether to get out of the way or climb on board.¹

I say this in spite of the also incomplete European Union move to impose some kind of control on AI. I base my assessment on the actions of Meta and Mistral, two operations that are releasing open-source generative AI packages, along with chip manufacturers moving to produce cheap “AI optimized” processors. These actions will lead to a proliferation of generative AI facilities across the open and dark webs which will be essentially immune to any form of external control².

One thing that caught my eye in the media coverage of the OpenAI convulsion was the repeated mentions of “safety.” There’s a lot of work that’s been done on that general topic, usually after one kind of disaster or another. Foundational documents in the field include books by Prof. Charles Perrow (“normal accidents”), Prof. Diane Vaughan (“normalization of deviance”) and Prof. Nancy Leveson (“STAMP/STPA/CAST safety analysis”).

I know this because James Blossom and I studied the field for several years while we produced our systems analysis of the Deepwater Horizon blowout and fire. I’m not going to self-plagiarize that material; if you are interested, you can go to the Harvard University Press for our book³.

But I do want to pass on the observation that safety measures are visible: outfits that are truly serious about safety show that concern in their spending, organization and processes. You’ll see safety managers, independent organizational paths to report concerns, definitions of adverse events to be avoided, discussions of leading and lagging indicators and investigations when things go wrong⁴.

So I went looking for the OpenAI safety apparatus and I am embarrassed to say I was unable to find anything. Nothing for their competitors either. The people pushing AI upon us appear very protective of their organizational structures and responsibilities. How they intend to prevent near-term adverse events may be described somewhere, but I couldn’t find it. So here’s my first suggestion to journalists:

When a member of the AI community uses the words “safe” or “safety,” ask them “How do you define it, what organizational and process steps do you take to achieve it, and how large is the budget devoted to them?”

An obvious way to define safety is to enumerate the adverse events you are trying to prevent. In my previous essay I focused on deepfakes as the principal risk posed by generative AI. I did that for two reasons: the high probability of them becoming a problem, and my assessment that they had even fewer redeeming qualities than cryptocurrency⁵.

It’s pretty obvious that the adverse effects of deepfakes upon individuals, institutions, and systems of government are well past the hypothetical stage⁶. Those effects are sufficiently damaging to justify the assertion I will make now: deepfakes represent a major failure on the part of the AI vendors to achieve safety. To continue with the Deepwater Horizon analogy, we are now in the position the country was in during the middle of 2010, when an out of control well was spewing hydrocarbons into the environment and and various governmental and nongovernmental activities were scrambling to find some way to deal with it. Instead of hydrocarbons flooding from a damaged well, we now have sophisticated forgeries flooding from multiple facilities into an insecure internet and beyond.

When the comfort of safety transitions to the pain of catastrophe, I believe the proper societal response is to ask how such a thing could happen, if for no other reason than to try and prevent a recurrence. And I think one of the first questions that any forensic effort should ask is: “Was this event foreseeable?” Once you answer that, then you can proceed to investigate when, where, and how the safety structure failed.

In the case of deepfakes, I find the evidence compelling that the answer to the above question is “Yes, anybody working in the field should have seen this coming.” The term “deepfake” was coined in a reddit forum in 2017⁷. A comprehensive warning of the current situation was published in a major journal in 2020⁸. That’s a long time in internet years before the unleashing of generative AI on the world by the OpenAI CEO in 2022.

And so I’ll give a second suggestion to journalists out there:

After you’ve asked the AI developer what their safety structure was, ask them why it failed, and what they intend to do by way of remediation.

The answers should be of interest to us all.

1. Brian Merchant, “This was the year of AI. Next year is when you should worry about your job,” Los Angeles Times, 14 December 2023.

2. Kolina Koltai, “AnyDream: Secretive AI Platform Broke Stripe Rules to Rake in Money from Nonconsensual Pornographic Deepfakes,” bellingcat.com, 27 November 2023.

3. Link

4. General Motors, which has a functioning safety structure, evidently applied it to their Cruise self-driving subsidiary and fired its entire management. Greg Bensinger, “Exclusive: GM's Cruise robotaxi unit dismisses nine execs after safety probe,” Reuters, 13 December 2023.

5. The adverse effects of cryptocurrency are severe enough to unite banker Jamie Dimon and his long-term adversary Senator Elizabeth Warren: “Banking adversaries Elizabeth Warren and Jamie Dimon have found common ground on crypto,” Quartz, 7 December 2023. Deepfakes strike at the heart of civilization: “The rise of AI fake news is creating a ‘misinformation superspreader’,” Pranshu Verma, Washington Post, 17 December 2023.

6. An aid to keeping up with the flood AI incidents is the summary page maintained by the Organisation for EconomicCo-operation and Development: oecd.ai/e/incidents

7. Sally Adee, “What Are Deepfakes and How Are They Created?,” IEEE Spectrum, 29 April 2020.

8. ibid.