Microsoft and the Taylor Swift genAI deepfake problem
The last few weeks have been a PR bonanza for Taylor Swift in both good ways and bad. On the good side, her boyfriend Travis Kelce was on the winning team at the Super Bowl, and her reactions during the game got plenty of air time. On the much, much worse side, generative AI-created fake nude images of her have recently flooded the internet.
As you would expect, condemnation of the creation and distribution of those images followed swiftly, including from generative AI (genAI) companies and, notably, Microsoft CEO Satya Nadella. In addition to denouncing what happened, Nadella shared his thoughts on a solution: “I go back to what I think’s our responsibility, which is all of the guardrails that we need to place around the technology so that there’s more safe content that’s being produced.”
Microsoft weighed in on the issue of deepfakes again yesterday (though without mentioning Swift). In a blog post, Microsoft Vice Chair and President Brad Smith decried the proliferation of deepfakes and said the company is taking steps to limit their spread.
“Tools unfortunately also become weapons, and this pattern is repeating itself,” he wrote. “We’re currently witnessing a rapid expansion in the abuse of these new AI tools by bad actors, including through deepfakes based on AI-generated video, audio, and images. This trend poses new threats for elections, financial fraud, harassment through nonconsensual pornography, and the next generation of cyber bullying.”
Smith pledged “a robust and comprehensive approach” from Microsoft, adding: “We’re committed to ongoing innovation that will help users quickly determine if an image or video is AI generated or manipulated.”
As far as it goes, the Microsoft view is certainly true, and is the typical all-purpose, knee-jerk response one would expect from the world’s biggest and most influential genAI company. But what Nadella and Smith left out is that there’s evidence the company’s AI tools created the Swift images; even more damning, a Microsoft AI developer says he warned the company ahead of time that proper guardrails didn’t exist, and Microsoft did nothing about it.
Evidence that Microsoft tools were used to create the deepfakes comes from a 404 Media article, which claims they originated in a Telegram community dedicated to creating “non-consensual porn;” it recommends that Microsoft Designer be used to generate the porn images. The article notes that “Designer theoretically refuses to produce images of famous people, but AI generators are easy to bamboozle, and 404 found you could break its rules with small tweaks to prompts.”
More damning still, a Microsoft AI engineer allegedly warned Microsoft in December that the safety guardrails of OpenAI’s image generator DALL-E, the brains behind Microsoft Designer, could be bypassed to create explicit and violent images. He claims Microsoft ignored his warnings and tried to get him to not say anything publicly about what he found.
The engineer, Shane Jones, wrote in a letter to US Sens. Patty Murray (D-WA) and Maria Cantwell (D-WA); Rep. Adam Smith (D-WA), and Washington state Attorney General Bob Ferguson that he “discovered a security vulnerability that allowed me to bypass some of the guardrails that are designed to prevent the [DALL-E] model from creating and distributing harmful images…. I reached the conclusion that DALL·E 3 posed a public safety risk and should be removed from public use until OpenAI could address the risks associated with this model.
“The vulnerabilities in DALL·E 3, and products like Microsoft Designer that use DALL·E 3, makes it easier for people to abuse AI in generating harmful images. Microsoft was aware of these vulnerabilities and the potential for abuse.”
Jones claimed Microsoft refused to act, posted a public letter about the issue on LinkedIn, and then was told by his manager to delete the letter because Microsoft’s legal department demanded it.
In his letter, Jones mentions the explicit images of Swift and says, “This is an example of the type of abuse I was concerned about and the reason why I urged OpenAI to remove DALL·E 3 from public use and reported my concerns to Microsoft.”
According to GeekWire, Microsoft in a statement said the company “investigated the employee’s report and confirmed that the techniques he shared did not bypass our safety filters in any of our AI-powered image generation solutions.”
All of this is, to a certain extent, circumstantial evidence. There’s no confirmation the images were created with Microsoft Designer, and we don’t know whether to trust Microsoft or Jones. But we do know that Microsoft has a history of downplaying or ignoring the dangers of genAI.
As I wrote last May, Microsoft slashed the staffing of a 30-member team that was responsible for making sure genAI was being developed ethically at the company — and then eliminated the team entirely. The slashing took place several months before the release of Microsoft’s genAI chatbot; the team’s elimination was several months after.
Before the release of the chatbot, John Montgomery, Microsoft corporate vice president of AI, told the team why it was being decimated: “The pressure from [CTO] Kevin [Scott] and [CEO] Satya [Nadella] is very, very high to take these most recent OpenAI models and the ones that come after them and move them into customers’ hands at a very high speed.”
He added that the ethics team stood in the way of that.
When a team member responded that there are significant dangers in AI that need to be addressed — and asked him to reconsider— Montgomery answered, “Can I reconsider? I don’t think I will. ’Cause unfortunately the pressures remain the same. You don’t have the view that I have, and probably you can be thankful for that. There’s a lot of stuff being ground up into the sausage.”
Once the team was gone, Microsoft was off and running with genAI. And that accomplished exactly what the company wanted. The company’s stock has skyrocketed, and thanks to AI, it’s become the most valuable company in the world — the second company (behind Apple) to be valued at more than $3 trillion.
That’s three trillion reasons you shouldn’t expect Microsoft to change its tune about the potential dangers of AI, whether or not Microsoft Designer was used to create the Taylor Swift deepfakes. And it doesn’t bode well for the chances of a tsunami of deepfakes in the year ahead, especially with a contested presidential election in the US.