OpenAI Said Its Code Was Risky. Two Grads Recreated It Anyway
Credit to Author: Tom Simonite| Date: Mon, 26 Aug 2019 11:00:00 +0000
In February, an artificial intelligence lab cofounded by Elon Musk informed the world that its latest breakthrough was too risky to release to the public. OpenAI claimed it had made language software so fluent at generating text that it might be adapted to crank out fake news or spam.
On Thursday, two recent master's graduates in computer science released what they say is a recreation of OpenAI’s withheld software onto the internet for anyone to download and use.
Aaron Gokaslan, 23, and Vanya Cohen, 24, say they aren’t out to cause havoc and don’t believe such software poses much risk to society yet. The pair say their release was intended to show that you don’t have to be an elite lab rich in dollars and PhDs to create this kind of software: They used an estimated $50,000 worth of free cloud computing from Google, which hands out credits to academic institutions. And they argue that setting their creation free can help others explore and prepare for future advances—good or bad.
“This allows everyone to have an important conversation about security, and researchers to help secure against future potential abuses,” says Cohen, who notes language software also has many positive uses. “I’ve gotten scores of messages, and most of them have been like, ‘Way to go.’”
The duo’s experiment, like OpenAI’s, involved giving machine learning software text from millions of webpages gathered by harvesting links shared on Reddit. After the software internalizes patterns of language from the text, it can then be adapted to tasks such as translation, powering chatbots, or generating new text in response to a prompt.
The text Gokaslan and Cohen’s software generates can be impressively fluid. When WIRED gave it the prompt “The problem with America is” it added “that, because everything is a narrative, we're all imprisoned in our own set of lies.” A few sentences later it praised Donald Trump for being able to “give voice to those who had been left voiceless.”
That text showed similarities to what WIRED saw when playing with the (ultimately withheld) model OpenAI developed earlier this year, called GPT-2. That one riffed about connections between Hilary Clinton and George Soros. Both versions of the software show the signs of training on content linked from Reddit, where political debates can be fiery.
But neither project can generate perfect prose: Machine learning software picks up the statistical patterns of language, not a true understanding of the world. Text from both the original and wannabe software often makes nonsensical leaps. Neither can be directed to include particular facts or points of view.
Those shortcomings have caused some AI researchers to greet OpenAI’s claims of an imminent threat to society with derision. Humans can—and do—write more potent misleading text.
Tuesday, OpenAI released a report saying it was aware of more than five other groups that had replicated its work at full scale, but that none had released the software. The report also said that a smaller version of GPT-2 OpenAI had released was roughly as good as the full withheld one at creating fake news articles. (You can try that smaller version online.)
Gokaslan and Cohen took the report’s data to mean that their own software wouldn’t be significantly more dangerous than what OpenAI had already released, if it was dangerous at all. They wanted to show the world that similar projects are now within reach of anyone with some programming skills and motivation. "If you gave a high school student guidance, they could probably do it,” Gokaslan says.
Miles Brundage, who works on policy at OpenAI, declines to say how dangerous the software the pair released might be. No one has had time to properly test it, he says, although figures released by Gokaslan and Cohen suggest it is slightly less powerful than the full GPT-2. Brundage adds OpenAI would like to eventually release that full version, but is waiting to feel “comfortable” there won’t be negative consequences.
Brundage acknowledges that Gokaslan and Cohen have shown how widening access to powerful computers and AI skills is increasing the number of people who can do such work. He still thinks anyone working on something similar should proceed with caution and talk through their release plans with OpenAI. “I encourage people to reach out to us,” he says.
Another AI safety lesson from the episode is to always read your email. Gokaslan and Cohen tried to inform OpenAI about their work by contacting the lead author on the lab’s technical paper about GPT-2. They say they never heard back, causing them to miss out on whatever OpenAI advises other researchers about the risks of software like its own.
A spokesperson for OpenAI said the researcher Gokaslan and Cohen tried to contact “gets a lot of email,” and that the lab's policy team has been monitoring a dedicated email address for discussions about GPT-2 previously publicized in blog posts.
Gokaslan and Cohen did make contact with OpenAI Thursday, after a tweet announcing their release began circulating among AI researchers. They say they’re looking forward to discussing their work and its implications. They’re also working on a research paper describing their project—and they plan to write it themselves.