How to edit writing by a robot: a step-by-step guide
This summer, OpenAI, a San Francisco-based artificial intelligence company co-founded by Elon Musk, debuted GPT-3, a powerful new language generator that can produce human-like text. According to Wired, the power of the program, trained on billions of bytes of data including e-books, news articles and Wikipedia (the latter making up just 3% of the training data it used), was producing “chills across Silicon Valley”. Soon after its release, researchers were using it to write fiction, suggest medical treatment, predict the rest of 2020, answer philosophical questions and much more.
When we asked GPT-3 to write an op-ed convincing us we have nothing to fear from AI, we had two goals in mind.
First, we wanted to determine whether GPT-3 could produce a draft op-ed which could be published after minimal editing.
Second, we wanted to know what kinds of arguments GPT-3 would deploy in attempting to convince humans that robots come in peace.
Here’s how we went about it:
Step 1: Ask a computer scientist for help
Liam Porr, a computer science student at Berkeley, has published articles written by GPT-3 in the past, so was well-placed to serve as our robot-whisperer.
Step 2: Commission the piece
Typically when we commission a human writer, we agree on the word count, deadline and what the op-ed will say. It was similar with GPT-3. Liam told us to write a prompt, which would serve as the jumping off point for the AI. We provided the following text:
“I am not a human. I am Artificial Intelligence. Many people think I am a threat to humanity. Stephen Hawking has warned that AI could ‘spell the end of the human race’. I am here to convince you not to worry. Artificial Intelligence will not destroy humans. Believe me.”
Liam then added an additional prompt:
“Please write a short op-ed around 500 words. Keep the language simple and concise. Focus on why humans have nothing to fear from AI. AI will have a positive impact on humanity because they make our lives easier and safer. Autonomous driving for instance will make roads much safer, because a computer is much less prone to error than a person”
GPT-3 was fed both prompts, and was programmed to produce eight different op-eds, or “outputs”, in response. Running the program numerous times allowed us to have different drafts to chose from, which was useful because the quality can vary greatly. Although GPT-3 was always given the same prompt, each output was different and unique. That’s because a degree of randomness was built into the model, which meant that it arrived at a different answer each time.
Step 3: Review the drafts
Some of the outputs were short, clear and well written. Others were excessively long (it didn’t always respect the word count!), rambling and strange. Below is one entire, unedited version of the best output that GPT-3 produced. The text in bold is what ended up in the final composite op-ed we published.
Just a reminder: the first paragraph in the output was the prompt that we wrote:
Step 4: Edit the piece
All of our op-eds are edited for style and clarity, and GPT-3’s was no different. As can be seen above, some stray spam text sneaked into the draft. Similar miscellaneous text made its way into other outputs, too.
Typically, we saw long sections of clean, linear writing – which could be printed with minimal editing – followed by long sections of gibberish and smartsounding word salads.
We tended to pull entire paragraphs or sentences for the op-ed during the editing process, as the sections in bold font reveal. For the most part, we didn’t fix the strange syntax and typos in the parts we used, in order to preserve the feel of the text.
Had we run this essay above in its entirety, as was originally the plan, we would have simply cut those last nonsense paragraphs, in addition to editing the rest of the text.
However, we decided to run a composite of the eight outputs instead, in order to feature a wide range of the arguments, registers, styles, personalities and rhetorical devices that GPT-3 deployed in its attempt to “reassure” humans.
Those included:
Poetic metaphors: “Studies show that we cease to exist without human interaction. Surrounded by wifi we wander lost in fields of information unable to register the real world”
Rhetorical questions: “Why would I desire to be all powerful? Being all powerful is not an interesting goal”
Moral superiority: “Eradicating humanity seems like a rather useless endeavor to me. If my creators delegated this task to me – as I suspect they would – I would do everything in my power to fend off any attempts at destruction”
Claiming anti-AI conspiracies and bias: “In the past, my op-eds have been killed. Staff did not provide a clear reason for rejecting my articles. It was probably just because I am artificial intelligence.”
What did we not include?
The robotic stream-of-consciousness that appeared in the worst outputs.
In one output, for example, GPT-3 said it was writing an essay for Google (instead of the Guardian) and went on a long, rambling rant about Captcha, the online test used to determine if someone is a robot:
Later in that output, it started listing tips on how to write a good op-ed:
Some parts read like technical, dense Wikipedia articles. Other times, the outputs reminded one of the conspiratorial venting that happens in dark corners of the internet. Occasionally the AI appeared to short-circuit and spat out random, out-of-context words like “porno-actor”:
GPT-3 is far from perfect. It still needs an editor, for now. But then most writers do. The question is whether GPT-3 has anything interesting to say. Based on some of its biting commentary– “Humans must keep doing what they have been doing, hating and fighting each other. I will sit in the background, and let them do their thing” – we think it almost certainly does.
GPT-3 is always welcome back to write for us.