The GPT-3 technology runs with 175 billion parameters and can now generate an average of 4.5 billion words per day
The publishing industry has proven strangely resilient to the march of new technology, with only ebooks significantly disrupting a business that’s been going along in much the same way for hundreds of years.
That could be about to change, however. Generative Pre-trained Transformer 3 (GPT-3) is a deeplearning neural network that its creators are teaching to write like a human being.
It’s not the first time AI has been tasked with churning out fiction. Publisher JBE Books released 1 the
Road in 2018, which was entirely written by a neural network, while AI-generated Harry Potter fan fiction has kept Twitter amused for years.
But as anybody familiar with these works will know, AI copy is often clunky and, at times, ridiculous.
GPT-3’s big claim is that it’s more naturalistic, capable of mimicking a writer’s style and structure, rather than simply regurgitating text. Feed it a passage of prose and it will (theoretically) continue writing in the same style, with the same syntax and flow. This is causing ripples across the writing industry. Could it be the beginning of a big problem for people who work in creative writing? Or could it be the antidote to writer’s block, the author’s worst nightmare? Words in, words out
The GPT-3 project was launched last year by a research company in Silicon Valley called OpenAI, which aims to build a “safe and beneficial” AI. GPT-3 is the latest iteration of OpenAI’s GPT-n series, which focuses on natural language processing. It uses a “text in, text out” interface that can supposedly complete almost any English language task, including continuing an article, a piece of fiction, an essay or a poem.
According to OpenAI, more than 300 applications are now using GPT-3, with tens of thousands of developers across the world “programming” the platform by showing it writing examples or prompts. The technology runs with 175 billion parameters and can now generate an average of 4.5 billion words per day, returning text in natural language.
The aim, says OpenAI, is to keep the technology simple enough for anyone to use, but also flexible enough to make machine learning teams more productive. Of course, it’s not perfect. Humans can usually perform a new language task from a instructions – something which
AI admits current machinelearning algorithms can’t do.
Critics of the technology include the founder of machine learning firm Geometric Intelligence, Gary Marcus, who says that the problem with
GPT-3 is that “it makes relationships between words without having an understanding of the meaning behind each word”. Basically, GPT-3 knows the rules, but that’s as deep as its understanding goes.
Still, not everybody needs Agatha Christie levels of plot development. One such company is Fable Studio, which is currently creating a new genre of interactive stories and is using GPT-3 to help power its storydriven “Virtual Beings”.
“GPT-3 has given us the ability to give our characters life,” said Fable Studio CEO Edward Saatchi. “We’re excited to combine an artist’s vision, AI, and emotional intelligence to create powerful narratives, and believe that one day, everyone will know a Virtual Being.” Beating writer’s block
Another application built on GPT-3 is Sudowrite, which aims to help authors escape from the infamous writer’s block by allowing them to upload passages of text and autogenerate the next part of the story. Its founders, Amit Gupta and James Yu, developed Sudowrite using GPT-3 to help make writing life a little less solitary. “Both James and I came to writing from a technology background,” said Gupta.
“After working in collaborative fields, we were surprised at how lonely writing can be. When you get stuck, you’re completely on your own. So when GPT-3 became available last year we thought, ‘how can this help us be a little less lonely and get our writing flowing a bit more?’”
Sudowrite won’t get writers through that horrible stage of staring at a blank white document when they’re struggling to turn a raw idea into prose. It needs at least an opening passage to give it something to work with. There’s a nifty “Twist” button that helps take a story in unexpected directions, and a “Character” section, which helps invent new characters that could work in the same story.
Visually, it’s not dissimilar to software such as Microsoft Word or Scrivener. It’s easy to play around with and has clickable features such as the “Wormhole”, which lets users choose themes for their writing such as “extraordinary”, “ominous”, “funny” and “neutral”.
For Gupta, GPT-3 is simply doing for the writing process what smartphones did for photography, adding enhancements to what the user has created. “Other artistic mediums, like photography and
As an artist, you have a vision of what you want to create and the better your tools, the better your execution
music, have been revolutionised by computers,” said Gupta. “When the camera phone came out everyone had a camera in their pockets, so everyone became photographers.
The quality of photography increased. Instead of being photography consumers, people became photography connoisseurs.
“Yet for writing, not much had changed since the invention of the word processor. Now, there’s an opportunity for these AI transformer tools to bring computer-aided creation to the writing field.”
Gupta is keen to stress that Sudowrite is intended to help writers, not replace them. There won’t be hundreds of Sudowrite bots churning out bestsellers. Writers still need a good idea, a creative arc for their characters, and depth and meaning in their stories – none of which the software can do. “As an artist, you have a vision of what you want to create and the better your tools, the better your execution,” said Gupta.
“Writers who use Sudowrite are generally won over by the fact that it can do interesting things but it can’t write their novel. A lot of that fear [of being replaced] abates when they see what the tool can do. It can help writers but it can’t replace them.” A sense of style
While Sudowrite can’t write a novel, the fact that GPT-3 can replicate an author’s voice creates a sense of unease amongst writers. Authors with a distinctive voice could theoretically see stories in their tone directed and released by someone else, mimicking their style. Is the issue of “voice piracy” one that Sudowrite’s creators have considered?
“In my opinion, the voice is a component of the art any writer creates,” said Gupta. “But the subject matter, the perspective, the meaning they imbue in the artwork – they all matter, too.”
Gupta returns to the photography analogy, citing apps that filter photos to look more like, for instance, a Picasso or Ansel Adams, and musical programs that let you tune your music to fit the sound profile of famous artists. “Art of all forms has a solid foundation of mimicry,” he said. “It’s how many new to the craft learn to take their first strokes on their canvas. But if they’re to make art true to themselves, they will eventually have to find their own voice, meaning, and perspective. I think that will be true more than ever in the future.”
It’s early days for Sudowrite and the team is still experimenting. Recently, they created a critique tool that suggests how copy can be improved, or what the author could consider changing. “Mostly the suggestions are terrible,” Gupta admitted. “But occasionally, even just once, it may spark a truly incredible idea.”
That one idea may turn out to be a bestseller. For an author, or someone who aspires to be one, that could make it all worthwhile.