The first AI writing tool can be traced back to Stanford’s spell check software in the seventies. AI writing assistants have come a long way since. Today, online writing tools use AI, predictive analytics and NLP to generate new ideas, check tone and structure stories. In addition, the rise of GPT-3 has further revolutionised the content ecosystem.
Amy Cuevas Schroeder, content director at Writer, has spoken about how specialists use AI to make the writing engaging and in tune with the brand voice. According to her, grammar models can be trained on a huge dataset of well-edited content, and deep learning will allow the model to pick up the nuts & bolts of syntax without a prescriptive or rule-based training.
In this article, we look at how AI writing assistants work and list some of the top writing softwares in the market.
Grammarly and Writer
Building and deploying grammar error correction (GEC) models based on data and not rules have become a lot easier with advances in AI. Many tools are still built on open-source grammatical error rule sets. Meaning, thousands and thousands of the most common mistakes writers make. Writer has built proprietary training and evaluation datasets based on native speakers and professional writers.
Grammarly uses NLP to address mistakes in spelling, punctuation, grammar and word choice. Its GEC system takes in a sentence with errors and outputs a corrected version. Thus, GEC is treated as a translation problem where sentences with mistakes are the source language, and mistake-free sentences are the target language.
Grammatical error correction
Transformer models sift through enormous amounts of text to refine the linguistics into statistical patterns. Additionally, transformers are capable of generalising and making local decisions to reach higher levels of accuracy.
For instance, suppose the model has to fill in the blank for the sentence ‘ the cat sat on the ____’. The model need not know the meanings to find an appropriate word- it studies the data and statistical patterns to infer that the closest match would be the word ‘mat’ and not ‘Roomba’.
But, transformers are not sophisticated enough, so the best performance is usually achieved using a hybrid approach like combining different methods such as custom-made rules, deep neural networks, and language models.
A comparison of Transformer language models (BERT, GPT-1, and GPT-2) against two recent similar systems on standard GEC datasets.
Source: Grammarly Engineering Blog
Tag, not write approach
Grammarly combines NMT and seq2seq to achieve custom translation which tags the sequence of words that are to be corrected. This reduces the task to a language understanding problem allowing researchers to parallelise the inference and make it run faster- thus, simplifying the training.
The model consists of 5,000 transformation tags covering some common mistakes, such as spelling, noun number, subject-verb agreement, and verb form. The vocabulary covers 98 percent of the errors present in the CoNLL-2014 (test) used to evaluate the model.
The GEC sequence tagging model is called GECToR- and is compared to Google’s BERT because of its encoder layer made up of a pre-trained transformer stacked with two linear layers, with softmax layers on the top. These are responsible for mistake detection and token tagging. The model is trained in three stages:
- Implementing a synthetic data set holding 9 million target sentence pairs with mistakes.
- Fine-tuning the model on real-world data sets consisting of 500,000 sentences.
- Fine-tuning the model on real-world data sets composed of 34,000 sentences.
The research showed the model predicted the tag-encoded transformations for each token in the input sequence that were further used to modify the output sequence.
Source: Grammarly Engineering Blog
Top AI writing assistants
Content creation
AI Writer – AI Writer generates fresh content for users with its auto writing and text generation features to produce error-free, information-dense content based on the user’s headline.
Acrolinx – Acrolinx helps create content driven by content goals for consistency, tone, inclusive language and scannability.
Rytr- Rytr writes a piece of content in any tone or format, including emails, ad copies and auto-generate catchy and creative texts.
Clear communication
Writer- Writer, is an AI-driven copy-editing software for the editorial teams. First, the team needs to define its guidelines, and then AI will identify all the inconsistencies and errors in the write-up that can potentially damage the brand image.
Wordtune – Its deep tech understands what the user is trying to say and suggests ways to make the writing clearer, compelling and authentic.
Grammarly- Grammarly’s algorithms flag potential issues in the text and make context-specific suggestions to help with grammar, spelling and usage, wordiness, style, punctuation, and even plagiarism.
Hemingway Editor- The app highlights lengthy, complex sentences and common errors, along with adjective & phrase suggestions and formatting.
Academic writing
Ginger- Along with spelling and grammar, Ginger takes into account full sentences to suggest context-based corrections.
ProWritingAid – ProWritingAid gives clear, easy steps to improve the writing with specific creative, business, and academic writing remarks.
