Advertisement

Active Hackathon

Tech Behind AI Writing Assistants

We look at how AI writing assistants work and list some of the top writing softwares in the market.

The first AI writing tool can be traced back to Stanford’s spell check software in the seventies. AI writing assistants have come a long way since. Today, online writing tools use AI, predictive analytics and NLP to generate new ideas, check tone and structure stories. In addition, the rise of GPT-3 has further revolutionised the content ecosystem.

Amy Cuevas Schroeder, content director at Writer, has spoken about how specialists use AI to make the writing engaging and in tune with the brand voice. According to her, grammar models can be trained on a huge dataset of well-edited content, and deep learning will allow the model to pick up the nuts & bolts of syntax without a prescriptive or rule-based training.

THE BELAMY

Sign up for your weekly dose of what's up in emerging technology.

In this article, we look at how AI writing assistants work and list some of the top writing softwares in the market. 

Grammarly and Writer

Building and deploying grammar error correction (GEC) models based on data and not rules have become a lot easier with advances in AI. Many tools are still built on open-source grammatical error rule sets. Meaning, thousands and thousands of the most common mistakes writers make. Writer has built proprietary training and evaluation datasets based on native speakers and professional writers.

Grammarly uses NLP to address mistakes in spelling, punctuation, grammar and word choice. Its GEC system takes in a sentence with errors and outputs a corrected version. Thus, GEC is treated as a translation problem where sentences with mistakes are the source language, and mistake-free sentences are the target language. 

Grammatical error correction

Transformer models sift through enormous amounts of text to refine the linguistics into statistical patterns. Additionally, transformers are capable of generalising and making local decisions to reach higher levels of accuracy. 

For instance, suppose the model has to fill in the blank for the sentence ‘ the cat sat on the ____’. The model need not know the meanings to find an appropriate word- it studies the data and statistical patterns to infer that the closest match would be the word ‘mat’ and not ‘Roomba’. 

But, transformers are not sophisticated enough, so the best performance is usually achieved using a hybrid approach like combining different methods such as custom-made rules, deep neural networks, and language models.

A comparison of Transformer language models (BERT, GPT-1, and GPT-2) against two recent similar systems on standard GEC datasets.

Source: Grammarly Engineering Blog

Tag, not write approach

Grammarly combines NMT and seq2seq to achieve custom translation which tags the sequence of words that are to be corrected. This reduces the task to a language understanding problem allowing researchers to parallelise the inference and make it run faster- thus, simplifying the training. 

The model consists of 5,000 transformation tags covering some common mistakes, such as spelling, noun number, subject-verb agreement, and verb form. The vocabulary covers 98 percent of the errors present in the CoNLL-2014 (test) used to evaluate the model. 

The GEC sequence tagging model is called GECToR- and is compared to Google’s BERT because of its encoder layer made up of a pre-trained transformer stacked with two linear layers, with softmax layers on the top. These are responsible for mistake detection and token tagging. The model is trained in three stages:

  1. Implementing a synthetic data set holding 9 million target sentence pairs with mistakes. 
  2. Fine-tuning the model on real-world data sets consisting of 500,000 sentences. 
  3. Fine-tuning the model on real-world data sets composed of 34,000 sentences. 

The research showed the model predicted the tag-encoded transformations for each token in the input sequence that were further used to modify the output sequence.

Source: Grammarly Engineering Blog

Top AI writing assistants 

Content creation

AI Writer – AI Writer generates fresh content for users with its auto writing and text generation features to produce error-free, information-dense content based on the user’s headline. 

Acrolinx – Acrolinx helps create content driven by content goals for consistency, tone, inclusive language and scannability.

Rytr- Rytr writes a piece of content in any tone or format, including emails, ad copies and auto-generate catchy and creative texts. 

Clear communication

Writer- Writer, is an AI-driven copy-editing software for the editorial teams. First, the team needs to define its guidelines, and then AI will identify all the inconsistencies and errors in the write-up that can potentially damage the brand image.

Wordtune – Its deep tech understands what the user is trying to say and suggests ways to make the writing clearer, compelling and authentic. 

Grammarly- Grammarly’s algorithms flag potential issues in the text and make context-specific suggestions to help with grammar, spelling and usage, wordiness, style, punctuation, and even plagiarism. 

Hemingway Editor- The app highlights lengthy, complex sentences and common errors, along with adjective & phrase suggestions and formatting. 

Academic writing

Ginger- Along with spelling and grammar, Ginger takes into account full sentences to suggest context-based corrections. 

ProWritingAid – ProWritingAid gives clear, easy steps to improve the writing with specific creative, business, and academic writing remarks. 

More Great AIM Stories

Avi Gopani
Avi Gopani is a technology journalist that seeks to analyse industry trends and developments from an interdisciplinary perspective at Analytics India Magazine. Her articles chronicle cultural, political and social stories that are curated with a focus on the evolving technologies of artificial intelligence and data analytics.

Our Upcoming Events

Conference, Virtual
Genpact Analytics Career Day
3rd Sep

Conference, in-person (Bangalore)
Cypher 2022
21-23rd Sep

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan, 2023

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
21st Apr, 2023

Conference, in-person (Bangalore)
MachineCon 2023
23rd Jun, 2023

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM
MOST POPULAR

Council Post: How to Evolve with Changing Workforce

The demand for digital roles is growing rapidly, and scouting for talent is becoming more and more difficult. If organisations do not change their ways to adapt and alter their strategy, it could have a significant business impact.

All Tech Giants: On your Mark, Get Set – Slow!

In September 2021, the FTC published a report on M&As of five top companies in the US that have escaped the antitrust laws. These were Alphabet/Google, Amazon, Apple, Facebook, and Microsoft.

The Digital Transformation Journey of Vedanta

In the current digital ecosystem, the evolving technologies can be seen both as an opportunity to gain new insights as well as a disruption by others, says Vineet Jaiswal, chief digital and technology officer at Vedanta Resources Limited

BlenderBot — Public, Yet Not Too Public

As a footnote, Meta cites access will be granted to academic researchers and people affiliated to government organisations, civil society groups, academia and global industry research labs.