Battle of AI-coding assistants: GitHub Copilot vs Amazon CodeWhisperer

Tech giants compete in everything. Now, the topic of interest seems to be AI-powered coding assistants. One of the major tech innovations last year was GitHub Copilot, an AI pair programmer developed by Microsoft and OpenAI. It created quite a stir in the tech world and received much appreciation (and some criticism too). 

Amazon has woken from its one-year slumber and brought out a direct competition to Copilot. It has been named CodeWhisperer.

This launch happened at the re:Mars conference held recently. 

CodeWhisperer is a machine learning-powered service that helps improve developer productivity by generating code recommendations based on developers’ comments in natural language and their code in the integrated development environment. During preview, CodeWhisperer is available for Java, JavaScript, and Python programming languages. It integrates with multiple IDEs, including JetBrains, Visual Studio Code, AWS Cloud9, and the AWS Lambda console, Amazon informed. 

Striking at the opportune moment

Interestingly, this comes in the backdrop of days after Microsoft has announced the general availability of GitHub Copilot to all developers for USD 10/month or USD 100/year. It will be free to use for verified students and maintainers of popular open-source projects. Until this announcement was made by GitHub, it was also available in technical preview. Currently, there are no details on the cost of CodeWhisperer.

Download our Mobile App

Github’s CEO, Thomas Dohmke, had informed, “With more than 1.2 million developers in our technical preview over the last 12 months, people who started using GitHub Copilot quickly told us it became an indispensable part of their daily workflows.”

He went on to further add that in files where Copilot is enabled, nearly 40% of code is being written by GitHub Copilot in popular coding languages, like Python, and the company expects this to only increase. 

GitHub Copilot draws context from comments and code to suggest individual lines and whole functions instantly. It is powered by Codex (generative pretrained language model from OpenAI) and is available as an extension for Visual Studio Code, Visual Studio, Neovim, and the JetBrains suite of IDEs.

Not just a copy of Copilot, says Amazon top exec

While interacting with TechCrunch, Vasi Philomin, Amazon’s VP for AI services, mentioned that Amazon has not created CodeWhisperer to give an alternative to Copilot. He said that with CodeGuru and DevOps Guru, the company set the groundwork for launch quite a few years ago.

• CodeGuru – In 2020, Amazon announced the general availability of Amazon CodeGuru. It is an ML-based developer tool that provides intelligent recommendations for improving code quality and identifying an application’s most expensive lines of code.

• DevOpsGuru – Last year, Amazon announced the general availability of Amazon DevOps Guru, an ML-powered service that gives users a simpler way to measure and improve an application’s operational performance and availability and reduce expensive downtime.

Delivering what it claims?

Since the launch of Copilot, it has generated much curiosity, and studies have been conducted to see if it delivers what it claims. According to the paper titled, “Is GitHub Copilot a Substitute for Human Pair-programming? An Empirical Study”, the authors conducted an experiment with 21 participants where they emphasised code productivity and code quality. 

For experimental design, a participant was given a project to code under three conditions presented in a randomised order. 

• Pair-programming with Copilot
• Human pair-programming as a driver
• As a navigator

The codes generated from the three trials were analysed to determine how many lines of code on average were added in each condition and how many lines of code on average were removed in the subsequent stage. “The results suggest that although Copilot increases productivity as measured by lines of code added, the quality of code produced is inferior by having more lines of code deleted in the subsequent trial,” the authors added.

Another study titled “Asleep at the Keyboard? Assessing the Security of GitHub Copilot’s Code Contributions” has shown that codes designed by Copilot could include bugs or design flaws that an attacker can potentially exploit. The authors created 89 scenarios for Copilot to develop code for, which resulted in 1,692 programs. The authors found out that about 40 per cent of these codes included bugs that could pose security risks.

CodeWhisperer keeps your coding style into consideration

Developers have to stay updated on different programming languages, frameworks, software libraries and cloud services. Now, Amazon claims that developers can speed up the development process with CodeWhisperer by simply writing a comment in their IDE’s code editor.

“CodeWhisperer automatically analyses the comment, determines which cloud services and public libraries are best suited for the specified task, and recommends a code snippet directly in the source code editor,” says Amazon.

Jeff Barr, Chief Evangelist for AWS, added in a blog post that CodeWhisperer will continually examine the user’s code and his/her comments and present them with syntactically correct recommendations. “The recommendations are synthesised based on your coding style and variable names and are not simply snippets,” he added.

Image: ML-powered coding companion – Amazon CodeWhisperer – Amazon Web Services

Security is also a focus area for CodeWhisperer

CodeWhisperer keeps security as a priority, too, claims Amazon. It provides security scans for Java and Python to help developers detect vulnerabilities in their projects and build applications responsibly. It also includes a reference tracker that detects whether a code recommendation might be similar to particular training data. Developers can then easily find and review the code example and decide whether to use the code in their project.