In July 2021, in association with OpenAI, GitHub launched a technical preview of GitHub CoPilot, an AI-based pair programmer that plugs into the editor and offers real-time coding suggestions. The objective was to empower developers with a new tool powered by AI so that they can write codes easier and faster. After that, in June 2022, it was made generally available to all developers.

While GitHub claims that the decision to make it available to all developers has been driven by the fact that GitHub CoPilot proved to be “one of the best tools to empower the next generation developers”, it is quite natural to understand how far and in what way has it empowered developers. 

Empowering developers

Empowering developers is a very broad term and could be interpreted in a variety of ways. For example, one way to interpret developers’ empowerment is by making them more productive. At the time of launch, the GitHub team was hopeful that the new AI-based tool would improve developers’ productivity. Thus, the GitHub team conducted a survey to assess if using GitHub CoPilot had indeed made developers feel more productive. 

Survey respondents provided gratifying feedback. According to them, GitHub Copilot has empowered them in multiple ways. First, it gives developers a suitable starting point for coding that saves their time and helps them stay in the flow. Second, GitHub CoPilot enables them to focus more on solving bigger problems and building better software while it takes care of the boring and repetitive tasks. 

Moreover, those who reported the highest productivity gains accepted the most number of suggestions provided by GitHub Copilot, as shown below.

Further, a qualitative investigation showed that GitHub Copilot makes developers’ work more meaningful and enjoyable. This is because the AI pair programmer can take care of the repetitive work while developers can invest adequate time in complex work requiring critical thinking. This leads to greater happiness, satisfaction and of course, performance. After all, developers interpret productivity as staying focused on the task at hand, making meaningful progress, and feeling good at the end of a day’s work.

In another survey, it was found that developers using GitHub Copilot completed tasks much faster—55% faster than those not using it. For example, while in usual circumstances (not using GitHub Copilot), it took an average of 2 hours 41 minutes to complete writing a web server in JavaScript, it took 1 hour 11 minutes to complete the same using GitHub Copilot. 

Developers’ productivity is not just about speed. It is about developers’ satisfaction and conserving mental energy. According to the SPACE productivity framework, GitHub Copilot impacts productivity in the following manner, as shown in the image below. 

Copilot 

Impressive numbers yet with glitches

Numbers undoubtedly hint at the impressive impact that GitHub Copilot has had on developers’ productivity but does that mean all is good with this new tool?

Well, not really. It does come with its own share of glitches. Experts opine that, at times, the code suggestions are ‘wonky’ and delayed. Thus, even before Copilot can provide a suggestion, the developer has already written the code. This happens mostly in the case of small production snippets. Also, Copilot often overrides automatic parenthesis closing. As a pair programmer, Copilot should ideally help developers question their assumptions, identify hidden problems and understand the larger context. However, instead of that, it outrightly considers their assumptions to be correct and focuses on generating codes based on the immediate context. 

A promising technology 

Despite the shortcomings, such has been the impact of GitHub Copilot that recently it has been made available for the teachers’ community. According to Paul Denny, associate professor at the University of Auckland, the technology is promising with respect to the course and content designing and newer ways of teaching. It can generate new programming assignments and also provide explanations for code examples that are supposedly helpful for students. For example, it can take casually defined problem input (in the English language) and generate well-structured code similar to the ones written by humans. 

The research community is trying to explore the possible implications of GitHub CoPilot in other spheres as well. These include security, labour market, developer practices and behaviour.