Finally something from Google in code completion

Google has described how the researchers have combined machine learning and semantic engines to develop a novel Transformer-based hybrid semantic ML code completion
Listen to this story

Microsoft has ‘Copilot’, Amazon has ‘Code Whisperer’ and Google has just launched one of their own coding assistants to help improve developer productivity at the end of a two-year research collaboration between Google Core, Google Brain and Google Research. 

In a blog post, Google detailed how the researchers have combined machine learning (ML) and semantic engines (SE) to develop Transformer-based hybrid semantic machine learning code completion. 

Available to Google developers now

THE BELAMY

Sign up for your weekly dose of what's up in emerging technology.

Currently, the coding assistant has only been made available to Google’s internal developers. So far, there is no indication from Google that such facilities could be made available to non-Googlers, but the possibility remains. The researchers then compared this semantic ML code completion of over 10,000 Google developers to a control group. The results were deemed impressive. The team saw a  6% reduction in coding iteration time and a 7% reduction in context switches when they were exposed to single-line ML completion, claims Google.

Hybrid semantic machine learning code completion

In the blog post, Google also delves into great detail about how the hybrid procedure is expected to work. 

  • The code was represented with sub-word tokens and a SentencePiece vocabulary and the researchers used encoder-decoder transformer models running on TPUs to make completion predictions. “Sequences are generated with a beam search or tree exploration on the decoder”, explains Google.
  • When the coder is typing in the IDE, the code completions are requested from the ML along with the SE in the backend. Google reports that while the semantic engines predict a single token, the machine learning models predict multiple tokens. Although, the researchers consider the first token only. 

Following these steps, the researchers consider the top three ML suggestions that are also contained in the SE suggestions and boost their rank to the top. Finally, the re-ranked results come up as suggestions for the coder in the IDE.

  • As a third step, the researchers use semantic engines to conduct fast semantic correctness checks within a given latency budget. Then, they use cached abstract syntax trees to enable a ‘full’ structural understanding, explains Google. 

GitHub Copilot developer productivity results

Copilot was one of the major highlights in tech in 2021. Quite recently, Microsoft announced the general availability of GitHub Copilot to all developers for USD 10/month or USD 100/year. GitHub Copilot draws context from comments and code to suggest individual lines and whole functions instantly. It is powered by Codex from OpenAI. 

Previously, Github’s CEO, Thomas Dohmke, praised Copilot, claiming that in files where Copilot is enabled, nearly 40% of the code is written by GitHub Copilot in popular coding languages. Dohmke believes that this value is expected to increase in the future. 

In July 2022, GitHub released a report on how Copilot helped increase developer productivity. It analysed survey data collected from more than 2,000 developers in the US to research the contribution Copilot had made to enhance these developers’ productivity. 

GitHub claims that the survey focused on three primary queries—if the developers felt that Copilot enhanced their productivity, if that feeling could be measured in any objective usage measurements and if any specific measurement reflected developers’ feeling of increased productivity better than others.

GitHub observed that the acceptance rate of completions was much higher for those who had reported the biggest productivity gains. It was found that the developers didn’t assign much value to reworking the suggestion as long as GitHub Copilot provided them with a suitable starting point. For more details, refer to this paper on GitHub.

“GitHub Copilot offers developers the parts but leaves it up to them to assemble and design the finished product”, adds Albert Ziegler, staff ML engineer at GitHub. 

The recent launch of Code Whisperer has created much anticipation around the impact it is expected to have on universal developer productivity.

Everyone wants to release a coding assistant now 

Of late, code generation is one of the key focus areas for big techs, along with other areas such as large language models and text-to-image generation. There have been a number of quick releases in coding assistant tools from leading global tech companies. 

Around the announcement of Copilot’s general availability, Amazon launched its ‘Code Whisperer’. At the time, Vasi Philomin, Amazon’s VP for AI services, emphasised that the Code Whisperer was not an imitation of GitHub’s ‘Copilot’ and had been in Amazon’s pipeline for years. 

Following suit, CRM software giant ‘Salesforce’ also revealed its CodeGen, which claimed to possess the capability to turn English prompts into executable code. Salesforce claimed that users would be able to solve simple coding problems with little to no coding experience with CodeGen. Likewise, Google-owned AI research lab ‘DeepMind’ also developed Alphacode with the capability to write computer programs.

More Great AIM Stories

Sreejani Bhattacharyya
I am a technology journalist at AIM. What gets me excited is deep-diving into new-age technologies and analysing how they impact us for the greater good. Reach me at sreejani.bhattacharyya@analyticsindiamag.com

Our Upcoming Events

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan, 2023

Conference, in-person (Bangalore)
Rising 2023 | Women in Tech Conference
16-17th Mar, 2023

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
27-28th Apr, 2023

Conference, in-person (Bangalore)
MachineCon 2023
23rd Jun, 2023

Conference, in-person (Bangalore)
Cypher 2023
20-22nd Sep, 2023

3 Ways to Join our Community

Whatsapp group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our newsletter

Get the latest updates from AIM