GitHub recently introduced Copilot Workspace, an AI-powered tool designed to ‘assist’ the developer. However, it’s not the first AI coding tool to make waves in the industry. Earlier this year, Cognition Labs announced Devin, which quickly grew popular.
The key difference between the two as explained by Jonathan Carter, head of GitHub Next were, “We don’t view GitHub Copilot Workplace as an ‘AI engineer’; we view it as an AI assistant to help developers be more productive and happier,”
While both Copilot Workspace and Devin aim to assist developers, they differ in their approaches and capabilities. And this space of building developer tools is gaining more competition as newer companies with fat investments enter it. Recently, we saw Augment AI, another company with impressive talent with the intention of easing the developer’s stress.
While Augment doesn’t have a product so far, the notable difference between Devin and GitHub Copilot Workspace is that Devin works more as an agent. Carter explained, “Devin includes a build/test/run agent that attempts to self-repair errors,” while “GitHub decided to focus on ‘optimising the core user experience’ for the Copilot Workspace technical preview, rather than including a similar feature.”
Are assistants better so far?
GitHub Copilot Workspace boasts several advantages that set it apart from Devin and other AI coding assistants.
Dan Shipper, a software engineer who tested Copilot Workspace, noted, “Copilot Workspace seamlessly integrates with GitHub features like issues, pull requests, and code reviews, making collaboration easier.”
This tight integration enables developers to collaborate more effectively and streamline their workflow, ultimately enhancing productivity.
Another advantage of Copilot Workspace is its mobile compatibility. As Shipper points out, “Developers can use Copilot Workspace on their mobile devices, allowing for greater flexibility and on-the-go collaboration.”
This feature proves particularly useful for quick code reviews or issue triaging, enabling developers to work efficiently even when away from their primary workstations.
Copilot Workspace also stands out for its adaptive learning capabilities. As developers interact with the tool, it learns from their codebase and adapts to their coding style over time. Shipper observed, “This leads to more accurate and relevant suggestions over time.”
This adaptive learning feature ensures the tool continuously improves its code suggestions, making it an increasingly valuable asset for developers.
On the other hand, Devin, the AI engineer, is a more autonomous entity capable of independently tackling complex development tasks. Devin’s ability to ‘self-repair errors’ suggests a higher level of autonomy and decision-making capacity compared to an AI assistant.
However, the actual performance of these tools may not always align with their marketed capabilities. In a video review of Devin by Santiago L. Valdarrama, a developer who got early access, tested the tool on various projects and found mixed results. While Devin successfully completed some tasks, such as building a Tic Tac Toe game and a digital classification application, it struggled with more complex projects like a lunar lander simulation.
Valdarrama noted that Devin often generated excessive or irrelevant code, requiring manual intervention and cleanup. He also observed that Devin’s ability to understand and implement complex requirements was limited, often leading to suboptimal or incorrect solutions.
While Devin generated significant buzz with its bold claims of being the world’s first AI software engineer, Copilot Workspace appears to be a more refined and developer-centric tool at this juncture. As Carter points out, “We definitely intend to explore a VS Code extension in the not-too-distant future.”
AI agents like Devin continue to advance in their reasoning and planning capabilities. While Devin’s current performance may not be impressive, it only completes tasks accurately about 14% of the time, it’s crucial to consider the rapid pace of AI development that led us here. When GPT was first introduced it couldn’t strategically reason.
We’ve witnessed significant advancements in AI models, and if this trend continues, we may soon find ourselves in a world where autonomous agents can effectively handle complex problems and engage in long-term planning.
