|
Listen to this story
|
The AI hype cycle is well and alive. Ever since the arrival of autonomous agent application AutoGPT on the scene, bloggers, VCs and anyone with a passing familiarity with ChatGPT have declared AutoGPT the future. AutoGPT is an open-source Python application that uses GPT-4 to complete an entire task on its own with little human intervention – this could mean that it may well be the future.
The current claims are still immense. Many of them have called ChatGPT a “thing of the past”, hailing AutoGPT as the next successor to the crown. But the ground realities are very different. Started as a project a month ago, the tool is still in its proof-of-concept stage.
Several researchers and developers who jumped to use the tool eventually found out that it was far from being perfect for production.
Stuck in a confused loop
The tool often tends to get stuck in loops with some complaining that at times the loop repeated itself for an entire night without yielding any results. This could be because AutoGPT isn’t able to differentiate between development and production, which means it cannot apply an action it has already learned and redo it.
For instance, if a user wants AutoGPT to come up with a recipe for Thanksgiving, it can do it. But if the user were to reuse the tool for a recipe for Christmas, AutoGPT starts the same process again without realising that the prompts are essentially the same minus one small change.
A Redditor described their experience with the tool – “I have tried giving it a few tasks only for it to get stuck in a loop and make me literally pay for it. 😂 I love the concept but for now it’ll only be a waste of money to mess around with it,” they stated. While another said, “It has completed zero or my requested tasks successfully.”
Too expensive for practical use
It is understandable that a lot of the quibbles that developers have with AutoGPT may just be its teething issues, but the fact remains that the tool is very costly. Each step in a task which is completed through chain-of-thought prompting requires a call to GPT-4 which is heavy on the pocket. Once the prompt is generated, AutoGPT works towards its goal by constantly improving its reasoning. It considers different routes and evaluates the pros and cons of each option before landing on the final solution. This dance that the tool does is what maxes out users’ tokens.
Some discussed this on a Hacker News thread, “I tried it and was disappointed given its hype. It will be amazing in a year or so, but right now it just wasted $10 of API access.”
And keep in mind, we’re just talking about testing here. To do anything that is worth doing, AutoGPT eats up far too many tokens. For instance, for VueGPT, which is an AI created by AutoGPT to build website applications on Vue JS, it simply costs far too much.
According to OpenAI, GPT-4 with an 8K context window charges USD 0.03 per 1,000 tokens for prompts and USD 0.06 per 1,000 tokens for results. If on an average an entire task takes 50 steps, it eventually becomes a considerable amount of money. Even if hypothetically AutoGPT were to be integrated in an organisation, it would mean running several tasks in a day for multiple employees. It is clear as day that companies wouldn’t be able to bear these cumulative costs.
Limited abilities
Up until now, AutoGPT is also limited by the range of functions it performs like web searching, coding, generating images and managing memory. Considering that GPT-4 is the most advanced AI model we have, its abilities to reason and break down an entire activity is also still limited. So, if you were expecting the tool to manage your finances or manage a social media account it’s best to tame your hopes.
Aleksa Gordic, a former researcher with DeepMind warned against the overblown fanfare while acknowledging that AutoGPT was an exciting development. “Web3/crypto/salesy vibes came to AI big time. I’m a techno-optimist but there is hyping and there is hyping so hard that you’re basically lying and spreading misinformation.
A concrete example is people saying AutoGPT basically solved everything or clearly implying it’s a generally intelligent system (AGI) like humans are.
The problem is that many of these people have very little understanding of how these systems actually work, and not just technical aspects but also the simple economics of it,” he stated.
While AutoGPT may be an initial piece of evidence that the system can self-prompt and complete a whole task seemingly using its own mind, there’s still a long way to go. And an early hype can also do more damage than good.
