|
Listen to this story
|
In a rather unsurprising turn of events, former director of AI at Tesla, Andrej Karpathy, has rejoined OpenAI. In his tweet announcing the transition, he said that the developments in AI are inspiring and having benefitted from them, he finds it exciting to “jump back and build”. Having been one of the founding members of OpenAI, Andrej left the company in 2017 to join Tesla and support Elon Musk in his endeavours. Musk was also one of the co-founders of OpenAI initially but had later left the Board member position in 2018 “to avoid future conflicts of interests due to Tesla developing AI”, OpenAI said.
At Tesla, Karpathy undertook multiple AI endeavours and, perhaps most importantly, was his work with Elon Musk on creating “Optimus”, the humanoid robot whose prototype was unveiled on AI Day, 30 October, 2022. In addition, Karpathy was leading the computer vision team of Tesla Autopilot and his team worked on data gathering, neural network training in addition to deployment in production. However, in July 2022, Andrej left Tesla without any “concrete plans” to pursue his goals of working extensively in AI technology, open source and education.
On Lex Fridman’s podcast in October 2022, Karpathy praised Tesla and Elon Musk, and said, “I’m happy to potentially, at some point, come back for Act 2”. He spoke about Optimus and Tesla AGI and how it was hard for him to leave the team and the organisation that he loved. Responding to Karpathy’s segment of the podcast, Elon Musk tweeted that “Andrej will always be welcome at Tesla”.
Andrej Karpathy holds a PhD in computer science from Stanford University and specialises in deep learning and computer vision. He has been contributing extensively towards the open-source community following his stint at Tesla. He made courses on building deep neural networks from code, including NanoGPT modelled codes from ‘Attention is All You Need’ paper and OpenAI’s GPT-2/GPT-3. Through his Youtube videos, viewers can code together and train on neural networks with the help of Jupyter notebooks that are then captured within the lectures directory. There are two categories of lectures. The first one being, “The spelled-out intro to neural networks and backpropagation: building micrograd”, wherein basic knowledge of Python and calculus is required as the lecture discusses backpropagation and training of neural networks. The second one, slightly more complex, is called, “The spelled-out intro to language modelling: building makemore”, and discusses bigram character-level language model which is implemented into a transformer language model like the GPT. This category focuses on introducing “torch.Tensor” along with its subtleties to efficiently evaluate neural networks. It will also focus on language modelling, including sampling, model training and loss evaluation.
At the beginning of the year, Karpathy released an updated version of minGPT, NanoGPT, a new fast repository for training and tuning medium-sized GPTs. This is in continuation with the minGPT library for GPT language model, which he released in 2020. That earlier version addressed the application of GPT on PyTorch.
Given the raging fascination for AI-generative language, his tutorials gained traction and he continued to stay relevant. It was pretty comprehensible that a larger picture was in plan.
Poster Child for Microsoft
On December 31, 2022, Karpathy went on to laud Microsoft-led GitHub Copilot, a cloud-based AI tool that helps developers write codes, for shortening his coding efforts. He claimed that the Copilot writes “80% of my code”. This, being a direct acknowledgement of using the application, couldn’t have been a clearer hint of him potentially associating with AI-powered organisations. Following this, Microsoft Chairman and CEO, Satya Nadella, has been using Andrej Karpathy as the poster child for promoting GitHUB Copilot in several events. In a podcast called “The Journal”, which is co-owned by Wall Street Journal, Nadella calls out Karpathy’s tweet as “one of the most elite AI developers” talking about improved productivity and that this will “change the productivity curve for software engineers”.
With the back-and-forth praising and acknowledgement of each other’s work since ChatGPT’s launch, Karpathy’s jump to OpenAI was long due.
Angel Investor
In April 2022, Andrej Karpathy invested in Adept, a machine learning research and product building company that works on general intelligence. Adept was founded by David Luan, who previously worked on developing GPT-2 and GPT-3 in OpenAI, after which he quit to lead large models efforts at Google. Karpathy’s investment came a few months before he left Tesla.
In other words, Andrej Karpathy has been keeping himself busy since his Tesla exit. Working autonomously on his forte of AI and building on what he likes was a plan in his short break. Along with this, he was also keenly following OpenAI’s launch and success. Through open acclaim for the success of AI generative platforms and chatbots, Karpathy’s hint of association with them was evident. He was warmly welcomed by Sam Altman.
Considering how Microsoft is heavily investing on ChatGPT, Andrej Karpathy will be a valuable resource in probably shaping ChatGPT into a bigger player than it already is. However, Tesla’s future with Andrej Karpathy looks bleak. Once “great friends”, Elon Musk has now stopped following Andrej Karpathy on Twitter.
