|
Listen to this story
|
OpenAI has released its latest version of the language model, GPT-4, which it calls a “milestone in our effort in scaling up deep learning”. While the company credits the achievement to a team effort, for OpenAI’s founder Sam Altman, one person stands out as a driving force behind the pretraining effort – Jakub Pachocki.
Pachocki has been with OpenAI since 2017, and his technical vision and leadership played a crucial role in the development of GPT-4. According to Altman, “we wouldn’t be here without him”.
Pachocki himself likens the process of building a language model like GPT-4 to building a spaceship – getting all the components right and making sure nothing breaks. In a recent interview with MIT, he said “That fundamental formula has not changed much for years,” talking about the evolution of GPT models since the first version released in 2018.
Pachocki’s fascination with artificial intelligence dates back to the release of AlphaGo, when he realised that deep learning had the potential to go far beyond what computers were capable of at the time. He played a key role in OpenAI’s development of a bot that learned to play Dota 2, an online multiplayer battle arena game, by playing against itself over and over until it reached the pro level.
“I always wanted to work on AI and push the limits of what computers could do,” he said in Artificial Gamer, a documentary about how OpenAI showed off its ability to beat top-tier champions at Dota 2.
Keeping It Low Key
Despite his pivotal role in the development of GPT4 and his success as a programmer, Pachocki has kept a low profile. He rarely makes media appearances and is not active on social media either. However, his achievements speak for themselves. In 2012, he won the title of Code Jam Champion and the $10,000 reward that came with it. He is also placed in the top ranks of other programming competitions, including the ACM ICPC World Finals 2012 (gold medal), TopCoder Open 2012, and Facebook Hacker Cup 2013.
Pachocki’s success as a programmer highlights the importance of cognitive ability in the field. As he explained in an interview, programming competitions like Google Code Jam are more like mathematical work or solving logic puzzles, and winning requires extraordinary cognitive ability. In that respect, Google’s 14% of employees without a college degree may not be so different from its other 86%.
As OpenAI continues to push the boundaries of what computers can do, it will be individuals like Pachocki who play a key role in making these breakthroughs possible. While he may prefer to keep a low profile, his achievements serve as an inspiration to others in the field who are striving to push the limits of what is possible.
Pachocki is not the only one shying away from the paparazzi. Mira Murati, OpenAI’s CTO is another such personality. Like some of her colleagues, there’s information about her work, but one would rarely find anything on the internet about her, aside from the fact that she is a fan of the sci-fi classic ‘2001: A Space Odyssey,’ poet Rainer Maria Rilke and the rock band, Radiohead.
There’s also Ilya Sutskever, the brain behind the company’s money machine ChatGPT, who also happens to be one of the most cited computer scientists in history. Polish computer scientist Wojciech Zaremba is also an integral part of the core OpenAI team. On a podcast with Lex Fridman in August 2021, Zaremba had said, “GPT learns from the whole internet. It gives them a decent backbone. Like a chameleon that can change its colour. But it should actively learn after deployment.”
GPT4 is available to OpenAI’s paying users via ChatGPT Plus and developers can access the API via this waitlist.
