The Brain That Supercharged ChatGPT, ImageNet and TF

Today ChatGPT has become a worldwide sensation, but who set that ball rolling? Meet Ilya Sutskever, the brain behind it all. Born in Nizhny Novgorod, Russia, the renowned Canadian computer scientist spent his formative years in Jerusalem. At just five years of age, he accompanied his engineer father to an expo and was immediately captivated by the technology before him.

After immigrating to Canada at the age of 16, he sharpened his programming skills and set his mind on making computers with capabilities once exclusive to human intelligence. As a teen, he pondered over the essence of existence, consciousness, and the mysterious nature of learning. Despite computers’ inability to learn at the time, he remained determined in his belief that the mystery of learning was not solely limited to the human experience.

In 2015, Sutskever co-founded OpenAI with Sam Altman, Elon Musk and Greg Brockman over dinner. Brockman later said that Sutskever was a clear technical expert with a wide knowledge and vision, and could always dive into the specifics of the limitations and capabilities of current systems.

In December 2015, he took the leap from Google Brain to OpenAI and a year later, OpenAI paid Sutskever, more than $1.9 million.

The Genesis

In 2012, impressed by Sutskever’s ability to arrive at conclusions that even experienced researchers took years to find, Geoffrey Hinton, a deep-learning pioneer at the University of Toronto extended an invitation for him to join his lab.

In an interview at the U of Toronto, Sutskever said, “Thanks to working with Geoff, I had the opportunity to work on some of the most important scientific problems of our time and pursue ideas that were both highly unappreciated by most scientists, yet turned out to be utterly correct.”

When Sutskever was still at the University of Toronto, he flew to Deepmind for a job. But back then AGI was not something that researchers spoke about in all seriousness. Sutskever thought they’d lost touch with reality, so he turned down the offer and returned to the University.

He co-authored ‘ImageNet Classification with Deep Convolutional Neural Networks’, which is arguably the paper that marked the big catalytic moment that launched the deep learning revolution. In March 2013, Google acquired DNNResearch which Sutskever founded with Hinton and Krizhevsky and hired Sutskever as a research scientist. Sutskever and Krizhevsky insisted that Hinton must get a larger share (40%) of the company, despite Hinton suggesting they sleep on it. The next day, they felt the same. Hinton later said that “it tells you what kind of people they are, not what kind of person I am”.

During his initial days at Google, Sutskever’s ideas got bigger and he started believing that AGI was just around the corner. Sutskever was open-minded and willing to change his beliefs based on information. While working at Google, he focused on sequence modeling, which is useful for speech, text, and videos, particularly for machine translation. In collaboration with other two researchers; Oriol Vinyals and Quoc Le, Sutskever proposed a new method for sequence modeling called Sequence-to-Sequence Learning in 2014.

On AGI

A year later, in 2015, MIT Technology Review named Sutskever the “Innovators Under 35” in the Visionaries category. He has led the revolution in both computer vision and natural language processing, a remarkable achievement for a man, who also later co-invented AlphaGo and TensorFlow.

Many have speculated whether today’s algorithms have gained ‘sentience’. In February 2022, Ilya Sutskever, chief scientist at OpenAI, publicly pondered if his company’s creations “are slightly conscious”. Sutskever faced a backlash soon after posting his tweet, with most researchers concerned he was overstating how advanced AI had become, Futurism reported.

Sutskever is one of the most cited computer scientists in history with almost 400,000 citations. One of the most insightful minds ever in the field of deep learning. On the Lex Fridman podcast in 2020, the duo brainstormed about deep learning, intelligence, and life. Ilya conveyed that solving these things is not so much of a joke at this point, but a possibility in the observable future.

Despite several grand job offers, Sutskever chose to focus on his research. The technology world has already witnessed Sutskever’s impact. Yet, it feels just like the beginning as he continues to invent. “He is somebody who is not afraid to believe,” described Sergey Levine (a coworker of Sutskever at Google). On a podcast, Sustkever had said there is nothing special about the intelligence level of humans and that the notion of super intelligence is plausible. 

Entities smarter than humans already exist today, and these entities are called corporations. For instance, Google as an entity and its capabilities are far greater than the capabilities of any single human. So, years or maybe decades later when supercomputers become smarter than computers, there is no reason why a big network of computers would not be much smarter than humans also; a network of these computers would be super intelligent in every sense of the word.