Think Like Yoshua

Widely recognised as one of the founding fathers of Deep Learning, Yoshua Bengio has played a central role in transforming artificial intelligence. He tutored a bevy of geniuses, including Ian Goodfellow. Bengio won the fabled Turing Award along with Yann LeCun and Geoffrey Hinton, the trio best known for tapping into the brainpower of machines.

Analytics India Magazine caught up with Yoshua Bengio to learn about the revolutionary’s journey in the field and his learnings spanning over three decades. 

Bengio considers his motivation to understand how things work and ask the ‘why’ questions an important factor that has kept him going. At some point, at the beginning of his grad studies, Bengio stumbled upon connectionist models. The term was used for the early work on neural networks—inspired by cognitive science and neuroscience—and applied to build learning machines, much like the work of Geoffrey Hinton. 

Hinton was praised for his work on artificial neural nets, especially how they can be designed to learn without the aid of a human teacher. It was considered the start of autonomous intelligent brain-like machines.

Science is about building on other people’s ideas

Interested in the notion of intelligence early on, young Bengio thought that researchers could understand human and animal intelligence to build intelligent machines, something he hadn’t considered possible. “I got very excited, and I still am. But, if you’re doing work in an area that you don’t feel a strong, emotional appeal to, then it’s not going to be the same as if you’re passionate about it,” he said. 

Moving forward, he emphasised the importance of collaboration. “There’s an incorrect view of how science happens that you see in movies, where scientists discover amazing things alone in their garage or hidden mountain. It’s not how science works. Science is about building on other people’s ideas and doing it in groups,” he said. 

One of Bengio’s most lauded works is with computer scientist and Google Brain researcher Ian Goodfellow on generative adversarial networks (GANs), which has revolutionised the fields of machine translation, image generation, audio synthesis, and text-to-speech systems.

Learning about each others’ insights and brainstorming sessions with other scientists as well as his students is something Bengio considers very powerful. “If I was doing my thing alone, I don’t think I would have achieved what I have. So, it’s not only like an intellectual thing, but when we’re together trying to understand or figure out something, we are motivated by the energy of others. We want to find solutions to problems. Especially when you’re younger, all of these human social factors of the collaboration really make a difference,” he added. 

Talking about his initial days at work in retrospective, Bengio said there was a period when he worked with a not very popular thing in the science community, i.e., neural networks. He had a hard time convincing his students to continue working on the topic but decided to follow his intuition. As a result, Bengio was one of the first to combine neural networks with probabilistic models of sequences—a concept that has been extended to contemporary speech recognition systems. 

In a paper published nearly two decades ago, he introduced the concept of word embeddings, a language modelling and feature learning paradigm in which words or phrases from a vocabulary are mapped to vectors of real numbers.

Research is an exploration

“The lesson is that people tend to do the same thing as others, and we need people to look in different directions. This is how we can be the most efficient. Research is an exploration,” Bengio said. Further, he explained that there’s exploration and there’s wasting time; the line between the two is not obvious. One needs to reason and be rigorous instead of being fooled by delusional ideas. Rationality is extremely important to fight the instinct to go with the flow in order to explore new territories. 

Lastly, Bengio values the sense of humility. He said people who are socially recognised are often blinded by ego. “It makes us think we are right when we run out of sense. Another thing which I’m 100% about is that you have to prove and renew every day. So, that sense of humility is something that needs to be cultivated.”