When “The Master Algorithm” was released in 2015, it acted as a window into the field of AI and machine learning, proving to be a great primer, especially for the ‘outside’ world. It was authored by Pedro Domingos, a pioneer in the field and someone who has lived through the evolution of the field, right from the early 90s to now in the 2020s. In a career that has spanned over three decades and counting, Domingos has been bestowed with several recognitions, including the SIGKDD award, which is widely considered the highest honour in data science. He is also credited with the invention of Markov Logic Networks.
Analytics India Magazine was recently in conversation with Domingos, talking about topics ranging from the evolution of machine learning, his book (s), to AGI and Metaverse!
Evolution of AI
“When I was younger, I came across this book on AI. It made me curious about machine learning – if you could make it work, it had the potential to take over the world. Later, when I decided to pursue a formal degree in this field, there were hardly any colleges offering such courses. The field was very primitive, making it very difficult to make significant contributions, as opposed to more mature fields like Physics or Biology. University of California, Irvine was one of the very few colleges that offered substantial machine learning courses, and that is where I got a PhD,” he said. He counts Geoff Hinton and Tom Mitchell among his earliest heroes of the field.
Times have drastically changed since then. As Domingos points out, the field of machine learning is dramatically changing, and it is growing bigger. The other thing that marks the evolution of the field in the last three decades is the development of a whole new industry for AI. “If you consider the top ten companies of the world, seven of them would say AI is essential to what they do,” Domingos pointed out.
While the progress of AI is encouraging, Domingos points out there is a downside to it. “It is becoming impossible to keep up with the field. It is good that we have a thriving industry because of its impact factor, but on the other hand, it becomes a lot more difficult to change. We will have to fight this inertia,” he said.
The Master Algorithm
“I thought about writing such a book back in the 90s with the then-ongoing data mining explosion. I thought people outside of the field would benefit from knowing the developments. My book was going to be directed toward people who were not necessarily working in the field of AI and machine learning. In that sense, it was a challenge because a popular science book cannot be a list of topics and should instead have a storytelling element to it. And I did not know at that time what that story would be,” said Domingos.
Close to over two decades later, two things persuaded him to finally write this book – the big data boom and the lack of knowledge about it that was causing costly mistakes being made. “There was a sense of urgency,” he said.
“What occurred to me is that machine learning is a quest for the ‘Master Algorithm’. Such an algorithm can learn anything. So I cast the book as being the search for such a Master Algorithm. Through this book, I take people on a tour of different approaches of different ‘tribes’ as I’d like to call them. These tribes are different schools of thought in machine learning – interestingly, each of them thinks that their way is the only right way when actually, all, individually, are just a piece of the puzzle. We need to find a unified theory of machine learning, building a standard like there exists in Physics or Biology,” he explained. The five tribes of machine learning are – Symbolists, Connectionists, Evolutionaries, Bayesians, and Analogies.
When asked which tribe just he aligns himself with, Domingos said, “I don’t align with any single one of them, which is unusual. But that was also a part of the reason that made me write this book, as opposed to somebody else who thinks their paradigm is the only way. My PhD thesis was on unifying these paradigms, and a large part of what I did throughout my career revolved around the same. I strongly believe that we need to combine ideas from each of these paradigms to truly have a general-purpose learning algorithm. It is more apparent and true today than ever before.”
On ethics and governance
“AI is a powerful technology, and like any other powerful technology, it creates a lot of issues. And when something starts to make a lot of impact in the real world, a lot of people start wanting to have a say – like politicians and those from other fields. This is not surprising but still worrisome. A lot of people who do not understand AI very well try to make big decisions about the same,” said Domingos.
“Westerners that were raised on the Bible in Genesis are always worried about the creation turning against the Creator—will the machines revolt against us— which is nonsense, but this gets so much airplay in Hollywood movies, from journalists, etc. On the other hand, the libertarians are very concerned about freedom, fairness and equality. A lot of time, the concerns are well-intentioned but misguided. Then we go on to solve a problem that wasn’t there in the first place. Let’s not jump the gun; trying to regulate something before you understand it is generally not a good idea,” Domingos said.
He further added that with strict data laws and regulations like the GDPR, the European government is more ‘trigger happy’ but calls America a little more sensible’ in this regard, with more lenient laws, except for legal requirements on algorithms that make decisions, especially in the field of medicine.
Interestingly, not very long ago, Domingos posted his views on the recently introduced ‘ethics review’ section in the papers that were submitted at the NeurIPS conference. “Since when are scientific conferences in the business of policing the perceived ethics of technical papers?” he had tweeted then.
He still holds his ground. “It could be a research work on how to speed up an algorithm, but the authors would now have to have a discussion of the ethical consequences of making an algorithm fast, and this makes no sense.”
AI today is intertwined with society, and it affects real people. What are Domingos’ thoughts on where should one draw the line; on this, he said that regulations could be imposed on applications of AI instead of the entire technology. “Trying to regulate AI to me is like trying to regulate mechanical engineering; what is the ethics of mechanical engineering? This makes no sense. There are specific applications of machine or mechanical engineering that have ethics involved and that need regulation. And it’s the same thing with AI,” he said.
Preparing the world for Metaverse
Domingos thinks that the idea of the Metaverse is a desirable goal; while it seems to be a long term goal, it is still very compelling. Calling it an extremely difficult problem, Domingos feels that there are several challenges to deal with, such as psychophysics.
The second challenge that he talks about is identifying and taking this kind of technology to the right kind of people and industries, which would, in turn, give more resources to conduct bigger research. “I think what Google did with Google Glass was a complete mistake – they rolled it out as a consumer product even before the technology could fully mature. Such a product would have instead worked very well for, say, a maintenance company, helping their staff save a lot of time and resources.”
