How Naftali Tishby’s Information Bottleneck Theory Can Break Open The Black Box Of Deep Learning

Deep learning has been making tremendous progress in the fields of computer vision , natural language processing and other fields of machine learning. But how these deep learning models work, at the scale they do has been an open question for quite a while. This mystery surrounding deep learning has made many researchers to focus specifically on understanding large deep learning models. Computational learning theory gives a formal framework to study the predictive powers and computational powers of machine learning models. It is mostly associated with knowing about the efficiency of data usage (sample complexity) and computation usage (time complexity). Recent research led by Dr. Naftali Tishby has made exciting connections between the field of computational learning theory and informati