Interview with the team behind Microsoft’s µTransfer

With machine learning models getting bigger in size, training them is becoming quite a challenging task. Especially the task of tuning, when the parameters run into the order of billions (even trillion now), makes it a highly cumbersome and resource-intensive process. Recently, researchers –  Edward Hu, PhD Student  Greg Yang, Senior Researcher Jianfeng Gao, distinguished scientist and vice president from Microsoft, introduced µ-Parametrization, which offers maximal feature learning even in infinite-width limit. The researchers further collaborated with OpenAI to demonstrate its practical advantages, which was recorded in this paper. We caught up with Edward Hu and Greg Yang to learn more about their research. Edited excerpts: AIM: How did you recognise this as a prob