Psst… Amazon Is Busy Transfer Learning

Amazon Alexa team uses transfer learning extensively to transfer knowledge across various language models, features, and better machine translation.
Psst… Amazon Is Busy Transfer Learning

Amazon, perhaps, is one of the only few players in the machine learning landscape to have heavily invested in transfer learning, besides Facebook, Microsoft and DeepMind. The success of this can be seen in its Alexa virtual assistant, which has reached significant strides in the last few years, competing closely with Google Assistant and Apple’s Siri.

For those unaware, transfer learning is a technique where learning in a new task is done via the transfer of knowledge from a related task that has already been learned. For instance, knowing how to ride a bicycle makes learning how to ride a motorcycle easier. Similarly, this idea can be applied to machine learning instead of learning or developing a large scale language model with billions of parameters from scratch. 

Doing so is not only time consuming but also expensive. Developers or researchers have to collect thousands of voice samples and annotate/label them manually, a process that can take months or weeks easily. That is why researchers at Amazon Alexa have pursued transfer learning, which leverages a neural network trained on a large dataset of previously labelled samples to train in a new domain/area with sparse data. 


Sign up for your weekly dose of what's up in emerging technology.

Today, the Amazon Alexa team uses transfer learning technology extensively to transfer knowledge across various language models, features, and better machine translation capabilities. 

Transfer Learning Everywhere

Not just Alexa, Amazon has been working on transfer learning across various areas, including product recommendation, AutoML, computer vision, and others. 

Download our Mobile App

Two years ago, Amazon introduced two new features called ‘Newscaster and Neural text-to-speech (TTS)’ to its cloud-based TTS service – Amazon Polly. Launched in 2016, Amazon Polly turns text into human-like speech, allowing users to develop speech-enabled products and applications. 

Besides Polly, Amazon offers multiple APIs that aim at executing tasks within text analysis, which can also explore transfer learning to further customise the inference of these models. Some of them include Amazon Personalize, Amazon Forecast, Amazon Transcribe, Amazon Rekognition, Amazon Comprehend, Amazon Lex, Amazon Textract, Amazon Translate, etc. 

Here’s a list of all the research work done in transfer learning by Amazon researchers in the last four years. 

Shaping Transfer Learning 

Four years ago, Amazon founder Jeff Bezos (executive chair), at the annual Amazon letter, described Alexa’s success and improvements from semi-supervised and transfer learning. He said they have dramatically reduced the amount of time required to teach Alexa new languages by using machine translation and transfer techniques, which allowed them to serve customers in more countries, including India and Japan.

Last month, Amazon released new tools like notebooks, text models and solutions for multimodal financial analysis within Amazon SageMaker JumpStart. Using these tools, you can easily retrieve public financial documents, including SEC filings, and process financial text documents with features like summarization and scoring for various attributes such as sentiment, risk and readability. 

In addition, users can access pre-trained language models trained on financial text for transfer learning and use example notebooks for data retrieval, text feature engineering, regression models and multimodal classification. 

Today, transfer learning has become one of the most popular techniques in deep learning as it can train deep neural networks with little or limited data in a short period of time. Previously, tech evangelist Andrew Ng, at NIPS 2016, had said that transfer learning would be – after supervised learning – the next driver of machine learning commercial success. Cut to the present; Amazon is most certainly leading the way for transfer learning.

More Great AIM Stories

Amit Raja Naik
Amit Raja Naik is a seasoned technology journalist who covers everything from data science to machine learning and artificial intelligence for Analytics India Magazine, where he examines the trends, challenges, ideas, and transformations across the industry.

AIM Upcoming Events

Regular Passes expire on 3rd Mar

Conference, in-person (Bangalore)
Rising 2023 | Women in Tech Conference
16-17th Mar, 2023

Early Bird Passes expire on 17th Feb

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
27-28th Apr, 2023

Conference, Virtual
Deep Learning DevCon 2023
27 May, 2023

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox