Now IBM wants to speed up automated transfer learning

This article looks into how IBM's CodeFlare significantly speeds up automated transfer learning tasks for foundation models

IBM introduced CodeFlare at the Ray Summit in June of 2021. The platform was introduced to drastically reduce the time required to set up, run, and scale machine-learning tests. For example, CodeFlare reduced the time to execute each pipeline from 4 hours to 15 minutes when one user used the framework to examine and improve approximately 100,000 pipelines for training machine learning mode. Recently, IBM announced that CodeFlare significantly reduces the time to automate transfer learning tasks for foundation models

What is CodeFlare?

CodeFlare is a hybrid multi-cloud platform that streamlines the integration, scalability, and acceleration of complicated multi-step analytics and machine learning pipelines. It is an open-source framework that makes it easier to integrate and scale big data and AI operations to the hybrid cloud. CodeFlare is developed on Ray, an open-source distributed computing framework for machine learning applications. Ray’s capabilities are expanded by CodeFlare, which adds specialised aspects that make scaling operations easier. 

CodeFlare uses Python and runs on IBM’s new serverless platform, IBM Cloud Code Engine and Red Hat OpenShift. Users can deploy the platform from just about anywhere, allowing researchers to reap the benefits of serverless computing. The platform also provides rich APIs and tools, allowing researchers to focus more on their research and less on configuration complexities. 

Also read: IBM Unveils New Data Fabric Capabilities & Advanced Data Privacy Features For IBM Cloud Satellite here.

How does CodeFlare speed up the automation of transfer learning tasks for foundation models?

The newly enhanced and upgraded CodeFlare effectively transforms it from a data science exploratory tool to a platform that can automate AI and machine learning workflows on IBM’s hybrid cloud. Businesses use foundation models for a variety of functions. A financial services firm, for example, may create a foundation model specifically for sentiment research. Currently, gathering and training an AI model on a relevant body of data and various upstream and downstream activities takes an incredible amount of time. CodeFlare speeds up the process by using a Python-based interface or foundational model pipelines, making it easier to integrate, duplicate, and share data. Moreover, on a hybrid cloud platform, these tasks of preprocessing, validating, and adapting foundation models for commercial use cases are now entirely automated.

For example, take the instance of sentiment analysis. CodeFlare begins by cleaning up the input data, which includes de-duplication and the removal of potentially harmful or biassed content. Then, it fine-tunes a foundation model for all the specific activities required for the sentiment analysis of the company. A data scientist may operationalise hundreds of such pipelines with only a few lines of code and automate these operations anytime they need to make changes. CodeFlare allows data scientists to use their own data without leaving the hybrid platform. 

Hybrid cloud is an essential part of IBM’s growth strategy. IBM’s total cloud revenue increased by 20% in the fiscal year 2020, thanks in part to hybrid cloud programmes supplied by IBM-owned Red Hat.

Also read: IBM’s Strategy For Hybrid Cloud Growth In India here

Download our Mobile App

Abhishree Choudhary
Abhishree is a budding tech journalist with a UGD in Political Science. In her free time, Abhishree can be found watching French new wave classic films and playing with dogs.

Subscribe to our newsletter

Join our editors every weekday evening as they steer you through the most significant news of the day.
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

Our Upcoming Events

15th June | Online

Building LLM powered applications using LangChain

17th June | Online

Mastering LangChain: A Hands-on Workshop for Building Generative AI Applications

Jun 23, 2023 | Bangalore

MachineCon 2023 India

26th June | Online

Accelerating inference for every workload with TensorRT

MachineCon 2023 USA

Jul 21, 2023 | New York

Cypher 2023

Oct 11-13, 2023 | Bangalore

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox