Key takeaways
- IBM's CodeFlare significantly speeds up automated transfer learning for foundation models.
- The platform reduces the time to execute machine learning pipelines from hours to minutes.
- CodeFlare is a hybrid multi-cloud platform that enhances the scalability of analytics and machine learning.
- Developed on Ray, CodeFlare adds features that simplify the integration of big data and AI operations.
- Users can deploy CodeFlare on various platforms, benefiting from serverless computing and rich APIs.
IBM introduced CodeFlare at the Ray Summit in June of 2021. The platform was introduced to drastically reduce the time required to set up, run, and scale machine-learning tests. For example, CodeFlare reduced the time to execute each pipeline from 4 hours to 15 minutes when one user used the framework to examine and improve approximately 100,000 pipelines for training machine learning mode. Recently, IBM announced that CodeFlare significantly reduces the time to automate transfer learning tasks for foundation models.
What is CodeFlare?
CodeFlare is a hybrid multi-cloud platform that streamlines the integration, scalability, and acceleration of complicated multi-step analytics and machine learning pipelines. It is an open-source framework that makes it easier to integrate and scale big data and AI operations to the hybrid cloud. CodeFlare is developed on Ray, an open-source distributed computing framework for machine learning applications. Ray's capabilities are expanded by CodeFlare, which adds specialised aspects that make scaling operations easier.
CodeFlare uses Python and runs on IBM's new serverless platform, IBM Cloud Code Engine and Red Hat OpenShift. Users can deploy the platform from just about anywhere, allowing researchers to reap the benefits of serverless computing. The platform also provides rich APIs and tools, allowing researchers to focus more on their research and less on configuration complexities.
Also read: IBM Unveils New Data Fabric Capabilities & Advanced Data Privacy Features For IBM Cloud Satellite here.
How does CodeFlare speed up the automation of transfer learning tasks for foundation models?
The newly enhanced and upgraded CodeFlare effectively transforms it from a data science exploratory tool to a platform that can automate AI and machine learning workflows on IBM's hybrid cloud. Businesses use foundation models for a variety of functions. A financial services firm, for example, may create a foundation model specifically for sentiment research. Currently, gathering and training an AI model on a relevant body of data and various upstream and downstream activities takes an incredible amount of time. CodeFlare speeds up the process by using a Python-based interface or foundational model pipelines, making it easier to integrate, duplicate, and share data. Moreover, on a hybrid cloud platform, these tasks of preprocessing, validating, and adapting foundation models for commercial use cases are now entirely automated.
For example, take the instance of sentiment analysis. CodeFlare begins by cleaning up the input data, which includes de-duplication and the removal of potentially harmful or biassed content. Then, it fine-tunes a foundation model for all the specific activities required for the sentiment analysis of the company. A data scientist may operationalise hundreds of such pipelines with only a few lines of code and automate these operations anytime they need to make changes. CodeFlare allows data scientists to use their own data without leaving the hybrid platform.
Hybrid cloud is an essential part of IBM's growth strategy. IBM's total cloud revenue increased by 20% in the fiscal year 2020, thanks in part to hybrid cloud programmes supplied by IBM-owned Red Hat.
Also read: IBM's Strategy For Hybrid Cloud Growth In India here.
