Machine Learning Deployment Is The Biggest Tech Trend In 2021

“What good is an ML model if it isn’t fast? doesn’t scale? isn’t accurate enough? takes weeks to deploy? and costs too much?”

Luis Ceze, CEO, OctoML

Having machine learning in a company’s portfolio used to be an investor magnet. Now, the market is bullish on MLaaS, with a new breed of companies offering machine learning services (libraries/APIs/frameworks) to help other companies get their job done better and faster. 

According to PwC, AI’s potential global economic impact will be worth $15.7 trillion by 2030. And, as interests slowly shift towards MLOps, it is possible that these companies, which promise to scale and accelerate ML deployment, might grab a bigger piece of the pie. Last week, OctoML raised $28 million. The Seattle-based startup offers a machine learning acceleration platform built on top of the open-source Apache TVM compiler framework project. The $28 million Series B funding brings the company’s total funding to $47 million. 

THE BELAMY

Sign up for your weekly dose of what's up in emerging technology.

Image credits: OctoML

“90% of machine learning models don’t make it to production.”

OctoML

For OctoML’s CEO, Luis Ceze, there is still a significant gap between building a model and making it production-ready. Between rapidly evolving ML models, wrote Ceze in a blog post, ML frameworks and a Cambrian explosion of hardware backends makes ML deployment challenging. “It is not easy to make sure your model runs fast enough and to benchmark it across different deployment hardware. Even if your determined machine learning team has hurtled through this gauntlet, they still have to go through a whole different set of challenges to package and deploy at the edge,” explained Ceze.

A good performance in ML models requires long hours of manual optimizations. These long hours will then translate into hefty cloud bills. Added to this is the model packaging which varies with devices and platforms. According to Ceze, there are no modern CI/CD integrations to keep up with model changes.

“What good is an ML model if it isn’t fast? doesn’t scale? isn’t accurate enough? takes weeks to deploy? and costs too much?,” questioned Ceze as he made a case for OctoML. 

OctoML addressed these pain points with their open-source machine learning compiler framework Apache TV, which according to the team, has quickly become the go-to solution for developers and ML engineers to maximize ML model performance on any hardware backend. “With OctoML we are establishing the first Machine Learning Acceleration Platform that will automatically maximize model performance while enabling seamless deployment on any hardware, cloud provider, or edge devices,” said Ceze.

Be it MLOps or XOps, these services are designed to ease the developers of technical debt that these mega ML models accumulate with changing complexities. Apart from OctoML, there are a few other startups that have succeeded in convincing the investors. Let’s take a look at couple of them:

Verta 

Funding till date: $10 million

The team at Verta is building software for data science teams to address the problem of model management — how to track, version, and audit models used across products. Verta MLOps software supports model development, deployment, operations, monitoring, and collaboration enabling data scientists to manage models across their lifecycle. So far, the company has $10 million in funding and it promises to make robust, scalable, mature deployable models a reality.

Algorithmia

Funding till date: $38.1 million 

Image credits: Algorithmia

“We’re obsessed with helping organizations get ML models into production because that’s the only way they can generate business value,” said the team at Algorithmia. Their enterprise MLOps platform manages all stages of the production ML lifecycle within existing operational processes, so you can put models into production quickly, securely, and cost-effectively. Unlike inefficient and expensive do-it-yourself MLOps management solutions that lock users into specific technology stacks, Algorithmia automates ML deployment, optimizes collaboration between operations and development, leverages existing SDLC and CI/CD systems, and provides advanced security and governance.

Algorithmia’s funding (Source: Crunchbase)

Today Algorithmia’s services are used by over 130,000 engineers and data scientists, including the United Nations, government intelligence agencies, and Fortune 500 companies.

“It’s [MLOps] going to be an essential component to enterprises industrializing their AI efforts in the future,” said Diego M. Oppenheimer, Algorithmia’s CEO in a recent interview with GitHub.

Databand.ai

Funding: $14.5 million

Databand brings in the similar flavor into the ML ecosystem. The team Databand is trying to solve the problems that arise due to increasing data workloads. The company founded by Josh Benamram, Victor Shafran and Evgeny Shulmanhelps helps data engineering teams catch data pipeline issues and trace the impact of those problems across end-to-end data flows. Databand’s platform includes an application for visualizing pipeline metadata, and an open source library for integrating with your Python, Java, Scala, or SQL data processes. Data pipeline monitoring is a key aspect of machine learning deployment. We can clearly see how targeting even a niche aspect of the whole ML deployment can land big investors.

Image credits: Gartner

Modern day software companies are in the process of or have already embraced machine learning as a key tool. Now they are at a crucial juncture where they can either leverage the MLOps services offered by these startups or build everything on their own. But, there are not many reasons why an organization looking to transition to ML will take the pain of MLOps. As companies look to leverage ML minus the deployment headache, niche players like OctoML will  continue to pop up. Even the latest Gartner survey lists scalability and acceleration of machine learning deployment as two driving forces that will continue to trend this year. According to Gartner, XOps— a variant of MLOps that deals with efficiencies in data, machine learning, model, platform will try to implement best DevOps practices and ensure reliability, reusability and repeatability. 

More Great AIM Stories

Ram Sagar
I have a master's degree in Robotics and I write about machine learning advancements.

Our Upcoming Events

Masterclass, Virtual
How to achieve real-time AI inference on your CPU
7th Jul

Masterclass, Virtual
How to power applications for the data-driven economy
20th Jul

Conference, in-person (Bangalore)
Cypher 2022
21-23rd Sep

Conference, Virtual
Deep Learning DevCon 2022
29th Oct

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM
MOST POPULAR

What can SEBI learn from casinos?

It is said that casino AI technology comes with superior risk management systems compared to traditional data analytics that regulators are currently using.

Will Tesla Make (it) in India?

Tesla has struggled with optimising their production because Musk has been intent on manufacturing all the car’s parts independent of other suppliers since 2017.