Machine Learning Deployment Is The Biggest Tech Trend In 2021

“What good is an ML model if it isn’t fast? doesn’t scale? isn’t accurate enough? takes weeks to deploy? and costs too much?”

Luis Ceze, CEO, OctoML

Having machine learning in a company’s portfolio used to be an investor magnet. Now, the market is bullish on MLaaS, with a new breed of companies offering machine learning services (libraries/APIs/frameworks) to help other companies get their job done better and faster. 

According to PwC, AI’s potential global economic impact will be worth $15.7 trillion by 2030. And, as interests slowly shift towards MLOps, it is possible that these companies, which promise to scale and accelerate ML deployment, might grab a bigger piece of the pie. Last week, OctoML raised $28 million. The Seattle-based startup offers a machine learning acceleration platform built on top of the open-source Apache TVM compiler framework project. The $28 million Series B funding brings the company’s total funding to $47 million. 

Image credits: OctoML

“90% of machine learning models don’t make it to production.”


For OctoML’s CEO, Luis Ceze, there is still a significant gap between building a model and making it production-ready. Between rapidly evolving ML models, wrote Ceze in a blog post, ML frameworks and a Cambrian explosion of hardware backends makes ML deployment challenging. “It is not easy to make sure your model runs fast enough and to benchmark it across different deployment hardware. Even if your determined machine learning team has hurtled through this gauntlet, they still have to go through a whole different set of challenges to package and deploy at the edge,” explained Ceze.

A good performance in ML models requires long hours of manual optimizations. These long hours will then translate into hefty cloud bills. Added to this is the model packaging which varies with devices and platforms. According to Ceze, there are no modern CI/CD integrations to keep up with model changes.

“What good is an ML model if it isn’t fast? doesn’t scale? isn’t accurate enough? takes weeks to deploy? and costs too much?,” questioned Ceze as he made a case for OctoML. 

OctoML addressed these pain points with their open-source machine learning compiler framework Apache TV, which according to the team, has quickly become the go-to solution for developers and ML engineers to maximize ML model performance on any hardware backend. “With OctoML we are establishing the first Machine Learning Acceleration Platform that will automatically maximize model performance while enabling seamless deployment on any hardware, cloud provider, or edge devices,” said Ceze.

Be it MLOps or XOps, these services are designed to ease the developers of technical debt that these mega ML models accumulate with changing complexities. Apart from OctoML, there are a few other startups that have succeeded in convincing the investors. Let’s take a look at couple of them:


Funding till date: $10 million

The team at Verta is building software for data science teams to address the problem of model management — how to track, version, and audit models used across products. Verta MLOps software supports model development, deployment, operations, monitoring, and collaboration enabling data scientists to manage models across their lifecycle. So far, the company has $10 million in funding and it promises to make robust, scalable, mature deployable models a reality.


Funding till date: $38.1 million 

Image credits: Algorithmia

“We’re obsessed with helping organizations get ML models into production because that’s the only way they can generate business value,” said the team at Algorithmia. Their enterprise MLOps platform manages all stages of the production ML lifecycle within existing operational processes, so you can put models into production quickly, securely, and cost-effectively. Unlike inefficient and expensive do-it-yourself MLOps management solutions that lock users into specific technology stacks, Algorithmia automates ML deployment, optimizes collaboration between operations and development, leverages existing SDLC and CI/CD systems, and provides advanced security and governance.

Algorithmia’s funding (Source: Crunchbase)

Today Algorithmia’s services are used by over 130,000 engineers and data scientists, including the United Nations, government intelligence agencies, and Fortune 500 companies.

“It’s [MLOps] going to be an essential component to enterprises industrializing their AI efforts in the future,” said Diego M. Oppenheimer, Algorithmia’s CEO in a recent interview with GitHub.

Funding: $14.5 million

Databand brings in the similar flavor into the ML ecosystem. The team Databand is trying to solve the problems that arise due to increasing data workloads. The company founded by Josh Benamram, Victor Shafran and Evgeny Shulmanhelps helps data engineering teams catch data pipeline issues and trace the impact of those problems across end-to-end data flows. Databand’s platform includes an application for visualizing pipeline metadata, and an open source library for integrating with your Python, Java, Scala, or SQL data processes. Data pipeline monitoring is a key aspect of machine learning deployment. We can clearly see how targeting even a niche aspect of the whole ML deployment can land big investors.

Image credits: Gartner

Modern day software companies are in the process of or have already embraced machine learning as a key tool. Now they are at a crucial juncture where they can either leverage the MLOps services offered by these startups or build everything on their own. But, there are not many reasons why an organization looking to transition to ML will take the pain of MLOps. As companies look to leverage ML minus the deployment headache, niche players like OctoML will  continue to pop up. Even the latest Gartner survey lists scalability and acceleration of machine learning deployment as two driving forces that will continue to trend this year. According to Gartner, XOps— a variant of MLOps that deals with efficiencies in data, machine learning, model, platform will try to implement best DevOps practices and ensure reliability, reusability and repeatability. 

Download our Mobile App

Ram Sagar
I have a master's degree in Robotics and I write about machine learning advancements.

Subscribe to our newsletter

Join our editors every weekday evening as they steer you through the most significant news of the day.
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

Our Recent Stories

Our Upcoming Events

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox

6 IDEs Built for Rust

Rust IDEs aid efficient code development by offering features like code completion, syntax highlighting, linting, debugging tools, and code refactoring