MLOps, also known as DevOps for machine learning, is facing a talent crunch. GitLab’s latest survey also shows that developers’ roles are shifting toward the operations side. Most developers are taking on test and ops tasks, especially around cloud infrastructure and security.
Further, the report found close to 38% of developers now define or create the infrastructure their app runs on, about 13% monitor and respond to that infrastructure, and 26% of developers instrument the code they’ve written for production monitoring (up by 18% from last year).
Sign up for your weekly dose of what's up in emerging technology.
Nearly 43% of the survey respondents have been doing DevOps for three to five years. “That’s a sweet spot where they have known success and are well-seasoned,” highlighted the report.
MLOps engineering is poised to take off in a big way. We have curated a list of top MLOps learning resources to help you get a handle on the subject.
DeepLearning.AI recently introduced a new specialised source called Machine Learning Engineering for Production (MLOps) Specialisation. The course is currently available on Coursera. Curated by tech evangelists Andrew Ng, Robert Crowe, Laurence Moroney and Cristian Bartolomé Arámburu, the course will help individuals become machine learning experts and enhance production engineering capabilities.
Here’s an overall highlight of the course:
- Design a machine learning production system end-to-end: Project scoping, data needs, modelling strategies, deployment requirements and more.
- Establish a model baseline, address concept drift, develop and deploy prototype, and continuously improve a productised machine learning application.
- Build data pipelines by gathering, cleaning, and validating datasets. Establish data lifecycle by using data lineage and provenance metadata tools.
- Apply best industry practices and progressive delivery techniques to maintain and monitor a continuously operating production system.
Click here to know more about the course.
Coauthored by Mark Treveil and the team Dataiku, this book covers the following aspects:
- Fulfil data science value by reducing friction throughout machine learning pipelines
- Refine machine learning models through retraining, periodic tuning, and complete remodelling to ensure long-term accuracy.
- Design the MLOps life cycle to minimise organisational risks with fair, unbiased, and explainable models.
- Operationalise machine learning models for pipeline deployment and external business systems that are more complex and less standardised.
The platform offers a one-stop solution to discover, learn and build all things machine learning. It provides a series of lessons around machine learning and MLOps, which includes the basics of applying machine learning to building production-grade applications and products. Goku Mohandas curated the course.
The course covers various aspects of the machine learning pipeline, including data, cost, utility and trust. The courses have been created to educate the community on developing, deploying, and maintaining applications built using ML.
Computer scientist Chip Huyen’s blog post has summarised all the latest technologies/tools used in MLOps. The complete list is available here.
In this list, there are about 285 MLOps tools. Interestingly, out of 180 startups present in the list, 65 startups had raised funds in 2020, and a large majority of them are still in the data pipeline category. Some of the Indian MLOPs startups mentioned in the list include Playment, Dataturks, Scribble Data and Dockship.
The website offers collective resources for facilitating machine learning operations with GitHub. It gives access to use GitHub for automation, collaboration and reproducibility in machine learning workflows.
The site has blog posts explaining how to GitHub for data science and MLOps; open-source GitHub Actions tool that facilitates MLOps; documents and resources for getting started with MLOps; repository templates, examples and related projects that demonstrate various GitHub features for data science and MLOps; recorded talks, demos and tutorials and more.
The website is a collection of resources to understand MLops, starting from books, newsletters, workflow management, data engineering in MLOps (DataOps), communities, articles, feature stores, model deployment and serving, infrastructure, economics and more.
A complete list of links and resources for MLOps is available on GitHub.
Modelled on a Kubernetes SIG, the MLOps community is an open platform where machine learning enthusiasts, developers and industry professionals collaborate and discuss the best practices around machine learning operations (MLOps or DevOps for ML).