Dell has released Omnia, an open-source software package to simplify AI and compute-intensive workload deployment and management. Omnia automates the management of high-performance computing, AI, and data analytics to create a repository of hardware resources. Omnia has been developed by Dell’s HPC and AI Innovation Lab along with Intel and Arizona State University (ASU).
Advanced computing workflows that use different techniques to solve complex problems have led to the convergence of artificial intelligence, HPC, and data analytics. While this confluence is promoting better innovation and faster discovery, it is also putting pressure on IT to support environments that are growing in complexity. The IT teams are required to carry out manual configuration and reconfigurations of servers, storage, and networking between clusters to provide the resources needed for shifting workload demands.
Omnia is a deployment tool to configure Dell EMC PowerEdge servers that run standard RPM-based Linux OS images into clusters that are capable of supporting HPC, AI, and data analytics workloads. Omnia gives the IT flexibility to run workloads and advanced applications such as machine learning and deep learning, high-throughput computing, data analytics, and simulation, in the same environment, across a single interface for cluster provisioning and deployment.
The Omnia software stack helps in simplifying and speeding up the deployment process for mixed workloads. Omnia uses Slurm (job scheduler for Linux), Kubernetes, and other packages to manage jobs and run different workloads on the same converged solution. Dell describes Omnia as a set of Ansible playbooks that speed the deployment of converged workloads. In a single day, a stack of servers, networking switches, and storage arrays can be transformed into a cluster for running all HPC, AI, and data analytics workloads.
Red Hat originally created Ansible to help in configuration management and application development. It is a simple to use IT automation engine that converts inefficient and repetitive tasks to predictable, simple, and scalable solutions. With Ansible, developers can spend more time on their work and help operations support deployment pipelines. With these capabilities, developers get a quick, comprehensive, and coordinated approach to delivering business value.
“It will put the right software on each server based on your use case whether it’s HPC simulations [or] neural networks for AI and reduce your time to time deployments. One of the coolest things about Omnia is that you can compose and recompose the stack based on what you need, really taking an infrastructure as code approach here,” said Caitlin Gordon, vice president, product management, Dell.
Further, Omnia uses community-supported package repositories for Linux and automates all the steps for running a functional multi-mode Kubernetes cluster. Omnia offers best-of-breed deployments for Kubernetes through Helm and OperatorHub. It provides configuration files for dynamic and persistent storage, deploys machine learning platforms like Kubeflow automatically, and points to optimised containers in DockerHub, Nvidia GPU Cloud etc in accelerated and unaccelerated workloads.
Dell expands on HPC on-demand
HPC has been long used by data scientists for solving problems at scale. Organisations from different industries are accelerating the adoption of HPC for artificial intelligence and other applications. HPC systems are complex and require specific technical expertise and massive funding to forward the projects. Many companies lack the budget, skills and time to build, optimise, and manage HPC clusters. For this, Dell offers flexible HPC on-demand with white-glove managed services.
As per the recent announcement, Dell’s HPC on-demand is now supporting VMware environments. Customers can now use advanced infrastructure to meet their computing needs using HPC on-demand and R system with the addition of VMware Cloud Foundation, VMware Cloud Director, and VMware vRealize Operations.
Mercury Marine, a manufacturer of marine propulsion systems, uses computer-aided hydrodynamics simulations. Using Dell’s HPC on-demand, the company has been able to reduce the simulation time from 48 hours to just 2 hours.