Micron Technology has announced the world’s first open-source, heterogeneous-memory storage engine, which has been designed explicitly for solid-state drives and storage-class memory.
According to the company website — the legacy storage engines was born in the era of hard disk drives, which has failed to architecturally provide for the increased performance and reduced latency of next-generation non-volatile media.
The company claims to develop HSE, which is now available on GitHub, has been designed for developers using all-flash infrastructure who require the benefits of open-source software, including the ability to customise or enhance code for their unique use cases.
When asked, Derek Dicker, corporate VP and general manager of the Storage Business Unit at Micron said, “As the only company developing storage class memory, flash and DRAM technologies, Micron is uniquely positioned to build a software stack that accelerates applications running in today’s flash-based storage environments as well as storage class memory-based infrastructure of the future.”
He further said, “We have delivered a first-of-its-kind innovation for open-source storage developers, unlocking the full potential of high-performance storage applications.”
The company stated that along with delivering performance and endurance improvements, heterogeneous-memory storage engine reduces latency, especially for large-scale data sets, through intelligent data placement. Apart from that, it also improves the throughput of particular storage applications by up to six times, reduces latency 11 times and improves SSD endurance by seven times.
The company further stated on their release that heterogeneous-memory storage engine could also take advantage of multiple classes of media concurrently, such as flash and 3D XPoint technology. When a Micron X100 NVMe SSD, the world’s fastest SSD, is added to a set of four Micron 5210 QLC SSDs, throughput more than doubles and read latency improves nearly four times.
Some of the features and benefits of the HSE are mentioned below:
- HSE has the capability of integrating with MongoDB, the world’s most popular NoSQL database, that delivers significant performance improvements, reduces latency and takes full advantage of modern memory and storage technologies. It can also be integrated with other storage applications like NoSQL databases and object stores.
- HSE is ideal when performance at scale matters, including very large data size, large key counts (billions), high operation concurrency (thousands), or deployment of multiple classes of media.
- The platform is designed to be extensible to new interfaces and new storage devices, enabling use with a broad range of applications and solutions, including databases, internet of things (IoT), 5G, artificial intelligence (AI), high-performance computing (HPC) and object storage.
- HSE is capable of delivering additional performance for software-defined storage, such as Red Hat Ceph Storage and Scality RING, that enables cloud-native applications through containerised platforms like Red Hat OpenShift as well as tiered performance for file, block and object storage protocols for multiple use cases.
- HSE is delivered as an embeddable key-value database; Micron will maintain the code repository on GitHub.
On the same occasion, Stefanie Chiras, vice president and general manager of Red Hat Enterprise Linux said that “We see enormous potential in the technologies being introduced by Micron, especially as it takes an innovative approach in lowering the latency between compute, memory and storage resources.”
Further added, “We look forward to working with Micron in the open-source community to build out these innovations further, ultimately bringing new choices to the storage world derived from open standards and concepts.”
Brad King, field chief technology officer and co-founder of Scality also said, “As the demand for object-based storage continues to increase and it’s deployed for more and more workloads, it’s no surprise that our customers are increasingly interested in fast object storage.”
“While our storage software can support ‘cheap and deep’ on the lowest-cost commodity hardware for the simplest workloads, it can also exploit the performance benefits of technologies like flash, storage class memory and SSDs for very demanding workloads. Micron’s HSE technology enhances our ability to continue optimising flash performance, latency and SSD endurance without trade-offs,” concluded King.
