At the ongoing annual cloud conference, AWS re:Invent event, the e-commerce giant has been announcing a staggering number of product launches, tools, frameworks, services and more. From AI chips to enterprise applications, the event is covering all the aspects and addressing needs of a developer to build innovative products.
Here is a list of six libraries and tools, in no particular order, that was launched so far at the AWS re:Invent 2020.
SageMaker Distributed Data-Parallel (SDP)
AWS re:Invent 2020 introduced two new distributed training libraries for Amazon SageMaker, one of them is the SageMaker Distributed Data-Parallel (SDP). The libraries are meant to provide integrated methods for the users to quickly train large deep learning models. SageMaker distributed data-parallel extends SageMaker’s training capabilities on deep learning models with near-linear scaling efficiency in order to achieve fast time-to-train with minimal code changes.
Some of the benefits of this library are —
- SDP optimises the training job for AWS network infrastructure and Amazon EC2 instance topology.
- SDP takes advantage of gradient updates to communicate between nodes with a custom AllReduce algorithm.
SageMaker Distributed Model Parallel (SMP)
The second library announced as the distributed training library for Amazon SageMaker is SageMaker Distributed Model Parallel or SMP. Amazon SageMaker distributed model parallel is a model parallelism library that can be used for training complex deep learning models which were previously challenging to train because of the limitations in GPU memory.
Some of its benefits are —
- This library automatically as well as efficiently splits a model over multiple GPUs and instances, and coordinates model training. This allows users to increase the prediction accuracy by creating massive models with extended parameters.
- One can use SMP to automatically create partitions in the existing TensorFlow and PyTorch workloads across multiple GPUs with minimal code changes.
AWS IoT SDK Includes OTA Library
AWS re:Invent 2020 event also announced AWS IoT Device SDK for Embedded C (C-SDK) version 202012.00, which now includes an over-the-air update (OTA) library and a PKCS #11 implementation (corePKCS11). According to a blog post, the OTA library makes it easier to manage notifications, download, and perform cryptographic verification of firmware updates. The OTA and corePKCS11 libraries have been optimised for memory usage and modularity, and have undergone code quality checks.
AWS IDE Toolkit For AWS Cloud9
In another announcement, re:Invent 2020 launched the AWS Toolkit for AWS Cloud9 to enable users of the browser-based IDE to efficiently handle the core AWS services by the graphical user interface. A key component of this toolkit is the Resource Explorer, which is a view that permits users to navigate as well as interact with the popular AWS resources such as AWS Lambda, Amazon API Gateway, Amazon S3, among others. Also, the AWS Toolkit extends AWS Cloud9’s support for AWS Lambda functions, allowing users to quickly invoke, import, deploy or delete functions.
Amazon Kendra Launches Connector Library
The connector library for Amazon Kendra was also launched to make it easy to synchronise data from multiple content repositories. Connectors can be scheduled to automatically sync indexes with the data source, such that one can securely search through the most updated content. Also, the Amazon Kendra connector library offers native connectors, as well as an expanding list of partner connectors from AWS partners like Raytion.
Amazon Braket Now Supports PennyLane
Amazon Braket announced the support for PennyLane, which is an open-source software framework designed for hybrid quantum computing. Pennylane provides interfaces to common ML libraries, including PyTorch and TensorFlow. The integration of this framework with Amazon Braket allows users to test and fine-tune algorithms faster at a larger scale on scalable and fully managed simulators and run them on the choice of quantum computing hardware.
