Deep learning is a broader field of machine learning, which uses artificial neural networks(ANN) to derive high-level features from the inputs. Deep learning or deep neural networks(DNN) architecture consists of multiple layers, specifically the hidden layers between the input and output layers.
Some common deep learning architectures are Convolutional neural networks(CNNs), Recurrent neural networks(RNNs), Long Short Term Memory(LSTM), deep belief networks(DBN), which have been applied to computer vision, audio and speech recognition and natural language processing(NLP) use cases. Even structured and unstructured tabular data have shown good performance using deep learning models.
Source: Link
In this article, I’ll discuss the deep learning frameworks available for different programming language interfaces.
A deep learning framework allows researchers and developers to achieve the state-of-art compactly and robustly. It helps in training and testing the model using APIs. These provide high-level performance and better management of dependencies.
1. Tensorflow
Developed by Google Brain, Tensorflow is by far, one of the most used deep learning frameworks. Tensorflow provided a wide range of APIs(application programming language) from preprocessing to the modelling of data. It is written in Python, C++ and CUDA. Runs on almost all platforms – Linux, Windows, macOS, iOS and Android. For Android and iOS, Tensorflow provides the TensorflowLite version. For deployment, Tensorflow has powerful clustering support. Tensorflow can be used with CPUs, GPUs and TPUs as well. Tensorflow has excellent documentation on its official website containing all modules. Latest version Tensorflow 2.0 has undergone major advances.
Tensorflow API is widely used in Python, which is completely under stable releases. Other language APIs are under development and not in stable releases; they use it without API backward compatibility – C++, Javascript, Java, Go. Some languages even use it as 3rd party packages – Haskell, C#, R, Julia, Scala, Ruby, MATLAB.
Applications – Google Teachable Machine(No code ML training platform), RankBrain(SEO), Deep Speech(speech translation), Nsynth(for making music), Uber, Delivery Hero, Ruangguru, Hepsiburada, 9GAG, Channel.io.
2. PyTorch
Developed by Facebook’s AI Research Lab, PyTorch is another widely used deep learning framework mainly for its Python interface. PyTorch is built on top of the Torch library. PyTorch has similarities with Tensorflow and thus in major competition with it. PyTorch was mainly developed for research and production deployment purpose. Compatible with Linux, Windows, macOS, Android and iOS. Using TorchServe, PyTorch faster speed and ease of access to deploy models.TorchScript provides flexibility in the transition of graph mode functionalities. Distributed training allows PyTorch to provide optimisation in models. Active development community for computer vision and NLP to reinforcement learning techniques. Some milestones by PyTorch- HuggingFaces Transformers, PyTorch Lightening, Tesla Autopilot, Uber Pyro, Catalyst.
API – Mainly used in Python but also has an interface for C++.
Applications – NVIDIA, Apple, Robin Hood, Lyft, Ford Motor Company
3. Caffe
Developed by the University of California, Berkeley, Caffe supports various architectures for image segmentation and classification. Caffe is written in C++. Compatible with Linux, Windows, macOS. Works on CPUs but better performance with GPU acceleration. Caffe is preferred for its speed and industry deployment. Can process up to 60million images with NVIDIA GPU. Has proper documentation and active community of developers to support prototypes for startups and research works.
API – Python and Matlab
Applications – CaffeOnSpark(A venture from Yahoo with integration on Apache Spark), Caffe2(by Facebook), Snap Inc., Cadence Design Systems, Qualcomm
4. MXNet
Developed by Apache Software Foundation, MXNet is an open-source deep learning framework built for high scalability and support by various programming languages. MXNet is written in multiple languages – C++, Python, Java, Scala, Julia, R, Javascript, Perl, Go and Wolfram Language. It is known for its fast model training. Compatible with Windows, macOS, Linux. Can run on smart devices as its lightweight and memory efficient. By its hybrid nature through distributed training MXNet allows combined symbolic and imperative operations, making it efficient, flexible and portable. Its active community is by far the most efficient one having synchronised ideas on AI and deep learning.
API – Gluon Python API, support by Scala, Julia, R, Java, Perl, Clojure
Applications – AWS(as DL framework), Warner Bros. Entertainment Group of Companies, Elbit Systems of America, Kwai
5. Chainer
Development by Preferred Networks in venture with Intel, Nvidia, IBM, and Microsoft. Chainer is a cross-platform deep learning framework completely built-in Python based on Numpy and CuPy libraries. Chainer emerged due to its dynamic computational graphs which are readily available through APIs. This feature is known as define-by-run approach. Chainer has 4 extension libraries – ChainerRL(for reinforcement learning), ChainerCV(for computer vision), ChainerMN(for multiple GPU usage), ChainerUI(for management and visualisation).
API – Python
Applications – PaintsChainer(automatic colourization), JPMorgan Chase, Novu LLC, Facebook ADP, Mad Street Den.
6. DeepLearning4j
DL4j or deep learning for Java is the only deep learning framework to be built on Java for JVM(Java Virtual Machine) and written in Java, CUDA, C++, C. It is developed by Eclipse. It covers a wide range of deep learning algorithms. Operating systems supported are Linux, Windows, macOS, iOS and Android. Has support from distributed systems Apache Spark and Hadoop.
API – Support for all JVM based languages Java, Scala, Clojure, Kotlin.
Applications – Cybersecurity, Fraud Detection, anomaly detection and also Used in RapidMiner and Weka, U.S. Bank, Livongo, Thermo Fisher Scientific, NovoDynamics Inc.
7. Flux
The only machine learning library to be written in Julia. Flux supports deep learning architectures CNNs, RNNs, LSTMs. Flux supports the building of pipelines to integrate Flux models. It has extensions such as DiffEqFlux.jl for neural differential equations through differential programming and Zygote.jl for automatic differentiation.
API – Julia
Applications – used in homomorphic encrypted data(an encryption technique which doesn’t need decryption of data)
8. Matlab – Deep Learning Toolbox
Developed by MathWorks, the MATLAB deep learning Toolbox is written in C, C++, Java and MATLAB. It can run on operating systems such as Linux, Windows and macOS. This framework supports all kinds of deep learning architectures CNNs, RNNs, LSTMs, also GANs and pretrained models such as ResNets The toolbox provides parallel training with GPUs. The Deep Designer app keeps track of the parameters being used.
9. Microsoft Cognitive Toolkit
Microsoft Cognitive Toolkit(CNTK) developed by Microsoft Research and written in C++. Supported by the operating system – Windows and Linux. CNTK represents neural networks as a directed graph wherein the leaf nodes are for input parameters and other nodes are for matrix operations. CNTK provides deep learning architectures feed-forward networks DNNs, CNNs, RNNs, LSTMs. Has automatic differentiation and parallelism support multiple GPUs and servers.
API – Python, C#, C++
Application – Sequence2Sequence, LightRNN, Reinforcement Learning with DQN, HOPE Technik Pte LTD, Rockwell Collins, SparkCognition
10. Keras
Cross-platform neural network library is written in Python, developed by Francis Chollet. Keras is a high-level API built on top of Tensorflow. Keras is the most used deep learning framework in Kaggle. Keras best runs on GPUs and TPUs. Keras models can easily be deployed to the web, iOS and Android. Keras is known for its fast computation, user-friendliness and ease of access. Keras has an active community and thus under constant development.
API – Python
Applications – Used by scientific organisations such as NASA, CERN, NIH and LHC, Lockheed Martin, Ford Motor Company
11. Theano
It was developed by Montreal Institute of Learning Algorithms, written in Python and CUDA. Theano is used in the optimization of multi-dimensional arrays. Theano provides faster computation. Theano is built on top of Numpy. It is designed to perform high dimensional numeric calculations of neural networks. Runs on operating systems such as Linux, Windows and macOS. Performs better on GPU powered systems.
API – Python
Applications: WellsFargo, SparkCognition, Data Economy, Northrop Grumman, HOPE Technik Pte LTD
