There’s no denying the fact that Deep Learning as we know it, how awesome it is when we can see that with minimal or no human-intervention a job can be done. It’s almost creating your own self but for a super-specific task. Since, Machine Learning, Deep Learning is dubbed to be one of the sexiest jobs of the 21st century(hyped?) so there has to be some starting point, a sort of a roadmap that you can follow to reach to the other side.
Luckily, we can now approach it relatively easier with modern frameworks like Tensorflow, PyTorch which gives you a high-level interface to build awesome stuff! Let’s discuss why you should start with PyTorch.
1. The Pythonic Nature
One of the USP’s of the PyTorch is its pythonic nature. That means line-by-line execution of the code and simultaneous building of the computation graphs just like in python. This makes it quite readable and easy to debug which is what all Pythonistas want.
2. The Conceptual Help
Although these frameworks are there in the first place to make your life easier, however, it’s also equally important to know the basics of Deep Learning, how it works, the secret ingredient, Backpropagation!! which tremendously increase your knowledge about Deep Neural Networks.
With PyTorch, if your concepts aren’t crystal clear you’ll not be able to build things pretty well so by choosing PyTorch, not only you’ll learn it the hard way but with a stronger foundation. You must be thinking that Tensorflow can also do the same and I’m not opposing you but trust me you’ll wind up with Keras much often if you google stuff and you don’t want Keras at the start of your learning as it might spoil you with its easiness.
3. Dynamic Computation Graphs
As discussed in the “Pythonic” section as well, PyTorch creates dynamic computation graphs just like python. This helps to comprehend the process better.
4. Easy switch to Accelerators
PyTorch comes with this awesome feature of utilizing CUDA cores and switching to available accelerators pretty quickly and easily. This will transfer the Tensor calculation to the GPU and make your computations faster.
device = torch.device(“cuda:0” if torch.cuda.is_available() else “cpu”)
model = models.resnet18(pretrained=True)
num_ftrs = model_ft.fc.in_features
# Here the size of each output sample is set to 2.
# Alternatively, it can be generalized to nn.Linear(num_ftrs, len(class_names)).
model.fc = nn.Linear(num_ftrs, 2)
# switching to Accelerator if available
model = model.to(device)
5. Support for Data Parallelism
PyTorch was built on the idea of using multi-core systems efficiently and it proudly delivers that. With the DataLoader mechanism in place which will handle large datasets with ease without hogging your system memory.
PyTorch’s heart is in Parallelism!!
data_transforms = {
‘train’: transforms.Compose([
transforms.ToTensor(),
transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]) ]),
‘val’: transforms.Compose([
transforms.ToTensor(),
transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]) ]),
}
data_dir = ‘data/hymenoptera_data’
image_datasets = {x: datasets.ImageFolder(os.path.join(data_dir, x), data_transforms[x])
for x in [‘train’, ‘val’]}
dataloaders = {x: torch.utils.data.DataLoader(image_datasets[x], batch_size=4, shuffle=True, num_workers=4)
for x in [‘train’, ‘val’]}
dataset_sizes = {x: len(image_datasets[x]) for x in [‘train’, ‘val’]}
class_names = image_datasets[‘train’].classes
You can put your model to the distributed model like the snippet below and this also proves helpful for multi-GPU setup.
import torch.distributed as dist
from torch.nn.parallel import DistributedDataParallel
dist.init_process_group(backend=’gloo’)
model = DistributedDataParallel(model)
6. The Ever Amazing TorchScript
Python is known for many things like its readability, easiness to learn, and interpretable execution process but not performance as compared to compiled languages like C++, Java. Don’t worry PyTorch got you covered in this area as well!. Torch engineers suspected this way before and made a TorchScript which is PyTorch but in C++ and made it super-modular.
Inferences in C++ makes 10x performance boosts
You can build and train models in PyTorch employing the readability and relatively easy debugging process of Python and convert it to a TorchScript and make inferences in C++ which will be a significant boost in performance. You can build and train models in PyTorch as well. Let’s look at some examples of both:
Converting PyTorch Model to TorchScript:
# Reference: #TODO- Add Link
from torchvision import models
# Download and load the pre-trained model
model = models.resnet18(pretrained=True)
# Set upgrading the gradients to False
for param in model.parameters():
param.requires_grad = False
# Save the model except the final FC Layer
resnet18 = torch.nn.Sequential(*list(resnet18.children())[:-1])
example_input = torch.rand(1, 3, 224, 224)
script_module = torch.jit.trace(resnet18, example_input)
script_module.save(‘resnet18_without_last_layer.pt’)
Inference in TorchScript:
torch::jit::script::Module module;
// argv[1] should be the path to the model
module = torch::jit::load(argv[1]);
/* We need to convert last layer input and output features from (512, 1000) to (512, 2) since we only have 2 classes */
torch::nn::Linear linear_layer(512, 2);
// Define the optimizer on parameters of linear_layer with learning_rate = 1e-3
torch::optim::Adam optimizer(linear_layer->parameters(), torch::optim::AdamOptions(1e-3))
7. Deployment
Serving Models has never been easier thanks to all the cloud services and a much simpler setup process. PyTorch supports a large band of devices like mobile (android) and web. A service as basic as Flask to using some sort of packaging mechanism like a Docker container. Then that service is often made network accessible via some sort of service, either using JSON over HTTP or an RPC technology like gRPC.
PyTorch is Mobile ready!
PyTorch comes with a support of ONNX (Open Neural Network Exchange) which helps you to export your models built in PyTorch to other mobile-friendly formats like Caffe2. Let’s see how you can export your model using ONNX support
import torch.onnx
import torchvision
dummy_input = torch.randn(1, 3, 224, 224)
model = torchvision.models.alexnet(pretrained=True)
torch.onnx.export(model, dummy_input, “alexnet.onnx”)
The following snippet is an example of utilizing torch models in android and ios.
## Save your model
torch.jit.script(model).save(“my_mobile_model.pt”)
## iOS prebuilt binary
pod ‘LibTorch’
## Android prebuilt binary
implementation ‘org.pytorch:pytorch_android:1.3.0’
## Run your model (Android example)
Tensor input = Tensor.fromBlob(data, new long[]{1, data.length});
IValue output = module.forward(IValue.tensor(input));
float[] scores = output.getTensor().getDataAsFloatArray();
You’ve reached the end!!
Making your concepts crystal clear in one of the stepping stones to your successful career in Deep Learning. Despite any level of abstraction you got to learn the things from the ground level and PyTorch can help you understand gradient calculation like an expert, catering till the end of the development pipeline as well. If you are a beginner try focussing on math and concepts and simultaneously applying to small using PyTorch. This will help you get a good command of Tensors as well. Rest, the internet is filled with great tutorials and anyone can learn anything.
Happy Learning!
