PyTorch is one of the most used libraries for building deep learning models, especially neural network-based models. In many tasks related to deep learning, we find the use of PyTorch because of its features and capabilities like production-ready, distributed training, robust ecosystem, and cloud support. In this article, we will learn how we can build a simple neural network using the PyTorch library in just a few steps. For this purpose, we will demonstrate a hands-on implementation where we will build a simple neural network for a classification problem.
We will accomplish this implementation in the following steps:-
Sign up for your weekly dose of what's up in emerging technology.
- Step 1: Creating the data
- Step 2: Loading the data using data loader
- Step 3: Building a neural network model
- Defining the neural net class
- Instantiating the classifier
- Step 4: Training the neural network model
- Optimizing loss curve
- Defining decision boundaries
- Step 5: Making Predictions
Let’s start with the first step, where we will create a dataset for implementation.
Step 1: Creating the data
In this article, we will understand the intuition behind defining and working the neural networks in PyTorch. For this purpose, we will build a neural network from scratch and apply it to a classification problem. For this purpose, we will create a dataset using make_classification under sklearn.datasets which we use in making a classification data set.
Using the following lines of code we can create the dataset for classification:
from sklearn.datasets import make_classification X, Y = make_classification( n_features=4, n_redundant=0, n_informative=3, n_clusters_per_class=2, n_classes=3 )
Using the above code we have created a dataset for classification wherein the dataset we have 4 features with 3 informative features and 3 classes.
Let’s visualize the dataset.
import matplotlib.pyplot as plt plt.title("Multi-class data, 4 informative features, 3 classes", fontsize="large") plt.scatter(X[:, 0], X[:, 1], marker="o", c=Y, s=25, edgecolor="k")
Here we can see our dataset in a two-dimensional space and points are clearly visible from three classes.
After making the dataset, we are ready to build a classification model. Since in this article, we are discussing a simple implementation of a neural network using the PyTorch, we will use a two-layer neural network where we can use sigmoid as our activation function. Data points in the above graph will be our input coordinates and classes related to the dots are the ground truth.
Step 2: Loading the data using data loader
Before defining the model, we are required to split our dataset into tests and train sets. That can be done using the following lines of codes.
from sklearn.model_selection import train_test_split X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.33, random_state=42)
Let’s check the shape of the split dataset.
X_train.shape, X_test.shape, Y_train.shape, Y_test.shape
After this, we are required to define the train data as a PyTorch tensor.
import torch Y_test = torch.from_numpy(X_test) Y_test = torch.from_numpy(np.asarray(Y_test))
To use the batches of the dataset we are required to put the data through the Dataloader module of PyTorch. Using the below lines of code we can do that.
from torch.utils.data import Dataset, DataLoader class Data(Dataset): def __init__(self): self.X=torch.from_numpy(X_train) self.Y=torch.from_numpy(Y_train) self.len=self.X.shape def __getitem__(self,index): return self.X[index], self.Y[index] def __len__(self): return self.len data=Data() loader=DataLoader(dataset=data,batch_size=64)
Checking the shape of the data under the tensor.
print(data.X[0:5]) print(data.X.shape) print(data.Y[0:5]) print(data.Y.shape)
Step 3: Building a neural network model
First of all, we will define the dimensions of the network.
input_dim=4 # how many Variables are in the dataset hidden_dim = 25 # hidden layers output_dim=3 # number of classes
Step 3.1: Defining the neural net class
A class for creating simple networks can be defined using the following lines of codes.
import torch.nn as nn class Net(nn.Module): def __init__(self,input,H,output): super(Net,self).__init__() self.linear1=nn.Linear(input,H) self.linear2=nn.Linear(H,output) def forward(self,x): x=torch.sigmoid(self.linear1(x)) x=self.linear2(x) return x
Step 3.2: Instantiating the classifier
Now with the above-defined dimensions, we can instantiate a model instance using the class defined in the last step.
Let’s check the information about the model
Here we can see that the model which we have prepared will take 4 features as input and will give 4 features as output.
Step 4: Training the neural network model
Before training, we are required to define the criterion to calculate gradients of parameters and optimizers to update the parameters.
criterion=nn.CrossEntropyLoss() optimizer=torch.optim.SGD(clf.parameters(), lr=0.1)
After defining the criterion and optimizer, we are ready to train our model. Using the following lines of codes we can train it.
learning_rate = 1e-1 loss_list =  for t in range(1000): y_pred = clf(x) loss = criterion(y_pred, y) loss_list.append(loss.item()) clf.zero_grad() loss.backward() with torch.no_grad(): for param in clf.parameters(): param -= learning_rate * param.grad
In this training, we have defined methods that will get the accumulated gradient to zero, append the loss on the loss_list, and will get a new gradient, and update parameters using the backward propagation.
Step 4.1: Optimizing loss curve
The loss curve will let us know about the performance of the model.
Visualizing loss curve
step = np.linspace(0,1000,1000) plt.plot(step,np.array(loss_list))
In the above output, we can see that our loss curve converged, and using this model we can also define our decision boundaries.
Step 4.2: Defining decision boundaries
params = list(clf.parameters()) w = params.detach().numpy() b = params.detach().numpy() t= params.detach().numpy() plt.scatter(X[:, 0], X[:, 1], c=Y,cmap='jet') u = np.linspace(X[:, 0].min(), X[:, 0].max(), 2) plt.plot(u, (0.5-b-w*u)/w) plt.plot(u, (0.5-t-w*u)/w) plt.xlim(X[:, 0].min()-0.5, X[:, 0].max()+0.5) plt.ylim(X[:, 1].min()-0.5, X[:, 1].max()+0.5)
In the above output, we can see that we have defined our decision boundaries and it is looking good. Now we can use the model for making predictions.
Let’s make the predictions with the trained neural network.
x_val = torch.from_numpy(X_test) z=clf(x_val) yhat=torch.max(z.data,1) yhat
In the above output, we can see the predictions of the class labels of the given input patterns. Using a few of the modifications like the number of layers or number of epochs we can improve the performance of our model.
In this article, we have learnt how to define a neural net in PyTorch in just a few steps. We have defined a dataset on which we have trained a double-layer neural network. We have used this network for classification and implemented that in PyTorch. This can be an example of a simple multiclass classification neural network also.