Hands-On Implementation Of Perceptron Algorithm in Python

Artificial Neural Networks(ANNs) are the newfound love for all data scientists. From classical machine learning techniques, it is now shifted towards deep learning. Neural networks mimic the human brain which passes information through neurons. Perceptron is the first neural network to be created. It was designed by Frank Rosenblatt in 1957. Perceptron is a single layer neural network. This is the only neural network without any hidden layer. Perceptron is used in supervised learning generally for binary classification. 

Source: link

The above picture is of a perceptron where inputs are acted upon by weights and summed to bias and lastly passes through an activation function to give the final output. 

Output = Activation function * (Bias + (Input Matrix * Weight matrix))

Input matrix X1 to Xn and Weight matrix is W1 to Wn,
Bias is to allow shift activation. Bias is taken as W0,
The activation function is used to introduce non-linearities into the network. 
Generally, this is sigmoid for binary classification. 

Perceptron Implementation in Python

Now let’s implement the perceptron algorithm in python from scratch

Numpy library for summation and product of arrays

import numpy as np

We’ll create the perceptron class and declare certain parameters such as learning rate(to keep track of the updations in weights), epochs(iterations) and random state and the constructor is defined to initialize the variables.

class Perceptron(object):
  def __init__(self, learning_rate=0.01, n_iter=100, random_state=1):
    self.learning_rate = learning_rate
    self.n_iter = n_iter
    self.random_state = random_state

The fit method is for mapping the training data inputs(X) with training labels or targets(y). This method will call the other two functions internally. All the necessary calculations take place here. Two matrices are maintained, one for weights updation and another for error updation. Weights are randomly selected and as the algorithm learns its updated with new values. Errors are maintained to keep track of misclassifications. As the algorithm learns errors will be improved.  

  def fit(self, X, y):
    rand = np.random.RandomState(self.random_state)
    self.weights = rand.normal(loc=0.0, scale=0.01, size=1 +  X.shape[1])
    self.errors_ = []

The weights array should have the same dimension as the input array otherwise dot product is not possible.

Errors are updated alongside wherever there’s updation for the summation value. Error is calculated using the learning rate and difference of actual and prediction value and added back to the weights array.

    for _ in range(self.n_iter):
      errors = 0
      for x, target in zip(X, y):
        update = self.learning_rate * (target - self.predict(x))
        self.weights[1:] += update * x
        self.weights[0] += update
        errors += int(update != 0.0)
        self.errors_.append(errors)
      return self

Bais is taken as the first value of the weights array. During each iteration/epoch summation of bais with the product of weight at that instant to input value(the dot product). 

  def net_input(self, X):
    z = np.dot(X, self.weights[1:]) + self.weights[0]
    return z

Prediction is made based upon the summation result  

  def predict(self, X):
    return np.where(self.net_input(X) >= 0, 1, -1)

Now our perceptron algorithm is ready its time we load a dataset and check predictions upon it

from sklearn.datasets import load_iris
X,y = load_iris(return_X_y=True)

I’m using the iris dataset from sklearn for demonstration

Visualization of the data 

import matplotlib.pyplot as plt
import numpy as np
%matplotlib inline
plt.scatter(X[:50, 0], X[:50, 1],
            color='green', marker='x', label='setosa')
plt.scatter(X[50:100, 0], X[50:100, 1],
            color='red', marker='o', label='versicolor')
plt.xlabel('sepal length')
plt.ylabel('petal length')
plt.legend(loc='upper right')
plt.show()

Finally, the perceptron class defined with required parameters and fit method is called 

per = Perceptron(learning_rate=0.1, n_iter=100, random_state=1)
per.fit(X, y)
plt.plot(range(1, len(per.errors_) + 1), per.errors_, marker='o')
plt.xlabel('Epochs')
plt.ylabel('Number of updates')
plt.show()

This plot shows the variation of the algorithm of how it has learnt with each epoch.

Conclusion

I’ve shown a basic implementation of the perceptron algorithm in Python to classify the flowers in the iris dataset. For starting with neural networks a beginner should know the working of a single neural network as all others are variations of it. Perceptron has variants such as multilayer perceptron(MLP) where more than 1 neuron will be used. 

The complete code of the above implementation is available at the AIM’s GitHub repository. Please visit this link to find the notebook of this code.