Hands-On Guide To Image Classification Using R

Image classification is an important Machine Learning task which assigns a label to an input image. It is quite a common practice among ML enthusiasts to implement the task of classifying images using Pythonic code. This article will guide you to perform image classification using R programming language.

Before proceeding, you can check our following articles related to the fundamental concepts of R and its uses cases other than image classification implemented using it.

• Basics of R
• Some free e-books for R
• Time series analysis and forecasting with R
• Exploratory Data Analysis (EDA) in Python vs R

Practical implementation using R

Here’s a demonstration of performing image classification using RStudio version 1.2.1335. We have used the Fashion-MNIST dataset with 28*28 dimensional gray-scale images categorized into 10 classes. The whole dataset has been partitioned into a training set of 60,000 images and a test set comprising 10,000 images. The step-wise explanation of the code is explained below:

1. Install TensorFlow package 

 library(tensorflow)
 install_tensorflow() 

2. Install Keras package

install.packages(“keras”)

3. Load the Keras package into the current session.

library(keras)

4. The R interface to Keras library has an in-built Fashion-MNIST dataset, which can be loaded using the dataset_fashion_mnist() function.

 data <- dataset_fashion_mnist()

Output:

5. Load training and test sets into four different arrays- two for images and labels of the training set and the other two for those of the test set. 

We c() function here to combine images and labels into a vector.  

 c(training_imgs, training_lbls) %<-% data$train
 c(testing_imgs, testing_lbls) %<-% data$test 

6. The output labels are integers in the range 0-9. We create a vector with class names corresponding to each integer label, which will later be used to visualise the classification output. The class labels can be found in the ‘Details’ section of this page.

output_categories = c('T-shirt/top','Trouser','Pullover','Dress','Coat','Sandal','Shirt','Sneaker','Bag','Ankle boot')

7. Explore the data by checking the dimensions of training and test set arrays for images and labels. The dim() function can be used to retrieve the images’ dimensions.

dim(training_imgs)

Output:  [1] 60000    28    28

dim(training_lbls)

Output: [1] 60000

dim(testing_imgs)

Output: [1] 10000    28    28

dim(testing_lbls)

Output: [1] 10000

The above outputs show that each of the training and test set has 28*28 dimensional images, with the count of images for the former set being 60,000 and that for the latter one being 10,000.

8. Preprocess the data before model training.

First, install and load the tidyr R package, making the data tidy(easy to handle) for visualization.

 install.packages(“tidyr”)
 library(tidyr) 

Install the ggplot2 graphics package for visualization.

 install.packages(“ggplot2”)
 library(ggplot2) 

Check if the set of training images is a dataset and assign hat dataset to a variable named ‘img’

img <- as.data.frame(training_imgs[2, , ])

Count the number of images in img using ncol(), create a sequence from 1 upto the number of columns using seq_len() and then retrieve the column names from the img set of images.

colnames(img) <- seq_len(ncol(img))

Retrieve the images from the dataset 

img$y <- seq_len(nrow(img))

Get the pixel values of images

 img <- gather(img, "x", "value", -y)
 img$x <- as.integer(img$x) 

Initialize a ggplot object. Provide img set as parameter and also specify the aesthetics of the plot using aes().

 ggplot(img, aes(x = x, y = y, fill = value)) +
 #Plot the geometric tiles
   geom_tile() +   
 #Create 2-color gradient (b/w) for the plot   
   scale_fill_gradient(low = "white", high = "black", na.value = NA) +
 #Create y scale marked from bottom to top
   scale_y_reverse() +
 #Create theme for data and non-data elements’ display
   theme_minimal() +
   theme(panel.grid = element_blank())   +
   theme(aspect.ratio = 1) +
   xlab("") +    #x-axis label
   ylab("")       #y-axis label 

Output:

9. The above output image shows that pixel values of the input images are in the range 0-255. We carry out normalization to convert the pixel values that fall in the range 0-1 before feeding the images to the neural network. Divide pixel values of each of the training and test set images by 255 for such normalization.

 training_imgs <- training_imgs / 255
 testing_imgs <- testing_imgs / 255 

10. Display some initial images.

 #Set graphical parameters using par() function. Fill 4*4 matrix column-wise (specified #using mfcol parameter) for displaying 16 images
 par(mfcol=c(4,4))
 #specify margin sizes using mar parameter
 par(mar=c(0, 0, 1.5, 0), xaxs='i', yaxs='i')
 #Plot the ith image
 for (x in 1:16) { 
   img <- training_imgs[x, , ]
   img <- t(apply(img, 2, rev)) 
 #Create grid of rectangles
   image(1:28, 1:28, img, col = gray((0:255)/255), xaxt = 'n', yaxt = 'n',
         main = paste(output_categories[training_lbls[x] + 1]))
 } 

Output:

11. Build the neural network model.

 #Create linear stack of network layers
 model <- keras_model_sequential()
 model %>%
 #Flatten the 28*28 images into 1D array
   layer_flatten(input_shape = c(28, 28)) %>%
 #Create hidden layer with 128 neurons and ReLU activation function
   layer_dense(units = 128, activation = 'relu') %>%
 #Output layer with 10 neurons (as there are 10 output classes) and softmax activation function
   layer_dense(units = 10, activation = 'softmax') 

12. Compile the model

 model %>% compile(
   optimizer = 'adam',    #model optimization technique
   loss = 'sparse_categorical_crossentropy',   loss function
   metrics = c('accuracy')    #evaluation metric
 ) 

13. Fit the model on training data

model %>% fit(training_imgs, training_lbls, epochs = 10, verbose = 2)

Output:

Accuracy and training loss graphs:

14. Model evaluation on test set images

eval <- model %>% evaluate(testing_imgs, testing_lbls, verbose = 0)

Print test loss

cat('Test loss:', eval[“loss”], "\n")

Output: Test loss: 0.3401631

Print test accuracy

cat('Test accuracy:', eval[“accuracy”], "\n")

Output: Test accuracy: 0.8849 

15. Make predictions on test set images.

predicted_class <- model %>% predict_classes(testing_imgs)

Print predicted classes for first 10 images in the test set.

predicted_class[1:10]

Output: [1] 9 2 1 1 6 1 4 6 5 7

16. Plot several images with predicted labels.

 #Create 4*4 matrix and fill it column-wise
 par(mfcol=c(4,4))
 #Set margins
 par(mar=c(0, 0, 1.5, 0), xaxs='i', yaxs='i')
 for (i in 1:16) { 
   img <- testing_imgs[i, , ]
   img <- t(apply(img, 2, rev)) 
 #subtract 1 from predicted label as predicted value will be in range 1-10 while actual as labels’ #range is 0- 9
   pred_label <- which.max(pred[i, ]) - 1
   actual_label <- testing_lbls[i]
 #If prediction is correct, display the labels in green color
   if (pred_label == actual_label) {
     color <- '#008800' 
 #Display labels in red for false prediction
   } else {
     color <- '#bb0000'
   }
 #Create the grid of images
   image(1:28, 1:28, img, col = gray((0:255)/255), xaxt = 'n', yaxt = 'n',
         main = paste0(output_categories[pred_label + 1], " (",
                       output_categories[actual_label + 1], ")"),
         col.main = color)
 } 

Output:

The above output plot shows that the model has misclassified 4th image in the 1st row.

17. Predict output label for a single image

Extract an image from the test set first. Maintain the dimension of the batch of images as it is needed for the model.

image <- testing_imgs[3, , , drop = FALSE]

Check dimension of the extracted image

dim(image)

Output: [1]  1 28 28

Make a prediction for the image

pred <- model %>% predict(image)

Display confidence score for each of the classes

pred

Output:

Subtract 1 from the predicted labels as actual labels start from 0

pred <- pred[1, ] - 1

Get the predicted class having maximum confidence score

which.max(pred)

Output: [1] 2

• Code source: Official RStudio tutorial
• .R file of the above implementation is available here