Now Reading
Complete Guide To Handling Categorical Data Using Scikit-Learn

Complete Guide To Handling Categorical Data Using Scikit-Learn

W3Schools

Dealing with categorical features is a common thing to preprocess before building machine learning models. In real-life data science scenario, it means that the dataset has an attribute stored as text such as days of the week(Monday, Tuesday,.., Sunday), time, colour(Red, Blue, …), or place names, etc. 

Categorical features have a lot to say about the dataset thus it should be converted to numerical to make it into a machine-readable format. Focusing only on numerical variables in the dataset isn’t enough to get good accuracy. Often categorical variables prove to be the most important factor and thus identify them for further analysis. Most of the machine learning algorithms do not support categorical data, only a few as ‘CatBoost’ do. 

There are a variety of techniques to handle categorical data which I will be discussing in this article with their advantages and disadvantages. 



Identifying Categorical Variables (Types):

Two major types of categorical features are

  • Nominal – These are variables which are not related to each other in any order such as colour (black, blue, green).
  • Ordinal – These are variables where a certain order can be found between them such as student grades (A, B, C, D, Fail).

The dataset I’m going to work with is: Melbourne housing price dataset from Kaggle https://www.kaggle.com/dansbecker/melbourne-housing-snapshot/home?select=melb_data.csv

Let’s Explore the Dataset

import pandas as pd
df = pd.read_csv('/content/drive/My Drive/melb_data.csv')
df.head()
categorical data

The dataset contains 13580 rows and 21 columns.

Let’s get the categorical data out of training data and print the list. The object dtype indicates a column has text.

s = (df.dtypes == 'object')
object_cols = list(s[s].index)
print("Categorical variables:")
print(object_cols)

Categorical variables:['Suburb', 'Address', 'Type', 'Method', 'SellerG', 'Date', 'CouncilArea', 'Regionname']

For simplicity, I’ve taken up only 3 categorical columns to illustrate encoding techniques. 

features = df[['Type','Method','Regionname']]
features.head()
categorical data

Handling Categorical Variables

Replace using the map function

This could be a very basic approach to manually replace categorical values with custom values.

mapping = {'h':1,
           'u':2,
           't':3
          }
features['type'] = features.Type.map(mapping) 
features.type.value_counts()

OUTPUT

1    9449
2    3017
3    1114
Name: type, dtype: int64

This is not a good approach when it comes to large categories.

Label Encoding

Label encoding can uniquely number the different categories from 0 to n-1. Thus also termed as Integer encoding. LabelEncoder class from the scikit-learn library is used for this purpose. 

Before Label Encoding:

features.Regionname.value_counts()

Southern Metropolitan         4695
Northern Metropolitan         3890
Western Metropolitan          2948
Eastern Metropolitan          1471
South-Eastern Metropolitan     450
Eastern Victoria                53
Northern Victoria               41
Western Victoria                32
Name: Regionname, dtype: int64

After label Encoding:

from sklearn.preprocessing import LabelEncoder
le = LabelEncoder()
df1 = features[['Regionname']]
df1['Region'] = le.fit_transform(features['Regionname'])
df1.value_counts()
OUTPUT:
Regionname                  Region
Southern Metropolitan       5         4695
Northern Metropolitan       2         3890
Western Metropolitan        6         2948
Eastern Metropolitan        0         1471
South-Eastern Metropolitan  4          450
Eastern Victoria            1           53
Northern Victoria           3           41
Western Victoria            7           32
dtype: int64

fit_transform(y) – fits the label encoder and then returns encoded labels.

LabelEncoder cannot handle missing values so it’s important to impute them. LabelEncoder can be used to store values using less disk space. This is simple to use and works well on tree-based algorithms. It cannot work for linear models, SVMs, or neural networks as their data needs to be standardized.

One Hot Encoding

One hot encoding is a binary encoding applied to categorical values. To increase performance one can also first perform label encoding then those integer variables to binary values which will become the most desired form of machine-readable.

Pandas get_dummies() converts categorical variables into dummy/indicator variables.

See Also

Before OHE:

features.Method.value_counts()
S     9022
SP    1703
PI    1564
VB    1199
SA      92
Name: Method, dtype: int64

After OHE:

df2 = pd.get_dummies(features['Method'])
df2.value_counts()
OUTPUT:
PI  S  SA  SP  VB
0   1  0   0   0     9022
    0  0   1   0     1703
1   0  0   0   0     1564
0   0  0   0   1     1199
       1   0   0       92
dtype: int64

One hot encoding overcomes the limitations of label encoding and can be used in both tree-based and non-tree-based machine learning algorithms. The disadvantage is that for high cardinality, the feature space can really blow up quickly. The binary variables are often called “dummy variables” in statistics.

Label Binarizer

Scikit-learn also supports binary encoding by using the LabelBinarizer. We use a similar process as above to transform the data for the process of creating a pandas DataFrame.

from sklearn.preprocessing import LabelBinarizer
lb_style = LabelBinarizer()
lb_results = lb_style.fit_transform(features["Type"])
pd.DataFrame(lb_results, columns=lb_style.classes_).value_counts()
OUTPUT:
h  t  u
1  0  0    9449
0  0  1    3017
   1  0    1114
dtype: int64

Count/Frequency Encoder

Another way to refer to variables that have a multitude of categories is to call them variables with high cardinality. If we have categorical variables containing many multiple labels or high cardinality, then by using one-hot encoding, we will expand the feature space dramatically. Replacing categorical variables with their frequency this is the number of times each label appears in the dataset. 

Before encoding:

20 h 9449
21 u 1114
22 t 3017

After encoding:

df_frequency_map = features.Type.value_counts().to_dict()
features.Type = features.Type.map(df_frequency_map)
features.Type.iloc[20:23]
OUTPUT:
20    9449
21    1114
22    3017
Name: Type, dtype: int64

It is very simple to implement and does not increase the feature dimensional space. But if some of the labels have the same count, then they will be replaced with the same count and they will lose some valuable information. Adds somewhat arbitrary numbers, and therefore weights to the different labels, that may not be related to their predictive power.

Conclusion

There are many more ways by which categorical variables can be changed to numeric I’ve discussed some of the important and commonly used ones. Handling categorical variables is an important step for feature engineering. New variables can be formed by categorical variables and get more insight into the dataset. 

The complete code of the above implementation is available in the AIM’s GitHub repository. Please visit this link to find the notebook of the above codes.

What Do You Think?

If you loved this story, do join our Telegram Community.


Also, you can write for us and be one of the 500+ experts who have contributed stories at AIM. Share your nominations here.
What's Your Reaction?
Excited
0
Happy
0
In Love
0
Not Sure
0
Silly
0

Copyright Analytics India Magazine Pvt Ltd

Scroll To Top