Hands-On Tutorial On Principal Component Analysis In Python

Key takeaways

Real-world data complexity requires identifying strong independent variables for accurate predictions.
Dimensionality reduction techniques help simplify datasets by focusing on the most effective independent variables.
Principal Component Analysis (PCA) is a key method for reducing the number of independent variables in a problem.
Data must be properly scaled before analysis to avoid inaccuracies in predictions, as larger values can skew results.

Real-world data may not be as simple as predicting a person's salary against his or her experience. While this is just one example, there can be so many factors that may affect an employee’s salary. Real-world data is much more complex and therefore identifying and predicting a dependent factor against so many independent factors can reduce the probability of getting a correct prediction. That is why it is important to identify strong independent variables. Dimensionality Reduction is a technique that allows us to understand the independent variables and their variance thus helping to identify a minimum number of independent variables that has the highest variance with respect to the dependent variables. In simple terms, the dimensionality reduction technique helps us reduce the number to independent variables in a problem by identifying new and most effective ones

Implementing Principal Component Analysis In Python

In this simple tutorial, we will learn how to implement a dimensionality reduction technique called Principal Component Analysis (PCA) that helps to reduce the number to independent variables in a problem by identifying Principle Components.We will take a step by step approach to PCA.

Scaling The Data

Before jumping in to identify the strongest factors in your dataset, let it be any, we must make sure that all the data are in the same scale. If the data is not properly scaled it will lead to a false and inaccurate prediction as larger values will show larger effect.

from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform(X_test)

We have used the StandardScaler class of sklearn.preprocessing package for scaling the dataset.

X_train : training set containing only the independent features
X_test : test set containing only the independent features

Applying PCA to understand the Independent Factors

After the data is properly scaled We can apply Dimensionality Reduction to identify a set of new strong features or Principal Components

from sklearn.decomposition import PCA
pca = PCA(n_components =  None)
pca.fit(X_train)
variance = pca.explained_variance_ratio_

n_components: number of principal components to identify

Here we have used the sklearn. decomposition library to import the PCA class. We then initialised the PCA class with n_components = None as we have no prior knowledge about the variance of factors. The PCA object is then fitted with with the independent variable set to calculates the variance. The explained_variance_ratio_ of the PCA object returns a numpy array containing variances of Principal Components sorted in descending order. (The number of principal Components will be same as the number of factors in X_train). The higher value in the numpy array denotes high variance. From the obtained variances, choose the minimum number of principal components with the highest variances.

Applying PCA And Transforming The Datasets

After obtaining the minimum number of features(Principal Components with high variance), reinitialise the PCA with n_components as the number of Principal Components.Transform the training set and test set

pca = PCA(n_components = number of Principal Components )
X_train = pca.fit_transform(X_train)
X_test = pca.transform(X_test)
explained_variance = pca.explained_variance_ratio_

The above code will transform the X_train and X_test to training sets containing only the specified number of principal components.

Using Principal Components For Prediction

X_train and X_test can now be fitted to any predictive model depending on the nature of the problem Example:

#Fitting Logistic Regression to the Training set
classifier = LogisticRegression()
classifier.fit(X_train, y_train)
#Predicting the Test set results
y_pred = classifier.predict(X_test)

y_train: training set containing only the dependent factor

Important things to note:

PCA will take all the original training set variables and decompose them in a manner to make a new set of variables with high explained variance.
Principal component analysis involves extracting linear composites of observed variables.
PCA can be used to determine what amount of variability the independent variables can explain for the dependent variable and cannot be used to see whIch independent variables are more important for prediction.