My first Web Scraping project – Analyzing Flipkart Product Reviews using Text Mining

My first Web Scraping project - Analyzing Flipkart Product Reviews using Text Mining - Scrape and Analyze customer reviews
Flipkart review scraping

E-commerce websites generate large amounts of textual data. These firms hire data science professionals to refine this unstructured data and gather meaningful insights from it that can help in understanding the end-user in a better way. For example, by analyzing product reviews, Flipkart can understand the insights of the product, Netflix can find users’ likeness on their content and we can’t imagine doing this analysis will happen without Text analytics.

Topics we cover in this article:

  • How to Extract Product reviews from Flipkart website
  • Preprocessing of the Extracted reviews
  • Extracting and Analyzing Positive reviews 
  • Extracting and Negative reviews 

In this article, we will extract the reviews of Macbook air laptop from the Flipkart website and perform text analysis.

Hands-on implementation of Flipkart review scarping

#Importing required libraries
import requests   
from bs4 import BeautifulSoup as bs 
import re 
import nltk
import matplotlib.pyplot as plt
from wordcloud import WordCloud
import os

Extracting reviews from Flipkart for MacBook Air

Here we are going to extract reviews of Macbook air laptops from the URL.

Subscribe to our Newsletter

Join our editors every weekday evening as they steer you through the most significant news of the day, introduce you to fresh perspectives, and provide unexpected moments of joy
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

#Scraping review using beautifulsoup
for i in range(1,30):
  response = requests.get(url)
  soup = bs(response.content,"html.parser")# creating soup object to iterate over the extracted content 
  reviews = soup.findAll("div",attrs={"class","qwjRop"})# Extracting the content under specific tags  
  for i in range(len(reviews)):
#here we saving the extracted data 
with open("macbook.txt","w",encoding='utf8') as output:

Until here we extracted reviews from the website and stored them in a file named macbook_reviews.


The extracted product reviews include unwanted characters like spaces, capital letters, symbols, smiley emojis.  We don’t want to include those unwanted characters in text analysis, so in preprocessing we need to clean the data by removing unwanted characters.


# Joining all the reviews into single paragraph 
mac_rev_string = " ".join(macbook_reviews) 

# Removing unwanted symbols incase if exists
mac_rev_string = re.sub("[^A-Za-z" "]+"," ",mac_rev_string).lower()
mac_rev_string = re.sub("[0-9" "]+"," ",mac_rev_string)   

#here we are splitting the words as individual string
mac_reviews_words = mac_rev_string.split(" ")

#removing the stop words
#stop_words = stopwords('english')

In the below code snippet, we will gather the words from the reviews and display it using the word cloud

with open("/content/stop.txt","r") as sw:
    stopwords =
temp = ["this","is","awsome","Data","Science"]
[i for i in temp if i not in "is"]
mac_reviews_words = [w for w in mac_reviews_words if not w in stopwords]
mac_rev_string = " ".join(mac_reviews_words)
#creating word cloud for all words
wordcloud_mac = WordCloud(

From word cloud output the words like good, read, the laptop appears in the bigger size that illustrates these words are repeated more times in the MacBook air reviews. By observing this word cloud, we can see the highlighted words like performance, battery, delivery, the laptop, we can’t conclude how the battery works and, how it performs, to get insights from this output we need to divide this into a positive and negative word cloud.

In the below code snippet we will extract Positive words from product reviews

with open("/content/positive-words.txt","r") as pos:
  poswords ="\n")  
  poswords = poswords[36:]

mac_pos_in_pos = " ".join ([w for w in mac_reviews_words if w in poswords])
wordcloud_pos_in_pos = WordCloud(

#here we get wordcloud of all postive words in reviews
Flipkart review scraping 

Here, through this positive word cloud, we can get some insights like the product was good, smooth, fast, awesome product, recommend to others, portable to use, beautiful product, these are the positive insights from the MacBook air product.

In the below code snippet we will extract Negative words from product reviews 

with open("/content/negative-words.txt","r",encoding = "ISO-8859-1") as neg:
  negwords ="\n")
  negwords = negwords[37:]

# negative word cloud
# Choosing the only words which are present in negwords
mac_neg_in_neg = " ".join ([w for w in mac_reviews_words if w in negwords])

wordcloud_neg_in_neg = WordCloud(

#here we are getting the most repeated negative Wordcloud
Flipkart review scraping 

Now through this negative word cloud, we can illustrate that the product was lag, slow, crashed, we have issues in the product, it was so expensive, pathetic.


By analyzing the product reviews using text mining we gathered most appeared positive and negative words using the word clouds. We can conclude that text mining gains insights into customer sentiment and can help companies in addressing the problems. This technique provides an opportunity to improve the overall customer experience which returns huge profits.

Prudhvi varma
AI enthusiast, Currently working with Analytics India Magazine. I have experience of working with Machine learning, Deep learning real-time problems, Neural networks, structuring and machine learning projects. I am a Computer Vision researcher and I am Interested in solving real-time computer vision problems.

Download our Mobile App


AI Hackathons, Coding & Learning

Host Hackathons & Recruit Great Data Talent!

AIM Research

Pioneering advanced AI market research

Request Customised Insights & Surveys for the AI Industry

The Gold Standard for Recognizing Excellence in Data Science and Tech Workplaces

With Best Firm Certification, you can effortlessly delve into the minds of your employees, unveil invaluable perspectives, and gain distinguished acclaim for fostering an exceptional company culture.

AIM Leaders Council

World’s Biggest Community Exclusively For Senior Executives In Data Science And Analytics.

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox

Subscribe to Our Newsletter

The Belamy, our weekly Newsletter is a rage. Just enter your email below.