How The Indian Government Is Using Machine Learning And Big Data For Better Metrics

[caption id="attachment_17007" align="alignnone" width="1600"]

Reuters/Anindito Mukherjee[/caption] For a country as big as India, being able to get the data right, is a herculean task. At least before data analytics can come into picture and made things accurate not just by intuition but with numbers to back it up. India has already set foot on the path to building smart cities. According to the mid-year economic survey released last week, so far as the priority sector interventions are concerned, 22 of the 60 cities have already initiated the smart roads and 18 cities have initiated integrated command and control projects. Additionally, 20 cities have initiated smart water projects and 26 cities have started implementing the solar rooftop projects. Architectural, place-making and city beautification projects have been initiated in 18 cities.

Government turns to Geospatial Analytics

And with that, the government also seems to be committed to use a more complex but accurate form of decision making using machine learning. “With recent advances in remote sensing technology and machine learning for processing satellite images, we can get much more granular data on how urbanisation is happening across India,” said the mid-year review survey. The second volume of the Economic Survey released by the Union finance ministry shows that the use of geospatial analytics has begun to make its overdue presence felt in India as well. “Based on publically available data from the Global Human Settlement Layer (GHSL), we look at how built-up areas show the evolution of human settlements across India since 1975. It is also possible to disaggregate official census population numbers according to the density and form of these settlements to get granular population figures across the country.” The two metrics based on which India defines its urbanisation are: i) Administrative metric, ii) Census metric. But economists are now beginning to use machine learning—or the use of computer algorithms that learn from data—to extract information from satellite images.

Big Data, Big Information

This may be the first time that the Indian government is using geospatial analytics for deeper understanding of urbanization, but the use of big data is not new. Earlier this year, economists used monthly data of unreserved railway passengers over a period of five years to access the internal migration in the country. Surprisingly, the extent of migration in the analysis was far higher than what the census suggested. The big data metrics also allowed the economists to get a sense of labour flows throughout the country. In another example, the government also used big data analytics to get an estimate of trade in the country. The economists used Central sales tax invoices for trade between two states to estimate the extent to which states trade with each other. Moreover, the Indian government is also planning to use big data and analytics information available on corporate houses and individuals for income tax assessments. Several reports reveal that the government will start collecting statistics from usual sources like banks as well as online transactions and social media sites to match the spending and lifestyle patterns of a citizen with income declarations. The Department of Science and Technology (DST), under the Ministry of Science and Technology and Earth Sciences has been tasked to develop Big Data Analytics (BDA) ecosystem. National Data Sharing and Accessibility Policy (NDSAP) 2012 of DST is designed to promote data sharing and enable access to government owned data. Big Data Analytics infrastructure development in India is being steered by the C-DAC (Centre for Development of Advanced Computing), Ministry of Electronics and Information Technology. C-DAC regularly conducts training on "Hadoop for Big Data Analytics" and "Analytics using Apache Spark" for various agencies including Defence. With all the above stated examples and initiatives of the government, it is evident that big data, machine learning is the key to accuracy and efficient data collection. However, one will need to wait and watch as to how long it takes for the government to fully digitise India and take data collection to the next level.