Now Reading
How Process Manufacturing Industry Can Utilize Big Data Analytics

How Process Manufacturing Industry Can Utilize Big Data Analytics

Any process manufacturing plant has sensor measurement data, operator log book data either in electronic form or in handwritten form, CCTV monitoring data, audio data either from CCTV or separate system, thermal imaging data for hot areas of the plant, weather information, logistics information etc.

   Table 1. Details of different data types stored in process manufacturing plants

Data available in process manufacturing plants have following characteristics in the context of big data definition

  • Variety      numeric, text, video and audio data types
  • Velocity    scan rate of Mili-second to minutes (multi scale data)
  • Volume    a refinery is designed for 2GB/day data logging capacity
  • Value        novel insights for operations, energy and asset management etc.
  • Veracity    data quality and metadata management processes

Can process manufacturing data be termed BIG DATA

It is evident that the process manufacturing plant data has all the characteristics for it to be termed big data. Despite fulfilling the big data definition criteria, process manufacturing data cannot be used for big data analysis because different data types are not available for combined analysis.

As described in Table 1. Sensor measurement data is available in different DCS systems, PLCs and data loggers in the plant; sensor data is not combined in real time hence rendering it unusable for combined analysis. CCTV data store is from different vendors and developed on different database technology than sensor measurement data and hence cannot be used for combined analysis.

Similarly, operations and maintenance logbooks contain text data that is either stored in Manufacturing Execution System (MES) database or in handwritten records; thermal imaging and audio data systems are from different vendors and developed on different database technology than sensor measurement data. Moreover, there is no technology platform available for combined analysis of different types of data available in process manufacturing plant.

How can process manufacturing data be transformed to BIG DATA

Data integration and combined data analysis platform are the enablers for big data analysis of process manufacturing data. New technologies are available for common storage and management of different data types. Unstructured databases with distributed file system architecture are now available which can handle variety of data types in scalable way. Apache Hadoop is one such open source unstructured database which can handle numeric, text, video and image data types.

Apache Hadoop is scalable because of its Distributed File System [DFS] architecture. In memory data stores such as GridGain and aggregate oriented databases such as mongoDB provide different choices for real time and unstructured data analysis. These new technologies enable combined data analysis at large scale in near real time scenarios. Parallel computing platforms such as Apache Hadoop’s MapReduce framework and many other open source and proprietary MapReduce implementations can carry out data mining computations for large amount of data in short period of time.

Fig. 1. Schematic of big data technology deployment for process manufacturing industry

Process manufacturing industry can benefit from new big data technologies to solve complex problems which have not been attempted so far because of data unavailability or high computational requirements.

Relevance of Big Data to Process Manufacturing Industry

More sources of data will now be available for analysis with the help of big data technologies. It will be possible to deploy a solution for combined analysis of sensor measurement data and other data types such as text, audio, thermal imaging etc.

Fig. 2. Potential benefits of new data

Fig. 2 describes that new data can impact process manufacturing plant operations in two ways – new questions that can be answered with the availability of new data & technologies and existing questions that can be better answered by including new data sources and analytics techniques. Five new questions and three current questions are listed as use cases in this section to intuit potential benefits of big data analytics technologies for process manufacturing plants.

See Also
After Face Swap, China’s Deepfake App Zao Sparks Security Concern

What new questions can be asked?

  1. What are the best parameter settings for operating process plant?
  2. When is the next process disturbance or asset abnormal operation could occur?
  3. What is the root cause of process disturbance or asset abnormal operation?
  4. What are the best corrective actions to restore the operation?
  5. What would be the quality of product being produced?

How existing questions can be answered better

  1. How to smoothly operate with least possible losses and inventory
  2. Real time integration of shop floor KPI to organization KPIs
  3. Traceability across the value chain

Role of Big Data Stakeholders

There are three stakeholders in realizing benefits of big data for process manufacturing plants. Refer Fig. 3.

  • Computing technology vendors
  • Automation technology vendors
  • Process manufacturing Industry

Fig. 3. Stakeholders for Big data technology deployment in process manufacturing plants

Contribution from each of the three stakeholders is critical for realizing benefits of big data technologies for process manufacturing plants. Rest of the section outlines potential contribution of the three stakeholders towards big data technology adoption in process manufacturing plants.

Computing technology vendor

Computing technology vendors have been traditionally supplying IT hardware and basic computing environment. In the Big Data deployment scenario, computing vendor could bring following components –

  • Unstructured and Distributed File System (DFS)database (e.g. Apache Hadoop)
  • Parallel computing environment (e.g. Apache Hadoop’s MapReduce)
  • Analytics modeling platform and computing libraries (e.g. Apache Mahout, IBM BIG data platform)

Automation technology vendor

Automation technology vendors have been traditionally supplying control and plant information management systems. In the big data deployment scenario, automation technology vendor could bring following components –

  • Domain, control system and industrial communications knowledge
  • Application of new computing technologies (DFS, MapReduce) to process manufacturing plant problems
  • Operational roll out, refinement and field demonstration of new solutions

Process manufacturing industry

Process manufacturing industry have been traditionally adopting technology solutions and partnering with automation companies for new technology development e.g. advanced control and optimization solutions. In the big data deployment scenario, process manufacturing industry could actively engage with all the stakeholders in developing ecosystem for manufacturing analytics solutions. Process manufacturing industry could provide –

  • Analytics ecosystem development and leadership
  • Actual process manufacturing plant data which is key element of the puzzle
  • Business critical problem statements i.e. information about what to look for in the data
  • Operational and maintenance nuances which are of critical important for realizing successful operational roll out


Process manufacturing plants have variety of data sources such as sensors, CCTV systems, thermal videography, operation and maintenance logbooks etc. These data sources generate different types of data such as numeric, video, text etc. Different data types are used for specific analysis e.g. sensor data for process control, CCTV for facility monitoring, thermal videography for operational and safety etc. Current automation technologies do not allow for combined analysis of different data types. Big data technologies could enable complex data analysis in near real time to unlock operational efficiencies for process manufacturing plants.

Provide your comments below


Copyright Analytics India Magazine Pvt Ltd

Scroll To Top