How Does The Energy Sector Handle Data at Scale

The use of AI in the energy sector is being used through the entire process. Data driven solutions are already being used in seismic processing, predictive maintenance, operational optimization, high-frequency trading, reservoir simulation, and computational chemistry just to name a few. 

Chiranjib Sur who is the Head of Engineering and Scientific Software at Shell explains just how they are doing this at scale. He has been working with the company for over eleven years as a computational and data scientist. With a PhD in Physics, he brings a profound understanding in multidisciplinary fields. He spoke about what and who is responsible for scaling the software in the energy sector. 

Data driven decision making in the energy sector

“What needs to be done isn’t really rocket science.” explains Chiranjib Sur. “We apply some heuristic principles, some rules and human intelligence. Even the actual rocket scientists does the same,” he jokes. 

To gain insight first it is important to set the problem statement. “The problem with data is that we can represent the same data in multiple ways, both the insights might be accurate and yet very different.”

He then asked,“The next question is, If we want to make the right decision for the next business model, based on some data, how do we do that?” In the context of Shell, Sur gave the example of trying to figure out how many barrels of oil will be produced in the near future based on the history of oil production of the specific region. 

“Now there are two parts  - data and science,” he says. With the energy sector the ‘science’ part of it requires interdisciplinary expertise. “To find out where the oil pockets are, we need to understand geology,” he said. 

This is the first step. Understanding that data doesn’t work independently but in conjunction with larger context. “Oil extraction isn’t an easy process as they stick to rocks, it is imperative that our probabilities align to give us the maximum accuracy,” he explains. 

From millions of parameters in the database a few hundred observations are made leading to the final decision that leads to a prediction that the net amount of oil from this well would be in a range. 

Without getting into very technical details he tried to convey that different kinds of data driven techniques start with a stochastic optimisation. Solving this, leads to structured data from which you can make observations by predictive modeling.  

How this becomes an engineering problem is not to do with the data itself but how the data is hosted and communicated. When we deal with data that's as large as 100 terabytes, we have to figure out a way that data scientists and AI engineers have to work with that finally has to benefit the business model. 

“With all this accuracy is of utmost importance. We cannot afford to say that this message is up for predictive maintenance. We need to be deterministic. And to be deterministic, we need to be right,” he explains. 

To do this at scale

In the case of Shell, we collaborate and work with experts in each field. “This is an interdisciplinary field and we get different teams to collaborate,” Sur explains. Recently it was announced that Shell is collaborating with C3 AI. Scale is a critical challenge in the oil and gas industry, and this collaboration is addressing it effectively is a common objective. 

“AI contributes to the evolution of predictive analytics, enabling the system to teach and learn from itself, ultimately resulting in better and more accurate predictions.” Sur concluded.