It is often said that data science is a multidisciplinary field. Not only does it require good coding skills and an analytical mindset, but also domain expertise, which is what sets it apart from data analytics. Since data science has become an integral part of the success of nearly every industry, data scientists are required to have an understanding of the particular sphere they are working in. For example, a data specialist in the financial sector must be aware of the terminology used in the world of finance and be up to date on the latest developments.
In this article, we’ll explore the world of the oil & gas industry. We’ll walk through a simple introduction to manufacturing, transporting, and selling oil. Then, we’ll move to the technical side of things and try to answer what data scientists do in this field and what skills they need to possess to get ahead. We'll also look at some successful applications of data science in the oil and gas industry. And finally, as usual, we’ll provide some pointers on the next steps to becoming an oil & gas data scientist.
Table of Contents
- Introduction to the Oil and Gas Industry
- Types of Data in the Oil and Gas Industry
- How to Become an Oil and Gas Data Scientist
- What Is the Role of an Oil and Gas Data Scientist?
- Applications of Data Science in Oil and Gas
- How to Become a Data Scientist in the Oil and Gas Industry: Next Steps
Introduction to the Oil and Gas Industry
As stated by IBIS World, the oil and gas industry has generated global revenue of $5 trillion in the years 2017-2022. And according to The Guardian, this industry has been generating $2.8 billion a day in the last 50 years. So in dollar value, it’s the biggest sector in the world. Given the high significance and sensitivity of this industry, you can easily guess that it’s pretty complex in terms of technical processes, investment, international agreements, and so on.
Major energy companies in the world drill to extract raw materials for manufacturing gas and crude oil. To understand what raw materials mean, let’s first understand how oil is actually formed. Plainly speaking, the remains of decaying (decomposing) animals and plants deposit and accumulate over time in limestone and sandstone deep in the oceans, with the right pressure and temperature they form what is known as “hydrocarbons” which are the raw material for making oil and gas. Hydrocarbons are simply organic compounds that consist basically of carbon and hydrogen atoms. Even though the process looks simple, it’s highly complicated, costly, and time-consuming. To simplify things, the oil & gas industry is divided into 3 stages:
- Exploration & Production (Upstream)
As the title suggests, energy companies explore locations around the world searching for raw materials. This stage takes a lot of time and resources because the search for reserves and wells is difficult and sometimes a company invests huge amounts of money and uses expensive and labor-intensive machines and still may not find what they’re looking for. Organizations approach this problem by making contracts with drilling contractors instead of using their own equipment.
- Transportation (Midstream)
In the second stage, the company transports the extracted materials to refineries and factories to start processing them.
- Converting & Selling (Downstream)
Finally, the refineries remove impurities and convert the extracted materials into oil-based products and derivatives and release them to the market.
Types of Data in the Oil and Gas Industry
As we mentioned before, data science has become an integral part of the success of nearly every industry and this applies especially to oil & gas. In this field, there is a growing demand for big data analytics to improve the techniques and processes involved in exploring and manufacturing oil.
In a 2020 research article by Mehdi Mohammadpoor and Farshid Torabi, the authors illustrate that with the advancement of technological methods in exploration and production, massive amounts of data are being generated every day. So the need for big data analytics in the oil and gas industry has increased enormously. Companies operating in the field collect data from two main types of sources - structured and unstructured.
Structured data sources:
- Risk and project management reports
- Surface and subsurface facilities
- Drilling data
- Production data
- Market prices
- Weather data
Unstructured data sources:
- Well logs
- Daily written reports of drilling
- CAD drawing
How to Become an Oil and Gas Data Scientist?
There’s a common stereotype that you should have some sort of engineering degree to become a data scientist in the field. While this is true for some positions where companies require a major in petroleum or production engineering, it's not always the case. Not all healthcare data scientists are originally medical doctors or pharmacists. The same applies here. Still, a data scientist should have domain expertise. So if you’re not an engineering graduate and want to be a data scientist in an engineering field, equip yourself with all the basic knowledge, terminology and skills to keep pace with the rapid growth of the industry.
If you’re already an engineer or have an engineering background, this will save you much time as you’ll only need to build data skills and perhaps freshen up your engineering knowledge to get ahead of competitors. To become a data scientist in the oil & gas industry, you need the following skills:
1. Domain knowledge
If you don’t have any previous experience in the oil & gas industry, the topics you need to be familiar are:
- Basic oil & gas operations and processes
- The oil & gas industry market
- Energy and oil terminology
- Drilling process, components, and site preparation
- Characteristics of oil & gas products
2. Programming language(s)
Just like in any other industry, you need at least one programming language to deal with the data at hand. And since most of the data in this field is big data, you may as well learn how to use SQL which allows you to manage, analyse and store massive datasets. Also, you need to master coding in Python, which is relatively easy to do. If you already know your way around Python and want to take your programming skills to the next level, you can start learning Java since most of the big data platforms are written in Java and it can be used in different data science techniques.
3. Statistics
Studying statistics is an important skill in almost every domain but for data science, statistics is a foundational building block. You don’t have to be a math guru but you need a basic understanding of the key concepts and methods used to transform, analyse, and leverage the power of data. These are the key concepts to start with:
- Descriptive statistics
As the name implies, this branch of statistics is used to describe the main characteristics of data. It includes the calculation of mean, median, and mode.
- Inferential statistics
A second branch of statistics that’s concerned with analyzing random samples to draw conclusions about a population. This branch is divided into hypothesis testing and regression analysis.
- Variability
Variability includes parameters like range, standard deviation, and variance.
- Correlation
Correlation is a simple method used to measure the relationship between two variables. There are 2 types of correlation:
a. Positive correlation, where a variable increases by the increase of the other variable. I.e. they move in the same direction.
b. Negative correlation, where a variable increases by the decrease of the other variable or vice versa. I.e. they move in the opposite direction.
4. Machine learning and deep learning
Machine learning has brought about unforeseen progress in many industries. This also applies to the oil and gas industry, it improves almost all types of operations and reduces costs. The following are ways machine learning and deep learning are changing the industry for the better:
- Rock facies classification by using XGBoost
- Detecting hidden patterns in unstructured data using deep learning
- Predict the best locations for drilling using reinforcement learning
- Estimate reservoir recovery with auto machine learning
- Predict the success of Enhanced Oil Recovery (EOR) process using Generative Adversarial Network (GAN)
- Fracture characterization
- Liquid loading detection
What Is the Role of an Oil and Gas Data Scientist?
Like any data scientist, you’re expected to have a strong analytical mindset and be able to deal with different types of data. In a typical position, you’ll be required to do the following:
- Run experiments and simulations
- Perform statistical analysis and pattern recognition
- Manage and analyse big data
- Contribute to training, building and deploying machine learning models
Applications of Data Science in Oil and Gas
Data science can improve each and every aspect of the oil and gas industry, from analysing seismic data to predicting well logs. Here are the most common applications of data science in the oil & gas industry:
- Rate of penetration estimation
- Improving the transportation process
- Pressure, volume, temperature (PVT) estimation
- Reducing drilling time
- Predict location of oil pockets
- Estimate best locations for drilling
- Predictive maintenance
How to Become a Data Scientist in the Oil and Gas Industry: Next Steps
If our introduction to the key applications of data science in oil and gas has got you excited, you can now start mastering the analytics skills you need to make a real impact on this crucial industry. From comprehensive introductions to Python and R to key considerations in Machine Learning, the 365 Data Science Program has everything you need to break into the field. Under the guidance of leading industry experts, you will learn by doing with a myriad of practical exercises and real-world business cases. If you want to see how the training works, start with a selection of free lessons by signing up below.
