I suggest we solve this problem by using the big data that none of us bothered to collect. QUIRKY JOKE

Click image to enlarge

IOT - cornerstones

internet transit prices - DrPeering.net





What is IoT?

IoT is the practice of capturing, analyzing, and acting on data generated by networked objects and machines. The Internet, which was introduced to connect people, is now connecting objects to people and is constantly collecting, analyzing, and communicating data. It is becoming a sort of sixth sense for humans. The key drivers in adaptation of IoT are sensors, networks, storage, and big data analytics.


The Institute of Electrical and Electronics Engineers provides a formal definition for sensors:

“An electronic device that produces electrical, optical, or digital data derived from a physical condition or event. Data produced from sensors is then electronically transformed, by another device, into information (output) that is useful in decision making done by ‘intelligent’ devices or individuals.”

The technological complement to a sensor is an actuator, a device that converts an electrical signal into action, often by converting the signal to nonelectrical energy, such as motion. A simple example of an actuator is an electric motor that converts electrical energy into mechanical energy. Sensors and actuators belong to the broader category of transducers. A sensor converts energy of different forms into electrical energy; a transducer is a device that converts one form of energy into another. For example, a loudspeaker is a transducer because it converts an electrical signal into a magnetic field and, subsequently, into acoustic waves. Sensors are becoming smarter and smaller every day. The price of sensors has drastically fallen in the past 10 years. These factors are driving adaptation of IoT in the industry.


The information that sensors collect has to be communicated and collected at some location. This involves transmission of data via networks. The first step in the process of transferring data from one machine to another via a network is to uniquely identify each of the machines. Network protocols are a set of rules that define how computers identify each other. Open protocols such as Internet Protocol allows for interpretability and scalability in the system.

When data have to be transferred over short distances, devices can use wireless Personal Area Network technologies such as Bluetooth and ZigBee. For bigger spaces such as buildings Local Area Networks are instituted. A Wide Area Network is used when data are communicated across cities. Data transfer rates and energy requirements are two key considerations when selecting a network technology for a given application. Technologies such as 4G (LTE, LTE-A) and 5G are favorable for IoT applications, given their high data transfer rates. Technologies such as Bluetooth Low Energy and Low Power Wi-Fi are well suited for energy-constrained devices.

The data transfer rates have increased in the past 30 years from 2 kbps to 1 gbps, enabling heavy data transfer quickly and seamlessly. Moreover, Internet transit prices have also come down. According to Holdowsky et al. (2015), “In 2003, it cost USD 120 to transfer 1 Mbps in the United States; as of 2015, the cost has come down to 63 cents.”


Sensors generate huge amounts of data every second that are communicated through networks and stored. In IoT, data are handled in two ways: real-time frontend data, which interacts directly with IoT-connected devices, and offline backend data which are stored for in-depth analysis. These storage methods are mostly completed on the cloud or in big storage facilities. Due to progress in technology, the price for data storage has fallen drastically in the past 10 years. However, the protection of the collected data is a big challenge and has become a major expense.

Big Data Analytics

The collected data have to be analyzed to give some insight into the working of “things.” Unfortunately in the oil and gas industry most of the companies are still using MS Excel as a main analysis tool. Big-data tools developed in recent years can handle a much higher volume of data, such as in petabytes (1 petabyte = 1,000 terabytes). Big-data tools are also considered to be better suited to handle the variety of incoming data, both structured and unstructured. There are three ways in which data analytics work:

i) Descriptive analysis, which describes what has happened

ii) Predictive analysis, which answers what could happen

iii) Prescriptive analysis, which answers what should happen

To complete these type of analyses on the amount of data expected to be available in the oil and gas industry, tools have to be developed to sort, sift, and process the incoming data into something that is usable. In 2004, Google introduced an open-source tool called MapReduce, a framework for processing large datasets. MapReduce allows a user to specify a “map” function that processes a key/value pair out of a data set. The “reduce” function then filters and merges all of the data associated with the generated key/value in the map step. This system allows for the automated processing of data coming into a system from hundreds or even thousands of input devices to be quickly broken down into key data streams that can then be sent to an end user or software for analysis.

The maturity of the development of a company’s analytical tools will determine what types of data analytics they will be utilizing. Early phase data gathering usually leads to descriptive analysis. This type of analysis is typically used for determining root causes of a given occurrence. As the IoT footprint grows along with the availability of more sophisticated analytical tools, our industry will begin to push toward prescriptive analysis. The turn toward prescriptive analysis techniques will prove to be a turning point in the planning, efficiency, and profitability across all sectors of the industry.

IoT in the Oil and Gas Sector

In the current industry environment, oil and gas companies have to start treating data just as they treat hydrocarbon. Data need to be generated, communicated, stored, and processed. Companies can reap considerable value by developing an integrated IoT strategy with an aim to transform the business.

Applications of IoT in the oil and gas sector are endless. The upstream industry loses billions of dollars every year due to nonproductive time (NPT). IoT could be used to help reduce NPT events by using real-time data to predict breakdowns and schedule preventative maintenance. Huge amounts of reservoir data can be integrated with real-time field data to plan well placement and flow rates. Accidents can be prevented and processes can be optimized.

The midstream industry has the challenging task of transporting variable volumes and grades of products from multiple locations to new end-users and markets. Connecting pipeline networks, sensors, leak detection, alarms, and emergency shutdowns to interact seamlessly and to be available for analysis and interpretation in real time would significantly reduce some of the major risks that this sector of the industry deals with.

The downstream industry is commercially the most challenging branch of the business. Refinery shutdowns, handling various grades of crude oil, and changing environmental regulations are pushing gross refining margins down to a bare minimum. With the use of IoT, refineries can plan their shutdowns, minimize their downtime, and improve safety records.

IoT is the next step in the evolution of the oil and gas industry. Data will have to flow in real time and seamlessly from fields to boardrooms. This will enable better decision making, better optimization of process, and a safer working environment. In an industry that spends trillions of dollars every year on exploration, development, and maintenance of assets, even a single-digit percentage improvement through IoT will have a huge impact.