How AI and Machine Learning are Reshaping the Future of Inline Inspection

As artificial intelligence (AI) and machine learning (ML) rapidly transform the energy sector, their potential to augment pipeline integrity management and inline inspection (ILI) becomes more pronounced.

In an industry where precision and safety are paramount, these technologies can be leveraged to uncover hidden trends, manage vast data volumes, and enhance performance through robotics and advanced analytics. It’s safe to say that AI and ML are here to stay. In this article, we’ll explore how AI and ML transform ILI and the industry’s approach to pipeline integrity management. We’ll also delve into how NDT Global leverages them to enhance our inline inspection services and how we apply best practices to mitigate the potential risks associated with these transformative technologies.

What Are AI, Machine Learning, and Deep Learning?

At its most basic level, AI uses computer science and data to enable problem-solving in machines. By mimicking human cognitive functions associated with human intelligence, such as decision-making, pattern recognition, and language learning, AI systems can tackle complex challenges at scale and with accuracy. AI signals a paradigm shift in computing with vast potential in pipeline integrity management. Some of the ways AI can optimize efficiency and accuracy include:

Pipeline integrity management: AI can be leveraged to identify and predict defects, optimize maintenance schedules, and enhance safety protocols.
Predictive maintenance: AI can use sensor data and historical patterns to predict equipment failures before they occur.
Robotics and automation: AI can drive autonomous inspections and repair in challenging, difficult-to-access pipeline environments.

Machine learning (ML), a subset of AI, refers to computer systems that learn and adapt automatically from experience. ML specializes in specific tasks using advanced algorithms and learns to predict outcomes, detect anomalies, and make data-driven decisions. ML is also narrower and more tailored to a specific use case or subject than AI. Some common examples include:

Spam or not spam detection
Speech-to-text applications
Fault detection for websites and e-commerce
Credit and banking application checks

So, in short, ML is AI that can automatically adapt with minimal human intervention and interface. ML also enables a system to learn and improve from the data it ingests. Deep Learning is a subset of machine learning that layers algorithms and computing units—or neurons—into an artificial neural network. These neural networks were originally inspired by the structure of the human brain. Data passes through these connections in a non-linear fashion, similar to how the human brain processes information. While machine learning algorithms can learn from relatively small data sets, Deep Learning algorithms usually require large data sets that might include diverse and unstructured data.

Let’s illustrate the use cases and capabilities of ML and Deep Learning by comparing them to the traditional problem-solving approach for detecting and diagnosing metal loss within a pipeline using human analysis. In traditional problem-solving analysis, knowledge-based inputs are encoded to create a set of rules. As all the information comes from humans, the amount of data the experts encode is limited to their knowledge, observations and the ability to translate their innate knowledge into a set of representative numbers or rules. The diagram below depicts a standard case of ultrasonic metal loss inspection. The red channels at the top of the diagram indicate a reduction in wall thickness. By comparing this data with human-defined rules, analysts will conclude that internal metal loss is indeed present in this particular section of the pipeline because, at certain points, they observe:

A wall thickness of less than 5mm over a predefined area
A stand-off increase at the same location
A continuous sloping contour
A well-defined pattern in well-defined patterns in single signals

So, here, humans precisely define the rules which are applied to arrive at the conclusion that internal metal loss is present.

Now, let’s consider how the same problem might be solved using machine learning.

First, the model learns from human-crafted descriptors (or features in the ML world) that are representative numbers from actual data. These may include minimal wall thickness readings, histograms showing the stand-off feature distribution, and the number of pixels grouped together that indicate a reduction in wall thickness. Over time, the model will learn which descriptors are more important than others and generate rules, arriving at decisions or conclusions based on those. So, over time, the model will derive the same conclusion that metal loss is present in the pipeline at this location.

AI with Machine Learning

Step 1: Train a model. Models learn from human-crafted data and generates automatic rules.

When applying Deep Learning to this scenario, the model will start learning automatically directly from the rules and data—without any human intervention—to arrive at the conclusion that metal loss is present in the pipeline. But to accomplish this successfully, it requires a large amount of data.

Step 2: Apply the trained model and the learned rules

Considerations for Ensuring Accuracy and Ethics

While ML and Deep Learning are increasingly used to perform in-depth analysis and make complex decisions, we should be mindful that their algorithms are not always 100% accurate or entirely ethical. It’s important to remember that these models rely on probabilities, so their performance will depend on the amount and quality of the data, the methodology used, and any way humans may unintentionally limit them through their choice of data sets and descriptors. For example, if a particular aspect or attribute is vital for accurate decision-making but there’s no descriptor tied to or reflecting it, the model will never function optimally. There are two primary error sources for machine learning models: concept drift and covariate drift. Let’s explore each in more detail:

Concept Drift

Concept drift can occur if an environment changes. For example, in the realm of email spam detection, what spam emails looked like ten years ago vastly differs from what they do today. They were easy to spot in the past due to their poor use of language, grammar, and design elements and lack of personalization. Today, cybercriminals have honed their craft and distributed more sophisticated emails that are easy for users to mistake for genuine communications from trusted sources. Imagine you have a machine learning algorithm that’s trained using spam mail from a decade ago. It will never be able to detect modern email scams as it has never been trained using senders’ latest tools and tactics. So, when an environment changes and models don’t receive the appropriate information about those changes, the risk of concept drift is introduced.

Covariate Drift

Covariate drift can occur if data changes. To illustrate the effects of covariate drift, let’s consider medical science. If physicians have an algorithm that’s been trained to assess the health of the hearts of patients under the age of 40, it will struggle to accurately assess the heart health of patients in their 60s or 70s because the data gathered from older patients will be vastly different to what it was trained on. Finally, we should remember that ML and Deep Learning techniques and the underlying systems in which they operate are extremely complex. It’s not always easy to understand how these technologies work and precisely why they’re arriving at their decisions.

How is NDT Global Currently Using AI Safely in Inline Inspection?

NDT Global is committed to developing safe and effective AI that enhances the performance of its core technologies. Our team comprises a diverse group of experts, ranging from experienced human analysts to data scientists. This mix ensures a diverse set of algorithms that prevents a narrow or limited scope of capabilities. The team is also looking for more than speed; our focus is on the quality of the results and the value AI can add to various areas of ILI. For example, AI can be used in warehouse situations to identify and prepare the right tools and parts. It can also be used in quality control to ensure that tools are set up correctly before they go into the pipeline. The most promising applications of AI lie in complex data analysis and integrity assessments. With the proper design and training, AI promises significant enhancements in quality and speed.

AI Meets ILI

NDT Global currently focuses AI activities on the data analysis workflows. We work on applications of AI in our data analysis chain, from data preparation and cleaning, to pipe tally generation, feature detection and classification, and quality control and reporting.

Spotlight On AI for ILI Data Analysis

Let’s take a moment to explore how AI-enabled data analysis can assist in delivering high quality results.

Some of our clients’ pipelines run for hundreds of kilometers, and our teams are tasked with searching for small defects or features—sometimes less than a centimeter in length or diameter. And while some sections of longer pipelines are riddled with anomalies, long stretches of pipeline often have no relevant features whatsoever.

Ten years ago, for some technologies we had to take a manual, entirely human-based approach to checking the integrity of every centimeter of a pipeline. For other technologies millions of false calls had to be checked. Measurement data was converted into hundreds or thousands of screen images, which an analyst would have to review to detect the presence of features. Then, we’d need to determine which features were relevant and which were not.

If a feature was deemed relevant, the analyst would then have to classify it, e.g., as metal loss or a crack, and record its depth and position relative to the inside of the pipeline wall and any welds.

Today, we’re successfully using AI to reduce the burden on analysts in all these areas, such as:

Finding the features in the pipeline: As this task involves straightforward image recognition, algorithms are trained to detect every anomaly in a pipeline run without the need for any human intervention.
Determining which features are relevant: Machine learning models are developed to remove obviously irrelevant features. Human analysts only need to concern themselves with the remaining relevant features.
Feature analysis: Models perform an initial analysis, including the feature’s basic parameters, which are then checked by an analyst.

As you can see, our approach centers on getting people and machine learning models working cooperatively side-by-side. This collaborative approach has advantages for NDT Global and our clients, including greater consistency, focus, and speed while preserving the high level of safety.

Consistency: Using these algorithms, NDT Global is shifting decision-making from subjective to objective and 100% reproducible.

Focus: Scrolling through images of hundreds of meters of pipeline daily is a demanding and mentally draining task. Now, our team of human analysts can focus on what matters most. By eliminating the need to focus on obviously irrelevant features and instead dedicate time and energy to more relevant, complex ones, the risk of human error by fatigue is reduced.

Speed: Because analysts are focusing only on critical features and don’t need to spend hours trawling through images of long sections of healthy pipe, conclusions and critical recommendations are delivered faster.

Introducing AI into Your Pipeline System – Steps for Building Trust

Trusting AI in a pipeline system requires using a well-planned program to adapt AI to your specific needs. Importantly, it includes constant oversight at the human level.

NDT Global is proceeding with such a program with the same structured and proven approach we’ve used to develop the best technology and train the best people.

Our approach includes:

Problem Definition – Each program starts by defining the problem(s) which have to be solved, gathering requirements, and setting metrics as targets.
Data Acquisition and Preparation – This includes data collection, cleaning, preparation, and exploration.
Model Development – Next comes the iterative process of developing models until we target metrics are achieved.
Model Evaluation – As the model coalesces, the developers start testing it on unseen data and analyzing the quality and accuracy of its predictions.
Deployment – This involves introducing the model into the live production environment.
Monitoring and Maintenance – During deployment the model’s performance is tracked based on new data (vs. historical data) and refined using user feedback or the outputs of the new data.

NDT Global’s Best Practices for Working with AI in Inline Inspections

Here are some recommended best practices for bringing the power of AI to bear on ILI projects based on our extensive experience in this area:

Use Recent Inspection Technologies

Avoid using data gathered from inspections performed by previous ILI tools; they may behave very differently to your current technologies, and may confuse the model.

Work with a Diverse Group of Stakeholders, Pipelines, Technologies, and Data Quality

Collaborate closely with a diverse group of stakeholders and clients involved in ILI projects to ensure that:

Runs are involved with deep metal loss and other unique sets of features
Data sets cover different pipeline conditions like geographical locations, pipelines of varying age and pipeline status (many features per kilometer vs. relatively few), different medium types, wall thickness, welding methodologies
All available technologies and tool designs which are deployed are represented
Specific data reporting requests or requirements from the client are avoided

Wherever Possible, Use Analyst-Labeled Data

In machine learning, data labeling is the process of identifying raw data and adding one or more meaningful and informative labels to provide context so that the model can learn from it.

Properly labeled data provides the “ground truth” (i.e., how labels reflect “real world” scenarios) for testing and iterating subsequent models. Data labeling will help ensure the data you use is unbiased and representative, improving the usability of data variables within a model. The scope of most ML models is to assist analysis work, so using analyst-labeled data is the key.

Evaluate Your Model on Unseen Inspections

Your selected inspections should always follow data collection guidelines.
Pipelines should differ from those used during the training phase.
Be sure to include as many inspections as possible that involve important or unique features.

Calculate Metrics Separately for Every Inspection and Pay Attention to Important Features

All inspections should achieve your target metrics, but you’ll also need to develop specific, stricter metrics for pipelines with unique features. If you see misdetections, look for patterns. If there are no clear patterns, analyze the model’s predictions with ML-specific tools to understand why it made those predictions.

Navigate the Future of Inline Inspection with Confidence

In the realm of pipeline integrity management, AI and ML are powerful enabling technologies that are here to stay. However, they require precise application and diligent oversight to deliver trustworthy results.

As AI and ML evolve, NDT Global is committed to enhancing all ILI processes with the responsible use of these cutting-edge technologies to ensure quality, safety, and unparalleled value for clients. By blending technological excellence with world-class people expertise, we’re pushing the boundaries of pipeline integrity management transformation.

Discover the Power of Clarity. Whether you’re ready to schedule an inspection or have questions about our process, our team can help get you started. Contact Us.