Case Study • Surival Analysis • Predictive Maintenance • Deep Models • ML Models • Deep Learning • Machine Learning • Semi-Parametric Models >

Revolutionizing Predictive Maintenance - A Hybrid Deep Approach to Remaining Useful Life (RUL) Estimation

A hybrid deep learning approach for predictive maintenance, combining time-frequency feature extraction, advanced AI architectures, and innovative bearing analysis to optimize Remaining Useful Life (RUL) predictions, tackling data sparsity and complex degradation dynamics for industrial reliability.

George Deac

George Deac

— 5 min read
Revolutionizing Predictive Maintenance - A Hybrid Deep Approach to Remaining Useful Life (RUL) Estimation
3D Waterfall Diagram of the Wavelet Scalogram Energy Spectrum of a degradation trajectory

Overview

This study addresses the challenge of sparse data availability for predicting the Remaining Useful Life (RUL) of industrial bearings. By combining time-frequency feature extraction techniques with an advanced deep learning framework, our hybrid model offers enhanced accuracy and robustness.

This approach leverages a state of the art convolutional backbone models for deep ferature extraction, integrated with probabilistic regression methods and informed statistical degradation pattern assumptions (semi-parametric trained deep model), paving the way for more reliable predictive maintenance systems in industrial settings.

Moreover, our framework can constitute a Mixture of Experts (MoE) configuration allowing the model to better generalize on previously untrained new components under monitoring.

Venn diagram of a semi-parametric deep modelling approach of the degradation process

The Problem

Predictive maintenance relies heavily on accurate estimation of RUL to minimize downtime and optimize operational efficiency.

Traditional methods face several challenges:

  • Complexity of Degradation Dynamics: Bearings exhibit nonlinear degradation patterns influenced by diverse operational conditions and dynamic loads.
  • Data Sparsity: Open-source datasets for RUL estimation, such as NASA IMS and PRONOSTIA, are limited in size and variability, and factories don't usually collect high sample rate data within data lakes.
  • Noisy Predictions: Regression outputs from existing methods often lack reliability, especially when applied to sparse datasets, and don't provide quantifyable measures of model uncertainty.

Problem Analysis

Challenges in RUL Estimation

  • Feature Representation: Traditional time-domain feature based models fail to accurately capture the full spectrum of degradation information, this is because in the time domain there is inherently more sparsity in the distribution of the informational entropy of the data.
  • Uncertain Regression Outputs: Classical regression methods are prone to overfitting and noise, particularly in sparse data scenarios.
  • Uninformed / unguided training: Most deep model implementations don't used an informed semi-parametric approach during training, this can lead to poorer model training from the data, reduced generalizability, or spurious correlations.
  • Scalability in Industrial Applications: Adapting predictive models for real-time, edge-based deployment remains a significant hurdle.

Our Approach

Pipeline Overview

Our hybrid model pipeline integrates time-frequency feature extraction, deep learning, and probabilistic regression.

The overall data pipeline
The overall model pipeline

The key components of the pipeline:

Time-Frequency Analysis (TFA) Stage
Extracts 2D spectral feature representations (scalograms) using methods like Continuous Wavelet Transform (CWT) and Tunable Q-Wavelet Transform for optimal time-frequency localization. Optional wavelet packet decomposition (WPD) filtering stage.

Feature Engineering - Time-Frequency Analysis (TFA) Stage

Degradation Label Engineering
Models approximate degradation paths for bearings based on survival analysis theory. These paths act as training labels for the deep regression models.

Common Degradation Models:

  • Linear Degradation Model: Assumes a constant rate of degradation, suitable for high-stress testing environments where wear progresses uniformly.
  • Single-Sided Weibull Model: Applies when degradation accelerates over time, reflecting real-world scenarios of cumulative damage.
  • Bathtub Curve Model: Represents degradation with three phases—initial burn-in, stable operation, and accelerated wear—common in components with varying stress loads.
  • Cox Proportional Hazards Model: Uses covariates to estimate failure probabilities dynamically, offering flexibility in scenarios with sufficient historical data.
Survival Analysis Statistical Degradation Path Assumption: pdf(t) = h(t)⋅SF(t) = h(t)⋅ (1-CDF(t))
Mapping of the assumed training degradation paths and the scalogram features

Feature Extraction and Regression
Backbone: Tested multiple CNN architectures (e.g., ResNet18, SqueezeNet, EfficientNet v2), custom CNN-LSTM architectures, vision transformer based architectures, tailored for time-frequency data.
Regression Neck: Includes traditional regression, quantile regression (pinball loss), or deep kernel learning (DKL) for probabilistic forecasting.

Secondary Regression Stage
Implements a Non-Stationary Gaussian Process Regression (NSGPR) to refine incomplete RUL trajectories point clouds into complete degradation paths with confidence intervals.

The two stage regression depicted

Meta-Layer Head for Online Learning
Incorporates a mixture of experts (MoE) configuration to aggregate knowledge from multiple specialized models.

Innovative Techniques

  • Heteroscedastic Quantile Regression: Carries uncertainty information into the NSGPR stage for robust predictions.
  • Backpropagation of NSGPR Error: Experimentally integrates probabilistic feedback during model training for improved accuracy.
  • Knowledge Distillation: Combines multiple trained models into a single generalized model for scalable industrial applications.
The model detailed pipeline flow diagram
Example prediction vs a perfectly linear assumed degradation path

Results and Contributions

Key Findings

  1. Improved Prediction Accuracy
    - The hybrid model significantly outperformed state-of-the-art methods in RUL estimation on sparse datasets (NASA IMS, PRONOSTIA).
    - Incorporation of time-normalized input variables enhanced model guidance and reduced noise.

  2. Robust Degradation Curve Estimation
    - NSGPR provided smooth, probabilistic RUL trajectories, reducing uncertainties inherent in raw regression outputs.
    - Quantile regression effectively captured heteroscedasticity in data, improving model reliability.

  3. Scalable Deployment
    - The mixture of experts (MoE) approach enabled knowledge aggregation and more importantly distillation allowing generalization for new components.

Impact on Predictive Maintenance

  • Operational Efficiency: The proposed framework supports precise predictive maintenance scheduling, reducing unplanned downtime.
  • Scalability: Adaptable to various industrial setups, including edge-based and centralized deployments or decentralized deployments.
  • Real-Time Applications: Enables real-time RUL estimation in dynamic environments with minimal computational overhead.
  • Generalizability: The pipeline can be adapted for other industrial components beyond bearings.

Summary

This study introduces a novel hybrid model for RUL estimation that integrates advanced feature extraction, deep learning, and probabilistic regression. By addressing challenges in data sparsity and noise, the framework sets a new benchmark for predictive maintenance systems.

To learn more about our methodology or explore collaboration opportunities, contact us. Together, let’s advance predictive maintenance to the next level.

Contact Us

Read more