Revision and implementation of metrics to evaluate the performance of prognostics models

Abstract

Prognostics is used in predictive maintenance to estimate the remaining time to the end of the life of a system or component. Among the many challenges of prognostics is the need for model verification and validation. Over the years, several objective metrics have been utilized by the community. Some of these metrics came from statistics, others from forecasting, and others have been proposed specifically for prognostics. A single “perfect” metric has not yet been put forward. Finding one metric that can excel in all evaluation dimensions and case studies is an open question. In this review, we analyze the most important metrics of prognostics. A set of 19 metrics is subject to analysis and implementation. The metrics are implemented on a public GitHub project. Our analysis focuses only on metrics for deterministic predictions. Stochastic predictions are out of the scope. The paper describes properties, advantages, disadvantages, and industrial applicability of each metric. We also discuss potential modifications to the existing metrics and the development of new metrics. A final table summarizes the main properties of the metrics. Our goal is to raise awareness about prognostics metrics and help establish a common evaluation procedure. Code available at: MetricsForPrognostics.