A Review on Reinforcement Learning in Condition-based Maintenance

Tran, Quang Khai; Huynh, Khac Tuan; Grall, Antoine; Langeron, Yves; Mosayebi Omshi, Elham

Permalink

https://hdl.handle.net/2142/121799 Copy

Description

Title

A Review on Reinforcement Learning in Condition-based Maintenance

Author(s)

• Tran, Quang Khai
• Huynh, Khac Tuan
• Grall, Antoine
• Langeron, Yves
• Mosayebi Omshi, Elham

Issue Date

2023

Keyword(s)

• Condition-based maintenance
• Reinforcement learning
• Dynamic programming
• Review

Abstract

In this paper, we examine the implementation of reinforcement learning in the field of condition-based maintenance. It begins with a brief overview of the pertinent areas. In response to the query of why reinforcement learning is becoming an attractive tool in condition-based maintenance, an extensive review of reinforcement learning in this field is provided. Finally, the prospective direction of research is discussed. Condition-based maintenance involves monitoring the condition of a system and performing maintenance only when necessary, as opposed to traditional time-based maintenance, which maintains systems on a regular schedule regardless of their current state. Numerous industrial cases and academic studies have shown that condition-based maintenance yields superior results compared to time-based maintenance in a range of situations. Condition-based maintenance, like other maintenance strategies, seeks to maximize the system's availability while minimizing the maintenance cost. There are different approaches to describing a condition-based maintenance issue. A common one is to apply a parametric structure to the system under consideration. In other words, the maintained system is structured based on parameters such as the preventive maintenance threshold and the interval between inspections. Decision-makers must determine the optimal value of these parameters in order to minimize the average maintenance cost. This strategy has been studied exhaustively over the past several decades. Modularity is a substantial advantage of this method. We can alter the system's underlying degeneration process without altering the problem's overall structure. While it is possible to discover optimal values for the structure's parameters, it is challenging to demonstrate that this structure is the optimal structure for the problem. To surmount this limitation, recent research examines the condition-based maintenance problem as a sequential decision-making problem and formulates it as a Markov decision process and its variations. Besides the renowned dynamic programming method that can be used when the transition function of the MDP is known, reinforcement learning is a modern and flexible tool that can also be used to obtain solutions to MDP problems. Reinforcement learning is a class of machine learning techniques used to solve problems involving sequential decision-making. It is a method founded on rewards that guides a decision-maker to act rationally in a stochastic environment. In contrast to the structural parametric approach, the decision-maker must determine the maintenance action at each state encountered in order to minimize the maintenance cost. In other words, we seek a policy that minimizes the cost of maintenance. It is the key distinction between the parametric approach and the sequential decision-making approach, and it will be analyzed in detail in this paper: in the former, the policy is given at the outset, and optimal parameter values are sought for the parameterized policy, whereas in the latter, the policy must be determined. This new approach relaxes the predetermined structure and permits greater flexibility in the maintenance policy. This review also provides a classification of reinforcement learning algorithms according to the system's characteristics.

Type of Resource

text

Language

eng

Handle URL

https://hdl.handle.net/2142/121799

Owning Collections