A Q-learning based method of optimal fault diagnostic policy with imperfect tests

Article type: Research Article

Authors: Liang, Yajun^{; *} | Xiao, Mingqing | Tang, Xilang | Ge, Yawei | Wang, Xiaofei

Affiliations: Aeronautics Engineering College, Air Force Engineering University, Xi’an, China

Correspondence: [*] Corresponding authors. Yajun Liang, Aeronautics Engineering College, Air Force Engineering University, Xi’an, 710038, China. Tel.: +86 84 18981236690; E-mails: sunliyangna@stu.xjtu.edu.cn and 1214102891@qq.com.

Abstract: As an efficient way for fault diagnosis, diagnostic policy is to build a test sequence to achieve specified fault diagnosis and accurate failure location with minimal test cost. Traditional diagnostic policy building methods are not suitable for unreliable and imperfect tests due to the uncertainty caused by various outside or inside interference in the real-world systems extensively. Therefore, in this paper, a novel method for diagnostic policy based on Q-learning is proposed to get the optimal policy of long run in the realistic systems. We construct the diagnostic policy model based on the framework of RL. Then the probability of false alarm or misdetection can be learnt from the environment not partial statistic data through Q-learning algorithm. And the optimal diagnostic policy with imperfect tests is obtained with high efficiency, also considering the test cost and information gain. The proposed method is illustrated by a real-world application. The effectiveness and feasibility of this method are also verified by comparison results.

Keywords: Optimal diagnostic policy, imperfect tests, reinforcement learning, Q-learning

DOI: 10.3233/JIFS-181799

Journal: Journal of Intelligent & Fuzzy Systems, vol. 36, no. 6, pp. 6013-6024, 2019