Fault Detection in Nuclear Power Plants using Deep Leaning based Image Classification with Imaged Time-series Data


  • Yong Shi University of Chinese Academy of Sciences, Beijing, China
  • Xiaodong Xue University of Chinese Academy of Sciences, Beijing, China
  • Jiayu Xue University of Chinese Academy of Sciences, Beijing, China
  • Yi Qu University of Chinese Academy of Sciences, Beijing, China




fault detection, nuclear power plants, deep learning, image classification, imaged time-series data


Fault detection is critical to ensure the safely routine operations in nuclear power plants (NPPs), requiring very high accuracy and efficiency. Meanwhile, the rapid development of modern information technologies have profoundly changed and promoted various sectors including nuclear industry. Inspired by the great progress and promising performance of deep learning based image classification recent years, a two-stage fault detection methodology in NPPs has been proposed in this paper. First the time-series data describing the operating status of NPPs have been transformed into two-dimensional images by four methods, preserving the time-series information in images and converting the fault detection problem into a supervised image classification task. Then four specific image classifying models based on three primary deep learning architectures have been separately experimented on the imaged time-series data, achieving excellent accuracies. Further the performances of different combinations of transforming means and classifying models have been compared and discussed with extensive experiments and detailed analysis of throughput for four transforming methods. This methodology proposed has obtained remarkable results by reshaping data format and structure, making image classifying models applicable, which not only efficiently detect and warn possible faults in NPPs but also enhances the capability for safety management in nuclear power systems.


[1] Aizpurua, J. I.; Mcarthur, S.; Stewart, B. G.; Lambert, B.; Cross, J. G.; Catterson, V. M. (2019). Adaptive Power Transformer Lifetime Predictions Through Machine Learning and Uncertainty Modeling in Nuclear Power Plants, IEEE Transactions on Industrial Electronics, 66, 4726-4737, 2019. https://doi.org/10.1109/TIE.2018.2860532

[2] Ayodeji, A.; Liu, Y. (2019). PWR heat exchanger tube defects: Trends, signatures and diagnostic techniques, Progress in Nuclear Energy, 112, 171-184, 2019. https://doi.org/10.1016/j.pnucene.2018.12.017

[3] Caron, M.; Touvron, H.; Misra, I.; J'egou, H.; Mairal, J.; Bojanowski, P.; Joulin, A. (2021). Emerging Properties in Self-Supervised Vision Transformers, ArXiv, abs/2104.14294, Avbailable: https://arxiv.org/abs/2104.14294, 2021.

[4] Chen, F.; Jahanshahi, M. R. (2018). NB-CNN: Deep Learning-Based Crack Detection Using Convolutional Neural Network and Naí¯ve Bayes Data Fusion, IEEE Transactions on Industrial Electronics, 65, 4392-4400, 2018. https://doi.org/10.1109/TIE.2017.2764844

[5] Chen, L.; Papandreou, G.; Kokkinos, I.; Murphy, K. P.; Yuille, A. L. (2018). DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs, IEEE Transactions on Pattern Analysis and Machine Intelligence, 40, 834-848, 2018. https://doi.org/10.1109/TPAMI.2017.2699184

[6] Deleplace, A.; Atamuradov, V.; Allali, A.; Pelle, J. T.; Plana, R.; Alleaume, G. (2020). Ensemble Learning-based Fault Detection in Nuclear Power Plant Screen Cleaners, IFAC-PapersOnLine, 53, 10354-10359, 2020. https://doi.org/10.1016/j.ifacol.2020.12.2773

[7] Deng, J.; Dong, W.; Socher, R.; Li, L.; Li, K.; Fei-Fei, L. (2009). ImageNet: A large-scale hierarchical image database, 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 248-255, 2009. https://doi.org/10.1109/CVPR.2009.5206848

[8] Dong, C.; Loy, C. C.; He, K.; Tang, X. (2016). Image Super-Resolution Using Deep Convolutional Networks, IEEE Transactions on Pattern Analysis and Machine Intelligence, 38, 295-307, 2016. https://doi.org/10.1109/TPAMI.2015.2439281

[9] Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; Uszkoreit, J.; Houlsby, N. (2021). An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale, 9th International Conference on Learning Representations (ICLR), 2021.

[10] Eckmann, J.; Kamphorst, S. O.; Ruelle, D. (1987). Recurrence Plots of Dynamical Systems, Europhysics Letters, 4, 973-977, 1987. https://doi.org/10.1209/0295-5075/4/9/004

[11] Guo, Z.; Wu, Z.; Liu, S.; Ma, X.; Wang, C.; Yan, D.; Niu, F. (2020). Defect detection of nuclear fuel assembly based on deep neural network, Annals of Nuclear Energy, 137, 107078, 2020. https://doi.org/10.1016/j.anucene.2019.107078

[12] He, C.; Ge, D.; Yang, M.; Yong, N.; Wang, J.; Yu, J. (2021). A data-driven adaptive fault diagnosis methodology for nuclear power systems based on NSGAII-CNN, Annals of Nuclear Energy, 159, 108326, 2021. https://doi.org/10.1016/j.anucene.2021.108326

[13] He, K.; Gkioxari, G.; Dollár, P.; Girshick, R. B. (2020). Mask R-CNN, IEEE Transactions on Pattern Analysis and Machine Intelligence, 42, 386-397, 2020. https://doi.org/10.1109/TPAMI.2018.2844175

[14] He, K.; Zhang, X.; Ren, S.; Sun, J. (2016). Deep Residual Learning for Image Recognition, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770-778, 2016. https://doi.org/10.1109/CVPR.2016.90

[15] Hendrycks, D.; Gimpel, K. (2016). Gaussian Error Linear Units (GELUs), ArXiv, abs/1606.08415, Available: http://arxiv.org/abs/1606.08415, 2016.

[16] Holdsworth, A. F.; George, K. E.; Adams, S. J.; Sharrad, C. A. (2021). An accessible statistical regression approach for the estimation of spent nuclear fuel compositions and decay heats to support the development of nuclear fuel management strategies, Progress in Nuclear Energy, 141, 103935, 2021. https://doi.org/10.1016/j.pnucene.2021.103935

[17] Hosaka, T. (2019). Bankruptcy prediction using imaged financial ratios and convolutional neural networks, Expert Systems with Applications, 117, 287-299, 2019. https://doi.org/10.1016/j.eswa.2018.09.039

[18] Hu, G.; Zhou, T.; Liu, Q. (2021). Data-Driven Machine Learning for Fault Detection and Diagnosis in Nuclear Power Plants: A Review, Frontiers in Energy Research, 9, 1-12, 2021. https://doi.org/10.3389/fenrg.2021.663296

[19] Jolliffe, I. T. (1986). Principal Component Analysis, Springer, 1986. https://doi.org/10.1007/978-1-4757-1904-8

[20] Kim, J.; Lee, J. K.; Lee, K. M. (2016). Accurate Image Super-Resolution Using Very Deep Convolutional Networks, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1646-1654, 2016. https://doi.org/10.1109/CVPR.2016.182

[21] Krizhevsky, A.; Sutskever, I.; Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks, Advances in Neural Information Processing Systems (NIPS), 25, 2012.

[22] Lee, G.; Lee, S. J.; Lee, C. (2021). A convolutional neural network model for abnormality diagnosis in a nuclear power plant, Applied Soft Computing, 99, 106874, 2021. https://doi.org/10.1016/j.asoc.2020.106874

[23] Li, J.; Lin, M. (2021). Ensemble learning with diversified base models for fault diagnosis in nuclear power plants, Annals of Nuclear Energy, 158, 108265, 2021. https://doi.org/10.1016/j.anucene.2021.108265

[24] Lin, T.; Dollár, P.; Girshick, R. B.; He, K.; Hariharan, B.; Belongie, S. J. (2017). Feature Pyramid Networks for Object Detection, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 936-944, 2017. https://doi.org/10.1109/CVPR.2017.106

[25] Ling, J.; Liu, G.; Li, J.; Shen, X.; You, D. (2020). Fault prediction method for nuclear power machinery based on Bayesian PPCA recurrent neural network model, Nuclear Science and Techniques, 31, 1-11, 2020. https://doi.org/10.1007/s41365-020-00792-9

[26] Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S. E.; Fu, C.; Berg, A. C. (2016). SSD: Single Shot MultiBox Detector, 14th European Conference on Computer Vision (ECCV), 21-37, 2016. https://doi.org/10.1007/978-3-319-46448-0_2

[27] Liu, Z.; Lin, Y.; Cao, Y.; Hu, H.; Wei, Y.; Zhang, Z.; Lin, S.; Guo, B. (2021). Swin Transformer: Hierarchical Vision Transformer using Shifted Windows, ArXiv, abs/2103.14030, Available: http://arxiv.org/abs/2103.14030, 2021.

[28] Loshchilov, I.; Hutter, F. (2019). Decoupled Weight Decay Regularization, 7th International Conference on Learning Representations (ICLR), 2019.

[29] Mandal, S.; Santhi, B.; Sridhar, S.; Vinolia, K.; Swaminathan, P. (2017). Nuclear Power Plant Thermocouple Sensor-Fault Detection and Classification Using Deep Learning and Generalized Likelihood Ratio Test, IEEE Transactions on Nuclear Science, 64, 1526-1534, 2017. https://doi.org/10.1109/TNS.2017.2697919

[30] Meng, J.; Su, Y.; Xie, S. (2020). Loose parts detection method combining blind deconvolution with support vector machine, Annals of Nuclear Energy, 149, 107782, 2020. https://doi.org/10.1016/j.anucene.2020.107782

[31] Mo, K.; Lee, S. J.; Seong, P.H. (2007). A dynamic neural network aggregation model for transient diagnosis in nuclear power plants, Progress in Nuclear Energy, 49, 262-272, 2007. https://doi.org/10.1016/j.pnucene.2007.01.002

[32] Mohapatra, D.; Subudhi, B.; Daniel, R. (2020). Real-time sensor fault detection in Tokamak using different machine learning algorithms, Fusion Engineering and Design, 151, 111401, 2020. https://doi.org/10.1016/j.fusengdes.2019.111401

[33] Nicolau, A. D.; Augusto, J. P.; Schirru, R. (2017). Accident diagnosis system based on real-time decision tree expert system, AIP Conference Proceeding, 1836, 020017, 2017. https://doi.org/10.1063/1.4981957

[34] Oh, C.; Lee, J. I. (2020). Real time nuclear power plant operating state cognitive algorithm development using dynamic Bayesian network, Reliability Engineering and System Safety, 198, 106879, 2020. https://doi.org/10.1016/j.ress.2020.106879

[35] Po, L. (2019). Conceptual Design of an Accident Prevention System for Light Water Reactors Using Artificial Neural Network and High-Speed Simulator, Nuclear Technology, 206, 505-513, 2019. https://doi.org/10.1080/00295450.2019.1641877

[36] Racheal, S.; Liu, Y.; Ernest, M.; Ayodeji, A. (2021). A Systematic Review of PCTRAN-Based Pressurized Water Reactor Transient Analysis, Proceedings of the 2021 28th International Conference on Nuclear Engineering, 2021. https://doi.org/10.1115/ICONE28-65663

[37] Redmon, J.; Divvala, S. K.; Girshick, R. B.; Farhadi, A. (2016). You Only Look Once: Unified, Real-Time Object Detection, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 779-788, 2016. https://doi.org/10.1109/CVPR.2016.91

[38] Rezaeianjouybari, B.; Shang, Y. (2020). Deep learning for prognostics and health management: State of the art, challenges, and opportunities, Measurement, 163, 107929, 2020. https://doi.org/10.1016/j.measurement.2020.107929

[39] Robbins, H. E. (2007). A Stochastic Approximation Method, Annals of Mathematical Statistics, 22, 400-407, 1951. https://doi.org/10.1214/aoms/1177729586

[40] Ronneberger, O.; Fischer, P.; Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation, 18th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), 234-241, 2015. https://doi.org/10.1007/978-3-319-24574-4_28

[41] Saeed, H. A.; Peng, M.; Wang, H.; Zhang, B. (2020). Novel fault diagnosis scheme utilizing deep learning networks, Progress in Nuclear Energy, 118, 103066, 2020. https://doi.org/10.1016/j.pnucene.2019.103066

[42] Shi, Y.; Xue, X.; Qu, Y.; Xue, J.; Zhang, L. (2021). Machine Learning and Deep Learning Methods used in Safety Management of Nuclear Power Plants : A Survey, 2021 International Conference on Data Mining Workshops (ICDMW), 917-924, 2021. https://doi.org/10.1109/ICDMW53433.2021.00120

[43] Simonyan, K.; Zisserman, A. (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition, 3rd International Conference on Learning Representations (ICLR), 2015.

[44] Smith, C. L.; Borgonovo, E. (2007). Decision making during nuclear power plant incidents: a new approach to the evaluation of precursor events, Risk analysis, 27, 1027-1042, 2007. https://doi.org/10.1111/j.1539-6924.2007.00941.x

[45] Tan, M.; Le, Q. V. (2019). EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks, Proceedings of the 36th International Conference on Machine Learning (ICML), 6105- 6114, 2019.

[46] Tian, W.; Wu, J.; Cui, H.; Hu, T. (2021). Drought Prediction Based on Feature-Based Transfer Learning and Time Series Imaging, IEEE Access, 9, 101454-101468, 2021. https://doi.org/10.1109/ACCESS.2021.3097353

[47] Tolstikhin, I. O.; Houlsby, N.; Kolesnikov, A.; Beyer, L.; Zhai, X.; Unterthiner, T.; Yung, J.; Keysers, D.; Uszkoreit, J.; Lucic, M.; Dosovitskiy, A. (2021). MLP-Mixer: An all-MLP Architecture for Vision, ArXiv, abs/2105.01601, Available: http://arxiv.org/abs/2105.01601, 2021.

[48] Touvron, H.; Bojanowski, P.; Caron, M.; Cord, M.; El-Nouby, A.; Grave, E.; Izacard, G.; Joulin, A.; Synnaeve, G.; Verbeek, J.; J'egou, H. (2021). ResMLP: Feedforward networks for image classification with data-efficient training, ArXiv, abs/2105.03404, Available: http://arxiv.org/abs/2105.03404, 2021.

[49] Touvron, H.; Cord, M.; Douze, M.; Massa, F.; Sablayrolles, A.; J'egou, H. (2021). Training dataefficient image transformers & distillation through attention, Proceedings of the 38th International Conference on Machine Learning (ICML, 10347-10357, 2021.

[50] Vaswani, A.; Shazeer, N. M.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, L.; Polosukhin, I. (2017). Attention is All you Need, Advances in Neural Information Processing Systems (NIPS), 30, 2017.

[51] Wang, Z.; Oates, T. (2015). Imaging Time-Series to Improve Classification and Imputation, Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI), 3939- 3945, 2015.

[52] Wang, H.; Peng, M.; Wesley Hines, J.; Zheng, G.; Liu, Y.; Upadhyaya, B. R. (2019). A hybrid fault diagnosis methodology with support vector machine and improved particle swarm optimization for nuclear power plants, ISA transactions, 95, 358-371, 2019. https://doi.org/10.1016/j.isatra.2019.05.016

[53] Wang, X.; Xu, Z. S.; Dzitac, I. (2019). Bibliometric Analysis on Research Trends of International Journal of Computers Communications & Control, International Journal of Computers Communications & Control, 14, 711-732, 2019. https://doi.org/10.15837/ijccc.2019.5.3685

[54] Yang, C.; Chen, Z.; Yang, C. (2020). Sensor Classification Using Convolutional Neural Network by Encoding Multivariate Time Series as Two-Dimensional Colored Images, Sensors, 20, 1, 168, 2020. https://doi.org/10.3390/s20010168

[55] Yao, Y.; Wang, J.; Long, P.; Xie, M.; Wang, J. (2020). Small-batch-size convolutional neural network based fault diagnosis system for nuclear energy production safety with big-data environment, International Journal of Energy Research, 44, 5841-5855, 2020. https://doi.org/10.1002/er.5348

Additional Files



Most read articles by the same author(s)

Obs.: This plugin requires at least one statistics/report plugin to be enabled. If your statistics plugins provide more than one metric then please also select a main metric on the admin's site settings page and/or on the journal manager's settings pages.