A Stochastic Max Pooling Strategy for Convolutional Neural Network Trained by Noisy Samples

  • Shuai Sun Department of Electrical Engineering, Zhengzhou Electric Power College, Zhengzhou 450000, China
  • Bin Hu Department of Electrical Engineering, Zhengzhou Electric Power College, Zhengzhou 450000, China
  • Zhou Yu Department of Electrical Engineering, Zhengzhou Electric Power College, Zhengzhou 450000, China
  • Xiaona Song College of Mechanical Engineering, North China University of Water Resources and Electric Power, Zhengzhou 450000, China

Abstract

The deep convolutional neural network (CNN) has made remarkable progress in image classification. However, this network performs poorly and even cannot converge in many actual applications, where the training and test samples contain lots of noises. To solve the problems, this paper puts forward a network training strategy based on stochastic max pooling. Unlike the traditional max pooling, the proposed strategy first ranks all the values in each receptive field, and then selects a random value from the top-n values as the pooling result. Compared with common pooling methods, stochastic max pooling can limit the pooling selection to a larger value that represents the main information of the pooling area which reduces the chance of introducing noises into the network, and enhances the robustness of extracting noisy image features. Experimental results show that the CNN used stochastic max pooling Strategy can converge better than traditional CNN and classified noisy images much more accurately than traditional pooling methods.

References

[1] Benkaddour, M.K.; Bounoua, A. (2017). Feature extraction and classification using deep convolutional neural networks, PCA and SVC for face recognition, Traitement du Signal, 34(1-2), 77-91, 2017.
https://doi.org/10.3166/ts.34.77-91

[2] Boureau, Y.L.; Bach, F.; LeCun, Y.; Ponce, J. (2010). Learning mid-level features for recognition, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2559- 2566, 2010.
https://doi.org/10.1109/CVPR.2010.5539963

[3] Gidaris, S.; Komodakis, N. (2015). Object detection via a multi-region and semantic segmentationaware CNN model, Proceedings of the IEEE International Conference on Computer Vision, 1134- 1142, 2015.
https://doi.org/10.1109/ICCV.2015.135

[4] Girshick, R.; Donahue, J.; Darrell, T.; Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation, Proceedings of the IEEE conference on computer vision and pattern recognition, 580-587, 2014.
https://doi.org/10.1109/CVPR.2014.81

[5] He, K.; Zhang, X.; Ren, S.; Sun, J. (2016). Deep residual learning for image recognition, 2016 IEEE Conference on Computer Vision and Pattern Recognition, 770-778, 2016.
https://doi.org/10.1109/CVPR.2016.90

[6] Krizhevsky, A.; Sutskever, I.; Hinton, G.E. (2012). Imagenet classification with deep convolutional neural networks, Advances in Neural Information Processing Systems, 25(2), 1097-1105, 2012.

[7] Lakshmipathi, A.N.; Battula, B.P. (2018). Deep convolutional neural networks for product recommendation, Ingénierie des Systèmes d'Information, 23(6), 161-172, 2018.
https://doi.org/10.3166/isi.23.6.161-172

[8] Nair, V.; Hinton, G.E. (2010). Rectified linear units improve restricted boltzmann machines, Proceedings of the 27th international conference on machine learning (ICML-10), 807-814, 2010.

[9] Neelapu, R.; Devi, G.L.; Rao, K.S. (2018). Deep learning based conventional neural network architecture for medical image classification, Traitement du Signal, 35(2), 169-182, 2018.
https://doi.org/10.3166/ts.35.169-182

[10] Raguram, L.S.B.; Shanmugam, V.M. (2017). Deep belief networks for phoneme recognition in continuous Tamil speech-an analysis, Traitement du Signal, 34(3-4), 137-151, 2017.
https://doi.org/10.3166/ts.34.137-151

[11] Ren, S.; He, K.; Girshick, R.; Sun, J. (2017). Faster R-CNN: towards real-time object detection with region proposal networks, IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137-1149, 2017.
https://doi.org/10.1109/TPAMI.2016.2577031

[12] Simonyan, K.; Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition, arXiv.

[13] Srivastava, N.; Hinton, G.E.; Krizhevsky, A.; Sutskever, I.; Salakhutdinov, R. (2014). Dropout: A simple way to prevent neural networks from overfitting, Journal of Machine Learning Research, 15(1), 1929-1958, 2014.

[14] Wen, Y.; Zhang, K.; Li, Z.; Qiao, Y. (2016). A discriminative feature learning approach for deep face recognition, European Conference on Computer Vision, 499-515, 2016.
https://doi.org/10.1007/978-3-319-46478-7_31

[15] Zeiler, M.D.; Fergus, R. (2013). Stochastic pooling for regularization of deep convolutional neural networks, arXiv.
Published
2020-02-03
How to Cite
SUN, Shuai et al. A Stochastic Max Pooling Strategy for Convolutional Neural Network Trained by Noisy Samples. INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL, [S.l.], v. 15, n. 1, feb. 2020. ISSN 1841-9844. Available at: <http://univagora.ro/jour/index.php/ijccc/article/view/1007>. Date accessed: 01 dec. 2020. doi: https://doi.org/10.15837/ijccc.2020.1.3712.

Keywords

image classification, deep learning, convolutional neural network (CNN), stochastic max pooling