A Social Network Image Classification Algorithm Based on Multimodal Deep Learning
Keywords:
multimodal deep learning, social network, image classification, three-dimensional neural network (3D NN)Abstract
The complex data structure and massive image data of social networks pose a huge challenge to the mining of associations between social information. For accurate classification of social network images, this paper proposes a social network image classification algorithm based on multimodal deep learning. Firstly, a social network association clustering model (SNACM) was established, and used to calculate trust and similarity, which represent the degree of similarity between users. Based on artificial ant colony algorithm, the SNACM was subject to weighted stacking, and the social network image association network was constructed. After that, the social network images of three modes, i.e. RGB (red-green-blue) image, grayscale image, and depth image, were fused. Finally, a three-dimensional neural network (3D NN) was constructed to extract the features of the multimodal social network image. The proposed algorithm was proved valid and accurate through experiments. The research results provide a reference for applying multimodal deep learning to classify the images in other fields.
References
[2] Arepalli, P.G.; Narayana, V.L.; Venkatesh, R.; Kumar, N.A. (2019). Certified node frequency in social network using parallel diffusion methods, Ingenierie des Systemesd'Information, 24(1), 113-117, 2019. https://doi.org/10.18280/isi.240117
[3] Chirra, V.R.R.; Uyyala, S.R.; Kolli, V.K.K. (2019). Deep CNN: A machine learning approach for driver drowsiness detection based on eye state, Revue d'Intelligence Artificielle, 33(6), 461-466, 2019. https://doi.org/10.18280/ria.330609
[4] Claude, U. (2020). Predicting tourism demands by google trends: A hidden markov models based study, Journal of System and Management Sciences, 10(1), 106-120, 2020.
[5] De Salve, A.; Di Pietro, R.; Mori, P.; Ricci, L. (2017). A logical key hierarchy based approach to preserve content privacy in decentralized online social networks, IEEE Transactions on Dependable and Secure Computing, 17(1), 2-21, 2017. https://doi.org/10.1109/TDSC.2017.2729553
[6] Gothania, J.; Rathore, S.K. (2019). Performance metrics for chromatic correlation clustering for social network analysis, Revue d'Intelligence Artificielle, 33(5), 373-378, 2019. https://doi.org/10.18280/ria.330507
[7] Gu, Y.; Chanussot, J.; Jia, X.; Benediktsson, J.A. (2017). Multiple kernel learning for hyperspectral image classification: A review, IEEE Transactions on Geoscience and Remote Sensing, 55(11), 6547-6565, 2017. https://doi.org/10.1109/TGRS.2017.2729882
[8] Hang, R.; Liu, Q.; Hong, D.; Ghamisi, P. (2019). Cascaded recurrent neural networks for hyperspectral image classification, IEEE Transactions on Geoscience and Remote Sensing, 57(8), 5384-5394, 2019.
[9] Huang, W.D.; Wang, Q.; Cao, J. (2018). Tracing public opinion propagation and emotional evolution based on public emergencies in social networks, International Journal of Computers Communications & Control, 13(1), 129-142, 2018.
[10] Kim, J.H.; Kim, M.S.; Hong, R.K.; Ko, J.W. (2019). Continuous use intention of corporate mobile SNS users and its determinants: application of extended technology acceptance model, Journal of System and Management Sciences, 9(4), 12-28, 2019.
[11] Kirsal, Y.; Paranthaman, V.V.; Mapp, G. (2018). Exploring Analytical Models for Proactive Resource Management in Highly Mobile International Journal of Computers Communications & Control, 13(5), 837-852, 2018. https://doi.org/10.15837/ijccc.2018.5.3349
[12] Kuhnle, A.; Pan, T.; Alim, M.A.; Thai, M.T. (2017). Scalable bicriteria algorithms for the threshold activation problem in online social networks, In IEEE INFOCOM 2017-IEEE Conference on Computer Communications, IEEE, 1-9, 2017. https://doi.org/10.1109/INFOCOM.2017.8057068
[13] Li, S.; Song, W.; Fang, L.; Chen, Y.; Ghamisi, P.; Benediktsson, J.A. (2019). Deep learning for hyperspectral image classification: An overview, IEEE Transactions on Geoscience and Remote Sensing, 57(9), 6690-6709, 2019. https://doi.org/10.1109/TGRS.2019.2907932
[14] Mallick, P.K.; Ryu, S.H.; Satapathy, S.K.; Mishra, S.; Nguyen, G.N.; Tiwari, P. (2019). Brain MRI image classification for cancer detection using deep wavelet autoencoder-based deep neural network, IEEE Access, 7, 46278-46287, 2019. https://doi.org/10.1109/ACCESS.2019.2902252
[15] Meng, W.L.; Mao, C.Z.; Zhang, J.; Wen, J.; Wu, D.H. (2019). A fast recognition algorithm of online social network images based on deep learning, Traitement du Signal, 36(6), 575-580, 2019. https://doi.org/10.18280/ts.360613
[16] Minaev, V.A.; Dvoryankin, S.V. (2016). Foundation and description of informational and psychological destructive nature influences dynamics model in social networks, Bezopasnost informatsionnykh tekhnologiy= IT Security, 23(3), 40-52, 2016.
[17] Paoletti, M.E.; Haut, J.M.; Plaza, J.; Plaza, A. (2018). A new deep convolutional neural network for fast hyperspectral image classification, ISPRS journal of photogrammetry and remote sensing, 145, 120-147, 2018.
[18] Pensa, R.G.; Di Blasi, G.; Bioglio, L. (2019). Network-aware privacy risk estimation in online social networks, Social Network Analysis and Mining, 9(1), 15, 2019. https://doi.org/10.1007/s13278-019-0558-x
[19] Roy, S.K.; Krishna, G.; Dubey, S.R.; Chaudhuri, B.B. (2019). HybridSN: Exploring 3-D-2-D CNN feature hierarchy for hyperspectral image classification, IEEE Geoscience and Remote Sensing Letters, 17(2), 277-281, 2019.
[20] Sajja, T.K.; Devarapalli, R.M.; Kalluri, H.K. (2019). Lung cancer detection based on CT scan images by using deep transfer learning, Traitement du Signal, 36(4), 339-344, 2019. https://doi.org/10.18280/ts.360406
[21] Shu, K.; Bernard, H.R.; Liu, H. (2019). Studying fake news via network analysis: detection and mitigation, In Emerging Research Challenges and Opportunities in Computational Social Network Analysis and Mining, Springer, Cham, 43-65, 2019. https://doi.org/10.1007/978-3-319-94105-9_3
[22] Van Schaik, P.; Jansen, J.; Onibokun, J.; Camp, J.; Kusev, P. (2018). Security and privacy in online social networking: Risk perceptions and precautionary behaviour, Computers in Human Behavior, 78, 283-297, 2018. https://doi.org/10.1016/j.chb.2017.10.007
[23] Venkatesan, S.; Oleshchuk, V.A.; Chellappan, C.; Prakash, S. (2016). Analysis of key management protocols for social networks, Social Network Analysis and Mining, 6(1), 3, 2016. https://doi.org/10.1007/s13278-015-0310-0
[24] Wajeed, M.A.; Sreenivasulu, V. (2019). Image based tumor cells identification using convolutional neural network and auto encoders, Traitement du Signal, 36(5), 445-453, 2019. https://doi.org/10.18280/ts.360510
[25] Zhang, X.F.; Chen, X.L.; Seng, D.W.; Fang, X.J. (2019). A factored similarity model with trust and social influence for top-N recommendation, International Journal of Computers Communications & Control, 14(4), 590-607, 2019. https://doi.org/10.15837/ijccc.2019.4.3577
[26] Zhu, L.; Chen, Y.; Ghamisi, P.; Benediktsson, J.A. (2018). Generative adversarial networks for hyperspectral image classification, IEEE Transactions on Geoscience and Remote Sensing, 56(9), 5046-5063, 2018. https://doi.org/10.1109/TGRS.2018.2805286
Additional Files
Published
Issue
Section
License
ONLINE OPEN ACCES: Acces to full text of each article and each issue are allowed for free in respect of Attribution-NonCommercial 4.0 International (CC BY-NC 4.0.
You are free to:
-Share: copy and redistribute the material in any medium or format;
-Adapt: remix, transform, and build upon the material.
The licensor cannot revoke these freedoms as long as you follow the license terms.
DISCLAIMER: The author(s) of each article appearing in International Journal of Computers Communications & Control is/are solely responsible for the content thereof; the publication of an article shall not constitute or be deemed to constitute any representation by the Editors or Agora University Press that the data presented therein are original, correct or sufficient to support the conclusions reached or that the experiment design or methodology is adequate.
