A Hybrid Model for Concurrent Interaction Recognition from Videos


  • M. Sivarathinabala Anna University
  • S. Abirami Department of Information Science and Technology, Anna University, Chennai, India.


Pose prediction, interaction recognition, layered HMM


Human behavior analysis plays an important role in understanding the high-level human activities from surveillance videos. Human behavior has been identified using gestures, postures, actions, interactions and multiple activities of humans. This paper has been analyzed by identifying concurrent interactions, that takes place between multiple peoples. In order to capture the concurrency, a hybrid model has been designed with the combination of Layered Hidden Markov Model (LHMM) and Coupled HMM (CHMM). The model has three layers called as pose layer, action layer and interaction layer, in which pose and action of the single person has been defined in the layered model and the interaction of two persons or multiple persons are defined using CHMM. This hybrid model reduces the training parameters and the temporal correlations over the frames are maintained. The spatial and temporal information are extracted and from the body part attributes, the simple human actions as well as concurrent actions/interactions are predicted. In addition, we further evaluated the results on various datasets also, for analyzing the concurrent interaction between the peoples.

Author Biography

M. Sivarathinabala, Anna University

Department of Information Science and Technology


Alexandros Andre Chaaraoui, Pau Climent-Perez, Francisco Florez-Revuelta(2013); Silhouette-based human action recognition using sequences of key poses, Pattern Recognition Letters, 34(15): 1799-1807. http://dx.doi.org/10.1016/j.patrec.2013.01.021

Aggarwal, J. K.and Ryoo, M. S. (2011); Human activity analysis: A review, ACM Computing Survey, 43(3): 16:1-16:43.

Arnold Wiliem,Vamsi Madasu, Wageeh Boles and Prasad Yarlagadda (2012); A suspicious behaviour detection using a context space model for smart surveillance systems, computer vision and Image Understanding, 116(2): 194-209.

Fadime sener and Nazli Ikizler-cinbis (2015); Two Person Interaction Recognition via spatial Multiple Instance Embedding, Journal of Visual Communication and Image Representation, 32: 63-73. http://dx.doi.org/10.1016/j.jvcir.2015.07.016

Gowsikhaa.D, Abirami.S and Baskaran.R. (2012); Automated human behavior analysis from surveillance videos: a survey, Artificial Intelligence Review, 1-19.

Gowsikhaa.D, Manjunath and Abirami S. (2012); Suspicious Human activity detection from Surveillance videos, International Journal on Internet and Distributed Computing Systems, 2(2): 141-149.

Junji Yamato, Jun Ohya and Kenichiro Ishii (1992); Recognizing Human Action in Time- Sequential Images using Hidden Markov Model, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 379- 385.

Matthew Brand, Nuria Oliver, and Alex Pentland (1997); Coupled Hidden Markov Models for Complex Activity Recognition, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 994 - 999.

Nuria Oliver, Ashutosh Garg and Eric Horvitz (2004); Layered Representations for learning and inferring office activity from multiple sensor channels,Computer Vision and Image Understanding, 96: 163-180. http://dx.doi.org/10.1016/j.cviu.2004.02.004

Roberto Melfi, Shripad Kondra and Alfredo Petrosino (2013); Human activity modeling by spatio temporal textural appearance, Pattern Recognition Letters,34(15): 1990-1994. http://dx.doi.org/10.1016/j.patrec.2013.04.025

Ryoo M.S. (2011); Human Activity Prediction: Early Recognition of Ongoing Activities from Streaming Videos, Proceedings of IEEE International Conference on Computer Vision (ICCV), 1036-1043. http://dx.doi.org/10.1109/iccv.2011.6126349

Ryoo,M.S, and Aggarwal, J.K. (2010); UT Interaction Dataset, Proc. of ICPR Contest on Semantic Description of Human activities, http://cvrc.ece.utexas.edu/SDHA2010/HumanI nteraction.html.

Sangho Park and J.K. Aggarwal (2004); Semantic-level Understanding of Human Actions and Interactions using Event Hierarchy, Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop, 1-12.

Sang Min Yoon, Arjan Kuijper (2013); Human action recognition based on skeleton splitting, Expert systems with Applications, 40(17): 6848-6855. http://dx.doi.org/10.1016/j.eswa.2013.06.024

Sivarathinabala M. and Abirami S. (2014); Motion Tracking of Humans under Occlusion using Blobs, Proceedings of Advanced Computing, Networking and Informatics- Volume 1, Smart Innovation, Systems and Technologies, 27: 251-258.

Shih-Kuan Liao, Baug-Yu Liu,(2010); An edge-based approach to improve optical flow algorithm, Proceedings of Third International Conference on Advanced Computer Theory and Engineering, 6: 45-61.

Shizhong and Joydeep Ghosh (2001); A New formulation of Coupled Hidden Markov Models.

S. J. Blunsden and R. B. Fisher (2010); The BEHAVE video dataset: ground truthed video for multi-person behavior classification, Annals of the BMVA, 4: 1-12.

Teddy Ko (2010); A Survey on Behavior Analysis in Video Surveillance Applications. Proceedings of IEEE, Applied Imagery Pattern Recognition Workshop, 1-8.

Thomas Brox, Bodo Rosenhahn, Juergen Gall, and Daniel Cremers (2010); Combined Region and Motion-Based 3D Tracking of Rigid and Articulated Objects, IEEE Transactions on Pattern Analysis And Machine Intelligence, 32(3): 402-415. http://dx.doi.org/10.1109/TPAMI.2009.32

Weilun Lao, Jungong Han, and Peter H. N. deWith (2010); Flexible Human Behavior Analysis Framework for Video Surveillance Applications. International Journal of Digital Multimedia Broadcasting,ID: 920121, 1-9. http://dx.doi.org/10.1155/2010/920121

Weiyao Lin, Ming-Ting Sun, Radha Poovendran and Zhengyou Zhang (2010); Group Event Detection with a Varying Number of Group Members for Video Surveillance, IEEE Transactions on Circuits and Systems for Video Technology, 20(8): 1503.00082.

Weiming Hu, Guodong Tian, Xi Li, Stephen Maybank (2013); An Improved Hierarchical Dirichlet Process-Hidden Markov Model and Its Application to Trajectory Modeling and Retrieval, Int J Comput Vis, 105:246-268. http://dx.doi.org/10.1007/s11263-013-0638-8

Dong Zhang, Daniel Gatica-Perez, Samy Bengio, Iain McCowan, and Guillaume Lathoud (2004); Modeling Individual and Group Actions in Meetings: a Two-Layer HMM Framework, the Second IEEE Workshop on Event Mining: Detection and Recognition of Events in Video, In Association with CVPR, 1-8.

Gildas Morvan, Daniel Dupont,Jean-Baptiste Soyez, Rochdi Merzouki (2012); Engineering hierarchical complex systems: an agent-based approach, The case of flexible manufacturing systems, Chapter - Service Orientation in Holonic and Multi-Agent Manufacturing Control, series Studies in Computational Intelligence, 402: 49-60.

Cho, Sunyoung and Kwak, Sooyeong and Byun, Hyeran (2013); Recognizing Human-human Interaction Activities Using Visual and Textual Information, Pattern Recogn. Lett., 34(15):1840- 1848.

Manuel J. Marin-Jimenez, Enrique Yeguas, Nicolas Perez de la Blanca (2013); Exploring STIPbased models for recognizing human interactions in TV videos, Pattern Recognition Letters, 34: 1819 -1828.

Hejin Yuan (2015); A Semi-supervised Human Action Recognition Algorithm Based on Skeleton Feature, Journal of Information Hiding and Multimedia Signal Processing, 6(1): 175-181.



Most read articles by the same author(s)

Obs.: This plugin requires at least one statistics/report plugin to be enabled. If your statistics plugins provide more than one metric then please also select a main metric on the admin's site settings page and/or on the journal manager's settings pages.