A Hybrid Model for Concurrent Interaction Recognition from Videos

M. Sivarathinabala, S. Abirami


Human behavior analysis plays an important role in understanding the high-level human activities from surveillance videos. Human behavior has been identified using gestures, postures, actions, interactions and multiple activities of humans. This paper has been analyzed by identifying concurrent interactions, that takes place between multiple peoples. In order to capture the concurrency, a hybrid model has been designed with the combination of Layered Hidden Markov Model (LHMM) and Coupled HMM (CHMM). The model has three layers called as pose layer, action layer and interaction layer, in which pose and action of the single person has been defined in the layered model and the interaction of two persons or multiple persons are defined using CHMM. This hybrid model reduces the training parameters and the temporal correlations over the frames are maintained. The spatial and temporal information are extracted and from the body part attributes, the simple human actions as well as concurrent actions/interactions are predicted. In addition, we further evaluated the results on various datasets also, for analyzing the concurrent interaction between the peoples.


Pose prediction, interaction recognition, layered HMM

Full Text:



Alexandros Andre Chaaraoui, Pau Climent-Perez, Francisco Florez-Revuelta(2013); Silhouette-based human action recognition using sequences of key poses, Pattern Recognition Letters, 34(15): 1799-1807.

Aggarwal, J. K.and Ryoo, M. S. (2011); Human activity analysis: A review, ACM Computing Survey, 43(3): 16:1–16:43.

Arnold Wiliem,Vamsi Madasu, Wageeh Boles and Prasad Yarlagadda (2012); A suspicious behaviour detection using a context space model for smart surveillance systems, computer vision and Image Understanding, 116(2): 194-209.

Fadime sener and Nazli Ikizler-cinbis (2015); Two Person Interaction Recognition via spatial Multiple Instance Embedding, Journal of Visual Communication and Image Representation, 32: 63-73.

Gowsikhaa.D, Abirami.S and Baskaran.R. (2012); Automated human behavior analysis from surveillance videos: a survey, Artificial Intelligence Review, 1-19.

Gowsikhaa.D, Manjunath and Abirami S. (2012); Suspicious Human activity detection from Surveillance videos, International Journal on Internet and Distributed Computing Systems, 2(2): 141-149.

Junji Yamato, Jun Ohya and Kenichiro Ishii (1992); Recognizing Human Action in Time- Sequential Images using Hidden Markov Model, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 379- 385.

Matthew Brand, Nuria Oliver, and Alex Pentland (1997); Coupled Hidden Markov Models for Complex Activity Recognition, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 994 - 999.

Nuria Oliver, Ashutosh Garg and Eric Horvitz (2004); Layered Representations for learning and inferring office activity from multiple sensor channels,Computer Vision and Image Understanding, 96: 163-180.

Roberto Melfi, Shripad Kondra and Alfredo Petrosino (2013); Human activity modeling by spatio temporal textural appearance, Pattern Recognition Letters,34(15): 1990-1994.

Ryoo M.S. (2011); Human Activity Prediction: Early Recognition of Ongoing Activities from Streaming Videos, Proceedings of IEEE International Conference on Computer Vision (ICCV), 1036-1043.

Ryoo,M.S, and Aggarwal, J.K. (2010); UT Interaction Dataset, Proc. of ICPR Contest on Semantic Description of Human activities, http://cvrc.ece.utexas.edu/SDHA2010/HumanI nteraction.html.

Sangho Park and J.K. Aggarwal (2004); Semantic-level Understanding of Human Actions and Interactions using Event Hierarchy, Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop, 1-12.

Sang Min Yoon, Arjan Kuijper (2013); Human action recognition based on skeleton splitting, Expert systems with Applications, 40(17): 6848-6855.

Sivarathinabala M. and Abirami S. (2014); Motion Tracking of Humans under Occlusion using Blobs, Proceedings of Advanced Computing, Networking and Informatics- Volume 1, Smart Innovation, Systems and Technologies, 27: 251-258.

Shih-Kuan Liao, Baug-Yu Liu,(2010); An edge-based approach to improve optical flow algorithm, Proceedings of Third International Conference on Advanced Computer Theory and Engineering, 6: 45-61.

Shizhong and Joydeep Ghosh (2001); A New formulation of Coupled Hidden Markov Models.

S. J. Blunsden and R. B. Fisher (2010); The BEHAVE video dataset: ground truthed video for multi-person behavior classification, Annals of the BMVA, 4: 1-12.

Teddy Ko (2010); A Survey on Behavior Analysis in Video Surveillance Applications. Proceedings of IEEE, Applied Imagery Pattern Recognition Workshop, 1-8.

Thomas Brox, Bodo Rosenhahn, Juergen Gall, and Daniel Cremers (2010); Combined Region and Motion-Based 3D Tracking of Rigid and Articulated Objects, IEEE Transactions on Pattern Analysis And Machine Intelligence, 32(3): 402-415.

Weilun Lao, Jungong Han, and Peter H. N. deWith (2010); Flexible Human Behavior Analysis Framework for Video Surveillance Applications. International Journal of Digital Multimedia Broadcasting,ID: 920121, 1-9.

Weiyao Lin, Ming-Ting Sun, Radha Poovendran and Zhengyou Zhang (2010); Group Event Detection with a Varying Number of Group Members for Video Surveillance, IEEE Transactions on Circuits and Systems for Video Technology, 20(8): 1503.00082.

Weiming Hu, Guodong Tian, Xi Li, Stephen Maybank (2013); An Improved Hierarchical Dirichlet Process-Hidden Markov Model and Its Application to Trajectory Modeling and Retrieval, Int J Comput Vis, 105:246-268.

Dong Zhang, Daniel Gatica-Perez, Samy Bengio, Iain McCowan, and Guillaume Lathoud (2004); Modeling Individual and Group Actions in Meetings: a Two-Layer HMM Framework, the Second IEEE Workshop on Event Mining: Detection and Recognition of Events in Video, In Association with CVPR, 1-8.

Gildas Morvan, Daniel Dupont,Jean-Baptiste Soyez, Rochdi Merzouki (2012); Engineering hierarchical complex systems: an agent-based approach, The case of flexible manufacturing systems, Chapter - Service Orientation in Holonic and Multi-Agent Manufacturing Control, series Studies in Computational Intelligence, 402: 49-60.

Cho, Sunyoung and Kwak, Sooyeong and Byun, Hyeran (2013); Recognizing Human-human Interaction Activities Using Visual and Textual Information, Pattern Recogn. Lett., 34(15):1840- 1848.

Manuel J. Marin-Jimenez, Enrique Yeguas, Nicolas Perez de la Blanca (2013); Exploring STIPbased models for recognizing human interactions in TV videos, Pattern Recognition Letters, 34: 1819 -1828.

Hejin Yuan (2015); A Semi-supervised Human Action Recognition Algorithm Based on Skeleton Feature, Journal of Information Hiding and Multimedia Signal Processing, 6(1): 175-181.

DOI: https://doi.org/10.15837/ijccc.2016.4.2524

Copyright (c) 2017 M. Sivarathinabala, S. Abirami

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

IJCCC is an Open Access Journal : CC-BY-NC.

Articles published in IJCCC user license are protected by copyright.

Users can access, download, copy, translate the IJCCC articles for non-commercial purposes provided that users, but cannot redistribute, display or adapt:

  • Cite the article using an appropriate bibliographic citation: author(s), article title, journal, volume, issue, page numbers, year of publication, DOI, and the link to the definitive published version on IJCCC website;
  • Maintain the integrity of the IJCCC article;
  • Retain the copyright notices and links to these terms and conditions so it is clear to other users what can and what cannot be done with the  article;
  • Ensure that, for any content in the IJCCC article that is identified as belonging to a third party, any re-use complies with the copyright policies of that third party;
  • Any translations must prominently display the statement: "This is an unofficial translation of an article that appeared in IJCCC. Agora University  has not endorsed this translation."

This is a non commercial license where the use of published articles for commercial purposes is forbiden. 

Commercial purposes include: 

  • Copying or downloading IJCCC articles, or linking to such postings, for further redistribution, sale or licensing, for a fee;
  • Copying, downloading or posting by a site or service that incorporates advertising with such content;
  • The inclusion or incorporation of article content in other works or services (other than normal quotations with an appropriate citation) that is then available for sale or licensing, for a fee;
  • Use of IJCCC articles or article content (other than normal quotations with appropriate citation) by for-profit organizations for promotional purposes, whether for a fee or otherwise;
  • Use for the purposes of monetary reward by means of sale, resale, license, loan, transfer or other form of commercial exploitation;

    The licensor cannot revoke these freedoms as long as you follow the license terms.

[End of CC-BY-NC  License for Website User]

INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL (IJCCC), With Emphasis on the Integration of Three Technologies (C & C & C),  ISSN 1841-9836.

IJCCC was founded in 2006,  at Agora University, by  Ioan DZITAC (Editor-in-Chief),  Florin Gheorghe FILIP (Editor-in-Chief), and  Misu-Jan MANOLESCU (Managing Editor).

Ethics: This journal is a member of, and subscribes to the principles of, the Committee on Publication Ethics (COPE).

Ioan  DZITAC (Editor-in-Chief) at COPE European Seminar, Bruxelles, 2015:

IJCCC is covered/indexed/abstracted in Science Citation Index Expanded (since vol.1(S),  2006); JCR2018: IF=1.585..

IJCCC is indexed in Scopus from 2008 (CiteScore2018 = 1.56):

Nomination by Elsevier for Journal Excellence Award Romania 2015 (SNIP2014 = 1.029): Elsevier/ Scopus

IJCCC was nominated by Elsevier for Journal Excellence Award - "Scopus Awards Romania 2015" (SNIP2014 = 1.029).

IJCCC is in Top 3 of 157 Romanian journals indexed by Scopus (in all fields) and No.1 in Computer Science field by Elsevier/ Scopus.


 Impact Factor in JCR2018 (Clarivate Analytics/SCI Expanded/ISI Web of Science): IF=1.585 (Q3). Scopus: CiteScore2018=1.56 (Q2);

SCImago Journal & Country Rank

Editors-in-Chief: Ioan DZITAC & Florin Gheorghe FILIP.