An Approach for Construction of Augmented Reality Systems using Natural Markers and Mobile Sensors in Industrial Fields

  • Daniel Lima Gomes Jr Federal Institute of Maranhão
  • Paulo Roberto Jansen dos Reis Federal University of Maranhão
  • Anselmo Cardoso de Paiva Federal University of Maranhão
  • Aristófanes Corrêa Silva Federal University of Maranhão
  • Geraldo Braz Jr Federal University of Maranhão
  • Marcelo Gattass Tecgraf Institute / PUC-Rio
  • Antônio Sérgio de Araújo


This paper presents a methodology for the development of augmented reality (AR) visualization applications in industrial scenarios. The proposal presents the use of georreferenced natural markers detected in real time, which enables the construction of AR systems. This use of augmented visualization allows the creation of tools that can aid on-site maintenance activities for operators. AR use makes possible including information about the equipment during a specific procedure. In this work, the detection of natural markers in the scene are based on Haar-like features associated with equipment geolocalization. This approach enable the detection of equipment in multiple user’s viewpoints in the industrial scenario and makes it possible the inclusion of real information about those equipment in real time as AR annotations. In this way, beyond a methodology approach, this paper presents a new way for Power System information visualization in the field that can be used in both for training and for control operations.

Author Biographies

Daniel Lima Gomes Jr, Federal Institute of Maranhão
Received his M.Sc. in Electrical Engineering (2010) from Federal University of Maranhão (UFMA). Now he is PhD student of informatics at UFMA, Brazil.Since 2010 he is a professor of Federal Technology Institute of Maranhão (IFMA), Brazil. His current research interests include Image Processing, Virtual and Augmented Reality and Computer Vision
Paulo Roberto Jansen dos Reis, Federal University of Maranhão
Received a M.Sc. degree in Computer Science (2014) by theFederal University of Maranhao (UFMA). Currently he is a researcher of Applied Computing Center (NCA) and his research interests are Virtual and Augmented Reality with emphasis on collaborative industrial applications.
Anselmo Cardoso de Paiva, Federal University of Maranhão
Received a PhD degree in Informatics from Pontiphical CatholicUniversity of Rio de Janeiro - Brazil in 2001. He is founder member of the Applied Computer Center at Federal University of Maranhao (UFMA), where he is currently a Professor. His research interests include GIS, analysis and processing of medical images, Virtual and Augmented Reality.
Aristófanes Corrêa Silva, Federal University of Maranhão
Received a PhD degree in Informatics from Pontiphical CatholicUniversity of Rio de Janeiro - Brazil in 2004. Currently he is a Professor at the Federal University of Maranhão (UFMA), Brazil. He teaches image processing, pattern recognition and programming language. His research interests include image processing, image understanding, medical image processing, machine vision, artificial intelligence, pattern recognition and, machine learning.
Geraldo Braz Jr, Federal University of Maranhão
Received PhD degree in Electrical Engineering (2014) from FederalUniversity of Maranhao (UFMA), Brazil. Currently he is professor at the Federal University of Maranhao. He teaches Computer Vision, programming language, Software Engineering. His research interests include Computer Vision, Image Processing, Machine Learning and Augmented Reality.
Marcelo Gattass, Tecgraf Institute / PUC-Rio
Director of the Tecgraf Institute at Pontifical Catholic University of Riode Janeiro (Brazil) since 1987. He received his PhD degree in Computer Graphics & Civil Engineering (1982) from Cornell University, USA. His research interests include Augmented Reality and Scientific Visualization.
Antônio Sérgio de Araújo
Supervisor of the Regional Operation Center from East System in Hydroeletric Company of Sao Francisco (CHESF) in Recife, Brazil. He received his bachelor degree in Organizational Psicology at UNICAP (1993). Post-graduate in SRH at FAFIRE (1990),Eletrotechnical Univerisity Extension at UFPE (2000), Specialization in Operation Security Analysis in Power Systems at Fundação COGE (2004) and Eletric Systems Automation at FUPAI (2006).


[1] Antonijevic M.; Sucic S.; Keserica H. (2016); Augmented Reality for Substation Automation by Utilizing IEC 61850 Communication, 39th International Convention on Information and Communication Technology, Electronics and Microelectronics - Distributed Computing, Visualization and Biomedical Engineering, Opatija, Croatia, 2016.

[2] Aukstakalnis S.; Blatner D. (1992); Silicon Mirage - The Art and Science of Virtual Reality, Peachpit Press, ISBN: 978-0938151821, 317, 1992.

[3] Azuma R.T. (1997); A Survey of Augmented Reality, Presence: Teleoperators and Virtual Environments, 355-385, 1997.

[4] Bay H.; Ess A.; Tuytelaars T.; Gool L. V. (2008); Speeded-up Robust Features (SURF), Computer Vision and Image Understanding, 110(3), 346-359, 2008.

[5] CEPEL (2004); Open Energy Management System (Sistema Aberto de Gerenciamento de Energia), [Online] Avaiable:

[6] Côté S.; Barnard J.; Snyder R. (2013); Offline Spatial Panoramic Video Augmentation for Visual Communication in the AEC Industry, 13th International Conference on Construction Applications of Virtual Reality (CONVR 2013), 82-89, 2013.

[7] Dai J.; Zhang L. (2011); Registered Localization Method-Based on OpenCV for Augmented Reality, International Conference of Soft Computing and Pattern Recognition (SoCPaR), 492-495, 2011.

[8] Farag H. E. Z.; El-Saadany E.F. (2012); A Novel Cooperative Protocol for Distributed Voltage Control in Active Distribution Systems, IEEE Transactions on Power Systems, 1645-1656, 2012.

[9] Google (2016); Google Cardboard, [Online] Available at

[10] GPS (2014); GPS Accuracy, Official U.S. Government information about the Global Positioning System, [Online] Available:

[11] Jian Y.; Xin W.; Xue Z.; Zhenyou D. (2015); Cloud Computing and Visual Attention Based Object Detection for Power Substation Surveillance Robots, IEEE 28th Canadian Conference on Electrical and Computer Engineering, 337-342, 2015.

[12] Kim J.; Jun H. (2011); Implementation of Image Processing and Augmented Reality Programs for Smart Mobile Device, The 6th International Forum on Strategic Technology, 1070- 1073, 2011.

[13] Li Z.; Wang J.; Sun H.; Guo Q. (2015); Transmission Contingency Analysis Based on Integrated Transmission and Distribution Power Flow in Smart Grid, IEEE Transactions on Power Systems, 3356-3367, 2015.

[14] Liang M.; Min H.; Luo R.; Zhu J. (2015); Simultaneous Recognition and Modeling for Learning 3-D Object Models From Everyday Scenes, IEEE Transactions on Cybernetics, 45(10), 2237-2248, 2015.

[15] Lienhart, R.; Maydt, J. (2002); An Extended Set of Haar-like Features for Rapid Object Detection, IEEE International Conference on Image Processing (ICIP), 900-903, 2002.

[16] Lin H.; Sambamoorthy S.; Shukla S.; Thorp, J.; Mili L. (2011); Power System and Communication Network Co-Simulation for Smart Grid Applications, Innovative Smart Grid Technologies (ISGT), 1-6, 2011.

[17] Lowe D.G. (2004); Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, 60(2), 91-110, 2004.

[18] Mahadev P. M.; Christie R. D. (1993); Envisioning Power System Data: Concepts and a Prototype System State Rep-resentation, IEEE Transactions on Power Systems, 8(3), 1084-1090, 1993.

[19] Mahadev P. M.; Christie R. D. (1994); Envisioning power system data: vulnerability and severity representations for static security assessment, IEEE Transactions on Power Systems, 9(4), 1915-1920, 1994.

[20] Mikkelsen C.; Johansson J.; Cooper M. (2012); Visualization of Power System Data on Situation Overview Displays, 16th International Conference on Information Visualisation, 188-197, 2012.

[21] Milgram P.; Kishino F. (1994); A Taxonomy of Mixed Reality Visual Displays, IEICE Trans. Information and Systems, 1321-1329, 1994.

[22] Nigam S.; Deb K.; Khare A. (2013); Moment Invariants based Object Recognition for Different Pose and Appearances in Real Scenes, International Conference on Informatics, Electronics & Vision (ICIEV), ISBN: 978-1-4799-0397-9, 1-5, 2013.

[23] Overbye T. J.; Weber J. D. (2000); New Methods for the Visualization of Electric Power System Information, IEEE Symposium on Information Visualization, ISSN: 1522-404X, 1-9, 2000.

[24] Overbye T. J.; Weber J. D. (2000); Visualization of power system data, Proceedings of the 33rd Annual Hawaii International Conference on System Sciences, 1-7, 2000.

[25] Pribeanu C. (2012); Specification and Validation of a Formative Index to Evaluate the Ergonomic Quality of an AR-based Educational Platform, International Journal of Computers Communications & Control, 7(4), 721-732, 2012.

[26] Prochazk D.; Stencl M.; Popelka O.; Stastny J. (2011); Mobile Augmented Reality Applications, Proceedings of Mendel 2011: 17th International Conference on Soft Computing, 469-476, 2011.

[27] Ribeiro T. R.; Reis P. R. J.; Braz Jr., G.; Paiva A. C.; Silva A. C.; Maia I. M. O.; Araújo A. S. (2014); Agito: Virtual reality environment for power systems substations operators training, International Conference on Augmented and Virtual Reality (AVR), 113-123, 2014.

[28] Rublee E.; Rabaud, V.; Konolige K.; Bradski, G. (2011); ORB: an efficient alternative to SIFT or SURF, IEEE International Conference on Computer Vision, 2564-2571, 2011.

[29] Schwegmann C.P.; Kleynhans W.; Salmon B.P. (2014); Ship detection in South African oceans using SAR, CFAR, and a Haar-like feature classifier, Geoscience and Remote Sensing Symposium (IGARSS), 557-570, 2014.

[30] Szeliski R. (2011); Computer Vision: Algorithms and Applications, Springer-Verlag London, ISBN: 978-1-84882-935-0, 812, 2011.

[31] Vargas H.; Farias G.; Sanchez J.; Dormido S.; Esquembre F. (2013); Using Augmented Reality in Remote Laboratories, International Journal of Computers Communications & Control, 8(4), 622-634, 2013.

[32] Viola P.; Jones M. J. (2001); Rapid Object Detection using a Boosted Cascade of Simple Features, Computer Vision and Pattern Recognition (CVPR), 511-518, 2001.

[33] Weber J. D.; Overbye T. J. (2000); Voltage Contours for Power System Visualization, IEEE Transactions on Power Systems, 15(1), 404-409, 2000.
How to Cite
GOMES JR, Daniel Lima et al. An Approach for Construction of Augmented Reality Systems using Natural Markers and Mobile Sensors in Industrial Fields. INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL, [S.l.], v. 12, n. 4, p. 507-518, june 2017. ISSN 1841-9844. Available at: <>. Date accessed: 05 july 2020. doi:


Augmented Reality, Data Visualization, Natural Markers