Integrated System for Stereoscopic Cognitive Vision, Localization, Mapping, and Communication with a Mobile Service Robot


  • Cătălin Buiu POLITEHNICA University of Bucharest Department of Automatic Control and Systems Engineering Spl. Independentei 313, 060042 Bucharest, Romania


control, communication, localization, mapping, mobile robot, stereoscopic vision, virtual reality


This paper describes a stereo-vision-based mobile robot that can navigate and explore its environment autonomously and safely and simultaneously building a tridimensional virtual map of the environment. The control strategy is rule-based and the interaction with robot is done via Bluetooth. The stereoscopic vision allows the robot to recognize objects and to determine the distance to the analyzed objects. The robot is able to generate and simultaneously update a full colour 3D map of the environment that is being explored. The position and type of each detected and recognized object is marked in this 3D map. Furthermore, the robot will be able to use a gripper in order to collect detected objects and carry them to dedicated collecting bins, and so will be able to work in commercial waste cleanup applications. This application represents a successful integration of computers, control and communication techniques in mobile service robot control.


C. Buiu (Editor), Cognitive Robots (in Romanian), Editura Universitara, 2008.

C. M. Soria, R. Carelli, R., J. M. Ibarra Zannatha, Coordinated Control Of Mobile Robots Based On Artifcial Vision, International Journal of Computers, Communications, and Control, Vol. I (2006), No. 2, pp. 85-94.

K.D. Do, Bounded Controllers for Decentralized Formation Control of Mobile Robots with Limited Sensing, International Journal of Computers, Communications, and Control, Vol. II (2007), No. 4, pp. 340-354.

P. Elinas, R. Sim, J. J. Little, SigmaSLAM: Stereo vision SLAM using the Rao-Blackwellised particle filter and a novel mixture proposal distribution, In Proc. of the IEEE Int. Conf. on Robotics and Automation (ICRA), Florida, USA, 2006.

D. G. Lowe, Object recognition from local scale-invariant features, In Int. Conf. on Computer Vision, Corfu, Greece, September 1999, pp. 1150-1157.

R. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision, Cambridge, UK: Cambridge Univ. Pr., 2000.

D. Nister, O. Naroditsky, J. Bergen, Visual odometry, In Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2004), 2004, pp. 652-659.

M. Agrawal, K. Konolige, Rough terrain visual odometry, In Proceedings of the International Conference on Advanced Robotics (ICAR), August 2007.

K. Konolige, M. Agrawal, Frame-frame matching for realtime consistent visual mapping, In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), April 2007.

M. J. Procopio, T. Strohmann, A. R. Bates, G. Grudic, Jane Mulligan, Using Binary Classifiers to Augment Stereo Vision for Enhanced Autonomous Robot Navigation, University of Colorado at Boulder Technical Report CU-CS-1027-07, April 2007

Wang, L. K., S. Hsieh, E. C. Hsueh, F. Hsaio, K. Huang, Complete pose determination for low altitude unmanned aerial vehicle using stereo vision, In Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2005), pp. 108 - 113.

Lee, S., D. Jang, E. Kim, S. Hong, J. Han, A real-time 3D workspace modeling with stereo camera, In Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2005), pp. 2140 - 2147.

T. Chinapirom, U. Witkowski, R. Ulrich, Stereoscopic Camera for Autonomous Mini-Robots Applied in KheperaSot League, Research Report, University of Paderborn, Germania, 2007

D. R. Karuppiah, Z. Zhu, P. Shenoy, E. M. Riseman, A fault-tolerant distributed vision system architecture for object tracking in a smart room, In B. Schiele and G. Sagerer (Eds.), Springer Lecture Notes in Computer Science 2095, pp 201-219, 2007

S. Florczyk, Robot Vision: Video-based Indoor Exploration with Autonomous and Mobile Robots, Weinheim: Wiley-VCH, 2005.

M. F. Ahmed, Development of a Stereo Vision system for Outdoor Mobile Robots, M.S. thesis, University of Florida, 2006.

F. Rovira-Más, S. Han, J. Wei, J. F. Reid, Autonomous Guidance of a Corn Harvester using Stereo Vision, Agricultural Engineering International: the CIGR Ejournal, Manuscript ATOE 07 013, Vol. IX. July, 2007.

W. Lovegrove, B. Brame, Single-camera stereo vision for obstacle detection in mobile robots, In Intelligent Robots and Computer Vision XXV: Algorithms, Techniques, and Active Vision., Proceedings of the SPIE, Volume 6764, pp. 67640T, 2007

C. Buiu, F. Cazan, R. Ciurlea, Developing of a Service Robot to Recognize and SortWaste, In: 16th International Conference on Control Systems and Computer Science, pp. 298-303. POLITEHNICA Press, Bucharest, 2007.

Ana Pavel, C. Vasile, C. Buiu, Cognitive Vision System for an Ecological Mobile Robot, In Proceedings of SINTES 13, The International Symposium on System Theory, Automation, Robotics, Computers, Informatics, Electronics and Instrumentation, pp. 267-272, Universitaria Press, Craiova, 2007.

C. Buiu, Design and development of a waste cleanup service robot, In Proceedings of the First International EUROBOT Conference, Heidelberg, pp. 194-202, 2008.

A.G. Cohn, D. Magee, A. Galata, D. Hogg, S. Hazarika, Towards an architecture for cognitive vision using qualitative spatio-temporal representations and abduction, In C. Freksa, W. Brauer, C. Habel, K.F. Wender (editors), Spatial Cognition III, Routes and Navigation, Human Memory and Learning, Spatial Representation and Spatial Learning, pp. 232-248, Springer-Verlag, 2003.



Most read articles by the same author(s)

Obs.: This plugin requires at least one statistics/report plugin to be enabled. If your statistics plugins provide more than one metric then please also select a main metric on the admin's site settings page and/or on the journal manager's settings pages.