Gaze, Posture and Gesture Recognition to Minimize Focus Shifts for Intelligent Operating Rooms in a Collaborative Support System
Keywords:
posture recognition, behavior analysis, intelligent roomsAbstract
This paper describes the design of intelligent, collaborative operating rooms based on highly intuitive, natural and multimodal interaction. Intelligent operating rooms minimize surgeon’s focus shifts by minimizing both the focus spatial offset (distance moved by surgeon’s head or gaze to the new target) and the movement spatial offset (distance surgeon covers physically). These spatio-temporal measures have an impact on the surgeon’s performance in the operating room. I describe how machine vision techniques are used to extract spatio-temporal measures and to interact with the system, and how computer graphics techniques can be used to display visual medical information effectively and rapidly. Design considerations are discussed and examples showing the feasibility of the different approaches are presented.References
T.R. Hansen ,Focus Shift Analysis in the operating theatre,. 2004. Source:htt p : ==www:pervasive- interaction:org=publications=Focus_shi f t:pd f .
J. Graydon, M. Eysenck, Distraction and cognitive performance, Eur J Cogn Psychol.;1(2):161-179, 1989. http://dx.doi.org/10.1080/09541448908403078
V. Weerdesteyn, A. Schillings, G. Van Galen, D. Duysens, Distraction affects the performance of obstacle avoidance during walking, J Motor Behavior. 35(1):53-63, 2003. http://dx.doi.org/10.1080/00222890309602121
K. H. Goodell, C. G. Cao, S. D. Schwaitzberg, Effects of cognitive distraction on performance of laparoscopic surgical tasks, J Laparoendosc Adv Surg Tech A. Apr;16(2):94, 2006.
R. M. Satava, Disruptive visions: The operating room of the future. Surgical endoscopy, vol. 17, no1, pp. 104-107, 2003. http://dx.doi.org/10.1007/s00464-002-0018-x
J. V. Sutherland, W. J. Van den Heuvel, T. Ganous, M. M. Burton, A. Kumar, Towards an Intelligent Hospital Environment: Adaptive Workflow in the OR of the Future, Stud Health Technol. Inform.118:278-312, 2005.
P. Wellner,The DigitalDesk Calculator: Tangible Manipulation on a Desk Top Display, Proceedings of UIST'91. pp.27-33. 1991. http://dx.doi.org/10.1145/120782.120785
W. Newman, P. Wellner, A Desk Supporting Computer based interaction with paper, Proceedings of the Conference on Human Factors in Computing Systems (CHI'92). p587- 592. 1992. http://dx.doi.org/10.1145/142750.143007
Microsoft Surface. Available online: www.microsoft.com/surface/
R. A. Brooks, The Intelligent Room Project, Proceedings of the Second International Cognitive Technology Conference (CT'97), Aizu, Japan, August 1997.
K. S. Huang and M. M. Trivedi, Video Arrays for Real-Time Tracking of Persons, Head and Face in an Intelligent Room, Machine Vision and Applications, Special Issue, Jun. 2003.
M. Nicolescu and G. Medioni, Electronic pan-tilt-zoom: a solution for intelligent room systems, Proc. ICME'2000, 2000. http://dx.doi.org/10.1109/icme.2000.871071
J. C. Terrillon, A. Piplr, Y. Niwa, K. Yamamoto, Robust Face Detection and Japanese Sign Language Hand Posture Recognition for Human Human-Computer Interaction in an "Intelligent" Room, In Proc. Int'l Conf. Vision Interface, 2002.
N. Dimakis, J. K. Soldatos, L. Polymenakos, P. Fleury, D. Curin, Jan, J. Kleindienst, Integrated development of context-aware applications in smart spaces, Pervasive Computing, IEEE 7 (4), 71- 79, 2008. http://dx.doi.org/10.1109/MPRV.2008.75
T Skordas, G Metakides, Major Challenges in Ambient Intelligence, Studies in Informatics and Control, Vol.12, No.2, June 2003 Page 3. 2.3.
J. P. Tardif, S. Roy, M. Trudeau, Multi-projectors for arbitrary surfaces without explicit calibration nor reconstruction, In Proceedings of Fourth International Conference on 3-D Digital Imaging and Modeling. 3DIM 2003.. pp. 217-224. 2003.
A. Griesser and L. V. Gool, Automatic Interactive Calibration of Multi-Projector-Camera Systems, Proceedings of the 2006 Conference on Computer Vision and Pattern Recognition Workshop, 8, 2006. http://dx.doi.org/10.1109/CVPRW.2006.37
I. Mitsugami, N. Ukita, M. Kidode,.'Displaying a Moving Image By Multiple Steerable Projectors', IEEE International Workshop on Projector-Camera Systems (Procams2007), 2007. http://dx.doi.org/10.1109/cvpr.2007.383472
Z. Jiang, Y. Mao, B. Qin and B. Zang, A High Resolution Video Display System by Seamlessly Tiling Multiple Projectors, IEEE International Conference on Multimedia and Expo, 2007. http://dx.doi.org/10.1109/icme.2007.4285089
E. Bhasker, R. Juang, A. Majumder, Registration Techniques for Using Imperfect and Par tially Calibrated Devices in Planar Multi-Projector Displays, IEEE Transactions on Visualization and Computer Graphics, 13:6, 1368-1375, 2007. http://dx.doi.org/10.1109/TVCG.2007.70586
J. Summet, M. Flagg, TJ. Cham, JM. Rehg and R. Sukthankar, Shadow Elimination and Blinding Light Suppression for Interactive Projected Displays,IEEE Transactions on Visualization and Computer Graphics, 13:3,508-517, 2007. http://dx.doi.org/10.1109/TVCG.2007.1007
T. Johnson, G.Welch, H. Fuchs, E. La Force, H. Towles, A Distributed Cooperative Framework for Continuous Multi-Projector Pose Estimation in Proceedings IEEE VR 2009.
A. Nishikawa, T. Hosoi, K. Koara, D. Negoro, A. Hikita, S. Asano, H. Kakutani, F. Miyazaki, M. Sekimoto, M. Yasui, Y. Miyake, S. Takiguchi, and M. Monden. FAce MOUSe: A Novel Human- Machine Interface for Controlling the Position of a Laparoscope, IEEE Trans. on Robotics and Automation, 19:5, 825-841, 2003. http://dx.doi.org/10.1109/TRA.2003.817093
C. Graetzel, T.W. Fong, S. Grange, and C. Baur, A non-contact mouse for surgeon-computer interaction, Technology and Health Care, 12:3, 245-257, 2004.
J. Wachs, H. Stern, Y. Edan, M. Gillam, C. Feied, M. Smith, and J. Handler, A hand gesture sterile tool for browsing MRI images in the OR, Journal of the American Medical Informatics Association. 15(3), 2008. http://dx.doi.org/10.1197/jamia.M2410
P. Lukowicz, A. Timm-Giel, H. Lawo, O. Herzog, WearITwork: Toward Real-World Industrial Wearable Computing, IEEE Pervasive Computing. 6:4,8-13, 2007. http://dx.doi.org/10.1109/MPRV.2007.89
Y. Kuno, T. Murashima, N. Shimada, and Y. Shirai, Intelligent Wheelchair Remotely Controlled by Interactive Gestures, In Proceedings of 15th Intl. Conf. on Pattern Recognition, 4,672-675, 2000. http://dx.doi.org/10.1109/icpr.2000.903007
T. Starner, J. Auxier, D. Ashbrook, and M. Gandy. The Gesture Pendant: A Self-illuminating, Wearable, Infrared Computer Vision System for Home Automation Control and Medical Monitoring, In 4th Intl. Symposium on Wearable Computers, 87-94, 2000. http://dx.doi.org/10.1109/iswc.2000.888469
D. A. Becker and A. Pentland, Staying Alive: A Virtual Reality Visualization Tool for Cancer Patients, Proc. of the AAAI'96 Workshop on Entertainment and Alife/AI, 1996.
R. Boian, R. Sharma, C. Han, A. Merians, G Burdea, S. Adamovich, M. Recce, M. Tremaine, H. Poizner, Virtual reality-based post-stroke hand rehabilitation,Studies in Health and Technology Information, 85, 64-70, 2002.
M. Ashdown and Y. Sato, Steerable projector calibration, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), 2005. http://dx.doi.org/10.1109/cvpr.2005.533
E. Moschos, R. Coleman, Acquiring laparoscopic skill proficiency: Does orientation matter?, American Journal of Obstetrics and Gynecology, 191:5, 1782-1787, 2004. http://dx.doi.org/10.1016/j.ajog.2004.07.073
R. Yang and G. Welch.,Automatic and Continuous Projector Display Surface Estimation Using Every-Day Imagery, 9th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision, 2001.
K. Murphy, A. Torralba, D. Eaton,W. T. Freeman,Object detection and localization using local and global features, Lecture Notes in Computer Science. Sicily workshop on object recognition, 2005.
J. Wachs, D. Goshorn and M. Kölsch, Recognizing Human Postures and Poses in Monocular Still Images, In Proceeding of the International Conference on Image Processing, Computer Vision, and Signal Processing. (IPCV09) Las Vegas, Nevada. July 2009.
J.Wachs, H. Stern, Y. Edan, M. Gillam, C. Feied, M. Smith, and J. Handler, A Real-Time Hand Gesture Interface for a Medical Image Guided System, International Journal of Intelligent Computing in Medical Sciences and Image Processing, 1:3:1, 175-185, 2008.
G. R. Bradski, Computer vision face tracking for use in a perceptual user interface, Intel Technical Journal, 1-15, 1998.
S. T. Shivappa, M. M. Trivedi, and B. D. Rao, Hierarchical audio-visual cue integration framework for activity analysis in intelligent meeting rooms, IEEE CVPR 2009, 2009.
M. Coen, Design principles for intelligent environments, In Proc. of AAAI, Madison, Wisconsin, 1998.
Published
Issue
Section
License
ONLINE OPEN ACCES: Acces to full text of each article and each issue are allowed for free in respect of Attribution-NonCommercial 4.0 International (CC BY-NC 4.0.
You are free to:
-Share: copy and redistribute the material in any medium or format;
-Adapt: remix, transform, and build upon the material.
The licensor cannot revoke these freedoms as long as you follow the license terms.
DISCLAIMER: The author(s) of each article appearing in International Journal of Computers Communications & Control is/are solely responsible for the content thereof; the publication of an article shall not constitute or be deemed to constitute any representation by the Editors or Agora University Press that the data presented therein are original, correct or sufficient to support the conclusions reached or that the experiment design or methodology is adequate.
