A Real-Time Tracking Algorithm for Human Following Mobile Robot using 3D Sensor
Main Article Content
Abstract
Detecting and tracking particular person are considered main tasks of mobile robot. In this paper, we propose a real-time mobile robot system using 3D Kinect sensor for automatically detecting, tracking and following human. This method is based on depth information, skeleton and color of human from 3D camera. Firstly, the depth image is taken from 3D Kinect to segment the individual region. After that, we calculate the body length, shoulder length and arm length in combination with color of target’s clothes to gather as the material for tracking task. Finally, the mobile robot which is controlled by voice command can recognize and follow the single target person. The effectiveness and robustness of the proposed method are evaluated comparing with method based on single skeleton or color of objective. Moreover, our proposed method can identify the target again when it disappears and appears again in the frame. All experiments are implemented with the support of model designed to integrate 3D Kinect camera on a wheeled mobile robot. The velocity and direction of wheeled mobile robot are controlled by a proportional-integral-derivative controller to keep a constant velocity all the time. The experiment result is shown that proposed system has worked effectively, stably and flexibly and the success rate is more than 90%.
Keywords
detection, 3D depth image, tracking, mobile robot, real time
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
References
[1] Jarret Web, James Ashley, Beginning Kinect Programming with the Microsoft Kinect SDK 1st Ed, Apress Publishing, USA, 2012
[2] Soroush Falahati, Open NI Cookbook, 1st Ed, Packt Publishing, USA, 2013
[3] Webb, Jarret, Ashley, James, Beginning Kinect Program with the Microsoft Kinect SDK, 1st Ed, Apress, USA, 2012. https://doi.org/10.1007/978-1-4302-4105-8_1
[4] Shiying Sin, Ning, An, Human recognize for following robot with a kinect sensor, ROBIO, Qingdao, China, 2016, pp.16709654
[5] Abdel-Mehsen Ahmad and Hiba, 3D Sensor-based moving human tracking robot with obstacle avoidance, IMCET, Beirut, Lebanon, 2016, pp.16523074
[6] Purvi Agarwal, Pranjal Gautam, Anmoal Agarwal, Vijai Singh, Human follower robot using kinect, IRJET, India, 2017, pp.2395-0072
[7] Doan Thi Huong Giang, Vu Hai, Tran Thi Thanh Hai, Utilizing depth image from kinect sensor: error analysis and its applications, FAIR, Thai Nguyen, Vietnam, 2014
[8] Cheng-An Yang and Kai-Tai Song, Control design for robotic human-following and obstacle avoidance using an RGB-D camera, ICCAS, Jeju, Korea, 2020, pp.19301946. https://doi.org/10.23919/ICCAS47443.2019.8971754
[9] Zhang, Huang, Rui Jun Yan, Wen Shen Zhou, Long Sheng, Binocular vision sensor (kinect)-based pedestrian following mobile robot, Applied Mechanics and Materials, Trans Tech Publications, Ltd., October 2014. https://doi.org/10.4028/www.scientific.net/AMM.670-671.1326
[10] Shih, Ching-Long, Chao-Cheng Li, A people-following mobile robot using kinect and a laser scanner, Robot Autom Eng J (2018) pp. 1-8. https://doi.org/10.19080/RAEJ.2018.02.555578
[11] Armando Nava, Leonardo Garrido and Ramon F. Brena, Recognizing activities using a kinect skeleton tracking and hidden markov models, MICAI, Tuxtla Gutierrez, Mexico, 2015, pp.15413297. https://doi.org/10.1109/MICAI.2014.18
[12] José-Juan Hernández-López, Ana-Linnet Quintanilla-Olvera, José-Luis López-Ramírez, Francisco-Javier Rangel-Butanda, Mario-Alberto Ibarra-Manzano, Dora-Luz Almanza-Ojeda, Detecting objects using color and depth segmentation with Kinect sensor, Procedia Technology, Vol 3, pp.196-204, 2012. https://doi.org/10.1016/j.protcy.2012.03.021
[2] Soroush Falahati, Open NI Cookbook, 1st Ed, Packt Publishing, USA, 2013
[3] Webb, Jarret, Ashley, James, Beginning Kinect Program with the Microsoft Kinect SDK, 1st Ed, Apress, USA, 2012. https://doi.org/10.1007/978-1-4302-4105-8_1
[4] Shiying Sin, Ning, An, Human recognize for following robot with a kinect sensor, ROBIO, Qingdao, China, 2016, pp.16709654
[5] Abdel-Mehsen Ahmad and Hiba, 3D Sensor-based moving human tracking robot with obstacle avoidance, IMCET, Beirut, Lebanon, 2016, pp.16523074
[6] Purvi Agarwal, Pranjal Gautam, Anmoal Agarwal, Vijai Singh, Human follower robot using kinect, IRJET, India, 2017, pp.2395-0072
[7] Doan Thi Huong Giang, Vu Hai, Tran Thi Thanh Hai, Utilizing depth image from kinect sensor: error analysis and its applications, FAIR, Thai Nguyen, Vietnam, 2014
[8] Cheng-An Yang and Kai-Tai Song, Control design for robotic human-following and obstacle avoidance using an RGB-D camera, ICCAS, Jeju, Korea, 2020, pp.19301946. https://doi.org/10.23919/ICCAS47443.2019.8971754
[9] Zhang, Huang, Rui Jun Yan, Wen Shen Zhou, Long Sheng, Binocular vision sensor (kinect)-based pedestrian following mobile robot, Applied Mechanics and Materials, Trans Tech Publications, Ltd., October 2014. https://doi.org/10.4028/www.scientific.net/AMM.670-671.1326
[10] Shih, Ching-Long, Chao-Cheng Li, A people-following mobile robot using kinect and a laser scanner, Robot Autom Eng J (2018) pp. 1-8. https://doi.org/10.19080/RAEJ.2018.02.555578
[11] Armando Nava, Leonardo Garrido and Ramon F. Brena, Recognizing activities using a kinect skeleton tracking and hidden markov models, MICAI, Tuxtla Gutierrez, Mexico, 2015, pp.15413297. https://doi.org/10.1109/MICAI.2014.18
[12] José-Juan Hernández-López, Ana-Linnet Quintanilla-Olvera, José-Luis López-Ramírez, Francisco-Javier Rangel-Butanda, Mario-Alberto Ibarra-Manzano, Dora-Luz Almanza-Ojeda, Detecting objects using color and depth segmentation with Kinect sensor, Procedia Technology, Vol 3, pp.196-204, 2012. https://doi.org/10.1016/j.protcy.2012.03.021