Towards A High-Performance and Causal Stabilization System for Video Captured from Moving Camera
Main Article Content
Abstract
Video shot from camera attached to moving devices like smartphone, and drone are often shaken because unwanted movements of the image sensors, which are caused by unstable motions of the devices during their operation (e.g. moving, fly). This phenomenon impacts on effectiveness of systems that use camera videos as input data such as security surveillance and object tracking. In this paper, we propose a novel software-based system to stabilize camera videos in real-time by combining several general models. The main contribution of proposed system is the capability of processing instantaneously video achieved from moving devices to meet quality requirements by using Harris with Optical-flow, and Lucas-Kanade methods for motion estimation. We also propose several mechanisms including frame partition and matching for corner detector when applying Harris method to ensure processing quality and system performance. In our system, we also use Kalman filter for prediction model of motion compensation. Our experiments proved that the average processing speed of our system can reach 35 fps, which satisfies the real-time requirement.
Keywords
Causal system, motion prediction, performance, real-time, video stabilization
Article Details
References
[1] Vermeulen, Eddy, Real-time video stabilization for moving platforms, 21st Bristol UAV Systems Conference, 2007.
[2] Shen, Guturu, et al., Video stabilization using principal component analysis and scale invariant feature transform in particle filter framework, IEEE Transactions on Consumer Electronics 55.3 (2009) 1714-1721.
[3] Lim, Anli, et al., Real-time optical flow-based video stabilization for unmanned aerial vehicles, Journal of Real-Time Image Processing (2017) 1-11.
[4] Shen, Pan, et al., Fast video stabilization algorithm for UAV, Intelligent Computing and Intelligent Systems, 2009. ICIS 2009. IEEE International Conference on. Vol. 4. IEEE, 2009.
[5] Vazquez Marynel, and Carolina Chang, Real-time video smoothing for small RC helicopters. Systems, Man and Cybernetics, 2009. SMC 2009. IEEE International Conference on. IEEE, 2009.
[6] Wang, Yue, et al., Real-Time Video Stabilization for Unmanned Aerial Vehicles. MVA. 2011.
[7] Kalman, Rudolph E., and Richard S. Bucy, New results in linear filtering and prediction theory. Journal of basic engineering 83.1 (1961) 95-108.
[8] Lucas, Bruce D., and Takeo Kanade, An iterative image registration technique with an application to stereo vision, (1981) 674-679.
[9] Pan, et al., A dual pass video stabilization system using iterative motion estimation and adaptive motion smoothing. Pattern Recognition (ICPR), 2010 20th International Conference on. IEEE, 2010.
[10] Harris, Chris, and Mike Stephens, A combined corner and edge detector. Alvey vision conference. Vol. 15. No. 50. 1988.
[11] Fleet, David, and Yair Weiss. Optical flow estimation. Handbook of mathematical models in computer vision. Springer US, 2006. 237-257.
[12] Holden, Mark, A review of geometric transformations for non-rigid body registration. IEEE transactions on medical imaging 27.1 (2008) 111-128.
[2] Shen, Guturu, et al., Video stabilization using principal component analysis and scale invariant feature transform in particle filter framework, IEEE Transactions on Consumer Electronics 55.3 (2009) 1714-1721.
[3] Lim, Anli, et al., Real-time optical flow-based video stabilization for unmanned aerial vehicles, Journal of Real-Time Image Processing (2017) 1-11.
[4] Shen, Pan, et al., Fast video stabilization algorithm for UAV, Intelligent Computing and Intelligent Systems, 2009. ICIS 2009. IEEE International Conference on. Vol. 4. IEEE, 2009.
[5] Vazquez Marynel, and Carolina Chang, Real-time video smoothing for small RC helicopters. Systems, Man and Cybernetics, 2009. SMC 2009. IEEE International Conference on. IEEE, 2009.
[6] Wang, Yue, et al., Real-Time Video Stabilization for Unmanned Aerial Vehicles. MVA. 2011.
[7] Kalman, Rudolph E., and Richard S. Bucy, New results in linear filtering and prediction theory. Journal of basic engineering 83.1 (1961) 95-108.
[8] Lucas, Bruce D., and Takeo Kanade, An iterative image registration technique with an application to stereo vision, (1981) 674-679.
[9] Pan, et al., A dual pass video stabilization system using iterative motion estimation and adaptive motion smoothing. Pattern Recognition (ICPR), 2010 20th International Conference on. IEEE, 2010.
[10] Harris, Chris, and Mike Stephens, A combined corner and edge detector. Alvey vision conference. Vol. 15. No. 50. 1988.
[11] Fleet, David, and Yair Weiss. Optical flow estimation. Handbook of mathematical models in computer vision. Springer US, 2006. 237-257.
[12] Holden, Mark, A review of geometric transformations for non-rigid body registration. IEEE transactions on medical imaging 27.1 (2008) 111-128.