Video inpainting using object motion rate and color variance in spatiotemporal domain

Article type: Research Article

Authors: Huang, Hui-Yu^{; *} | Lin, Chih-Hung

Affiliations: Department of Computer Science and Information Engineering, National Formosa University, Huwei Township, Taiwan

Correspondence: [*] Corresponding author. Hui-Yu Huang. E-mail: hyhuang@nfu.edu.tw.

Abstract: Inpainting is a technique to enhance digital videos. Based on the spatiotemporal domain, we herein propose a video inpainting method to repair the removal objects in the videos. The method consists of an adaptive foreground model, the motion rate estimation of objects, and a repairing scheme. Initially, the adaptive foreground model based on the background subtraction method is developed. The model is used to estimate the motion rate for each moving object in the frame. According to the estimated motion rate, the model specifies an adaptive interval between the forwarding reference frame and backward reference frame to obtain the useful information and to repair the removal objects. The remaining un-repaired areas are filled using an exemplar-based inpainting technique with color variance. The results show that the proposed method can produce visually pleasing results. Additionally, it reduces the inpainting time and provides efficient computing.

Keywords: Video inpainting, image inpainting, exemplar-based inpainting, spatiotemporal domain

DOI: 10.3233/JIFS-200542

Journal: Journal of Intelligent & Fuzzy Systems, vol. 40, no. 3, pp. 5609-5622, 2021