Manuscript received May 9, 2024; revised May 17, 2024; accepted July 29, 2024; published January 27, 2025.

Abstract—Advertising (AD) on video backgrounds reduces the disturbing of audiences. However, integrating virtual AD on videos’ dynamic backgrounds is difficult. The object detection method cannot follow the background moving correctly, so that the AD replacement shows flicking on the video. Furthermore, the target background object may move behind the virtual logo and foreground objects. The virtual AD should be inserted behind these things. To overcome those challenges, this research integrates multiple Deep Neural Networks (DNN). First, object detection DNN identified object locations. Thus, tracking of these locations was done through an optical flow DNN. Moreover, an image in-painting DNN reconstructed the blocked objects, which helps the detection approach. Based on the detection and tracking, the virtual AD is pasted on the video, then object segmentation is utilized to put foreground objects back on top of the virtual AD. The experimental results show that, in the dynamic background scenario, the AD replacement has a sensitivity of approximately 81.1%, a specificity of at least 92.57%and a successful rate of more than 83.3%. This means that, in most cases, the virtual AD can be integrated into the appropriate position on the moving background.

Keywords—deep learning, neural networks cooperation, product replacement, dynamic background detection, background inpainting, foreground segmentation

Cite: Cheng Yang, Fucheng Zheng, Duaa Zuhair Al-Hamid, Peter Han Joo Chong, and Patrick Lam, "Deep Learning Based Advertisement Replacement on Dynamic Background Videos," Journal of Image and Graphics, Vol. 13, No. 1, pp. 34-45, 2025.

Copyright © 2025 by the authors. This is an open access article distributed under the Creative Commons Attribution License (CC-BY-4.0), which permits use, distribution and reproduction in any medium, provided that the article is properly cited, the use is non-commercial and no modifications or adaptations are made.