Automated Product Placement using GANs (2019) ¶
Supervisor - Srinivasan Govindaraj
Members - Advait Rane, Krut Patel
I worked as an ML intern at Happiest Minds Technologies, Bangalore to automate contextual product placement for advertising in videos.
The aim of this project was to automate the placement of certain products into video content post-production, so as to fit in with the context of the scene. To achieve this, we identified a similar object in the video and replaced it with the product to be advertised. We created a pipeline which involved (1) Object detection and (2) Product substitution.
We evaluated the use of different object detection models like YOLOv3, Faster R-CNN, and RetinaNet. We preferred YOLO since it produced results faster, which would be a major asset if we wanted to run the pipeline real time.
After identifying the object to be substituted, we used image-to-image translation GANs to replace it with the product. These are neural networks which take an image of a particular class as input and convert it to a similar image of a different class.
We evaluated several models such as CycleGAN, UNITGAN, and GANimorph. UNITGAN and GANimorph gave the best results. As a part of this stage, I implemented the GANimorph model using Python and PyTorch. The image below, taken from the GANimorph paper, describes the structure of the different models.
We trained these GANs to replace a bottle with another of a particular brand. The results obtained can be seen in the images below.
You can find the code for the my GANimorph implementation here