Traffic sign data augmentation using Stable Diffusion conditioned with ControlNet

Abstract

With the recent trend towards autonomous vehicles, traffic sign recognition has becomean important automotive task. Like other machine learning tasks, it requires large amountsof data. The datasets are often created using dashcam or similar videos and then manuallyannotated. As traffic signs are specific to each country, the datasets are expensive. Therefore,there’s a need to augment existing data to increase its diversity and size without incurring thehigh costs of manual annotation.Our goal is to apply strong augmentations to create a diverse dataset that remains realistic.We use the Stable Diffusion image generation model1 (Rombach et al. (2022)) and condition itusing ControlNet (Zhang (2023)) - a model specifically designed to condition Stable Diffusionusing another image. We train ControlNet to use Canny’s edge detection map as an input tocondition Stable Diffusion. This allows us to generate new realistic images as an extension tothe existing dataset.