Object Detection in Illustrated Imagery

Abstract

In contrast to the prevalent focus on real photos in computer vision research, we present a contribution by making the Ot & Sien dataset machine learning-ready for object detection tasks in illustrations. We refer to the new dataset as Ot & Sien++ that is composed of scanned images of children’s book illustrations, thereby venturing into an unexplored domain. The primary objective of this research is to investigate the generalization capabilities of existing object detection models to this unique dataset and establish benchmarks for this dataset.
To evaluate the performance of existing object detection models on our proposed dataset, we employed the widely used YOLOv5 as a benchmark. To mitigate the inherent imbalance of the dataset, various data augmentation techniques were applied. The results demonstrated the effectiveness of the object detection model and data augmentation in the context of children’s book illustrations. In addition, this research also explored applying few-shot learning models to the dataset. Baseline models were investigated to examine the potential of few-shot learning in the context of object detection in illustrations.
The proposed dataset elicits new challenges in object detection and will serve as a valuable resource for researchers in this domain. Our dataset can be found at https://data.4tu.nl/datasets/d1f3ca5c-f1e4-48f5-9a04-0564572d2b9c/1.