On-Device Split Inference for Edge Devices

Abstract

Nowadays, the popularity of machine learning and artificial intelligence algorithms is very high. A new research direction has emerged where the machine learning algorithms are executed on resource-constrained embedded devices. With the development of the Internet of Things paradigm, these edge devices are deployed in a lot of places. Due to the limited resources of embedded devices, it is difficult to bring machine learning algorithms to them. This is where the on-device split inference comes in. It is possible to distribute the inference between multiple edge devices and the cloud so that the edge devices can execute the inference of complex machine learning models. This paper presents a systematized literature review of papers that focus on on-device split inference for edge devices. The papers are analyzed and compared based on pre-determined questions and displayed based on several categories.