Towards On-Device Semantic Search using LLMs

Abstract

Traditional search engines rely on centralized databases and powerful servers to process and retrieve information. Developing alternatives to key-value search engine databases in distributed computing environments is a significant challenge, particularly when dealing with limited computational resources. This study explores the use of large language models (LLMs) to address this problem. We focus on environments with constrained computing power, such as mobile devices, to investigate the feasibility of using LLMs as a localized search solution. Through experiments with the state-of-the-art LLMs BERT and T5, we demonstrate their ability to memorize and retrieve unstructured data, specifically YouTube video IDs, based on partial information derived from video titles or tags. Our results show that the explored models can achieve 100\% precision and recall when retrieving 48266 video IDs. The findings suggest that LLMs have the potential to effectively function as a search engine database, offering semantic search capabilities while operating within the constraints of limited computational resources.