Designing Memory for Social Robots: Facilitating Conversations in Elderly Care during an Art Activity using Large Language Models

Abstract

This research explores the integration of episodic memory and visual short-term memory in social robots to enhance engagement and interaction with older adults, particularly those with dementia, through art activities such as painting. Dementia significantly impacts millions globally, being one of the leading causes of death worldwide. Furthermore, it creates a substantial burden on caregivers and healthcare systems. This research aims to leverage social robots to provide meaningful engagement, companionship, and emotional support to individuals with dementia.

The study involved developing a painting application using Android Studio and a conversational agent using generative AI. The AI model was enhanced with episodic memory and visual short-term memory, enabling it to reference past interactions through episodic memory and analyze painting progress through recent visual inputs with short-term memory. This qualitative study included 8 participants, and post-session interviews were conducted to gain deeper insights into the participants' experiences with the system.

The study suggests that memory-enhanced social robots can improve engagement and interaction quality among older adults interested in painting, but have limited impact on those uninterested in the activity. Mixed responses were observed regarding the robot's role as a companion, with some participants feeling a sense of company while others did not. Overall, participants did not experience frustration due to the robot's presence.

Challenges include response time and real-time awareness, highlighting the need for a more intelligent system. Given the small sample size of eight participants, future research should involve larger groups for more comprehensive and reliable data.