Augmented Body Communicator: Enhancing daily body expression for people with upper limb limitations through LLM and a robotic arm

Individuals with upper limb movement limitations face difficulties interacting with others. Currently, the use of robotic arms is focused on addressing functional tasks, but there is still significant room for exploration around enhancing users’ body language capabilities during social interactions. This paper proposes an Augmented Body Communicator system that combines robotic arms and a large language model. Thanks to the incorporation of kinetic memory disabled users and their supporters can collaboratively create actions for the robot arm. The LLM system then provides recommendations about which action is most appropriate based on contextual cues in interactions with others. The system underwent in-depth user testing with 6 participants with conditions affecting upper limb mobility. Results indicate that the system enhances users’ ability to express themselves. Based on our findings we propose recommendations for robotic arms able to support disabled individuals with body language capabilities and functional tasks.