A Focus Group Study on Visualization-Based Reinforcement Learning Interpretability

Abstract

Deep reinforcement learning is a dynamic field that has been successfully applied in various engineering and scientific disciplines. However, like many areas of automated learning, it presents a significant challenge: understanding its models, which can hinder human trust in the decisions made by these algorithms. To address this issue, we conducted a focus group and interviews with experts in machine learning and reinforcement learning to gain insights into the perceptions and preferences surrounding interpretability techniques and tools in reinforcement learning systems. The discussion of the focus groups used an interactive dashboard to simplify the analysis of reinforcement learning environments. The focus group results showed many problems to tackle with the current approaches regarding RL agent training, like data manipulation, environment representation and reward misinterpretation. The focus group discussion also presented some key features of a visualization approach to interpretability in RL: contextual presentation of information, integration with existing pipelines, and ease of distribution.