How good is my story? Towards quantitative metrics for evaluating LLM-generated XAI narratives

A rapidly developing application of LLMs in XAI is to convert quantitative\nexplanations such as SHAP into user-friendly narratives to explain the\ndecisions made by smaller prediction models. Evaluating the narratives without\nrelying on human preference studies or surveys is becoming increasingly\nimportant in this field. In this work we propose a framework and explore\nseveral automated metrics to evaluate LLM-generated narratives for explanations\nof tabular classification tasks. We apply our approach to compare several\nstate-of-the-art LLMs across different datasets and prompt types. As a\ndemonstration of their utility, these metrics allow us to identify new\nchallenges related to LLM hallucinations for XAI narratives.\n