Visualizing the Content of a Children’s Story in a Virtual World: Lessons Learned

Abstract

We present the problem of “bringing text to life” via 3D interactive storytelling, where nat-ural language processing (NLP) techniques are used to transform narrative text into events in a virtual world that the user can interact with. This is a challenging problem, which requires deep understanding of the semantics of a story and the ability to ground those semantic elements to the actors and events of the 3D world’s graphical engine. We show how this problem has motivated interesting extensions to some classic NLP tasks, identify some of the key lessons learned from the work so far, and pro-pose some future research directions.