METHOD AND APPATATUS FOR EMBODIED CONVERSATIONAL CHARACTERS WITH MULTIMODAL INPUT/OUTPUT IN AN INTERFACE DEVICE

Deliberative and reactive processing are combined to process multi-modal inputs and direct movements and speech of a synthetic character that operates as an interface between a user and a piece of equipment. The synthetic character is constructed as an ally, working with and helping the user learn and operate the equipment. The synthetic character interacts with both a virtual space where the character is displayed, and a physical space (domain) that includes the user. Real-time reactive processing provides lifelike and engaging responses to user queries and conversations. Deliberative processing provides responses to inputs that require more processing time (deep linguistic processing, for example). Knowledge bases are maintained for both dynamic (discourse model, for example) and static (e.g., knowledge about the domain or discourse plans) information types. A rule based system is utilized against the knowledge bases and selected inputs from said user to determine input meanings, and follow a predetermined discourse plan.