System and Method for Interacting with Live Agents in an Automated Call Center

Embodiments of an interface system that enables a call center agent to access and intervene in an interaction between an automated call center system and a caller whenever necessary for complex application tasks is described. The system includes a user interface that presents the agent with one or more categories of information, including the conversation flow, obtained semantic information, the recognized utterances, and access to the utterance waveforms. This information is cross-linked and attached with a confidence level for better access and navigation within the dialog system for the generation of appropriate responses to the caller.