Eliciting user interaction though the incremental confirmation of user understanding using multimodal cues

We propose and evaluate a suite of dialogue strategies designed to elicit user-initiated interaction in multimodal dialogue systems. Effective communication relies on active contributions from all participants; however, users interacting with dialogue systems often exhibit less engagement compared to human–human interactions. We hypothesize that this discrepancy arises from the lack of incremental confirmation of user understanding by the system, depriving the user from opportunity to show interaction. To address this, we introduce a strategy that incrementally confirms user understanding though lexical, acoustic, temporal, and visual cues. We implement this strategy in a multimodal dialogue system and conduct a comparative study against a baseline system without the strategy. User interaction behaviors were analyzed using objectively measurable features. Results demonstrate that our approach significantly enhances user interaction, as evidenced by increased frequencies of backchanneling (+22.0%), turn exchanges (+11.2%), and number of questions (+25.8%). Notably, the system elicits active user participation, reflected in more questions and frequent turn exchanges, going beyond the backchannel-oriented engagement achieved in prior studies. Furthermore, we examine correlations between these behavioral metrics and users’ subjective evaluations of engagement.