IDALC: A Semi-Supervised Framework for Intent Detection and Active Learning based Correction

Voice-controlled dialog systems have become immensely popular due to their ability to perform a wide range of actions in response to diverse user queries. These agents possess a predefined set of skills or intents to fulfill specific user tasks. But every system has its own limitations. There are instances where even for known intents, if any model exhibits low confidence, it results in rejection of utterances that necessitate manual annotation. Additionally, as time progresses, there may be a need to retrain these agents with new intents from the system-rejected queries to carry out additional tasks. Labeling all these emerging intents and rejected utterances over time is impractical, thus calling for an efficient mechanism to reduce annotation costs. In this article, we introduce intent detection and active learning based correction (IDALC), a semisupervised framework designed to detect user intents and rectify system-rejected utterances while minimizing the need for human annotation. Empirical findings on various benchmark datasets demonstrate that our system surpasses baseline methods, achieving a <inline-formula><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula>5%–10% higher accuracy and a <inline-formula><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula>4%–8% improvement in macro-F1. Remarkably, we maintain the overall annotation cost at just <inline-formula><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula>6%–10% of the unlabeled data available to the system. The overall framework of IDALC is shown in <xref ref-type="fig" rid="fig1">Fig. 1</xref>.