Hierarchical Voting Decoder for Resolving Knowledge Conflicts

Large Language Models (LLMs) acquire extensive general knowledge during pre-training but often require external contextual evidence for real-world applications. Conflicts between internal knowledge and external context—known as knowledge conflicts—pose a major challenge to reliable inference. Existing solutions based on contrastive decoding alleviate this issue to some extent but fail to capture the hierarchical nature of knowledge representations across different model layers. To address this gap, we propose the Hierarchical Voting Decoder (HVD), a novel decoding framework that combines intra-layer contrastive decoding with inter-layer voting to exploit knowledge disparities across shallow, intermediate, and deep layers. This hierarchical strategy enables dynamic selection of trustworthy knowledge paths, thereby improving both robustness and faithfulness under conflict scenarios. Extensive experiments on multiple real-world datasets demonstrate that HVD consistently outperforms state-of-the-art baselines in mitigating knowledge conflicts. Our code is publicly available at: https://github.com/AKak23/HVD.