Self-Supervised Metric Learning With Graph Clustering For Speaker\n Diarization

In this paper, we propose a novel algorithm for speaker diarization using\nmetric learning for graph based clustering. The graph clustering algorithms use\nan adjacency matrix consisting of similarity scores. These scores are computed\nbetween speaker embeddings extracted from pairs of audio segments within the\ngiven recording. In this paper, we propose an approach that jointly learns the\nspeaker embeddings and the similarity metric using principles of\nself-supervised learning. The metric learning network implements a neural model\nof the probabilistic linear discriminant analysis (PLDA). The self-supervision\nis derived from the pseudo labels obtained from a previous iteration of\nclustering. The entire model of representation learning and metric learning is\ntrained with a binary cross entropy loss. By combining the self-supervision\nbased metric learning along with the graph-based clustering algorithm, we\nachieve significant relative improvements of 60% and 7% over the x-vector PLDA\nagglomerative hierarchical clustering (AHC) approach on AMI and the DIHARD\ndatasets respectively in terms of diarization error rates (DER).\n