Two provably consistent divide and conquer clustering algorithms for large networks

Significance In this paper, we present two divide-and-conquer algorithms for clustering large graphs. Both algorithms apply a base algorithm on several small subgraphs and then use these individual local clusterings to obtain a global clustering. We show that our methods help scale computationally intensive base clustering algorithms to large networks, and improve the algorithmic stability of some well-known algorithms. In this article, we advance divide-and-conquer strategies for solving the community detection problem in networks. We propose two algorithms that perform clustering on several small subgraphs and finally patch the results into a single clustering. The main advantage of these algorithms is that they significantly bring down the computational cost of traditional algorithms, including spectral clustering, semidefinite programs, modularity-based methods, likelihood-based methods, etc., without losing accuracy, and even improving accuracy at times. These algorithms are also, by nature, parallelizable. Since most traditional algorithms are accurate, and the corresponding optimization problems are much simpler in small problems, our divide-and-conquer methods provide an omnibus recipe for scaling traditional algorithms up to large networks. We prove the consistency of these algorithms under various subgraph selection procedures and perform extensive simulations and real-data analysis to understand the advantages of the divide-and-conquer approach in various settings.