Federated learning (FL) enables multiple clients to collaboratively train a\nshared model without disclosing their local datasets. This is achieved by\nexchanging local model updates with the help of a parameter server (PS).\nHowever, due to the increasing size of the trained models, the communication\nload due to the iterative exchanges between the clients and the PS often\nbecomes a bottleneck in the performance. Sparse communication is often employed\nto reduce the communication load, where only a small subset of the model\nupdates are communicated from the clients to the PS. In this paper, we\nintroduce a novel time-correlated sparsification (TCS) scheme, which builds\nupon the notion that sparse communication framework can be considered as\nidentifying the most significant elements of the underlying model. Hence, TCS\nseeks a certain correlation between the sparse representations used at\nconsecutive iterations in FL, so that the overhead due to encoding and\ntransmission of the sparse representation can be significantly reduced without\ncompromising the test accuracy. Through extensive simulations on the CIFAR-10\ndataset, we show that TCS can achieve centralized training accuracy with 100\ntimes sparsification, and up to 2000 times reduction in the communication load\nwhen employed together with quantization.\n