Graph Convolutional Networks (GCNs) have emerged as powerful tools for\nlearning on network structured data. Although empirically successful, GCNs\nexhibit certain behaviour that has no rigorous explanation -- for instance, the\nperformance of GCNs significantly degrades with increasing network depth,\nwhereas it improves marginally with depth using skip connections. This paper\nfocuses on semi-supervised learning on graphs, and explains the above\nobservations through the lens of Neural Tangent Kernels (NTKs). We derive NTKs\ncorresponding to infinitely wide GCNs (with and without skip connections).\nSubsequently, we use the derived NTKs to identify that, with suitable\nnormalisation, network depth does not always drastically reduce the performance\nof GCNs -- a fact that we also validate through extensive simulation.\nFurthermore, we propose NTK as an efficient `surrogate model' for GCNs that\ndoes not suffer from performance fluctuations due to hyper-parameter tuning\nsince it is a hyper-parameter free deterministic kernel. The efficacy of this\nidea is demonstrated through a comparison of different skip connections for\nGCNs using the surrogate NTKs.\n