Learning generative models for graph-structured data is challenging because\ngraphs are discrete, combinatorial, and the underlying data distribution is\ninvariant to the ordering of nodes. However, most of the existing generative\nmodels for graphs are not invariant to the chosen ordering, which might lead to\nan undesirable bias in the learned distribution. To address this difficulty, we\npropose a permutation invariant approach to modeling graphs, using the recent\nframework of score-based generative modeling. In particular, we design a\npermutation equivariant, multi-channel graph neural network to model the\ngradient of the data distribution at the input graph (a.k.a., the score\nfunction). This permutation equivariant model of gradients implicitly defines a\npermutation invariant distribution for graphs. We train this graph neural\nnetwork with score matching and sample from it with annealed Langevin dynamics.\nIn our experiments, we first demonstrate the capacity of this new architecture\nin learning discrete graph algorithms. For graph generation, we find that our\nlearning approach achieves better or comparable results to existing models on\nbenchmark datasets.\n