Multi-dimensional Hawkes process (MHP) is a class of self and mutually\nexciting point processes that find wide range of applications -- from\nprediction of earthquakes to modelling of order books in high frequency\ntrading. This paper makes two major contributions, we first find an unbiased\nestimator for the log-likelihood estimator of the Hawkes process to enable\nefficient use of the stochastic gradient descent method for maximum likelihood\nestimation. The second contribution is, we propose a specific single hidden\nlayered neural network for the non-parametric estimation of the underlying\nkernels of the MHP. We evaluate the proposed model on both synthetic and real\ndatasets, and find the method has comparable or better performance than\nexisting estimation methods. The use of shallow neural network ensures that we\ndo not compromise on the interpretability of the Hawkes model, while at the\nsame time have the flexibility to estimate any non-standard Hawkes excitation\nkernel.\n