One of the most important parts of Artificial Neural Networks is minimizing\nthe loss functions which tells us how good or bad our model is. To minimize\nthese losses we need to tune the weights and biases. Also to calculate the\nminimum value of a function we need gradient. And to update our weights we need\ngradient descent. But there are some problems with regular gradient descent ie.\nit is quite slow and not that accurate. This article aims to give an\nintroduction to optimization strategies to gradient descent. In addition, we\nshall also discuss the architecture of these algorithms and further\noptimization of Neural Networks in general\n