Many adaptive optimization methods have been proposed and used in deep\nlearning, in which Adam is regarded as the default algorithm and widely used in\nmany deep learning frameworks. Recently, many variants of Adam, such as\nAdabound, RAdam and Adabelief, have been proposed and show better performance\nthan Adam. However, these variants mainly focus on changing the stepsize by\nmaking differences on the gradient or the square of it. Motivated by the fact\nthat suitable damping is important for the success of powerful second-order\noptimizers, we discuss the impact of the constant $\\epsilon$ for Adam in this\npaper. Surprisingly, we can obtain better performance than Adam simply changing\nthe position of $\\epsilon$. Based on this finding, we propose a new variant of\nAdam called EAdam, which doesn't need extra hyper-parameters or computational\ncosts. We also discuss the relationships and differences between our method and\nAdam. Finally, we conduct extensive experiments on various popular tasks and\nmodels. Experimental results show that our method can bring significant\nimprovement compared with Adam. Our code is available at\nhttps://github.com/yuanwei2019/EAdam-optimizer.\n