Knowledge Graph (KG) completion has been excessively studied with a massive\nnumber of models proposed for the Link Prediction (LP) task. The main\nlimitation of such models is their insensitivity to time. Indeed, the temporal\naspect of stored facts is often ignored. To this end, more and more works\nconsider time as a parameter to complete KGs. In this paper, we first\ndemonstrate that, by simply increasing the number of negative samples, the\nrecent AttH model can achieve competitive or even better performance than the\nstate-of-the-art on Temporal KGs (TKGs), albeit its nontemporality. We further\npropose Hercules, a time-aware extension of AttH model, which defines the\ncurvature of a Riemannian manifold as the product of both relation and time.\nOur experiments show that both Hercules and AttH achieve competitive or new\nstate-of-the-art performances on ICEWS04 and ICEWS05-15 datasets. Therefore,\none should raise awareness when learning TKGs representations to identify\nwhether time truly boosts performances.\n