Battery-powered sensors deployed in the Internet of Things (IoT) require energy-efficient solutions to prolong their lifetime. When these sensors observe a physical phenomenon distributed in space and evolving in time, the collected observations are expected to be correlated. In this paper, we propose an updating mechanism leveraging Reinforcement Learning (RL) to take advantage of the exhibited correlation in the information collected. We implement the proposed updating mechanism employing deep Q-learning. Our mechanism is capable of learning the correlation in the information collected and determine the frequency with which sensors should transmit their updates, while taking into consideration a highly dynamic environment. We evaluate our solution using environmental observations, namely temperature and humidity, obtained in a real deployment. We demonstrate that our mechanism is capable of adapting the transmission frequency of sensors' updates according to the ever-changing environment. We show that our proposed mechanism is capable of significantly extending battery-powered sensors' lifetime without compromising the accuracy of the observations provided to the IoT service.