Recent work on exploration in reinforcement learning (RL) has led to a series\nof increasingly complex solutions to the problem. This increase in complexity\noften comes at the expense of generality. Recent empirical studies suggest\nthat, when applied to a broader set of domains, some sophisticated exploration\nmethods are outperformed by simpler counterparts, such as {\\epsilon}-greedy. In\nthis paper we propose an exploration algorithm that retains the simplicity of\n{\\epsilon}-greedy while reducing dithering. We build on a simple hypothesis:\nthe main limitation of {\\epsilon}-greedy exploration is its lack of temporal\npersistence, which limits its ability to escape local optima. We propose a\ntemporally extended form of {\\epsilon}-greedy that simply repeats the sampled\naction for a random duration. It turns out that, for many duration\ndistributions, this suffices to improve exploration on a large set of domains.\nInterestingly, a class of distributions inspired by ecological models of animal\nforaging behaviour yields particularly strong performance.\n