Due to the large memory footprint of untrimmed videos, current\nstate-of-the-art video localization methods operate atop precomputed video clip\nfeatures. These features are extracted from video encoders typically trained\nfor trimmed action classification tasks, making such features not necessarily\nsuitable for temporal localization. In this work, we propose a novel supervised\npretraining paradigm for clip features that not only trains to classify\nactivities but also considers background clips and global video information to\nimprove temporal sensitivity. Extensive experiments show that using features\ntrained with our novel pretraining strategy significantly improves the\nperformance of recent state-of-the-art methods on three tasks: Temporal Action\nLocalization, Action Proposal Generation, and Dense Video Captioning. We also\nshow that our pretraining approach is effective across three encoder\narchitectures and two pretraining datasets. We believe video feature encoding\nis an important building block for localization algorithms, and extracting\ntemporally-sensitive features should be of paramount importance in building\nmore accurate models. The code and pretrained models are available on our\nproject website.\n