Temporal action localization is an important step towards video\nunderstanding. Most current action localization methods depend on untrimmed\nvideos with full temporal annotations of action instances. However, it is\nexpensive and time-consuming to annotate both action labels and temporal\nboundaries of videos. To this end, we propose a weakly supervised temporal\naction localization method that only requires video-level action instances as\nsupervision during training. We propose a classification module to generate\naction labels for each segment in the video, and a deep metric learning module\nto learn the similarity between different action instances. We jointly optimize\na balanced binary cross-entropy loss and a metric loss using a standard\nbackpropagation algorithm. Extensive experiments demonstrate the effectiveness\nof both of these components in temporal localization. We evaluate our algorithm\non two challenging untrimmed video datasets: THUMOS14 and ActivityNet1.2. Our\napproach improves the current state-of-the-art result for THUMOS14 by 6.5% mAP\nat IoU threshold 0.5, and achieves competitive performance for ActivityNet1.2.\n