In this paper, we address the problem of action recognition from still images\nand videos. Traditional local features such as SIFT, STIP etc. invariably pose\ntwo potential problems: 1) they are not evenly distributed in different\nentities of a given category and 2) many of such features are not exclusive of\nthe visual concept the entities represent. In order to generate a dictionary\ntaking the aforementioned issues into account, we propose a novel\ndiscriminative method for identifying robust and category specific local\nfeatures which maximize the class separability to a greater extent.\nSpecifically, we pose the selection of potent local descriptors as filtering\nbased feature selection problem which ranks the local features per category\nbased on a novel measure of distinctiveness. The underlying visual entities are\nsubsequently represented based on the learned dictionary and this stage is\nfollowed by action classification using the random forest model followed by\nlabel propagation refinement. The framework is validated on the action\nrecognition datasets based on still images (Stanford-40) as well as videos\n(UCF-50) and exhibits superior performances than the representative methods\nfrom the literature.\n