Industrial sponsored search system (SSS) can be logically divided into three modules: keywords matching, ad retrieving, and ranking. The ad candidates grow exponentially during ad retrieving. Due to limited latency and computing resources, the candidates have to be pruned earlier. Suppose we set a pruning line to cut SSS into two parts: upstream and downstream. The problem we are going to address is: how to pick out the best K items from N candidates provided by the upstream to maximize the total system’s revenue. Since the industrial downstream is very complicated and updated quickly, a crucial restriction in this problem is that the selection scheme should get adapted to the downstream. In this paper, we propose a novel model-free reinforcement learning approach to fixing this problem. Our approach considers the downstream as a black-box environment, and the agent sequentially selects items and finally feeds into the downstream, where revenue would be estimated and used as a reward to improve the selection policy. The idea has been successfully realized in Baidu’s sponsored search system, and online long time A/B test shows remarkable improvements on revenue.