In this study we provided a scheduling procedure which is combination of machine learning and mathematical programming. Outpatients who request for appointment in healthcare facilities have different priorities. Determining the priority of outpatients and allocating the capacity based on the priority classes are important concepts that have to be considered in scheduling of outpatients. Two stages are defined for scheduling an incoming patient. In the first stage, We applied and compared different clustering methods such as k-mean clustering and agglomerative hierarchical clustering methods to classify outpatients into priority classes and suggested the best pattern to cluster the outpatients. In the second stage, we modeled the scheduling problem as a Markov Decision Process (MDP) problem that aims to decrease waiting time of higher priority outpatients. Due to the curse of dimensionality, we used fluid approximation method to estimate the optimal solution of the MDP. We applied our methodology on a dataset of Shaheed Rajaei Medical and Research Center in Iran, and we showed how our models work in prioritizing and scheduling of outpatients.