In this paper, we propose a new deep neural network classifier that\nsimultaneously maximizes the inter-class separation and minimizes the\nintra-class variation by using the polyhedral conic classification function.\nThe proposed method has one loss term that allows the margin maximization to\nmaximize the inter-class separation and another loss term that controls the\ncompactness of the class acceptance regions. Our proposed method has a nice\ngeometric interpretation using polyhedral conic function geometry. We tested\nthe proposed method on various visual classification problems including\nclosed/open set recognition and anomaly detection. The experimental results\nshow that the proposed method typically outperforms other state-of-the art\nmethods, and becomes a better choice compared to other tested methods\nespecially for open set recognition type problems.\n