This paper aims to define, quantify, and analyze the feature complexity that\nis learned by a DNN. We propose a generic definition for the feature\ncomplexity. Given the feature of a certain layer in the DNN, our method\ndisentangles feature components of different complexity orders from the\nfeature. We further design a set of metrics to evaluate the reliability, the\neffectiveness, and the significance of over-fitting of these feature\ncomponents. Furthermore, we successfully discover a close relationship between\nthe feature complexity and the performance of DNNs. As a generic mathematical\ntool, the feature complexity and the proposed metrics can also be used to\nanalyze the success of network compression and knowledge distillation.\n