Confidence Dimension for Deep Learning based on Hoeffding Inequality and Relative Evaluation

Research on the generalization ability of deep neural networks (DNNs) has\nrecently attracted a great deal of attention. However, due to their complex\narchitectures and large numbers of parameters, measuring the generalization\nability of specific DNN models remains an open challenge. In this paper, we\npropose to use multiple factors to measure and rank the relative generalization\nof DNNs based on a new concept of confidence dimension (CD). Furthermore, we\nprovide a feasible framework in our CD to theoretically calculate the upper\nbound of generalization based on the conventional Vapnik-Chervonenk dimension\n(VC-dimension) and Hoeffding's inequality. Experimental results on image\nclassification and object detection demonstrate that our CD can reflect the\nrelative generalization ability for different DNNs. In addition to\nfull-precision DNNs, we also analyze the generalization ability of binary\nneural networks (BNNs), whose generalization ability remains an unsolved\nproblem. Our CD yields a consistent and reliable measure and ranking for both\nfull-precision DNNs and BNNs on all the tasks.\n