Learning with Kernel Random Field and Linear SVM

Abstract

Deep learning methods, which include feature extraction in the training process, are achieving success in pattern recognition and machine learning fields but require huge parameter setting, and need the selection from various methods. On the contrary, Support Vector Machines (SVMs) have been popularly used in these fields in light of the simple algorithm and solid reasons based on the learning theory. However, it is difficult to improve recognition performance in SVMs beyond a certain level of capacity, in that higher dimensional feature space can only assure the linear separability of data as opposed to separation of the data manifolds themselves. We propose a new framework of kernel machine that generates essentially linearly separable kernel features. Our method utilizes pretraining process based on a kernel generative model and the mean field Fisher score with a higher-order autocorrelation kernel. Thus derived features are to be separated by a liner SVM, which exhibits far better generalization performance than any kernel-based SVMs. We show the experiments on the face detection using the appearance based approach, and that our method can attain comparable results with the state-of-the-art face detection methods based on AdaBoost, SURF, and cascade despite of smaller data size and no preprocessing.