TOTALLY CORRECTIVE BOOSTING WITH CARDINALITY PENALIZATION

Methods, systems, and apparatus, for totally corrective boosting with cardinality penalization are described. One of the methods includes obtaining initialization data identifying training examples, a dictionary of weak classifiers, and an active weak classifier matrix. Iterations of a totally corrective boosting with cardinality penalization process are performed, wherein each iteration performs operations comprising selecting a weak classifier from the dictionary of weak classifiers that most violates a constraint of a dual of the primal problem. The selected weak classifier is included in the active weak classifier matrix. The primal problem is optimized, and a discrete weight vector is determined. Weak classifiers are identified from the active weak classifier matrix with respective discrete weights greater than a threshold. The regularized risk is optimized, and a continuous weight vector is determined. The classifier is determined as an ensemble identified by the weak classifiers and the continuous weight vector.