PowerEvaluationBALD: Efficient Evaluation-Oriented Deep (Bayesian) Active Learning with Stochastic Acquisition Functions

We develop BatchEvaluationBALD, a new acquisition function for deep Bayesian\nactive learning, as an expansion of BatchBALD that takes into account an\nevaluation set of unlabeled data, for example, the pool set. We also develop a\nvariant for the non-Bayesian setting, which we call Evaluation Information\nGain. To reduce computational requirements and allow these methods to scale to\nlarger acquisition batch sizes, we introduce stochastic acquisition functions\nthat use importance sampling of tempered acquisition scores. We call this\nmethod PowerEvaluationBALD. We show in a few initial experiments that\nPowerEvaluationBALD works on par with BatchEvaluationBALD, which outperforms\nBatchBALD on Repeated MNIST (MNISTx2), while massively reducing the\ncomputational requirements compared to BatchBALD or BatchEvaluationBALD.\n