Ensembles of deep learning models can be used for estimating predictive uncertainty. Existing ensemble approaches, however, introduce a high computational and memory cost limiting their applicability to real-time biosignal applications (e.g. ECG, EEG). To address these issues, we propose early exit ensembles (EEEs) for estimating predictive uncertainty via an implicit ensemble of early exits. In particular, EEEs are a collection of weight sharing sub-networks created by adding exit branches to any backbone neural network architecture. Empirical evaluation of EEEs demonstrates strong performance in accuracy and uncertainty metrics as well as computation gain highlighting the benefit of combining multiple structurally diverse models that can be jointly trained. Compared to state-of-the-art baselines (with an ensemble size of 5), EEEs can improve uncertainty metrics up to 2× while providing test-time speed-up and memory reduction of approx. 5×. Additionally, EEEs can improve accuracy up to 3.8 percentage points compared to single model baselines.
Paper
Full text
Robust and Efficient Uncertainty Aware Biosignal Classification via Early Exit Ensembles
OpenAlex · Anomaly Detection Techniques and Applications · 2022
Abstract
Ensembles of deep learning models can be used for estimating predictive uncertainty. Existing ensemble approaches, however, introduce a high computational and memory cost limiting their applicability to real-time biosignal applications (e.g. ECG, EEG). To address these issues, we propose early exit ensembles (EEEs) for estimating predictive uncertainty via an implicit ensemble of early exits. In particular, EEEs are a collection of weight sharing sub-networks created by adding exit branches to any backbone neural network architecture. Empirical evaluation of EEEs demonstrates strong performance in accuracy and uncertainty metrics as well as computation gain highlighting the benefit of combining multiple structurally diverse models that can be jointly trained. Compared to state-of-the-art baselines (with an ensemble size of 5), EEEs can improve uncertainty metrics up to 2× while providing test-time speed-up and memory reduction of approx. 5×. Additionally, EEEs can improve accuracy up to 3.8 percentage points compared to single model baselines.