HeartBERT: A Self-Supervised ECG Embedding Model for Efficient and Effective Medical Signal Analysis

The analysis of Electrocardiogram (ECG) signals is critical for clinical applications, but current machine learning methods often face limitations when dealing with smaller datasets or intricate signal patterns. We introduce HeartBERT, a novel model designed to enhance ECG signal analysis using self-supervised learning. Drawing inspiration from Bidirectional Encoder Representations from Transformers (BERT) in natural language processing and leveraging the RoBERTa architecture, HeartBERT generates sophisticated embeddings optimized for biomedical signal analysis. To demonstrate the efficacy of the proposed model, two key downstream tasks are selected: sleep-stage classification and heartbeat classification. HeartBERT-based systems, utilizing bidirectional LSTM heads, are designed to tackle complex challenges. A series of practical experiments is conducted to reveal the superiority and advancements of HeartBERT, particularly in its ability to perform well with smaller training datasets. The code and data are publicly available at https://github.com/ecgResearch/HeartBert.