K. N. Toosi University of Technology (Tahery, Hamid Akhlaghi, Amirsoleimani, Farzi)
Self-supervised ECG embedding model inspired by BERT/RoBERTa from natural language processing, designed for efficient medical signal analysis. HeartBERT translates ECG signals into an intermediate synthetic 'language' via signal quantization and discretization (Lloyd-Max quantization), then trains a RoBERTa-style encoder from scratch on this text-like representation using the MIT-BIH Arrhythmia Database, PTB-XL, and European ST-T Database. The resulting embeddings are evaluated on two downstream tasks -- sleep-stage classification and heartbeat classification -- using bidirectional LSTM heads, showing particular strength when only small labeled training datasets are available.
Architecture
Hybrid
RoBERTa-style transformer encoder trained from scratch on ECG signals converted to a synthetic textual representation via signal quantization/discretization, producing self-supervised embeddings used with bidirectional LSTM heads for downstream tasks
Added to catalog
2026-08-14
European ST-T Database
48 half-hour excerpts of two-channel ambulatory ECG recordings from 47 subjects studied by the BIH Arrhythmia Laboratory; used as a cross-dataset, lead-missing/noise-robustness benchmark distinct from the pretraining data.
52% male / 48% female; age range 0-95 (median ~62). 21,837 10-second 12-lead ECG records.
Self-supervised ECG embeddings used for downstream sleep-stage classification and heartbeat classification