Rice University
Multimodal ECG model that pairs a 1D ConvNeXtV2 signal encoder with a BioLinkBERT text encoder, trained with a joint contrastive-and-captioning objective using LLM-generated descriptions of ECG demographics and waveform patterns in place of raw clinical reports. Validated on arrhythmia diagnosis and ECG-based subject identification, reaching an AUROC of 0.938 fine-tuned and 0.812 zero-shot on PTB-XL diagnostic classification. Developed at Rice University.
Architecture
Hybrid
1D ConvNeXtV2-base ECG signal encoder + BioLinkBERT text encoder, trained with a joint contrastive and captioning loss (COCA-style)
Framework
PyTorch
Added to catalog
2026-07-10
GPL 3.0
License for model weights only. Associated code may be licensed seperately, check code source for specific terms.
45,152 12-lead, 10-second ECGs from Chapman University / Shaoxing People's Hospital / Ningbo First Hospital with arrhythmia diagnoses; ~56% male / 44% female.
52% male / 48% female; age range 0-95 (median ~62). 21,837 10-second 12-lead ECG records.
PTB-XL diagnostic classification (fine-tuned)
PTB-XL diagnostic classification (linear probe)
PTB-XL diagnostic classification (zero-shot)
Embedding