National University of Singapore / Peking University (Lan, Feng et al.)
First multimodal LLM to unify ECG time series, 12-lead ECG images, and text for grounded, clinician-aligned ECG interpretation. A dual-encoder framework (ECG-CoCa time-series encoder plus a LLaVA-style vision-language backbone) extracts complementary time-series and image features with cross-modal alignment, trained on knowledge-guided instruction data (ECG-Grounding, linking diagnoses to measurable waveform parameters such as QRS/PR intervals) plus the 1.15-million-conversation ECG-Instruct corpus. Introduces the "Grounded ECG Understanding" benchmark and improves predictive performance, explainability, and grounding over prior ECG-language models such as ECG-Chat and PULSE.
Architecture
Hybrid
Dual-encoder MLLM: ECG-CoCa contrastive time-series encoder plus a LLaVA-style vision-language architecture (built on PULSE-7B / LLaVA-1.6-Vicuna-7B) fused via cross-modal alignment for grounded ECG report generation and question answering
Framework
PyTorch
Added to catalog
2026-08-10
Apache 2.0
License for model weights only. Associated code may be licensed seperately, check code source for specific terms.
Publicly available 15% subset of the CODE (Clinical Outcomes in Digital Electrocardiology) dataset from the Telehealth Network of Minas Gerais; full CODE dataset (>2 million exams) is request-only.
12-lead ECG multi-label arrhythmia-classification dataset released for the 2018 China Physiological Signal Challenge.
45,152 12-lead, 10-second ECGs from Chapman University / Shaoxing People's Hospital / Ningbo First Hospital with arrhythmia diagnoses; ~56% male / 44% female.
Georgia 12-lead ECG Challenge (G12EC) database, part of the PhysioNet/CinC 2020-2021 Challenge training data.
800,035 12-lead ECG-report pairs from 161,352 subjects at Beth Israel Deaconess Medical Center.
52% male / 48% female; age range 0-95 (median ~62). 21,837 10-second 12-lead ECG records.
Grounded ECG interpretation report generation, linking diagnoses to measurable waveform parameters (e.g. QRS/PR intervals)
Multi-label ECG diagnostic classification (ECG-Bench evaluation)