Deep learning strategy for cost-effective, comprehensive cardiac screening from ECG alone, by transferring domain-specific structural information from cardiac magnetic resonance (CMR) imaging into ECG representations. Combines multimodal contrastive learning with masked data modelling during pretraining on paired ECG-CMR data, then uses only ECG at inference. On 40,044 UK Biobank subjects, the multimodal pretraining improved subject-specific CVD risk prediction by up to 12.19% and cardiac phenotype prediction by up to 27.59% versus ECG-only baselines, with learned ECG representations shown to incorporate information from CMR regions of interest.

Self-supervised deep learning model that extracts cardiovascular-risk-relevant patterns from multimodal polysomnography (PSG) signals -- EEG, ECG, and respiratory signals -- without relying on manual sleep-stage annotations. Trained on 4,398 participants, the model derives 'projection scores' by contrasting embeddings from individuals with and without cardiovascular disease (CVD) outcomes. Externally validated in an independent cohort of 1,093 participants, ECG-derived projection scores were predictive of prevalent and incident cardiac conditions (particularly CVD mortality), and combining projection scores with the Framingham Risk Score consistently improved prediction (AUC 0.607-0.965 internally, 0.710-0.807 externally across most outcomes).

Foundation vision-language model for cardiac MRI that learns pathophysiological visual representations directly from the natural-language radiology reports accompanying each scan, rather than from hand-labeled targets. A Multi-scale Vision Transformer (MViT, Kinetics-400-initialized) video encoder for cine CMR sequences is contrastively pretrained (InfoNCE) against a PubMed-pretrained BERT text encoder over 19,041 multi-institutional CMR studies. The frozen vision encoder transfers with strong performance to left-ventricular ejection-fraction regression (MAE 3.34% on a UK Biobank hold-out of ~4,259-45,623 participants) and detecting HFrEF (LVEF<40%, AUC 0.880), and the paper reports emergent zero-/few-shot performance across 39 cardiac and non-cardiac conditions including cardiac amyloidosis and hypertrophic cardiomyopathy. Code and pretrained MViT encoder weights are both released (Hugging Face, CC BY-NC 4.0).

Generalist reconstruction foundation model for accelerating cardiac MRI (CMR) acquisition, designed to recover diagnostic-quality images from highly undersampled (8x-24x) multi-coil k-space data across heterogeneous scanners, field strengths, and cardiovascular diseases. Combines a CLIP-ViT-based module for semantic/contextual understanding of the anatomy being imaged with a physics-informed data-consistency reconstruction network, trained on MMCMR-427K -- the largest public multimodal CMR k-space database to date. Intended as an upstream substrate that feeds downstream segmentation, phenotyping, and diagnosis models (e.g. automated cardiac-phenotype extraction via nnU-Net) rather than replacing them. Released by the CMRxRecon-challenge consortium; code and the underlying database are public for academic, non-commercial use, but no separately downloadable pretrained checkpoint is provided.

Multimodal large language model for ECG medical-report generation and cardiology conversational question-answering. An ECG-CoCa encoder (contrastive ECG-report pretraining in the style of OpenCLIP) is paired with a LLaVA-style vision-language architecture and an LLM backbone, fine-tuned on a purpose-built 45k-example ECG-instruction dataset (19k diagnosis examples + 25k multi-turn dialogue examples) built from five public 12-lead ECG datasets. Produces free-text diagnostic reports and supports zero-shot ECG-report retrieval classification.

First multimodal LLM to unify ECG time series, 12-lead ECG images, and text for grounded, clinician-aligned ECG interpretation. A dual-encoder framework (ECG-CoCa time-series encoder plus a LLaVA-style vision-language backbone) extracts complementary time-series and image features with cross-modal alignment, trained on knowledge-guided instruction data (ECG-Grounding, linking diagnoses to measurable waveform parameters such as QRS/PR intervals) plus the 1.15-million-conversation ECG-Instruct corpus. Introduces the "Grounded ECG Understanding" benchmark and improves predictive performance, explainability, and grounding over prior ECG-language models such as ECG-Chat and PULSE.

Multi-scale ECG-language pretraining model that aligns 12-lead ECG signals with clinical text reports at three granularities -- token, beat, and rhythm level -- rather than a single global embedding. First fine-tunes a cardiology-specialized text encoder to improve understanding of ECG report language, then trains an ECG-FM-initialized ECG encoder against it with hierarchical contrastive supervision. Outperforms prior ECG-language and self-supervised baselines including MERL, ST-MEM, and HeartLang on zero-shot classification, linear probing, and ECG report generation, with especially large gains at low label fractions. Developed at the University of Hong Kong (HKU-MedAI).

Multimodal cardiac MRI foundation model that fuses 3D+T cine CMR (short-axis and long-axis views) with tabular patient health records (demographics, metabolic, and lifestyle factors) from 42,000 UK Biobank participants. Two-stage self-supervised pretraining -- masked-image reconstruction, then imaging-tabular contrastive alignment -- produces representations that transfer to whole-heart segmentation, cardiac phenotype/physiological-feature regression, and cardiac/metabolic disease classification within one unified framework.

Multi-agent LLM framework, deployed as a Software-as-a-Medical-Device on AWS, that assists cardiologists reading 24-hour Holter/patch ECG monitoring studies. Three fine-tuned LLM agents divide the diagnostic workflow the way a cardiologist would: a table-to-text agent (Llama-3.1-8B) extracts findings from tabular arrhythmia metrics, an image-to-text agent (LLaVA-v1.5-13B) extracts findings from ECG tracing images, and a findings-to-interpretation agent (Llama-3.1-8B) synthesizes both against clinical guidelines with a fact-checking step. Each agent is instruction-tuned on cardiologist-adjudicated reports from 2,000+ real-world patients and further steered at inference with in-context demonstrations matched to the patient's age, sex and arrhythmia class. In blinded cardiologist ratings across eight clinical/security metrics (1-5 scale), ZODIAC outperformed GPT-4o, Gemini-Pro, Llama-3.1-405B, Mixtral-8x22B, and medical-specialist LLMs (BioGPT, Meditron, Med42) on every metric while using under 30B total parameters, and has been integrated into commercial ECG monitoring devices. This is a proprietary product; no public code or model weights have been released.

Vision-language model that jointly embeds a cardiac MRI study, treated as video, with the impression section of its clinical report. Combines a video encoder over cine/LGE frame sequences with a Bio+ClinicalBERT text encoder using CLIP-style contrastive training. Supports zero-shot and few-shot classification of cardiomyopathies, amyloidosis, and LV dysfunction, plus image/report retrieval and structured report drafting. Trained on a private, single-institution corpus of roughly 11,000-14,000 CMR study-report pairs from Cleveland Clinic and Case Western.

Multimodal ECG model that pairs a 1D ConvNeXtV2 signal encoder with a BioLinkBERT text encoder, trained with a joint contrastive-and-captioning objective using LLM-generated descriptions of ECG demographics and waveform patterns in place of raw clinical reports. Validated on arrhythmia diagnosis and ECG-based subject identification, reaching an AUROC of 0.938 fine-tuned and 0.812 zero-shot on PTB-XL diagnostic classification. Developed at Rice University.

Vision-language foundation model fine-tuned from CLIP on more than one million private echocardiogram video-report pairs, enabling zero-shot cardiac function assessment, device identification, and image/text retrieval without task-specific training. Combines a ConvNeXt-Base video encoder with a GPT-2-style text encoder under contrastive pretraining. Training data is private, but model weights and code are public. Developed by Cedars-Sinai's Ouyang lab.

Vision-language foundation model that interprets an entire transthoracic echocardiogram study rather than a single view or video: it classifies the view type of every clip, applies view-informed attention across the full study, and generates or retrieves comprehensive study-level interpretations in English or Italian. Pretrained on a private Cedars-Sinai corpus of 12 million echo video-report pairs. Developed by the Smidt Heart Institute and Stanford's Ouyang lab.

Distills knowledge from EchoCLIP, a vision-language echocardiography model, into ECG embeddings, aiming to improve how well ECG signals alone can predict echo-derived measures of cardiac function. Combines a 1D ECG encoder with a BioBERT text encoder under a probabilistic cross-modal embedding objective that captures uncertainty. Published at MICCAI 2025 by the University of Toronto's McIntosh Lab.

Multimodal model that learns a shared representation space for ECG signals and their clinical text reports, pretrained on paired MIMIC-IV-ECG recordings and reports. Supports zero-shot ECG classification via text prompts, tested across six public benchmark datasets including PTB-XL and CPSC2018 without any downstream training data. Developed at Imperial College London and published at ICML 2024.