Multimodal large language model for ECG medical-report generation and cardiology conversational question-answering. An ECG-CoCa encoder (contrastive ECG-report pretraining in the style of OpenCLIP) is paired with a LLaVA-style vision-language architecture and an LLM backbone, fine-tuned on a purpose-built 45k-example ECG-instruction dataset (19k diagnosis examples + 25k multi-turn dialogue examples) built from five public 12-lead ECG datasets. Produces free-text diagnostic reports and supports zero-shot ECG-report retrieval classification.

Systematic study of domain specialization for large language models in electrocardiography, comparing supervised fine-tuning (QLoRA) against retrieval-augmented generation (RAG) as two paths to inject ECG/cardiology knowledge into open-weight Llama 3.1 models (8B and 70B). Question-answer and multiple-choice pairs were generated from ECG/cardiology literature and used both for fine-tuning and for a multi-layered evaluation (multiple-choice accuracy, text-similarity metrics, LLM-as-a-judge, and blinded human-cardiologist review). The fine-tuned Llama 3.1 70B ranked first overall, exceeding the RAG variants and Claude Sonnet 3.7 on in-distribution multiple-choice and text-similarity metrics, though RAG and Claude generalized better to semantically complex, out-of-distribution questions. Developed by AI4Health at the University of Oldenburg with Charite Berlin; the finetuning/RAG/evaluation code is public, but per the paper's data-availability statement neither the training corpus nor the fine-tuned weights are released (German copyright law, section 60d UrhG).

First multimodal LLM to unify ECG time series, 12-lead ECG images, and text for grounded, clinician-aligned ECG interpretation. A dual-encoder framework (ECG-CoCa time-series encoder plus a LLaVA-style vision-language backbone) extracts complementary time-series and image features with cross-modal alignment, trained on knowledge-guided instruction data (ECG-Grounding, linking diagnoses to measurable waveform parameters such as QRS/PR intervals) plus the 1.15-million-conversation ECG-Instruct corpus. Introduces the "Grounded ECG Understanding" benchmark and improves predictive performance, explainability, and grounding over prior ECG-language models such as ECG-Chat and PULSE.

Multi-agent LLM framework, deployed as a Software-as-a-Medical-Device on AWS, that assists cardiologists reading 24-hour Holter/patch ECG monitoring studies. Three fine-tuned LLM agents divide the diagnostic workflow the way a cardiologist would: a table-to-text agent (Llama-3.1-8B) extracts findings from tabular arrhythmia metrics, an image-to-text agent (LLaVA-v1.5-13B) extracts findings from ECG tracing images, and a findings-to-interpretation agent (Llama-3.1-8B) synthesizes both against clinical guidelines with a fact-checking step. Each agent is instruction-tuned on cardiologist-adjudicated reports from 2,000+ real-world patients and further steered at inference with in-context demonstrations matched to the patient's age, sex and arrhythmia class. In blinded cardiologist ratings across eight clinical/security metrics (1-5 scale), ZODIAC outperformed GPT-4o, Gemini-Pro, Llama-3.1-405B, Mixtral-8x22B, and medical-specialist LLMs (BioGPT, Meditron, Med42) on every metric while using under 30B total parameters, and has been integrated into commercial ECG monitoring devices. This is a proprietary product; no public code or model weights have been released.