University of Nevada Las Vegas / Concorde Career Colleges (Young & Matthews)
LoRA-adapted domain-specialized cardiology text embedding model built on BioLinkBERT (340M parameters), identified as the top performer among 10 encoder- and decoder-style transformer architectures benchmarked head-to-head for cardiology semantic retrieval. LoRA fine-tuning on ~150,000 cardiology-textbook-derived sentence pairs raised its cardiology semantic-separation score from 0.033 (zero-shot) to 0.510, the highest of any evaluated architecture (including decoder models up to 10x larger), while remaining Pareto-optimal for the separation/throughput trade-off at 143.5 embeddings/sec and a 1.51GB memory footprint.
Architecture
Transformer
BioLinkBERT (340M-parameter BERT-style encoder, pretrained on PubMed abstracts with document-link prediction) adapted via LoRA (rank 16, attention query/value projections) with an InfoNCE contrastive objective on cardiology textbook sentence pairs
Framework
PyTorch
Added to catalog
2026-08-10
Apache 2.0
License for model weights only. Associated code may be licensed seperately, check code source for specific terms.
Cardiology Textbook Sentence-Pair Corpus (CardioEmbed family)
~150,000 anchor-positive sentence pairs derived from 7 cardiology textbooks; text corpus, not human-subject data
Domain-specialized cardiology text embedding for semantic retrieval and similarity search