LAMAE Learns Multimodal Cardiac Representations Across ECG, Imaging, and Clinical Data

Summary

Cardiovascular diagnosis depends on combining ECG, echocardiography, chest radiographs, and clinical variables, but many medical foundation models learn each modality separately and combine them only later. The paper introduces Latent-Attention Masked Autoencoders (LAMAE), a self-supervised, structure-aware architecture that learns patient-level representations by exchanging information directly in latent space. Its shared latent-attention module operates across a study-view-entity hierarchy, allowing the model to aggregate variable observations and handle missing modalities. LAMAE was pretrained on more than 1.2 million MIMIC-IV hospital stays. Across multimodal tasks including in-hospital mortality prediction, ICD-10 and DRG coding, and length-of-stay prediction, it outperformed modality-specific pretraining and strong contrastive and vision-language baselines. It remained competitive on unimodal tasks, and its gains persisted when only one modality was available at test time. The results suggest that jointly modeling structure within and between modalities can produce more robust and transferable cardiac representations.