Intelligent multi-modal cardiovascular disease detection framework using hybrid deep learning and explainable artificial intelligence: a clinical decision support perspective

Abstract

Cardiovascular diseases (CVDs) remain the leading cause of mortality worldwide, accounting for nearly 17.9 million deaths annually. Early and accurate diagnosis is essential for reducing morbidity and enabling timely therapeutic intervention. Although significant advancements have been achieved in clinical diagnostic technologies, conventional machine learning approaches still face challenges related to representation learning, heterogeneous multi-modal data integration, and clinical interpretability. To address these limitations, this study proposes an Intelligent Multi-Modal Clinical Decision Support System (IM-CDSS) based on a hybrid architecture integrating Convolutional Neural Networks (CNN), Bidirectional Long Short-Term Memory (BiLSTM), and Bahdanau attention mechanisms for comprehensive analysis of electrocardiogram (ECG) signals, echocardiographic images, and clinical biomarkers. The proposed framework was trained and evaluated using 12,847 de-identified patient records compiled from publicly available datasets, including the MIT-BIH Arrhythmia Database, PhysioNet MIMIC-IV, and Cleveland Heart Disease Dataset. Model interpretability was enhanced using SHapley Additive exPlanations (SHAP) and Gradient-weighted Class Activation Mapping (Grad-CAM), while Bayesian optimization with Optuna was employed for hyperparameter tuning. Five-fold cross-validation ensured robust evaluation. Experimental results demonstrate that IM-CDSS achieved 97.6% accuracy, a macro F1-score of 97.2%, and an AUC of 0.986 across five cardiovascular disease categories. Ablation studies confirmed the effectiveness of individual architectural components, while SHAP analysis identified Heart Rate Variability, QRS Duration, and ST Elevation as the most influential diagnostic features. The proposed framework offers improved accuracy, interpretability, and computational efficiency, supporting its potential deployment in resource-constrained clinical environments.