A Hybrid Multimodal Explainable Deep Learning Framework for Early Heart Disease Detection Using ECG, Retinal Images, and Clinical Features
Keywords:
Medical, Cardiovascular, Artificial Intelligence, Explainable AIAbstract
Cardiovascular diseases are amongst the leading cause of deaths in the world today, stressing the importance of reliable and early prediction mechanisms. Classical prediction methods use a single modality technique to identify heart disease patients. This paper seeks to propose a new framework known as a Hybrid Multimodal Explainable Deep Learning Model to predict heart diseases at an early stage using ECG signals, retinal fundus images, and clinical information. In the proposed model, ECG signal processing branch is used to process temporal signal information, a convolutional neural network for the retinal images branch to extract visual data, and a dense neural network for processing clinical parameters. Feature extraction and fusion were done using multimodal features integration method. The proposed model was evaluated using publicly available databases of ECG, retinal images, and clinical information. The model showed a better classification accuracy of 98%. Precision, recall, F1-score, ROC analysis, and confusion matrix performance evaluation were other evaluation techniques that proved the efficiency of the model.

