A Multi-Seed Comparative Evaluation of Class-Imbalance Mitigation Strategies For Four-Class Alzheimer’s MRI Image Classification
Keywords:
Magnetic Resonance Imaging (MRI), Alzheimer's disease (AD), Deep learning, ResNet-50, Class imbalance, weighted random sampling, Weighted cross-entropy, Macro-F1 score, Balanced accuracy, multi-seed evaluation.Abstract
Even though there is growing evidence that magnetic resonance imaging (MRI) can be used to classify Alzheimer's disease, the classification is still difficult to accomplish due to the unequal distribution of diagnostic classes, especially the lack of the higher stages of the disease in typical image sets. Four class-imbalance mitigation strategies are examined in a controlled fashion for the classification of Alzheimer's MRI images in four classes in this paper. Standard cross entropy loss (CE), weighted cross entropy loss (WCE), weighted random sampling (WRS), and class-balanced focal loss with logit adjustment (CBFL-LA) functions are used to test the ImageNet pretrained ResNet-50 model. In the experiments the MRI images are simply categorized as either Mild Demented, Moderate Demented, or Severely Demented. The MRI images are divided into the following categories: Mild Demented, Moderate Demented and Severely Demented in the experiments. Non-Demented, Very Mild Demented. The image-level training, validation and test splitting approach is used and each approach is tested on four random seeds with accuracy, macro-averaged F1-score and balanced accuracy. The outcomes demonstrate that the WRS has the best mean test accuracy of 96.43% ± 0.33%, and the WCE has the best mean macro-F1-score of 94.11% ± 3.31%. The best mean balanced accuracy in the study is obtained with Standard CE (93.80% ± 3.73%). In the setting evaluated, CBFL-LA’s mean performance is lower than that of the other metrics. These results show the effectiveness of imbalance mitigation is related to the chosen evaluation criterion and that the significant increase of the overall accuracy is not necessarily the best measure of class-balanced performance. The study compares in an empirical way commonly used imbalance-handling strategies in a common model and common evaluation protocol. Yet the number of Moderate Demented test images is small and results from splitting the images at the level of the image lead to conclusions that are limited for minority class reliability and patients’ independent generalization. The results can guide future studies on Alzheimer's MRI classification, which focus on balancing the methods for handling these imbalances.

