Comparative Analysis of Voting and Stacking Ensemble Learning for Heart Disease Prediction: A Machine Learning Approach

Abstract

Heart disease remains a leading cause of mortality worldwide, necessitating the development of accurate predictive models for early diagnosis and intervention. This study investigates the effectiveness of ensemble learning approaches, particularly Voting and Stacking classifiers, in comparison to traditional machine learning models and deep learning architectures. Using a dataset containing clinical and diagnostic attributes, preprocessing steps such as label encoding and standardization were applied to ensure compatibility with machine learning models. The ensemble classifiers were constructed using base learners, including Random Forest, Gradient Boosting, and XGBoost, with soft voting aggregation and logistic regression meta-learning for the Stacking approach. The models were evaluated using stratified ten-fold cross-validation based on precision, recall, F1-score, and ROC-AUC. The results indicate that the Voting classifier achieved the highest overall F1-score (0.8882) and ROC-AUC (0.8697), surpassing the Stacking classifier. Additionally, ensemble models demonstrated competitive performance compared to deep learning architectures, with Random Forest and Gradient Boosting achieving the highest ROC-AUC scores of 0.9313 and 0.9279, respectively. The findings suggest that ensemble methods provide an effective, interpretable, and computationally efficient alternative to deep learning for heart disease prediction. This study highlights the potential of ensemble learning in medical applications and provides valuable insights into optimizing classification models for structured tabular healthcare datasets.