Improving the architecture of a two-level heterogeneous ensemble for solvingImprovingmachine learning problems

Abstract

The article investigates an approach to improving the architectures of two-level heterogeneous ensembles of models for solving machine learning problems. An improved ensemble architecture is proposed. In which the boosting method is used at the first level of ensemble learning to gradually improve the solutions of the base models. At the second level, the stacking method is used to aggregate the solutions of the base models using a metamodel. The base models used were a model based on multiple linear regression, a decision tree model, a random forest model, a support vector model, a KNN model, a model based on an artificial neural network, and a multivariate adaptive regression spline model. These models are divided into two groups: undertrained and over trained. The experimental part of the study was carried out on solving the problem of predicting the electricity generation indicators of hybrid power plants based on environmental indicators. The use of the improved architecture of a two-level heterogeneous ensemble demonstrated an increase in forecast accuracy compared to other ensemble architects and solutions based on any of the base models. The proposed approach is effective in solving machine learning problems.