Systematic Approach to Data Normalization and Standardization in Machine Learning Problems

Abstract

The article presents a systematic approach to normalization and standardization at the stage of data analysis and pre-processing when solving machine learning tasks. Data normalization is a necessary initial stage of data processing in the systematic use of many multivariate statistical methods. Features of the system approach to normalization are described. The stages of the system approach are defined. At the first stage, the initial data set, the machine learning task and the modeling method are analyzed for the need for normalization. At the second stage, the type of data distribution is determined and normality is checked. At the third stage, a check is carried out for the presence of emissions in the set. At the fourth stage, data is normalized. The classification of normalization methods is given. The main methods of normalization are described and the features of linear and non-linear normalization methods are considered. An example of systematic use of normalization methods is given. The importance and effectiveness of the system approach to solving normalization tasks at the stage of data analysis and pre-processing of machine learning tasks is proven.