Big Data Dimensionality Reduction Technique with Deep Autoencoder

O.P.  Nweke; D.  Matthias; E.O.  Bennett

Abstract

Dimensionality reduction refers to the act of narrowing the focus from an abundance of potential variables to a more manageable set. Google created the open-source Flutter framework for creating mobile apps. The issue of high-dimensional data hindered the system's effectiveness in terms of providing correct findings (the quantity of false-positive, negative, true-positive, and negative outcomes). The process of solving a classification problem, carrying out a precise visualization, and transmitting vast amounts of data will be made easier by the reduction of high-dimensional data. Because of the high dimensional problem and big data, this paper proposes a reduction technique named Deep Autoencoder Network that was used in reducing high-dimension data to low-dimension data. The proposed technique was applied to big data for dimensionality reduction. The outcome of the Deep Autoencoder demonstrates how the big data runtime was decreased and improved accuracy was attained. For test data, 78% accuracy was achieved. The output of the Deep Autoencoder was evaluated by principal component analysis (PCA) and singular value decomposition. SVD and PCA were used to assess the Deep Autoencoder's (SVD) output. Runtime and accuracy were also contrasted. The results show that the Deep Autoencoder outperforms PCA and SVD with an accuracy level of 78%, and runtime of 10 seconds over 14% for PCA, 12% for SVD, with a runtime of 46 seconds, and 53 seconds, respectively.

Abstract

Keywords