Common problems in classifying large data are revealed as long processing time and a lot of training data in order to maintain high accuracy. To solve these problems, researchers study methods for classifying large data to reduce the use of large amounts of training data without sacrificing high classification efficiency. The proposed method reduces the size of the training data by combining K-means and deep learning. To study the effectiveness of the proposed method, the accuracy and AUC values were determined. In addition, it was compared with the original deep learning method using 80% and 90% training data out of the total data and was compared with the original deep learning using the same amount of training data. The results show that the proposed method can significantly reduce the size of the training data. Less than 1% of the total data size was used as training data, but the method yielded the high average percent of accuracy and the high average AUC of the classification. In the case of normal distribution and the size is 1,000,000 × 5 (N × Feature), the proposed method exhibits the average percent of accuracy as high as 97.4878% and the average AUC as 0.9735. When the proposed method was compared with the deep learning method using training data about 80% and 90% of the total data size, classification efficiency was relatively as high as that of the deep learning, but the classification time was 2–4 times less than the processing time of the deep learning method.
Keywords
Large-scale Data; K-means Clustering; Outlier Detection; Deep Learning; Classification
THE JOURNAL OF KMUTNB
Published by : King Mongkut's University of Technology North Bangkok Contributions welcome at : http://www.journal.kmutnb.ac.th
By using our website, you acknowledge that you have read and understand our Cookie Policy and Privacy Policy.