A Framework of Data Quality Assurance using Machine Learning

Abstract

This paper presented a framework of data quality assurance by applying machine learning. Data quality assurance often a major concern in the decision-making derivation since decades. Poor quality of the data will eventually affect the result of the decision making. Despite the frequent occurrence of missing data, reliability of data and inconsistency of data, there are a few machine learning algorithms that can perform prevention of poor-quality data at early stage of the data acquisition and data cleansing effectively. This research aims to solve the challenges mentioned above through the framework designed. The framework is designed to integrate two algorithms into the data management lifecycle. Preventive algorithm meant to be applied in the earlier stage of the data management lifecycle while predictive algorithm is applied for the purpose of data cleansing. It is foresee that this framework is able to improve the data quality. Future work will be the implementation of the framework.