Managing data quality through automation.

ABSTRACT

Traditional definitions of data quality deal primarily with individual data sets and the data collection process. Today's standards for ensuring data quality have not changed with respect to the desired results, but have simply been expanded to take advantage of modern technology. Computers are used to acquire, review, store, analyze, and report data. Because each of these steps can be automated, the need for human intervention and manual review is minimized. As a result, the potential for invalid data to reach the data analysis stage has increased significantly. To reduce this potential, efforts must be devoted to developing automated procedures that cover every conceivable validation possibility. Relationships between data and data sets must be well defined [1], and data base support that facilitates ready access to the data for the purpose of analysis must be provided. For small data sets, automation may therefore be impractical; but for large, interrelated data sets, automation is highly desirable. Computer automation has therefore expanded the traditional concept of ensuring data quality to include a complex array of interrelated tasks that must be properly managed to achieve the desired results.