Data quality's in the eye of the beholder
Data quality management can be defined as the various tools and/or processes that are implemented within a business that result in the development of data that is accurate, complete and valid. Data quality is about having the right (correct) information at the right time, in the right place and for the right people. This data then, in return, is used to make accurate decisions - to allow the business to serve its customers effectively and achieve company goals.
Janine Sutherland, a business intelligence consultant of the PBT Group, said:"The above sounds simple and standard, right? However, the reality is that multiple departments within a business look at the same piece of data, yet the information value derived from that data is actually vastly different.
Different value
"For example; let's look at a customer account record, which contains the customer's name, surname, account/product type and account balance. The data found here is consumed by the marketing, finance and collections department; however each department finds different value in this data provided."
For the marketing department, the customer detail is directly related to the value it places on customer behaviour and, based on this, possible cross-sell opportunities. For the finance department, the customer himself only plays a minor role in budget reporting. However, this data and the related details play a very critical role in collections department - especially for debt collection and recovery.
Continued Sutherland: "For the marketing and collections department, having the correct account/product type and up-to-date contact details and data would be of the utmost importance. Incorrect data could lead to potential business revenue loss (where doubtful debt could not be collected) or the marketing department could miss potential cross-sell opportunities. In contrast, the finance department would only require the contact details data for the purposes of invoicing."
The problem, however, comes in when, for example, the finance department notices that the contact detail data is incorrect. "Should this happen, the finance department flags the data with a record as poor-quality data; however, this data would still be most suitable for marketing purposes as the account type and product have been verified as correct. Irrespective of which data is incorrect, the whole record should not be deemed as being of 'poor data quality', but rather should be looked at in context, to define its fit and purpose for use."
Data-quality algorithms
Quality of the data can be measured by implementing various data-quality algorithms associated to each data-quality dimension - these being: accuracy, completeness, consistency, currency, precision, privacy, reasonable, referential integrity, timeliness, uniqueness, and validity.
Based on the business area where the data is used and the business value add or importance, weightings can be assigned to each of the key measures and their respective evaluations would then assist in disseminating and highlighting any problem areas. Additionally, you can determine an overall data-quality index by summing up the overall scores for each data-quality dimension and you could apply weighting factors according to importance.
"In the scenario above, the objective of 'fit for purpose' data quality could have been achieved by a lower referential integrity score, but with a much higher currency and accuracy weighting for the finance department. The inverse would apply to the marketing department when it comes to their referential integrity, currency and accuracy weighting scores," added Sutherland.
By aligning the data-quality dimensions and by assigning weightings and business rules to the information value that business derives from data, "bad data" can no longer be a general quality statement, as it is now expressed in terms of being "fit for purpose".
Concluded Sutherland: "This 'fit for purpose' approach, if implemented within a business, would be more pleasing to the consumer of the data - in other words, the individual who needs to use the data for decision making. As such, data quality is in fact in the eye of the beholder."
