Managing Data Quality. Tim King

unnecessarily – I want to be unique and original.

      Challenges when exploiting and managing data

      23

       5. Being forced to look different from time to time – I like to be consistent.

       6. Bits of me being removed – I like to be complete.

       7. Incompetent people being near me – I will tell them lies or, even worse, I will delete myself and not come back.

       8. The assumption I want to be housed in an expensive system – I am equally happy in a simple spreadsheet if I’m properly looked after and appreciated.

       9. Being ignored – because if you show me you don’t care I will just not help you and I might even become obsolete.

       10. Unstructured data – don’t get me started! Oh, how I hate whiteboards.

      In summary, if you handle me correctly, I promise I will serve you in the right way and shall be around as long as you need me. Treat me badly and I promise I can, and will, mess up your decisions. I do like to be user friendly, so can you please be data friendly?

      The challenge of managing enterprise data quality

      This chapter has illustrated a number of factors and considerations that show why data quality is difficult to manage at an enterprise level. These can be summarised as follows:

      There can be many data stores, particularly if you include unofficial ones, locally and in the cloud.

      The number of data stores grows rapidly (and uncontrollably) as people create new spreadsheets and exploit cloud data stores.

      Ownership and stewardship of these data stores is weak, if present at all.

      Similarly, there can be many different software systems that include their own data stores and also undertake many data updates.

      There are many different business processes that use and update data and are run by different parts of the organisation.

      There are many users of systems and processes, some of which do not have the correct view of what data are required, what the process is or where the correct place to store data is.

      It is virtually impossible to ‘rewind’ data back to the point at which they were ‘good’ in order to resolve data quality problems.

      Data exploitation might not use the correct balance of tools, subject matter expertise and awareness of data quality, leading to worse perceptions of data quality. Poorer quality data might result in a different choice of tools to exploit it.

      Users could have different levels of diligence or personal business need, with some users only entering the bare minimum of data into any system they interact with.

      Managing Data Quality

      24

      Historic or legacy interventions can have created many data issues that are unlikely to be corrected. Such interventions include, for example, poorly defined and governed business processes, poor quality data migrations or bulk updates to data that have resulted in data corruptions.

      Summary

      Data flow across organisations and become of interest to many different processes, individuals and teams.

      Technology is only a tool to help deliver effective data quality management, and cannot achieve this on its own.

      Data have specific fundamental features that demand careful consideration.

      The next chapter explains how people are at the heart of the data quality story in any organisation.

      25

      This chapter introduces the effect of people and their behaviours on data quality. We discuss the ‘Data Zoo’ concept, to illustrate the generic behaviours that people exhibit towards data quality, and then consider how behaviours are influenced when individuals are working as part of a team. Finally, we consider some of the factors that encourage particular behaviour types.