Data governance is no longer optional: regulations such as GDPR will start to be enforced; and organizations are finally realizing the value of data as an asset that needs to be protected, managed and maintained to increase asset value. Because data is a digital asset, and has mostly been managed within the realm of IT, organizations are quick to look at technology, expecting to find data governance software and solutions; but technology is only part of the solution.
Data governance, as defined by IDC in a Data Governance Maturityscape, is an organizational discipline that requires a vision and strategy, people and organizational structures, process, data, in addition to technology. I’m confident this will be debated, but I don’t believe that data governance software exists; rather I believe that software exists to support the process of data governance. I call it data intelligence software.
Data intelligence software is a collection of capabilities that helps organizations with grade school concept of answering the five (5) W’s (of data): Who, What, When, Where, Why, and How? Who is using the data, who created the data or asset, and who is responsible for it? What does the data represent, what is the data being used for? When was the data created, when is the data being used, and when will the data expire? Where is the data in the organization, and where is it being consumed? Why does the data exist, why is the data being persisted, and why is it being used? How was the data created or captured, and how is it being used? And Data intelligence also adds another dimension: Relationship.– what relationships are inherent within the data and between the people that are generating and consuming the data?
Having answers to these questions is what will inform and guide use cases such as data governance, data quality management and self-service data. Getting answers to these questions requires harnessing the power of metadata that is generated every time data is captured at a source, accessed by users, moves through an organization, is profiled, cleansed, aggregated, augmented and used for analytics for operational or strategic decision making. Data intelligence software goes beyond metadata management, it also includes data cataloging, master data definition and control, data profiling and data stewardship.
I have also spoken with many organizations that need (and want) data governance but cannot use the word “governance” in any attempt to gain buy-in; either because it implies constraints, or because past efforts have failed (miserably). Data intelligence, on the other hand, is a fresh and almost intriguing term because it doesn’t imply constraints, but promises opportunity – to learn more about the data itself and how the organization uses data. Not only capturing the intelligence to answer the 5 W’s, but also analysis of the intelligence, combined with the content of the data itself may yield a whole new level of insight in this age of big data and digital transformation.
I believe that one of the reasons past attempts at data governance have failed is because intelligence about data was created and maintained manually, and the variety of intelligence required to answer the 5 W’s has not existed. In a recent survey conducted by IDC, we discovered that spreadsheets, custom software, documents, and word of mouth were among the top most frequently used methods of cataloging data. Manual processes have never been able to keep up with the pace of change, in business or technology; and now in the age of digital transformation, change is even more rapid and constant, within increasingly complex technical and business environments.
Data is at the heart of digital transformation and deserves specific attention as a critical building block of the digital platform. Organizations are facing more complex ecosystems and business environments within an expansion of data characteristics, types, constructs, behaviors, domains, social contexts, and hybrid technical environments. For the past several years, data professionals have focused on the three V’s of big data: volume, velocity, and variety. These big data characteristics still exist and have become ingrained in data operations, but to get value from data in the era of digital transformation, data professionals need to shift their focus toward the three A’s: awareness, augmentation, and automation.
Data intelligence software is providing organizations with an awareness of where their data is, who is using the data, and why and how the data is being used. Data intelligence software is also augmenting data with technical, semantic and business metadata, adding key knowledge elements so that data, and the usage of data is better understood. Data intelligence demands automation of data discovery, definition, duplication, consistency, usage, and protection in this age of big data where the 3 V’s, combined with the increased distribution of data across the digital transformation platform makes any manual efforts, useless.
Expect to see more research and content forthcoming from IDC on data intelligence software and the functional use cases it supports, including data governance, data quality management and self-service data. Let the debate about the existence of data governance software begin….
Explore the future of data intelligence with IDC’s FutureScape: Worldwide Data, Integration, and Analytics 2020 Predictions:
