Implementing the right metadata management solution can positively impact an organization’s data strategy to maximize data use and reuse. Read previous blog posts on metadata management best practices and metadata management frameworks to see how metadata management can impact your organization’s data strategy to drive value from data. This blog takes the journey forward and explores how active metadata graphs and machine learning can help build a foundation for Data Intelligence within your organization.
See how Froedtert & the Medical College of Wisconsin transformed their healthcare operations with Collibra’s help.
Read their story and learn how you can achieve similar results!
What are active metadata graphs?
Active metadata graphs span diverse information asset types and connect metadata about these assets to give rich content and context to data so that data can be trusted and acted upon. Native connectivity for metadata ingestion from data sources like data lakes, data warehouses, master data repositories, OLTP databases, spreadsheets, APIs and BI tools helps build the metadata graph.
It’s the metadata of the following information assets (and many more) that are connected to provide rich content and context to all users across an enterprise.
- business glossary terms
- systems and data sources
- policies
- processes
- data sets
- APIs
- ML models
- data products
Active metadata graphs blend machine learning and human intelligence to continuously improve context around the information stored in the data ecosystem. With streamlined, contextual discovery and natural language search, you can efficiently shop for trusted data to drive reliable business outcomes. Active metadata graphs help enterprises break organizational and data silos to get to outcomes fast.
Data governance and privacy professionals attach policies to the metadata, ensuring that access is granted to the right users of data. Data stewards link physical data assets (e.g. tables and columns) to logical data assets (e.g. entities and fields) based on data classification. Data access requests automatically create data usage agreements, which generate an audit trail that also becomes part of the active metadata graph.
Business analysts can find data sets or reports, data scientists can search for ML models, data engineers can view technical lineage, and privacy managers can access policies. The BI reports, data products, additional data sets, or other artifacts these users create from shopped data automatically contribute back to the graph.
Powerful search functionality opens the door to the graph. A user can come in through any door, depending on their business objectives. The value added by the graph continuously grows through increased number of users and higher usage, creating strong network effects.
Displaying collaboration among data professionals via an active metadata graph
The benefits of utilizing metadata graphs and machine learning
According to the Forrester Insights-Driven Business report, businesses are drowning in data but starving for insights. For trustworthy insights, the data driving the analysis must be trustworthy and analysts must be able to find the best data for their purpose. You can ensure better understanding of and access to trusted data through extensive metadata associated with each data asset, providing important and accurate business context.
Machine learning powers the building of active metadata graphs by intelligently automating data classification, cataloging, lineage and policy management to add rich context to data assets at scale.
Active metadata graphs help you:
- Generate greater visibility into the data landscape: With ML-powered automatic data classification and auto-linking of data sets, business terms, policies, processes and more, data curators and data consumers can collaborate on business semantics for trusted data.
- Evaluate the right data for your needs: Data profiling, data scoring, and crowdsourced ratings and reviews strengthen data context and allow business analysts and data scientists to evaluate and choose the best data for their purposes.
- Enhance data shopping experience: With highly relevant and rich business context around data, users have a more intuitive and simplified data shopping experience, which provides the right data with the right context to the right users.
- Deliver faster insights: Automated discovery, understanding, and collaborative data access for business analysts and data scientists reduce time to insights.
Building a Data Intelligence foundation with an active metadata graph powered by machine learning
Imagine you want to compare the current quarter sales with the same quarter sales in the previous year, a standard scenario for sales teams. The comparison appears simple, but when you start searching for the right data, you face several questions.
- Which data should we use? Order management or accounting?
- What is the definition of order date? Order placement, fulfillment, or payment?
- If the order fulfillment spans two quarters, how should we consider it?
- Is the percentage incentive accounted for?
- Is the data complete and consistent?
- Are there missing, incorrect, or duplicate values?
- What’s the approved data structure for storing customer data?
- How was total sales calculated? Show me the data lineage of calculations.
Data Intelligence answers these questions and delivers trusted data access so that you can have confidence in the analyses used to drive your business decisions.
Data Intelligence is the ability to understand and use your data effectively.
An active metadata graph powered by ML is the foundation for Data Intelligence, connecting data assets, insights, and models and offering real-time, compliant and self-service access to trusted data enterprise-wide.
How Collibra’s Data Intelligence Cloud can accelerate trusted business outcomes
Built on collaboration across all data citizens, an active metadata graph provides both technical users (data architects and data engineers) and business users (business analysts and data scientists) the right data in the right context.
Breaking organizational silos maximizes the use and reuse of your data assets, empowering all your data citizens to optimize processes, increase efficiency, and accelerate business outcomes.
With a robust Data Intelligence foundation, you will be able to shop for trusted data for your quarterly sales comparison without any worry about duplicate or missing data, and use a shared understanding of the business terms such as order or accounting year. Strong confidence in your analyses will lead to reliable insights that can drive your critical business decisions.
Gartner Research expects that by 2023, organizations utilizing active metadata and machine learning to dynamically connect and automate data management processes will reduce their time to data delivery and impact on value by 30%. A powerful combination to ensure that you have a robust Data Intelligence foundation and begin a rewarding journey towards relevant and trusted knowledge.
