Migrating to the cloud provides CDOs with the opportunity to derive value from increasing volumes of data. Using cloud data warehouses like Google BigQuery, ensures the ability to scale access to data at speed. As a result, more CDOs have migrated to cloud data platforms or are in the process of migrating. They see the cloud as a strategic investment and many turn to BigQuery as their cloud data warehouse. But despite the power of BigQuery, cloud migration isn’t always easy and many CDOs fear that moving to the cloud will just replicate their on-prem issues. These CDOs fear ending up with a cloud platform where there is more unusable data because of lack of context, ownership and quality of data. They are also worried about sending data to the cloud that shouldn’t be in the cloud or should be very restricted or encrypted. Ultimately, CDOs do not want to migrate over bad data which can lead to bad reporting, lack of trust in reports and machine learning models. More specially, CDOs often face three distinct challenges while migrating:
- Ownership and accountability
- Identifying, classifying and managing sensitive data as well as enabling security policies
- Managing ongoing data quality to get the most of your data
Luckily, Collibra’s integration with Google Cloud offers organizations a way to help alleviate these worries by building an internal data marketplace in which data consumers and data producers publish and shop for the best quality data. Our joint value proposition brings Google Cloud and Collibra’s capabilities together to help customers accelerate their data journey and successfully migrate to the cloud.
Building an internal data marketplace
In order to achieve a successful cloud migration, CDOs must first pick the right technology. BigQuery is the best cloud data warehouse available, and with Collibra, CDOs can build an internal data marketplace on top of the warehouse. Collibra provides the governance foundation that is crucial to a successful internal data marketplace.
Collibra Data Catalog provides an easy way to get every piece of data identified during the data migration process. As LOBs sign up for the cloud platform, they identify data sources for ingestion. Rather than lift and shift, Collibra will ingest metadata first to make sense of the data in the source. This is crucial because it provides insight into what data is available across the organization, including context such as ownership, data policies (should this data be shared, if yes under what conditions) and usage (what BI reports depend on this data). This governed ingestion process ensures only the right data is being migrated to the cloud. Data Offices can then follow the technical and business lineage from BigQuery and CloudSQL to provide more visibility into where the data is, where it came from and where it is going in the migration process.
Embed security policies with Collibra Policy Manager
Once the data is successfully ingested into BigQuery, CDOs want to enable their team, and in some cases the entire organization, to use this data. But how do CDOs and CISCOs know the data is safe and being used in a compliant manner? While a data marketplace provides great access to the entire organization through the ability to easily “shop for data,” policies still need to be in place to ensure the compliant use of data. With Collibra Policy Manager, as data moves to the cloud, Collibra can scan and identify sensitive data. As the data is ingested, it is linked to policies, which can in turn be linked to Google Cloud policy tags. These tags can be used to write down the policy to Google Cloud Data Catalog, IAM and BigQuery. This ensures the complaint use and sharing of data across the organization.
Solve data quality issues with Collibra
The number one issue organizations face while migrating to the cloud is data quality. A recent Harvard Business Review journal reported that 47% of recently created data records have at least one critical error and that 25% of revenue for most companies is lost due to bad data. Collibra Data Quality solves these challenges with machine learning. Collibra Data Quality automatically learns data quality rules and continuously observes the data. This means that the more data you send to the cloud, the better it becomes. Users can monitor the data in BigQuery and observe the data, auto discover the data, and build the rules necessary to ensure the highest quality data. This enables self-service data quality for the entire organization, thus reducing cost.
Collibra together with BigQuery
Collibra’s integration with Google Cloud makes it easier for enterprises to understand and use the data that is vital to their business. Our partnership provides our customers a strong foundation for governance that enables them to take control of their data, scale access and accelerate analytics all while maintaining compliance with privacy and policy practices. Together, BigQuery and Collibra provide the tools for a successful cloud migration and help ensure CDOs do not face roadblocks on their journey. By adopting data governance, lineage and data quality, businesses can keep their end-to-end data pipelines clean to fast-track their digital transformation journey. Through metadata management and data quality, businesses can decide if data assets are fit for consumption for a given business context. And finally, Collibra Data Catalog and Collibra Data Quality together, help businesses continuously monitor and cleanse their critical data sources and data elements.
