In this series, we’re focusing on what it’s like to take a data strategy into the real world, build operations with it and derive tangible benefits. Specifically, we’re considering the Data Office, a core resource to develop and disseminate data-related skills and services throughout the organization. We’ve previously examined the steps needed to create a data office and explored what a data office will do within the company.
We’re using our own environment at Collibra as the proving ground. Like all of our clients, we use data to grow the business. We face the same pressure points and seek to take advantage of market opportunities. In every sense, we drink our own champagne.
Now it’s time to look deeper. What kind of products can we see coming from the Data Office?
Some aspects of Data Intelligence can seem conceptual; a product is very real; it brings home the possibilities of an operating philosophy better than any strategy document. While a department or function has goals and missions, a particular product solves particular problems. It enhances efficiency, minimizes risk, cuts costs, drives growth, etc. It delivers real value.
So let’s explore three aspects: what goes into building data products, the Collibra approach to data product development, and how this might apply to data offices in other organizations.
What are Data Products?
To be clear, even a real data product is digital, but the benefits can be seen and felt in every way. The best description I’ve seen is in MIT Sloan Management Review, which describes digital offerings as “information-enriched solutions wrapped in a seamless, personalized customer experience.” If anything, it’s better than a real product.
Think of it this way: in an online store, you get not only the item of your choice but also helpful information about the item (description, reviews, specs), recommendations tailored to your buying patterns and browsing habits, details on possible savings, reminders on what you might still need, and so much more. This is a data product powering your shopping experience — and the better the recommendations, the more you’ll buy. It’s good for you and good for the store.
Data products can take many forms. As Mike Ferguson illustrates in this graphic, we can have data assets, SQL queries, reports and much more. I think of it as a spectrum. On one end, we have basic, generally manual data products, such as a self-service report. It usually requires some human intervention: the person consuming the report has to receive, review and analyze it, then perhaps make a decision and take further action. On the other end are advanced options, such as recommendations generated through a machine learning algorithm. In between those lie plenty of variety, with offerings based on everything from internal skills to industry priorities. (Imagine if the recommendations didn’t show up as native suggestions seamlessly integrated into the online store, but instead appeared in a PDF highlighting specific details deemed most relevant, such as the most popular items of the week.)
Let’s remember that we’re still early in the data era; most organizations only have enough maturity to handle low-hanging fruit like basic data products. The more advanced options require change: training, new infrastructures, function-specific applications and adaptable processes. If they’re not here yet, they’re on the way.
How does Collibra create Data Products?
Collibra is proudly a creation of the technology era: from our humble origins (in 2008), we’ve developed and refined approaches to create products. We have mechanisms in place to identify the features most needed by the market, guidelines to follow during the development process, and specific procedures to launch, deliver and support each solution throughout its lifecycle. We adapt our product management practices to every data product.
For now, let’s zero in on two aspects of data product development: functional and technical.
On the functional front, the goal is to identify market needs for possible capabilities and establish a priority ranking for each. The data office exists to serve business needs, so we focus on specific business problems, evaluate ways in which data and analytics can help solve those problems, and develop objectives for each product accordingly.
At Collibra, these processes have matured over time: our Data Office has built relationships throughout the business, conducts ongoing interviews to monitor trends and trouble spots, and cross-checks findings with Objectives and Key Results in the performance management solution. We prioritize transparency and collaboration to identify potential data products and map them in a simple matrix to visualize cost versus value. (No data office has unlimited resources; let’s make the most of what we have.)
On the technical side, here’s one way to break down what a data product needs.
- Data: Yes, a data product needs data — from business applications (e.g., manually entered into the CRM), log files (e.g., generated automatically from different apps), sensor readings, third-party sources, social media channels, etc.
- Data Platform: Data requires an infrastructure for storing, moving and processing. We’ve set up a cloud-native data platform to ingest data from source systems; build, test and deploy pipelines; monitor data products in production, etc.
- Data Pipelines: Once the data lands in the platform, and after ingestion, we need to process, move and blend it with other data. These pipelines feed fresh data to the data products continuously. Yummy!
- Insights, Decisions, Actions: If a data product can’t deliver insights that lead to decisions and actions, then it’s not worth much. At the very least, it must deliver insights, even a basic report that delivers new information to the consumer. An advanced data product can make decisions and deliver actions automatically (e.g. recommendations for purchase).
- Marketplace: The Collibra platform features a Catalog that already hosts data products such as raw or curated data sets (e.g., registered on ingestion into the data platform) and reports (e.g., synchronized from Tableau). This is the perfect place to display data products for wider distribution: metadata, tags, ratings, comments, owners, etc.
How can other Data Offices develop their own processes to build Data Products?
This is how it works for us; here are some ideas for you on your Data Intelligence journey.
First, remember that data products solve business needs and that only comes with company wide collaboration. Every data product should have an ‘owner’ to clarify needs, assess value, determine requirements, etc. Manage expectations with communication and prioritization; working in isolation is a recipe for failure.
Second, collaboration might mean deep involvement in the business. Find partners in HR, Finance, Sales and other functions who can advocate for change and drive adoption. The technical issues are vital here — as mentioned earlier, the online store or app recommendation must show up natively or the process will create friction.
Third, envision and build a data platform that can support both basic and advanced data products. If your organization has a data warehouse or data lake, it might be possible to leverage those infrastructures; if not, marshal resources to build a sophisticated foundation that is adaptable, reliable and always on. (Even if that’s not necessary yet, it will be soon.) Also, make sure it’s a cloud-native architecture, and feel free to borrow best practices from software and data engineering disciplines.
Finally, keep the data front of mind — no data product can function without a reliable and trusted supply.
Now, go forth and let the data products roll.
