Recently, our Head of Data Office, Jay Militscher posted a very helpful general overview of data mesh. Although the four pillars of data mesh are inextricably linked, this blog will focus on the first pillar: domain-driven ownership.
Where experts control the ecosystem
The data mesh framework is social and technical. It assumes that one of the primary challenges of managing analytical workloads with legacy architecture is knowledge. And repositioning the domain expert — as opposed to a centralized source — at the center of the data ecosystem is data mesh’s first principle.
Four data mesh principles
- Domain-driven ownership
- Data as a product
- Self-service data infrastructure
- Federated computational governance
Domain-driven ownership recognizes that marketing knows marketing and sales knows sales, and the centralization to facilitate analytics is inefficient. When organizations pump everything into a central data lake, it separates the data from the subject matter experts, which decreases productivity. Innovation slows. Organizations become stagnant.
‘We’ve seen the data warehouse evolve, big data platforms cluster, data lakes streaming and all of the above rapidly commoditizing in the cloud,’ mentions Collibra co-founder Stijn “Stan” Christiaens. ‘No matter how advanced the technology, the consistent obstacle has always been business ownership.’1
So the first pillar of data mesh puts domain experts in charge of their data ecosystem. In the data mesh model, the domain experts not only control the data ecosystem, they are also skilled at leveraging data. Since they know the context and the business priorities, they’re responsible for cleansing, enriching, and making data readily available to data consumers throughout the organization.
From a business perspective [decentralized ownership] makes things easier as it maps much more closely to the actual structure of your business. Domains can be followed from one end of the business to the other. Each team is accountable for their data, and their processes can be scaled without impacting other teams.
– James Serra, ‘Data Mesh: Centralized ownership vs decentralized ownership
These domain owners establish and maintain the quality of the data and provide necessary facts and documentation. Centralized data offices no longer take this on. This removes friction and streamlines time-to-insight simply by virtue of commingling data with the business talent.
However, domain-driven ownership is only the first pillar of a four-pillar framework. The dynamics of organizational evolution is unique for each company. In fact, our experience at Collibra is an interesting case study in the evolution of a data mesh organization.
Collibra: Domain-driven ownership at our core
Even before Zhamak Dehghani coined the term ‘data mesh’ in 2019, our co-founders Stijn “Stan” Christiaens and Felix Van de Maele subscribed to an organizational strategy that looked a lot like data mesh, especially when it came to domain-driven ownership.
At Collibra, the approach to building out internal enterprise analytics was to staff data analysts inside the business department, rather than as a centralized team. And this worked for a long time as a guiding principle. Sales analytics people sat in Sales. Marketing analytics people sat in Marketing. And so on.
However, what became clear was domain-driven ownership without the other three pillars of data mesh couldn’t scale, which we believe is one of the key metrics to evaluating data mesh.
Read how Cambia Health Solutions improved member experience and established data trust with Collibra’s data quality and governance solutions.
Learn from their success story now!
Meshing it up with the Collibra Data Office
While domain-driven ownership drove productivity within business units, over time we discovered there were limits to this approach without a broader framework for sharing data across the organization.
We had built a discipline within the company of high-performing distributed expertise. But discoverability was limited. Everyone was a little bit on their own and doing things their own way. Sales analysts created sales analytics in Excel. Marketing did theirs in Google Sheets. Finance in yet another application. It became clear that we needed to rethink how we enabled analytics and the analysts that supported this critical capability.
In 2020, the Collibra Data Office was created with a mandate to create ‘data ecosystems’ so our data analysts across the organization could truly excel in their roles. We leverage data mesh as our framework for advancing our mission.
Beyond domain-driven ownership
What’s been clear in our multi-year data mesh journey is the interrelationship between the four data mesh pillars.
While we’ll focus on pillars 2 through 4 in future blogs, it’s helpful to understand the dynamics of the four-pillar model. At Collibra, we developed a deep domain-driven expertise — and many of our analysts viewed their work with the data-as-a-product lens, which is the second pillar of data mesh. But we didn’t have a self-service infrastructure or federated computational governance. And this limited the reach and impact of our analysts’ work.
Today, we are actively operationalizing a comprehensive data mesh framework. And we’re using our own product — Collibra Data Catalog — to do it. How great is that? We get to activate data mesh and help our data analysts do their jobs better, and we’re using our own product to drive discoverability, governance, quality, and much more.
The journey to data mesh
Wherever you are on your data mesh journey, it’s important to understand that it is a process. You begin with the strengths of your current organization — and build.
As ever-greater volumes of data are managed in a centralized repository, it’s the Data Office or IT who are tasked with the overwhelming responsibility of managing, curating, and delivering massively complex data sets that are only getting more complicated by the day.
If you’ve reached the point where your centralized monolithic model is creating real impediments to discovering, understanding, and leveraging data to its fullest potential, then data mesh offers a useful corrective framework.
Check out our The Data Download podcast to learn how you can put your organization on the path to data mesh.
____
1Forbes, ‘Data Mesh: A New Hope For Data.’ March 2022.
