The Office of the Secretary of Defense (OSD) and their Comptroller Office of the Undersecretary of Defense are responsible for US defense policy, planning, resource management and program evaluation. In carrying out those duties, the OSD’s day-to-day decisions impact a wide range of operations, including human resources, weapons acquisition, research, intelligence and fiscal policy across all of the US armed forces (Army, Navy, Marine Corps and Air Force). Steering an organization of that size means data is crucial to its decision-making processes. However, ensuring decision makers have access to the right data, can trust in its accuracy and understand its context is by no means a simple task.
Addressing the challenge to become data-driven
Recognizing that challenge, Greg Little, Director, CFO Data Transformation Office, partnered with Collibra to launch their Advana (Advancing Analytics) Data Catalog to create a centralized platform for data and analytics across the organization.
Utilizing the technology provided by Collibra, the Advana program directly responded to the need for users and stakeholders across the OSD to find, understand and trust their data, ultimately empowering and enabling better decision-making.
Little saw the Advana program as a strategic initiative with a focus on eventually expanding the program to encompass numerous use cases across the OSD. Little chose Collibra Data Catalog as the technical solution because of its enterprise capabilities and flexibility.
However, the value of the Advana program wasn’t realized overnight. It’s an ongoing process that began with a targeted implementation and internal champions with experience to help drive the program’s success.
A targeted implementation
Through the Advana program, Little empowered the OSD team to identify various opportunities for data cataloging across the organization. This strategy helped the team focus the implementation on a specific use case and then expand from there. Little also worked closely with the senior leaders to understand their data governance and data catalog needs and then built out a strong data team within the OSD to bring his vision to a reality.
When Greg Little hired Matt Piester as their Data as a Service Lead for the Advana program, the program was still in its infancy. Using his past experience working in different roles across numerous governmental organizations—a submarine officer in the Navy, a Defense contractor while serving in the reserves, several years in the Department of Health and Human Services, and now at OSD—Piester saw the value of this program and identified opportunities to communicate and implement this technology across the organization.
Getting started with driving the success of Advana at OSD, Little and Piester identified some immediate hurdles to overcome. The biggest challenge? Scoping down the project to narrow in on a singular use case. More specifically, Piester recalls, “it is important to start with a specific use case and find answers to key questions from decision makers.”
At the time, the OSD team determined that the primary use case was financial management. The CFO’s office needed to be able to track spending and carry out their audit function more effectively. By cataloguing all data sources in scope of the OSD’s financial audit function, the team helped simplify what had been a very challenging process. Collibra Data Catalog provided a “single source of truth” for the CFO’s office and enabled financial analysts to have a complete view of the available data within the department.
But preparing for financial audits is just one use case. As the CFO’s office began to get more comfortable with Collibra Data Catalog, Little and Piester looked to reorganize the data in the catalog based on different lines of business such as procurement or HR. To support that goal, Piester looked to catalog various data sets in Collibra by having the data ingested into the AWS infrastructure automatically populate Collibra Data Catalog.
Piester’s tip for implementation: Don’t bite off more than you can chew. Start with a singular use case that you can really tackle and find success, then scale out from there.
By succeeding in that first use case, the financial management team not only became early adopters of Advana, but also advocates for the program and keen to share their experience to drive further adoption across the organization. Piester applied the same framework used in the financial services case study to different data types across the larger organization.
Tackling additional use cases and driving adoption
It is not easy to scale a data program across a large organization. As Little and Piester began expanding the Advana program to include different use cases, they quickly discovered that different groups were at varying levels of maturity from the perspective of Data Intelligence. “It has not necessarily been a one size fits all approach, because other lines of business are at different phases in their data journey,” says Piester.
But with Little and Piester’s past experience, they were able to effectively communicate across the organization and found a common goal among the teams within the OSD. They understood the needs and the challenges that many of their stakeholders had on a day-to-day basis and knew that every team was trying to solve for essentially the same problem regardless of their level of data maturity—finding, understanding and trusting their data.
Piester’s tip for adoption: It’s not always a one-size-fits-all approach when it comes to driving adoption. Some groups are at different maturity levels in their data journeys. To gain that initial buy-in, find a common pain point or challenge to solve for and/or a common goal, then implement your model to their use case and customize as needed to help them realize value. Once they realize value, leverage that group as advocates as you continue rolling out your program.
With this common goal in mind, they were able to apply the structure of the financial use case to these different lines of business and make adjustments as needed to customize the use case. In that process, he recognized that this first-implemented financial use case can serve as a model for what other teams can do with a data catalog and he could leverage members of the financial management team as advocates for the Advana program across the organization.
Two approaches to measuring success
The Advana program is still relatively nascent, but has already made significant strides in its mission, with the team tracking key performance indicators to ensure they remain on the right path and can measure their progress. The team approaches measuring the success of the program in two ways:
- Quantitatively: They categorize success as usage and uptake. More specifically, they look at how many people are currently using the data catalog, who is using it and how often across the entire organization.
“Our metrics so far have been primarily around usage and uptake within the catalog. Just eight months after launch we now have almost 1000 registered users,” says Piester.
The team is also interested in how often a user accesses Collibra Data Catalog. Do they access the catalog daily, weekly or monthly? This is an important statistic because Piester considers a successful implementation and adoption as one where people rely on the data catalog for information, rather than asking a colleague or using a spreadsheet. He sees their success illustrated in the above-mentioned financial audits use case.
- Qualitatively: The team is currently working on a way to measure data quality. The team is putting together a data quality framework that not only aims to gauge levels of data quality but that also makes that information transparent to consumers and to help drive improvements. This data quality framework is developed and the team is planning to test its accuracy heading into 2021. To Piester and the team, this metric is crucial because it is important for users to know their data is high quality, trustworthy and accurate.
The broader mission
With a solid foundation in place, there are two ways in which the Advana program is set to evolve – both internally within the OSD and externally with its partners across the Department of Defense.
“Internally the focus is around increasing uptake within the community, making the catalog more comprehensive and making it part of business analysts’ day-to-day workflow,” says Piester. To support that goal, the team is seeking to catalog data sources beyond those currently aggregated and analyzed in Advana. Collecting pertinent metadata will help users know that data sets exist, even when accessing them may be more complex given their sensitive nature.
To succeed in that broader mission, the OSD team is working closely with colleagues across the Department of Defense, who have similar goals and who have initiated similar efforts in cataloging their data sources. Through this collaborative approach, the goal is to create a “federated data catalog” that spans key data sources across all organizations. Aligning these strategies will allow for the joint development of a strong data catalog and data management strategy across the entire Department of Defense.
“Ultimately, it should not necessarily matter who owns the data or where it is collected. I, as a user, will be able to come to Advana—or the Army, Navy or Air Force version—and know exactly what data is available, and we can ensure everyone is looking at the same universe,” says Piester.
With ambitious goals in mind, the OSD team is looking forward to accelerating their data strategy with Advana in 2021 while continuing to provide value to users and stakeholders across their organization.
