Data Culture ‘Disconnect’ Identified in New Index

(FGC/Shutterstock)

What does it take to succeed with data and analytics? Establishing a solid data culture, for starters, is often cited as a common characteristic among organizations that are successful with data and analytics. With that in mind, data catalog vendor Alation set out to quantify this quality by measuring various aspects of data culture with its new Data Culture Index, or DCI.

Alation introduced its DCI in its first State of Data Culture Report, which it released today. The DCI and the report reveal several interesting aspects of data culture as it exists among the 300 people who took the survey last month, including a curious “data culture disconnect,” according to Aaron Kalb, a co-founder of Alation and its chief data and analytics officer.

This disconnect reflects a gap between two main measures included in the State of Data Culture study. For starters, Alation asked folks to grade their data culture development, which is a “top down” measurement. Only 12% gave themselves an A, while nearly two-thirds gave themselves a C, D, or F.

Then there was more of a “bottoms up” measure, which is the DCI itself and accounts for the bulk of the survey. This involved more in-depth questions that sought to measure the relative maturity of three critical aspects of data culture across individual departments in an organization, including: data search and discovery; data literacy; and data governance. These are the three main measures of the maturity of data culture in an organization, Kalb says.

“The interesting finding is, when we say ‘How’s your data culture?’, we actually see a lot of grade inflation,” Kalb says. “People aren’t necessarily coming to the same place when asked overall, top down, compared to when we ask bottom up.”

The DCI measures three elements of data culture on a score from 0 to 4 (Source: Alation State of Data Culture Report)

A perfect score on the DCI is 12, and the worst is 0. About one-third of organizations have a DCI of 8 or higher, one-third have a DCI of 6 or 7, while the bottom-third have DCI’s of 1 to 5.

According to Kalb, the organizations that scored lower of in their bottom-up DCI scores tend to think they’re actually doing pretty good with data culture. This reflects a Dunning-Kruger-life effect, where those with the least ability were the least likely to recognize their own lack of ability, he says.

“So in some ways, the worse you’re doing at data search and discovery, data literacy, and data governance — the cornerstones of data culture — the more likely you are to say, ‘I’m actually an A-minus,” he tells Datanami.

It’s a new take on that old empirical gotcha: You don’t know what you don’t know.

“My hypothesis about that,” Kalb says, “is if you’ve really been working at this and putting in all the blood, sweat, and tears to get 3s and 4s across the category, you realize how hard it is and you get a sense of the shape of the curve, whereas folks who are earlier in their journeys, maybe it doesn’t seem as daunting yet, and they’ll be in for a bit of a surprise.”

Aaron Kalb is the chief data and analytics officer at Alation

There was another disconnect revealed when it comes to executives and their relationship with data. In general, executives tend to think they are data-driven. That tends to be how executives describe themselves. But according to the DCI, two-thirds of respondents said the executives in their organizations “sometimes, often, or always” ignore the data.

After gathering, preparing, and analyzing fast sums of data, it would seem, many executives just go with their gut and ignore what the data says, the DCI suggests.

“I don’t think anybody would admit it, that ‘In my organization, we collect a lot of data, and then we shred it and go with my gut.’ Nobody is going to say that brazenly,” Kalb says. “Executives don’t think they ignore data as much as people who work for them think they ignore data. And who’s to say who’s right there? It’s not shocking.”

There could also be other elements at play there. For example, executives may be ignoring the data because they don’t believe that it actually reflects reality, Kalb says. They may have been burned by bad data in the past, or burned by somebody misrepresenting good data, and therefore have a good reason to believe their own gut instinct.

That is all a part of having a good data culture. “Data quality was seen as the number one challenge to being data driven,” Kalb says. “I think if you invest in really good data search and discovery capabilities. Our customers often invest in a data catalog as a way to say, yeah there’s good data and there’s bad data. So if you can separate the good from the bad, then you can place more trust in data than if they’re indistinguishable.”

What’s holding organizations back from getting value from data (Source: Alation State of Data Culture Report)

Several other patterns emerged from the data, including that organizational structure appears to matter. There is a correlation between higher DCI scores and organizations where the chief data officer, chief analytics officer, or chief data and analytics officer (CDO, CAO, CDAO, respectively) reports directly to the CEO or the executive board, compared to those where the CDO, CAO, or CDAO reports to the CIO or the head of the IT department.

Whether there’s causality behind that, or it’s just a random blip in the first of what will hopefully be a long series of quarterly reports exploring data culture, has yet to be seen. “The hope would be that, actually tracking this over time, will let us make claims that more approach causality,” Kalb says.

Another interesting finding is that moving up the data culture ladder may be easier than some folks at the lower end of the spectrum may believe. We addressed the “unknown unknown” phenomenon earlier, but this one actually could work in favor of those in the bottom tertial of the DCI.

“There’s a few heartening observations,” Kalb says. “One is, among the bottom third of the folks on the DCI, what they claim were the biggest blockers weren’t things that are known to be super hard to solve, like data quality. But it was actually executive buy in. Whether that’s the hardest thing or the easiest thing depends on your organization. But it’s possible that putting a few people who really care about this and finding a champion might be all it takes to get to that middle place and get to those harder problems.”

Wakefield Research conducted the DCI surveys in August. It involved 300 folks at the director, executive, and officer levels at midsize-to-large organizations (at least 2,500 employees) in the US, the UK, Germany, Denmark, Norway, and Sweden. The plan calls for repeating the DCI every quarter.

You can download the first Alation State of Data Culture Report at www.alation.com/state-data-culture-report-2020/.

Related Items:

Testing Data Literacy on Main Street

VC Ben Horowitz Dishes on Hadoop, AI, and Data Culture

How to Build a Big Data Culture