The unfulfilled promise of data analytics (part 1)

There is an elephant in the room when it comes to data science. More companies than ever are making big investments in data, but the truth is, data science projects often fail.

Despite having more access to customer data than ever before, you are not alone if you’ve completed a data project that left you with more questions than answers. Based on my observations in this space, here are three reasons data projects do not succeed and how to avoid these pitfalls when starting your next data project.

‘What we’ve got here is failure to communicate’

People who are technically oriented are typically wired to be inside-out thinkers; in other words, they start with a focus on the details. It’s not unusual for business-oriented types to be outside-in thinkers: They start with the big picture. It’s fair to say that each side often approaches the other with a sense of foreboding.

It’s imperative that there not be a communication gap between the data science team and the project owner (i.e., CMO, COO or CEO). Communication is a two-way street. Data scientists must be able to present results in a way that stakeholders can grasp. The project owner shouldn’t be left feeling like they need a data science degree to understand what is happening with their data project. In turn, the data science team needs the business owners to try to understand technical details beyond the buzzwords of the moment, such as “big data,” “machine learning” and “AI.” Clear communication is imperative to a successful data project.

It’s not my job

The number of data science projects I’ve seen without clear, measurable objectives is disheartening. I’ve concluded that residue from the communication gap is the likely culprit here, with both data science teams and business owners reluctant to take the lead on defining success. The translation of business requirements into technical requirements is often the sticking point, and that leads to diminished outcomes. A business requirements analysis can help everyone avoid a mismatch between what is being analyzed and what is actually needed. A project manager can be instrumental in guaranteeing that the project serves a specific business objective.

Frustration can also encroach on a project when the wrong questions are asked. When the data team truly understands the business, they don’t just wait for the business to ask for a problem to solve. They dig deep and ask the business questions that push everyone to take a step back and make sure the data project answers the right question instead of the client’s question, distinguishing between what is asked and what is needed.

Garbage in, garbage out

I’ve seen projects fail due to data quality challenges, particularly with data scientists who have recently graduated. In school, these recent grads generally had the luxury of working with “clean” data because the focus was on the concept being taught. The reality is, the real world is not like this, and data is almost never pristine or organized in a way that is conducive to the analytical construct.

While it may not be the “fun” part of a project, there’s often some work to do with the data before digging in to answer the big question, assuming it’s been identified (which is a big assumption). Data quality is a significant factor in a successful data project, and if the organization doesn’t take the time to upgrade to a modern data strategy for one source of truth, the project will not succeed.