Kurt Drowning In Data

By Kurt Janssen

Data: we’re drowning in it.

It’s growing at an exponential rate - essentially doubling every two years. But, as Big Data guru Bernard Marr says, less than 0.5% of all data is analysed, and that number is shrinking. Not growing – shrinking!

That means that what we’ve got is a lot of information but very little insight, and comparatively miniscule knowledge about what’s really happening in our world and in our businesses.

That’s mind-blowing when you stop and think about it.

How did we end up in this data bog, and more to the point, how can we get out of it? How can we know which data is worthless, which data has value and how to extract that value?

Dual-edged sword

Here’s the kicker: to know whether your data has value, you first need to understand it. And to understand it, you need to visualise it so that it becomes human-relatable.

We have to get better and smarter at visualising data quickly to understand if there is value there or not. It’s only through this mechanism that you can get it in front of the user to validate its worth, ensure you understand where to spend limited time and on which datasets to spend scarce resources. Best bang for buck!

Here’s an example: imagine you’ve got a data set that shows the flow of people around a city. It’s a big investment to get that data set. But if it’s just sitting in a database and you’re not actually visualising it, then decision makers can’t see it to make decisions based on insights. How do you know if it’s valuable and how can you justify the continued spend on the dataset?

What we need to do is:

1) Hypothesise and use scientific theory to get in there really quick to understand the data, learn about the data, value the data and get it to the point where we can make decisions about whether it’s worth the investment.

2) Make the complex stuff simple so that people can understand if it provides knowledge and informs decision-making. It might be found to only provide “noise” to the organisation.

3) Most importantly, do the above quickly, cheaply, and at scale.

Define “valuable”

Data has value if it’s helping to solve key pain points in your business or if it is allowing an organisation to get closer to achieving its strategic goals and vision.

A good starting point is defining those pain points. Which data is important to those pain points and how do you test and hypothesise that quickly?

This doesn’t have to mean sinking in truckloads of cash to buy or develop software that you don’t need in order to find out that your data’s off the mark. This is where a quick and lean approach – a pilot project or a minimum viable product (MVP) – can quickly tell you what you need to know before you’re too far in the red. It’s a win-win situation: if your data is gold then a pilot or MVP can be rolled into a more substantial development that will enable you to extract value from the data on an ongoing basis. If your data isn’t suitable, you can stop investing in that data set or change the way you collect the data to ensure you’re getting what you need out of the investment.

Get tech and business talking to each other

Another major way to dredge the data bog is to have the technology side of your business and the business side of the business working much more closely together.

Think about your tech team. Do they understand what the pain points are in your business? Do your business and tech teams talk to each other about the problems that need solving?

Communication is everything. It’s easy to get siloed and there’s this thing about silos in a business that sort of self-propagates. You get an asset management system and you get experts around that. You get a financial system and you get experts around that. And because the systems are different and they often don’t interoperate well, and the people who run them have different expertise, it just become difficult.

You have to create and maintain bridges to have that cross-information flow of both data information and business uses and pain points. If you can work together to find solutions that are linked to pain points in the business, you are 100 times more likely to get buy-in from the bosses. That’s when you see the real value in communication because together, you’re solving problems that neither one could do alone.

That’s when your good data becomes great data. That’s the 1 + 1 = 10 scenario, where you see the potential you can’t see in any one silo. It’s a value multiplier.

The glue that sticks it all together

When you’re looking at all this disparate stuff and systems and data and problems, I firmly believe it’s geography and the location element that ties those things together.

Everything and everyone are somewhere on the planet. So, if you can figure out a way to layer up your asset management systems, your ERP systems, your financial systems and your GIS systems, it’s the geographic aspect that ties all those components together.

Geography is the glue that will hold information-rich organisations together in the coming decades.

One last thought - open data or accessible data?

Adding to the data bog is the whole open data paradigm.

Don’t get me wrong – I love open data. It’s really admirable and it allows other businesses to get different value out of data sets that were collected often with a single purpose in mind. Someone else with a very different set of problems comes along and says “hey, I can repurpose this data and actually solve this “other” problem.” So, the value for the community, for the nation, increases.

But there’s a caveat to that: 98% of the population aren’t geeks, developers or data experts. It’s easy to tick a box and say, “I’ve opened up data to my community,” but no-one can use it unless they are a data developer or geek or a company like us. If you can’t develop on a large CSV, a database connection or utilise an API, then even if you know about the open data you are not able to effectively utilise it to help solve your pain points. Its utility is almost non-existent to that individual or organisation. So open data is not reaching its goal of transparency now. But what if it was open, accessible data?

Rather than raw data, it would tell stories and display creative visuals that make it accessible to the general public and other stakeholders who want to see it, understand it, and gain insights from it.

“Accessible” could mean different things for different groups, right? It could mean a website. It could mean a map or a graph. But the main point is that it’s not a developer-focused resource. Someone can go to the open data and make a decision straight away based on the insights from it, instead of having to download, spend time, integrate it and build an application around it.

Visualising data with the intended audience in mind is when you truly start to extract its intrinsic value.