Disparate Data – tackling data silos to unlock value

“Data is the new oil” – you may have heard this phrase branded about in the past 10/15 years as organisations start to harness and collect all the data that comes from their processes, customers and the like.

However, like crude oil, data (when it’s not refined) can be difficult to extract value from and you often get a lot of wastage and a lack of clarity on what it all means. When it comes to the public sector and local authorities taking their first steps into IoT and working with big data on a city-wide scale, this can cause major headaches and frustrations. This not only impacts those in the thick of managing those projects internally, but teams across a local authority who have increasing reliance on data to inform day to day decision making and manage multi-agency responses.

If you look at private enterprises, there are lots of examples of the use of platforms and data aggregation to improve data efficiency and allow them to understand their data more clearly, in a structured way. CRM systems, shared databases, monitoring tools and dashboards are all commonplace within these environments, but what are the major challenges for public organisations in taking similar steps and how can they be overcome?


Setting the Scene

This appetite to move to a data and analytics led approach isn’t necessarily new, even in the public sector. The McKinsey Global Institute estimates that data and analytics could create value worth between $9.5 trillion and $15.4 trillion a year if embedded at scale – and $1.2 trillion of that in the public and social sectors. That’s even in the context of recent McKinsey surveys suggesting that half of the respondents were still not using artificial intelligence (AI) anywhere within their organisations.

However, even with the value of data and analytics being touted at these huge amounts, the public sector does face a bigger and trickier challenge when it comes to adoption and value generation. UKCloud’s “The State of Digital and Data” survey in 2021 found that 97 per cent of UK public sector respondents are, at the very least, evaluating digital technology and its potential to improve the outcomes and services being delivered to citizens. Yet only half of those respondents believed that they have the resources necessary to understand and drive efficiencies from the data they have, meaning they can‘t determine its true value. So why is this the case and what are the key challenges that face the public sector when it comes to creating value from data?


Bigger and trickier challenges in the public sector

McKinsey’s research into the public sector suggests that whilst many have data and analytics strategies they usually share the common weakness in that they are too broad. Whilst it’s important to have bold aspirations as a starting point for transformational change, broad ambitions lack the clarity and measurable goals to maintain focus and gather internal momentum and support.

For example, if your aim is to improve public services without clear quantitative targets it’s easy to get caught up in collecting every bit of data possible from public services without accountability as to how that data is actually used for the benefit of the citizen or service delivery.

Data from UK Public Services suggest that this issue is more common than you might think, when asked where the majority of data resides, those surveyed demonstrated genuine uncertainty by providing contrary responses, suggesting a lack of complete data oversight. With the majority of organisations not dedicating more than a few days each month for employees to innovate and research ways to unlock more value from data, it suggests that there isn’t enough focus and alignment in discovering new ways of working and challenging the status quo.

This issue of siloed data seems to be a running theme with the British Standards Institute who provide guidance on Smart City projects within the UK, suggesting that silos are one of the key barriers to obtaining and using the right data in smart city projects. Their examples from councils suggest that even close colleagues are reluctant to share data with the lack of cooperation being caused by long-establish behaviours and not having common goals or objectives across teams to work towards.

So how do these challenges contribute to the success or failure of smart city projects?


Why we can’t keep working like this

Well, the statistics and data indicate that there are massive inefficiencies in the use of data and these behaviours and approaches aren’t complementing the “refining” of data or helping improve confidence in adopting new ways of working and using data to support decision making.

The rising costs of providing public services and the cost of living for citizens puts pressure on councils to increase council tax or find more efficient ways to deliver services to meet the increasing demands of citizens. So, wasting spend on collecting data or smart city projects that don’t deliver a return on investment in terms of time or operational savings will lead to new initiatives having a tough time getting off the ground.

Whilst intentions are there to adopt new ways of collecting and using data and analytics, local authorities will be hesitant in making major changes if the consensus is that it’s a waste of time and resources. So, it’s vital that councils take the right approach and share successes to open up the right dialogue and knowledge sharing across stakeholders.

BSI’s City data survey report highlighted 14 categories of datasets that are required to support city projects (see figure 1), suggesting the need for clarity of data across different areas of a local authority to make smart city projects a success.

Credit: BSI


It’s clear that the opportunity in getting a grip of your data is being able to take the “oil to the refinery” and establish an understanding from your datasets to unlock the value within. This means breaking down these silos, establishing clear alignment across teams and having clear quantitative targets for your organisation to anchor your approach to.


How do you get there?


Take a top-down approach

According to the BSI, vision and strong leadership are the two vital components that need to be developed to break down the silos and ultimately improve sharing and interoperability.

If behaviours are led and demonstrated from the top down, it’s more likely that these approaches will be adopted further down the organisation to encourage teams to work with one another, understand the benefit of collaboration and create a culture of inquisitiveness when extracting the value from your data.

Having a clear strategy that is communicated from the top and implemented through teams helps improve understanding across and organisation as well as create a data-led culture.


Start small before going broad

Small pilots aimed at a broad aim are likely to fail to scale as the goalposts have been set too wide.

Being clear on what you want to achieve and specific to what you’re measuring ensures that efforts stay focused and people across your organisation understand what will be gained. This helps that first step toward winning workforce support.

McKinsey’s research showed that many of the best-in-class organizations build a lighthouse—that is, they implement 10 to 15 use cases within one organizational unit or focused upon one topic. The concentration delivers change that can be seen, not incremental improvements, and so builds support for broader adoption.


Focus on the outcomes before the technology

A common issue we find when consulting with public organisations is that sometimes there can be an over reliance on placing technology for technologies sake. For example, “we need to have this type of IoT connectivity… or this type of hardware to collect specific data”. Trying to fit use cases with a certain technology is usually more problematic than finding the right technology to fit with a use case.

When you start with an outcome that you want to achieve you are much more likely to deliver on that outcome as you plan a solution with that as the starting point and it’s the north star that you’re constantly focussed on when building a solution.

Issues can arise when you start with technology as you are always trying to fit solutions into specific frameworks and this can lead to issues such vendor lock in, a lack of technology-agnostic solutions, more expensive projects and sometimes outcomes that look different to what was initially scoped. This, all in all, affects returns and questions the effectiveness of the data that you have and can feel like a backwards step rather than progressive digital transformation.


Focus on the outcome you want by framing your use cases as questions to establish outcomes rather than a technology to use/create Credit: McKinsey


How we can help

To provide some context as to how this relates to us at Connexin, we work with a number of public sector organisations to provide end-to-end IoT solutions to help them collect and understand their data in a way in which they can extract insights and outcomes from.

Often, we act on a consultative basis to help organisations understand the outcomes and value that they want to drive and build solutions and frameworks that give them the best chance of success.

In this case we can speak to your stakeholders, understand your challenges and propose how that would work across an end-to-end solution from how you can collect data (whether that’s from existing sources, sensors etc.), put in place the right infrastructure, bring the data together in a data platform, analyse the data to identify insights that aid decision making, and ultimately help you towards the outcomes you’re looking for.


Connexin’s end to end approach


If you are looking for support in your digital transformation or looking to better utilise your data, we’d be happy to have informal chat to discuss how we could potentially help you.

Fill in the form below and one of our team will reach out.