The secrets of a successful big data implementation

In the face of increasing austerity, the UK public sector must explore new ways of increasing operational efficiencies in order to generate cost savings, while improving the quality of services it provides to citizens.

[easy-tweet tweet=”The UK public sector must explore new ways of increasing operational efficiencies” user=”comparethecloud” hashtags=”bigdata”]

For the public sector, adopting big data and predictive analytics can deliver enormous benefits that reveal new insights and solutions to complex problems, such as medical advances that can save lives. Big data can also help drive efficiencies by enabling more precise targeting of resources and delivering improvements in citizen services.

Preparing your internal organisation for big data

To get maximum value from big data, internal organisation is a must, with at least a baseline understanding of its potential and the right governance in place to ensure the consistent building of solutions. An organisation needs to have the right resources in the right places. This may mean having data scientists in each department, or perhaps a science centre of excellence, depending on need.

An organisation needs to have the right resources in the right places

Managing the cloud infrastructure

Although big data solutions can be deployed internally, the volume and connectivity requirements can exceed the data centre capacity available; and it can be challenging for a non-specialist IT team to scale out big data infrastructure on demand.

Ultimately, the cost of ensuring the availability and durability of data managed in house tends to become prohibitive. For these reasons, many big data projects fail to realise their full potential, or may be abandoned. Cloud-based solutions, however, change the way that infrastructure is deployed and paid for, helping organisations break down the capacity and operational management barriers to big data solutions.

many big data projects fail to realise their full potential

Frameworks like G-Cloud enable public sector organisations to contract, order and set up cloud services in a matter of hours, putting an end to long procurement cycles. G-Cloud suppliers’ consumption-based commercial models and usage-based pricing make costs transparent, so budget and expenditure can be accurately tracked.

The open data question

Data that can be freely used by, modified by and shared with other public sector organisations is the most effective route to genuine insight and actionable intelligence.

The UK government has a well-established open data agenda with aims that range from helping people understand how government works and how policies are made; to making raw data available to enable people to create useful new applications.

[easy-tweet tweet=”The UK government has a well-established open data agenda” user=”comparethecloud” hashtags=”data”]

For an organisation to work effectively with shared data, it needs a highly available, highly connected repository with scalable capacity for storing and processing the data. Cloud computing is the most viable, cost-effective and rapidly deployable solution to this challenge. But organisations need to choose their provider carefully, to be confident that the private data stored alongside the public data will be secure, protected and accessible only by the organisation that created it and that it follows data protection regulations.

Selecting the right tools and technology

For public sector organisations, using open-source tools and technology is often preferable to their proprietary equivalents. The nature of open source helps avoid the vendor lock-in, which many have suffered in the past, finding themselves held to ransom on price or proprietary solutions.

With the chance that the technology landscape will have moved on before a proof of concept is finished, an organisation needs to maintain close contact with the open-source community, and keep up to date with toolset development and overall development direction.

However, there is a risk of encountering some dead-ends, committing time to a tool only to find the industry heads off in a different direction, leaving it unsupported and unmaintained. To minimise the risk of this happening, organisations should try to favour tools with a longer life which have been widely adopted. To reduce the risk further, tools where the contributors on the project come from more than one company should be chosen. Furthermore, organisations must make sure the technology choices support the business’ long-term aspirations, or can be extended to do so.

Deliver maximum ROI with big data

In order to demonstrate value to stakeholders, the programme should be structured to deliver short-term value as quickly as possible, and include a vision for the longer term, with an agile structure that’s responsive to change, enabling the capitalisation of new opportunities as they occur.

A big data implementation is often, in reality, a programme of business change, as well as a technology programme.

A big data implementation is often, in reality, a programme of business change, as well as a technology programme. Big data analyses can unlock insight hidden deep within data, which may challenge the organisation’s existing understanding of its users, its customers, or the environment and lead to change within the business processes.

To maximise the potential for success of a big data programme, a public sector organisation needs to start its big data journey by carefully assessing its business objectives and establishing a strategy before implementation. The most likely route to success is a business-first, technology-second approach.

Overall, there is a huge potential for government departments to share data in a federated way that will allow each department to be more efficient, and give all departments a better overall picture of a given scenario. By using scalable and open-source technologies, the public sector can make huge savings and look at use cases that were previously seen as too complex or costly. Adopting a big data strategy will also ultimately lead to a better experience for citizens when they interact with government services.

+ posts

Steve Dixon, Product Manager, Skyscape Cloud Services

With more than 20 years in the IT Industry, Steve comes from a background in ecommerce and cloud services, with roles ranging from IT service management through to product management.

Steve is an evangelist of cloud technologies and the transformational benefits digital adoption has to offer. He has an expert knowledge in innovation and thought leadership within IT, and has specialist experience in IT strategy, product management, project management, and using technology coupled with process improvement to drive operational efficiencies.

Unlocking Cloud Secrets and How to Stay Ahead in Tech with James Moore

Newsletter

Related articles

How AI is Transforming Customer Communication Management

Business communication has evolved over the years. Today, it's...

Investment Opportunities for Startups and Technologies in AI 

Although artificial intelligence developed from niche technology has become...

Four Surprising Lessons I’ve Learned Leading Tech Teams

Techies. Geeks. Boffins. Whatever your organisation calls its IT...

A Business Continuity Cheat Sheet

Right, let's be honest. When you hear "business continuity,"...

Challenges of Cloud & Ultima’s Solution to Transform Business

With the way that AWS and Microsoft dominate technology...