Every business wants to do more with their data. At the same time, terms like digital transformation, big data and service-orientated technology have been flung around for so many years, it can be difficult to understand their real-world value.
Instead, we should focus on what we want to achieve. When I hear the phrase ‘digital transformation’ from a company, that tells me they want to create better user experiences. When a company talks about big data, I see a business that wants to improve how they collect, analyse and use data. Service-orientated technology is a little more technical and covers distributed computing, but the result is a company that can develop, release, iterate and maintain software faster and with more agility.
Making data the star
Using data in this way involves making use of cloud computing. Rather than simply lifting everything into the cloud, you can now make use of ‘cloud-native’ services that can take advantage of new and more dynamic environments. We’re moving away from the hardware world of racking up servers and bringing up ports to having large scale infrastructure and this mesh is communicating with each other. Each of these elements has layers and they are all being watched, maintained, monitored and secured.
However, we have to go further than this. For developers, they could care less about individual infrastructure elements, even if they can deliver scale and availability for their applications. Instead, we should be looking at how to make it even easier to work with data over time. How can new features be delivered in a single sprint and not in months.
This involves another layer of abstraction, but by using automation we can make it easier for everyone to work with data, rather than having to understand and run the databases themselves.
It may be easier to think of this approach as similar to self-driving cars: while trials are taking place around autonomous vehicles, we have not got there yet and it will need to develop over time. Instead, what we do have is more assistance in place to make driving easier and safer automatically. For example, if a car pulls out in front of me, my car can brake and stop before I can even think about it.
For cloud-native databases, that same level of automation can help deliver more automation and guidance on how things should be running in the background. Rather than relying on individuals to configure and run these installations, automation can help put together the most appropriate and efficient set-up, then make recommendations on what to do next if there are changes needed.
Working at speed around data
In real-world terms, what all this automation does is make it easier to innovate, and enable a business to match its infrastructure to consumer patterns. A good recent example of this is the explosive growth of kerbside pickup and delivery. Almost overnight, this became a critical service for consumers during the pandemic. For large retailers like Home Depot, Walmart, and Target, in-store pickup went from a niche offering to being the most used service for customers in the second quarter.
As an example, Home Depot had to launch this in thirty days as COVID took hold. The company saw sales across its digital platforms increase by approximately 100% in the quarter, with its new pickup service being used more than 60% of the time. Home Depot’s SVP of Information Technology, Fahim Siddiqui, told the Inspired Execution podcast it had “everything to do with having the people, process and technology basis to scale out and really provide unparalleled service.”
Making the move to cloud-native
What we’re seeing in this data-driven world is more automation, based on the reliability and ubiquity of cloud. However, this goal relies on how data and IT operations management tasks can be handled in the background. DataOps is almost the last frontier for large scale application infrastructure. What that means is that DataOps is going to be more of a service; you’re probably going to be renting your data operations, you’re not going to be deploying it. It will be cloud native, so wherever you are in the world and whatever you need data for, it’s available.
This will rely on automation to help us make better decisions, save money and make sure everything is as efficient as possible. It will ensure infrastructure is in the right configuration for the use cases that we have.
The world we’re living in right now has the database at the bottom layer or what we call the backend. But it is affecting everything: developers want APIs, the queues go to events, microservices and queries, every part of this should be dynamic, scalable and self healing. If you had this block diagram deployed as your application infrastructure. I think you would be very happy, because you can answer questions, you can respond to changes in the world, and you can grow as you need to. If the business wants to extract value out of another part of your business you have the ability to do it quickly.
Patrick McFadin is the VP of Developer Relations at DataStax, where he leads a team devoted to making users of Apache Cassandra successful. He has also worked as Chief Evangelist for Apache Cassandra and consultant for DataStax, where he helped build some of the largest and most exciting deployments in production. Previous to DataStax, he was Chief Architect at Hobsons and an Oracle DBA/Developer for over 15 years.