DataOps techniques are set to transform corporate data management processes in more ways than one.
With the global COVID-19 pandemic accelerating the need for faster, higher quality and resilient data and analytics in the face of rapid change, organisations have been tasked with making quicker and more well-informed decisions surrounding business processes including automation, continuous value delivery and real-time risk assessment and mitigation, with primary stress on agility.
A primary cause for this, of course, is the total amount of enterprise data businesses are generating — and the fact that it is rising by almost 40-60% on an annual basis. According to technologist Bernard Marr, “With requirements changing every day and the need for data access continuing to grow…organizations have to find ways to improve their data management processes.” This is where DataOps comes in.
DataOps techniques have been slated to be the primary means of providing a more cost-effective, agile and collaborative approach to the building and maintenance of data pipelines. Vice President of consulting giant Gartner, Ted Fried man believes, “the point of DataOps is to change how people collaborate around data and how it is used in the organization”, thereby making it an integral pivot in the analytics engines of firms worldwide.
‘Can we do this?’
DataOps is, at its core, an agile operations methodology — ‘a collaborative data management practice’, if you will — aimed at improving communication, integration and automation of data flows between managers and consumers across an organisation. By aligning data management tools and processes with data goals, it primarily focuses on altering the question from ‘Can we do this?’ to “How do we provide an optimized, governed data-driven product?”: thereby seeking to develop data pipelines so that actions become a collaborative exercise throughout the organisation (including technical experts such as engineers and data scientists) rather than throwing it at a ‘virtual wall’ for it to be someone else’s problem. Incidentally, research suggests that the use of DataOps also makes it four times more likely for organisations to hit their financial goals.
A major aspect of perfecting DataOps implementation is aligning processes with how data is consumed, instead of how it is created. According to Gartner, there are three major value propositions to be kept in mind in this process: (i) making data available to all as a utility; (ii) enabling businesses to make the right decisions using data insights and (iii) driving businesses forward using resilient analytics.
- “Adapt(ing)…DataOps strategy to a utility value proposition”: By focusing on removing silos and manual manipulation of data and stressing on creating central data repositories, data and analytics must be made readily available throughout the organisational structure. Gartner opines: “..because there are many relevant roles and not a single owner of the data, assign(ing) a data product manager to ensure data consumers’ needs are being met” will be crucial.
- “Us(ing) DataOps to support data’s use as a business enabler”: This involves the use of data and analytics in various specific use cases, including supply chain optimisation, inter-enterprise data sharing and fraud detection through collaboration with specific business unit stakeholders involved in the processes.
- “Support(ing) the data and analytics driver value proposition”: the use of data and analytics as a driving force in creating new products and services, entering new markets and generating greater revenue. Gartner lauds using DataOps in the development of optimised data-driven products to consumers from ideas borne in the laboratory.
Every business will have its own form of DataOps adoption: either a centralised version of all the aforementioned aspects, or decentralised deployment models for specific actions. According to Gartner, “Delivering DataOps using each value proposition will foster collaboration between stakeholders and data implementers delivering the right value proposition with the right data at the right time.”