The Future of BDA

Data Analytics would be central to transformation initiatives, paving a faster path from data to decision

Big Data Analytics (BDA) is defined as the process of analysing data to uncover patterns using computing algorithms, programming, and statistical modelling techniques to find valuable and timely correlations, resulting in actionable insights that drive business decisions inside an organization. The purpose of Big Data is to feed the analytics engines and therefore the two are inevitably connected.

Data explosion is going to be the hallmark of the future as more and more people around the world gain access to mobile devices. According to the Ericsson mobility report November 2020,global total mobile data traffic is estimated to reach around 51EB (exabytes) per month by the end of 2020 and is projected to grow by a factor of around 4.5 to reach 226EB per month in 2026.By then, over 6 billion people would be using smartphones, laptops and a multitude of new devices, said the report.

During the ongoing pandemic, BDA is being utilized to manage, diagnose, and develop a cure for COVID-19. A lot of big technology players are exploring options to increase the efficiency and reduce the time taken in drug discovery processes, including: IBM, SAP, CytoReason, SAS, Splunk, Hitachi Vantara (previously Pentaho), and XtalPi. Pfizer – the pharmaceutical giant at the forefront in the race for a COVID vaccine – is just one of the many companies who are leveraging BDA to create competitive advantage for themselves. Business analyst Frost & Sullivan estimates that the market revenue from BDA reached $14.85 billion in 2019; they predict it would continue to grow at a CAGR of 28.9% to reach $68.09 billion by 2025.The pandemic, and the way in which enterprises have responded to it, certainly appears to be exacerbating the divide between the ‘haves’ and ‘have-nots’ when it comes to data!

The new pace of business demands a faster path from data to decision to ensure that data and analytic plans are aligned with – and central to – transformation initiatives. This path from data to decision involves multiple steps, with the most significant being data ingestion and integration, data storage and processing, and data visualization and analysis.

Challenges however continue to prevent an even faster growth. Despite the drivers, overall market growth is restricted by the lack of customer data hygiene and data standardization. This makes it difficult for BDA customers to justify their return on investment (ROI), as they still scramble to manage and make sense of their data. Sunken investment in homegrown solutions, lack of skilled labour, and long sales cycles also restrict market growth.

Future Trends

According to Gartner,75% of enterprises will shift from piloting to operationalizing AIby the end of 2024,driving a 5X increase in streaming data and analytics infrastructures. Within the current pandemic context, AI techniques such as machine learning (ML), optimization and natural language processing (NLP) are providing vital insights and predictions about the spread of the virus and the effectiveness and impact of countermeasures. AI and machine learning are critical realigning supply and the supply chain to new demand patterns.

By 2023, more than 33% of large organizations will have analysts practicing decision intelligence, including decision modelling. Decision intelligence brings together a number of disciplines, including decision management and decision support. It encompasses applications in the field of complex adaptive systems that bring together multiple traditional and advanced disciplines.

It provides a framework to help data and analytics leaders design, compose, model, align, execute, monitor and tune decision models and processes in the context of business outcomes and behaviour. Explore using decision management and modelling technology when decisions need multiple logical and mathematical techniques, must be automated or semi-automated, or must be documented and audited.

By 2022, public cloud services will be essential for 90% of data and analytics innovation. As data and analytics moves to the cloud, data and analytics leaders still struggle to align the right services to the right use cases, which leads to unnecessary increased governance and integration overhead.

The question for data and analytics is moving from how much a given service costs to how it can meet the workload’s performance requirements beyond the list price. Data and analytics leaders need to prioritize workloads that can exploit cloud capabilities and focus on cost optimization and other benefits such as change and innovation acceleration when moving to cloud.

Data and analytics capabilities have traditionally been considered distinct capabilities  and managed accordingly. Vendors offering end-to-end workflows enabled by augmented analytics blur the distinction between once separate markets.

The collision of data and analytics will increase interaction and collaboration between historically separate data and analytics roles. This impacts not only the technologies and capabilities provided, but also the people and processes that support and use them. The spectrum of roles will extend from traditional data and analytics roles in IT to information explorer, consumer and citizen developer as an example.

To turn the collision into a constructive convergence, incorporate both data and analytics tools and capabilities into the analytics stack. Beyond tools, focus on people and processes to foster communication and collaboration. Leverage data and analytics ecosystems enabled by an augmented approach that have the potential to deliver coherent stacks.

By 2022, 35% of large organizations will be either sellers or buyers of data via formal online data marketplaces, up from 25% in 2020. Data marketplaces and exchanges provide single platforms to consolidate third-party data offerings. These marketplaces and exchanges provide centralized availability and access that create economies of scale to reduce costs for third-party data. To monetize data assets through data marketplaces, data and analytics leaders should establish a fair and transparent methodology by defining a data governance principle that ecosystems partners can rely on.