Page 199 - Building Big Data Applications

11

Data discovery and connectivity

In the world today, data is produced every second and can drive insights that are beyond
the imagination of the human threshold, which is funneled by a combination of artificial
intelligence, augmented neural networks, backend platform augmented intelligence,
streaming data, and machine learning algorithms. We have tipped the threshold of data
and analytics processing today with the integration of these tools and technologies. The
rate of accessing data as soon as it arrives or even as it is produced to provide insights
and impact has evolved into an everyday expectation. This evolution has been even
observed by the industry analysts and they are calling it an evolution of business in-
telligence with augmented intelligence within the solution of business intelligence.
Call it whichever way it makes most sense, data discovery as a process has changed
the way we work with data. Imagine if you can identify the data based on a pattern or add
newer patterns as they manifest, which makes it easy for analysis prior to integration, we
will save 20%e30% of work that is done in the traditional world today. The impact of
cloud as a platform is very large and to realize the overall business value, speed is very
important, and this platform capability will deliver that requirement. The management
of data across the stages of data processing has been changing, and this change requires
several steps to be added and modified in the process to ensure constant insights are
delivered. These steps have disrupted the traditional processing of data in the enterprise
data warehouse, and the disruption is creating a new wave of analytics and insights
which are preintegration level insights in many cases and are very useful for predictive
and prescriptive analytics which suffered due to lack of availability of data at the right
time to create and deliver powerful insights (Fig. 11.1).
Once, centrally administered data warehouses were the most authoritative and
comprehensiverepository available foranalysisofcorporate information.However,the data

FIGURE 11.1 Data management in the digital age.

Building Big Data Applications. https://doi.org/10.1016/B978-0-12-815746-6.00011-9 199
Copyright © 2020 Elsevier Inc. All rights reserved.