Page 178 - Building Big Data Applications

Chapter 10 Building the big data application 177

process to observe the outcomes in a slower state of study. This will provide two
immense benefits, the first is the research, which will validate the need for a better use
case, and will deliver instant results for a litmus test observation. In the nest practices
approach, this should be the first step taken for any application which will lead to
effective storyboards to be created and used by the teams.
Requirements for all applications are driven by the business experts as they have deeper
insights into what the enterprise is looking into and wanting to deliver. In the applications
world driven with infrastructure like Hadoop or NoSQL, the requirements are needed once
the researchoutcomes are shown bythe teams.The reasonfor this approach istofirst isolate
the conditions that cause the most impact, and then expand the requirements on those
conditions. We learned this lesson in implementing precision medicine initiative, where
every patient isspecifically isolatedand treatedbyusing their own genes which are extracted
and mutated in labs, and upon seeing the impact closest to the desired outcome, we inject
the mutated gene back into the patient and the outcomes are positive. The requirements
were solidified once the outcomes were understood; the same way if the research outcomes
are clearly explained to the business experts and executives, their questions will be a better
ask. We can stop worrying about that “yet another BI project” finally. The data outcome of
the research project will help in managing the requirements with examples and outcomes
desired, which is another step to the storyboard.
Log file collection and analysis is another process that building the application in the
new world of data mandates. The reason for this requirement is the ability to collect logs
as we execute applications is very easy to collect and once the logs are there, running
analysis on them will provide the teams a better understanding of what succeeded and
what needs to improve. In case of failures, which will be happening, the log analysis will
clearly point out where the failure occurred and it will aid in correction of the failure.
This is very much the use case for treating patients in the world today where diabetes has
become so common and often not treatable with all types of medicines and injections.
The doctors have a choice of inserting an IOT sensor that will keep a 24 7 365 trace
of sugars in the patient, and by this observation, they can treat the patient with the right
set of options to help them manage the diabetes.
Data requirements will be generated from the research projects and log file analysis.
This is a key differentiator in building the big data application, which will provide you
the best insights into areas that you are interested in understanding for the betterment of
outcomes. This data can be structured, semistructured, unstructured, internal or
external, picture, video or email, and can be used once or multiple times. The founda-
tional element is to build the storyboard, you will have the most accurate and precise
data that is needed. Metadata and master data associated with this data needs to be
documented and added to the storyboard. The reason for this requirement is the same
data can be called by different users with different names and it needs to be classified for
each team with their specific naming standards.
The next section discusses the actual design, development, and testing of the appli-
cation. The teams that participated in prior exercises of storyboarding, research, and log