Page 122 - Building Big Data Applications

Chapter 6 Visualization, storyboarding and applications 119

Each sequence will create its own set of audit trails and the logs from these audits are
very essential for the teams to have access to and analyze. The audit logs can be accessed
and analyzed using Apache Flume or Apache Elastic or Splunk depending on what the
enterprise wants to accomplish during the analysis. This is where we will use machine
learning and neural networks the most as the logs can be varying in size, content and
actions. The processing of unsupervised data is more useful to learn from without
actually interfering the actual view of the execution. This flow is an essential step in
creation of the actual visualization for the team.
Each audit analysis will produce the exact user interactions, system interactions and
resulting steps that was the behavior of the system and the user. Think of this in research
with the data from CERN particle accelerators and the discovery of the “god particle”. The
actual scattering of the data and its outcomes were not executable repeatedly, however if
the logs are analyzed and outcomes understood, the discovery can be accelerated with the
right corrections applied. This type of activity is what research teams have always wanted
and have never been successful in applying their team skills together.

Let us look at some of the use cases of big data
applications

Clickstream analysisdwhen you engage in an Internet search for a product, you can see
that along with results for the product you searched, you are provided details on sales,
promotions, and coupons for the product in your geographical area and the nearest 10
miles, and also promotions being offered by all internet retailers. Analyzing the data
needed to create the search results tailored to meet your individual search results, we can
see that the companies, who have targeted you for a promotional offer or a discount
coupon. How did these retailers offer me a coupon? The answer to this is the big data
analytic application in the retailer which provides details on web users’ behaviors gathered
with the process of clickstream analysis. The big data application is architected to provide
instant analytics on any user or users’ behaviors based on specific geographies or multiple
geographies compared in a mash-up. To make these analytics work seamlessly and provide
multiple user connects with offers, the data integration in this architecture is complex.
What changes here is the actual mechanism of building this application itself.
For search to be executed, we need to have an interface that will use a search box and
a submit button. This is great as all search engines provide you an interface. What
happens next is the intricate process of why an engine like Google is a success while Bing
or Yahoo is a distant second or third. The application development process here has to
boil down to all levels of details. For search to happen, the following big data application
strategy needs to be written as foundations:

Step 0dFoundational web interface screen
Search text box
Submit button
Minimal logo