Page 206 - Building Big Data Applications

206 Building Big Data Applications

scientist is no doubt interesting, for many organizations, doing so is simply getting too
far into the weeds.
AI is a combination of frameworks, algorithms, and hyperparameters, and these are
all the foundational elements of AI and ML. The underlying goal is the mode of usage
and implementation of the ecosystem. You need to decide if you are the driver, pas-
senger or the engineer that will work with the algorithms. To be an engineer, you need to
learn and experiment the algorithms, which is a deep dive into the subject. Today most
of the end consumers of these technologies just want to drive, and some people only
want to be passengers. Using AI in a “driver” or “passenger” capacity is possible; you just
need to know where to look. Armed with that knowledge, you can be more intrepid about
taking on AI in your own organization.
The passengers on the AI journey can benefit from the power of machine learning
without having to learn its rigors, which is deep and wide. That’s because many tech-
nology products now have AI built-in. For example, Microsoft has added an AI feature to
Outlook that sends you a daily message summarizing actions that you promised to take
in previous emails. Essentially, such applications capture to-do items from what you
wrote without requiring you to enter those as tasks in any explicit way.
If you would like to drive and not just ride, to get closer to the AI technology, yet still
not have to immerse yourself in the discipline, the major public cloud providers each
offer a number of “cognitive services” to help you do this. For example, Amazon
Comprehend can perform sentiment analysis on plain text; Microsoft offers its computer
vision service for image classification, activity recognition and more; Google Cloud Video
Intelligence extracts metadata from videos; and IBM’s Watson Language Translator
translates text from one language to another. These cognitive services use machine
learning models that are already built and trained by the cloud providers themselves.
The gas tank is already full, but would you rather choose your own fuel? You might look
for a cloud offering to which customers can bring their own training data. These avoid
the need for customers to imagine a predictive scenario, pick algorithms, and tune pa-
rameters but still provide a customized offering.
It is good to be a casual driver and get from A to B on your own. But what if you want
to drive a stick shift? Going further down, but still staying above the hardcore data sci-
ence fray, are companies that choose to develop “Auto-ML” systemsdsomething we do
at my company but that organizations can pursue independently as well. In such in-
stances, developers (not data scientists) can supply a data set; specify which column
within it represents the value they would like to predict and which columns they see as
having impactful input on that prediction.
Auto-ML systems can take it from there by selecting the best algorithm and param-
eter values, then training and testing the machine learning model that results. These
systems let developers create their own models without needing to become ML experts.
There are challenges that are to be overcome in this journey with AI. The challenges
hover around the same three areas of people, process, and technology. However, the
solutions are more deliverable as we have a new generation of engineers who do not