Page 173 - Building Big Data Applications

172 Building Big Data Applications

FIGURE 9.8 Search and
recommendation process.

3. John Doe also receives recommendations and personalized offers along with the
result sets.

How does the system know what else John Doe will be interested in purchasing and
how sure is the confidence score for such a recommendation? This is exactly where we
can apply the framework for machine learning as shown in Fig. 9.8.
The first step of the process is a user login or just anonymously executing a search on
a website. The search process executes and also simultaneously builds a profile for the
user. The search engine produces results that are shared to the user if needed as first pass
output, and adds the same to the user profile and as a second step executes the
personalized recommendation that provides an optimized search result along with
recommendations.
In this entire process after the first step, the rest of the search and recommendation
workflow follows the machine learning technique and is implemented with the collab-
orative filtering and clustering algorithms. The user’s search criteria and the basic user
coordinates including the website, clickstream activity, and geographical data are all
gathered as a user profile data, and are integrated with data from the knowledge re-
pository of similar prior user searches. All of this data is processed with machine learning
algorithms and multiple hypothesis results are iterated with confidence scores and the
highest score is returned as the closest match to the search. A second pass of the result
set and data from knowledge repository is processed to optimize the search and this data