Page 42 - Building Big Data Applications

36 Building Big Data Applications

Node
Manager

Container App Mstr
Client
Resource Node
Manager Manager
Client
App Mstr Container

MapReduce Status Node
Manager
Job Submission
Node Status
Resource Request Container Container

FIGURE 2.9 YARN architecture. Image sourcedApache Foundation.

AMContainer ManagerdComponent responsible for starting and stop-
ping the container of the AM by talking to the appropriate
NodeManager
AM MonitordComponent responsible for managing the aliveness of
the AM and responsible for restarting the AM if necessary

The resource manager stores snapshots of its state in the Zookeeper. In case of failure,
a very transparent restart is feasible and ensures availability.

Node Manager
Is a per-machine agent and is responsible for launching containers for applica-
tions once the scheduler allocates them to the application.
Container resource monitoring for ensuring that the allocated containers do not
exceed their allocated resource slices on the machine.
Setting up the environment of the container for the task execution including bi-
naries, libraries, and jars
Manages local storage on the node. Applications can continue to use the local
storage even when they do not have an active allocation on the node, thus
providing scalability and availability
ApplicationMaster
Perapplication
Negotiates resources with ResourceManager
Manages application scheduling and task execution with Node Managers
Recovers the application on its own failure. Will either recover the application
from saved persistent state or just run the application from the very beginning,
depending on recovery success