Basic Concepts

Basic Concepts for Xiaomi Elastic MapReduce Services * Instance Groups

Instance Groups

Xiaomi data services defines four roles for instances in clusters: master, control, core, and task. Each role forms an instance group: master instance group, control instance group, core instance group, and task instance group. The 4 instance groups assume responsibilities and deploy services differently.

Master Instance Groups

Within each cluster, there is only one master instance group. Within each master instance group, there is only one master instance (master node). After the cluster has been started, the master instance group may not be changed. Used to manage clusters, coordinate the deployment of services within clusters, and monitor the operation status of individual instances within the cluster. The master instance group allows users to log on through SSH.

NOTE: Refer to manage clusters for how to log onto the master instance

Control Instance Groups

Within each cluster, there is only one control instance group. Within each control instance group, there are only three control instances (control nodes). After the cluster has been started, the control instance group may not be changed. Control services such as zookeeper, zkfc in HDFS, namenode, and resourcemanager in YARN are deployed within the instance group.

Core Instance Groups

Within each cluster, there is only one core instance group. Within each core instance group, there is at least one core instance (core node). When the cluster is running, core instances may be added to the core instance group, but they may not be removed. Used for storing data and running tasks, and deploying storage services and computing services, such as datanode in HDFS and nodemanager in YARN etc.

Task Instance Groups

Task instance groups are optional for clusters. Task instance groups do not have to contain any task instances (task nodes). When the cluster is running, task instances may be added or removed. When computing resources are scarce but there is sufficient storage space, you can add task instances to run tasks. Only computing services are deployed in task instance groups, for example nodemanager in Yarn.