Yarn Usage Instructions

Basic Usage

cd $HADOOP_HOME # change to HADOOP_HOME
$HADOOP_HOME/bin/yarn jar ${jarpath} ${params} # Submit your job,
$HADOOP_HOME/bin/yarn logs --applicationId xxx # check the task's log.
other commands such as killing a task, check details through`$HADOOP_HOME/bin/yarn --help`.

Usage of galaxy-hadoop

Galaxy-hadoop mainly provides examples of Elastic MapReduce on SDS and FDS in the mr/spark framework

Compilation dependencies

See the galaxy-hadoop for detailed documents

Example

• Galaxy-hadoop MR processing of FDS data, for specific use see:

  • galaxy-hadoop/examples/fds-mapreduce/wordcount-fds

• Galaxy-hadoop MR processing of SDS data, for specific use see:

  • galaxy-hadoop/examples/sds-mapreduce/sds-multi-tables-manay-scans
  • galaxy-hadoop/examples/sds-mapreduce/sds-single-table-manay-scans
  • galaxy-hadoop/examples/sds-mapreduce/sds-single-table-one-scan
  • galaxy-hadoop/examples/sds-mapreduce/sds-global-index-one-scan

• Galaxy-hadoop spark processing of FDS data, for specific use see:

  • galaxy-hadoop/examples/fds-spark/fds-scan

• Galaxy-hadoop spark processing of SDS data, for specific use see:

  • galaxy-hadoop/examples/sds-spark/sds-table-copy

Usage of galaxy-exporter

Galaxy-exporter is a tool for converting SDS data to HDFS parquet files under the MR framework. Because this is an internal tool, there is no direct external development. Please send an email to emr-help@xiaomi.com to apply for access.