Connect to YARN via CLI

Elena Kostyuchenko

Connect

YARN provides a command line interface implemented as a bin/yarn script. To interact with YARN via CLI, connect to a cluster host with YARN via SSH and run the desired YARN command.

For example, check the YARN version by running:

$ yarn version

Output example:

Hadoop 3.1.2
Source code repository git@ssh.gitlab.adsw.io:arenadata/infrastructure/code/ci/prj_adh.git -r 2bfc95569d9993d795ded4878847f3f3db76e77c
Compiled by jenkins on 2023-09-07T07:57Z
Compiled with protoc 2.5.0
From source with checksum 38903f2495a81dfd8e2d8fc4f659a92
This command was run using /usr/lib/hadoop/hadoop-common-3.1.2.jar

Running yarn without any arguments prints the list of all YARN commands.

The commands have the following syntax:

$ yarn [SHELL_OPTIONS] COMMAND [GENERIC_OPTIONS] [SUB_COMMAND] [COMMAND_OPTIONS]

Useful commands

Here are examples of typical commands used when working with YARN.

Monitor cluster load

To see the dynamic list of running applications and the resources they consume, run:

$ yarn top

Output example:

YARN top - 11:54:03, up 0d, 2:9, 1 active users, queue(s): root
NodeManager(s): 1 total, 1 active, 0 unhealthy, 0 decommissioned, 0 lost, 0 rebooted
Queue(s) Applications: 1 running, 1 submitted, 0 pending, 0 completed, 0 killed, 0 failed
Queue(s) Mem(GB): 0 available, 6 allocated, 9 pending, 0 reserved
Queue(s) VCores: 0 available, 5 allocated, 9 pending, 0 reserved
Queue(s) Containers: 5 allocated, 9 pending, 0 reserved

                  APPLICATIONID USER             TYPE      QUEUE PRIOR   #CONT  #RCONT  VCORES RVCORES     MEM    RMEM  VCORESECS    MEMSECS %PROGR       T
 application_1705311887839_0001 yarn        mapreduce    default     0       5       0       5       0      6G      0G         57         75  16.25   00:00

This command prints out a list of applications that is constantly updating. To see a static list of running applications, use the application -list command.

List running applications

To see the list of applications running on the host, run:

$ yarn application -list

You can filter applications in the list by using the following options:

  • -appTypes — filter by application types;

  • -appStates — filter by application states;

  • -appTags — filter by application tags.

Output example:

2024-01-15 11:56:01,792 INFO client.RMProxy: Connecting to ResourceManager at elenas-adh2.ru-central1.internal/127.0.0.1:8032
2024-01-15 11:56:02,241 INFO client.AHSProxy: Connecting to Application History server at elenas-adh2.ru-central1.internal/127.0.0.1:10200
Total number of applications (application-types: [], states: [SUBMITTED, ACCEPTED, RUNNING] and tags: []):1
                Application-Id      Application-Name        Application-Type          User           Queue                   State             Final-State        Progress                         Tracking-URL
application_1705311887839_0002       QuasiMonteCarlo               MAPREDUCE          yarn         default                 RUNNING               UNDEFINED              5%  http://elenas-adh3.ru-central1.internal:33585

Terminate a frozen application

To terminate an application that stopped responding, run:

$ yarn application -kill <Application ID>

Where <Application ID> is the ID of the application you want to kill. You can provide a list of space-separated application IDs.

For more detailed information on the above commands, see YARN CLI.

Found a mistake? Seleсt text and press Ctrl+Enter to report it