Connect to YARN via CLI
Connect
YARN provides a command line interface implemented as a bin/yarn script. To interact with YARN via CLI, connect to a cluster host with YARN via SSH and run the desired YARN command.
For example, check the YARN version by running:
$ yarn version
Output example:
Hadoop 3.1.2 Source code repository git@ssh.gitlab.adsw.io:arenadata/infrastructure/code/ci/prj_adh.git -r 2bfc95569d9993d795ded4878847f3f3db76e77c Compiled by jenkins on 2023-09-07T07:57Z Compiled with protoc 2.5.0 From source with checksum 38903f2495a81dfd8e2d8fc4f659a92 This command was run using /usr/lib/hadoop/hadoop-common-3.1.2.jar
Running yarn
without any arguments prints the list of all YARN commands.
The commands have the following syntax:
$ yarn [SHELL_OPTIONS] COMMAND [GENERIC_OPTIONS] [SUB_COMMAND] [COMMAND_OPTIONS]
Useful commands
Here are examples of typical commands used when working with YARN.
Monitor cluster load
To see the dynamic list of running applications and the resources they consume, run:
$ yarn top
Output example:
YARN top - 11:54:03, up 0d, 2:9, 1 active users, queue(s): root NodeManager(s): 1 total, 1 active, 0 unhealthy, 0 decommissioned, 0 lost, 0 rebooted Queue(s) Applications: 1 running, 1 submitted, 0 pending, 0 completed, 0 killed, 0 failed Queue(s) Mem(GB): 0 available, 6 allocated, 9 pending, 0 reserved Queue(s) VCores: 0 available, 5 allocated, 9 pending, 0 reserved Queue(s) Containers: 5 allocated, 9 pending, 0 reserved APPLICATIONID USER TYPE QUEUE PRIOR #CONT #RCONT VCORES RVCORES MEM RMEM VCORESECS MEMSECS %PROGR T application_1705311887839_0001 yarn mapreduce default 0 5 0 5 0 6G 0G 57 75 16.25 00:00
This command prints out a list of applications that is constantly updating. To see a static list of running applications, use the application -list
command.
List running applications
To see the list of applications running on the host, run:
$ yarn application -list
You can filter applications in the list by using the following options:
-
-appTypes
— filter by application types; -
-appStates
— filter by application states; -
-appTags
— filter by application tags.
Output example:
2024-01-15 11:56:01,792 INFO client.RMProxy: Connecting to ResourceManager at elenas-adh2.ru-central1.internal/127.0.0.1:8032 2024-01-15 11:56:02,241 INFO client.AHSProxy: Connecting to Application History server at elenas-adh2.ru-central1.internal/127.0.0.1:10200 Total number of applications (application-types: [], states: [SUBMITTED, ACCEPTED, RUNNING] and tags: []):1 Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL application_1705311887839_0002 QuasiMonteCarlo MAPREDUCE yarn default RUNNING UNDEFINED 5% http://elenas-adh3.ru-central1.internal:33585
Terminate a frozen application
To terminate an application that stopped responding, run:
$ yarn application -kill <Application ID>
Where <Application ID>
is the ID of the application you want to kill. You can provide a list of space-separated application IDs.
For more detailed information on the above commands, see YARN CLI.