ADH network ports

Eugenia Kuzina

ADH network port mapping

The tables below list ports of ADH service components and servers. The Config column contains a file name where the port number can be redefined.

ADCM

Component/Server

Port

Protocol

Description

ADCM

TCP

ADCM connection port

Arenadata PostgreSQL

Component/Server

Port

Protocol

Description

Parameter

Config

Arenadata PostgreSQL

5432

TCP

PostgreSQL port for DB connections

port

postgresql.conf

Airflow

Component/Server

Port

Protocol

Description

Parameter

Config

Webserver

8080/8080

HTTP/HTTPS

Airflow Server Web UI port

web_server_port (server_port in ADCM)

airflow.cfg

Worker

8793/8793

HTTP/HTTPS

Celery Worker API port

worker_log_server_port (worker_port in ADCM)

airflow.cfg

Flower

5555/5555

HTTP/HTTPS

Flower Web UI port

flower_port

airflow.cfg

Broker

6379

TCP

Redis service port

broker_url (redis_port in ADCM)

airflow.cfg

Metastore

3307

TCP

MariaDB service port

sql_alchemy_conn, result_backend

airflow.cfg

Airflow2

Component/Server

Port

Protocol

Description

Parameter

Config

Webserver

8080/8080

HTTP/HTTPS

Airflow Server Web UI port

web_server_port (server_port in ADCM)

airflow.cfg

Worker

8793/8793

HTTP/HTTPS

Celery Worker API port

worker_log_server_port (worker_port in ADCM)

airflow.cfg

Flower

5555/5555

HTTP/HTTPS

Flower Web UI port

flower_port

airflow.cfg

Broker

6379

TCP

Redis service port

broker_url (redis_port in ADCM)

airflow.cfg

Metastore

5432

TCP

PostgreSQL service port

sql_alchemy_conn, result_backend

airflow.cfg

Flink

Component/Server

Port

Protocol

Description

Parameter

Config

Task Manager

Range (for example, 35122)

TCP

Data port (range)

taskmanager.data.port

—

Task Manager

Range (for example, 32390)

IPC

IPC port (range)

taskmanager.rpc.port

—

Job Manager

8081/8081

HTTP/HTTPS

Flink Job Manager Web UI port

rest.port

A custom parameter in flink-conf.yaml

Job Manager

6123

TCP

Flink Job Manager RPC port (non HA mode)

jobmanager.rpc.port

flink-conf.yaml

Job Manager

Range generated by Zookeeper (for example, 50100)

TCP

Flink Job Manager port (HA mode)

high-availability.jobmanager.port

—

HBase

Component/Server

Port

Protocol

Description

Parameter

Config

HBase Master

16000

IPC

HBase Master IPC port

hbase.master.port

A custom parameter in hbase-site.xml

HBase Master

16010/16010

HTTP/HTTPS

HBase Master Web UI port

hbase.master.info.port

A custom parameter in hbase-site.xml

HBase Region

16030/16030

HTTP/HTTPS

HBase Region Server Web UI port

hbase.regionserver.info.port

A custom parameter in hbase-site.xml

HBase Region

16020

IPC

HBase Region Server IPC port

hbase.regionserver.port

A custom parameter in hbase-site.xml

Phoenix Query Server

8765/8765

HTTP/HTTPS

API Port

phoenix.queryserver.http.port

A custom parameter in hbase-site.xml

HBase REST server

60080/60080

HTTP/HTTPS

API Port

hbase.rest.port

hbase-site.xml

HBase REST Server

8085

HTTP/HTTPS

Web UI Port

hbase.rest.info.port

A custom parameter in hbase-site.xml

HBaseThrift2 Server

Component/Server

Port

Protocol

Description

Parameter

Config

Thrift2 Server

9090

TCP

Thrift2 Server port

—

Thrift2 Server

9095/9095

HTTP/HTTPS

Thrift2 Server Web UI port

hbase.thrift.info.port

A custom parameter in hbase-site.xml

HDFS

Component/Server

Port

Protocol

Description

Parameter

Config

NameNode

9870/9871

HTTP/HTTPS

NameNode Web UI port

dfs.namenode.http-address, dfs.namenode.https-address

hdfs-site.xml

NameNode

8020

IPC

File system metadata operations

Embedded in dfs.namenode.rpc-address.<namespace>.<namenode_id>

hdfs-site.xml

ZKFC

8019

IPC (TCP, UDP)

ZooKeeper FailoverController for NN HA

dfs.ha.zkfc.port

A custom parameter in hdfs-site.xml

DataNode

9864/9865

HTTP/HTTPS

DataNode Web UI port

dfs.datanode.https.address

hdfs-site.xml

DataNode

9866

TCP

Data transfer

dfs.datanode.address

hdfs-site.xml

DataNode

9867

IPC

File system metadata operations

dfs.datanode.ipc.address

hdfs-site.xml

JournalNode

8480/8481

HTTP/HTTPS

JournalNode Web UI port

dfs.journalnode.http.address, dfs.journalnode.https.address

hdfs-site.xml

JournalNode

8485

IPC

File system metadata operations

dfs.journalnode.rpc-address

hdfs-site.xml

HttpFS server

14000/14000

HTTP/HTTPS

HTTPfs API port

httpfs.http.port

httpfs-site.xml

Hive

Component/Server

Port

Protocol

Description

Parameter

Config

Hive Server

10000

TCP or HTTP

Thrift/JDBC port

hive.server2.thrift.port

hive-site.xml

Hive Server

10002/10002

HTTP/HTTPS

Hive Server Web UI port

hive.server2.webui.port

A custom parameter in hive-site.xml

Hive Metastore

9083

TCP

Hive Metastore service port

hive.metastore.uris

hive-site.xml

Tez

9999/9999

HTTP/HTTPS

Tez Web UI port

Embedded in tez.tez-ui.history-url.base

tez-site.xml

HUE

Component/Server

Port

Protocol

Description

Parameter

Config

HUE Server

8000/8000

HTTP/HTTPS

HUE Server web UI port

desktop.http_port

hue.ini

Impala

Component/Server

Port

Protocol

Description

Parameter

Config

Impala Catalog Service

23020

TCP

Internal use only. The Catalog Server daemon listens on this port for updates from the StateStore daemon

state_store_subscriber_port

/etc/impala/conf/catalogstore.conf

Impala Catalog Service

25020

HTTP/HTTPS

The port on which the Catalog Service web server is running. See Impala web UIs overview

webserver_port

/etc/impala/conf/catalogstore.conf

Impala Catalog Service

26000

TCP

Internal use only. The Catalog Server uses this port to communicate with the Impala daemons

catalog_service_port

/etc/impala/conf/catalogstore.conf

Impala Daemon

21000

TCP

The port on which Impala daemons serve Beeswax client requests

beeswax_port

/etc/impala/conf/impalastore.conf

Impala Daemon

21050

TCP

The port on which Impala daemons serve HiveServer2 client requests

hs2_port

/etc/impala/conf/impalastore.conf

Impala Daemon

22000

TCP

Internal use only. Impala daemons use this port for Thrift-based communication with each other

be_port

/etc/impala/conf/impalastore.conf

Impala Daemon

23030

TCP

Internal use only. Impala daemons listen on this port for updates from the Statestore daemon

state_store_subscriber_port

/etc/impala/conf/impalastore.conf

Impala Daemon

25000

HTTP/HTTPS

The port where the Impala daemon web server is running. See Impala web UIs overview

webserver_port

/etc/impala/conf/impalastore.conf

Impala Daemon

27000

TCP

Internal use only. Impala daemons use this port for KRPC-based communication with each other

krpc_port

/etc/impala/conf/impalastore.conf

Impala Daemon

28000

HTTP

The port is used by client applications to transmit commands and receive results over HTTP via the HiveServer2 protocol

hs2_http_port

/etc/impala/conf/impalastore.conf

Impala Statestore

24000

TCP

The port on which the Impala Statestore component is running

state_store_port

/etc/impala/conf/statestore.conf

Impala Statestore

25010

HTTP/HTTPS

The port on which the Statestore web server is running. See Impala web UIs overview

webserver_port

/etc/impala/conf/statestore.conf

Kyuubi

Component/Server

Port

Protocol

Description

Parameter

Config

Kyuubi Server

10099

HTTP/HTTPS

Port on which the REST frontend service runs

kyuubi.frontend.rest.bind.port

kyuubi-defaults.conf

Kyuubi Server

10009

Binary

Port on which the Thrift frontend service runs via a binary protocol

kyuubi.frontend.thrift.binary.bind.port

kyuubi-defaults.conf

Kyuubi Server

10010

HTTP/HTTPS

Port on which the Thrift frontend service runs via HTTP

kyuubi.frontend.thrift.http.bind.port

kyuubi-defaults.conf

MariaDB

Component/Server

Port

Protocol

Description

Parameter

Config

MariaDB Master server

3306

TCP

MariaDB port for DB connections

port

my.cnf

Monitoring (the Monitoring cluster)

Component/Server

Port

Protocol

Description

Parameter

Config

Graphite

2015

TCP

Port to send metrics

port

diamond.conf

Solr

Component/Server

Port

Protocol

Description

Parameter

Config

Solr Server

8983/8985

HTTP/HTTPS

Solr Server Web UI and API port

hostPort

solr.xml

Spark

Component/Server

Port

Protocol

Description

Parameter

Config

History Server

18082/18082

HTTP/HTTPS

HS Web UI port

spark.historyServer.port, spark.ssl.historyServer.port

spark-defaults.conf

Thrift Server

4040/4040 (4041, … if more applications are running)

HTTP/HTTPS

Thrift Server Web UI

spark.ui.port, spark.ssl.ui.port

spark-defaults.com

Thrift Server

10016

TCP

Thrift Server service port

hive.server2.thrift.port

spark-env.sh

Livy Server

8998/8998 (8999, …)

HTTP/HTTPS

Livy Server Web UI port

livy.server.port

livy.conf

Spark3

Component/Server

Port

Protocol

Description

Parameter

Config

History Server

18092

HTTP/HTTPS

HS Web UI port

spark.history.ui.port, spark.ssl.historyServer.port

spark-defaults.conf

Livy Server

8999

HTTP/HTTPS

Livy Server Web UI port

livy.server.port

livy.conf

Spark3 Connect

15002

HTTP/HTTPS

The port number to connect to the Spark3 Connect gRPC endpoint

spark.connect.grpc.binding.port

spark-defaults.conf

SSM

Component/Server

Port

Protocol

Description

Parameter

Config

Smart Agent

7048

RPC

The port number used by SSM agents to communicate with the SSM Server

smart.agent.port

smart-site.xml

Smart Agent master port

7051

RPC

The port number used by the SSM Server to communicate with SSM agents

smart.agent.master.port

smart-site.xml

SSM RPC server port

7042

RPC

The port number used by the SSM Server to interact with NameNodes

smart.server.rpc.address

smart-site.xml

SSM web UI port

7045

HTTP/HTTPS

The port number used by the SSM web UI

smart.server.rpc.address

smart-site.xml

Sqoop

Component/Server

Port

Protocol

Description

Parameter

Config

Metastore

16100

TCP

Connection between Sqoop and the metastore (internal use)

sqoop.metastore.server.port

sqoop-site.xml

YARN

Component/Server

Port

Protocol

Description

Parameter

Config

ResourceManager

8088/8090

HTTP/HTTPS

RM Web UI Port

yarn.resourcemanager.webapp.address, yarn.resourcemanager.webapp.https.address

yarn-site.xml (only 8090)

ResourceManager

8030

HTTP

The port of the scheduler interface

yarn.resourcemanager.scheduler.address

A custom parameter in yarn-site.xml

ResourceManager

8031

IPC

Resource tracker port

yarn.resourcemanager.resource-tracker.address

A custom parameter in yarn-site.xml

Resource Manager

8032

IPC

The port of the application manager interface in the RM

yarn.resourcemanager.address

A custom parameter in yarn-site.xml

Resource Manager

8033

IPC

The port of the RM admin interface

yarn.resourcemanager.admin.address

A custom parameter in yarn-site.xml

NodeManager

8042/8044

HTTP/HTTPS

NM Web UI port

8042 is yarn.nodemanager.webapp.address, 8044 is yarn.nodemanager.webapp.https.address

yarn-site.xml (only 8044)

NodeManager

8040

IPC

Port where the localizer IPC is

yarn.nodemanager.localizer.address

A custom parameter in yarn-site.xml

NodeManager

8041

IPC

Port of container manager in the NM

yarn.nodemanager.address

yarn-site.xml

NodeManager

7337

TCP

Spark shuffle service port (The service is disabled by default)

spark.shuffle.service.port

custom spark-defaults.conf

NodeManager

7447

TCP

Spark shuffle service port

—

MapReduce History Server

19888/19890

HTTP/HTTPS

HS Web UI port

mapreduce.jobhistory.webapp.address, mapreduce.jobhistory.webapp.https.address

mapred-site.xml

MapReduce History Server

10020

IPC

MapReduce JobHistory Server port

mapreduce.jobhistory.address

mapred-site.xml

MapReduce History Server

10033

IPC

The port of the History server admin interface

mapreduce.jobhistory.admin.address

A custom parameter in mapred-site.xml

MapReduce

13562

IPC

MapReduce Shuffle port

mapreduce.shuffle.port

A custom parameter in mapred-site.xml

Timeline Server

8188/8190

HTTP/HTTPS

TS Web UI port

yarn.timeline-service.webapp.https.address

yarn-site.xml

Timeline Server

10200

IPC

TS Server port

Embedded in mapreduce.jobhistory.address

mapred-site.xml

Zeppelin

Component/Server

Port

Protocol

Description

Parameter

Config

Server

8180/8180

HTTP/HTTPS

Zeppelin Web UI port

zeppelin.server.port, zeppelin.server.ssl.port

zeppelin-site.xml

Zookeeper

Component/Server

Port

Protocol

Description

Parameter

Config

Zookeeper

2888

TCP

Port for cluster coordinating (internal cluster use only)

server.*

zoo.cfg

Zookeeper

3888

TCP

Port for cluster coordinating (internal cluster use only)

server.*

zoo.cfg

Zookeeper

2181

TCP

Port for clients connection

clientPort (client_port in ADCM)

zoo.cfg

Zookeeper Admin Server

5181

HTTP

Zookeeper HTTP API interface

admin.serverPort (admin.serverPort in ADCM)

zoo.cfg

JMX ports

Only the YARN RM port is defined in ADCM UI. Other JMX ports are hardcoded in service environment configs.

Service

Component/Server

Port

Protocol

Description

Parameter

Config

HBase

HBase Region

10102

TCP

JMX port (internal use with jmxtrans)

-Dcom.sun.management.jmxremote.port

hbase-regionserver-env.sh

HDFS

NameNode

1026

TCP

JMX port (internal use with jmxtrans)

-Dcom.sun.management.jmxremote.port

hadoop-env.sh

HDFS

DataNode

1027

TCP

JMX port (internal use with jmxtrans)

-Dcom.sun.management.jmxremote.port

hadoop-env.sh

Hive

Hive Server

1031

TCP

JMX port (internal use with jmxtrans)

-Dcom.sun.management.jmxremote.port

hive-server2-env.sh

Hive

Metastore

1032

TCP

JMX port (internal use with jmxtrans)

-Dcom.sun.management.jmxremote.port

hive-metastore-env.sh

Monitoring

jmxtrans

2101

TCP

JMX port for clients connection

-Dcom.sun.management.jmxremote.port

/etc/jmxtrans/wrapper.conf

Spark

History Server

18083

TCP

JMX port (internal use with jmxtrans)

-Dcom.sun.management.jmxremote.port

spark-env.sh

Spark3

History Server

18093

TCP

JMX port (internal use with jmxtrans)

-Dcom.sun.management.jmxremote.port

spark-env.sh

YARN

Resource Manager

1028

TCP

JMX port (internal use with jmxtrans)

-Dcom.sun.management.jmxremote.port (resource_manager_jmx_port in ADCM)

yarn-env.sh

YARN

NodeManager

1029

TCP

JMX port (internal use with jmxtrans)

-Dcom.sun.management.jmxremote.port

yarn-env.sh

YARN

MapReduce History Server

1030

TCP

JMX port (internal use with jmxtrans)

-Dcom.sun.management.jmxremote.port

mapred-env.sh

Ports redefined by Kerberos

When Kerberos is enabled, ADCM redefines ports from the table below by hardcoded values from Ansible config.

On disabling Kerberos, ADCM returns the default hardcoded values for these parameters.

Service

Component/Server

Port

Protocol

Description

Kerberos

Parameter

HDFS

DataNode

1022/50475

HTTP/HTTPS

DataNode Web UI port/ secure port

Enabled

dfs.datanode.http.address, dfs.datanode.https.address

HDFS

DataNode

1019

TCP

Data transfer port

Enabled

dfs.datanode.address

Client ports

Client ports are generated in the local port range. You can determine a specific port from this range using a known service port. Use the lsof command as follows:

$ sudo lsof -i -P -n | grep <known port namber>

Examples

The Diamond client is running on a local random port. It connects to Graphite that listens the 2015 port.

$ sudo lsof -i -P -n | grep 2015

The result:

python    6000       root    2u  IPv4 4811652      0t0  TCP 10.92.16.232:34758->10.92.16.30:2015 (ESTABLISHED)

The Hive client is running on a local random port. It connects to MariaDB that listens the 3306 port.

$ sudo lsof -i -P -n | grep 3306

The result:

java      20906       hive  549u  IPv6 14115081      0t0  TCP 10.92.17.88:35006->10.92.17.37:3306 (ESTABLISHED)

Examples of client ports in the local port range
Component	Port	Protocol	Description
Diamond (python-process, running by root)	34758 or another random port from local port range	TCP	Local jmxtrans client port for sending metrics to Monitoring cluster on port 2015 (to Graphite)
Hive	42598 or another random port from local port range	TCP	Local client port to connect to MariaDB database on 3306 port
HBase, Spark, and etc.	50382 or another random port from local port range	TCP	Local client port to connect to HDFS on port 8020
Flink JobManager	43162 or another random port from local port range	TCP	Flink client connects to Zookeeper: TCP 10.92.17.55:42844→10.92.17.55:2181 Flink internal connect between JobManager’s nodes: TCP 10.92.17.55:60390→10.92.17.58:33769

Found a mistake? Seleсt text and press Ctrl+Enter to report it