ADH network ports

ADH network port mapping

The tables below list ports of the ADH services and components. The Config column contains a file name where the port number can be redefined.

Arenadata PostgreSQL

Component/Server Port Protocol Description Parameter Config

Arenadata PostgreSQL

5432

TCP

PostgreSQL port for DB connections

port

postgresql.conf

Airflow2

Component/Server Port Protocol Description Parameter Config

Airflow2 Broker

6379

TCP

Redis service port

broker_url (redis_port in ADCM)

airflow.cfg

Airflow2 Flower

5555/5555

HTTP/HTTPS

Flower Web UI port

flower_port

airflow.cfg

Airflow2 Scheduler

5432

TCP

PostgreSQL service port

sql_alchemy_conn, result_backend

airflow.cfg

Airflow2 Server

8080/8080

HTTP/HTTPS

Airflow Server Web UI port

web_server_port (server_port in ADCM)

airflow.cfg

Airflow2 Worker

8793/8793

HTTP/HTTPS

Celery Worker API port

worker_log_server_port (worker_port in ADCM)

airflow.cfg

Component/Server Port Protocol Description Parameter Config

Flink History Server

8082

HTTP/HTTPS

Flink History Server web UI port

historyserver.web.port

flink-conf.yaml

Flink JobManager

8081/8081

HTTP/HTTPS

Flink Job Manager web UI port

rest.port

flink-conf.yaml

Flink JobManager

6123

TCP

Flink Job Manager RPC port (non-HA mode)

jobmanager.rpc.port

flink-conf.yaml

Flink JobManager

Range generated by Zookeeper (for example, 50100)

TCP

Flink Job Manager port (HA mode)

high-availability.jobmanager.port

 — 

Flink SQL-Gateway

8083

TCP

Port of the SQL Gateway REST endpoint

sql-gateway.endpoint.rest.port

flink-conf.yaml

Flink TaskManager

Range (for example, 35122)

TCP

Data port (range)

taskmanager.data.port

 — 

Flink TaskManager

Range (for example, 32390)

IPC

IPC port (range)

taskmanager.rpc.port

 — 

Flink2

Component/Server Port Protocol Description Parameter Config

Flink2 History Server

8092

HTTP/HTTPS

Port of the History Server’s web interface

historyserver.web.port

config.yaml

Flink2 SQL-Gateway

8093

TCP

Port of the SQL Gateway REST endpoint

sql-gateway.endpoint.rest.port

config.yaml

HBase

Component/Server Port Protocol Description Parameter Config

HBase Master Server

16000

IPC

HBase Master IPC port

hbase.master.port

A custom parameter in hbase-site.xml

HBase Master Server

16010/16010

HTTP/HTTPS

HBase Master Web UI port

hbase.master.info.port

A custom parameter in hbase-site.xml

HBase Master Server

9211

HTTP/HTTPS

Sets the Prometheus metrics port

Prometheus metrics port

Monitoring

HBase Phoenix Query Server

8765/8765

HTTP/HTTPS

API Port

phoenix.queryserver.http.port

A custom parameter in hbase-site.xml

HBase Phoenix Query Server

9213

HTTP

Sets the Prometheus metrics port

Prometheus metrics port

Monitoring

HBase REST Server

60080/60080

HTTP/HTTPS

API Port

hbase.rest.port

hbase-site.xml

HBase REST Server

8085

HTTP/HTTPS

Web UI Port

hbase.rest.info.port

A custom parameter in hbase-site.xml

HBase REST Server

9215

HTTP

Sets the Prometheus metrics port

Prometheus metrics port

Monitoring

HBase RegionServer

16030/16030

HTTP/HTTPS

HBase RegionServer Web UI port

hbase.regionserver.info.port

A custom parameter in hbase-site.xml

HBase RegionServer

16020

IPC

HBase RegionServer IPC port

hbase.regionserver.port

A custom parameter in hbase-site.xml

HBase RegionServer

9212

HTTP

Sets the Prometheus metrics port

Prometheus metrics port

Monitoring

HBaseThrift2 Server

Component/Server Port Protocol Description Parameter Config

HBase Thrift2 Server

9090

TCP

Thrift2 Server port

 — 

 — 

HBase Thrift2 Server

9095/9095

HTTP/HTTPS

Thrift2 Server Web UI port

hbase.thrift.info.port

A custom parameter in hbase-site.xml

HBase Thrift2 Server

9214

HTTP

Sets the Prometheus metrics port

Prometheus metrics port

Monitoring

HDFS

Component/Server Port Protocol Description Parameter Config

HDFS DataNode

9864/9865

HTTP/HTTPS

DataNode Web UI port

dfs.datanode.https.address

hdfs-site.xml

HDFS DataNode

9866

TCP

Data transfer

dfs.datanode.address

hdfs-site.xml

HDFS DataNode

9867

IPC

File system metadata operations

dfs.datanode.ipc.address

hdfs-site.xml

HDFS DataNode

9202

HTTP

Port to share monitoring metrics

Prometheus metrics port

Monitoring

HDFS HttpFS server

14000/14000

HTTP/HTTPS

HTTPfs API port

httpfs.http.port

httpfs-site.xml

HDFS JournalNode

8480/8481

HTTP/HTTPS

JournalNode Web UI port

dfs.journalnode.http.address, dfs.journalnode.https.address

hdfs-site.xml

HDFS JournalNode

8485

IPC

File system metadata operations

dfs.journalnode.rpc-address

hdfs-site.xml

HDFS JournalNode

9203

HTTP

Port to share monitoring metrics

Prometheus metrics port

Monitoring

HDFS NameNode

9870/9871

HTTP/HTTPS

NameNode Web UI port

dfs.namenode.http-address, dfs.namenode.https-address

hdfs-site.xml

HDFS NameNode

8020

IPC

File system metadata operations

Embedded in dfs.namenode.rpc-address.<namespace>.<namenode_id>

hdfs-site.xml

HDFS NameNode

9201

HTTP

Port to share monitoring metrics

Prometheus metrics port

Monitoring

HDFS ZKFC

8019

IPC (TCP, UDP)

ZooKeeper FailoverController for NN HA

dfs.ha.zkfc.port

A custom parameter in hdfs-site.xml

Hive

Component/Server Port Protocol Description Parameter Config

Hive HiveServer2

10000

TCP or HTTP

Thrift/JDBC port

hive.server2.thrift.port

hive-site.xml

Hive HiveServer2

10002/10002

HTTP/HTTPS

Hive Server Web UI port

hive.server2.webui.port

A custom parameter in hive-site.xml

Hive HiveServer2

9208

HTTP

Port to share monitoring metrics

Prometheus metrics port

Monitoring

Hive Metastore

9083

TCP

Hive Metastore service port

hive.metastore.uris

hive-site.xml

Hive Metastore

9207

HTTP

Port to share monitoring metrics

Prometheus metrics port

Monitoring

Hive Tez UI

8089

HTTP/HTTPS

Tez Web UI port

Embedded in tez.tez-ui.history-url.base

tez-site.xml

HUE

Component/Server Port Protocol Description Parameter Config

HUE Server

8000/8000

HTTP/HTTPS

HUE Server web UI port

desktop.http_port

hue.ini

HUE Server

8000

HTTP

Port to share monitoring metrics

desktop.http_port

hue.ini

Impala

Component/Server Port Protocol Description Parameter Config

Impala Catalog Service

23020

TCP

Internal use only. The Catalog Server daemon listens on this port for updates from the StateStore daemon

state_store_subscriber_port

/etc/impala/conf/catalogstore.conf

Impala Catalog Service

25020

HTTP/HTTPS

Port on which the Catalog Service web server is running. See Impala web UIs overview

webserver_port

/etc/impala/conf/catalogstore.conf

Impala Catalog Service

26000

TCP

Internal use only. The Catalog Server uses this port to communicate with the Impala daemons

catalog_service_port

/etc/impala/conf/catalogstore.conf

Impala Catalog Service

25020

HTTP

Port to share monitoring metrics

webserver_port

catalogstore.conf

Impala Daemon

21000

TCP

Port on which Impala daemons serve Beeswax client requests

beeswax_port

/etc/impala/conf/impalastore.conf

Impala Daemon

21050

TCP

Port on which Impala daemons serve HiveServer2 client requests

hs2_port

/etc/impala/conf/impalastore.conf

Impala Daemon

22000

TCP

Internal use only. Impala daemons use this port for Thrift-based communication with each other

be_port

/etc/impala/conf/impalastore.conf

Impala Daemon

23030

TCP

Internal use only. Impala daemons listen on this port for updates from the Statestore daemon

state_store_subscriber_port

/etc/impala/conf/impalastore.conf

Impala Daemon

25000

HTTP/HTTPS

Port where the Impala daemon web server is running. See Impala web UIs overview

webserver_port

/etc/impala/conf/impalastore.conf

Impala Daemon

27000

TCP

Internal use only. Impala daemons use this port for KRPC-based communication with each other

krpc_port

/etc/impala/conf/impalastore.conf

Impala Daemon

28000

HTTP

Port is used by client applications to transmit commands and receive results over HTTP via the HiveServer2 protocol

hs2_http_port

/etc/impala/conf/impalastore.conf

Impala Daemon

25000

HTTP

Port to share monitoring metrics

webserver_port

impalastore.conf

Impala Statestore

24000

TCP

Port on which the Impala Statestore component is running

state_store_port

/etc/impala/conf/statestore.conf

Impala Statestore

25010

HTTP/HTTPS

Port on which the Statestore web server is running. See Impala web UIs overview

webserver_port

/etc/impala/conf/statestore.conf

Impala Statestore

25010

HTTP

Port to share monitoring metrics

webserver_port

/etc/impala/conf/statestore.conf

Kyuubi

Component/Server Port Protocol Description Parameter Config

Kyuubi Server

10099

HTTP/HTTPS

Port on which the REST frontend service runs

kyuubi.frontend.rest.bind.port

kyuubi-defaults.conf

Kyuubi Server

10009

Binary

Port on which the Thrift frontend service runs via a binary protocol

kyuubi.frontend.thrift.binary.bind.port

kyuubi-defaults.conf

Kyuubi Server

10010

HTTP/HTTPS

Port on which the Thrift frontend service runs via HTTP

kyuubi.frontend.thrift.http.bind.port

kyuubi-defaults.conf

Kyuubi Server

10019

HTTP

Port to share monitoring metrics

kyuubi.metrics.prometheus.port

kyuubi-defaults.conf

Monitoring (service)

Component/Server Port Protocol Description Parameter Config

Prometheus

11200

HTTP

Port used by Prometheus for incoming HTTP requests (UI, API, metrics)

web.listen-address

Prometheus settings

Grafana

11210

HTTP

Port, on which the Grafana server listens for incoming requests

Grafana listen port

Grafana settings

Node exporter

11203

HTTP

Port, on which the Node exporter server listens for incoming requests

Listen port

Node Exporter settings

Monitoring (cluster)

Component/Server Port Protocol Description Parameter Config

Graphite

2015

TCP

Port to send metrics

port

diamond.conf

Ozone

Component/Server Port Protocol Description Parameter Config

Ozone Datanode

9882/9883

HTTP/HTTPS

HTTP address for the DataNode web interface

hdds.datanode.http-address/hdds.datanode.https-address

ozone-site.xml

Ozone Datanode

9855

RPC

Port used for Ratis DataStream on the DataNode

hdds.container.ratis.datastream.port

ozone-site.xml

Ozone Manager

9862

RPC

Port of the Ozone Manager

ozone.om.address

ozone-site.xml

Ozone Manager

9874/9875

HTTP/HTTPS

Specifies the HTTP address for the Ozone Manager web interface

ozone.om.http-address/ozone.om.https-address

ozone-site.xml

Ozone Manager

9858

RPC

Port for Ratis communication used by Ozone Manager

ozone.om.ratis.port

ozone-site.xml

Ozone Manager

9874

HTTP

Port to share monitoring metrics

ozone.om.http-address

ozone-site.xml

Ozone Recon

9888/9889

HTTP/HTTPS

HTTP address for the Recon web interface, used to view metrics and reports for the Ozone cluster

ozone.recon.http-address/ozone.recon.https-address

ozone-site.xml

Ozone S3G

9878/9879

HTTP/HTTPS

HTTP address for the S3 Gateway, which provides the S3-compatible API interface for Ozone

ozone.s3g.http-address/ozone.s3g.https-address

ozone-site.xml

Ozone Storage Container Manager

9876/9877

HTTP/HTTPS

Specifies the HTTP address for the SCM web interface

ozone.scm.http-address/ozone.scm.https-address

ozone-site.xml

Ozone Storage Container Manager

9894

RPC

Port for Ratis communication used by SCM for high availability

ozone.scm.ratis.port

ozone-site.xml

Ozone Storage Container Manager

9876

HTTP

Port to share monitoring metrics

ozone.scm.http-address

ozone-site.xml

Solr

Component/Server Port Protocol Description Parameter Config

Solr Server

8983/8985

HTTP/HTTPS

Solr Server Web UI and API port

hostPort

solr.xml

Solr prometheus exporter

9983

HTTP

Solr exporter HTTP port

Option port

 — 

Spark3

Component/Server Port Protocol Description Parameter Config

Spark3 Connect

15002

HTTP/HTTPS

Port number to connect to the Spark3 Connect gRPC endpoint

spark.connect.grpc.binding.port

spark-defaults.conf

Spark3 History Server

18092

HTTP/HTTPS

HS Web UI port

spark.history.ui.port, spark.ssl.historyServer.port

spark-defaults.conf

Spark3 Livy Server

8999

HTTP/HTTPS

Livy Server Web UI port

livy.server.port

livy.conf

SSM

Component/Server Port Protocol Description Parameter Config

SSM Agent

7044

RPC

Port number used by SSM agents to communicate with the SSM Server

smart.agent.port

smart-site.xml

SSM Agent

7051

RPC

Port number used by the SSM Server to communicate with SSM agents

smart.agent.master.port

smart-site.xml

SSM Server

7042

RPC

Port number used by the SSM Server to interact with NameNodes

smart.server.rpc.address

smart-site.xml

SSM Server

7045

HTTP/HTTPS

Port number used by the SSM web UI

smart.server.rpc.address

smart-site.xml

Trino

Component/Server Port Protocol Description Parameter Config

Coordinator

18188

HTTP/HTTPS

Prometheus metrics port

http-server.http.port

config.properties

Coordinator

9209

HTTP/HTTPS

Sets the Prometheus metrics port

Prometheus metrics port

Monitoring

HAProxy

7001

HTTP/HTTPS

Sets the HTTP port for the server

http-server.http.port

config.properties

Worker

18189

HTTP/HTTPS

Sets the HTTP port for the server

http-server.http.port

config.properties

Worker

9210

HTTP/HTTPS

Sets the Prometheus metrics port

Prometheus metrics port

Monitoring

YARN

Component/Server Port Protocol Description Parameter Config

MapReduce History Server

19888/19890

HTTP/HTTPS

HS Web UI port

mapreduce.jobhistory.webapp.address, mapreduce.jobhistory.webapp.https.address

mapred-site.xml

MapReduce History Server

10020

IPC

MapReduce JobHistory Server port

mapreduce.jobhistory.address

mapred-site.xml

MapReduce History Server

10033

IPC

Port of the History server admin interface

mapreduce.jobhistory.admin.address

A custom parameter in mapred-site.xml

MapReduce History Server

13562

IPC

MapReduce Shuffle port

mapreduce.shuffle.port

A custom parameter in mapred-site.xml

YARN NodeManager

8042/8044

HTTP/HTTPS

NM Web UI port

8042 is yarn.nodemanager.webapp.address, 8044 is yarn.nodemanager.webapp.https.address

yarn-site.xml (only 8044)

YARN NodeManager

8040

IPC

Port where the localizer IPC is

yarn.nodemanager.localizer.address

A custom parameter in yarn-site.xml

YARN NodeManager

8041

IPC

Port of container manager in the NM

yarn.nodemanager.address

yarn-site.xml

YARN NodeManager

7337

TCP

Spark shuffle service port (The service is disabled by default)

spark.shuffle.service.port

custom spark-defaults.conf

YARN NodeManager

7447

TCP

Spark shuffle service port

 — 

 — 

YARN NodeManager

9205

HTTP

Port to share monitoring metrics

Prometheus metrics port

Monitoring

YARN ResourceManager

8088/8090

HTTP/HTTPS

RM Web UI Port

yarn.resourcemanager.webapp.address, yarn.resourcemanager.webapp.https.address

yarn-site.xml (only 8090)

YARN ResourceManager

8030

HTTP

Port of the scheduler interface

yarn.resourcemanager.scheduler.address

A custom parameter in yarn-site.xml

YARN ResourceManager

8031

IPC

Resource tracker port

yarn.resourcemanager.resource-tracker.address

A custom parameter in yarn-site.xml

YARN ResourceManager

8032

IPC

Port of the application manager interface in the RM

yarn.resourcemanager.address

A custom parameter in yarn-site.xml

YARN ResourceManager

8033

IPC

Port of the RM admin interface

yarn.resourcemanager.admin.address

A custom parameter in yarn-site.xml

YARN ResourceManager

9204

HTTP

Port to share monitoring metrics

Prometheus metrics port

Monitoring

YARN Timeline Server

8188/8190

HTTP/HTTPS

TS Web UI port

yarn.timeline-service.webapp.https.address

yarn-site.xml

YARN Timeline Server

10200

IPC

TS Server port

Embedded in mapreduce.jobhistory.address

mapred-site.xml

YARN Timeline Server

9206

HTTP

Port to share monitoring metrics

Prometheus metrics port

Monitoring

Zeppelin

Component/Server Port Protocol Description Parameter Config

Zeppelin Server

8180/8180

HTTP/HTTPS

Zeppelin Web UI port

zeppelin.server.port, zeppelin.server.ssl.port

zeppelin-site.xml

Zeppelin Server

8180

HTTP

Prometheus metrics port

zeppelin.server.port

zeppelin-site.xml

Zookeeper

Component/Server Port Protocol Description Parameter Config

Zookeeper Server

2888

TCP

Port for cluster coordinating (internal cluster use only)

server.*

zoo.cfg

Zookeeper Server

3888

TCP

Port for cluster coordinating (internal cluster use only)

server.*

zoo.cfg

Zookeeper Server

2181

TCP

Port for clients connection

clientPort (client_port in ADCM)

zoo.cfg

Zookeeper Server

5181

HTTP

Zookeeper HTTP API interface

admin.serverPort (admin.serverPort in ADCM)

zoo.cfg

Zookeeper Server

7000

HTTP

Sets the Prometheus metrics port

metricsProvider.httpPort

zoo.cfg

JMX ports

Only the YARN RM port is defined in ADCM UI. Other JMX ports are hardcoded in service environment configs.

Service Component/Server Port Protocol Description Parameter Config

HBase

HBase Region

10102

TCP

JMX port (internal use with jmxtrans)

-Dcom.sun.management.jmxremote.port

hbase-regionserver-env.sh

HDFS

NameNode

1026

TCP

JMX port (internal use with jmxtrans)

-Dcom.sun.management.jmxremote.port

hadoop-env.sh

HDFS

DataNode

1027

TCP

JMX port (internal use with jmxtrans)

-Dcom.sun.management.jmxremote.port

hadoop-env.sh

Hive

Hive Server

1031

TCP

JMX port (internal use with jmxtrans)

-Dcom.sun.management.jmxremote.port

hive-server2-env.sh

Hive

Metastore

1032

TCP

JMX port (internal use with jmxtrans)

-Dcom.sun.management.jmxremote.port

hive-metastore-env.sh

Monitoring

jmxtrans

2101

TCP

JMX port for clients connection

-Dcom.sun.management.jmxremote.port

/etc/jmxtrans/wrapper.conf

Spark

History Server

18083

TCP

JMX port (internal use with jmxtrans)

-Dcom.sun.management.jmxremote.port

spark-env.sh

Spark3

History Server

18093

TCP

JMX port (internal use with jmxtrans)

-Dcom.sun.management.jmxremote.port

spark-env.sh

YARN

Resource Manager

1028

TCP

JMX port (internal use with jmxtrans)

-Dcom.sun.management.jmxremote.port (resource_manager_jmx_port in ADCM)

yarn-env.sh

YARN

NodeManager

1029

TCP

JMX port (internal use with jmxtrans)

-Dcom.sun.management.jmxremote.port

yarn-env.sh

YARN

MapReduce History Server

1030

TCP

JMX port (internal use with jmxtrans)

-Dcom.sun.management.jmxremote.port

mapred-env.sh

Ports redefined by Kerberos

When Kerberos is enabled, ADCM redefines ports from the table below by hardcoded values from Ansible config.

On disabling Kerberos, ADCM returns the default hardcoded values for these parameters.

Service Component/Server Port Protocol Description Kerberos Parameter

HDFS

DataNode

1022/50475

HTTP/HTTPS

DataNode Web UI port/ secure port

Enabled

dfs.datanode.http.address, dfs.datanode.https.address

HDFS

DataNode

1019

TCP

Data transfer port

Enabled

dfs.datanode.address

Client ports

Client ports are generated in the local port range. You can determine a specific port from this range using a known service port. Use the lsof command as follows:

$ sudo lsof -i -P -n | grep <known port namber>

Examples

  1. The Diamond client is running on a local random port. It connects to Graphite that listens the 2015 port.

    $ sudo lsof -i -P -n | grep 2015

    The result:

    python    6000       root    2u  IPv4 4811652      0t0  TCP 10.92.16.232:34758->10.92.16.30:2015 (ESTABLISHED)
  2. The Hive client is running on a local random port. It connects to ADPG that listens the 5432 port.

    $ sudo lsof -i -P -n | grep 5432

    The result:

    java       1161       hive  783u  IPv6 2043472      0t0  TCP 10.92.41.49:58094->10.92.41.242:5432 (ESTABLISHED)
Examples of client ports in the local port range
Component Port Protocol Description

Diamond (python-process, running by root)

34758 or another random port from local port range

TCP

Local jmxtrans client port for sending metrics to Monitoring cluster on port 2015 (to Graphite)

Hive

58094 or another random port from local port range

TCP

Local client port to connect to ADPG database on 5432 port

HBase, Spark, and etc.

50382 or another random port from local port range

TCP

Local client port to connect to HDFS on port 8020

Flink JobManager

43162 or another random port from local port range

TCP

Flink client connects to Zookeeper: TCP 10.92.17.55:42844→10.92.17.55:2181

Flink internal connect between JobManager’s nodes: TCP 10.92.17.55:60390→10.92.17.58:33769

ADCM ports

Port Protocol Description

22

TCP

ADCM connection port

Arenadata Enterprise Tools ports

Port Protocol Description

443

TCP

Port that is used by Docker Registry

81

TCP

Port that is used to access repositories during offline installation

Arenadata Monitoring ports

Port Protocol Description

3000

HTTP

HTTP port to access the Grafana web interface. You can change the default value when installing the monitoring cluster on the configuration page of the Grafana service in the ip_and_ports section

80

TCP

TCP port to access the Graphite web interface. You can change the default value when installing the monitoring cluster on the configuration page of the Graphite service in the ip_and_ports section

Found a mistake? Seleсt text and press Ctrl+Enter to report it