Configure a cluster

Olga Semme, Daria Barysheva

Contents

Primary configuration
Configuration groups
Ansible settings

IMPORTANT

There is usually no need to change cluster configuration parameters. You can leave all parameters at the default values.

After creation a new cluster, you can configure it by performing the following steps:

Select a cluster on the Clusters page. To do this, click a cluster name in the Name column.

Select a cluster
Open the Configuration tab on the cluster page. The Configuration tab includes the following sections: Primary configuration, Configuration groups, Ansible settings.
Fill in all necessary parameters on the selected tab and click Save.

Primary configuration

The Configuration → Primary configuration tab allows you to configure the general cluster settings.

Cluster configuration parameters are listed below:

Reliability control — parameters of this section are intended to set the timeout of the DNF software package manager unlocking wait. Displayed if the Show advanced switch is activated. In Ubuntu, after the host reboot, automatic update starts, which locks DNF.

Name

Description

Default value

retries

The maximum number of retries

delay

The time delay between retries (in seconds)

timeout

Timeout for network ports checks (in seconds)

300

Paths to the repositories that are used during the installation process depending on the cluster operating system (Ubuntu 22, Altlinux 10, Astra Linux, RedOS 7). Only the marked repositories are registered on the cluster servers.

Repositories

Repository name Parameter Description Default value

monitoring

desc

Repository description

Monitoring components url

url

Arenadata repository path

Specified depending on OS

enabled

Enables use of the repository

true

ADS

desc

Repository description

ADS repository

url

Arenadata repository path

Specified depending on OS

enabled

Enables use of the repository

true

prometheus_repo

desc

Repository description

Prometheus repository

url

Arenadata repository path

Specified depending on OS

enabled

Enables use of the repository

true

arenadata_java

desc

Repository description

Arenadata repository for OpenJDK

url

Arenadata repository path

Specified depending on OS

enabled

Enables use of the repository. If set to true before the Install, Add/Remove components, or Update action for a cluster or service, the latest version of Java 17 will be installed on hosts

false

ranger plugins

desc

Repository description

Arenadata Ranger plugins repository

url

Arenadata repository path

Specified depending on OS

enabled

Enables use of the repository

true

Precheck packages — enables package availability check before installation.
Docker registry url — specifies the location of Arenadata Docker Registry. Displayed if the Show advanced switch is activated.
Znode path for cluster in Zookeeper service — Znode path for the ADS cluster in the Zookeeper service. Displayed if the Show advanced switch is activated.
BASIC Authentication — configuration for BASIC authentication.

For a description of enabling authentication, see the article Basic authentication.
SASL_PLAINTEXT Authentication — configuration for SASL PLAINTEXT authentication.

For a description of enabling authentication, see the article SASL PLAINTEXT.
Kerberos — configuration for Kerberos authentication.

The description of enabling authentication is given in the articles:
java.io.tmpdir — a Java system property that determines where the JVM writes temporary files.

Custom Java config — a template for a custom configuration file that determines the versions of Java used for the cluster and services, as well as installation paths. This parameter can be used to explicitly specify the Java version. Starting from ADS 3.9.1.2.b1, it is used to explicitly define the Java version. For more details, see Software Requirements — Java.

The default section configures the default behavior when working without specifying a particular service.
The autodetect section defines the algorithm for automatically detecting different versions of Java. For each version (currently, 21), the following is configured:
- env_var — a special environment variable for this version (e.g. JAVA21_HOME).
- paths — a list of paths to search for the Java installation.
The services section defines the services for which specific versions of Java are configured. The settings are stored in separate YAML files.

AD runtime utils config template

default:
  runtimes:
    java:
      version: "{{ java_default_version }}"
      env_var: JAVA_HOME

autodetect:
  runtimes:
    java:
      "8":
        env_var: JAVA8_HOME
        paths:
          - /usr/java/jdk1.8
          - /usr/java/jre1.8
          - /usr/lib/jvm/j2sdk1.8-oracle
          - /usr/lib/jvm/j2sdk1.8-oracle/jre
          - /usr/lib/jvm/java-8-oracle
          - /usr/lib/jdk8-latest
          - /usr/lib/jvm/java-1.8.0
          - /usr/lib/jvm/java-1.8.0-oracle
          - /usr/lib/jvm/bellsoft-java8-amd64
          - /usr/lib/jvm/java-arenadata-openjdk-8

      "17":
        env_var: JAVA17_HOME
        paths:
          - /usr/lib/jvm/bellsoft-java17-amd64
          - /usr/lib/jvm/bellsoft-java17
          - /usr/lib/jvm/liberica-jdk-17*
          - /usr/lib/jvm/liberica-jre-17*
          - /usr/lib/jvm/zulu-17*
          - /usr/lib/jvm/temurin-17*
          - /usr/lib/jvm/adoptopenjdk-17*
          - /usr/lib/jvm/jdk-17*
          - /usr/lib/jvm/java-17*
          - /usr/lib/jvm/openjdk-17*
          - /usr/lib/jvm/openjdk-17*
          - /opt/java/bellsoft-java17-amd64
          - /opt/java/temurin-17*
          - /usr/local/java/bellsoft-java17-amd64
          - /usr/lib/jvm/java-arenadata-openjdk-17
          - /usr/lib64/jvm/java-17*

      "21":
        env_var: JAVA21_HOME
        paths:
          - /usr/lib/jvm/java-21*
          - /usr/lib/jvm/jdk-21*
          - /usr/lib/jvm/java-arenadata-openjdk-21

      "23":
        env_var: JAVA23_HOME
        paths:
          - /usr/lib/jvm/bellsoft-java23-amd64
          - /usr/lib/jvm/bellsoft-java23
          - /usr/lib/jvm/liberica-jdk-23*
          - /usr/lib/jvm/liberica-jre-23*
          - /usr/lib/jvm/zulu-23*
          - /usr/lib/jvm/temurin-23*
          - /usr/lib/jvm/adoptopenjdk-23*
          - /usr/lib/jvm/jdk-23*
          - /usr/lib/jvm/java-23-openjdk
          - /usr/lib/jvm/openjdk-23*
          - /opt/java/bellsoft-java23-amd64
          - /opt/java/temurin-23*
          - /usr/local/java/bellsoft-java23-amd64
          - /usr/lib/jvm/java-arenadata-openjdk-23

      "24":
        env_var: JAVA24_HOME
        paths:
          - /usr/lib/jvm/bellsoft-java24-amd64
          - /usr/lib/jvm/bellsoft-java24
          - /usr/lib/jvm/liberica-jdk-24*
          - /usr/lib/jvm/liberica-jre-24*
          - /usr/lib/jvm/zulu-24*
          - /usr/lib/jvm/temurin-24*
          - /usr/lib/jvm/adoptopenjdk-24*
          - /usr/lib/jvm/jdk-24*
          - /usr/lib/jvm/java-24-openjdk
          - /usr/lib/jvm/openjdk-24*
          - /opt/java/bellsoft-java24-amd64
          - /opt/java/temurin-24*
          - /usr/local/java/bellsoft-java24-amd64
          - /usr/lib/jvm/java-arenadata-openjdk-24

services:
  ZOOKEEPER:
    path: /etc/zookeeper/conf/zookeeper-java.yaml
  KAFKA:
    path: /etc/kafka/conf/kafka-java.yaml
  KAFKA_CONTROLLER:
    path: /etc/kafka/conf/kafka-controller-java.yaml
  KAFKA_CONNECT:
    path: /etc/kafka-connect/config/kafka-connect-java.yaml
  CRUISE_CONTROL:
    path: /etc/cruise-control/conf/cruise-control-java.yaml
  KSQL:
    path: /etc/ksqldb/ksqldb-java.yaml
  KAFKA_REST:
    path: /etc/kafka-rest/kafka-rest-java.yaml
  SCHEMA_REGISTRY:
    path: /etc/schema-registry/schema-registry-java.yaml
  NIFI:
    path: /etc/nifi/conf/nifi-java.yaml
  NIFI_REGISTRY:
    path: /etc/nifi-registry/conf/nifi-registry-java.yaml
  MINIFI:
    path: /etc/minifi/conf/minifi-java.yaml
  MINIFI_C2:
    path: /etc/minifi-c2/conf/minific2-java.yaml
  NIFI2:
    path: /etc/nifi2/conf/nifi-java.yaml
  NIFI2_REGISTRY:
    path: /etc/nifi2-registry/conf/nifi-registry-java.yaml

Cluster UUID — displays the universally unique identifier of the cluster.

Cluster configuration window

Configuration groups

The Configuration → Configuration groups tab is designed to set cluster configuration groups.

Ansible settings

The Configuration → Ansible settings tab allows you to set Ansible configuration options at the cluster level. The tab is available starting with ADCM 2.2.0.

Ansible settings
Name	Description	Default
forks	The number of parallel processes to spawn when communicating with remote hosts	5

Found a mistake? Seleсt text and press Ctrl+Enter to report it