SSL encryption

Elena Dvoryadkina

For secure connections to ADQM clusters and protection of data exchange between service components, ADQM Control supports SSL encryption. To use this feature in your ADQM Control cluster, obtain certificates and enable SSL in ADCM UI.

 
The example in this article uses an ADQM Control cluster that includes two hosts, on which components of ADQM Control’s services are distributed as follows:

  • 10.92.41.249, dev-adqm-c-01.arenadata.local — all components of the ADQM Control service and the Zookeeper Server component of the Zookeeper service;

  • 10.92.41.23, dev-adqm-c-02.arenadata.local — the Arenadata PostgreSQL component of the ADPG service.

ADQM Control is integrated with an ADQM cluster for which SSL encryption of connections is enabled according to the tutorial from the Enable SSL encryption article of the ADQM documentation.

For demonstration and testing purposes, server SSL certificates are signed with a self-signed CA certificate (its creation is also described in the article with an example of setting up SSL for ADQM — see the steps 1 and 2 in the Create SSL certificates section). In production systems, use a trusted certificate authority to sign certificates.

Create SSL certificates

To create and register SSL certificates with the commands listed below, use the openssl utility (minimum version 1.1.1).

  1. For each host of the ADQM Control cluster, generate a key and create a certificate signing request (CSR):

    $ openssl req -newkey rsa:2048 -nodes \
  -subj "/CN=dev-adqm-c-01.arenadata.local" \
  -addext "subjectAltName = DNS:dev-adqm-c-01.arenadata.local,IP:10.92.41.249" \
  -keyout adqm-c-host-01.key -out adqm-c-host-01.csr
    $ openssl req -newkey rsa:2048 -nodes \
  -subj "/CN=dev-adqm-c-02.arenadata.local" \
  -addext "subjectAltName = DNS:dev-adqm-c-02.arenadata.local,IP:10.92.41.23" \
  -keyout adqm-c-host-02.key -out adqm-c-host-02.csr

    The CN (Common Name) certificate identifier should be the fully qualified domain name (FQDN) of a host for which the certificate is being created. Otherwise, the certificate cannot be verified.

  2. For each host, create and sign a certificate using the CA certificate and private key:

    $ openssl x509 -req \
  -in adqm-c-host-01.csr \
  -out adqm-c-host-01.crt \
  -CAcreateserial -CA arenadata_ca.crt -CAkey arenadata_ca.key \
  -days 365
    $ openssl x509 -req \
  -in adqm-c-host-02.csr \
  -out adqm-c-host-02.crt \
  -CAcreateserial -CA arenadata_ca.crt -CAkey arenadata_ca.key \
  -days 365

    As mentioned above, the server certificates in this example are signed by the same certificate authority as the certificates in the example of configuring SSL for an ADQM cluster — see the Enable SSL encryption article in the ADQM documentation for a description of how to create the CA certificate (arenadata_ca.crt) and private key (arenadata_ca.key) files.

  3. Copy files of server certificates and keys, as well as the CA certificate to the cluster hosts where components of ADQM Control’s services are installed. For the Backend, Agents, Alert Generator, and Alert Receiver components of the ADQM Control service, these files should be stored in the same path and named the same on each host where these components are installed. For the Alertmanager component and components of other services (ADPG, Zookeeper, Monitoring), separate directories are used to store these files. Paths and file names of server certificates/keys and CA certificates are specified in the SSL-related settings of the corresponding services of ADQM Control (see the SSL parameters in service configurations section below).

     
    In the current example, SSL will be enabled for all components of the ADQM Control service and for the ADPG service. Therefore, it is necessary to place the created certificate/key files on hosts of the ADQM Control cluster as follows:

    • For the Backend, Agents, Alert Generator, and Alert Receiver components of the ADQM Control service, copy files to the dev-adqm-c-01.arenadata.local host:

      • adqm-c-host-01.crt to /etc/adqmc/server.crt;

      • adqm-c-host-01.key to /etc/adqmc/server.key;

      • arenadata_ca.crt to /etc/adqmc/ca.crt.

    • For the Alertmanager component of the ADQM Control service, copy files to the dev-adqm-c-01.arenadata.local host:

      • adqm-c-host-01.crt to /etc/admprom/alert-manager/server.crt;

      • adqm-c-host-01.key to /etc/admprom/alert-manager/server.key;

      • arenadata_ca.crt to /etc/admprom/alert-manager/ca.crt.

    • For the ADPG service, copy files to the dev-adqm-c-02.arenadata.local host:

      • adqm-c-host-02.crt to /etc/adpg16/server.crt;

      • adqm-c-host-02.key to /etc/adpg16/server.key;

      • arenadata_ca.crt to /etc/adpg16/ca.crt.

    The /etc/adqmc/, /etc/admprom/alert-manager/, /etc/adpg16/ directories and the server.crt, server.key, ca.crt file names are set by default in the Certificate file, Private key file, Certificate authority file parameters in the SSL configuration configuration section of the ADQM Control and ADPG services. If the certificate and key files on your ADQM Control hosts have different names or are located in different directories, specify the corresponding paths in these parameters.

Configure ADQM Control to enable SSL encryption

You can enable SSL encryption of connections in ADQM Control using the ADCM interface, which provides the following functionality for this:

Manage SSL cluster action

  1. On the Clusters page, click the icon actions default dark actions default light in the ADQM Control cluster row to open the list of cluster actions and select Manage SSL.

    Launch the Manage SSL action
    Launch the "Manage SSL" action

    ADCM will display a dialog window. When you turn on the Enable SSL switch, it provides options allowing you to specify which services of ADQM Control should use secure connections.

    Options for enabling SSL in the Manage SSL action configuration window
    Options for enabling SSL in the "Manage SSL" action configuration window

  2. Fill in the required parameters in the Manage SSL action configuration window:

    • Parameters in the Enable SSL section — to enable SSL encryption of incoming and internal connections for various services of ADQM Control.

      Parameters to enable SSL for services

       

      Parameter values ​​specified in the Manage SSL action window overwrite the values ​​of the corresponding parameters in service configurations after the action execution — see the Service parameter column in the table below.

      Manage SSL parameter Parameter description Service parameter

      [ADPG] → Secure ADPG

      Enables SSL encryption of connections to the data storage

      ADPG service: Enable SSL parameter

      [ADQM Control] → Secure Backend

      Enables SSL encryption of connections to the Backend component of the ADQM Control service

      ADQM Control service: [ADQM Control’s Backend] → Enable SSL parameter

      [ADQM Control] → Secure ADQM Agent

      Enables SSL encryption of connections to the Agents component of the ADQM Control service

      ADQM Control service: [ADQM Control’s ADQM Agent] → Enable SSL parameter

      [ADQM Control] → Secure Alert Generator

      Enables SSL encryption of connections to the Alert Generator component of the ADQM Control service

      ADQM Control service: [ADQM Control’s Alert Generator] → Enable SSL parameter

      [ADQM Control] → Secure Alert Receiver

      Enables SSL encryption of connections to the Alert Receiver component of the ADQM Control service

      ADQM Control service: [ADQM Control’s Alert Receiver] → Enable SSL parameter

      [ADQM Control] → Secure Alertmanager

      Enables SSL encryption of connections to the Alertmanager component of the ADQM Control service

      ADQM Control service: [Alertmanager] → Enable SSL parameter

      [Zookeeper] → Secure Incoming Connections

      Enables SSL encryption of connections to ZooKeeper

      Zookeeper service: Enable SSL parameter

      [Zookeeper] → Secure SSL Quorum

      Enables SSL-encrypted quorum communication in ZooKeeper

      Zookeeper service: sslQuorum parameter

      [Monitoring] → Secure Prometheus

      Enables SSL encryption of connections to the Prometheus Server component of the Monitoring service

      Monitoring service: [Prometheus] → Enable SSL parameter

      [Monitoring] → Secure Grafana

      Enables SSL encryption of connections to the Grafana component of the Monitoring service

      Monitoring service: [Grafana] → Enable SSL parameter

      [Monitoring] → Secure Node-exporter

      Enables SSL encryption of connections to the Node Exporter component of the Monitoring service

      Monitoring service: [Node-exporter] → Enable SSL parameter

    • The Run service checks parameter — to specify whether to check the cluster health and availability after applying the settings.

  3. To launch the action, click Next and then Run in the confirmation window.

During the Manage SSL action execution, ADCM stops all services one by one, reconfigures them to enable or disable SSL encryption, then starts the services again and checks them.

You can view the result of the action and its execution process on the Jobs page, which displays the execution details for all internal steps of actions, including verbose Ansible stdout/stderr outputs.

SSL parameters in service configurations

An alternative way to set up secure connections in ADQM Control is to configure SSL settings for each service separately.

  1. Open the Services tab of your ADQM Control cluster and select the service for which you want to enable SSL encryption — ADPG, ADQM Control, Zookeeper, or Monitoring.

  2. On the Primary configuration tab, configure the SSL parameters — the name of the section where these parameters are located depends on the service.

    ADPG: SSL Configuration

     
    In the SSL Configuration section of the ADPG service configuration, the following parameters are available.

    Parameter Description Default value

    Enable SSL

    Enables secure incoming connections in the data storage

    Disabled

    Certificate file

    Path to the server SSL certificate file in the CRT format (*.crt file). The certificate file should be located at the specified path on the cluster host where the Arenadata PostgreSQL component of the ADPG service is installed

    /etc/adpg16/server.crt

    Private key file

    Path to the file with the private key of the server SSL certificate (*.key file). The key file should be located at the specified path on the cluster host where the Arenadata PostgreSQL component of the ADPG service is installed

    /etc/adpg16/server.key

    Certificate authority file

    Path to the CA certificate file in the CRT format (*.crt file). The CA certificate file should be located at the specified path on the cluster host where the Arenadata PostgreSQL component of the ADPG service is installed

    /etc/adpg16/ca.crt
    ADPG service configuration parameters to check SSL certificates of connections
    ADPG service configuration parameters to check SSL certificates of connections
    ADQM Control: SSL configuration

     
    In the SSL Configuration section of the ADQM Control service configuration, the following parameters are available.

    Parameter Description Default value

    [ADQM Control’s Backend] → Enable SSL

    Enables SSL encryption of incoming connections for the Backend component

    Disabled

    [ADQM Control’s ADQM Agent] → Enable SSL

    Enables SSL encryption of incoming connections for the Agents component

    Disabled

    [ADQM Control’s Alert Generator] → Enable SSL

    Enables SSL encryption of incoming connections for the Alert Generator component

    Disabled

    [ADQM Control’s Alert Receiver] → Enable SSL

    Enables SSL encryption of incoming connections for the Alert Receiver component

    Disabled

    [Alertmanager] → Enable SSL

    Enables SSL encryption of incoming connections for the Alertmanager component

    Disabled

    [ADQM Control Services] → Certificate file

    Path to the server SSL certificate file in the CRT format (*.crt file) for the Backend, Agents, Alert Generator, and Alert Receiver components. The certificate file should be located at the specified path on each host of the cluster where these components are installed

    /etc/adqmc/server.crt

    [ADQM Control Services] → Private key file

    Path to the file with the private key of the server SSL certificate (*.key file) for the Backend, Agents, Alert Generator, and Alert Receiver components. The key file should be located at the specified path on each host of the cluster where these components are installed

    /etc/adqmc/server.key

    [ADQM Control Services] → Certificate authority file

    Path to the CA certificate file in the CRT format (*.crt file) for the Backend, Agents, Alert Generator, and Alert Receiver components. The CA certificate file should be located at the specified path on each host of the cluster where these components are installed

    /etc/adqmc/ca.crt

    [Alertmanager] → Certificate file

    Path to the server SSL certificate file in the CRT format (*.crt file) for the Alertmanager component. The certificate file should be located at the specified path on the cluster host where the Alertmanager component is installed

    /etc/admprom/alert-manager/server.crt

    [Alertmanager] → Private key file

    Path to the file with the private key of the server SSL certificate (*.key file) for the Alertmanager component. The key file should be located at the specified path on the cluster host where the Alertmanager component is installed

    /etc/admprom/alert-manager/server.key

    [Alertmanager] → Certificate authority file

    Path to the CA certificate file in the CRT format (*.crt file) for the Alertmanager component. The CA certificate file should be located at the specified path on the cluster host where the Alertmanager component is installed

    /etc/admprom/alert-manager/ca.crt
    Configuration parameters of the ADQM Control service to check SSL certificates of connections
    Configuration parameters of the ADQM Control service to check SSL certificates of connections
    ADQM Control: ADQM management

     
    The Connection type parameter in the ADQM management section of the ADQM Control service configuration allows you to enable the use of only secure TCP connections between ADQM Control and ADQM. If the ADQMDB service’s Network → Enable tcp parameter in ADQM is set to false, the Connection type parameter of ADQM Control should be set to Secure (TCP secure).

    Enable the use of only secure TCP connections to ADQM
    Enable the use of only secure TCP connections to ADQM
    ADQM Control: Zookeeper configuration

     

    In the Zookeeper configuration section of the ADQM Control service configuration, the SSL connection parameter allows you to enable SSL encryption of connections to ZooKeeper.

    Turn on secure connections to Zookeeper
    Turn on secure connections to Zookeeper
    Zookeeper: SSL configuration

     

    The SSL configuration section on the Zookeeper service configuration page contains the following parameters to set up SSL encryption of incoming and internal connections in ZooKeeper (keystore and truststore files should be stored at the specified paths on all cluster hosts where Zookeeper Server is installed).

    Parameter Description Default value

    Enable SSL

    Enables secure incoming connections in ZooKeeper

    Disabled

    sslQuorum

    Enables encrypted quorum communication

    Disabled

    serverCnxnFactory

    Specifies ServerCnxnFactory implementation (to use TLS-based server communication, set it to NettyServerCnxnFactory)

    org.apache.zookeeper.server.NettyServerCnxnFactory

    ssl.quorum.keyStore.location

    Path to the server keystore file

    /etc/zookeeper/server.jks

    ssl.quorum.keyStore.password

    Password for the keystore specified in ssl.quorum.keyStore.location

     — 

    ssl.quorum.trustStore.location

    Path to the server truststore file

    /etc/zookeeper/ca.jks

    ssl.quorum.trustStore.password

    Password for the truststore specified in ssl.quorum.trustStore.location

     — 

    ssl.keyStore.location

    Path to the server keystore file to be used for client TLS connections

    /etc/zookeeper/server.jks

    ssl.keyStore.password

    Password for the keystore specified in ssl.keyStore.location

     — 

    ssl.trustStore.location

    Path to the server truststore file to be used for client TLS connections

    /etc/zookeeper/ca.jks

    ssl.trustStore.password

    Password for the truststore specified in ssl.trustStore.location

     — 

    ssl.protocol

    Protocol to be used in client TLS negotiation

    TLSv1.2

    ssl.quorum.protocol

    Protocol to be used in quorum TLS negotiation

    TLSv1.2

    secureClientPort

    Port to be used for secure client connections

    2281
    Zookeeper service configuration parameters for SSL connections
    Zookeeper service configuration parameters for SSL connections
    Monitoring: SSL configuration

     

    The following parameters in the SSL configuration section of the Monitoring service configuration allows you to enable SSL-encrypted connections to service components (certificate/key files should be stored at the specified paths on all cluster hosts where the corresponding components of monitoring are installed).

    Parameter Description Default value

    [Prometheus] → Enable SSL

    Enables SSL encryption of incoming connections for Prometheus Server

    Disabled

    [Prometheus] → Certificate file

    Path to the server SSL certificate file in the CRT format (*.crt file) for Prometheus Server

    /etc/admprom/prometheus/server.crt

    [Prometheus] → Private key file

    Path to the file with the private key of the server SSL certificate (*.key file) for Prometheus Server

    /etc/admprom/prometheus/server.key

    [Prometheus] → Certificate authority file

    Path to the file with the CA certificate in the CRT format (*.crt file) to verify Prometheus targets

    /etc/admprom/prometheus/ca.crt

    [Grafana] → Enable SSL

    Enables SSL encryption of incoming connections for Grafana

    Disabled

    [Grafana] → Certificate file

    Path to the server SSL certificate file in the CRT format (*.crt file) for Grafana

    /etc/admprom/grafana/server.crt

    [Grafana] → Private key file

    Path to the file with the private key of the server SSL certificate (*.key file) for Grafana

    /etc/admprom/grafana/server.key

    [Grafana] → Certificate authority file

    Path to the file with the CA certificate in the CRT format (*.crt file) to verify Prometheus certificate

    /etc/admprom/grafana/ca.crt

    [Node-exporter] → Enable SSL

    Enables SSL encryption of incoming connections for node exporters

    Disabled

    [Node-exporter] → Certificate file

    Path to the server SSL certificate file in the CRT format (*.crt file) for node exporters

    /etc/ssl/server.crt

    [Node-exporter] → Private key file

    Path to the file with the private key of the server SSL certificate (*.key file) for node exporters

    /etc/ssl/server.key
    Configuration parameters of the Monitoring service for SSL connections
    Configuration parameters of the Monitoring service for SSL connections

  3. After specifying all the necessary parameters for configuring SSL connections, click Save and run Reconfigure and restart for a service to apply the changes.

Example

  1. In the ADQM management section of the ADQM Control service configuration, set the Connection type parameter to Secure (TCP secure) so that ADQM Control can connect to ADQM only through the TCP port for SSL-secured communication (ADQM secure port).

    Option to enable the use of only secure TCP connections to ADQM
    Option to enable the use of only secure TCP connections to ADQM

    Click Save.

  2. Run the Manage SSL action for the cluster. In the configuration window that appears, activate Enable SSL and also turn on the following options to encrypt traffic between ADQM Control service components:

    • [ADPG] → Secure ADPG

    • [ADQM Control] → Secure Backend

    • [ADQM Control] → Secure ADQM Agent

    • [ADQM Control] → Secure Alert Generator

    • [ADQM Control] → Secure Alert Receiver

    • [ADQM Control] → Secure Alertmanager

      Enable SSL in ADQM Control via the Manage SSL cluster action
      Enable SSL in ADQM Control via the "Manage SSL" cluster action

Test a connection

To check that ADQM Control works in secure mode via HTTPS, you can open its web interface at https://10.92.41.249:5555 (10.92.41.249 in this example is the IP address of the host where the Backend component is installed). Note that you cannot now access ADQM Control via HTTP.

Disable SSL

To disable SSL, run the Manage SSL action with the Enable SSL switch inactive or turn off the Enable SSL option in the configuration of a service.

Disable SSL with the Manage SSL action
Disable SSL with the "Manage SSL" action
Found a mistake? Seleсt text and press Ctrl+Enter to report it