Use ADCM Installer

Adelya Zvyagintseva

ADCM Installer is a tool written in Go to help simplify the processes of installation, upgrade, and configuration of the ADCM installations. It also allows you to extend the Docker Compose functionality by secret storage.

ADCM Installer consists of the following packages:

  • Docker Compose — controls the life cycle of Docker containers.

  • age-encryption — encodes and decodes sensitive data (such as username and password to connect to the database).

For details of possible ADCM Installer use cases, see Use cases.

Quick start with ADCM Installer

NOTE
If you use CentOS, the operating system version must be at least version 8.

Follow the instructions below to install ADCM Installer:

  1. Install Docker. For more information, refer to the Docker documentation.

  2. Download the adi file to the /usr/local/bin directory and set permissions of the adi file as follows:

    $ sudo wget -O /usr/local/bin/adi https://github.com/arenadata/adcm-installer/releases/latest/download/adi
$ sudo chmod +x /usr/local/bin/adi

    Wait for the file download to finish.

IMPORTANT

For offline installation, you need to use the URL of the local Container Registry:

$ sudo wget -O /usr/local/bin/adi <local_container_registry_url>/arenadata/adcm-installer/releases/download/<version>/adi

To get a brief background on the ADCM Installer commands, use the following command:

$ adi --help

The expected command execution result:

Command line tool for installing Arenadata products

Usage:
  adcm-installer [flags]
  adcm-installer [command]

Available Commands:
  adcm-versions List versions of Arenadata products
  apply         Apply a configuration by file name
  completion    Generate the autocompletion script for the specified shell
  components    Manage installed components
  delete        Delete resources by file name or installation name
  help          Help about any command
  init          Initialize a new configuration
  list          List running ADCM installation
  secrets       Manage secrets

Flags:
  -h, --help      help for adcm-installer
      --version   Print the version and exit

Use cases

ADCM Installer supports the following commands:

  • init

  • apply

  • completion

  • list

  • adcm-versions

  • secrets

  • delete

The command descriptions and examples of their usage are given below.

Initialize a configuration file

The init command generates the configuration file described below, which is necessary for the ADCM installation deployment.

For example, to initialize the configuration file of the ADCM installation using PostgreSQL as a database, use the following command with the --adpg flag:

$ adi init <installation_name> --adpg

where <installation_name> is the name of the ADCM installation such as my-adcm, that includes the Docker containers: my-adcm-adpg and my-adcm-adcm.

To specify the database credentials and configure the database connection settings, use the interactive mode by specifying the -i or --interactive flag:

$ adi init <installation_name> --adpg -i
NOTE
Without the --adpg flag, settings for connection to an external PostgreSQL database will be requested.

Database connection parameters can also be specified in a file using the --from-config flag.

Parameters that can be specified in the file are presented in the table below. The table also describes a correlation between parameters that can be configured in the interactive mode and parameters that can be specified in the file.

Parameters
Parameter used in the file Parameter displayed in interactive mode Type Description Default value

adcm-count

Number of ADCM instances

uint8

Number of the ADCM instances. The parameter is available in ADCM Installer starting with version 1.1

1

adcm-image

ADCM image

string

ADCM container image repository (in the above example — my-adcm-adcm)

hub.arenadata.io/adcm/adcm

adcm-tag

ADCM image tag

string

Docker image tag used for the ADCM container (in the above example — my-adcm-adcm)

2.7.1

adcm-publish-port

ADCM publish port

uint16

Port of localhost on which the ADCM container runs (in the above example — my-adcm-adcm)

If you have one ADCM instance, the default value is 8000. If you have several ADCM instances, the amount of the increment for each instance is 1

adcm-db-host

ADCM database host

string

ADCM database server address. This parameter is ignored when specifying the --adpg flag in the interactive mode

 — 

adcm-db-port

ADCM database port

uint16

Port number for connecting to the ADCM database. This parameter is ignored when specifying the --adpg flag in the interactive mode

5432

adcm-db-name

ADCM database name

string

Name of the ADCM database

If you have one ADCM instance, the default value is adcm. If you have several ADCM instances, the default value is adcm_<instance_serial_number>

adcm-db-user

ADCM database user

string

Username for connecting to the ADCM database

If you have one ADCM instance, the default value is adcm. If you have several ADCM instances, the default value is adcm_<instance_serial_number>

adcm-db-pass

ADCM database password

string

Password for connecting to the ADCM database

 — 

adcm-db-ssl-mode

Postgres SSL mode

string

Flag used to connect to the ADCM database for establishing a TLS connection. This parameter is ignored when specifying the --adpg flag in the interactive mode. Possible values:

  • disable

  • allow

  • prefer

  • require

  • verify ca

  • verify full

The description of each value is given in the Installation article

disable

adcm-db-ssl-ca-file

ADCM database SSL CA file path

string

Path to the certificate authority (CA) certificate file for establishing a TLS connection to the ADCM database

 — 

adcm-db-ssl-cert-file

ADCM database SSL certificate file path

string

Path to the certificate file in PEM format for establishing a TLS connection to the ADCM database

 — 

adcm-db-ssl-key-file

ADCM database SSL private key file path

string

Path to the private key file in PEM format for establishing a TLS connection to the ADCM database

 — 

adcm-url

ADCM url

uint16

URL of the host where the ADCM container is deployed. Use the following format: http://<ip_address>:<port>/

If you have one ADCM instance, the default value is http://<ip_address>:8000/. If you have several ADCM instances, port number for each instance increases by 1. For example, for the first instance, the default value is http://<ip_address>:8001/

adcm-volume

ADCM volume name or path

string

Name of the ADCM container volume (in the above example — my-adcm-adcm)

<installation_name>-adcm-<instance_serial_number>

adcm-publish-ssl-port

ADCM publish SSL port

uint16

Localhost port for the ADCM container (in the above example — my-adcm-adcm) to establish TLS connection

8443

adcm-ssl-cert-file

ADCM SSL Certificate file path

string

Path to the certificate file in PEM format for establishing a TLS connection to ADCM

 — 

adcm-ssl-key-file

ADCM SSL Private Key file path

string

Path to the private key file in PEM format for establishing a TLS connection to ADCM

 — 

adpg-pass

ADPG superuser password

string

Password of the superuser role which is set during the PostgreSQL database initialization. The parameter is available when specifying the --adpg flag in the interactive mode

 — 

adpg-image

ADPG image

string

ADPG container image repository (in the above example — my-adcm-adpg). The parameter is available when specifying the --adpg flag in the interactive mode

hub.arenadata.io/adcm/postgres

adpg-tag

ADPG image tag

string

Docker image tag used for the ADPG container (in the above example — my-adcm-adpg). The parameter is available when specifying the --adpg flag in the interactive mode

v16.3.1

adpg-publish-port

ADPG publish port

uint16

Port of localhost on which the ADPG container runs (in the above example — my-adcm-adpg). The parameter is available when specifying the --adpg flag in the interactive mode

5432

adpg-volume

ADPG volume name or path

string

Name of the ADPG container volume (in the above example — my-adcm-adpg). The parameter is available when specifying the --adpg flag in the interactive mode

<installation_name>-adpg

consul-image

Consul image

string

Consul container image repository. The parameter is available in ADCM Installer starting with version 1.1 (in the interactive mode, you need to specify the --consul flag)

hub.arenadata.io/adcm/consul

consul-tag

Consul image tag

string

Docker image tag used for the Consul container. The parameter is available in ADCM Installer starting with version 1.1 (in the interactive mode, you need to specify the --consul flag)

v0.0.0

consul-publish-port

Consul publish port

uint16

Port of localhost on which the Consul container runs. The parameter is available in ADCM Installer starting with version 1.1 (in the interactive mode, you need to specify the --consul flag)

8500

vault-mode

Vault Deployment mode

boolean

Flag used to select the mode in which OpenBao will be launched. The parameter is available in ADCM Installer starting with version 1.1 (in the interactive mode, you need to specify the --vault flag). Possible values:

  • non-ha — persistent mode;

  • ha — high availability (HA) mode;

  • dev — development mode.

Description of the modes is given below

non-ha

vault-ui

Vault enable UI

boolean

Flag used to enable access to the OpenBao web interface (for example, to create or update secrets). The parameter is available in ADCM Installer starting with version 1.1 (in the interactive mode, you need to specify the --vault flag). Possible values:

  • true — enebles access to the web interface;

  • false — disables access to the web interface.

true

vault-image

Vault image

string

OpenBao container image repository. The parameter is available in ADCM Installer starting with version 1.1 (in the interactive mode, you need to specify the --vault flag)

openbao/openbao

vault-tag

Vault image tag

string

Docker image tag used for the OpenBao container. The parameter is available in ADCM Installer starting with version 1.1 (in the interactive mode, you need to specify the --vault flag)

2.2.0

vault-publish-port

Vault publish port

uint16

Port of localhost on which the OpenBao container runs. The parameter is available in ADCM Installer starting with version 1.1 (in the interactive mode, you need to specify the --vault flag)

8200

vault-db-host

Vault database host

string

OpenBao database server address. The parameter is available in ADCM Installer starting with version 1.1 (in the interactive mode, you need to specify the --vault flag)

1

vault-db-port

Vault database port

uint16

Port number for connecting to the OpenBao database. The parameter is available in ADCM Installer starting with version 1.1 (in the interactive mode, you need to specify the --vault flag)

5432

vault-db-name

Vault database name

string

Name of the OpenBao database. The parameter is available in ADCM Installer starting with version 1.1 (in the interactive mode, you need to specify the --vault flag)

vault

vault-db-user

Vault database user

string

Username for connecting to the OpenBao database. The parameter is available in ADCM Installer starting with version 1.1 (in the interactive mode, you need to specify the --vault flag)

vault

vault-db-pass

Vault database password

string

Password for connecting to the OpenBao database. The parameter is available in ADCM Installer starting with version 1.1 (in the interactive mode, you need to specify the --vault flag)

 — 

vault-db-ssl-mode

Postgres SSL mode

string

Flag used to connect to the OpenBao database for establishing a TLS connection. The parameter is available when specifying the --vault flag in the interactive mode. Possible values:

  • disable

  • allow

  • prefer

  • require

  • verify ca

  • verify full

disable

vault-db-ssl-ca-file

Vault database SSL CA file path

string

Path to the certificate authority (CA) certificate file for establishing a TLS connection to the OpenBao database. The parameter is available in ADCM Installer starting with version 1.1 (in the interactive mode, you need to specify the --vault flag)

 — 

vault-db-ssl-cert-file

Vault database SSL certificate file path

string

Path to the certificate file in PEM format for establishing a TLS connection to the OpenBao database. The parameter is available in ADCM Installer starting with version 1.1 (in the interactive mode, you need to specify the --vault flag)

 — 

vault-db-ssl-key-file

Vault database SSL private key file path

string

Path to the private key file in PEM format for establishing a TLS connection to the OpenBao database. The parameter is available in ADCM Installer starting with version 1.1 (in the interactive mode, you need to specify the --vault flag)

 — 

vault-ssl-cert-file

Vault SSL Certificate file path

string

Path to the certificate file in PEM format for establishing a TLS connection to OpenBao. The parameter is available in ADCM Installer starting with version 1.1 (in the interactive mode, you need to specify the --vault flag)

 — 

vault-ssl-key-file

Vault SSL Private Key file path

string

Path to the private key file in PEM format for establishing a TLS connection to OpenBao. The parameter is available in ADCM Installer starting with version 1.1 (in the interactive mode, you need to specify the --vault flag)

 — 

If parameter values are not specified, default values will be used for names and Docker container configuration parameters and passwords will be generated automatically.

After the command mentioned above is executed, the following files will be created:

  • adcm.yaml (or adcm.yml) is the ADCM installation configuration file in the YAML format;

  • age.key is the file with the private key in the AGE-SECRET-KEY-1… format to access sensitive data (secrets) stored in the adcm.yaml (or adcm.yml) file.

To take advantage of additional command options, use the following flags:

  • --force — allows you to overwrite the existing configuration file;

  • -o or --output — allows you to save the configuration file with the given name, for example: --output /<file_path>, where <file_path> is the absolute path to the configuration file in the YAML format including the file name;

  • --age-key-file — determines the location of the file with the private key;

  • --age-key — determines the private key, for example:

    $ adi init <installation_name> --age-key AGE-SECRET-KEY-1...
    NOTE
    The private key can also be passed via the AGE_KEY variable, for example: export AGE_KEY=AGE-SECRET-KEY-1…​.
Example of the configuration file for deploying ADCM with a PostgreSQL database
# Project name
name: my-adcm
# Configurations of the following Docker containers: my-adcm-adcm and my-adcm-adpg
services:
  adcm:
    # Add specified Linux capabilities to the Docker container. The list of supported capabilities is given in the Linux documentation
    cap_add:
      - CAP_CHOWN
      - CAP_SETUID
      - CAP_SETGID
    # Remove specified Linux capabilities from the Docker container
    cap_drop:
      - ALL
    # Order in which the Docker containers are started. In this case, the my-adcm-adcm container will be started after the my-adcm-adpg container startup
    depends_on:
      adpg:
        # Condition under which the my-adcm-adcm container is started. In this case -- wait until the my-adcm-adpg container is healthy
        condition: service_healthy
        restart: true
        required: true
    # Custom hostname for the Docker container
    hostname: my-adcm-adcm
    # Image name
    image: hub.arenadata.io/adcm/adcm:2.5.0
    # Metadata to add to the Docker container
    labels:
      app.arenadata.io/type: adcm
    # Create a dedicated network for the Docker container on the host
    networks:
      default: null
    # Map ports between the Docker container and the host
    ports:
      - mode: ingress
        # Port of the Docker container
        target: 8000
        # Published port
        published: "8000"
        # Port protocol
        protocol: tcp
    # Pull the Docker image from the registry for every container startup
    pull_policy: always
    # Policy of the Docker container restart. In this case, the policy restarts the Docker container in all cases, except forcible stop of the Docker container
    restart: unless-stopped
    # Provide the Docker container with access to sensitive data defined by the x-secrets top-level element
    secrets:
      - source: adcm-db-name
        target: /run/secrets/adcm-db-name
      - source: adcm-db-user
        target: /run/secrets/adcm-db-user
      - source: adcm-db-pass
        target: /run/secrets/adcm-db-pass
    # Named volumes that are accessible by the Docker container
    volumes:
      - type: volume
        source: my-adcm-adcm
        target: /adcm/data
        volume: {}
  adpg:
    cap_drop:
      - ALL
    hostname: my-adcm-adpg
    # Docker container health check using the pg_isready utility
    healthcheck:
      test:
        - CMD-SHELL
        - pg-entrypoint isready postgres
      timeout: 3s
      interval: 10s
      retries: 3
    image: mybackspace/adpg:16.4
    labels:
      app.arenadata.io/type: adpg
    networks:
      default: null
    pull_policy: always
    # Run the Docker container with a read-only internal filesystem
    read_only: true
    restart: unless-stopped
    # Prevent the Docker container processes from gaining additional privileges
    security_opt:
      - no-new-privileges
    # User used to run the Docker container
    user: 10001:10001
    volumes:
      - type: volume
        source: my-adcm-adpg
        target: /data
        volume: {}
# Named network that allows Docker containers to communicate with each other
networks:
  default:
    name: my-adcm_default
# Named volumes that will be created with the Docker container initial startup
volumes:
  my-adcm-adcm:
    name: my-adcm-adcm
  my-adcm-adpg:
    name: my-adcm-adpg
# Sensitive data that is granted to the Docker containers
x-secrets:
  age_recipient: age1htxwzyy75rlrhut6jry0uet3ks7cw44w80jl2gktukp9m800auyqspyfws
  data:
    adcm-db-name: YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBoMWh2d2E5TU8xR0RkQVdCalB6ZUEvZUh5d2ZqN3hpbEJBK2taMHZCM3hvCkNzbVNudHlSVUIxY0hCT3lDaHJNVld0VHlHK1hRU2RjMzRnUGxjaUdBUDAKLS0tIEFIcmVEVnpQVE5iVFdwQ0NRcVpNQ0w5TWRRQTB3ZmZBemZVbFcySk5ZOEEK3Y7P8Lgl0+Gx9+UwJQOzRheRbVxox/XaXEj6M1xMukn56AgJ
    adcm-db-pass: YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBWRXR6eEduL3piZk1oODd5QnNENUZPTHhzMXhxczRMNTVhd0VrUWJoYlNVCnV5OHBadTlpMlpTWnc4SXFoMThqbHE4UVBheVloNnk1U0o1N2E2dHZaZFEKLS0tIFViUm96eXJFWWJJY3o1WHlNYmdneVRQUTVRL3ZKbFlhVFF2Z2l4cDZ0QUUKr7EYAMlS/h6TeoZV4bgBKqlVDUysOKGgAU1qWjCOezUWA/2hARb1Kt928jB+Qn8i
    adcm-db-user: YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAzYm95bDZDV1NrbGpuYWdXN21IaUZwSFVaSHlVNVZXd2tGR1RhWE1XcTFjCnY0RUFJQnZwcUxWWk40bjUxWi9xdHM3TFpOUTZlYWk0elpSU21mUnlTbWcKLS0tIHJvYjZHNGk4UUNDMnFCd3NFVGsvV0tyYWprQUw2Wm1pMGNHMWRYYkIwTlEK4pYnqi8iDd2GefTUK2SLz8Q3BO5Re4DHmV33+TKiUQcjTXV6
    adpg-password: YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBObFI3MUptcW1ZdSs0ZGc2Tlp3SXprRGptdWhld01RekxjakZNQ1NIWFZNCnZDbzYzUXVzNUtvS3dYbkxhNnNNV2k2TXJCZWNQeVI2Y2c0aXlKdjk0bUUKLS0tIDZEUWFNRGRScFN0Y2s5WUQwMzUzQTJKNTAzRTBmVXhWNFQ0V2tXZUN6S00KTA1a/+YkjQ1bzr4/9I5GTHAnFHgCwblCW656C8O2qyw2NmbZtgeypIWf6n1yz2wg

OpenBao

Starting with ADCM Installer 1.1, it is possible to configure and deploy OpenBao for identity-based secrets and encryption management.

ADCM Installer allows you to run OpenBao in the following modes:

  • Dev mode (by default) — enables the following options:

    • automatically initialize and unseal the OpenBao server;

    • store data (in an encrypted form) in memory without requiring access rights;

    • store an OpenBao root token that can be used to access the OpenBao command line interface (CLI).

    IMPORTANT
    It is not recommended to use the development mode in a production environment, as data will be lost after restarting the OpenBao server. Use this mode only for testing purposes.

  • Persistent mode — the exact opposite of the dev mode which allows you to use the PostgreSQL storage backend to persist data after restarting the OpenBao server.

  • High availability (HA) mode — protects from failures by running several OpenBao servers using the PostgreSQL storage backend.

    NOTE
    You can run OpenBao in the high availability mode by running ADCM Installer on the required hosts.

For more information on OpenBao features and functions, please refer to the OpenBao documentation.

For example, to create a configuration file for deploying the ADCM installation with a PostgreSQL database and installing OpenBao, perform the following steps:

  1. Run the init command by specifying the --vault flag:

    $ adi init <installation_name> --adpg --vault -i

  2. Determine the values ​​of parameters required to deploy the ADCM installation with a PostgreSQL database and to install OpenBao. The parameters are described in the table.

HashiCorp Consul

Starting with ADCM Installer 1.1, it is possible to configure and deploy HashiCorp Consul in the dev mode. HashiCorp Consul is a solution for network services that allows you to coordinate secure communication between services and check their availability. HashiCorp Consul can be used instead of the status-checker in all products (ADB Control, ADH, and so on).

NOTE
HashiCorp Consul provides the ability to register services, but it does not automatically do it. Products should register their services in HashiCorp Consul on their own.

To create a configuration file for deploying the ADCM installation with a PostgreSQL database and installing HashiCorp Consul, perform the following steps:

  1. Run the init command by specifying the --consul flag:

    $ adi init <installation_name> --adpg --consul -i

  2. Determine the values ​​of parameters required to deploy the ADCM installation with a PostgreSQL database and to install HashiCorp Consul. The parameters are described in the table.

Deploy the ADCM installation

The apply command allows you to deploy the ADCM installation with the created files adcm.yaml (or adcm.yml), and age.key.

$ sudo adi apply

When you run the command, the docker-compose.yaml file is generated which participates in the following actions:

  1. Creating and running the my-adcm-adpg container.

  2. Creating the PostgreSQL database.

  3. Creating and running the my-adcm-adcm container.

To ensure that the ADCM installation is deployed successfully, run the commands described in the Installation article.

To deploy the ADCM installation in debug mode, specify the --dry-run flag. As a result, the docker-compose.yaml file is created without creating and running containers.

For more information on other flags that can be specified when running the apply command, use the following command:

$ adi apply --help
NOTE
With the --help flag, you can also get a brief description of other commands described below.

Create scripts with auto-complete

The completion command allows you to generate scripts with auto-complete for the ADCM Installer commands using the Tab key for the following shells:

  • Bash

  • fish

  • PowerShell

  • Zsh

NOTE
For Bash, first install the bash-completion package required for the script.

For more detailed instructions on how to use a generated script, run the following command:

$ adi completion <shell> --help

where <shell> is the shell name.

List running ADCM installations

The list command allows you to obtain the following information on the ADCM installations:

  • NAMES — the ADCM installation name;

  • STATUS — the status and the number of the Docker containers;

  • CONFIG FILES — the path to the configuration file.

Command example:

$ sudo adi list

The example of the command execution result:

NAMES               STATUS              CONFIG FILES
my-adcm             running(3)          /home/adelya/adcm-installer/adcm.yaml

To display information on stopped ADCM installations, specify the --all flag.

List available ADCM versions

The adcm-versions command allows you to check for the new ADCM image versions available for download from Arenadata Docker Registry:

$ adi adcm-versions --all

The example of the command execution result:

2.5.1
2.5.0
2.4.0
2.3.0
2.2.0
2.1.2
2.1.1
2.1.0
2.0.0

To display the last five versions, run the adcm-versions command without specifying the --all flag.

Manage secrets

The secrets command allows you to manage secrets specified in the configuration file as follows:

  • secrets new-key — creates a new private key which by default is saved in the age.key file;

  • secrets show — shows secrets defined during initialization;

  • secrets set — determines new secret values, for example:

    $ adi secrets set adcm-db-pass 1234

Stop and remove running ADCM installations

The delete command stops running ADCM installations using the specified configuration file or running installation name.

If adcm.yaml (or adcm.yml) or age.key are located in the directory in which the command is run, the delete command will be executed automatically without specifying the configuration file or the ADCM installation name:

$ sudo adi delete

As a result, the ADCM installation will be stopped with the preservation of data specified during initialization of the configuration file.

To remove a running ADCM installation along with volumes related to the installation, specify the --volumes flag. Confirm the removal during the execution of the command.

IMPORTANT
Note that running this command will permanently delete all information.
Found a mistake? Seleсt text and press Ctrl+Enter to report it