Migration of ADB Control and ADBM metadata during ADB ES installation

Starting with ADB 6.30.0.1, ADB Control and ADBM come with a separate ADB Enterprise Services (ADB ES) bundle. On the ADB side, only agents are left: ADBM agents and ADBC agents. During initial ADB ES cluster installation, you should perform the following steps for a proper ADB upgrade (to version 6.30.0.1), including ADB Control and ADBM metadata migration from ADB to ADB ES.

IMPORTANT
  • The following steps should be performed one time during the initial ADB ES installation and ADB upgrade to version 6.30.0.1 if ADB previously used ADB Control or ADBM. If these services were not installed in your ADB cluster before, none of the following steps is required — instead, perform the standard ADB cluster upgrade and offline installation of ADB ES (then, add ADBC and ADBM agents to ADB, import ADB ES configuration to ADB, and install agents in ADB).

  • The minimal ADB version for upgrade is 6.27.1.63.

Prerequisites

Names of the external PostgreSQL databases for ADB Control and ADBM services in the ADB ES cluster (regardless of whether these databases were added earlier or will be created when you follow the steps listed below) should be generated according to the following pattern:

  • ADB Control — <prefix>_adcc

  • ADBM — <prefix>_adbm

The <prefix> value should be equal for both services. Subsequently, that value should be specified in the Database service parameter External postgres parameters → Database name prefix. Examples of valid database names: external_adcc and external_adbm (where external is a prefix).

If external databases were previously created with names that do not meet the above-mentioned pattern, perform renaming after ADB upgrade before ADB ES installation (see Step 3. Rename external PostgreSQL databases).

Step 1. Migrate metadata stored in PostgreSQL

IMPORTANT
  • You should perform step 1 if an external PostgreSQL database was not used for ADB Control or ADBM in ADB. If an external database was used for both services, you should proceed to the next step without performing metadata migration. If an external database was not used for any of the services — perform step 1 only for that service.

  • Since ADB ES provides one common service for storing PostgreSQL data in ADB Control and ADBM (namely Database), if different PostgreSQL servers were previously used to store external databases for these services, it is necessary to select one PostgreSQL server, transfer both external databases to it, and then specify its settings in the Database service. Also, both databases should have the same owner.

  1. Depending on which PostgreSQL target databases you plan to use in ADB ES, perform the following:

  2. Migrate metadata from previously used internal databases to new ones (external or delivered in ADB ES). For this, you can use the pg_dump utility on the host where target databases are deployed. Example:

    $ sudo -u postgres pg_dump -h <host> -p <port> --format=custom --no-owner --no-privileges -U <user> -f /tmp/<dump>.dump <database_name>

    where:

    • -h <host> — IP address of the host where the internal database is deployed (from which you perform metadata migration).

    • -p <port> — port number for connection to the internal database (by default, 5433).

    • --format=custom — specifies creation of a compressed dump suitable for the pg_restore utility.

    • --no-owner — excludes owner information from a dump (owners will be set during restore).

    • --no-privileges — excludes privilege information from a dump (can be restored separately if needed).

    • -U <user> — username to connect to the internal database (by default, postgres).

    • -f /tmp/<dump>.dump — path to a dump. Replace <dump> by the target file name.

    • <database_name> — name of the internal database (by default, adbm for ADBM and adcc for ADB Control).

      IMPORTANT
      • You need to create separate dumps for ADB Control and ADBM since their metadata is stored in different databases.

      • When running the utility, you will be asked for a password to connect to the internal database. The default password is 123 (however, in your environment it may be different — see a note below).

      • In the example, the default credentials are used. However, for ADB Control, you should replace them with those you specified when configuring this service in your ADB cluster (see ADB Control → Database parameters in the ADB documentation versions prior to 6.30.0). For ADBM, use default values listed above.

  3. Restore data from each dump you created. For this, on the host where target databases are deployed, run the pg_restore utility:

    $ sudo -u postgres pg_restore -p <target_port> -d <target_db> --clean --no-owner /tmp/<dump>.dump

    where:

    • -p <target_port> — port number for connection to the target database. In case you use the internal database that comes with ADB ES, port number is 5433.

    • -d <target_db> — name of the target database. In case you use the internal database that comes with ADB ES, database name is adcc for ADB Control and adbm for ADBM.

    • --clean — drop existing database objects before restoring.

    • --no-owner — do not restore information on database object ownership (owners will be set manually in the next step).

  4. Ensure that metadata is loaded to the target database. You can do this by comparing the number of rows in the key tables of the source and target databases:

    SELECT count(*) FROM <schema>.<table>;

    Query example for ADBM:

    SELECT count(*) FROM adbm.adbm_restore_point;
  5. Check that the correct owner is set for the adcc (in case of ADB Control) and adbm (in case of ADBM) schemas in target databases. If needed, change the owner manually. Example for adbm:

    $ sudo -u postgres psql -p <target_port> -d <target_db> -c "ALTER SCHEMA adbm OWNER TO <target_owner>;"

    where <target_owner>:

  6. For the quartz schema in ADBM, you should restore ownership for all its tables and sequences (if privileges were lost during migration).

    For tables:

    DO $$ DECLARE r record;
    BEGIN
        FOR r IN SELECT tablename FROM pg_tables WHERE schemaname = 'quartz' LOOP
            EXECUTE 'ALTER TABLE quartz.' || quote_ident(r.tablename) || ' OWNER TO <target_owner>;';
        END LOOP;
    END $$;

    For sequences:

    DO $$ DECLARE r record;
    BEGIN
        FOR r IN SELECT sequence_name FROM information_schema.sequences WHERE sequence_schema = 'quartz' LOOP
            EXECUTE 'ALTER SEQUENCE quartz.' || quote_ident(r.sequence_name) || ' OWNER TO <target_owner>;';
        END LOOP;
    END $$;

    where <target_owner> is the same as in the previous step.

Step 2. Upgrade ADB clusters

For all ADB clusters that were monitored in the previous version of ADB Control (before upgrade):

  1. Remove ADBM from ADB using the Uninstall service action. In the Erase data field, set false.

  2. Remove ADB Control from ADB using the Uninstall service action. In the Erase data field, set false.

  3. Perform ADB cluster upgrade to 6.30.0.1.

Step 3. Rename external PostgreSQL databases

If external databases were previously created with names that do not meet the above-mentioned naming pattern, you should rename them before performing the next steps. However, renaming changes the Liquibase checksums, which can adversely affect future updates. To avoid this, after renaming databases, execute the following command in each external PostgreSQL database (ADB Control and ADBM) to reset checksums:

UPDATE DATABASECHANGELOG SET MD5SUM = NULL;

Step 4. Attach external PostgreSQL databases to ADB ES

IMPORTANT

Perform step 4 only if you plan to use external PostgreSQL (not included in ADB ES).

  1. Add one Database service to ADB ES and map its components to hosts

  2. On the Database service configuration page, select External postgres parameters in the Database type field and fill in all parameters in the list that opens. The assignment of parameters is described in the Configuration parameters → Database → External postgres parameters section.

    Parameters for connecting to the external PostgreSQL DB
    Parameters for connecting to the external PostgreSQL DB
  3. Click Save to save the Database configuration.

Step 5. Install ADB ES and agents in ADB

  1. In the ADB Control database, to where you migrated metadata (external or internal), run the following query to delete data about the existing ADB cluster (needed to prevent conflicts in the future):

    DELETE FROM adcc.system_user WHERE cluster_name = '<cluster_name>';

    where <cluster_name> is an ADB cluster name.

  2. In ADB ES, add other services (except Database) and map their components to hosts.

  3. Perform ADB ES installation using the Install cluster action.

  4. Add the ADBC agents and ADBM agents services to all ADB clusters that were monitored in the previous version of ADB Control.

  5. Import ADB ES configuration to ADB clusters.

  6. Install ADBC agents and ADBM agents in ADB clusters.

Step 6. Perform post-installation actions

  1. In case of errors with GUC management in the ADB Control web interface after upgrade — on the host with ADB Control UI, add the following record to /etc/hosts:

    <master_IP> <master_hostname>

    Since the master ADB Control agent is registered in Eureka using the ADB master hostname, there can be resolution errors when ADB Control UI tries to reach the agent to manage GUC settings.

  2. On the ADB master host, run the following command to remove all existing configuration files of the pgbackrest utility:

    $ gpssh -f arenadata_configs/arenadata_all_hosts.hosts -v -e 'rm -rf /home/gpadmin/arenadata_configs/pgbackrest/*'
  3. Using the web interface of the ADBM service (installed in ADB ES), run the Verify action to recreate configuration files.

  4. Using the web interface of the ADBM service (installed in ADB ES), run the Create configuration action to reapply the current backup configuration.

Step 7. Check results

As a successful result of the previous steps, all ADB clusters registered for monitoring in the previous version of ADB Control (before upgrade) should be displayed on the Configuration → Clusters page of the ADB Control web interface.

Found a mistake? Seleсt text and press Ctrl+Enter to report it