Rolling upgrade

The Rolling upgrade functionality allows you to sequentially upgrade the ADQMDB service on separate ADQM hosts without a long shutdown of a cluster.

IMPORTANT
  • Rolling upgrade is available for upgrading an ADQM version from 23.3.2.37.1.b1 (minimum) to 24.3.2.23.2.b1 (and higher) in ADCM starting with version 2.3.

  • Rolling upgrade can only be applied to ADQM clusters in the installed state.

The process of ADQM rolling upgrade via the ADCM interface is described below.

Step 1. Prepare for the update

  1. On the Bundles page, upload a new ADQM bundle — see the step Upload a cluster bundle to ADCM in the Create a cluster article.

  2. Open the Clusters page, where the icon upgrade default becomes active for an ADQM cluster in the Actions column after a new bundle is loaded — click this icon.

    Preparing the cluster for upgrade
    Preparing a cluster for the upgrade

    In the Upgrade Cluster window that opens, select a version you want to update ADQM to and click Upgrade.

    Selecting a version to update ADQM
    Selecting a version to update ADQM

    In the confirmation window that appears, click Run.

    Confirming preparation for the update
    Confirming preparation for the update

    When the action is completed, ADCM will change the cluster status to upgrading — the cluster is ready to be upgraded.

Step 2. Update ADQMDB on host groups

  1. On the ADQMDB service configuration page (Clusters → <ADQM_cluster> → Services → ADQMDB), open the Action hosts groups tab and combine cluster hosts into groups to then upgrade the ADQMDB version for each host group sequentially. For example, you can create host groups so that each group contains one replica of each shard. The image below shows an example of dividing hosts into groups according to this principle for the following cluster topology:

    • shard 1 with two replicas — dev-adqm-01.ru-central1.internal and dev-adqm-02.ru-central1.internal;

    • shard 2 with two replicas — dev-adqm-03.ru-central1.internal and dev-adqm-04.ru-central1.internal.

    Distributing hosts into groups for sequential ADQMDB service updates
    Distributing hosts into groups for sequential ADQMDB service updates
  2. Perform the ADQMDB service upgrade for one host group. To do this, in the host group row, click the icon actions default dark actions default light in the Actions column and run the Upgrade action.

    Initiating ADQMDB update for a group of hosts
    Initiating ADQMDB update for a group of hosts

    In the window that opens, click Run to confirm the upgrade. After the ADQMDB service upgrade for a host group is complete, the cluster will remain in the upgrading state.

    You can re-run the Upgrade action for a group of hosts if, for example, the upgrade failed on some hosts or new hosts were added to the group.

  3. After the Upgrade action has been successfully completed, make sure that the new version of ClickHouse is installed on all hosts in the group (for example, you can run the select version() command in the clickhouse-client command-line client) and that databases work correctly (ClickHouse guarantees technical compatibility of two versions within a year, but business behavior may change, so it is recommended to check the applied result for each version separately). Further actions depend on the check result:

    • If the check is successful, you can continue upgrading ADQMDB on other hosts of the cluster (run the Upgrade action for the next group of hosts) or proceed to completing the cluster upgrade.

    • If you have detected any problems while testing the new version of ClickHouse, you can abort the cluster upgrade and roll back the previous version of the ADQMDB service on the previously upgraded hosts. To do this, switch to the Clusters page and run the Abort the upgrade action for the cluster.

      Canceling the ADQMDB service update
      Canceling the ADQMDB service update

      After the update is canceled, the cluster bundle will revert to the previous version (and ClickHouse on all cluster hosts accordingly too), and the cluster state will be set to installed.

      If you failed to cancel the upgrade (for example, because a file with information needed to roll back to the previous version is corrupted or lost, or the archive with the configuration files of the previous version of the ADQMDB service is missing), you can run the Abort the upgrade action with the Forced rollback option enabled. This will perform a forced rollback to the previous version of the ADQM bundle, after which you will need to run the Reconfig & restart action for the ADQMDB service.

      Starting with ADQM 24.3.11.7.1.b1, the Abort the upgrade action with the Forced rollback flag set restores ADQM repositories and removes ADQM packages if they have been updated.

      Canceling the update with the Forced rollback option
      Canceling the update with the Forced rollback option

      After you fill in the action parameters on the Configuration tab, click Next and confirm the action on the Confirmation tab by clicking Run.

      Confirm canceling the update
      Confirm canceling the update

Step 3. Complete the update

Once you have successfully upgraded the ADQMDB service to the new version on hosts where a preliminary upgrade was required, complete the cluster upgrade with the Complete the upgrade action (on the Clusters page) — it will upgrade ADQMDB on the remaining hosts, as well as all other ADQM services in the cluster. Once this action is performed, there is no option to revert to the previous version of ADQM.

Completing the ADQM cluster upgrade
Completing the ADQM cluster upgrade

When the upgrade process is complete, ADCM will change the cluster state to installed.

NOTE

The Complete the upgrade action runs the usual upgrade of the cluster, i.e. the upgrade of all ADQM services on all hosts (as the Upgrade action does for a cluster being upgraded to an ADQM version up to and including 24.3.2.23.1.b1).

Found a mistake? Seleсt text and press Ctrl+Enter to report it