Rolling upgrade
The Rolling upgrade functionality allows you to sequentially upgrade the ADQMDB service on separate ADQM hosts without a long shutdown of a cluster.
IMPORTANT
|
The process of ADQM rolling upgrade via the ADCM interface is described below.
Step 1. Prepare for the update
-
On the Bundles page, upload a new ADQM bundle — see the step Upload a cluster bundle to ADCM in the Create a cluster article.
-
Open the Clusters page, where the icon becomes active for an ADQM cluster in the Actions column after a new bundle is loaded — click this icon.
Preparing a cluster for the upgradeIn the Upgrade Cluster window that opens, select a version you want to update ADQM to and click Upgrade.
Selecting a version to update ADQMIn the confirmation window that appears, click Run.
Confirming preparation for the updateWhen the action is completed, ADCM will change the cluster status to
upgrading
— the cluster is ready to be upgraded.
Step 2. Update ADQMDB on host groups
-
On the ADQMDB service configuration page (Clusters → <ADQM_cluster> → Services → ADQMDB), open the Action hosts groups tab and combine cluster hosts into groups to then upgrade the ADQMDB version for each host group sequentially. For example, you can create host groups so that each group contains one replica of each shard. The image below shows an example of dividing hosts into groups according to this principle for the following cluster topology:
-
shard 1 with two replicas — dev-adqm-01.ru-central1.internal and dev-adqm-02.ru-central1.internal;
-
shard 2 with two replicas — dev-adqm-03.ru-central1.internal and dev-adqm-04.ru-central1.internal.
Distributing hosts into groups for sequential ADQMDB service updates -
-
Perform the ADQMDB service upgrade for one host group. To do this, in the host group row, click the icon in the Actions column and run the Upgrade action.
Initiating ADQMDB update for a group of hostsIn the window that opens, click Run to confirm the upgrade. After the ADQMDB service upgrade for a host group is complete, the cluster will remain in the
upgrading
state.You can re-run the Upgrade action for a group of hosts if, for example, the upgrade failed on some hosts or new hosts were added to the group.
-
After the Upgrade action has been successfully completed, make sure that the new version of ClickHouse is installed on all hosts in the group (for example, you can run the
select version()
command in the clickhouse-client command-line client) and that databases work correctly (ClickHouse guarantees technical compatibility of two versions within a year, but business behavior may change, so it is recommended to check the applied result for each version separately). Further actions depend on the check result:-
If the check is successful, you can continue upgrading ADQMDB on other hosts of the cluster (run the Upgrade action for the next group of hosts) or proceed to completing the cluster upgrade.
-
If you have detected any problems while testing the new version of ClickHouse, you can abort the cluster upgrade and roll back the previous version of the ADQMDB service on the previously upgraded hosts. To do this, switch to the Clusters page and run the Abort the upgrade action for the cluster.
Canceling the ADQMDB service updateAfter the update is canceled, the cluster bundle will revert to the previous version (and ClickHouse on all cluster hosts accordingly too), and the cluster state will be set to
installed
.If you failed to cancel the upgrade (for example, because a file with information needed to roll back to the previous version is corrupted or lost, or the archive with the configuration files of the previous version of the ADQMDB service is missing), you can run the Abort the upgrade action with the Forced rollback option enabled. This will perform a forced rollback to the previous version of the ADQM bundle, after which you will need to run the Reconfig & restart action for the ADQMDB service.
Starting with ADQM 24.3.11.7.1.b1, the Abort the upgrade action with the Forced rollback flag set restores ADQM repositories and removes ADQM packages if they have been updated.
Canceling the update with the Forced rollback optionAfter you fill in the action parameters on the Configuration tab, click Next and confirm the action on the Confirmation tab by clicking Run.
Confirm canceling the update
-
Step 3. Complete the update
Once you have successfully upgraded the ADQMDB service to the new version on hosts where a preliminary upgrade was required, complete the cluster upgrade with the Complete the upgrade action (on the Clusters page) — it will upgrade ADQMDB on the remaining hosts, as well as all other ADQM services in the cluster. Once this action is performed, there is no option to revert to the previous version of ADQM.
When the upgrade process is complete, ADCM will change the cluster state to installed
.
NOTE
The Complete the upgrade action runs the usual upgrade of the cluster, i.e. the upgrade of all ADQM services on all hosts (as the Upgrade action does for a cluster being upgraded to an ADQM version up to and including 24.3.2.23.1.b1). |