Manage configurations in ADBM

Daria Barysheva, Anton Vasilev

Backup configuration

Overview

In ADBM, a backup configuration means a set of options that are used to manage binary database backups. These options include paths for saving backups, log levels, timeouts, compression settings, etc. ADBM supports several configuration versions. But there can be only one active version, which is marked as current in the user interface.

Possible operations with configurations are described below. All operations are performed on the Configuration → Backup configuration tab. To open this tab, follow the steps:

  1. Open the Backup manager page via the ADB Control web interface.

  2. In the table that is located in the Clusters section of the opened page, click the ADB cluster name.

    Select the ADB cluster
    Select the ADB cluster
    Select the ADB cluster
    Select the ADB cluster
    NOTE

    Starting with ADBM 2.5.0, a unique cluster identifier is displayed next to each cluster name.

  3. Select the Configuration → Backup configuration tab.

    The Configuration → Backup configuration tab
    The Configuration → Backup configuration tab
    The Configuration → Backup configuration tab
    The Configuration → Backup configuration tab

Initially, there are no configurations. To start work with backups in ADBM, add the first configuration.

Limitations

  • ADBM does not support topology of ADB clusters that consist of a single segment host with mirroring enabled. For these clusters, an error occurs when saving the configuration. Without this restriction, there would be duplicates in the pgBackRest configuration for each stanza.

  • If the cluster topology has changed before the configuration editing (via the Expand action in ADCM), the configuration will not apply as long as there is at least one valid backup in ADBM. This is due to the fact that after applying the configuration on the cluster with the changed topology, all previous backups automatically become invalid. Therefore, in such cases, it is necessary to explicitly remove all existing backups and then edit the configuration.

Add a new configuration

IMPORTANT

  • Remember that enabling any configuration (even without configured schedules) applies archive_command to your cluster and starts the WAL generating process.

  • The repository that is used in configuration settings may be the same for multiple backup configurations. Every configuration can have a list of backups taken with that particular configuration. Thus, we can get sets of backups that are associated with different configurations but are actually stored in the same repository. ADBM marks backups for deletion by cleanup policies considering combination of the repository type, path, and endpoint (in case of S3). So, in case if you use different endpoints for the same S3 repository, the Cleanup action will treat them as different repositories and delete backups that should be kept.

  1. Open the Configuration → Backup configuration tab as described above.

  2. Fill in the parameters that are located in the General configuration, Timeout, Repository, and Compression sections. When filling the fields, pay attention to the following:

    • Mandatory parameters are highlighted in red.

    • For those parameters that specify time values, you can use the following units:

      • ms — milliseconds;

      • sec — seconds;

      • min — minutes;

      • hr — hours;

      • d — days;

      • w — weeks.

    • For those parameters that specify data amount, you can use the following units:

      • b — bytes;

      • kb — kilobytes;

      • mb — megabytes;

      • gb — gigabytes.

    • For those parameters that specify time or data amount, use the <value><unit> format (without spaces). For example, 1min, 2hr. If you need a complex value, use a space (e.g. 30min 30sec).

    • By clicking the qm dark qm light icon (which appears to the right of some fields during their filling), you can view hints: available measurement units, minimum and maximum values, and so on.

      General configuration
      Parameter Description Default value

      Number of full backups

      A maximum number of full backups. After this limit is reached, the oldest extra full backups will be removed during the next automatic or manual Cleanup action. For more details, see Cleanup

      3

      Number of differential backups

      A maximum number of differential backups. After this limit is reached, the oldest extra differential backups will be removed during the next automatic or manual Cleanup action. For more details, see Cleanup

      6

      Total number of backups

      A total number of backups allowed for the current cluster, including backups in the Stopped and Failed statuses that are not deleted by the Cleanup action policies. The total amount of backups will be checked before making a new backup (whether it hits the specified limit)

      40

      File Log level

      A level for file logging.

      Possible values:

      • off — no logs;

      • error;

      • warn;

      • info;

      • detail;

      • debug;

      • trace.

      info

      Archive command

      The command that is to be executed when WAL (Write Ahead Log) files are transferred to the archive directory. If you change the Archive command field value, then the ADB cluster is automatically restarted when the configuration is applied (to change the value of the archive_command GUC).

       
      The stanza configuration name prefix for segments that is used in the Archive command should be equal to the Stanza template value (see below): --stanza=<Stanza template>%c

      		 
      PGOPTIONS=\"-c gp_session_role=utility\" /usr/lib/gpdb/bin/pgbackrest --stanza=seg%c archive-push %p

      Archive timeout

      Sets maximum time to wait for each WAL segment (required for backup consistency) to reach the archive repository

      1min

      Stanza template

      A stanza configuration name prefix for segments. Should be equal to the prefix that is used in the Archive command value (see above): --stanza=<Stanza template>%c

      seg

      Log path

      A path where log files are to be stored

      /home/gpadmin/gpAdminLogs

      Exclude path

      Defines the paths and files that should be excluded from backups

      • pg_log/gp_era

      • backups

      • gpperfmon/data/

      • gpbackup_history.yaml
      The General configuration section
      The General configuration section
      The General configuration section
      The General configuration section
      Timeout
      Parameter Description Default value

      Database query timeout

      Sets the timeout for queries against the database including the backup start/stop functions which can take a significant amount of time. The timeout should be kept high unless you know that these functions will return quickly. The Database query timeout value should be less than the Protocol timeout value

      30min

      Protocol timeout

      Sets the timeout that the local or the remote process can wait for a new message to be received on the protocol layer. The timeout prevents processes from the indefinite waiting. The Protocol timeout value should be greater than the Database query timeout value

      30min 30sec

      I/O timeout

      A timeout that is used for connections and read/write operations. The entire read/write operation does not need to complete within this timeout but some progress should be made, even if it is only a single byte

      1min

      Job wait timeout

      A timeout for a lock capture at the cluster level before the job starts

      15min
      The Timeout section
      The Timeout section
      The Timeout section
      The Timeout section
      Storage
      Parameter Description Default value

      Storage

      One of the storages that you added on the Storages tab

       — 

      Relative path

      A path to a repository for backups relative to the Repository path of the selected storage. You can use the same storage for multiple configurations and assign each of them its own separate repository within that storage

       — 

      Bundle size

      A target size for file bundles. A bundle means grouping single files into one file before the backup launch. It is not recommended to set the option too high because backup retries will need to redo the entire bundle

      100mb

      Bundle limit

      A limit for the files that are included in bundles (see Bundle size above). The files larger than this size are stored separately. Bundled files cannot be reused when a backup is resumed, so this option, in fact, controls the files that can be resumed

      2mb

      Use block incremental backup

      A switch that indicates whether to make more granular backups by splitting files into blocks that can be backed up independently

      false
      The Storage section
      The Storage section
      The Storage section
      The Storage section
      Workspace
      Parameter Description Default value

      Temporary directory

      The working directory that is used during the Partial restore action for temporary data allocation.

      To reduce the impact on cluster performance during the table restore process, it is recommended to configure this directory on a disk different from the cluster disks. Since all operations during the table restore do not require random disk access, the HDD disk is sufficient

      /tmp/adbm/table_restore

      Port base

      The base port number that is used during the Partial restore action. The actual port number of each segment is equal to <Port base> + <dbId> where <dbId> is the gp_segment_configuration.dbid value corresponding to the segment.

      Allowed values: [1025,65035]

      12000

      Max processes per host

      The maximum number of processes that can work in parallel on one segment host during the Partial restore action.

      Allowed values: [1,100]

      1
      The Workspace section
      The Workspace section
      The Workspace section
      The Workspace section
      Compression
      Parameter Description Default value

      Compression type

      A compression type.

      Possible values:

      • none — no compression;

      • bz2;

      • gz;

      • lz4;

      • zst.

      gz

      Compression level

      A compression level. Possible values: 1 — 9

      6

      Max processes to use for compress/transfer

      The maximum number of processes to be used on every segment for compression/transferring

      1

      Buffer size

      A buffer size for I/O operations

      1mb
      The Compression section
      The Compression section
      The Compression section
      The Compression section

  3. Click Save. You can find this button both before and after the sections listed above.

    Save the configuration
    Save the configuration
    Save the configuration
    Save the configuration

  4. As a result, the Apply backup configuration action runs. This action, in turn, generates several subactions. You can see all of them on the Actions tab (for more details, see View actions in ADBM).

    Starting with ADBM 2.6.5, you can open the Actions tab and view the action details by clicking the Apply backup configuration link, which is available in the Action created field after you save the configuration.

Until all necessary actions are completed successfully, the configuration will have the draft status. . As a successful result of all actions, the configuration acquires the current status. Along with status, the following information about the configuration is available:

  • The version number (starting with 1).

  • The user who created a configuration version.

  • The creation date in the DD/MM/YYYY format.

    The successful result
    The successful result
    The successful result
    The successful result

Edit a configuration

After you created the configuration, you cannot edit it. But it is possible to add a new configuration version based on one of the previously created. To do this, perform the following steps:

  1. Open the Configuration → Backup configuration tab as described above.

  2. In the list of available configuration versions, click the version on base of which you want to create a new one. All parameters of the selected version are displayed under the list with available versions (in the read-only mode).

    NOTE

    You can create a configuration based on any previous version, not just the latest.
    Select a version
    Select a version
    Select a version
    Select a version

  3. Click Create new.

  4. Apply necessary changes to the parameters that are located in the General configuration, Timeout, Repository, and Compression sections. The parameters are the same as described above.

  5. Click Save to add a new configuration version. You can find this button both before and after the sections with parameters.

    Save a new configuration version
    Save a new configuration version
    Save a new configuration version
    Save a new configuration version

  6. As a result, the Apply backup configuration action runs. This action, in turn, generates several subactions. You can see all of them on the Actions tab (for more details, see View actions in ADBM).

  7. After all necessary actions are completed successfully, the new configuration version becomes available on the Configuration tab and has the current status. The previous configuration acquires the previous status.

    NOTE

    • ADBM uses the latest configuration version you added.

    • Inactive configuration versions that have no related backups will be removed during the next automatic or manual Cleanup action.
    The new configuration version
    The new configuration version
    The new configuration version
    The new configuration version

Draft configurations

Starting with the version 1.6.3, ADBM supports an initial status draft for backup configurations. This status is assigned to all configurations when attempting to save them until all actions are successfully applied and the status is changed to current. However, if a new configuration cannot be validated at the moment of its creation, it is saved as a draft. Thus, you can avoid re-entering all information during the next applying attempt. Configuration drafts can be modified (and re-applied) or removed by clicking Edit and Delete, respectively.

Starting with ADBM 1.7.3, the verified status is also available for backup configurations. This status means that validation of the backup configuration is successfully completed, but some errors occurred during its application. Configurations in the verified status can be edited or removed like drafts.

Configuration in the draft status
Configuration in the draft status
Configuration in the draft status
Configuration in the draft status

Schedule

The Configuration → Schedule tab allows you to select a schedule for various operations related to backups, such as creating various types of backups and restore points, data cleanups, etc. The tab is available starting with ADBM 2.5.0, before that schedules were part of a backup configuration and can be filled in along with other settings in the Configuration → General configuration section.

The Configuration → Schedule tab
The Configuration → Schedule tab
The Configuration → Schedule tab
The Configuration → Schedule tab

Below is the description of schedules that can be filled in on the Configuration → Schedule tab.

The "Configuration → Schedule" tab fields
Parameter Description

Full Backup schedule

A schedule for automatic creation of full backups. If you are going to create backups only manually, do not activate the switcher that is located next to the field

Differential backup schedule

A schedule for automatic creation of differential backups. If you are going to create backups only manually, do not activate the switcher that is located next to the field

Incremental backup schedule

A schedule for automatic creation of incremental backups. If you are going to create backups only manually, do not activate the switcher that is located next to the field

Restore point creation schedule

A schedule for automatic creation of restore points. If you are going to create restore points only manually, do not activate the switcher that is located next to the field

Cleanup schedule

A schedule for deleting extra backups (see Number of full backups and Number of differential backups above) and backups with the failed and invalid statuses. If you are going to run the cleanup action only manually, do not activate the switcher that is located next to the field

To fill in any of the schedules mentioned above, activate the corresponding switch and enter the cron expression in the field that becomes available.

To switch to the alternative schedule fill mode, click the cron dark cron light icon. As a result, the Cron expression generator window opens where you can set a schedule by selecting seconds, minutes, hours, days, months, and years. The selected values are automatically transformed into the cron expression that will be used as the field value. You can see it in the top part of the window along with its description. To save the current expression and return to the Configuration → Schedule main form, click Save.

Cron expression generator
Cron expression generator
Cron expression generator
Cron expression generator

To save changes made on the Configuration → Schedule tab, click Save. To undo changes, click Revert.

Storages

In ADBM, a backup storage is a location where of your data backup is stored. Starting with ADBM 2.6.5, it’s a separate entity that you can configure, edit, and share among different backup configurations. To use a storage for backup, you can select it in the Storage section of your backup configuration.

Add a new storage

  1. Open the Configuration → Storages tab and click Add storage.

    The Configuration → Storages tab
    The Configuration → Storages tab
    The Configuration → Storages tab
    The Configuration → Storages tab

  2. Fill in the parameters that are located in the Alert settings for storage and General sections described below. For S3 storages, you also need to fill in the parameters of the s3 configuration section.

  3. Click Save.

Alert settings for storage
Parameter Description Default value

Almost Full

The storage usage threshold (in percent) at which the storage configuration will have a yellow indicator and the Almost Full status

80

Critical

The storage usage threshold (in percent) at which the storage configuration will have a red indicator and the Critical status

90
General
Parameter Description Default value

Storage name

A unique name for the storage by which you can identify it when selecting a storage in a backup configuration

 — 

Repository type

A repository type.

Possible values:

  • posix

  • s3

  • cifs

posix

Repository path

The repository where backups and archived WAL segments are to be stored. The repository directory should be pre-created and available from Master and Segment hosts. You can use the NFS shared directory, which is connected to all cluster segments. Do not use local directories on segments if mirroring is switched on — in that case, the hosts with mirror segments won’t have the required stanzas.

Ensure that you set up the same repository path for complementary primary and mirror segments

 — 

S3 bucket size

The amount of storage space that you want to allocate for backups in the bucket. Displayed if s3 is selected in the Repository type field of the General section.

The parameter is required to calculate the S3 storage utilization metrics and status

 — 
The General section
The General section
The General section
The General section
s3 configuration

 
The following parameters are available if s3 is selected in the Repository type field.

Parameter Description Default value

URI Type

An endpoint type (see Endpoint below). Displayed if s3 is selected in the Repository type field. Possible values:

  • host

  • path

host

Bucket

A bucket name in the S3 storage. A bucket name cannot contain dots

 — 

Endpoint

An endpoint of the S3 storage

 — 

Key

A key to access the S3 storage

 — 

Key secret

A secret key to access the S3 storage

 — 

Region

A region of the S3 storage

 — 

Upload chunk size

Size of the uploaded data chunk (part size in S3 terms). Limitations shown in the tooltip (64kb-1tb) come from pgbackrest. Note that S3 storages may have their own restrictions

5mb

Server-side encryption options

Indicates whether to perform data encryption on the server side. After this switch is activated, the Type and Key fields become visible and mandatory (see below)

false

Type

A type of the server-side data encryption:

  • kms-key-id — using the specified AWS key management service key.

  • sse-customer-key — using the specified customer key.

This field is mandatory if the Server-side encryption options switch is activated. You should also specify a key value in the Key field (see below)

 — 

Key

A key that is used for data encryption on the server side. The key type is specified in the Type field (see above).

This field is mandatory if the Server-side encryption options switch is activated. The field becomes enabled after you select a value from the Type drop-down list

 — 

Repository storage CA file

Certificate Authority (CA) PEM file to access the S3 storage. To select the file, click Upload

 — 

Certificate verification

Defines whether to verify a repository storage certificate. If you activate this switch, the Repository storage CA file becomes enabled

true
s3 configuration section
The s3 configuration section
s3 configuration section
The s3 configuration section

Monitor storages

To monitor the storage status and space utilization, use the Overview section, which is available after applying the configuration with this storage. The Overview section allows you to monitor the following parameters:

  • Availability. Shows whether the storage is available. If ADBM cannot access the storage, it won’t perform a backup.

  • Space utilization: Total capacity, Used space, Free space, and Fill percentage.

    The fill percentage shows the ratio between the total capacity and used space. For POSIX and CIFS storages, the total capacity is calculated automatically. For S3 storages, you should manually specify the total capacity using the S3 bucket size field. If no value is set, the S3 storage utilization metrics and status won’t be available.

  • Current status. Depending on the fill percentage value, each storage may have one of the following statuses:

    • Sufficient space — the fill percentage value is below the percentage configured under Alert settings for storage, in the Almost Full limit.

    • Almost Full — the fill percentage value is above the Almost Full limit, but below the Critical limit.

    • Critical — the fill percentage value is above the Critical limit. To avoid storage overflow during a backup, you can use the Fail on low space flag, which ensures the backup won’t proceed if the storage is in the Critical state.

These metrics are first calculated when you apply the backup configuration that uses this storage. Afterward, the metrics and the status are checked and updated every minute.

Storage status
Storage status
Storage status
Storage status
Found a mistake? Seleсt text and press Ctrl+Enter to report it