ADBM releases

Fixed a bug where deleting a backup (as a result of manual or automatic Cleanup) could delete all backups and the WAL archive of that backup configuration. The bug occurred if the configuration was changed so that it pointed to the same repository, but the path in Relative path differed only by slashes — for example, c1 and /c1. In this case, the system treated them as different repositories. After the fix, such paths are recognized as the same repository

Fixed a bug that occurred when restoring a table whose name requires quoting (for example, due to mixed-case letters, special characters, or spaces). A different table could be restored, whose name matched the name of the intended table in the lower case

Fixed a bug that occurred when resuming a Streaming data flow in a cluster group: if one of the segments lost access to the storage while the others recovered successfully, the Resume action failed with an error even after access was restored. Now resuming does not require manual intervention, and all segments are correctly restored to the target restore point

Fixed a bug that made it impossible to resume a backup that started according to a schedule and was then stopped by the Terminate action

Added the ability to compare files by size and modification time during scheduled backups and restore instead of calculating checksums and performing a full overwrite, which reduces resource consumption. For this, the Use checksums flag is added to the backup configuration, and the Override files drop-down list is added to the Common restore action and to the data flow creation form

Added the ability to configure backup cleanup rules. To support this, the Target only flag is added to the Cleanup action; when it’s selected, only backups that depend on the target restore point are deleted without taking retention policies into account

During partial restore, tables to be restored can now be specified not only by mask but also as a list

The logic for displaying cluster group actions is changed: now only actions from the period when the cluster actually belonged to this group are displayed. Actions performed when the cluster belonged to another group are no longer displayed

Fixed incorrect display of data restore progress when executing the Stream Copy action

Fixed vulnerabilities in third-party components of ADBM

Fixed the list of actions available after upgrading an ADB cluster to version 6.31.0 and above

Added a wrapper around the pgBackRest archive-push command to verify WAL segment existence before archiving

Introduced a new Deleted status for backups

Added progress tracking by segment for the Restore step in Streaming data flows

Added the Check free space toggle to the storage settings to disable free space checks for S3 storage

Added configurable checksum calculation and algorithm settings for S3 storage configuration

Introduced cluster groups

Added an option to restart streaming (see the Restart action in the On restore points streaming failure description in Disaster recovery)

Improved stream resume behavior

Allowed restore point creation during backup operations

Added detection of segment timeline changes during backup to prevent invalid restore

Fixed restore of mirrors during Stream Copy after a previous stream failure

Fixed an issue where table restore would fail on tables with a high number of partitions

ADBM now supports ADB 7

pgbackrest updated to version 2.54

Fixed an issue where the status was changed to failed when an error occurred

A backup that failed after the Resume action now shows 100% progress

Added timeline ID comparison before and after the backup

Fixed a bug where the Cleanup action would delete valid backups

Added audit of operations with clusters and storage configurations

Added the Storages tab and modified backup configurations accordingly

Added representation of storage space utilization and availability

Implemented compatibility verification for database and agent versions

Added processing of a new XLogRecord type during the Partial restore action

Fixed: WAL archives for backup were not verified during the Verify action

Fixed: agents were not displayed on the Backup manager page when ADB was stopped

Fixed: backup progress was sometimes incorrectly updated

Changed the name of the stream entity to data flow and modified naming in UI accordingly

Changed errors in jobs to more informative ones

Storage access was checked only from the master, which could cause incorrect backup status acquisition

After launching a stream of the Copy type with the parallel mirror restore option, the pg_hba.conf file was missing on target segments if the source segments had no mirrors

Added a check of the corresponding WAL chain during the Verify action

Added a check of the PostgreSQL backend availability during the table restore

Fixed filtering WAL for headers that were divided across two pages

Fixed handling backup blocks during filtering WAL in pgbackrest to avoid gpdb assertion triggering during the Partial restore action

In pgbackrest, fixed handling a long record at the end of a previous WAL file during the Partial restore action

Fixed errors that occurred during the Partial restore task execution in temporary PostgreSQL instances when running the pgbackrest archive-get command

Fixed the search for neighboring WAL files in other directories during filtering WAL

In filtering WAL, replaced the error of missing the next WAL file with a warning

Fixed the overwrite contrecord (continuation record) handling at the beginning of the next file when filtering the WAL

Fixed the record is too big error when filtering WAL

Fixed filtering WAL when a checksum is in the next segment file

Fixed filtering WAL when there is no unfinished record at the end of the segment file

Fixed handling WAL records larger than a WAL segment

Fixed: a segment did not select timeline of a restore point during Partial restore after switching to a mirror

Fixed reading an unfinished record at the end of the file when the next file is partial

Added display of the Common restore action progress

Removed the Skip validation option from the data restore form if a cluster is down. Now it is detected automatically and works for data restore, copy and streaming, giving the ability to recover a stopped cluster. ADBM agents should be alive on the stopped cluster hosts for this option to work

Reworked the Verify action: backup files are now verified along with metadata. Added the corresponding option Verify to the Common restore, Partial restore, stream launch actions

Changed the archive_mode setting value to on to avoid the Duplicates found in archive for WAL segment error that occurred when primary and mirror segments tried to archive the same file. Note: ADB cluster will be restarted after ADBM upgrade because of the archive_mode change

Implemented storing the ic_proxy_port_delta parameter on a cluster level for proper handling of the ic_proxy configuration during data restore

Created the new page Backup manager → Configuration → Schedule to manage scheduling for a cluster. Also, reworked a schema of scheduling data

Reworked a schema of cluster-related data and changed cluster addressing from a name to GUID

In the ADBM agent, improved processing of the mirrors absence

Fixed handling of auto terminate actions for timed out backups

Fixed the A conflicting conditional operation is currently in progress against this resource error that occurred when checking S3 writability

Estimated progress is now shown for backups

Added server-side encryption options for S3: kms-key-id and sse-customer-key (see Server-side encryption options in Manage configurations in ADBM → Add a new configuration → Repository)

Added a choice of the restore point streaming error handling mode on stream creation (see On restore points streaming failure in Disaster recovery). Added the Paused status for streams and Resume/Cancel options for this status

Added storing of the gp_interconnect_type value in the cluster topology to support force restore on clusters with configured ic_proxy

Changed the Apply backup configuration action logic to restart a cluster only when the archive_mode value is different from always and has to be set

Added caching of the Recover segments action on the agent side for high availability (HA) purposes

Adjusted the await-segment-ready parameters in the ADBM agent properties to optimize recovery time and avoid streaming failures by timeout

Data restore with the Force and Skip validation flags set could fail if there was switch to standby

In ADBM CLI, some parameters were not returned in the command output and parameter compatibility was not checked

Removed the etcd component from the ADBM architecture and assigned its functions to PostgreSQL

To avoid illogical assignment of the Invalid status to backups, modified the Verify storage action to check repository accessibility from all agents instead of the master host only

A list of cluster actions now contains only currently allowed actions to eliminate errors that occurred when the Verify action was not applied after cluster upgrade

For the backup report command of ADBM CLI, added the ability to select the output format in the --output option

Added actualization of agent roles before data restore with the Force option — to support force restore on a target cluster with a lost dispatcher role

Added filters, sorting, and pagination to the table on the restore details page

Fixed handling of gp_interconnect_proxy_addresses during data restore in an ADB cluster

Fixed: the Partial restore action launched on a source cluster could terminate streaming

Fixed: a filtering configuration for Partial restore was not used

Fixed: a target cluster was shown in the ADB Control UI with the Up status while streaming

Introduced an option of partial recovery from binary backups

Added parallel Cleanup execution on agents to avoid termination of this action by timeout on large installations

Updated the backup configuration version display

In the Subactions section on the page with action details, added line breaks to the display of long action names

In the process of cluster reinstallation, added restart after the ADBM client installation to set the PGBACKREST_CONFIG environment variable

The isQuiet, isVerbose, backupDir options did not work for the backup and backup config commands in ADBM CLI

Some elements were not aligned in the ADBM action details

ADBM restarted ADB upon backup configuration changes even when archive_command was not changed

The shortened version of the database option for commands did not work in ADBM CLI

Backups in the Invalid status were not deleted during the Cleanup action

The archiver process failed after the host recovery

Implemented new parameters for S3 in backup configurations: Upload chunk size and Certificate verification (see the Repository section in Manage configurations in ADBM)

Added integration with Vault

The Resume action functionality is now available only for full backups

Restricted the number of backups in the Stopped status to 1 and added a history of backup status changes

Added a limit for a total number of backups

Added validation for a compression level in a backup configuration

Fixed the Cleanup action for backups in the Unknown status

Fixed: backups were not transferred to the Invalid status

Fixed: gprecoverseg failed on an agent during the Restore action due to running data backup or restore in ADB

Fixed: no information on a backup after changing its status from Unknown to Done

Implemented CLI for logical backups

Enabled the feature of block-level incremental backups: the corresponding flag is added to the backup configuration page

Added support for expanding an ADB cluster for ADBM

Added the Verify backup action to a stream flow

Removed the ability to delete a configuration for an active stream

Fixed: the restore process failed for a restore point created after one of the segments went down

Question marks were displayed in the Exclude path field if its value was added in Russian

Fixed the incorrect Warning status of the Terminate action applied to a stream after several clicks on the Run button in the Terminate stream modal window

Fixed: the only cluster backup configuration was deleted after several streams and cleanups were launched

A backup with the Applied configuration failed if the Draft configuration existed

Backups failed after upgrading an ADB cluster from 6.25.1 to 6.26.2

ADB master stopped during the restart of adbm-agent

Added a new option for Disaster Recovery: standby cluster can now be restored from WAL changes of a source cluster. Restore from a source backup is required only on the initial step, not on every iteration

Improved handling of an S3 repository in ADBM actions considering essential configuration parameters

Cleanup logic was updated to delete all incremental and differential backups that might be left by policy

Added the liquibase.releaseLocks parameter for ADBM migration service

Fixed: copy restore failed after stopping one of the source cluster segments

Fixed a stream status type on applying impossible streaming configuration

Fixed: the Unknown status for backups was absent

Added new statuses for management of backup configurations

Implemented changes in the Cleanup action to support modifications of a repository type or path, and to protect backups during Disaster Recovery (DR) copy stream

Added handling of repository unavailability to the process of backup status determination

The Backup action flow was being stopped when subactions were in the Warning status

Some details of the Restore action were not visible in the light theme

Implemented the ability to restore mirrors from binary backups in parallel with primary segments — via the Restore mirrors field. Note that it is an experimental feature

Verifying the repository accessibility was added for a copy stream command flow

Added the Warning status for ADBM actions

Date filter defaults were changed to the last day on the Backups and Restores pages

Added the forced Skip validation flag display to the Restore modal window for restoring data in stopped ADB clusters

Fixed the error of running incremental backups connected to zero-sized files

Fixed the verification error that occurred when an S3 repository with self-signed certificates was used

Fixed the error of creating ADBM configuration for S3 storages with the path URI type

Fixed downgrade in the ADBM UI performance when one of clusters was unavailable

Fixed the error of restoring from a backup after a successful copy stream on a DR cluster

CIFS repository support was added

Added the option of automated backup copying to a target cluster when a new backup on a source cluster is completed

Action id was added into action details

Auto detection of an active master host was added to support a backup process after switching to standby

It was impossible to restore backups in clusters with non-unique names within one ADBM

Self-restore (via the Restore action) failed after changing roles on mirrors

S3 certificate was deleted from segments after the host reboot

Added the restore with mirrors option for Disaster Recovery (DR) copy process

The repository validation step was added before saving a backup configuration

Backup topology data is now sorted by Segment

Restores list was empty for clusters with spaces in names

DR copy action failed when source and target clusters had different standby configurations

Restore for standby master was implemented

Automatic checking of ADB versions compatibility on source and target clusters was added to the Disaster Recovery (DR) copy process

Availability check for a backup configuration on a target cluster was added to the DR copy process

Copy stream failed on a cluster with standby

Could not restore a backup on DR cluster with a number of datanodes different from a source cluster

Choosing particular databases from the database list caused the Disaster Recovery (DR) copy process to fail

DR cluster could not be started after restore

Implemented Disaster Recovery with Cold Standby — ability to recover a cluster from a backup located on another cluster. Beta version! Please pay attention to the ADBM known issues section and backup a target cluster before starting a stream

Added pgbackrest options (-delta, --force, --max-process) for the restore action

Added an option to save a backup configuration without schedule for running backups on request

ADBM action tree view was enhanced

Enabled resuming failed backups considering segments that have already been backed up

Fixed: the Apply backup config action failed when S3 was selected

Added support for PowerPC-based computers

Added the ability to restore data from backups without mirror segments

Added the ability to filter restore points for the Restore retry mechanism

Added the Restores tab to view Restore actions

Creating an additional configuration when reinstalling without creating a backup

The error with using an S3 repository to store backups

Added the openapi-yaml generator to generate OpenAPI files

Added a timeout to read an OpenAPI file

The list of available cluster actions is supported now

Added the ability to delete the last backup from the stanza. It expands the cleaning functionality logic

Added the ability to restore specific databases from backups

Added the ability to use the Force flag when a cluster is being restored from a backup. It means bypassing all checks

Added the validation for the cron expressions that are used when ADBM configuration is being created or edited

Added a check for archive_command when saving the ADBM configuration. It allows avoiding unnecessary restart of the ADB cluster

Implemented the first ADBM version. For more details, see Arenadata DB Backup Manager overview