ADB Control releases

Changed collection and calculation of CPU and I/O metrics to make calculation of differences by processes in buckets more precise

Added SSL support for external databases used in ADB Control and ADBM (Metrics DB, Query DB, Offload DB)

Changed relative metrics calculation. Skew for all metrics is now calculated based on data from all segments, not only the ones that are active during the command execution. CPU average usage percent summary calculation is changed to consider logical segment mapping on hosts

Optimized a time frame for segment system metrics refreshment in command details

Eliminated the direct GpRole usage on the ADB Control side for the Greenplum 7 support

Added validation for fields with domain names or IP addresses in the ADB Control/ADBM service configuration

Added a custom temp directory for the adcc_scheduler Docker service to eliminate errors caused by the libzstd inavailability

Fixed: a modal window opened instead of the main login page

Added support of LDAPS for ADB Control users

Added the ability to configure a dedicated subnet for ADB Control and ADBM (see ADB Control → Use dedicated network in Configuration parameters)

Changed display of erroneous slices in the Plan & progress section on the page with command details

Reworked processing a command counter in the ADB Control extension

In the adcc extension, added optimizations to the process of the query_band usage that help to avoid errors in SELECT FROM get_stat_activity()

Upgraded Spring Boot to 3.2.3

Extended the sql_id type for compatibility with ADB 7

Changed CPU usage calculation to average instead of sum on the Dashboard page

Updated names of some elements on the Dashboard page

Added the custom Login mount point parameter for integration with the Vault storage (see Vault integration → Login mount point in Configure a cluster)

Unified API conventions for the ADB Control and ADBM HTTP REST services, including datetime ranges

The session registered under one user moved to another user if cookies were not cleared

After transaction errors, plan nodes were still in the Executing status

For some commands with errors, an error text was not displayed in the command details

The metrics offload job failed after ADB upgrade to 6.27.1.56

The adcc extension queries could delay cluster stop

Added the ability to connect to an external Hashicorp Vault instance and store secrets in it

Added display of blocked sessions and commands waiting to be queued within resource groups

Improved the layout of charts on the Dashboard page

Added pre-checks for cluster actions to ensure ADB Control has enough privileges in the external database

Extended filtering and sorting by segment system metrics on the command details page

Removed the outdated authentication mechanism for system users

The Information page did not display the cluster info

Fixed escaping of special characters for values entered in filters with partial search

Fixed the executing status on command plan nodes after transaction cancellation

Fixed: a failed attempt to log in to ADB as LDAP user was registered in ADB Control as the ADB authorization type instead of ADB LDAP

Fixed occasional fails of applying a metrics offload configuration

Added support for clustered ClickHouse as an external database for ADB Control installation

Added the new role — ADBM Administrator

Reworked the Configuration → Clusters tab

The planchecker service now has a configuration file

The CPU average usage metric is now averaged across processes instead of cluster segments

A connection URL is now changed in ADB Control after Standby is activated in ADB

Added re-creation of the ClickHouse container volume on the ADB Control service upgrade

Improved the container security by enabling ReadonlyRootfs and the no-new-privileges options

Removed Nginx from ADB Control and made the ADB Control UI accessible with SSH tunneling in isolated environments

Added masking passwords to the migration service logs

Inserts to audit_auth are now splitted into smaller batches to avoid shutdown of Postgres in ADB Control

Moved the cluster spill metrics to a separate table to avoid deadlocks in the ADB Control Postgres instance

Added a relation schema name to the plan node details

Fixed absence of elapsed time on plan slices for queries in progress

The CPU Total metric had a wrong calculation basis

The Spill Skew value sometimes shown more than 100%

The session table blinked on update in the UI

Pagination did not work on the Resource group → Hosts tab

Fixed the migration error that occurred on configuring ADB Control with external Postgres

Added the DB credentials setup option for ADB Control and ADBM service configurations

Introduced the query tagging feature in ADB. Tags are now visible in ADB Control

Added ability to set GUCs that manage ADB Control behavior

Refactored the detailed plan progress view for convenience

Added session management of ADB Control users. Session interruption is now available for users with the Admin and Owner roles. Additionally, owners can set the maximum duration for user sessions (see the User session timeout option)

Improved Docker security with container user restrictions

Reworked external database interaction with ADB Control from security perspective: sensitive settings are now made by a customer

Added the LDAP group list actualization button

Updated PlanChecker components: deleted JQuery, bumped Bootstrap to 5.3.3 for security purposes

Added the Session field to the Audit/Operations pages

Fixed the display of Total CPU usage percent metric on the Dashboard page: decimal value was displayed instead of percentage

Fixed the absence of highlighting for selected filters

Removed the ability to set incorrect values in the calendar

Fixed the rights setting for keystore.jks, which caused the Permission denied error on agents

Fixed: the CPU Avg metric for some queries was marked as Infinity or NaN

Fixed: information for some segments was absent in command details in case of a large number of cluster segments

Introduced an option to configure SSL/TLS for interaction of ADB Control/ADBM services

Added the Application name parameter for each session on the Monitoring → Sessions tab

Implemented dynamic loading of an LDAP group list based on the user input on the Users → LDAP group mapping tab

Moved charts to a separate Dashboard page

Implemented changes in API and a system user management process to improve security of interaction between ADB Control and ADCM, and the process of registering external ADB clusters

Modified TTL (time-to-live) for ClickHouse tables to provide lower disk space consumption

Fixed different values of the Session field displayed for Auth and Logout audit events within the same session on the Audit → Authorizations tab

Fixed the Export sessions checkbox absence on the Configuration → Job policy → Metrics offload tab

Fixed the Start time field mapping on the Monitoring → Sessions tab

Fixed aggregation of metrics for cluster Summary charts

Added the Sessions tab to the Monitoring page with the session termination option

Improved a process of filtering in agents for gathering system metrics

Added an automated cluster archivation when a new cluster with the same JDBC URL is added

Removed the Create connection button from the Configuration → Clusters page due to the existing feature of automated adding external clusters via ADCM

Support for SSL/TLS in gRPC services was added

A user IP address was not displayed in the Host column on the Audit → Operations and Audit → Autorizations pages

Fixed an absence of a record for a denied action attempt on the Audit → Operations page

The custom timezone setting is now available on the Configuration tab

New parameters for the metrics offloading job (Export job) were added

Changed the order of side menu items

Deleting the last user with the Owner role is now prohibited

The Show and Frequency settings for jobs and actions lists were not saved after switching to another page

Fixed the error of filtering by Job group on the Jobs → Audit page

ADB is now available as a target database for exporting metrics from ADB Control

Filters for Object name, Username, Host columns on the Audit → Operations and Audit → Authorizations pages are now case insensitive and partial (allow you to search by the first characters entered)

Password for an offload database was shown on the Audit → Operations page

Sorting by columns was not available on the Jobs page

User role was displayed as a digit on the Audit → Operations page

After selecting a cluster, agents for other clusters were displayed on the Information page

It was impossible to switch to the next list page in the Resource groups → Overview section if there were more than 5 segments in the list

New tables and filters were implemented

Error when changing the user role

Error when launching the metrics offload job

Execution time was not showed in plan nodes after the command finished

Details for relation audit were not showing

Audit operation for updating the resource group configuration was not registered

Audit for unsuccessful ADB logon attempts was added

Audit for ADB Control users management actions is available now

Checking that a new password differs from the current one was implemented

Added an archiving action for a cluster to stop monitoring of the selected cluster with the ability to activate it again

Added filtering of queries shorter than a threshold on the adcc-agent side to improve agents stability

SQL signature (sql_id) was added for every command

Start time/End time filters are now visible by default on Commands History, Transactions History, and Backups pages

Status names formatting in filters and table columns is unified

No message was displayed when a connection was successfully created

Toggle button for a table was enabled when there was no further information

Reset button didn’t reset sorting

Displayed NaN for Avg load rate / sec on the ADCC agent information page

Cluster information data didn’t update immediately after switching the cluster on Actions, Backups, Restores tabs

Black screen was displayed when switching to another cluster from restore details

Backup details action row was not displayed

Page path not changed when switching tabs

Performance metrics list for commands/transactions had duplicates

Arenadata DB Command Center (ADBCC) is renamed to Arenadata DB Control (ADB Control)

Added the ability to export metrics to an external database for a long-term storage

Added the CSRF (Cross-Site Request Forgery) protection support

Added a forced password change at the first user authentication in the system

Added the temporary blocking of the user account after several failed authentication attempts

Added calculation of total monitoring metrics. For active commands and transactions, cpu-total, read-total, and write-total are calculated at the current time from the start of execution

Added pagination for the Job monitoring page

Fixed checking the name of the user that is being created. Only latin characters, numbers, and special characters in the user name are available

Fixed: no pop-up window when receiving an error

Fixed: a query could become "frozen" in the unknown status after 14-16 seconds

Decreased the log level in gpadcc

Using ADBCC along with gpperfmon within the same ADB cluster is now supported

LDAP search in several OUs during ADBCC authentication is now available

Added the ability to sort by almost all columns on the Command online/Command history and Transaction online/Transaction history pages

Added the ability to show all available filters on the Command online/Command history and Transaction online/Transaction history pages

Added the ability to edit the JDBC URL and the Username cluster connection settings after the cluster creation

Added a pop-up window with information about the application of the resource group settings

On the detailed transaction page in the table of commands belonging to the transaction, renamed the Workload column to Resource group

Added the ability to specify the sorting direction on the Resource groups page

Fixed breadcrumbs that are shown when viewing resource groups. Instead of cluster and group IDs, resource group names are now displayed

Sometimes the cpu skew metric obtained negative values

The statistics calculation for an interval of more than an hour

The calculation for the Total queue duration column on the Resource groups page

Removed the ability to enter negative values for the Batch size and Expire duration settings on the Configuration page for a job policy

The pagination error on the User page

The calculation for the number of commands in the running status on the Command online page

The search for a command in the text when there is a newline character in the query text

Added a new user interface for ADBCC

Added a new module for ADB binary backups management — Arenadata DB Backup Manager (ADBM)

Added a new module responsible for performing scheduled tasks — adcc-scheduler

All datetime metrics are now stored in UTC

For clusters being removed from the ADBCC configuration, uncompleted commands and transactions are not updated

The status, finishAt, and system metrics (readTotal, writeTotal, and cpuUsageTotal) of the current active command are now updated immediately for a terminated transaction. The status of the command is set to CANCELLED

The calculation of Spill Skew on the Command monitoring and Command history pages

The calculation of the node execution time for a command in PLAN & PROGRESS

The calculation of system metrics with negative values

Added new performance metrics for commands and transactions: Cpu usage total, Read bytes total, Write bytes total

Added the ability to repeatedly change a resource group for a transaction

Removed the ability to stop execution of a separate command by canceling or terminating it

Renamed the Workload column to Resource Group on the Monitoring and History data pages for commands and transactions, as well as for detailed pages

Removed the ability to change a resource group for a command on the Monitor Command page

For commands and transactions, rows are now sorted by the Status column value

Excluded the large bold font style for system metrics on detailed pages of commands and transactions

Corrected the data sorting on the Audit page based on numbers, symbols, letters

It was impossible to enter data using the keyboard into the Start Time filter on the Monitor/Transaction pages

The error with appearance of multiple filters by cluster and by database, if a filter by database was being added from the filter list

Fixed the mapping of memory dimension from MiB to Mb

Fixed calculating the number of tuples affected by the request without updating the page

Included the Planchecker database objects into migration

Added a new column Planner

Added the ability to filter commands by a planner

Added the ability to cancel a transaction on the Detailed transaction page

Added a button to reset and delete all previously applied filters on the Monitor and History command pages

Added the ability to reassign queries to another resource group

Added a configuration for a Planchecker address

Changed the display of the Spill Files (skew), Row Skew (in the Node panel), Cpu Skew, Memory Skew, Read Skew, Write Skew metrics to the uniform format: <metric_name> Skew <metric_value>%

The ADB unavailability error in case of the OutOfMemoryError error in ADBCC

Fixed an actual tuples count for commands

Fixed the incorrect display of the command status on the Detailed transaction page if the transaction was cancelled

Added a Planchecker image to the ADBCC service (docker-compose) in the ADB bundle

Processing of NodeStatusMetric messages is supported

Added transaction pages: Transactions monitoring, Transactions history, page with detailed information about the transaction and a list of requests made within this transaction

Added the ability to filter data by schema and relation on the Query history page

Added a link to Transactions history in order to navigate to queries from the Audit page

Added a link to the parent transaction in the Command details tab

The SQL statement groups DDL, DML, DCL, and TCL are now displayed on the Monitor and History pages

Added Spill and Spill Skew columns to the customization table on the Monitor/Command and History/Command pages

A non-blocking socket is now used to communicate with an agent

Long values of the Query ID column overlapped the next column with data on the History/Command page

The incorrect sorting according to the schema with a heavy load on the Relation audit page

When a user moved from the statistics to the list of commands on the Relation audit page, the command was being duplicated

The missing tooltip for the Reset button

The NullPointerException exception occurred during attempts to get system metrics

Empty values were at the end of the list when the data was being sorted

The queuedTime field displayed different values on the Monitor page and the Command details page

The cache did not work properly, and the data was obtained directly from the database

After the Cluster name column was added, the cluster value was not displayed in the table on the Monitor/Command and History/Command pages

The Duplicate key value violates unique constraint error

The bug with sorting on the Monitor and History pages when all the records became hidden

Integrated the Avg cluster query metrics

Implemented compression for huge queries

Added a configuration for Registry Service to configure the addresses of backend servers

The actual statistics from EXPLAIN ANALYZE for finished queries is processed now

The Plan&Progress and Text panels are hidden now if the content is empty

ADBCC exceeded the limit of connections to ADB

Fixed the relation name display for a Dynamic Seq Scan node

The color-coding for plan node statuses was broken

The login error with cyrillic passwords

Migrated the adcc-extension to the adbcc repository

Filtering invalid metrics from gpperfmon

Data audit support

Background jobs history

Support for virtual process memory in system metrics of commands

Service load ratio support

Changed the sort order for metrics with NULLS LAST on the Query Monitor/History pages

Reworked top menu

Bumped log4j2 dependencies

New system endpoints

HTTPS support in ADBCC SSL

The new maxMessageKbSize parameter in the adcc-agent configuration file

The agent stopped processing query metrics due to RejectedExecutionException

Added the maxMessageKbSize setting to agent.properties

Added a message size check

Fixed re-sending of incorrect messages

Fixed getting system metrics from ClickHouse

Multi-clusters

New system metrics: CPU, RAM, IO

Adding columns dynamically to the History and Monitoring pages

Information about background processes on the system status information page

Actualization for hanging queries

adcc-extension: database and username are now shown in a plan message

adcc-extension: lock polling is now disabled to avoid sending huge locks snapshots to agents

Using a ClickHouse database for saving draft system metrics

Added an agent build for Power8 LE platform (ppc64le arch)

Time-based Retention Policy

Fine-grained access to multiple databases for Advanced users

Extended information on errors

Column search filters and sorting on the Query Monitor and History pages

Information on ADBCC/ADB versions and documentation

Scaling of the plan modal window

UI updates

New UI for Backend Server

Ability to cancel and terminate queries on the Query Details and Monitoring pages

Ability to view locks and blocks on the Query Details page

Support of displaying information for the ADBCC agent

LDAP authentication

Truncating of query plan details