Release Notes

6.21.1_arenadata36

Features and Improvements

  • Sync with upstream 6.21.1
  • Enabling core dump files for ADB processes
  • Update docs on appending echo to arenadata_segments_monitor.sh in case of Zabbix integration
  • Update docs on system requirement for ADCC server
  • ADCC Release 3.6.0:
    • Using a non-blocking socket to communicate with an agent
    • The display of the following metrics Spill Files (skew), Row Skew in Node panel, Cpu Skew, Memory Skew, Read Skew, Write Skew has been changed to a uniform format: <metric_name> Skew <metril_value>%
    • Added new column Planner
    • Added the ability to filter commands by planner
    • Added the ability to cancel a transaction on the detailed transaction page
    • Added a button to reset and delete all previously applied filters on the monitor and history command pages
    • Reassign queries to other resource group
    • ADCC UI. Add configuration for planchecker address
    • Add planchecker image to service ADCC (docker-compose) to ADB bundle
    • Fixed ADB unavailability error in case of OutOfMemoryError error on ADCC
    • Fixed actual tuples count for commands
    • Fixed the error of incorrect display of the command status on the detailed transaction page if the transaction was canceled

Fixes

  • Tkhemali can’t process text fields with New line character
  • Release datum stream read when partition scan is over
  • EvalPlanQual routine erroneously arises for DELETE operation over partitioned AO relation
  • Fixed when gp_toolkit.pg_resgroup_move_query may lose RG slots
  • Avoid running auto_explain somewhere besides dispatcher process
  • Fixed when ALTER TABLE erase pg_appendonly values
  • Fixed gang loss during running transaction when we tried to call gp_dist_wait_status within a transaction
  • Fixed when entry db process initializes whole plan tree

6.21.0_arenadata35

Features and Improvements

  • Sync with upstream 6.21.0
  • Optimize DML queries under partitioned tables to avoid further planning if partition was pruned
  • Exclude gpmon background process from shared memory user list
  • Refactor gpdb specific code for dispatch result waiting
  • Fallback to Postgres for empty target list in CTE producer
  • ADCC Release 3.5.1:
    • Using a non-blocking socket to communicate with an agent
    • Support processing of NodeStatusMetric messages
    • Added transaction pages: Transactions monitoring page, Transactions history page, Page with detailed information about the transaction and a list of requests made within this transaction
    • Added the ability to filter data by schema and relation on query history page
    • Added link To transactions history to navigate to queries from the audit page
    • Added a link to the parent transaction in Command Details tab
    • SQL statement groups DDL, DML, DCL, and TCL are now displayed on Monitor and History pages
    • Added Spill and Spill Skew columns in table customization on Monitor/Command and History/Command pages
    • Fixed: History/Command: When generating long “Query ID” sizes, overlaps the next column with data
    • Fixed: Relation audit: When moving from statistics to a list of commands, the command is duplicated.
    • Fixed: Relation audit: Incorrect sorting according to the Scheme, with a heavy load
    • Fixed: Relation audit: When moving from statistics to a list of commands, the command is duplicated.
    • Fixed: Reset button tooltip missing
    • Fixed: NullPointerException when trying to get system metrics
    • Fixed: When sorting data, empty values are at the end of the list
    • Fixed: queuedTime displayed different values on the monitor page and the detailed command page
    • Fixed: Cache was not working properly, and the data was obtained directly from database
    • Fixed: After adding the Cluster Name column, the cluster value was not displayed in the table on Monitor/Command and History/Command pages
    • Fixed: the Duplicate key value violates unique constraint error
    • Fixed: bug with sorting on Monitor and History pages, when all the records became hidden

Fixes

  • Fix cross-slice interaction detection for subplans
  • Subplans ordering is broken when planner copy them for partitions
  • gpinitsystem fails on machine with custom locale
  • [gptkh] gptkh can’t parse engine settings with dot symbol
  • [kafka-adb] Re-added kafka-client-2.5.0.jar to kadb-fdw connector

6.20.4_arenadata34

Features and Improvements

  • Sync with upstream 6.20.4
  • ADB Bundle: Possibility to specify cluster network to distinguish high-speed network for the interconnect purpose
  • Monitoring: Provide instruction on how to add database-specific metrics to Zabbix

Fixes

  • ADCC: Fixed when ADCC Container Memory Usage panel is blank under certain conditions

6.20.3_arenadata34

Features and Improvements

  • Sync with upstream 6.20.3

Fixes

  • PXF: When migrating a PXF service, the rights to the directory are not checked
  • PXF: Can’t change PXF_JVM_OPTS parameters
  • Аrchive gpexpand_schema.tar is created in the directory /home/gpadmin and may overflow space
  • Compatibiliy platmorm is incorrect for Altlinux on community bundle

6.20.1_arenadata33

Features and Improvements

  • Sync with upstream 6.20.1
  • PXF: Include PXF 6.3.0 into ADB Bundle with upgrade process from PXF 5.x
  • PXF: Activate pxf cluster sync command
  • PXF: Add pxf bin directory to PATH var
  • PXF: Add ability to override datatypes mapping in external table for PXF
  • Add Tkhemali connector ver 1.0.1 with a lot of refactoring
  • Make editing /etc/hosts optional
  • Add ADB Bundle Action Delete Tkhemali
  • Add ADB Bundle Action Delete Kafka-to-ADB
  • Add ADB Bundle Action Delete ADB-to-Kafka
  • Add parameter -Dliquibase.releaseLocks=true to migration containfer for reconfigure restart action
  • ADCC Release 3.4.0:
    • Process actual statistics from EXPLAIN ANALYZE for finished queries
    • Migrating adcc-extension to adbcc repository
    • Integrate Avg cluster query metrics
    • Hide Plan&Progress and Text panels if content is empty
    • Huge query compression
    • Added configuration in Registry Service to configure the addresses of backend servers
    • Fixed: ADCC exceeds the limit of connections to ADB
    • Fixed: Display relation name for Dynamic Seq Scan node
    • Fixed: Color-coding for plan node statuses is broken
    • Fixed: Login error with Cyrillic password

Fixes

  • PXF: JDBC Hive. Error when one of the fields is a reserved word
  • ORCA fails on assert during transformation left join to inner join
  • mode aggregate doesn’t handle properly query finish command from QD
  • [kafka-adb] Set storage option for offset table explicitly to avoid overriding
  • ADCC installation fails if cluster name consists of more than one word
  • Interleaving of AO partitioned table removing and pg_total_relation_size call for one of its partition potentially causes SIGSERV
  • [gptkh] tnx() function error. Insert data into partition table and number of partitions more than 1
  • Tkhemali: Column names with spaces isnt handled properly
  • ORCA produces invalid plan for CTE with replicated tables
  • ORCA: enable merging Gather motion for singleton on segment
  • Monitor scripts are not added to crontab on action Activate Standby
  • Backend server is displayed as IP while agent as FQDN in a Version page

6.19.3_arenadata32

Features and Improvements

  • Sync with upstream 6.19.3

Fixes

  • ADCC Release 3.3.1:
    • Added filtering invalid metrics from gpperfmon

6.19.1_arenadata31

Features and Improvements

  • Sync with upstream 6.19.1
  • Hidden servername in https group adcc service
  • Added ADB Loader tools for RHEL 8
  • Added ability to deploy maintenance scripts for several databases
  • Enabled back log_lock_waits GUC
  • Built docker-compose rpm package 1.29.2
  • Added Delete PXF Action
  • Moved “disable firewall” option on cluster level from ADB service
  • ADCC Release 3.3.0:
    • Added support data audit
    • Bumped log4j2 dependencies
    • Added support background jobs history
    • Added support virtual memory for process
    • Added support for the service load ratio
    • Changed sort order for metrics with NULLS LAST in Query Monitor/History
    • Added new system endpoints
    • Reworked top menu

Fixes

  • Wrong column binding is used to extract values from SharedScan node
  • [gptkh] Distributed table test2_tmp_3054 looks at itself
  • [gptkh] Error in parsing settings for Distributed tables
  • Coordinator doesn’t use quote escaping for GUCs values when restore on QE
  • sshd option ‘MaxStartups’ must be added before ‘Match’

Known Issues and Workarounds

Issue

Gradual memory leak on mirror segments: issue # 13067

Workaround

Increase monitoring for memory consumption in a cluster, and if possible restart cluster during the maintenance. Issue will be patched in next release.

6.18.2_arenadata30

Features and Improvements

  • Sync with upstream 6.18.2
  • [6X backport] Implement archive_mode always
  • [6X backport] Zero fill pages skipped by a force WAL switch
  • Added kafka connector 1.0.4
  • Added plcontainer 2.1.5
  • ADCC Release 3.2.5:
    • Added supporting https in ADCC ssl
    • Added new parameter maxMessageKbSize to adcc-agent config file
    • Fixed agent stops processing query metrics due to RejectedExecutionException

Fixes

  • [power] Fixed unexpected subplans order leads to subselect_gp test fails

6.18.0_arenadata29

Fixes

  • Fixed when accessing from AO table Index returns the wrong result
  • kafka-fdw: fixed garbage in output at least for format text
  • gpbackup: fixed when it handles empty tables set for an incremental data backup properly
  • gpbackup: fixed when it hangs forever when interrupted
  • ADCC Release 3.2.4:
    • Added message size check
    • Fixed re-sending of incorrect messages
    • Added settings maxMessageKbSize to agent.properties
    • Fixed getting system metrics from clickhouse

6.18.0_arenadata28

Features and Improvements

  • Sync with upstream 6.18.0
  • gpbackup: added explicit order of tables by pg_class.relpages during gpbackup
  • PXF: added support for partitioning query for Sybase
  • ADCC Release 3.2.2:
    • adcc-extension: added database and username to plan message
    • adcc-extension: disabling lock polling to avoid sending huge locks snapshots to agents
    • Added support for multi clusters
    • Added system metrics: CPU, RAM, IO
    • Added actualization for hanging queries
    • Added Clickhouse Database for saving draft system metrics
    • Added dynamically adding columns to history and monitoring forms
    • Added information about background processes on the system status information page

Fixes

  • Fixed unexpected SIGSEGV error on Reindex operation
  • madlib on ppc64le arch included into build
  • Fixed when gpbackup may dump partitioned table’s data twice
  • Fixed Not enough memory reserved for the statement when calling the function multiple times for CTE SELECT statement in PL/pgSQL function on table with lot of partitions

6.17.5_arenadata26

Features and Improvements

  • Sync with upstream 6.17.5
  • ADB Bundle: added Delete action for Monitoring Clients
  • ADB Bundle: added Ready to upgrade status for bundle upgrade

Fixes

  • Fixed low CPU performance on Power with new added CGLAGS build options

6.17.2_arenadata26

Features and Improvements

  • Sync with upstream 6.17.2
  • Sync with pxf release-5.16.3
  • Sync with gpbackup 1.21.0 release
  • Sync with gpbackup-s3-plugin 1.7.0 release
  • Added build for Power8 LE platform (ppc64le arch)
  • ADCC Release 3.1.3

Known Issues and Workarounds

Issue

madlib on Power8 (ppc64le arch) breaks gpdb build

Workaround

No workaround. madlib is not included into Power8 (ppc64le arch) build.

6.17.1_arenadata25

Features and Improvements

  • Sync with upstream 6.17.1
  • Implement ZSTD compression support for gpbackup
  • adcc-extension: send schema name in Node message
  • ADB Bundle: external database connection for ADCC Database
  • ADCC Release 3.1.0:
    • Time-based Retention Policy
    • Fine-grained access to multiple databases for Advanced users
    • Show extended information about error in ADCC
    • Add info on ADCC/ADB versions and docs from ADCC web interface
    • Add scaling to plan modal window
    • Add column search filters and sorting for ADCC pages Query Monitor/History
    • ADCC UI updates

Fixes

  • Security Patch for cve-2020-25695 (sandbox escape)
  • kafka-fdw: fixed when segments fails after SELECT operation from KafkaToADB external table with batch size more than 40000 - <msg_count> * 40 bytes
  • Fixed when Postgres optimizer produces bad plan for replicated table with index
  • adcc-extension: fixed when expected segment distributed transaction context was ‘Segment Prepared’
  • adcc-extension: fixed when there is no possibility to differ QueryStatus for slices that executed on master
  • [Power] fixed bogus compiller error during pg_upgrade compilation

6.16.3_arenadata24

Features and Improvements

  • Sync with upstream 6.16.3
  • [6X backport] Detect POLLHUP/POLLRDHUP while running queries
  • adcc-extension: Retrieve and send error text
  • ADB Bundle: Define adjustable parameters for ADCC
  • ADCC Release 3.0.3:
    • New UI Backend Server
    • Query Details and Monitoring: canceling and terminate
    • Query Details: Locks and Blocks
    • Support displaying information for the ADCC agent
    • Query plan details are truncated
    • Support LDAP authentication

Fixes

  • ERROR: header checksum does not match
  • [6X] Restrict execution for multilevel correlated queries
  • adcc-extension: view with security barrier

Known Issues and Workarounds

Issue

kafka-fdw of versions 0.11-0.12 (they were ported since 6.15.0_arenadata17) is known to have a bug when batch size exceeds 40000 - <msg_count> * 40 bytes. SELECT operation from KafkaToADB external causes SEGFAULT. The fix on the issue is expected in a next ADB release.

Workaround

Do not use batch size larger than 40000 - <msg_count> * 40 bytes if you read more that one message (when reading just one message no any limits exist).

6.16.2_arenadata21

Features and Improvements

  • Sync with upstream 6.16.2
  • PXF: remove tuple count check for jdbc INSERT
  • [6X Backport] Shrink to zero relation’s segment files on truncate and delete
  • ADCC: Add jvm arguments for logging

Fixes

  • Fixed gpbackup failure when foreign tables are present in a backuping database: gpbackup tries to acquire lock on foreign table

Important

For installing or upgrading to 6.16.2_arenadata21 use ADCM version 2021.05.26.12 or higher

6.16.1_arenadata20

Features and Improvements

  • Sync with upstream 6.16.1
  • ADCC Release 2.1.1
  • [6X Backport] ORCA fails with SIGSEGV on queries to view under table with dropped columns
  • [6X backport] elog function ignore multibyte encoding when truncate long messages
  • ADB Bundle: Set netcat timeout for send metrics to monitoring in Altlinux
  • Kafka-ADB Connector: Allow user to set custom librdkafka options

Fixes

  • Fixed when PartitionSelector mechanism doesn’t return tuples in case of type mismatch
  • Fixed adcc-extension: unknown node with id==5 for subplan in values
  • Fixed adcc-extension: lost locally planed mark for node status messages for query with custom aggregate
  • Fixed wrong CCNTs for plans dispatched after InitPlans were executed
  • Removed redundant Gpmon_Incr_Rows_Out() calls from internal sort functions
  • Fixed invalid variable specified in segment directory path check in ADCM

Known Issues and Workarounds

Issue

gpbackup utility of versions 1.20.1-1.20.4 is known to have a bug when foreign tables are present in a backuping database. Backup fails with error SQLSTATE 42809. The fix on the issue is expected in a next ADB release.

Workaround

Replace current gpbackup with version 1.20.0 which included in ADB 6.13.0_arenadata12.

6.16.0_arenadata19

Features and Improvements

  • Sync with upstream 6.16.0
  • Implement diskquota extension update
  • Add gp_enable_gpperfmon=on parameter on master and segment server for ADCC

6.15.0_arenadata18

Features and Improvements

  • [6X backport gpdb PR 11625] pg_rewind: avoid removing log files
  • Add $PXF_CONF and $PXF_HOME env variable on PXF hosts
  • Kafka-ADB Connector: Implement signal handlers to interrupt consuming
  • Port ADB to Alt Linux 8.2

Fixes

  • Fixed invalid values in prodataaccess attribute of pg_proc relation after cluster initialization/upgrade
  • Fixed when Kafka-ADB Connector do not consider partitions empty when a query for their watermark offsets fails on timeout
  • [6X Backport] ORCA produces incorrect plan for NOT IN clause

6.15.0_arenadata17

Features and Improvements

  • Implement Arenadata Command Center 2.0

Fixes

  • Broken URL in ADB bundle service description

6.15.0_arenadata16

Features and Improvements

  • Sync with upstream 6.15.0
  • Implement switchover from master to standby via ADCM

Fixes

  • Kafka-ADB-connector: issue with converting AVRO dates before UNIX epoch to GPDB representation
  • Kafka-ADB-connector: issue with GPDB representation of AVRO double number that has more than 6 digits after radix point
  • Kafka-ADB-connector: issue with wrong interpretation for AVRO timestamps that below zero

6.14.1_arenadata14

Features and Improvements

  • Sync with upstream 6.14.1
  • Add link to main page in ADCC logo

Fixes

  • Fix resource group waiting queue corruption
  • pg_aocsseg inconsistency after column addition rollback
  • Remove online loading of static resources from the ADCC web interface
  • Cluster failed after single segment-host lost due to FTS misbehavior
  • Multiline text in CSV output from gplogfilter is incorrect

6.14.0_arenadata13_b2

Features and Improvements

  • Fixup incorrect links in repos of b1 release

6.14.0_arenadata13

Sync only release

Features and Improvements

  • Sync with upstream 6.14.1

6.13.0_arenadata12

Features and Improvements

  • Tkhemali-connector: Avoid intermediate conversion to byte array
  • Remove obsolete batching options in ADQM Connector (sinse we are using TEXT now)
  • Sync with upstream 6.13.0 codebase
  • Bump to latest version of additional components
  • Reduce noise in logs about gpstate
  • Support auxilliary relations that belongs to AO table to be income to pgstattupe extension to exactly estimate bloat of these relations

Fixes

  • GPDB fails after activation developer gucs of PostgreSQL
  • Orca optimizer builds inefficient query plan
  • Column-wise statistics target parameter doesn’t apply to partitioning tables
  • Error gpactivatestandby when run it too fast after power off master
  • Mistake in the parsing of decimal part for time logical types
  • Scale is applied incorrectly for decimal values what has a length less than precision

6.12.1_arenadata11

New Features

  • Sync code base with upstream GPDB 6.12.1
  • Add ofset function to Kafka (ADS) connector
  • Add rest commited function to Kafka (ADS) connector
  • Implement ‘text’ format for kafka connector
  • Provide JVM_OPTS setting for PXF
  • Support AVRO logical types in Kafka-ADB connector
  • Switch to new way of postgis installation
  • Kafka-ADB: Use rd_kafka_query_watermark_offsets to validate partition-offset pairs

Fixes

  • ADCC disk space usage troubles in case of coredumps
  • gprestore - ERROR: missing data for column
  • “The limit of distributed transactions has been reached” error on secondary
  • WindowFunc with winref 1 assigned to WindowAgg with winref 2 (nodeWindowAgg.c:2264)
  • Wrong behavior with grouping function
  • Exclusive lock on any partitioned table locks the pg_partitions view.
  • Server locks kadb.offsets table