Release Notes¶
6.21.1_arenadata36¶
Features and Improvements¶
- Sync with upstream 6.21.1
- Enabling core dump files for ADB processes
- Update docs on appending echo to
arenadata_segments_monitor.sh
in case of Zabbix integration - Update docs on system requirement for ADCC server
- ADCC Release 3.6.0:
- Using a non-blocking socket to communicate with an agent
- The display of the following metrics Spill Files (skew), Row Skew in Node panel, Cpu Skew, Memory Skew, Read Skew, Write Skew has been changed to a uniform format:
<metric_name> Skew <metril_value>%
- Added new column Planner
- Added the ability to filter commands by planner
- Added the ability to cancel a transaction on the detailed transaction page
- Added a button to reset and delete all previously applied filters on the monitor and history command pages
- Reassign queries to other resource group
- ADCC UI. Add configuration for planchecker address
- Add planchecker image to service ADCC (
docker-compose
) to ADB bundle - Fixed ADB unavailability error in case of
OutOfMemoryError
error on ADCC - Fixed actual tuples count for commands
- Fixed the error of incorrect display of the command status on the detailed transaction page if the transaction was canceled
Fixes¶
- Tkhemali can’t process text fields with New line character
- Release datum stream read when partition scan is over
EvalPlanQual
routine erroneously arises for DELETE operation over partitioned AO relation- Fixed when
gp_toolkit.pg_resgroup_move_query
may lose RG slots - Avoid running
auto_explain
somewhere besides dispatcher process - Fixed when ALTER TABLE erase
pg_appendonly
values - Fixed gang loss during running transaction when we tried to call
gp_dist_wait_status
within a transaction - Fixed when entry db process initializes whole plan tree
6.21.0_arenadata35¶
Features and Improvements¶
- Sync with upstream 6.21.0
- Optimize
DML
queries under partitioned tables to avoid further planning if partition was pruned - Exclude
gpmon
background process from shared memory user list - Refactor gpdb specific code for dispatch result waiting
- Fallback to Postgres for empty target list in CTE producer
- ADCC Release 3.5.1:
- Using a non-blocking socket to communicate with an agent
- Support processing of NodeStatusMetric messages
- Added transaction pages: Transactions monitoring page, Transactions history page, Page with detailed information about the transaction and a list of requests made within this transaction
- Added the ability to filter data by schema and relation on query history page
- Added link To transactions history to navigate to queries from the audit page
- Added a link to the parent transaction in
Command Details
tab - SQL statement groups
DDL
,DML
,DCL
, andTCL
are now displayed on Monitor and History pages - Added
Spill
andSpill Skew
columns in table customization on Monitor/Command and History/Command pages - Fixed: History/Command: When generating long “Query ID” sizes, overlaps the next column with data
- Fixed: Relation audit: When moving from statistics to a list of commands, the command is duplicated.
- Fixed: Relation audit: Incorrect sorting according to the Scheme, with a heavy load
- Fixed: Relation audit: When moving from statistics to a list of commands, the command is duplicated.
- Fixed: Reset button tooltip missing
- Fixed:
NullPointerException
when trying to get system metrics - Fixed: When sorting data, empty values are at the end of the list
- Fixed:
queuedTime
displayed different values on the monitor page and the detailed command page - Fixed: Cache was not working properly, and the data was obtained directly from database
- Fixed: After adding the Cluster Name column, the cluster value was not displayed in the table on Monitor/Command and History/Command pages
- Fixed: the
Duplicate key value violates unique constraint
error - Fixed: bug with sorting on Monitor and History pages, when all the records became hidden
Fixes¶
- Fix cross-slice interaction detection for subplans
- Subplans ordering is broken when planner copy them for partitions
- gpinitsystem fails on machine with custom locale
- [gptkh] gptkh can’t parse engine settings with dot symbol
- [kafka-adb] Re-added kafka-client-2.5.0.jar to kadb-fdw connector
6.20.4_arenadata34¶
Features and Improvements
- Sync with upstream 6.20.4
- ADB Bundle: Possibility to specify cluster network to distinguish high-speed network for the interconnect purpose
- Monitoring: Provide instruction on how to add database-specific metrics to Zabbix
Fixes¶
- ADCC: Fixed when ADCC Container Memory Usage panel is blank under certain conditions
6.20.3_arenadata34¶
Features and Improvements
- Sync with upstream 6.20.3
Fixes¶
- PXF: When migrating a PXF service, the rights to the directory are not checked
- PXF: Can’t change PXF_JVM_OPTS parameters
- Аrchive gpexpand_schema.tar is created in the directory /home/gpadmin and may overflow space
- Compatibiliy platmorm is incorrect for Altlinux on community bundle
6.20.1_arenadata33¶
Features and Improvements¶
- Sync with upstream 6.20.1
- PXF: Include PXF 6.3.0 into ADB Bundle with upgrade process from PXF 5.x
- PXF: Activate pxf cluster sync command
- PXF: Add pxf bin directory to PATH var
- PXF: Add ability to override datatypes mapping in external table for PXF
- Add Tkhemali connector ver 1.0.1 with a lot of refactoring
- Make editing /etc/hosts optional
- Add ADB Bundle Action Delete Tkhemali
- Add ADB Bundle Action Delete Kafka-to-ADB
- Add ADB Bundle Action Delete ADB-to-Kafka
- Add parameter
-Dliquibase.releaseLocks=true
to migration containfer for reconfigure restart action - ADCC Release 3.4.0:
- Process actual statistics from EXPLAIN ANALYZE for finished queries
- Migrating adcc-extension to adbcc repository
- Integrate Avg cluster query metrics
- Hide Plan&Progress and Text panels if content is empty
- Huge query compression
- Added configuration in Registry Service to configure the addresses of backend servers
- Fixed: ADCC exceeds the limit of connections to ADB
- Fixed: Display relation name for Dynamic Seq Scan node
- Fixed: Color-coding for plan node statuses is broken
- Fixed: Login error with Cyrillic password
Fixes¶
- PXF: JDBC Hive. Error when one of the fields is a reserved word
- ORCA fails on assert during transformation left join to inner join
- mode aggregate doesn’t handle properly query finish command from QD
- [kafka-adb] Set storage option for offset table explicitly to avoid overriding
- ADCC installation fails if cluster name consists of more than one word
- Interleaving of AO partitioned table removing and pg_total_relation_size call for one of its partition potentially causes SIGSERV
- [gptkh] tnx() function error. Insert data into partition table and number of partitions more than 1
- Tkhemali: Column names with spaces isnt handled properly
- ORCA produces invalid plan for CTE with replicated tables
- ORCA: enable merging Gather motion for singleton on segment
- Monitor scripts are not added to crontab on action Activate Standby
- Backend server is displayed as IP while agent as FQDN in a Version page
6.19.3_arenadata32¶
Features and Improvements¶
- Sync with upstream 6.19.3
Fixes¶
- ADCC Release 3.3.1:
- Added filtering invalid metrics from gpperfmon
6.19.1_arenadata31¶
Features and Improvements¶
- Sync with upstream 6.19.1
- Hidden servername in https group adcc service
- Added ADB Loader tools for RHEL 8
- Added ability to deploy maintenance scripts for several databases
- Enabled back log_lock_waits GUC
- Built docker-compose rpm package 1.29.2
- Added Delete PXF Action
- Moved “disable firewall” option on cluster level from ADB service
- ADCC Release 3.3.0:
- Added support data audit
- Bumped log4j2 dependencies
- Added support background jobs history
- Added support virtual memory for process
- Added support for the service load ratio
- Changed sort order for metrics with NULLS LAST in Query Monitor/History
- Added new system endpoints
- Reworked top menu
Fixes¶
- Wrong column binding is used to extract values from SharedScan node
- [gptkh] Distributed table test2_tmp_3054 looks at itself
- [gptkh] Error in parsing settings for Distributed tables
- Coordinator doesn’t use quote escaping for GUCs values when restore on QE
- sshd option ‘MaxStartups’ must be added before ‘Match’
6.18.2_arenadata30¶
Features and Improvements¶
- Sync with upstream 6.18.2
- [6X backport] Implement archive_mode always
- [6X backport] Zero fill pages skipped by a force WAL switch
- Added kafka connector 1.0.4
- Added plcontainer 2.1.5
- ADCC Release 3.2.5:
- Added supporting https in ADCC ssl
- Added new parameter maxMessageKbSize to adcc-agent config file
- Fixed agent stops processing query metrics due to RejectedExecutionException
Fixes¶
- [power] Fixed unexpected subplans order leads to subselect_gp test fails
6.18.0_arenadata29¶
Fixes¶
- Fixed when accessing from AO table Index returns the wrong result
- kafka-fdw: fixed garbage in output at least for format text
- gpbackup: fixed when it handles empty tables set for an incremental data backup properly
- gpbackup: fixed when it hangs forever when interrupted
- ADCC Release 3.2.4:
- Added message size check
- Fixed re-sending of incorrect messages
- Added settings
maxMessageKbSize
to agent.properties - Fixed getting system metrics from clickhouse
6.18.0_arenadata28¶
Features and Improvements¶
- Sync with upstream 6.18.0
- gpbackup: added explicit order of tables by
pg_class.relpages
during gpbackup - PXF: added support for partitioning query for Sybase
- ADCC Release 3.2.2:
- adcc-extension: added database and username to plan message
- adcc-extension: disabling lock polling to avoid sending huge locks snapshots to agents
- Added support for multi clusters
- Added system metrics: CPU, RAM, IO
- Added actualization for hanging queries
- Added Clickhouse Database for saving draft system metrics
- Added dynamically adding columns to history and monitoring forms
- Added information about background processes on the system status information page
Fixes¶
- Fixed unexpected SIGSEGV error on Reindex operation
- madlib on ppc64le arch included into build
- Fixed when gpbackup may dump partitioned table’s data twice
- Fixed
Not enough memory reserved for the statement
when calling the function multiple times for CTE SELECT statement in PL/pgSQL function on table with lot of partitions
6.17.5_arenadata26¶
Features and Improvements¶
- Sync with upstream 6.17.5
- ADB Bundle: added Delete action for Monitoring Clients
- ADB Bundle: added Ready to upgrade status for bundle upgrade
Fixes¶
- Fixed low CPU performance on Power with new added CGLAGS build options
6.17.2_arenadata26¶
Features and Improvements¶
- Sync with upstream 6.17.2
- Sync with pxf release-5.16.3
- Sync with gpbackup 1.21.0 release
- Sync with gpbackup-s3-plugin 1.7.0 release
- Added build for Power8 LE platform (ppc64le arch)
- ADCC Release 3.1.3
Known Issues and Workarounds¶
Issue
madlib on Power8 (ppc64le arch) breaks gpdb build
Workaround
No workaround. madlib is not included into Power8 (ppc64le arch) build.
6.17.1_arenadata25¶
Features and Improvements¶
- Sync with upstream 6.17.1
- Implement ZSTD compression support for gpbackup
- adcc-extension: send schema name in Node message
- ADB Bundle: external database connection for ADCC Database
- ADCC Release 3.1.0:
- Time-based Retention Policy
- Fine-grained access to multiple databases for Advanced users
- Show extended information about error in ADCC
- Add info on ADCC/ADB versions and docs from ADCC web interface
- Add scaling to plan modal window
- Add column search filters and sorting for ADCC pages Query Monitor/History
- ADCC UI updates
Fixes¶
- Security Patch for cve-2020-25695 (sandbox escape)
- kafka-fdw: fixed when segments fails after SELECT operation from KafkaToADB external table with batch size more than
40000 - <msg_count> * 40
bytes - Fixed when Postgres optimizer produces bad plan for replicated table with index
- adcc-extension: fixed when expected segment distributed transaction context was ‘Segment Prepared’
- adcc-extension: fixed when there is no possibility to differ QueryStatus for slices that executed on master
- [Power] fixed bogus compiller error during pg_upgrade compilation
6.16.3_arenadata24¶
Features and Improvements¶
- Sync with upstream 6.16.3
- [6X backport] Detect POLLHUP/POLLRDHUP while running queries
- adcc-extension: Retrieve and send error text
- ADB Bundle: Define adjustable parameters for ADCC
- ADCC Release 3.0.3:
- New UI Backend Server
- Query Details and Monitoring: canceling and terminate
- Query Details: Locks and Blocks
- Support displaying information for the ADCC agent
- Query plan details are truncated
- Support LDAP authentication
Fixes¶
- ERROR: header checksum does not match
- [6X] Restrict execution for multilevel correlated queries
- adcc-extension: view with security barrier
Known Issues and Workarounds¶
Issue
kafka-fdw of versions 0.11-0.12 (they were ported since 6.15.0_arenadata17) is known to have a bug when batch size exceeds 40000 - <msg_count> * 40
bytes. SELECT operation from KafkaToADB external causes SEGFAULT. The fix on the issue is expected in a next ADB release.
Workaround
Do not use batch size larger than 40000 - <msg_count> * 40
bytes if you read more that one message (when reading just one message no any limits exist).
6.16.2_arenadata21¶
Features and Improvements¶
- Sync with upstream 6.16.2
- PXF: remove tuple count check for jdbc INSERT
- [6X Backport] Shrink to zero relation’s segment files on truncate and delete
- ADCC: Add jvm arguments for logging
Fixes¶
- Fixed gpbackup failure when foreign tables are present in a backuping database: gpbackup tries to acquire lock on foreign table
Important
For installing or upgrading to 6.16.2_arenadata21 use ADCM version 2021.05.26.12 or higher
6.16.1_arenadata20¶
Features and Improvements¶
- Sync with upstream 6.16.1
- ADCC Release 2.1.1
- [6X Backport] ORCA fails with SIGSEGV on queries to view under table with dropped columns
- [6X backport] elog function ignore multibyte encoding when truncate long messages
- ADB Bundle: Set netcat timeout for send metrics to monitoring in Altlinux
- Kafka-ADB Connector: Allow user to set custom librdkafka options
Fixes¶
- Fixed when PartitionSelector mechanism doesn’t return tuples in case of type mismatch
- Fixed adcc-extension: unknown node with id==5 for subplan in values
- Fixed adcc-extension: lost locally planed mark for node status messages for query with custom aggregate
- Fixed wrong CCNTs for plans dispatched after InitPlans were executed
- Removed redundant Gpmon_Incr_Rows_Out() calls from internal sort functions
- Fixed invalid variable specified in segment directory path check in ADCM
Known Issues and Workarounds¶
Issue
gpbackup utility of versions 1.20.1-1.20.4 is known to have a bug when foreign tables are present in a backuping database. Backup fails with error SQLSTATE 42809. The fix on the issue is expected in a next ADB release.
Workaround
Replace current gpbackup with version 1.20.0 which included in ADB 6.13.0_arenadata12.
6.16.0_arenadata19¶
Features and Improvements¶
- Sync with upstream 6.16.0
- Implement diskquota extension update
- Add gp_enable_gpperfmon=on parameter on master and segment server for ADCC
6.15.0_arenadata18¶
Features and Improvements¶
- [6X backport gpdb PR 11625] pg_rewind: avoid removing log files
- Add $PXF_CONF and $PXF_HOME env variable on PXF hosts
- Kafka-ADB Connector: Implement signal handlers to interrupt consuming
- Port ADB to Alt Linux 8.2
Fixes¶
- Fixed invalid values in prodataaccess attribute of pg_proc relation after cluster initialization/upgrade
- Fixed when Kafka-ADB Connector do not consider partitions empty when a query for their watermark offsets fails on timeout
- [6X Backport] ORCA produces incorrect plan for NOT IN clause
6.15.0_arenadata17¶
Features and Improvements¶
- Implement Arenadata Command Center 2.0
Fixes¶
- Broken URL in ADB bundle service description
6.15.0_arenadata16¶
Features and Improvements¶
- Sync with upstream 6.15.0
- Implement switchover from master to standby via ADCM
Fixes¶
- Kafka-ADB-connector: issue with converting AVRO dates before UNIX epoch to GPDB representation
- Kafka-ADB-connector: issue with GPDB representation of AVRO double number that has more than 6 digits after radix point
- Kafka-ADB-connector: issue with wrong interpretation for AVRO timestamps that below zero
6.14.1_arenadata14¶
Features and Improvements¶
- Sync with upstream 6.14.1
- Add link to main page in ADCC logo
Fixes¶
- Fix resource group waiting queue corruption
- pg_aocsseg inconsistency after column addition rollback
- Remove online loading of static resources from the ADCC web interface
- Cluster failed after single segment-host lost due to FTS misbehavior
- Multiline text in CSV output from gplogfilter is incorrect
6.13.0_arenadata12¶
Features and Improvements¶
- Tkhemali-connector: Avoid intermediate conversion to byte array
- Remove obsolete batching options in ADQM Connector (sinse we are using TEXT now)
- Sync with upstream 6.13.0 codebase
- Bump to latest version of additional components
- Reduce noise in logs about gpstate
- Support auxilliary relations that belongs to AO table to be income to pgstattupe extension to exactly estimate bloat of these relations
Fixes¶
- GPDB fails after activation developer gucs of PostgreSQL
- Orca optimizer builds inefficient query plan
- Column-wise statistics target parameter doesn’t apply to partitioning tables
- Error gpactivatestandby when run it too fast after power off master
- Mistake in the parsing of decimal part for time logical types
- Scale is applied incorrectly for decimal values what has a length less than precision
6.12.1_arenadata11¶
New Features¶
- Sync code base with upstream GPDB 6.12.1
- Add ofset function to Kafka (ADS) connector
- Add rest commited function to Kafka (ADS) connector
- Implement ‘text’ format for kafka connector
- Provide JVM_OPTS setting for PXF
- Support AVRO logical types in Kafka-ADB connector
- Switch to new way of postgis installation
- Kafka-ADB: Use rd_kafka_query_watermark_offsets to validate partition-offset pairs
Fixes¶
- ADCC disk space usage troubles in case of coredumps
- gprestore - ERROR: missing data for column
- “The limit of distributed transactions has been reached” error on secondary
- WindowFunc with winref 1 assigned to WindowAgg with winref 2 (nodeWindowAgg.c:2264)
- Wrong behavior with grouping function
- Exclusive lock on any partitioned table locks the pg_partitions view.
- Server locks kadb.offsets table