Release Notes ============= 6.20.3_arenadata34 ------------------ Features and Improvements * Sync with upstream 6.20.3 Fixes ~~~~~ * PXF: When migrating a PXF service, the rights to the directory are not checked * PXF: Can't change PXF_JVM_OPTS parameters * Аrchive gpexpand_schema.tar is created in the directory /home/gpadmin and may overflow space * Compatibiliy platmorm is incorrect for Altlinux on community bundle 6.20.1_arenadata33 ------------------ Features and Improvements ~~~~~~~~~~~~~~~~~~~~~~~~~ * Sync with upstream 6.20.1 * PXF: Include PXF 6.3.0 into ADB Bundle with upgrade process from PXF 5.x * PXF: Activate pxf cluster sync command * PXF: Add pxf bin directory to `PATH` var * PXF: Add ability to override datatypes mapping in external table for PXF * Add Tkhemali connector ver 1.0.1 with a lot of refactoring * Make editing `/etc/hosts` optional * Add ADB Bundle Action `Delete Tkhemali` * Add ADB Bundle Action `Delete Kafka-to-ADB` * Add ADB Bundle Action `Delete ADB-to-Kafka` * Add parameter ``-Dliquibase.releaseLocks=true`` to migration containfer for reconfigure restart action * ADCC Release 3.4.0: * Process actual statistics from EXPLAIN ANALYZE for finished queries * Migrating `adcc-extension` to adbcc repository * Integrate Avg cluster query metrics * Hide `Plan&Progress` and `Text` panels if content is empty * Huge query compression * Added configuration in Registry Service to configure the addresses of backend servers * Fixed: ADCC exceeds the limit of connections to ADB * Fixed: Display relation name for Dynamic Seq Scan node * Fixed: Color-coding for plan node statuses is broken * Fixed: Login error with Cyrillic password Fixes ~~~~~ * PXF: JDBC Hive. Error when one of the fields is a reserved word * ORCA fails on assert during transformation left join to inner join * mode aggregate doesn't handle properly query finish command from QD * [kafka-adb] Set storage option for offset table explicitly to avoid overriding * ADCC installation fails if cluster name consists of more than one word * Interleaving of AO partitioned table removing and pg_total_relation_size call for one of its partition potentially causes SIGSERV * [gptkh] `tnx()` function error. Insert data into partition table and number of partitions more than 1 * Tkhemali: Column names with spaces isnt handled properly * ORCA produces invalid plan for CTE with replicated tables * ORCA: enable merging Gather motion for singleton on segment * Monitor scripts are not added to crontab on action `Activate Standby` * Backend server is displayed as IP while agent as FQDN in a Version page 6.19.3_arenadata32 ------------------ Features and Improvements ~~~~~~~~~~~~~~~~~~~~~~~~~ * Sync with upstream 6.19.3 Fixes ~~~~~ * ADCC Release 3.3.1: * Added filtering invalid metrics from gpperfmon 6.19.1_arenadata31 ------------------ Features and Improvements ~~~~~~~~~~~~~~~~~~~~~~~~~ * Sync with upstream 6.19.1 * Hidden servername in https group adcc service * Added ADB Loader tools for RHEL 8 * Added ability to deploy maintenance scripts for several databases * Enabled back log_lock_waits GUC * Built docker-compose rpm package 1.29.2 * Added Delete PXF Action * Moved "disable firewall" option on cluster level from ADB service * ADCC Release 3.3.0: * Added support data audit * Bumped log4j2 dependencies * Added support background jobs history * Added support virtual memory for process * Added support for the service load ratio * Changed sort order for metrics with NULLS LAST in Query Monitor/History * Added new system endpoints * Reworked top menu Fixes ~~~~~ * Wrong column binding is used to extract values from SharedScan node * [gptkh] Distributed table test2_tmp_3054 looks at itself * [gptkh] Error in parsing settings for Distributed tables * Coordinator doesn't use quote escaping for GUCs values when restore on QE * sshd option 'MaxStartups' must be added before 'Match' Known Issues and Workarounds ~~~~~ **Issue** Gradual memory leak on mirror segments: issue # `13067 `_ **Workaround** Increase monitoring for memory consumption in a cluster, and if possible restart cluster during the maintenance. Issue will be patched in next release. 6.18.2_arenadata30 ------------------ Features and Improvements ~~~~~~~~~~~~~~~~~~~~~~~~~ * Sync with upstream 6.18.2 * [6X backport] Implement archive_mode always * [6X backport] Zero fill pages skipped by a force WAL switch * Added kafka connector 1.0.4 * Added plcontainer 2.1.5 * ADCC Release 3.2.5: * Added supporting https in ADCC ssl * Added new parameter maxMessageKbSize to adcc-agent config file * Fixed agent stops processing query metrics due to RejectedExecutionException Fixes ~~~~~ * [power] Fixed unexpected subplans order leads to subselect_gp test fails 6.18.0_arenadata29 ------------------ Fixes ~~~~~ * Fixed when accessing from AO table Index returns the wrong result * `kafka-fdw`: fixed garbage in output at least for format text * `gpbackup`: fixed when it handles empty tables set for an incremental data backup properly * `gpbackup`: fixed when it hangs forever when interrupted * ADCC Release 3.2.4: * Added message size check * Fixed re-sending of incorrect messages * Added settings ``maxMessageKbSize`` to agent.properties * Fixed getting system metrics from clickhouse 6.18.0_arenadata28 ------------------ Features and Improvements ~~~~~~~~~~~~~~~~~~~~~~~~~ * Sync with upstream 6.18.0 * gpbackup: added explicit order of tables by ``pg_class.relpages`` during gpbackup * PXF: added support for partitioning query for Sybase * ADCC Release 3.2.2: * adcc-extension: added database and username to plan message * adcc-extension: disabling lock polling to avoid sending huge locks snapshots to agents * Added support for multi clusters * Added system metrics: CPU, RAM, IO * Added actualization for hanging queries * Added Clickhouse Database for saving draft system metrics * Added dynamically adding columns to history and monitoring forms * Added information about background processes on the system status information page Fixes ~~~~~ * Fixed unexpected SIGSEGV error on Reindex operation * `madlib` on ppc64le arch included into build * Fixed when `gpbackup` may dump partitioned table's data twice * Fixed ``Not enough memory reserved for the statement`` when calling the function multiple times for CTE SELECT statement in PL/pgSQL function on table with lot of partitions 6.17.5_arenadata26 ------------------ Features and Improvements ~~~~~~~~~~~~~~~~~~~~~~~~~ * Sync with upstream 6.17.5 * ADB Bundle: added `Delete` action for Monitoring Clients * ADB Bundle: added `Ready to upgrade` status for bundle upgrade Fixes ~~~~~ * Fixed low CPU performance on Power with new added CGLAGS build options 6.17.2_arenadata26 ------------------ Features and Improvements ~~~~~~~~~~~~~~~~~~~~~~~~~ * Sync with upstream 6.17.2 * Sync with pxf release-5.16.3 * Sync with gpbackup 1.21.0 release * Sync with gpbackup-s3-plugin 1.7.0 release * Added build for Power8 LE platform (ppc64le arch) * ADCC Release 3.1.3 Known Issues and Workarounds ~~~~~ **Issue** `madlib` on Power8 (ppc64le arch) breaks gpdb build **Workaround** No workaround. `madlib` is not included into Power8 (ppc64le arch) build. 6.17.1_arenadata25 ------------------ Features and Improvements ~~~~~~~~~~~~~~~~~~~~~~~~~ * Sync with upstream 6.17.1 * Implement ZSTD compression support for gpbackup * adcc-extension: send schema name in Node message * ADB Bundle: external database connection for ADCC Database * ADCC Release 3.1.0: * Time-based Retention Policy * Fine-grained access to multiple databases for Advanced users * Show extended information about error in ADCC * Add info on ADCC/ADB versions and docs from ADCC web interface * Add scaling to plan modal window * Add column search filters and sorting for ADCC pages Query Monitor/History * ADCC UI updates Fixes ~~~~~ * Security Patch for `cve-2020-25695 `_ (sandbox escape) * `kafka-fdw`: fixed when segments fails after SELECT operation from KafkaToADB external table with batch size more than ``40000 - * 40`` bytes * Fixed when Postgres optimizer produces bad plan for replicated table with index * adcc-extension: fixed when expected segment distributed transaction context was 'Segment Prepared' * adcc-extension: fixed when there is no possibility to differ QueryStatus for slices that executed on master * [Power] fixed bogus compiller error during pg_upgrade compilation 6.16.3_arenadata24 ------------------ Features and Improvements ~~~~~~~~~~~~~~~~~~~~~~~~~ * Sync with upstream 6.16.3 * [6X backport] Detect POLLHUP/POLLRDHUP while running queries * adcc-extension: Retrieve and send error text * ADB Bundle: Define adjustable parameters for ADCC * ADCC Release 3.0.3: * New UI Backend Server * Query Details and Monitoring: canceling and terminate * Query Details: Locks and Blocks * Support displaying information for the ADCC agent * Query plan details are truncated * Support LDAP authentication Fixes ~~~~~ * ERROR: header checksum does not match * [6X] Restrict execution for multilevel correlated queries * adcc-extension: view with security barrier Known Issues and Workarounds ~~~~~ **Issue** `kafka-fdw` of versions 0.11-0.12 (they were ported since *6.15.0_arenadata17*) is known to have a bug when batch size exceeds ``40000 - * 40`` bytes. SELECT operation from KafkaToADB external causes SEGFAULT. The fix on the issue is expected in a next ADB release. **Workaround** Do not use batch size larger than ``40000 - * 40`` bytes if you read more that one message (when reading just one message no any limits exist). 6.16.2_arenadata21 ------------------ Features and Improvements ~~~~~~~~~~~~~~~~~~~~~~~~~ * Sync with upstream 6.16.2 * PXF: remove tuple count check for jdbc INSERT * [6X Backport] Shrink to zero relation's segment files on truncate and delete * ADCC: Add jvm arguments for logging Fixes ~~~~~ * Fixed gpbackup failure when foreign tables are present in a backuping database: gpbackup tries to acquire lock on foreign table .. important:: For installing or upgrading to 6.16.2_arenadata21 use ADCM version 2021.05.26.12 or higher 6.16.1_arenadata20 ------------------ Features and Improvements ~~~~~~~~~~~~~~~~~~~~~~~~~ * Sync with upstream 6.16.1 * ADCC Release 2.1.1 * [6X Backport] ORCA fails with SIGSEGV on queries to view under table with dropped columns * [6X backport] elog function ignore multibyte encoding when truncate long messages * ADB Bundle: Set netcat timeout for send metrics to monitoring in Altlinux * Kafka-ADB Connector: Allow user to set custom librdkafka options Fixes ~~~~~ * Fixed when PartitionSelector mechanism doesn't return tuples in case of type mismatch * Fixed adcc-extension: unknown node with id==5 for subplan in values * Fixed adcc-extension: lost locally planed mark for node status messages for query with custom aggregate * Fixed wrong CCNTs for plans dispatched after InitPlans were executed * Removed redundant Gpmon_Incr_Rows_Out() calls from internal sort functions * Fixed invalid variable specified in segment directory path check in ADCM Known Issues and Workarounds ~~~~~ **Issue** `gpbackup` utility of versions 1.20.1-1.20.4 is known to have a bug when foreign tables are present in a backuping database. Backup fails with error SQLSTATE 42809. The fix on the issue is expected in a next ADB release. **Workaround** Replace current `gpbackup` with version 1.20.0 which included in ADB 6.13.0_arenadata12. 6.16.0_arenadata19 ------------------ Features and Improvements ~~~~~~~~~~~~~~~~~~~~~~~~~ * Sync with upstream 6.16.0 * Implement diskquota extension update * Add gp_enable_gpperfmon=on parameter on master and segment server for ADCC 6.15.0_arenadata18 ------------------ Features and Improvements ~~~~~~~~~~~~~~~~~~~~~~~~~ * [6X backport gpdb PR `11625 `_] pg_rewind: avoid removing log files * Add $PXF_CONF and $PXF_HOME env variable on PXF hosts * Kafka-ADB Connector: Implement signal handlers to interrupt consuming * Port ADB to Alt Linux 8.2 Fixes ~~~~~ * Fixed invalid values in `prodataaccess` attribute of `pg_proc` relation after cluster initialization/upgrade * Fixed when Kafka-ADB Connector do not consider partitions empty when a query for their watermark offsets fails on timeout * [6X Backport] ORCA produces incorrect plan for NOT IN clause 6.15.0_arenadata17 ------------------ Features and Improvements ~~~~~~~~~~~~~~~~~~~~~~~~~ * Implement Arenadata Command Center 2.0 Fixes ~~~~~ * Broken URL in ADB bundle service description 6.15.0_arenadata16 ------------------ Features and Improvements ~~~~~~~~~~~~~~~~~~~~~~~~~ * Sync with upstream 6.15.0 * Implement switchover from master to standby via ADCM Fixes ~~~~~ * Kafka-ADB-connector: issue with converting AVRO dates before UNIX epoch to GPDB representation * Kafka-ADB-connector: issue with GPDB representation of AVRO double number that has more than 6 digits after radix point * Kafka-ADB-connector: issue with wrong interpretation for AVRO timestamps that below zero 6.14.1_arenadata14 ------------------ Features and Improvements ~~~~~~~~~~~~~~~~~~~~~~~~~ * Sync with upstream 6.14.1 * Add link to main page in ADCC logo Fixes ~~~~~ * Fix resource group waiting queue corruption * pg_aocsseg inconsistency after column addition rollback * Remove online loading of static resources from the ADCC web interface * Cluster failed after single segment-host lost due to FTS misbehavior * Multiline text in CSV output from gplogfilter is incorrect 6.14.0_arenadata13_b2 --------------------- Features and Improvements ~~~~~~~~~~~~~~~~~~~~~~~~~ * Fixup incorrect links in repos of b1 release 6.14.0_arenadata13 ------------------ Sync only release Features and Improvements ~~~~~~~~~~~~~~~~~~~~~~~~~ * Sync with upstream 6.14.1 6.13.0_arenadata12 ------------------ Features and Improvements ~~~~~~~~~~~~~~~~~~~~~~~~~ * Tkhemali-connector: Avoid intermediate conversion to byte array * Remove obsolete batching options in ADQM Connector (sinse we are using TEXT now) * Sync with upstream 6.13.0 codebase * Bump to latest version of additional components * Reduce noise in logs about gpstate * Support auxilliary relations that belongs to AO table to be income to pgstattupe extension to exactly estimate bloat of these relations Fixes ~~~~~ * GPDB fails after activation developer gucs of PostgreSQL * Orca optimizer builds inefficient query plan * Column-wise statistics target parameter doesn't apply to partitioning tables * Error gpactivatestandby when run it too fast after power off master * Mistake in the parsing of decimal part for time logical types * Scale is applied incorrectly for decimal values what has a length less than precision 6.12.1_arenadata11 ------------------ New Features ~~~~~~~~~~~~ * Sync code base with upstream GPDB 6.12.1 * Add ofset function to Kafka (ADS) connector * Add rest commited function to Kafka (ADS) connector * Implement 'text' format for kafka connector * Provide JVM_OPTS setting for PXF * Support AVRO logical types in Kafka-ADB connector * Switch to new way of postgis installation * Kafka-ADB: Use rd_kafka_query_watermark_offsets to validate partition-offset pairs Fixes ~~~~~ * ADCC disk space usage troubles in case of coredumps * gprestore - ERROR: missing data for column * "The limit of distributed transactions has been reached" error on secondary * WindowFunc with winref 1 assigned to WindowAgg with winref 2 (nodeWindowAgg.c:2264) * Wrong behavior with grouping function * Exclusive lock on any partitioned table locks the pg_partitions view. * Server locks kadb.offsets table