Releases

Reflected changes in the ADCM release 2.3.0 dated 05.09.2024:

Made the following fixes:

Added information about RED OS 7.3.2 support to the Prerequisites article.

Added the following articles:

Added a note with information on rolling upgrade to the Cluster actions article.

Reflected changes in the ADB release 6.27.1.57 dated 29.08.2024:

Added information about supported Docker version for CentOS to the Software requirements article (see point 3).

Added Accounts in ADB.

Added new articles:

In the Monitoring article, added the second schema (ADPG monitoring with Metrics storage from the same cluster).

Updated the Work with extensions article.

Added a note about the Backup: Restore cluster action to the Backup: Restore cluster and Backup and restore using pgBackRest articles.

In the Configure a cluster article, updated the last image and added the following sections: Configuration groups and Ansible settings.

In the Add components article, described the Hosts mode, which allows mapping components by hosts.

Instead of all articles in the installation guide related to hostproviders, left only one article Prepare hosts, which contains links to the detailed documentation on each hostprovider.

Added new articles:

In the Configure a cluster article, updated the last image, actualized parameters, and added the following sections: Configuration groups and Ansible settings.

In the Add components article, described the Hosts mode, which allows mapping components by hosts.

Instead of all articles in the installation guide related to hostproviders, left only one article Prepare hosts, which contains links to the detailed documentation on each hostprovider.

Updated the Sqoop overview article.

Added the Use filters in HBase article.

In the Configure a cluster article, updated the last image, actualized parameters, and added the following sections: Configuration groups and Ansible settings.

In the Add components article, described the Hosts mode, which allows mapping components by hosts.

Instead of all articles in the installation guide related to hostproviders, left only one article Prepare hosts, which contains links to the detailed documentation on each hostprovider.

In the Configure a cluster article, updated the last image, actualized parameters, and added the following sections: Configuration groups and Ansible settings.

In the Add components article, described the Hosts mode, which allows mapping components by hosts.

Instead of all articles in the installation guide related to hostproviders, left only one article Prepare hosts, which contains links to the detailed documentation on each hostprovider.

In the Configure a cluster article, added the following sections: Configuration groups and Ansible settings.

In the Add components article, described the Hosts mode, which allows mapping components by hosts.

Instead of all articles in the installation guide related to hostproviders, left only one article Prepare hosts, which contains links to the detailed documentation on each hostprovider.

In the Add components article, described the Hosts mode, which allows mapping components by hosts.

Instead of all articles in the installation guide related to hostproviders, left only one article Prepare hosts, which contains links to the detailed documentation on each hostprovider.

In the Create hosts with Yandex Cloud hostprovider article, added information on products that require FQDNs for hosts created via ADCM.

In the Manage a Yandex Cloud hostprovider article, added information that the Create hosts action is not designed for products that require FQDNs for hosts created via ADCM.

In the Create hosts with Cloud.ru Advanced hostprovider article, added information on products that require FQDNs for hosts created via ADCM.

In the Manage a Cloud.ru Advanced hostprovider article:

In the Create hosts with CROC Cloud hostprovider article, added information on products that require FQDNs for hosts created via ADCM.

In the Manage a CROC Cloud hostprovider article:

Reflected changes in the ADQM release 24.3.2.23.2.b1 dated 02.08.2024:

Added new articles:

Reflected changes in the ADB release 6.27.1.56:

Added a recommendation about usage of generic cores to the Requirements for Astra Linux section of the Software requirements article.

Updated the ADB known issues article (see the first table row).

Added the ADB → SSL section to the Configuration parameters article.

In the Configure a cluster article, updated the last image and added the following sections: Configuration groups and Ansible settings.

Reflected changes in the ADH release 3.2.4.3.b1 dated 23.07.2024:

Added articles for the Hive service:

Added articles for the Solr service:

Added articles for the Sqoop service:

Added articles for the MapReduce service:

For YARN, added the Configure and use GPU on YARN article.

In the Configuration parameters article, added a note about filling parameters in the Custom section.

Reflected changes in the ADPS release 1.1.2.b1 dated 23.07.2024:

Added conceptual articles:

In the Configuration parameters article, added a note about filling parameters in the Custom section.

Reflected changes in the ADS release 3.3.2.2.b1 dated 08.07.2024:

Added new articles:

In the Network requirements article, described the 443 port.

Added Release notes.

Described UI changes:

Reflected changes in the ADCM release 2.2.0 dated 05.07.2024:

Added Ansible inventory structure.

In the following articles, added a note that starting with ADCM 2.0.0 the usage of latest is not supported:

Made the following fixes:

Added Release notes for the ADB Control release 4.8.8.

Actualized ADB Control known issues.

Added Release notes for the ADBM release 2.0.4.

Actualized ADBM known issues.

Reflected changes in the ADPG release 14.3.7 dated 13.05.2024:

Added new articles:

Reflected changes in the ADH release dated 27.04.2024:

Added the Logging in YARN article.

Reflected changes in the ADPS release dated 27.04.2024:

Added articles for Ranger:

Reflected changes in the ADQM release 24.3.2.23.1.b1 dated 25.04.2024:

Added new articles:

Reflected changes in the ADBM release 1.7.3 dated 11.04.2024:

Added a note about saving a configuration to the Manage configurations in ADBM article.

Reflected changes in the ADB Control release 4.7.5 dated 11.04.2024:

In the Metrics offload → Structure of offloaded data section of the Configure job policies article, added a note about choosing DB.

Added a note about upgrade to the Add a cluster section of the Cluster management article.

Reflected changes in the ADB release 6.26.2.55 dated 11.04.2024:

Added information about maximum supported version of ClickHouse/ADQM to the Tkhemali Connector 1.X installation article.

Reflected changes in the ADS release 3.3.2.1.b1 dated 11.04.2024:

Added articles:

Added a note about configuration files to the Cluster update section of the Cluster actions article.

In the Install a cluster article, added a note about installation on a clean host with no previous installations of related or similar software.

Added Release notes for the ADB release 6.26.2.55.

Added the Cluster actions article.

Updated ADB known issues (see 6.12.1.11).

Added Release notes for the ADBM release 1.7.3.

Updated ADBM known issues (see 1.7.0).

Reflected changes in the ADH release 3.2.4.2.b1 dated 27.03.2024:

Added Enable Kerberos with prepared Active Directory server and no admin permissions.

In the Requirements for SSL certificates article, added a note about using FQDN as a key alias in keystore.jks.

In the Configure Kerberos authentication based on FreeIPA via ADCM article, added information about FQDN usage in the KDC hosts and Kadmin server fields.

Added new articles on Hive:

Added new articles on YARN:

Added a note about configuration files to the Cluster upgrade article.

In the Install a cluster article, added a note about installation on a clean host with no previous installations of related or similar software.

Reflected changes in the ADPS release 1.1.1.b1 dated 27.03.2024:

Added new articles on Ranger:

Extended the Manage roles in Ranger Admin UI article.

Updated the Ranger → LDAP sync source for User synchronizer parameter description in the Configuration parameters article.

In the Configure Kerberos authentication based on FreeIPA via ADCM article, added information about FQDN usage in the KDC hosts and Kadmin server fields.

Added Supported extensions.

Added SQL commands not supported in ADB.

In the ADB Control → Database parameters section of the Configuration parameters article, added a note for the Data dir field.

Updated the ADB documentation home page. Instead of the full list of articles (which is still available in the left navigation menu), top 10 popular articles are displayed (based on the statisctics of the past month).

Reflected changes in the ADCM release 2.1.0 dated 20.03.2024:

In the config → Types of configuration parameters section of the Bundle DSL article, actualized the structure description: added a list of available attributes and possible match values, updated the configuration example.

Restructured the Ansible modules and plugins article. Added the adcm_change_flag module description.

Added Concerns.

In the ADCM role model article, made the Permissions and roles table view more convenient.

The Maintenance mode article was moved from the How to section to Concepts.

Reflected changes in the ADPG release 14.3.6 dated 05.03.2024:

Added the Views and materialized views article.

In the ADPG → Patroni ADPG configurations section of the Configuration parameters article, described the use_custom_location, patroni_log_dir, patroni_log_file_size, patroni_log_file_num, and patroni_log_level parameters.

Added the Integration between ADQM and ADPG article.

In the Service actions, described actions available for Clickhousekeeper.

Information about the need for applying Reconfig and restart to ADQMDB after running Expand/Shrink for Zookeeper/Clickhousekeeper was added to the following articles:

Fixed a description of the Astra Linux SE 1.7 Orel operating system in the Software requirements article.

Added the Use ADB DDBoost plugin article.

Made changes:

Added the Work with arenadata_toolkit article.

In the Network requirements article, gave information about the need for adding ports to an exception list if firewall is used.

Added Release notes for the ADH release 3.2.4.1.b2.

Added new articles on ZooKeeper:

Added the HiveServer2 UI overview article.

In the ADH architecture article, updated the Hadoop ecosystem schema and its description.

Reflected changes in the ADQM release 23.8.2.7.1.b1:

Added the Integration between ADQM and Kafka article.

Reflected changes in the ADB release 6.26.0.53:

Described two ways of integration between ADB and Kafka:

In the Configure a cluster article, recommended to use Java 11 in case of custom installation. The same note was added to Software requirements.

In the Network requirements article, added information about ports used by ADB to ADB Connector.

Updated ADB known issues.

Reflected changes in the ADB Control release 4.6.4:

Updated ADB known issues.

Reflected changes in the ADH release 3.2.4.1.b1:

Added new articles:

In the Use impala-shell to connect to Impala article, described how to fix the ImportError: no module named pkg_resources error.

Reflected changes in the ADPS release 1.1.0.b1:

Added new articles:

The Ranger audit description was updated and moved from the ADH/ADS documentation to the ADPS article Ranger audit.

In the Enable SSL on ADPS cluster with Knox article, described how to clear the gateway.tls.key.alias value in ADCM 2.0.0.

In all articles where ADCM web interface is mentioned, updated all texts and figures according to ADCM 2.0.0:

Added Release notes for the ADCM release 2.0.0.

In the Installation article, described environment variables (that can be optionally defined when a container is created) and recommended to use PostgreSQL in production.

In the Set the ADCM URL section of the Installation article, updated information about setting ADCM URL.

Updated the ADCM API v2 usage examples article: changed parameters and updated response examples for some queries.

Added the ADCM Auditor role description to the ADCM role model article.

In the Users management article, described how to unblock users.

Added information about the system and status users to the Audit UI article.

In the ADCM API article, gave a link to the API v2 specification and added information that API v1 is deprecated.

In the following articles, described the new versioning format, which is used starting with ADCM 2.0.0:

In the Upgrade article, recommended to backup ADCM data before upgrade.

Restructured the Release notes article: all releases are now located at the top level.

In all articles where ADCM web interface is mentioned, updated all texts and figures according to ADCM 2.0.0:

Added the Kafka API usage example article.

In all articles where ADCM web interface is mentioned, updated all texts and figures according to ADCM 2.0.0:

Added new articles:

In the Enable SSL/Disable SSL article, described the need to restart Solr if the Ranger Solr plugin is enabled.

In all articles where ADCM web interface is mentioned, updated all texts and figures according to ADCM 2.0.0:

Added new articles:

In all articles where ADCM web interface is mentioned, updated all texts and figures according to ADCM 2.0.0:

In the Software requirements article, recommended to switch off any antivirus software before ADB installation and during the ADB operation process (see point 6).

In all articles where ADCM web interface is mentioned, updated all texts and figures according to ADCM 2.0.0:

Added new articles on Hive:

Added new article on HDFS:

Updated a description of Ranger plugins:

Reflected changes in the ADB Control release 4.5.3:

Reflected changes in the ADBM release 1.5.2:

In the Hardware requirements article, changed requirements for ADB hosts during ADBM installation.

Reflected changes in the ADB release 6.25.2.52:

In the ADB to ADB Connector overview article, extended the third point of the first note — about usage of INSERT.

In the Add hosts article, updated the second note — about the ability to use FQDN in host names.

In the Configuration parameters article, extended a description of the JAVA_HOME parameter for PXF (described system default paths).

Updated ADB known issues.

Moved the ADPS documentation to a separate repository and made it available via the following link: Arenadata Platform Security. This link is also located on the ADH and ADS landings in the Connected products → Arenadata Platform Security section (see the Read documentation button).

Moved the following ADPS articles from the ADH and ADS documentation (with updates):

Added new articles:

Due to moving the Arenadata Platform Security (ADPS) documentation to a separate repository, made the following changes:

Added new articles:

In the Filesystem requirements article, described the umask option (user file-creation mode mask).

Reflected changes in the ADH release 3.1.2.1.b1:

Added new articles on Spark:

Added the Use JDBC to connect to Hive article.

Added new articles on HDFS:

Described actions available for ADH clusters via ADCM:

Described actions available for ADH services via ADCM:

Added Filesystem requirements.

Updated a description of the dfs.internal.nameservices HDFS configuration parameter in the Configuration parameters article.

Reflected changes in the ADS release 2.8.1.1.b1:

Added examples how to use ADS services:

Described ZooKeeper concepts:

Added some references:

In the SSL channel security article, added a note about the need to fill in san when creating certificates. Updated the generate.sh script, which is used to create self-signed certificates (considering san).

In the Metrics offload section of the Configure job policies article, added a subsection with a structure of offloaded tables and conditions for their offloading.

Added the General settings article, which describes how to configure Time zone. This functionality is available starting with ADB Control 4.4.0.

Updated all articles according to the current version of ADB Control (4.4.0):

Updated all articles according to the current version of ADBM (1.4.0):

In the Concepts → WAL section of the Arenadata DB Backup Manager overview article, changed the incorrect size of WAL segments from 16 MB to 64 MB.

At the beginning of the Maintenance mode article, added a note about availability conditions of this mode.

In the Prerequisites article, fixed a description of minimal requirements.

Added Release notes for the ADCM release 2023.09.28.17.

Added articles:

Changed the following articles:

Reflected changes in the ADB release 6.25.1.51:

Described requirements for ADB installation:

Updated ADB known issues.

Added Release notes for the ADB Control release 4.4.0.

Updated ADB Control known issues.

Added Release notes for the ADB Control release 4.3.3.

Added new articles:

In the Audit article, added the Relation audit details section.

In the table that describes access permissions in ADB Control, added links to the documentation sections where users can find details of the corresponding operations.

Divided the Add a cluster section in the Cluster management article into two parts: automatically and manually. Added notes at the beginning of each section.

Reflected changes in the ADBM release 1.3.3:

At the beginning of the Manage configurations in ADBM article, added a note about the need to create a new configuration when switching to standby master.

Added Release notes for the ADB release 6.25.1.49.

Updated the ADB known issues article.

Replaced the Arenadata Streaming Command Center (ADSCC) name by Arenadata Streaming Control (ADS Control) in all texts.

In the UI overview article, updated the section related to cluster management: described filters and legend.

In the ADS Control overview article, described user management in a separate section.

In the following articles, updated images for both themes and added icons:

Added Release notes.

Released the first version of the updated ADQM Spark Connector documentation, which includes the following articles:

In comparison with the previous version:

Added the documentation link to the ADH and ADQM landings.

Added the ability to select a language like for other products (via the toggle in the top right corner). Translated existing article into Russian. All new articles will be published in two languages simultaneously.

Added the ability to select a version like for other products (via the Select version drop-down list in the left).

Updated existing articles according to the internal style guides.

Reflected changes in the ADQM release 23.3.2.37.1.b1:

Added the Configure logical clusters in the ADCM interface article.

In the table of the Network requirements article, added the 9010 port for ClickHouse and the 9444, 9281 ports for ClickHouse Keeper.

Added Release notes.

Updated the Disaster recovery article:

Updated the ADBM known issues article. Closed problems that were fixed in ADBM 1.3.2.

Added Release notes for the ADB Control release 4.3.2.

Added ADB Control known issues.

Added Release notes for the ADB release 6.24.3.48.

Added ADB known issues.

Added the Disaster recovery article.

In the Manage configurations in ADBM article, updated figures and descriptions/default values of parameters. For the Full Backup schedule, Differential backup schedule, Incremental backup schedule, Restore point creation schedule, and Cleanup schedule options from the General configuration section, described switches, which define whether to perform actions manually or on schedule. Rules for parameters that specify time and data amount values were moved from tables to one place before information on adding a configuration.

Updated the Run actions in ADBM article:

In the View backups in ADBM article, updated images and filters. Added the Delta flag description to the backup details.

In the View restores in ADBM article, updated images, described the Restore type field (with possible values Self-restore and Copy). In the section with detailed restore information, described the Processes, Restore type, Action parameters; updated the Flags field.

Updated the ADB ClickHouse Connector installation article. There is no more need to load packages manually.

At the beginning of the Tkhemali Connector 1.X installation article, added a note about the connector status.

In all articles of the ADB installation guide, updated images and names of services/components due to renaming ADCC to ADB Control and Tkhemali to ADB ClickHouse Connector.

In the Configuration parameters article, updated names of the ADB Control parameters.

Reflected changes in the ADS release 1.8.2.b1:

Added new articles on NiFi:

Described configuration best practices for Kafka and NiFi:

Added the KsqlDB usage example article.

Added the Kafka REST Proxy usage example article.

Added Release notes for the ADCM release 2023.07.10.17.

Extended the Bundle DSL article. Added a description of the optional parameter requires.

Updated the Configure article. Changed the structure of parameters and described settings for authentication policies (to counteract brute-forcing methods).

Added examples of using the APT manager (in addition to YUM) to the Prerequisites article.

In the Users management article, described the ability to reset the login attempts counters of other users.

Added the Integration with the bundle article, which contains sequence diagrams for bundle load and action launch.

Removed incorrect quotes from examples in the Configure ADCM to use an external database article.

Reflected changes in the ADPG release 14.3.4:

Added new articles on the ADPG data model:

Added new articles with data query examples to the How to → Data querying section:

Described Kerberos authentication:

Described the Enable Kerberos, Disable Kerberos, and Reconfig Kerberos actions in the Manage a cluster section of the article that describes ADQM cluster actions.

Described the purpose and usage details of the following coordination services:

Added the Work with Livy article.

Extended the planning guide:

Described how to add HDFS data directories.

Updated the ADH installation guide:

Reflected changes in the ADH release 2.1.10.b1:

Described monitoring metrics available for ADH services.

Added Release notes for the ADCM release 2023.06.14.16.

Added the Bundle DSL article.

Updated the Ansible modules and plugins article. Described adcm_change_maintenance_mode.

Added the Ansible inventory groups article.

Renamed the new ADQM/ClickHouse connector from Tkhemali connector 2.0 to ADB ClickHouse Connector in the following articles:

In documentation on the previous connector version Tkhemali 1.X, replaced the Tkh profile name by TKH. The profile name should be written in uppercase.

Added Release notes for one of the previous ADB Control versions — 3.1.3.

Updated configuration parameters according to the ADQM release 22.8.12.45:

Updated descriptions of the parameters in the Proxying and Enable SSL encryption articles.

In the Add services article, added the Clickhousekeeper service to the table with ADQM services.

Reflected changes in the ADB release 6.23.3.44 dated 27.04.2023:

Added information on the previos connector version Tkhemali 1.X. You can use the new connector version in the beta mode simultaneously with the previous version. In comparison with old documentation, the new Tkhemali 1.X documentaion was actualized, populated with examples, and divided into three sections:

Reflected changes in the ADBM release 1.2.1 dated 27.04.2023:

Added a note about limitations for adding a configuration. See the CAUTION block before the Add a new configuration section in the Manage configurations in ADBM article.

Reflected changes in the ADQM release 22.8.12.45:

Added new articles to How to → Data querying → Query types:

In the Add components section, added information that when installing Clickhouse JDBC Bridge (optionally), a user should install it on all hosts with the Clickhouse Server component.

Added Release notes for the ADCM release 2023.04.21.12.

Removed extra ALTER USER command from the Configure ADCM to use an external database article.

Added Release notes.

Added Supported services.

Updated the Supported services article.

Added the AD Kerberos common problems article (with a description of ways to resolve them).

Fixed a typo with the Solr name on the second figure in the ADH architecture article.

Reflected changes in the ADS release 1.8.1.b1:

In the ADS configuration parameters article, added the Basic Auth properties group of parameters for ksqlDB, Kafka REST Proxy, Schema-Registry and the MiNiFi Agent bootstrap.conf group of parameters for MiNiFi.

Added new articles on NiFi:

ADB installation guide (via ADCM):

Articles on the ADB data model:

Description of work with tables:

Overview of server configuration parameters (Global User Configuration, GUC): Configure via GUC.

Overview of ADB parameters that can be configured via ADCM: Configuration parameters.

Release notes divided by services:

Overview of the ADBM features, architecture, and main concepts: Arenadata DB Backup Manager overview.

Description of connecting to ADBM via web interface: Connect to ADBM.

Overview of configurations that are used to launch backups and other actions: Manage configurations in ADBM.

Description of cluster actions: Run actions in ADBM.

How to view backups and their details: View backups in ADBM.

How to view actions and their details: View actions in ADBM.

Reflected changes in the ADCM release 2023.03.27.20:

Added the Ansible modules and plugins article.

Added the Import/export of services article.

Described how to view ADCM API.

Added author names to all articles.

Reflected changes in the ADPG release 14.3.3 dated 16.03.2023:

Added the Logical replication article.

Added the Use cases article.

Added articles related to the ADPG data model:

Updated and populated with usage examples the following articles:

Updated and added examples to the Use tables and column aliases in queries article.

Removed the following articles:

Added author names to all articles.

ADQM planning guide:

ADQM installation guide (via ADCM):

Specific features of ADQM as a database.

ADQM architecture:

ADQM data model:

Possible connection ways:

Authentication ways:

Authorization based on the RBAC model.

Actions available for ADQM at the cluster and service levels via ADCM:

SSL encryption.

Configuration parameters.

Release notes.

Added Release notes for the ADH release 2.1.8.b3.

Added the Cluster upgrade article, which describes the minor/major upgrade principles.

Added articles on FreeIPA:

Translated and fixed articles from the Hadoop Command-line section (with subsections)

Added author names to all articles.

Fixed the following:

Reflected changes in the ADS release 1.8.0.b1 dated 02.02.2023:

Described NiFi authorization:

Extended the Cluster actions article. Described actions that become available after running other actions (Create/Update/Delete user, Manage Basic Auth users, and several Disable actions).

Restructured articles related to authorization:

Fixed the following:

Added the NiFi Server UI overview and NiFi Registry UI overview articles.

Added the NiFi service authentication article.

Reflected changes in the ADS release 1.7.2.b2 dated 10.01.2023:

Restructured articles related to authentication:

Added author names to all articles.

Added Release notes for the ADSCC release 1.0.0.b2 dated 10.01.2023.

Added author names to all articles.

Added the ability to select a language like for other products (via the toggle in the top right corner). Translated existing article into Russian. All new articles will be published in two languages simultaneously.

Added the ability to select a version like for other products (via the Select version drop-down list in the left).

Updated existing articles according to the internal style guides, fixed some articles based on the technical review.

Restructured the left navigation menu (TOC) and the product home page:

Added new materials:

Reflected changes in the ADPG release 14.3.2 dated 29.12.2022:

Added new articles related to data backups:

Removed the How to → Working with actions article. Instead of it, added new articles:

Reflected changes in the ADS release 1.7.2.b1 dated 06.12.2022:

Added new articles:

Reflected changes in the ADCM release 2022.12.07.10 dated 07.12.2022:

Added the Creating a simple bundle article.

Restructured Release notes for previous releases by moving some notes from New features to Improvements and Misc/Internal.

Added new articles:

Restructured and extended the Password authentication article.

Updated the left navigation menu structure (to reduce a number of elements at the top level):

Added links for new articles to the product home page.

Added the ability to select a product version via the Select version drop-down list in the left.

Reflected changes in the ADPG release dated 30.06.2022:

Restructured the How to → Recommended ADPG settings and References → postgresql.conf articles. Instead of them, left one article References → Configuration parameters, which includes all ADPG parameters that can be configured via ADCM.

Added Logging.

Updated the Requirements for SSL certificates article.

Updated figures in the Tutorials → Security section (sizes and dark mode effects).

Actualized Release Notes.

Replaced the hadoop name in the documentation links by ADH.