Removed the following extensions: hstore_plperl, hstore_plperlu, jsonb_plperl, and jsonb_plperlu.
Updated the version of the timescaledb extension.
Described the following extensions: plperl, plperlu, plpython3u, pltcl, pltclu.
Added a note with information on additional packages for procedural languages.
In the Configure a cluster article, updated the image and described the Cluster UUID parameter (see Primary configuration → General configuration parameters).
Updated the Network requirements article: described ports used in a monitoring cluster and divided the table with all used ports into several tabs — one tab per one product.
Moved the ADS Control documentation to a separate repository and made it available via the following link: Arenadata Streaming Control. This link is also located on the ADS landing in the Connected products section (see the Read documentation button).
In the Monitoring of sessions article, updated all images and described new table fields: Resource group, Waiting for session, Waiting reason.
In the Work with resource groups article, renamed the Commands section to Commands → Running and added the Commands → Queueing section.
In the Performance section of the Monitoring of commands article, updated images and described the ability to sort and filter data in the table with segment metrics.
In the Repository collapsible block of the Manage configurations in ADBM article, updated the figure and described new fields: Upload chunk size and Certificate verification.
In the Configure a cluster article, updated the last image and added the following sections: Configuration groups and Ansible settings.
In the Add components article, described the Hosts mode, which allows mapping components by hosts.
Instead of all articles in the installation guide related to hostproviders, left only one article Prepare hosts, which contains links to the detailed documentation on each hostprovider.
In the Configure a cluster article, updated the last image, actualized parameters, and added the following sections: Configuration groups and Ansible settings.
In the Add components article, described the Hosts mode, which allows mapping components by hosts.
Instead of all articles in the installation guide related to hostproviders, left only one article Prepare hosts, which contains links to the detailed documentation on each hostprovider.
ADS Control documentation release 19.08.2024
In the Cluster actions, described the Manage Kerberos and Manage SSL actions.
In the Configure a cluster article, updated the last image, actualized parameters, and added the following sections: Configuration groups and Ansible settings.
In the Add components article, described the Hosts mode, which allows mapping components by hosts.
Instead of all articles in the installation guide related to hostproviders, left only one article Prepare hosts, which contains links to the detailed documentation on each hostprovider.
ADPS documentation release 19.08.2024
In the Configure a cluster article, updated the last image, actualized parameters, and added the following sections: Configuration groups and Ansible settings.
In the Add components article, described the Hosts mode, which allows mapping components by hosts.
Instead of all articles in the installation guide related to hostproviders, left only one article Prepare hosts, which contains links to the detailed documentation on each hostprovider.
ADQM documentation release 19.08.2024
In the Configure a cluster article, added the following sections: Configuration groups and Ansible settings.
In the Add components article, described the Hosts mode, which allows mapping components by hosts.
Instead of all articles in the installation guide related to hostproviders, left only one article Prepare hosts, which contains links to the detailed documentation on each hostprovider.
ADB documentation release 19.08.2024
In the Add components article, described the Hosts mode, which allows mapping components by hosts.
Instead of all articles in the installation guide related to hostproviders, left only one article Prepare hosts, which contains links to the detailed documentation on each hostprovider.
In the Manage a Yandex Cloud hostprovider article, added information that the Create hosts action is not designed for products that require FQDNs for hosts created via ADCM.
Renamed the JAAS options for the SASL PLAIN mechanism list to JAAS options for the SASL PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512 mechanism.
In the Kafka options list, added new possible values for the following parameters: SECURITY_PROTOCOL — SSL, SASL_MECHANISM — SCRAM-SHA-256 and SCRAM-SHA-512.
Added a file example for SASL_MECHANISM = SCRAM-SHA-256|SCRAM-SHA-512 to the JAAS_CONFIG_FILE section.
In the Use the SERVER option section, added the kafka.property.ssl.keystore.location and kafka.property.ssl.keystore.password options to the kafka-site.xml parameters list.
Described the Use block incremental backup option in the Add a new configuration → General configuration section of the Manage configurations in ADBM article.
ADH documentation release 23.07.2024
Reflected changes in the ADH release 3.2.4.3.b1 dated 23.07.2024:
In the ADB Control → External ADQM parameters section, described the ability to use clustered ClickHouse.
In the ADB Control → External ADB Control server parameters section, added the External ADB Control port field and changed the ADB Control address description.
Removed the Listening port field from the ADB Control → HTTPS parameters enable section.
Actualized the Manage Ranger plugin action description for Kafka and NiFi. Added the Override service policies field description, gave links to related articles in ADPS, updated ADCM screenshots, as well as Ranger screenshots in the following articles:
Described the asynchronous_insert_log parameter in the ADQM log types table of the Logging article and in the ADQMDB → Log settings section of the Configuration parameters article.
Removed a description of the manual stream launch.
Described the Backup Manager → Copy → History tab.
Described the Restore points section of a form with deatiled stream information.
Described the Replication lag field, which is available for streams with the Streaming type in the Streaming status on the Backup Manager → Copy → Streams tab and in the window with detailed stream information.
Described the tags field for the query and transaction tables in the Metrics offload → Structure of offloaded data section of the Configure job policies article.
Described the Manage user sessions permission in the Authorization article.
Described the Session column in the Operations section of the Audit article.
Described the session column for the audit_operation table in the Metrics offload → Structure of offloaded data section of the Configure job policies article.
Described the Planchecker service option in the General settings article.
In the View jobs article, updated all images and removed the Status column description from the Jobs → Monitoring → Schedule and Jobs → Backup manager → Schedule tabs.
Actualized the following sections: ADB Control → Database parameters, ADB Control → External database parameters, ADB Control → ADQM database parameters, ADB Control → External ADQM parameters.
Added the Service password description to the ADB Control → UI component parameters section.
In the Metrics offload → Structure of offloaded data section of the Configure job policies article, added a note about choosing DB.
Added a note about upgrade to the Add a cluster section of the Cluster management article.
ADB documentation release 22.04.2024
Reflected changes in the ADB release 6.26.2.55 dated 11.04.2024:
In the Redistribute → Run redistribution section of the Cluster actions article:
Added a description of the Number of parallel processes field.
Updated the ADPG documentation home page. Instead of the full list of articles (which is still available in the left navigation menu), top 10 popular articles are displayed (based on the statisctics of the past month).
ADCM documentation release 18.04.2024
Updated the ADCM documentation home page. Instead of the full list of articles (which is still available in the left navigation menu), top 10 popular articles are displayed (based on the statisctics of the past month).
ADQM documentation release 18.04.2024
Updated the ADQM documentation home page. Instead of the full list of articles (which is still available in the left navigation menu), top 10 popular articles are displayed (based on the statisctics of the past month).
ADS documentation release 18.04.2024
Updated the ADS documentation home page. Instead of the full list of articles (which is still available in the left navigation menu), top 10 popular articles are displayed (based on the statisctics of the past month).
ADH documentation release 18.04.2024
Updated the ADH documentation home page. Instead of the full list of articles (which is still available in the left navigation menu), top 10 popular articles are displayed (based on the statisctics of the past month).
ADPS documentation release 18.04.2024
Updated the ADPS documentation home page. Instead of the full list of articles (which is still available in the left navigation menu), top 10 popular articles are displayed (based on the statisctics of the past month).
ADB Cloud Bundle documentation release 16.04.2024
Released the first version of the ADB Cloud Bundle documentation, which includes the following articles:
Added the SSM and Kyuubi services to the Add services article.
In the Add components article, added examples of mapping the SSM and Kyuubi components to hosts.
In the ADH architecture article, actualized the second schema and its description (see SSM and Kyuubi).
Due to the fact that ADQM Spark 3 Connector now comes with the ADH bundle, the connector documentation was moved to the ADH documentation. In articles, small fixes were made related to Spark 3.
In the PySpark article, a path to the Python3 interpreter was changed to /opt/pyspark3-python/.
Information about the Ranger Service Name and Override service policies options was added to the following articles:
In the ADB Control → Database parameters section of the Configuration parameters article, added a note for the Data dir field.
Updated the ADB documentation home page. Instead of the full list of articles (which is still available in the left navigation menu), top 10 popular articles are displayed (based on the statisctics of the past month).
Described the PROVIDER and HOST groups in the Basic groups section.
All groups were assigned a bundle type (see the last column of tables on the page).
Added the second example to the Examples section.
In the config → Types of configuration parameters section of the Bundle DSL article, actualized the structure description: added a list of available attributes and possible match values, updated the configuration example.
In the ADPG → Patroni ADPG configurations section of the Configuration parameters article, described the use_custom_location, patroni_log_dir, patroni_log_file_size, patroni_log_file_num, and patroni_log_level parameters.
In the Service actions, described actions available for Clickhousekeeper.
Information about the need for applying Reconfig and restart to ADQMDB after running Expand/Shrink for Zookeeper/Clickhousekeeper was added to the following articles:
Added the Enable secure intra-cluster communication parameter to the Intra-cluster communication security section.
Updated descriptions of parameters in the Cache section (ex. Local cache).
In the JDBC Bridge configuration section, described jdbc bridge log path.
Removed the Specify a directory to store ZooKeeper log files article since in the new ADQM release the described actions are not needed. The article is still available for the previous ADQM versions.
In the Software requirements article, added specific requirements for Astra Linux related to cgroups v1.
In the Configure services article, updated the last figure since the PL/Python3u Extension parameter was added.
In the Configuration parameters article, described the PL/Python3u Extension parameter (see the Main section). Also, the Use nobarrier parameter was described (for master and segments separately).
Renamed the Metrics chart article to Dashboard and moved it to the lower level. All images in the article were updated.
In the SSL encryption article, updated images, actualized parameters in the HTTPS parameters enable section, and added some notes. Also, added new parameters to the Configuration parameters article (see ADB Control → HTTPS parameters).
At the beginning of the Cluster management article, added a note about the need for upgrading external ADB cluster in case of using ADB Control starting with 4.6.4.
In the Monitoring of sessions article, updated all images due to adding the Application name column and some fixes of the light theme in UI. The Application name description was added to the table.
In the Configure role mappings with LDAP groups section of the Authorization article, updated all images due to adding buttons and filters above the table in UI.
In the Metrics offload → Configuration section of the Configure job policies article, updated all images where the Export sessions flag is displayed.
In the Metrics offload → Structure of offloaded data section of the Configure job policies article, updated the offload condition for the session table due to adding the Export sessions flag.
In the Basic authentication article, updated all images since block and unblock icons replaced each other.
Added the Draft configurations section to the Manage configurations in ADBM article, which describes the new backup status Draft. In the Add a new configuration section, replaced the Creating status by Draft.
Actualized Hive configuration parameters: removed the External database section, added the javax.jdo.option.ConnectionDriverName parameter to the hive-site.xml section, added the default value for the javax.jdo.option.ConnectionURL parameter.
Actualized HDFS configuration parameters: renamed the HDFS heap memory settings section to Hadoop options and updated names and descriptions of its parameters.
Updated screenshots of the Airflow 2 web interface in the Airflow UI overview article.
In the Software requirements article, added information about support for Astra Linux 1.7 "Orel" SE with Axiom JDK (with the need for LC_TIME=en_US.utf8), fixed requirements for package managers, and gave a link to Version compatibility matrix.
In the description of YARN configuration parameters, said about the need to set min.user.id for correct YARN operation under Astra Linux (see container-executor.cfg).
In the Software requirements article, added information about support for Astra Linux 1.7 "Orel" SE with Axiom JDK (with the need for LC_TIME=en_US.utf8).
Actualized Ranger configuration parameters: removed the External database section, added the ranger.ks.jpa.jdbc.driver parameter to the dbks-site.xml section.
In the Installation article, described environment variables (that can be optionally defined when a container is created) and recommended to use PostgreSQL in production.
In the Set the ADCM URL section of the Installation article, updated information about setting ADCM URL.
Updated the ADCM API v2 usage examples article: changed parameters and updated response examples for some queries.
Added the ADCM Auditor role description to the ADCM role model article.
In the Users management article, described how to unblock users.
Added information about the system and status users to the Audit UI article.
In the ADCM API article, gave a link to the API v2 specification and added information that API v1 is deprecated.
In the following articles, described the new versioning format, which is used starting with ADCM 2.0.0:
In the Software requirements article, recommended to switch off any antivirus software before ADB installation and during the ADB operation process (see point 6).
In all articles where ADCM web interface is mentioned, updated all texts and figures according to ADCM 2.0.0:
In the Configure job policies article, described the session table and conditions for offloading its data. Updated images (as the UI page now displays information about the need to select a database for data offloading).
In the Audit article, described the session object type.
In the Cluster management article, removed information about adding a cluster manually; moved a description of cluster settings to the section that describes how to edit a cluster; updated all images (since the Create connection button was removed).
In the ADB Control section of the View jobs article, updated images since ADCC was replaced by Monitoring in breadcrumbs.
Updated figures in most articles where the light theme of the ADB Control UI was displayed.
Moved the ADPS documentation to a separate repository and made it available via the following link: Arenadata Platform Security. This link is also located on the ADH and ADS landings in the Connected products → Arenadata Platform Security section (see the Read documentation button).
Moved the following ADPS articles from the ADH and ADS documentation (with updates):
In the section related to authorization in ADH, left only one ADPS overview article — Arenadata Platform Security overview. This article contains links to the ADPS documentation.
Removed other articles that were added to the ADPS documentation.
In the section related to authorization in ADS, left only one ADPS overview article — Arenadata Platform Security overview. This article contains links to the ADPS documentation.
Removed other articles that were added to the ADPS documentation.
Due to the fact that ADB Spark 3 Connector now comes with the ADH bundle, the connector documentation was moved to the ADH documentation. The ADB Spark 3 Connector usage examples article was completely rewritten: prerequisites were described, all commands were divided into two parts (command and output). In other articles, small fixes were made related to Spark 3.
In the Cluster upgrade article, fixed a description of the bundle naming scheme.
In the SSL channel security article, added a note about the need to fill in san when creating certificates. Updated the generate.sh script, which is used to create self-signed certificates (considering san).
ADCM documentation release 11.10.2023
Added Release notes for the ADCM release 2023.10.10.08.
ADB Control documentation release 04.10.2023
In the Metrics offload section of the Configure job policies article, added a subsection with a structure of offloaded tables and conditions for their offloading.
Added the General settings article, which describes how to configure Time zone. This functionality is available starting with ADB Control 4.4.0.
Updated all articles according to the current version of ADB Control (4.4.0):
Updated all screenshots due to changes in the left navigation menu and adding a timezone in the top right corner of ADB Control UI. The light theme of the documentation now shows screenshots corresponding to the light theme of the application.
Updated a description of table filters in the following articles:
View jobs: removed the Error message column description from the both Audit tabs since its functionality was moved to the Job name column.
ADBM documentation release 04.10.2023
Updated all articles according to the current version of ADBM (1.4.0):
Updated all screenshots due to changes in the left navigation menu and adding a timezone in the top right corner of ADBM UI. The light theme of the documentation now shows screenshots corresponding to the light theme of the application.
Updated a description of table filters in the following articles:
Described the Autostart, Backup type options and a note related to them in the Disaster recovery article (see step 6 in the Configure and run section).
Described the Action ID field in the Subaction details table of the View actions in ADBM article.
In the Fields of the Overview section table of the View restores in ADBM article, described the ability to head to the backup configuration details via a link. Changed the order of the Databases and Restore types columns.
In the table that describes access permissions in ADB Control, added links to the documentation sections where users can find details of the corresponding operations.
Divided the Add a cluster section in the Cluster management article into two parts: automatically and manually. Added notes at the beginning of each section.
Updated the image that describes the Stream Info UI window.
Updated the ADBM known issues article. Closed problems that were fixed in ADBM 1.3.3.
At the beginning of the Manage configurations in ADBM article, added a note about the need to create a new configuration when switching to standby master.
Added the ability to select a language like for other products (via the toggle in the top right corner). Translated existing article into Russian. All new articles will be published in two languages simultaneously.
Added the ability to select a version like for other products (via the Select version drop-down list in the left).
Instead of the Storage Configuration parameter, added new sections: Enable additional local storage, Enable S3 storage, Enable local cache, Enable encrypted storage, and Storage policies.
Added the following parameters to the User password options section: Allow Implicit_NO_password, Length of password, Include Upper-case, Include Lower-case, Include Number, Include special symbols.
In the Manage configurations in ADBM article, updated figures and descriptions/default values of parameters. For the Full Backup schedule, Differential backup schedule, Incremental backup schedule, Restore point creation schedule, and Cleanup schedule options from the General configuration section, described switches, which define whether to perform actions manually or on schedule. Rules for parameters that specify time and data amount values were moved from tables to one place before information on adding a configuration.
For the Restore action, described the Processes parameter and the Delta, Force, Skip Validation flags.
Changed names of actions (e.g. Start Backup. Start → Start 'Backup' action and so on).
In the View backups in ADBM article, updated images and filters. Added the Delta flag description to the backup details.
In the View restores in ADBM article, updated images, described the Restore type field (with possible values Self-restore and Copy). In the section with detailed restore information, described the Processes, Restore type, Action parameters; updated the Flags field.
In all articles of the ADB installation guide, updated images and names of services/components due to renaming ADCC to ADB Control and Tkhemali to ADB ClickHouse Connector.
Updated a description of the Basic and SASL PLAINTEXT authentication:
In the Configure a cluster article, added a block with images that explain how to configure different types of authentication (as well as for offline).
Added Release notes for the ADCM release 2023.07.10.17.
Extended the Bundle DSL article. Added a description of the optional parameter requires.
Updated the Configure article. Changed the structure of parameters and described settings for authentication policies (to counteract brute-forcing methods).
Added examples of using the APT manager (in addition to YUM) to the Prerequisites article.
In the Users management article, described the ability to reset the login attempts counters of other users.
Added the Integration with the bundle article, which contains sequence diagrams for bundle load and action launch.
Described the Enable Kerberos, Disable Kerberos, and Reconfig Kerberos actions in the Manage a cluster section of the article that describes ADQM cluster actions.
In documentation on the previous connector version Tkhemali 1.X, replaced the Tkh profile name by TKH. The profile name should be written in uppercase.
Added Release notes for one of the previous ADB Control versions — 3.1.3.
Added information on the previos connector version Tkhemali 1.X. You can use the new connector version in the beta mode simultaneously with the previous version. In comparison with old documentation, the new Tkhemali 1.X documentaion was actualized, populated with examples, and divided into three sections:
Added a note about limitations for adding a configuration. See the CAUTION block before the Add a new configuration section in the Manage configurations in ADBM article.
In the Add components section, added information that when installing Clickhouse JDBC Bridge (optionally), a user should install it on all hosts with the Clickhouse Server component.
ADCM documentation release 24.04.2023
Added Release notes for the ADCM release 2023.04.21.12.
In the ADS configuration parameters article, added the Basic Auth properties group of parameters for ksqlDB, Kafka REST Proxy, Schema-Registry and the MiNiFi Agent bootstrap.conf group of parameters for MiNiFi.
In the Add services and Add components articles of the ADPG installation guide, described the Balancer service.
Updated the section that describes ADPG cluster and service actions: ADPG cluster management. Marked actions available for Balancer; removed actions that were previously used to install the contrib and PostGIS extensions (currently preinstalled in ADPG).
Extended the Cluster actions article. Described actions that become available after running other actions (Create/Update/Delete user, Manage Basic Auth users, and several Disable actions).
Restructured articles related to authorization:
Moved the common part (related to the authorization and audit via ADPS) from How to → Kafka → Access management → Authorization → ADPS to Tutorials → Access management → Authorization → ADPS. This part includes the following articles:
In the How to → Kafka → Access management → Authorization and How to → NiFi → Access management → Authorization sections, left descriptions that are specific for the Kafka and NiFi services accordingly. Linked these descriptions with common articles mentioned above.
Fixed the following:
Corrected the Shrink description for the MiNiFi service (in the DN description).
In the Mirror Maker 2 article, fixed an error with the /bin duplicate in one of the paths used.
Moved the common part (related to the authentcation configuration via ADCM) from How to → Kafka → Access management → Authentication to Tutorials → Access management → Authentication. This part includes the following articles:
In the How to → Kafka → Access management → Authentication and How to → NiFi → Access management → Authentication sections, left descriptions that are specific for the Kafka and NiFi services accordingly. Linked these descriptions with common articles mentioned above.
Added author names to all articles.
ADSCC documentation release 27.01.2023
Added Release notes for the ADSCC release 1.0.0.b2 dated 10.01.2023.
Added author names to all articles.
ADH documentation release 25.01.2023
Added the ability to select a language like for other products (via the toggle in the top right corner). Translated existing article into Russian. All new articles will be published in two languages simultaneously.
Added the ability to select a version like for other products (via the Select version drop-down list in the left).
Updated existing articles according to the internal style guides, fixed some articles based on the technical review.
Restructured the left navigation menu (TOC) and the product home page:
Expanded article titles so that users could clearly understand what services they work with when navigating from the search results. Removed duplicate titles, such as Example, How it works, etc.
Standardized names of articles in the left menu, on the home page, in article titles, and in cross links.
On the product home page, extended a number of articles for quick access. The sections are named according to other products.
In the left navigation menu, moved all articles related to concepts (architectural schemas, work principles) from How to to Concepts.
Added intermediate sections to the How to guide. Previously, all articles were located at the top level.
In the Concepts section of the left menu, stopped to divide ADH services into core and other. All services are located at the top level like in How to. All architectural schemas were moved to ADH architecture.
Added new materials:
In the Release notes article, described all ADH releases starting with 2.1.7.b1.
In the Solr collections article, updated commands that are used to work with Solr collections. Described two ways to run commands: via the /usr/lib/solr/bin/solr script and the solrctl utility.
Moved configuration parameters of all ADH services to a single article Configuration parameters like in other products. Added parameters implemented in the latest ADH releases.
ADPG documentation release 29.12.2022
Reflected changes in the ADPG release 14.3.2 dated 29.12.2022:
Described the high availability (HA) mode based on Patroni for the Enterprise edition of ADPG: ADPG high availability overview.
Described offline installation of ADPG, which is available in the Enterprise edition of ADPG: Offline installation (with subsections).
In the ADPG installation guide, updated the articles that describe how to add services, map components, and install a cluster: described the Etcd service, updated the example of mapping components, updated final service statuses.
In the ADPG installation guide, updated articles related to monitoring (online, offline): removed the Monitoring Clients service; described the Enable monitoring option and the Install monitoring agents cluster action. The updated guide on the monitoring configuration is available in the following articles: online, offline.
Updated the architectural schema in the ADPG architecture article. On the schema, new components related to HA were highlighted.
Updated options in the Configuration parameters article. Added parameters related to HA (see ADPG → Patroni ADPG configurations).
Removed the How to → Working with actions article. Instead of it, added new articles:
Cluster actions — description of actions available at the cluster level.
Service actions — with subsections that describe service actions (one action per one page).
ADCM documentation release 29.12.2022
Added Release notes for the ADCM release 2022.12.26.18 dated 26.12.2022.
ADSCC documentation release 26.12.2022
Released the first version of the Arenadata Streaming Command Center (ADSCC) documentation, which describes the ADSCC release 1.0.0.b1 dated 23.12.2022:
Reflected changes in the ADCM release 2022.12.07.10 dated 07.12.2022:
Added the Audit UI article, which describes two new tabs in the ADCM interface (Audit operations and Audit logins).
Moved the Maintenance mode article from Concepts → ADCM objects → Host → Maintenance mode to How to → Managing ADCM → Maintenance mode. Also, added information about MM for ADCM services and components.
In the UI overview article, described how to load logs on the JOBS tab.
Restructured the How to → Recommended ADPG settings and References → postgresql.conf articles. Instead of them, left one article References → Configuration parameters, which includes all ADPG parameters that can be configured via ADCM.