ADS releases

Fixed an issue with displaying the Grafana component control panel of the Prometheus service

Fixed HDFS import errors

Fixed a crash that occurred when running the Expand action for the ZooKeeper service

Fixed a NiFi configuration encryption error that occurred when working with сyrillic symbols on AltLinux sp 8.4

Fixed missing zk_keystore.pw in the NiFi configuration causing NiFi cluster to crash

Fixed the presence of Schema-Registry import after removing the service

Fixed 404 error related to NiFi Registry and Ranger when upgrading ADS cluster version 1.7.x

Fixed missing environment configuration for the Kafka REST Proxy service

Implemented support for Tiered Storage in the Kafka service

Implemented support for the Kafka Raft (KRaft) protocol for metadata management in Kafka

Implemented support for Debezium connectors in Kafka Connect for PostgreSQL and MS SQL servers

Implemented a new Monitoring service

Implemented ADH cluster import for automatic configuration of tiered storage

Fixed an issue with the password in the Initial Admin field when enabling authentication in the NiFi service

Fixed an issue with the Manage Kerberos action in an ADS cluster with NiFi and Monitoring clients services installed

Fixed the Error getting tags error in the NiFi and NiFi-Registry logs when Kerberos is enabled

Fixed an issue with updating NiFi SSL attributes after unsuccessful launch of the Enable SSL action

Important! The KRaft protocol for Kafka is provided as early access and is not recommended for production use. There is no possibility of migration from ZooKeeper to KRaft and back, only installation from scratch is possible

Working in the Kafka-Manager service is not available when using the KRaft protocol

Minimum ADCM version is 2.3.0

Minimum ADH version for HDFS import when setting up tiered storage is 3.3.6.2.b1

Implemented the kafka-rest-security plugin for authorization in the Kafka REST Proxy service

Changed the ZooKeeper service version to 3.8.4

Implemented support for Java 17

Added the ability to change /etc/kafka/conf/core-site.xml when enabling the Ranger plugin

Improved placement of parameters on the Kafka service configuration page

On the configuration page of the Kafka service, the appearance of the log.cleaner.enable Kafka parameter and its default value have been changed (false → true)

All user parameters entered using the Add key, value field will overwrite existing parameters

Fixed error when connecting LDAP groups and NiFi policies

Starting from the current version of ADS, the Java version used is 17. Before updating or installing ADS on Centos, RedHat, and AstraLinux operating systems, you should manually install Java 17 and also configure the JAVA_HOME variable

Implemented support for Samba DC

Implemented support for Ubuntu

Changed version of NiFi to 1.23.2

Added a new cluster action — Restart

Cluster actions Enable SSL and Disable SSL have been replaced with Manage SSL, which performs overall improved management of SSL channel security

The minimum ADS version that can be updated is 1.7

Added a new cluster configuration parameter Precheck packages that enables/disables package checks when performing actions Upgrade, Install, Expand. Checks are disabled by default

Added the ability to configure connection to JMX using login/password

Cluster actions Enable Kerberos, Disable Kerberos, and Reconfig Kerberos have been replaced with Manage Kerberos that performs general improved Kerberos management (enable, disable, configure)

Fixed issue with displaying incorrect information in NiFi configurations after enabling the Ranger plugin

Implemented support for RedOS in the Enterprise version of the ADS bundle

Added an additional GitFlow Provider to NiFi

Changed version of NiFi to 1.20

Implemented encryption of NiFi configuration parameters

Fixed a problem with displaying irrelevant data in the Kafka-Manager service

Maximum ADPS version — 1.0.4

Minimum ADCM version — 2023.10.10.08

Implemented support for Astra Linux in the Enterprise version of the ADS bundle

Reworked user authentication via SASL PLAINTEXT and Basic Auth to use secretmap entries in the ADCM interface

Implemented showing Confluent licenses in ADS when adding services during cluster installation

Implemented hostgroup settings for all services

Fixed issue with ansible_fqdn definition when triggering Expand and Shrink actions for Kafka Connect service

Fixed issue with bootstrapServers and kafkaConnectClusters export being overwritten after restarting Kafka and Kafka Connect services

Fixed issue with existing znode before creating ZooKeeper node

Fixed issue with deleting the Monitoring Clients service using the icon

Variables instead of hardcode for kafka-connect log4j

Changed version of ZooKeeper to 3.5.10

Changed version of Kafka Connect to 2.8.1

Added actions for Schema Registry service Expand and Shrink

Added installation of the Apache NiFi - MiNiFi - Toolkit at the same time as the MiNiFi Agent component. MiNiFi Toolkit is designed to create MiNiFi configuration files from exported NiFi templates

Added the ability to configure services for a group of hosts for MiNiFi service

Added the ability to configure authorization on the MiNiFi C2 Server for a group of hosts

Added logging level parameters to the components menu that can be changed during service components configuring

Auto restart of NiFi when authentication is being disabled (without using the Restart service action)

Implemented the ability to use the Kafka Connect service separately from the Kafka service

Added the Force rewrite services SSL configuration flag when starting Enable SSL action

Implemented support for the Hive 3 processor for the NiFi service

Implemented the connect-distributed-log4j.properties log level switch for the Kafka Connect Worker service component

Fixed an issue with ADS cluster data in ADSCC configuration when importing from ADS to ADSCC

Fixed a bug with NoNode ZooKeeper

Fixed Kafka Connect limit error in autotests

Fixed a bug when starting the Disable SSL action

Fixed an issue with using wrong ports for hostgroup configuration when triggering Enable SSL action

Fixed a bug in ZooKeeper when creating znode

Fixed an error with ZooKeeper installation

Fixed an cluster_info import overwrite issue when Kafka Connect Worker bean hostgroup is empty when running Shrink action for Kafka Connect service

Fixed a bug that occurred when installing more than one instance of Schema Registry

Fixed an Connection refused error in Kafka Manager health check after switching from curl tool to requests

Fixed issue with incorrect display of NiFi UI port overridden via configuration group for one host

Fixed a problem with creating a path when installing ZooKeeper

Fixed URL can’t contain control characters error in update autotests

Fixed an issue with restarting the NiFi-Registry when a configuration group for NiFi is created

Fixed an issue with starting Kafka after changing LOG_DIR via config group

Fixed an issue with starting Schema-Registry in tests when upgrading to version 1.7.X

Fixed an issue with disabling SASL PLAIN when SSL is enabled and NiFi is not installed

Fixed an issue with applying an SSL configuration change to a host

Fixed an issue with deleting multiple NiFi nodes

Removed double check when starting Enable SSL action

Changed name of KSQL service to ksqlDB

Disabled support for updating versions earlier than 1.4.11

Added the ability to delete a service from a cluster in the ADCM interface

Implemented support for Alt 8 SP in minifi.sh for NiFi service version 1.18

Fixed an issue with NiFi Server Ranger plugin activation when upgrading from 1.6.X, 1.7.0, and 1.7.X

Fixed an issue with exporting ADS cluster name to ADSCC cluster after upgrading ADSCC

Fixed an issue with updating basic information about services after applying actions Expand and Shrink

Added ability to clear user and authorizer files when enabling SSL in NiFi

Fixed an issue with creating service keytab

Fixed an issue with cluster crashing after misconfiguring NiFi settings when enabling SSL

Fixed an issue when enabling SSL in a cluster with NiFi service and monitoring without installing ADPS

For the NiFi Registry (a component of the NiFi service), Ranger authorization is implemented — access protection when storing and managing shared resources in one or more NiFi instances

Added the ability to manage all parameters using the ADCM user interface in all configuration files

Added support of AltLinux 8.4 operating system for ADS

For the NiFi service, the Ranger authorization plugin has been added, the ability to add or remove permissions for processing messages in NiFi has been implemented

Added support for Kafka Connect service and Mirror Maker 2 mechanism for ADS. For more information about Mirror Maker 2, see Mirror Maker 2

Fixed a bug when upgrading an ADS cluster with existing NiFi streams from 1.6.2.b4 to 1.7.0.b1

Added LDAP/AD authentication for NiFi service

For ADS in ADCM, the ability to delete a user is implemented

For the NiFi service, the ability to work with the _routing option using the NiFi Elasticsearch processor has been added

Switching of the logging level (log level) for ADS services is implemented

For ADS in ADCM, the ability to configure channel protection via the SSL protocol has been implemented

Kafka bump to new version stack

Fixed kdc_type setting error after update

Added the ability to use Active Directory as a Kerberos store for ADS

For the ZooKeeper service, the ability to add a set of logs for deletion (tear-down) has been implemented

Fixed a bug occurred when running the Reinstall status checker cluster action

Added full testing of the RedHat 7.9 volume license stack in ADS

Added Reinstall status checker action for ADS in ADCM

Implemented the ability to run status checker with passive components, such as client

Kafka topic data storage time not specified

Added autotests for installing ADS with Kerberos support by service

Added autotests for Upgrade with installed Kerberos

Updated status checker tests

Docker for ADS is released to use

Added a list of service log locations to the bundle configuration

The authentication protocol Kerberos is implemented for ADS

For ADS, a provisioning mechanism infrastructure has been created to enable Kerberos authentication

Fixed an issue with ADS and Enterprise Tools integration

For the ZooKeeper service, a bug with the Check action has been fixed — instead of performing a health check, it performed the installation and configuration of the service

Fixed a bug in starting offline installation on RH 7.9

Fixed incorrect execution of actions on the ZooKeeper service

Fixed offline installation error

For the NiFi service, incorrect work when passing metrics to the Graphite service of the Monitoring Clients cluster has been fixed

Fixed a bug with non-uniform hostnames for the Graphite service

Fixed a bug with duplicate configuration keys

Fixed intermittent validation error for Kafka service

For the NiFi service, an error has been fixed that occurred when the Shrink action was performed if the NiFi node did not have time to connect to the cluster

Fixed a bug for the NiFi service that occurred when the Expand action was performed when the wrong host was selected

For all services, a bug has been fixed: services did not change the state to Installed after being installed via the Install action

Changed ADS bundle format version

Disabled autostart for Shrink action

Changed maximum import version

Added branch parameter to .gitmodules

Integrated repository adcm_job_repo into ADS

Added the ability to update the bundle without rebuilding packages

Created a port map for ADS components

Created a pack-file for offline installation of CE with Enterprise Tools

Added autotests to check if Ranger works with Kerberos

Implemented SASL/PLAIN support for Kafka, KSQL, Schema-Registry, Kafka-Rest, Kafka-Manager services

Implemented the ability to add/update users for SASL/PLAIN

Implemented ADS integration with ADPS (Arenadata Platform Security)

Implemented support for Ranger Kafka Plugin

Implemented support for Service Dependency Tree

Implemented the Check service action to check services using the systemctl utility

Implemented the division of the ADS bundle into the Enterprise version and the Community version

Increased Kafka version to 2.6.0

Increased KSQL version to 0.11.0

ZooKeeper repository and Alt repository uploaded to Google

For the Schema-Registry service, the check has been improved

Improved test execution time and added new fixtures from the plugin

Improved Schema-Registry configuration options

Refactored Kafka Cluster post task in Kafka-Manager

Refactored adding NiFi Metric Report tasks

Refactored use of adcm_check as utils

Refactored the NiFi service health check

Refactored services_dependencies_check.yaml

Refactored adcm_role_jmxtrans

Refactored services

Refactored actions and states of the cluster and ADS services

Graphite and Grafana imports moved to Monitoring Clients service

Migrate Kafka-Manager to a Docker container

All service checks have become more informative

It is now possible to add parameters to the Schema-Registry service config

Fixed a problem with launching the Ranger plugin

Fixed issue with Monitoring service status after Ugrade action

Fixed issue with Kafka version support when upgrading from 1.4.X

Fixed a problem with starting the Schema-Registry service after the update

Fixed bug in Kafka-Manager

Fixed bugs in ansible-lint

Repository in config.yaml split with -r flag

ADS 1.6.0 uploaded to Google

Added Guava library to ranger-kafka-plugin assembly

Verified use of json_query

Added Monitoring_dependency_check tag to Expand action

Moved ADS autotests to SSH provider

For autotests, all file accesses are organized regardless of the current working directory

Added creation of clusters with a common function from the plugin in autotests

Refactored tests to use only scope fixtures

Added services_install_order to configuration

Updated tests for using the YCC 2.11 bundle

Added service dependency tree tests

Added processing of tests for different editions

Implemented build ZooKeeper 3.5.8

Implemented build Kafka 2.6.0

Implemented standardization of service checks

Implemented the assembly of the latest stable bundles for ADS 1.6.0

Refactored implementation of service dependencies

Added autotests of monitoring clients

Added MiNiFi autotests

Fixing bugs in offline mode for version 1.5.0.1

Preparing a bundle for offline installation 1.5.0

Creation and placement of offline bundles

Introduction of MiNiFi service for ADCM

Monitoring implemented for MiNiFi service

Added common actions for MiNiFi service for ADS in ADCM

Added support for Kafka 2.4.0 version in Kafka-Manager

Kafka-Manager downgraded to 1.3.3.23

Added the ability to monitor the NiFi service

Refactored Kafka-Manager user interface

Refactored ADS bundle for Altlinux

Refactored ZooKeeper role for Altlinux

Refactored service_main_info.yaml

Refactored Schema-Registry service

Refactored Kafka-Manager service

Refactored processing of cluster and service states

Changed the unit of Kafka metrics

Refactored Check action of Kafka service

Refactored calling the systemd role in the ZooKeeper service

Refactored actions of clusters and services for several types of scenarios

Refactored roles of repositories

Refactored NiFi Registry configuration for version 0.5.0

Refactored NiFi configuration for version 1.10.0

Refactored Kafka configuration for version 2.4.0

Refactored Kafka role

Refactored ZooKeeper role

Rebuilt MiNiFi package

Fixed a bug with building the Schema-registry service

Fixed a bug with creating custom NiFi service directories

Fixed a bug with installing the MiNiFi service (an Ansible task crashed)

Fixed issue with encoded template name in external link to MiNiFi C2 server

Fixed issue with performing Expand action for NiFi service

Fixed destructive actions when checking the repository

Fixed problem with updating child packages when updating main packages

Rebuilt ZooKeeper service package

Fixed Diamond collector issue for MiNiFi service

Fixed Run as user runtime issue for MiNiFi agent

Fixed Kafka-Manager service crash after version change

Fixed an issue with NiFi builds

Fixed issue — _verify_installation is not excluded when building bundles

Fixed LOG_DIR for all services

Changed the logic of loading the bundle version

For the ZooKeeper service, the Shrink and Expand service action tests have been Refactored when connected via an IP address

For the NiFi service, tests for the Shrink and Expand actions have been added

For the ZooKeeper service, tests for the Shrink and Expand actions have been added

Added check for Shrink and Expand results for ZooKeeper service

Added Kafka REST Proxy server update

Add a call to adcm_check after each service update task

A test for the operation of the Shrink action has been added for the Kafka service

Added adcm_check call after each task

Enable StatusLogger for MiNiFi Agent

For the MiNiFi service, a script has been developed to transfer metrics to Graphite

Added Upgrade action test for ADS 1.3.X and 1.4.X bundles

Moved service_rolling_restart.yaml to utils role

Added invisible configuration to config.yaml for computed service variable

Added update task for ZooKeeper in 3.5.6

Documents prepared for version 1.5.0.1

The latest stable packages for ADS 1.5.0 have been built

Migrated ADS to Altlinux

Added cluster upgrade tests

Added a default hidden parameter for the ZooKeeper repository to the service configuration

Added tests for Shrink and Expand actions for a cluster

Added NiFi dependency check for MiNiFi service

C2 MiNiFi server developed

Added restart tag to adcm-utils for task service_main_info.yaml

Added license acceptance in tests

Moved changes from branch-adsm-1.5-allinux to branch-adsm-1.5 for Centos and altlinux builds from the same branch

Updated license agreement