ADS Control configuration parameters

Olga Semme

This article describes the parameters that can be configured for ADS Control service via ADCM. To read about the configuration process, refer to the Configure services article.

NOTE
Some of the parameters become visible in the ADCM UI after the Show advanced flag being set.

The configuration descriptions use the concepts of the Eclipse Vert.x framework:

  • Verticle — a piece of code that is an event loop which can be deployed in the Vert.x environment.

  • Worker verticle — a verticle that runs using a thread from the pool of already created Vert.x worker threads, rather than using a new event loop.

General

 

Parameter Description Default value

server.http.port

Port to connect to ADS Control

8888

server.vertx.pools.eventLoopPoolSize

Maximum number of Event Loop verticles in the pool

8

server.vertx.pools.workerPoolSize

Maximum number of working verticles in the pool

20

server.vertx.config.clusterResponseTimeoutMs

Time interval to request getting metrics from Kafka cluster (in ms)

500

server.vertx.event-bus.requestTimeoutMs

Time interval for requesting the Event bus of the Vert.x server (in ms)

30000

server.vertx.event-bus.consumer.circuitBreaker.maxRetries

Maximum number of attempts for consumer requests on the Event bus before the connection is terminated by the circuit breaker pattern

3

server.vertx.event-bus.consumer.deliveryOptions.timeout

Time interval of parameters for delivering consumer requests to the Event bus

7200000

server.vertx.event-bus.producer.circuitBreaker.maxRetries

Maximum number of attempts to request manufacturer requests via the Event bus before the connection is terminated by the circuit breaker pattern

0

server.vertx.event-bus.producer.deliveryOptions.timeout

Time interval of parameters for delivery of manufacturer requests to the Event bus

7200000

server.file-extensions

Possible server file extensions

json,js,css,ico,png

server.session-consumer-cache.enabled

Enables caching for consumer sessions

true

server.session-consumer-cache.capacity

Consumer session cache data size

1000

server.session-consumer-cache.expireTime

Consumer session cache expiration time

20000

server.session-consumer-cache.clearDelay

Consumer session cache clear delay time

10000

server.producer-cache.capacity

Producers cache data size

1000

server.producer-cache.expireTime

Producers data cache expiration time

60000

server.streaming.consumer.group.idPrefix

Prefix that is added to the Kafka consumer group identifier

adscc

server.streaming.consumer.group.inactivityTimeout

Amount of time a Kafka consumer group can remain inactive (i.e. with no consumer signals) before the broker initiates a rebalance (in ms)

600000

server.streaming.consumer.group.inactiveGroupsCheckInterval

Interval between consumer group activity checks (in ms)

120000

server.streaming.consumer.messagePollTimeout

Timeout between polls for new messages from the Kafka server (in ms)

1000

server.streaming.consumer.maxPollRecords

The maximum number of records the consumer will retrieve in a single poll call

10
Kafka Workers

 

Parameter Description Default value

server.vertx.verticle.kafka-commit-offset-info-producer.instances

Number of instances of kafka-commit-offset-info-producer verticles that receive data about the last offset of the __consumer_offsets topic (data about the last message read by the consumer

1

server.vertx.verticle.application-state-verticle.instances

Number of application-state verticle instances collecting application state data

1

server.vertx.verticle.application-state-verticle.updatePeriodMilliseconds

Period between requests for application state data (in ms)

5000

server.vertx.verticle.application-state-verticle.updatePeriodOffsetsMilliseconds

Period between requests for data on the latest offsets (in ms)

30000
Kafka Consumer

 

Parameter Description Default value

server.vertx.kafka.lastOffsetPollTimeoutMs

Timeout of the last poll of the topic offset __consumer_offsets (data about the last message read by the consumer) (in ms)

200
Kafka Settings

 

Parameter Description Default value

server.kafka.timeout

Timeout waiting for a response to a request to the Kafka server

300000

server.kafka.describeTopicsTimeout

Timeout waiting for a response to a request for a topic description to the Kafka server

10000

server.kafka.messagePollTimeout

Timeout between polling new messages from the Kafka server (in ms)

2000

server.kafka.systemGroupId

GroupId for connecting to Kafka server

adscc
Basic authentication

 

Parameter Description Default value

Admin Password

ADS Control administrator password

 — 

Basic authentication users

Form for filling out data for new users:

  • Enter field name — enter the user name;

  • Enter secret — enter a password for the user;

  • Confirm — password confirmation.

For more information, see Manage users

 — 
LDAP Authentication

 

Parameter Description Default value

LDAP URL

LDAP server URL to enable user synchronization during authentication

 — 

Authentication Query

Query to find users in LDAP directory to enable user synchronization during authentication

 — 
Logging

 

    The logback_template parameter defines the log template.

    Default value:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <property name="LOG_PATH" value="logs/app.log"/>

    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n
            </pattern>
        </encoder>
    </appender>

    <appender name="FILE-ROLLING" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${LOG_PATH}</file>

        <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
            <fileNamePattern>logs/archived/app.%d{yyyy-MM-dd}.%i.log.gz</fileNamePattern>
            <!-- each archived file, size max 10MB -->
            <maxFileSize>10MB</maxFileSize>
            <!-- total size of all archive files, if total size > 20GB, it will delete old archived file -->
            <totalSizeCap>20GB</totalSizeCap>
            <!-- 60 days to keep -->
            <maxHistory>60</maxHistory>
        </rollingPolicy>

        <encoder>
            <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
        </encoder>
    </appender>
    <root level="INFO">
        <appender-ref ref="STDOUT"/>
    </root>
    <logger name="io.arenadata.adscc" level="DEBUG">
        <appender-ref ref="FILE-ROLLING"/>
    </logger>
    <logger name="io.arenadata.adscc" level="DEBUG">
        <appender-ref ref="STDOUT"/>
    </logger>
    <logger name="org.apache.kafka" level="ERROR">
        <appender-ref ref="STDOUT"/>
    </logger>
</configuration>
Monitoring

 

Parameter Description Default value

server.monitoring.openTelemetry.enable

Enables the monitoring system "OpenTelemetry"

ON

server.monitoring.prometheus.enable

Enables the monitoring system "Prometheus"

ON
Client

 

Parameter Description Default value

client.http.webClientOptions.maxPoolSize

Maximum pool size for connections

100

client.http.webClientOptions.verifyHost

Enables client host verification

false

client.http.circuitbreaker.maxFailures

Number of failures at which the Circuit Breaker pattern disables the execution of the operation

3

client.http.circuitbreaker.timeout

Value above which the request increases the value of the number of failures (in ms)

500

client.http.circuitbreaker.fallbackOnFailure

Sets whether or not the fallback is executed on failure, even when the circuit is closed

false

client.http.circuitbreaker.resetTimeout

Time after which the circuit breaker pattern will try to close the circuit again (in ms)

100
Kafka Clusters

 

    After clicking Set, the clusters field appears for integration with external Kafka Clusters.

Group Parameter Description

 — 

name

Kafka cluster name

 — 

bootstrapservers

Bootstrap Kafka cluster servers

 — 

schemaRegistryUrl

URL to connect to Schema Registry

 — 

jmxClientProperties

JMX port

kafkaConnectClusters

name

Kafka Connect cluster name

url

URL to connect to Kafka Connect

jmxClientProperties

JMX port
JAAS template file

 

    The custom jaas.conf file template is intended for specifying user data for connecting clients of other services to the current service (paths to keytab files, the useTicketCache parameter and others). For more information, see Configure a custom jaas.conf.

    Default value:

{% if cluster.config.kerberos_client and cluster.config.kerberos_client.enable_kerberos %}
KafkaClient {
    com.sun.security.auth.module.Krb5LoginModule required
    useKeyTab=true
    storeKey=true
    useTicketCache=false
    serviceName="kafka"
    keyTab="{{ cluster.config.kerberos_client.keytab_dir }}/adscc.service.keytab"
    principal="adscc/{{ ansible_fqdn }}@{{ cluster.config.kerberos_client.realm }}";
};
{%- endif %}
Advanced docker settings

 

Parameter Description Default value

Install docker package

Enables installation of the Docker package when installing the service

true

Disable seccomp

Disables the seccomp mechanism when installing a service

false
Parameter Description Default value

ADS Control Ranger plugin enabled

Indicates whether Ranger ADS Control plugin is enabled (auto-populated)

false
ranger-adscc-security.xml

 
Apache Ranger parameters

Parameter Description Default value

ranger.plugin.adscc.policy.rest.url

URL to Ranger Admin

 — 

ranger.plugin.adscc.service.name

Name of the Ranger Service containing policies for this ADS Control instance

 — 

ranger.plugin.adscc.policy.rest.ssl.config.file

Location of the file containing SSL data for connecting to Ranger Admin

 — 

ranger.plugin.adscc.policy.cache.dir

Directory where Ranger policies are cached after successful retrieval from the source

/tmp

ranger.plugin.adscc.policy.pollIntervalMs

How often to poll for changes in policies

 — 

ranger.plugin.adscc.policy.rest.client.connection.timeoutMs

ADS Control plugin RangerRestClient connection timeout in milliseconds

 — 

ranger.plugin.adscc.policy.rest.client.read.timeoutMs

ADS Control plugin RangerRestClient read timeout in milliseconds

 — 

ranger.plugin.adscc.use.rangerGroups

Enables the use of group information when evaluating user permissions

true

Add key, value

Parameters and their values ​​entered in this field override the parameters specified in the ADCM user interface. This field also allows you to set values ​​for all user parameters that are not displayed in the interface, but are allowed in the ranger-adscc-security.xml configuration file

 — 
ranger-adscc-audit.xml

 

Parameter Description Default value

xasecure.audit.is.enabled

Enables audit for Ranger plugins

true

xasecure.audit.destination.solr

Enables audit in Solr for Ranger plugins

true

xasecure.audit.destination.solr.urls

Defines Solr URL

 — 

xasecure.audit.destination.solr.batch.filespool.dir

Directory for Solr audit spool

/tmp

Add key, value

Parameters and their values ​​entered in this field override the parameters specified in the ADCM user interface. This field also allows you to set values ​​for all user parameters that are not displayed in the interface, but are allowed in the ranger-adscc-audit.xml configuration file

 — 
ranger-policymgr-ssl.xml

 

Parameter Description Default value

xasecure.policymgr.clientssl.keystore

Location of the keystore file

 — 

xasecure.policymgr.clientssl.keystore.password

Keystore password

 — 

xasecure.policymgr.clientssl.truststore

Location of the truststore file

 — 

xasecure.policymgr.clientssl.truststore.password

Truststore password

 — 

xasecure.policymgr.clientssl.keystore.credential.file

Location of the keystore credential file

/etc/adscc/certs/jceks/keystore.jceks

xasecure.policymgr.clientssl.truststore.credential.file

Location of the truststore credential file

/etc/adscc/certs/jceks/truststore.jceks

Add key, value

Parameters and their values ​​entered in this field override the parameters specified in the ADCM user interface. This field also allows you to set values ​​for all user parameters that are not displayed in the interface, but are allowed in the ranger-policymgr-ssl.xml configuration file

 — 
Found a mistake? Seleсt text and press Ctrl+Enter to report it