Configuration parameters

This article describes the parameters that can be configured for ADQM services via ADCM. To read about the configuring process, refer to the relevant article: Online installation or Offline installation.

NOTE
Some of the parameters become visible in the ADCM UI after the Advanced flag being set.

ADQMDB

SSL
NOTE
  • To access the configuration parameters, activate the Enable SSL option.

  • For more information on SSL encryption, see the Enable SSL encryption article.

Parameter Description Default value

HTTPS port

Port for connections via HTTPS

8443

TCP secure port

TCP port for SSL-encrypted connections

9440

Disable protocols

Protocols that are not allowed to be used

 — 

Certificate file

Path to the server SSL certificate file in the PEM format

/etc/clickhouse-server/server.crt

Private key file

Path to the file with the private key of the server SSL certificate

/etc/clickhouse-server/server.key

Certificate authority file

Path to the CA certificate file

/etc/clickhouse-server/ca.crt

Validate client certificate

Enables user authentication by SSL certificates signed by the CA certificate (the path to the CA certificate file should be specified in Certificate authority file)

Disabled

LDAP
NOTE
  • To access the configuration parameters, activate the Enable LDAP option.

  • For details on how a remote LDAP server can be used to authenticate users, refer to the LDAP authentication article.

Parameter Description Default value

LDAP hostname

LDAP server hostname or IP. This parameter is mandatory and cannot be empty

localhost

LDAP port

LDAP server port. Default is 636 if Enable TLS is set to yes, otherwise — 389

636

DN prefix

Prefix used to construct the DN to bind to. Effectively, the resulting DN will be constructed as the auth_dn_prefix + escape(user_name) + auth_dn_suffix string

uid=

DN suffix

Suffix used to construct the DN to bind to. Effectively, the resulting DN will be constructed as the auth_dn_prefix + escape(user_name) + auth_dn_suffix string. Note, that this implies that auth_dn_suffix should usually have comma as its first non-space character

,ou=users,dc=example,dc=com

Enable TLS

Flag to trigger the use of secure connection to the LDAP server.

The following values are available:

  • no — for the plain text protocol (ldap://), not recommended;

  • yes — for the SSL/TLS protocol (ldaps://), recommended;

  • starttls — for the StartTLS legacy protocol (the plain text protocol, upgraded to TLS).

no

TLS min version

Minimum version of the SSL/TLS protocol. Accepted values are: ssl2, ssl3, tls1.0, tls1.1, tls1.2

tls1.2

TLS require cert

SSL/TLS peer certificate verification behavior. Accepted values are: never, allow, try, demand

demand

TLS cert file

Path to the certificate file

/path/to/tls_cert_file

TLS key file

Path to the certificate key file

/path/to/tls_key_file

CA cert file

Path to the CA certificate file

/path/to/tls_ca_cert_file

path to CA certificates

Path to the directory containing CA certificates

/path/to/tls_ca_cert_dir

Cipher suite

Allowed cipher suite

 — 

Enable RBAC

Activates role-based access control

Disabled

Default role

Default role for all authenticated users

default

Base DN

Base DN to look for in LDAP tree

ou=Groups,dc=ad,dc=ranger-test

Search filter

Search filter to look for in LDAP tree with the {user_name}, {bind_dn}, and {user_dn} patterns

(&(objectClass=groupOfNames)(member={bind_dn}))

Attribute

Attribute to look for in LDAP tree

cn

Role prefix

Attribute value prefix to look for in LDAP tree

adqm_

Kafka engine
NOTE

To access the configuration parameters, activate the Kafka engine option.

Parameter Description Default value

Kafka Properties

In this field, specify parameters of Kafka engine. For example: <debug>all</debug> <auto_offset_reset>latest</auto_offset_reset> <compression_type>snappy</compression_type>. The field is required

 — 

Engine
Parameter Description Default value

System

Coordination system for data replication and distributed DDL queries execution

Zookeeper

Clickhouse Keeper
NOTE

To access the configuration parameters, activate the Clickhouse Keeper option.

Parameter Description Default value

Number of nodes in cluster

Number of nodes in a cluster to be used for ClickHouse Keeper

1

TCP port

Port for a client to connect

2129

Log storage path

Path to coordination logs

/var/lib/clickhouse-keeper/coordination/log

Snapshot storage path

Path to coordination snapshots

/var/lib/clickhouse-keeper/coordination/snapshots

Operation timeout ms

Timeout for a single client operation (in milliseconds)

10000

Session timeout ms

Timeout for client session (in milliseconds)

30000

Snapshot distance

Specifies how often ClickHouse Keeper will create new snapshots (in the number of records in logs)

100000

Raft logs level

Text logging level about coordination (trace, debug, and so on)

error

Rotate log storage interval

Number of log records to store in a single file

10000

Port

Port where a server listens for RAFT connections

9444

root_path

Znode that is used as the root for znodes used by the ClickHouse server

/clickhouse

Distributed DDL path

In this ZK-path ClickHouse posts DDL tasks. Otherwise, make sure there are no intersections in the configuration with other clusters

/clickhouse/task_queue/ddl

Zookeeper
Parameter Description Default value

root_path

Znode that is used as the root for znodes used by the ClickHouse server

/clickhouse

Distributed DDL zookeeper path

In this ZK-path ClickHouse posts DDL tasks. If this CH-cluster is the only one who uses ZooKeeper — don’t wory about this setting. Otherwise, make sure there are no intersections in the configuration with other clusters

/clickhouse/task_queue/ddl

External zookeeper
NOTE

To access the configuration parameters, activate External zookeeper.

Parameter Description Default value

zk_hosts

Comma-separated locations of all servers in the ensemble, the ports on which they communicate. For example, host1.mydomain.com:2181,host2.mydomain.com:2181,host3.mydomain.com:2181. The field is required

 — 

Limits
Parameter Description Default value

Max connections

Maximum number of incoming connections

4096

Max concurrent queries

Maximum number of simultaneously processed requests

100

Max concurrent threads

Limits the number of query processing threads. If the checkbox is selected, the parameter value is automatically calculated based on the number of logical cores

Enabled

Network
Parameter Description Default value

ClickHouse http port

Port for client or proxy connections

8123

Listen host

Restriction on hosts that requests can come from. Consider possible absence of IPv6 on a target host and use e.g. 0.0.0.0 in this case

::

Directories
Parameter Description Default value

Data path

Path to the directory containing data

/var/lib/clickhouse/

Log path

Path to the directory containing log files

/var/log/clickhouse-server/

Temp path

Path to temporary data for processing large queries

/var/lib/clickhouse/tmp/

User files path

Directory with user files. Used in the file() table function

/var/lib/clickhouse/user_files/

Access control path

Path to a folder where a ClickHouse server stores user and role configurations created by SQL commands

/var/lib/clickhouse/access/

Cluster configuration
Parameter Description Default value

Replication factor

The number of shards multiplied by replication factor is the total number of nodes in the ADQM physical cluster. Cluster topology and macros are generated once and not altered by any further cluster reconfiguration. For more information, read Distributed Table Engine

Cluster Configuration

Allows you to add cluster configuration parameters

 — 

Default cluster topology

Intra-cluster communication security
NOTE

To access the configuration parameters, activate Intra-cluster communication security.

Parameter Description Default value

Interserver http name

Username for authenthication between replicas

 — 

Interserver http password

Password for authenthication between replicas

 — 

Shard secret

Cluster secret to validate distributed queries on shards

 — 

Log settings
Parameter Description Default value

Database

Name of the system database the queries will be logged in

system

Flush interval milliseconds

Interval for flushing data from the buffer in memory to the table

7500

TTL

Lifetime of logs (in days)

30

query_log

Information about executed queries (for example, start time, duration of processing, error messages)

Disabled

trace_log

Stack traces collected by the sampling query and memory profilers. Useful for deep performance analysis

Disabled

query_views_log

Information about the dependent views executed when running a query (for example, the view type or the execution time)

Disabled

query_thread_log

Information about threads that execute queries (for example, thread name, thread start time, duration of query processing)

Disabled

part_log

Information about events that occurred with data parts in the MergeTree family tables, such as adding or merging data. This is an advanced log. It is turned on if checkbox is selected and it is not controlled per session basis

Disabled

crash_log

Information about stack traces for fatal errors. This is an advanced log. It is turned on if the checkbox is selected and it is not controlled per session basis

Disabled

session_log

Information about all successful and failed login and logout events. Turned on if the checkbox is selected and it is not controlled per session basis

Disabled

Default user and policy settings
Parameter Description Default value

default_profile_settings

Allows you to add default profile settings

max_memory_usage : 10000000000
use_uncompressed_cache : 0

Access Management

SQL-driven access control and account management for a default user

Enabled

Default user password

Password for a default user

 — 

Default user IP

User can connect to ClickHouse server only from the specified IP address or a subnetwork. Examples: 192.168.0.0/16, 2001:DB8::/32

 — 

Default user host

User host can be specified as FQDN. For example, mysite.com

 — 

Default user host_regexp

You can use pcre regular expressions when specifying user hosts. For example, .+*+\.mysite\.com

 — 

Advanced configuration parameters
Parameter Description Default value

Advanced parameters (name/value)

Configuration parameters for ClickHouse’s advanced.xml file

 — 

Storage configuration
Parameter Description Default value

Storage Configuration

Allows you to configure the storage

 — 

JDBC Bridge configuration
Parameter Description Default value

datasource.json

Data source for the JDBC Bridge

[
    {
        "$schema": "./datasource.jschema",
        "postgres13": {
            "converter": {
                "mappings": [
                    {
                        "nativeType": "bool",
                        "toType": "String"
                    }
                ]
            },
            "driverUrls": [
                "drivers/postgresql/postgresql-jdbc.jar"
            ],
            "driverClassName": "org.postgresql.Driver",
            "jdbcUrl": "jdbc:postgresql://localhost/test",
            "username": "",
            "password": "",
            "initializationFailTimeout": 0,
            "minimumIdle": 0,
            "maximumPoolSize": 10
        }
    },
    {
        "$schema": "./datasource.jschema",
        "ch-server": {
            "driverUrls": [
                "drivers/clickhouse/clickhouse-jdbc.jar"
            ],
            "driverClassName": "ru.yandex.clickhouse.ClickHouseDriver",
            "jdbcUrl": "jdbc:clickhouse://ch-server:8123/system",
            "username": "default",
            "password": "",
            "initializationFailTimeout": 0,
            "minimumIdle": 0,
            "maximumPoolSize": 10
        }
    }
]

server.json

Server configuration for the JDBC Bridge

"{\n \"requestTimeout\": 5000,\n \"queryTimeout\": 60000,\n \"configScanPeriod\": 5000,\n \"repositories\": [\n {\n \"entity\": \"ru.yandex.clickhouse.jdbcbridge.core.NamedDataSource\",\n \"repository\": \"ru.yandex.clickhouse.jdbcbridge.impl.JsonFileRepository\"\n },\n {\n \"entity\": \"ru.yandex.clickhouse.jdbcbridge.core.NamedSchema\",\n \"repository\": \"ru.yandex.clickhouse.jdbcbridge.impl.JsonFileRepository\"\n },\n {\n \"entity\": \"ru.yandex.clickhouse.jdbcbridge.core.NamedQuery\",\n \"repository\": \"ru.yandex.clickhouse.jdbcbridge.impl.JsonFileRepository\"\n }\n ],\n \"extensions\": [\n {\n \"class\": \"ru.yandex.clickhouse.jdbcbridge.impl.JdbcDataSource\"\n },\n {\n \"class\": \"ru.yandex.clickhouse.jdbcbridge.impl.ConfigDataSource\"\n },\n {\n \"class\": \"ru.yandex.clickhouse.jdbcbridge.impl.ScriptDataSource\"\n }\n ]\n}\n"

Other
Parameter Description Default value

Default database

Default database

default

Default profile

Default settings profile. Settings profiles are located in the file specified in the user_config parameter

default

Auto Core Dump Service

Indicates whether an automatic core dump is enabled. To enable or disable an automatic core dump, use the corresponding action of the ADQMDB service

Disabled

Chproxy

Server configuration

 

Parameter Description Default value

Listen port

Port to listen for HTTP requests

9090

Allowed networks

List of allowed networks. Each item may contain IP address or IP subnet mask. The 127.0.0.1 value is added to configuration by default. Chproxy tries detecting the most obvious configuration errors such as allowed_networks: ["0.0.0.0/0"]

 — 

ADQM cluster configuration

 

Parameter Description Default value

Interval

Interval for checking all cluster nodes for availability

5s

Timeout

Timeout of wait response from cluster nodes

5s

Zookeeper

Main

 

Parameter Description Default value

connect

ZooKeeper connection string used by other services or clusters. It is generated automatically

 — 

dataDir

Location where ZooKeeper stores the in-memory database snapshots and, unless specified otherwise, the transaction log of updates to the database. It is specified as the dataDir parameter in the zoo.cfg configuration file

/var/lib/zookeeper

admin.serverPort

AdminServer port. An embedded Jetty server that provides an HTTP interface to the four letter word commands

58080

zoo.cfg

 

Parameter Description Default value

clientPort

Port to listen for client connections, i.e. the port that clients attempt to connect to

2181

tickTime

Length of a single tick (in milliseconds) which is the basic time unit used by ZooKeeper to regulate heartbeats and timeouts

2000

initLimit

Amount of time (in ticks) to allow followers to connect and sync to a leader. Increase this value as needed, if the amount of data managed by ZooKeeper is large

5

syncLimit

Amount of time (in ticks) to allow followers to sync with ZooKeeper. If followers fall too far behind a leader, they will be dropped

2

maxClientCnxns

Limits the number of concurrent connections that a single client, identified by IP address, may make to a single member of the ZooKeeper ensemble

0

autopurge.snapRetainCount

Retains the specified number of the most recent snapshots and the corresponding transaction logs in dataDir and dataLogDir respectively, and deletes the rest. The minimum value is 3

3

autopurge.purgeInterval

Time interval (in hours) for which the purge task has to be triggered. Set a positive integer to enable the auto purging

24

Add key,value

In this field, you can define additional parameters that are not displayed in ADCM UI, but are allowed in the zoo.cfg configuration file

 — 

zookeeper-env.sh

 

Parameter Description Default value

ZOO_LOG_DIR

Directory to store the logs

/var/log/zookeeper

ZOOPIDFILE

Directory to store the ZooKeeper process ID

/var/run/zookeeper/zookeeper_server.pid

SERVER_JVMFLAGS

Sets different JVM parameters (for example, parameters related to garbage collection)

-Xmx1024m

JAVA

Path to Java

$JAVA_HOME/bin/java

ZOO_LOG4J_PROP

Sets the log4j logging level and which log appenders to turn on. If you turn on the CONSOLE log appender, logs go to stdout. If you turn on ROLLINGFILE, zookeeper.log file is created, rotated, and expired

INFO, CONSOLE, ROLLINGFILE

Found a mistake? Seleсt text and press Ctrl+Enter to report it