Configuration parameters

This article describes the parameters that can be configured for ADQM services via ADCM. To read about the configuring process, refer to the relevant article: Online installation or Offline installation.

NOTE
Some of the parameters become visible in the ADCM UI after the Advanced flag being set.

ADQMDB

SSL
NOTE
  • To access the configuration parameters, activate the Enable SSL option.

  • For more information on SSL encryption, see the Enable SSL encryption article.

Parameter Description Default value

Enable https

Enables a port for connection to ADQM via HTTPS

Enabled

HTTPS port

Port for connections via HTTPS

8443

Enable tcp secure

Enables a TCP port for SSL-encrypted connection

Enabled

TCP secure port

TCP port for SSL-encrypted connections

9440

Disable protocols

Protocols that are not allowed to be used

 — 

Certificate file

Path to the server SSL certificate file in the PEM format

/etc/clickhouse-server/server.crt

Private key file

Path to the file with the private key of the server SSL certificate

/etc/clickhouse-server/server.key

Certificate authority file

Path to the CA certificate file

/etc/clickhouse-server/ca.crt

Validate client certificate

Enables user authentication by SSL certificates signed by the CA certificate (the path to the CA certificate file should be specified in Certificate authority file)

Disabled

LDAP
NOTE
  • To access the configuration parameters, activate the Enable LDAP option.

  • For details on how a remote LDAP server can be used to authenticate users, refer to the LDAP authentication article.

Parameter Description Default value

LDAP hostname

LDAP server hostname or IP. This parameter is mandatory and cannot be empty

localhost

LDAP port

LDAP server port. Default is 636 if Enable TLS is set to yes, otherwise — 389

636

DN prefix

Prefix used to construct the DN to bind to. Effectively, the resulting DN will be constructed as the auth_dn_prefix + escape(user_name) + auth_dn_suffix string

uid=

DN suffix

Suffix used to construct the DN to bind to. Effectively, the resulting DN will be constructed as the auth_dn_prefix + escape(user_name) + auth_dn_suffix string. Note, that this implies that auth_dn_suffix should usually have comma as its first non-space character

,ou=users,dc=example,dc=com

Enable TLS

Flag to trigger the use of secure connection to the LDAP server.

The following values are available:

  • no — for the plain text protocol (ldap://), not recommended;

  • yes — for the SSL/TLS protocol (ldaps://), recommended;

  • starttls — for the StartTLS legacy protocol (the plain text protocol, upgraded to TLS).

no

TLS min version

Minimum version of the SSL/TLS protocol. Accepted values are: ssl2, ssl3, tls1.0, tls1.1, tls1.2

tls1.2

TLS require cert

SSL/TLS peer certificate verification behavior. Accepted values are: never, allow, try, demand

demand

TLS cert file

Path to the certificate file

/path/to/tls_cert_file

TLS key file

Path to the certificate key file

/path/to/tls_key_file

CA cert file

Path to the CA certificate file

/path/to/tls_ca_cert_file

path to CA certificates

Path to the directory containing CA certificates

/path/to/tls_ca_cert_dir

Cipher suite

Allowed cipher suite

 — 

Enable RBAC

Activates role-based access control

Disabled

Default role

Default role for all authenticated users

default

Base DN

Base DN to look for in LDAP tree

ou=Groups,dc=ad,dc=ranger-test

Search filter

Search filter to look for in LDAP tree with the {user_name}, {bind_dn}, and {user_dn} patterns

(&(objectClass=groupOfNames)(member={bind_dn}))

Attribute

Attribute to look for in LDAP tree

cn

Role prefix

Attribute value prefix to look for in LDAP tree

adqm_

Kafka engine
NOTE

To access the configuration parameters, activate the Kafka engine option.

Parameter Description Default value

Kafka Properties

In this field, specify parameters of Kafka engine. For example: <debug>all</debug> <auto_offset_reset>latest</auto_offset_reset> <compression_type>snappy</compression_type>. The field is required

 — 

Engine
Parameter Description Default value

System

Coordination system for data replication and distributed DDL queries execution. Possible values: Zookeeper, Clickhouse_keeper (integrated), Clickhouse_keeper (allocated)

Zookeeper

Clickhouse Keeper (integrated)
NOTE

To access the configuration parameters, activate the Clickhouse Keepe (integrated) option.

Parameter Description Default value

Number of nodes in cluster

Number of nodes in a cluster to be used for ClickHouse Keeper

1

Enable IPv6

Allows you to disable IPv6 if your hosts do not support this protocol

Enabled

TCP port

Port for a client to connect

2129

Log storage path

Path to coordination logs

/var/lib/clickhouse-keeper/coordination/log

Snapshot storage path

Path to coordination snapshots

/var/lib/clickhouse-keeper/coordination/snapshots

Operation timeout ms

Timeout for a single client operation (in milliseconds)

10000

Session timeout ms

Timeout for a client session (in milliseconds)

30000

Snapshot distance

Specifies how often ClickHouse Keeper will create new snapshots (in the number of records in logs)

100000

Raft logs level

Coordination logging level (trace, debug, and so on)

error

Rotate log storage interval

Number of log records to store in a single file

10000

Port

Port where a server listens for RAFT connections

9444

root_path

Znode that is used as the root for znodes used by the ClickHouse server

/clickhouse

Distributed DDL path

In this ZK-path ClickHouse posts DDL tasks. Otherwise, make sure there are no intersections in the configuration with other clusters

/clickhouse/task_queue/ddl

Zookeeper
Parameter Description Default value

root_path

Znode that is used as the root for znodes used by the ClickHouse server

/clickhouse

Distributed DDL zookeeper path

In this ZK-path ClickHouse posts DDL tasks. If this CH-cluster is the only one who uses ZooKeeper — don’t wory about this setting. Otherwise, make sure there are no intersections in the configuration with other clusters

/clickhouse/task_queue/ddl

External zookeeper
NOTE

To access the configuration parameters, activate External zookeeper.

Parameter Description Default value

zk_hosts

Comma-separated locations of all servers in the ensemble, the ports on which they communicate. For example, host1.mydomain.com:2181,host2.mydomain.com:2181,host3.mydomain.com:2181. The field is required

 — 

Limits
Parameter Description Default value

Max connections

Maximum number of incoming connections

4096

Max concurrent queries

Maximum number of simultaneously processed requests

100

Max concurrent threads

Maximum number of query processing threads allowed to run all queries (excluding threads for retrieving data from remote servers). If the limit is reached, at least one thread will still be allocated to execute a query. If the parameter value is 0, there are no restrictions

0

Max server memory usage

Limits total RAM usage by the ClickHouse server (in bytes). If the parameter value is 0, the amount of RAM is not restricted

0

Max thread pool size

Limits the number of threads in the Global Thread pool that ClickHouse uses to process queries (if there are no idle threads to process a query, then a new thread is created in the pool)

10000

Network
Parameter Description Default value

Enable http

Enables a port for HTTP connection

Enabled

ClickHouse http port

Port for client or proxy connections

8123

Enable tcp

Enables a port for connections via the Native protocol (ClickHouse TCP protocol)

Enabled

ClickHouse tcp port

Port for TCP connections

9000

Listen host

Restriction on hosts that requests can come from. Consider possible absence of IPv6 on a target host and use e.g. 0.0.0.0 in this case

::

Directories
Parameter Description Default value

Data path

Path to the directory containing data

/var/lib/clickhouse/

Log path

Path to the directory containing log files

/var/log/clickhouse-server/

Temp path

Path to temporary data for processing large queries

/var/lib/clickhouse/tmp/

User files path

Directory with user files. Used in the file() table function

/var/lib/clickhouse/user_files/

Access control path

Path to a folder where a ClickHouse server stores user and role configurations created by SQL commands

/var/lib/clickhouse/access/

Cluster configuration
Parameter Description Default value

Replication factor

The number of shards multiplied by replication factor is the total number of nodes in the ADQM physical cluster. Cluster topology and macros are generated once and not altered by any further cluster reconfiguration. For more information, read Distributed Table Engine

Cluster Configuration

Allows you to add cluster configuration parameters

 — 

Default cluster topology

Intra-cluster communication security
NOTE

To access the configuration parameters, activate Intra-cluster communication security.

Parameter Description Default value

Interserver protocol

Protocol for data exchange between ClickHouse servers. If using HTTPS, enable SSL

HTTP

Interserver protocol’s port

Port for exchanging data between ClickHouse servers over the interserver protocol

9009

Interserver http name

Username for authenthication between replicas

 — 

Interserver http password

Password for authenthication between replicas

 — 

Shard secret

Cluster secret to validate distributed queries on shards

 — 

Kerberos configuration
Parameter Description Default value

security

Used Kerberos module

 — 

keytab

Path to the keytab file

 — 

reinit_frequency

Kerberos reinitialization frequency

3600

principal

Principal — unique name for a client that is allowed to authenticate with Kerberos

 — 

Log settings
Parameter Description Default value

Logging level

Verbosity level of messages written to a log file

trace

Log file size, MB

Log file size (in MB). Once a file reaches the specified size, ClickHouse archives and renames it, and creates a new log file in its place

1000

Log files count

Number of archived log files that ClickHouse stores

10

Database

Name of the system database the queries will be logged in

system

Flush interval, milliseconds

Interval for flushing data from the buffer in memory to the table

7500

TTL, days

Lifetime of logs (in days)

30

query_log

Information about executed queries (for example, start time, duration of processing, error messages)

Disabled

trace_log

Stack traces collected by the sampling query and memory profilers. Useful for deep performance analysis

Disabled

query_views_log

Information about the dependent views executed when running a query (for example, the view type or the execution time)

Disabled

query_thread_log

Information about threads that execute queries (for example, thread name, thread start time, duration of query processing)

Disabled

part_log

Information about events that occurred with data parts in the MergeTree family tables, such as adding or merging data. This is an advanced log. It is turned on if checkbox is selected and it is not controlled per session basis

Disabled

crash_log

Information about stack traces for fatal errors. This is an advanced log. It is turned on if the checkbox is selected and it is not controlled per session basis

Disabled

metric_log

Indicates whether the history of metrics values from the system.metrics and system.events tables is recorded in the system.metric_log table

Disabled

asynchronous_metric_log

Indicates whether the history of metrics values from the system.asynchronous_metrics table is recorded in the system.asynchronous_metric_log table

Disabled

Default user and policy settings
Parameter Description Default value

default_profile_settings

Allows you to add default profile settings

max_memory_usage : 10000000000
use_uncompressed_cache : 0

Access Management

SQL-driven access control and account management for a default user

Enabled

Default user password

Password for a default user

 — 

Default user IP

User can connect to ClickHouse server only from the specified IP address or a subnetwork. Examples: 192.168.0.0/16, 2001:DB8::/32

 — 

Default user host

User host can be specified as FQDN. For example, mysite.com

 — 

Default user host_regexp

You can use pcre regular expressions when specifying user hosts. For example, .+*+\.mysite\.com

 — 

User password options
Parameter Description Default value

Access Management

Allows or forbids using plaintext passwords for user profiles

Enabled

Allow NO_password

Allows or forbids creating users with no passwords

Enabled

Advanced configuration parameters
Parameter Description Default value

Advanced parameters (name/value)

Configuration parameters for ClickHouse’s advanced.xml file

 — 

Storage configuration
Parameter Description Default value

Storage Configuration

Allows you to configure the storage

 — 

Data compression
NOTE

To access the configuration parameters, activate Data compression.

Parameter Description Default value

Minimum part size

Minimum size of a data part (in bytes)

10000000000

Minimum part size ratio

Ratio of the minimum size of a data part to the table size

0.01

Method

Compression method. Possible values: LZ4, LZ4HC, ZSTD

LZ4

Level

Compression level. Possible values:

  • for the LZ4HC method — [1, 12];

  • for the ZSTD method — [1, 22];

  • the LZ4 method has no compression levels — any value is ignored.

9

JDBC Bridge configuration
Parameter Description Default value

datasource.json

Data source for the JDBC Bridge

[
    {
        "$schema": "./datasource.jschema",
        "postgres13": {
            "converter": {
                "mappings": [
                    {
                        "nativeType": "bool",
                        "toType": "String"
                    }
                ]
            },
            "driverUrls": [
                "drivers/postgresql/postgresql-jdbc.jar"
            ],
            "driverClassName": "org.postgresql.Driver",
            "jdbcUrl": "jdbc:postgresql://localhost/test",
            "username": "",
            "password": "",
            "initializationFailTimeout": 0,
            "minimumIdle": 0,
            "maximumPoolSize": 10
        }
    },
    {
        "$schema": "./datasource.jschema",
        "ch-server": {
            "driverUrls": [
                "drivers/clickhouse/clickhouse-jdbc.jar"
            ],
            "driverClassName": "ru.yandex.clickhouse.ClickHouseDriver",
            "jdbcUrl": "jdbc:clickhouse://ch-server:8123/system",
            "username": "default",
            "password": "",
            "initializationFailTimeout": 0,
            "minimumIdle": 0,
            "maximumPoolSize": 10
        }
    }
]

server.json

Server configuration for the JDBC Bridge

"{\n \"requestTimeout\": 5000,\n \"queryTimeout\": 60000,\n \"configScanPeriod\": 5000,\n \"repositories\": [\n {\n \"entity\": \"ru.yandex.clickhouse.jdbcbridge.core.NamedDataSource\",\n \"repository\": \"ru.yandex.clickhouse.jdbcbridge.impl.JsonFileRepository\"\n },\n {\n \"entity\": \"ru.yandex.clickhouse.jdbcbridge.core.NamedSchema\",\n \"repository\": \"ru.yandex.clickhouse.jdbcbridge.impl.JsonFileRepository\"\n },\n {\n \"entity\": \"ru.yandex.clickhouse.jdbcbridge.core.NamedQuery\",\n \"repository\": \"ru.yandex.clickhouse.jdbcbridge.impl.JsonFileRepository\"\n }\n ],\n \"extensions\": [\n {\n \"class\": \"ru.yandex.clickhouse.jdbcbridge.impl.JdbcDataSource\"\n },\n {\n \"class\": \"ru.yandex.clickhouse.jdbcbridge.impl.ConfigDataSource\"\n },\n {\n \"class\": \"ru.yandex.clickhouse.jdbcbridge.impl.ScriptDataSource\"\n }\n ]\n}\n"

Other
Parameter Description Default value

Time zone

Time zone used for conversions between the String and DateTime formats (for example, when values from DateTime fields are output to text format, or when strings are converted to DateTime values). If the parameter value is System, ADQM uses the timezone parameter value set in the ADQM server configuration or in the operation system settings at the moment of the ADQM server start

System

Default database

Default database

default

Default profile

Default settings profile. Settings profiles are located in the file specified in the user_config parameter

default

Auto Core Dump Service

Indicates whether an automatic core dump is enabled. To enable or disable an automatic core dump, use the corresponding action of the ADQMDB service

Disabled

Chproxy

Network configuration

 

Parameter Description Default value

Enable HTTP

Enables listening for HTTP requests

Enabled

HTTP listen port

Port to listen for HTTP requests

9090

HTTP allowed networks

List of networks from which HTTP access is allowed. Each item may contain IP address or IP subnet mask. The 127.0.0.1 value is added to configuration by default. Chproxy tries detecting the most obvious configuration errors such as allowed_networks: ["0.0.0.0/0"]

 — 

Enable HTTPS

Enables listening for HTTPS requests

Disabled

HTTPS listen port

Port to listen for HTTPS requests

8543

HTTPS allowed networks

List of networks from which HTTPS access is allowed. Each item may contain IP address or IP subnet mask. The 127.0.0.1 value is added to configuration by default. Chproxy tries detecting the most obvious configuration errors such as allowed_networks: ["0.0.0.0/0"]

 — 

Certificate file

Path to the server SSL certificate file in the PEM format

/etc/chproxy/server.crt

Private key file

Path to the file with the private key of the server SSL certificate

/etc/chproxy/server.key

Proxy wildcarded users

 

Parameter Description Default value

List of user names

List of values used as prefixes in names of wildcarded users that look like prefix_* patterns

 — 

ADQM cluster configuration

 

Parameter Description Default value

Interval

Interval for checking all cluster nodes for availability

5s

Timeout

Timeout of wait response from cluster nodes

5s

Zookeeper

Main

 

Parameter Description Default value

connect

ZooKeeper connection string used by other services or clusters. It is generated automatically

 — 

dataDir

Location where ZooKeeper stores the in-memory database snapshots and, unless specified otherwise, the transaction log of updates to the database. It is specified as the dataDir parameter in the zoo.cfg configuration file

/var/lib/zookeeper

admin.serverPort

AdminServer port. An embedded Jetty server that provides an HTTP interface to the four letter word commands

58080

zoo.cfg

 

Parameter Description Default value

clientPort

Port to listen for client connections, i.e. the port that clients attempt to connect to

2181

tickTime

Length of a single tick (in milliseconds) which is the basic time unit used by ZooKeeper to regulate heartbeats and timeouts

2000

initLimit

Amount of time (in ticks) to allow followers to connect and sync to a leader. Increase this value as needed, if the amount of data managed by ZooKeeper is large

5

syncLimit

Amount of time (in ticks) to allow followers to sync with ZooKeeper. If followers fall too far behind a leader, they will be dropped

2

maxClientCnxns

Limits the number of concurrent connections that a single client, identified by IP address, may make to a single member of the ZooKeeper ensemble

0

autopurge.snapRetainCount

Retains the specified number of the most recent snapshots and the corresponding transaction logs in dataDir and dataLogDir respectively, and deletes the rest. The minimum value is 3

3

autopurge.purgeInterval

Time interval (in hours) for which the purge task has to be triggered. Set a positive integer to enable the auto purging

24

Add key,value

In this field, you can define additional parameters that are not displayed in ADCM UI, but are allowed in the zoo.cfg configuration file

 — 

zookeeper-env.sh

 

Parameter Description Default value

ZOO_LOG_DIR

Directory to store the logs

/var/log/zookeeper

ZOOPIDFILE

Directory to store the ZooKeeper process ID

/var/run/zookeeper/zookeeper_server.pid

SERVER_JVMFLAGS

Sets different JVM parameters (for example, parameters related to garbage collection)

-Xmx1024m

JAVA

Path to Java

$JAVA_HOME/bin/java

ZOO_LOG4J_PROP

Sets the log4j logging level and which log appenders to turn on. If you turn on the CONSOLE log appender, logs go to stdout. If you turn on ROLLINGFILE, zookeeper.log file is created, rotated, and expired

INFO, CONSOLE, ROLLINGFILE

Clickhousekeeper

General settings

 

Parameter Description Default value

Listen host

Restriction on hosts from which requests can come. Consider possible absence of IPv6 on a target host and use 0.0.0.0 in this case

::

Enable IPv6

Allows you to disable IPv6 if your hosts do not support this protocol

Активен

TCP port

Port for a client to connect

2129

Log storage path

Path to coordination logs

/var/lib/clickhouse-keeper/coordination/log

Snapshot storage path

Path to coordination snapshots

/var/lib/clickhouse-keeper/coordination/snapshots

root_path

Znode that is used as a root for znodes used by the ClickHouse server

/clickhouse

Distributed DDL path

ZK-path to which ClickHouse posts DDL tasks. Otherwise, make sure there are no intersections in the configuration with other clusters

/clickhouse/task_queue/ddl

Coordination settings

 

Parameter Description Default value

Operation timeout ms

Timeout for a single client operation (in milliseconds)

10000

Session timeout ms

Timeout for a client session (in milliseconds)

30000

Snapshot distance

Specifies how often ClickHouse Keeper will create new snapshots (in the number of records in logs)

100000

Raft logs level

Coordination logging level (trace, debug, and so on)

error

Rotate log storage interval

Number of log records to store in a single file

10000

Raft settings

 

Parameter Description Default value

Port

Port where a server listens for RAFT connections

9444

Logger options

 

Parameter Description Default value

Log level

Logging level (trace, debug, information, warning, or error)

Trace

Log path

Path to the directory containing log files

/var/log/clickhouse-keeper/

Log file size

Log file size (in MB). Once a file reaches the specified size, ClickHouse archives and renames it, and creates a new log file in its place

1000

Log file count

Number of archived log files that ClickHouse stores

5

Found a mistake? Seleсt text and press Ctrl+Enter to report it