Impala configuration parameters

To configure the service, use the following configuration parameters in ADCM.

NOTE
  • Some of the parameters become visible in the ADCM UI after the Advanced flag has been set.

  • The parameters that are set in the Custom group will overwrite the existing parameters even if they are read-only.

Parameter Description Default value

impala-env.sh

Contents of the impala-env.sh file that contains Impala environment settings

Custom impala-env.sh

Contents of the custom impala-env.sh file that contains custom Impala environment settings

Credential encryption
Parameter Description Default value

Encryption enable

Defines whether the credentials are encrypted

false

Credential provider path

Path to the credential provider for creating the .jceks files containing secret keys

jceks://hdfs/apps/impala/security/impala.jceks

Ranger plugin credential provider path

Path to the Ranger plugin credential provider

jceks://file/etc/impala/conf/ranger-impala.jceks

Custom jceks

Defines whether custom .jceks files located at the credential provider path are used (true) or auto-generated ones (false)

false

Password file name

Name of the password file in the classpath of the service if the password file is selected in the credstore options

impala_credstore_pass

ranger-hive-audit.xml
Parameter Description Default value

xasecure.audit.destination.solr.batch.filespool.dir

Spool directory path

/srv/ranger/hdfs_plugin/audit_solr_spool

xasecure.audit.destination.solr.urls

A URL of the Solr server to store audit events. Leave this property value empty or set it to NONE when using ZooKeeper to connect to Solr

 — 

xasecure.audit.destination.solr.zookeepers

Specifies the ZooKeeper connection string for the Solr destination

 — 

xasecure.audit.destination.solr.force.use.inmemory.jaas.config

Whether to use in-memory JAAS configuration file to connect to Solr

 — 

xasecure.audit.is.enabled

Enables Ranger audit

true

xasecure.audit.jaas.Client.loginModuleControlFlag

Specifies whether the success of the module is required, requisite, sufficient, or optional

 — 

xasecure.audit.jaas.Client.loginModuleName

Name of the authenticator class

 — 

xasecure.audit.jaas.Client.option.keyTab

Name of the keytab file to get the principal’s secret key

 — 

xasecure.audit.jaas.Client.option.principal

Name of the principal to be used

 — 

xasecure.audit.jaas.Client.option.serviceName

Name of a user or a service that wants to log in

 — 

xasecure.audit.jaas.Client.option.storeKey

Set this to true if you want the keytab or the principal’s key to be stored in the subject’s private credentials

false

xasecure.audit.jaas.Client.option.useKeyTab

Set this to true if you want the module to get the principal’s key from the keytab

false

ranger-hive-security.xml
Parameter Description Default value

ranger.plugin.hive.policy.rest.url

The URL to Ranger Admin

 — 

ranger.plugin.hive.service.name

Name of the Ranger service containing policies for this Impala instance

 — 

ranger.plugin.hive.policy.cache.dir

Directory, where Ranger policies are cached after a successful retrieval from the source

/srv/ranger/impala/policycache

ranger.plugin.hive.policy.pollIntervalMs

How often to poll for changes in policies in milliseconds

30000

ranger.plugin.hive.policy.rest.client.connection.timeoutMs

Impala plugin connection timeout in milliseconds

120000

ranger.plugin.hive.policy.rest.client.read.timeoutMs

Impala plugin read timeout in milliseconds

30000

xasecure.hive.update.xapolicies.on.grant.revoke

Specifies whether the Impala plugin should update the Ranger policies on the updates to permissions done using GRANT/REVOKE

true

ranger.plugin.hive.policy.rest.ssl.config.file

Path to the RangerRestClient SSL config file for HBase plugin

/etc/hbase/conf/ranger-hbase-policymgr-ssl.xml

ranger-hive-policymgr-ssl.xml
Parameter Description Default value

xasecure.policymgr.clientssl.keystore

Path to the keystore file used by Ranger

 — 

xasecure.policymgr.clientssl.keystore.credential.file

Path to the keystore credentials file

/etc/impala/conf/ranger-impala.jceks

xasecure.policymgr.clientssl.truststore.credential.file

Path to the truststore credentials file

/etc/impala/conf/ranger-impala.jceks

xasecure.policymgr.clientssl.truststore

Path to the truststore file used by Ranger

 — 

xasecure.policymgr.clientssl.keystore.password

Password to the keystore file

 — 

xasecure.policymgr.clientssl.truststore.password

Password to the truststore file

 — 

Enable LDAP
Parameter Description Default value

ldap_uri

URI of the LDAP server. Typically, the URI is prefixed with ldap:// or ldaps:// for SSL-based LDAP transport. The URI can optionally specify the port, for example: ldap://ldap_server.example.com:389

 — 

ldap_domain

Replaces the username with a string <username>@ldap_domain, where <username> is the name of the user trying to authenticate. Mutually exclusive with ldap_baseDN and ldap_bind_pattern

 — 

ldap_bind_dn

Distinguished name of the user to bind to for user/group searches. Required only if the user or group filters are being used and the LDAP server is not configured to allow anonymous searches

 — 

ldap_bind_password

Password of the user to bind to for user/group searches. Required only if the anonymous bind is not activated

 — 

ldap_bind_password_cmd

A Unix command the output of which returns the password to use with the --ldap_bind_dn option. The output of the command will be truncated to 1024 bytes and trimmed of trailing whitespace.

cat /etc/impala/conf/pass.pwd

ldap_user_search_basedn

Base DN for the LDAP subtree to search users

 — 

ldap_group_search_basedn

Base DN for the LDAP subtree to search groups

 — 

ldap_baseDN

Search base. Replaces the username with a DN of the form: uid=<userid>,ldap_baseDN, where <userid> is the username of the user trying to authenticate. Mutually exclusive with ldap_domain and ldap_bind_pattern

 — 

ldap_user_filter

A filter for both simple and search bind mechanisms. For a simple bind, it is a comma-separated list of user names. If specified, users must be on this list for authentication to succeed. For a search bind, it is an LDAP filter that will be used during an LDAP search, it can contain the {0} pattern which will be replaced with the user name

 — 

ldap_group_filter

Comma-separated list of groups. If specified, users must belong to one of these groups for authentication to succeed

 — 

ldap_allow_anonymous_binds

When true, LDAP authentication with a blank password (an anonymous bind) is allowed by Impala

false

ldap_search_bind_authentication

Allows switching between the search and simple bind user lookup methods when authenticating

true

ldap_ca_certificate

Specifies the location of the certificate in standard PEM format for SSL. Store this certificate on the local filesystem, in a location that only the impala user and other trusted users can read

 — 

ldap_passwords_in_clear_ok

Enables the webserver to start with the LDAP authentication even if SSL is not enabled. If set to true, the auth_creds_ok_in_clear parameter in the impalarc file gets set to true as well. A potentially unsecure configuration

false

ldap_bind_pattern

A string in which the #UID instance is replaced with the user id. For example, if this parameter is set to user=#UID,OU=foo,CN=bar and the user henry tries to authenticate, the constructed bind name will be user=henry,OU=foo,CN=bar. Mutually exclusive with ldap_domain and ldap_baseDN

 — 

allow_custom_ldap_filters_with_kerberos_auth

Specifies whether to allow custom LDAP user and group filters even if Kerberos is enabled

true

Other
Parameter Description Default value

Ranger plugin enabled

Whether or not Ranger plugin is enabled

false

Custom ranger-hive-audit.xml

In this section you can define values for custom parameters that are not displayed in ADCM UI, but are allowed in the configuration file ranger-hive-audit.xml

 — 

Custom ranger-hive-security.xml

In this section you can define values for custom parameters that are not displayed in ADCM UI, but are allowed in the configuration file ranger-hive-security.xml

 — 

Custom ranger-hive-policymgr-ssl.xml

In this section you can define values for custom parameters that are not displayed in ADCM UI, but are allowed in the configuration file ranger-hive-policymgr-ssl.xml

 — 

The Impala Catalog Service component
catalogstore.conf
Parameter Description Default value

hostname

The hostname to use for the Catalog Service daemon. If Kerberos is enabled, it is also used as a part of the Kerberos principal. If this option is not set, the system default is used

 — 

state_store_host

The host where the Impala Statestore component is running

 — 

state_store_port

The port on which the Impala Statestore component is running

24000

catalog_service_host

The host where the Impala Catalog Service component is running

 — 

catalog_service_port

The port on which the Impala Catalog Service component listens

26000

enable_webserver

Enables or disables the Catalog Service web server. Its Web UI includes information about the databases, tables, and other objects managed by Impala, in addition to the resource usage and configuration settings of the Catalog Service

True

webserver_require_spnego

Enables the Kerberos authentication for Hadoop HTTP web consoles for all roles of this service using the SPNEGO protocol. Use this option only if Kerberos is enabled for the HDFS service

False

webserver_port

The port on which the Catalog Service web server is running

25020

log_dir

The directory where the Catalog Service daemon places its log files

/var/log/impala/catalogd/

log_filename

The Prefix of the log filename — the full path is <log_dir>/<log_filename>

catalogd

max_log_files

Number of log files that are kept for each severity level (INFO, WARNING, ERROR, and FATAL) before older log files are removed. The number should be greater than 1 to keep at least the current log file to remain open. If set to 0, all log files are retained and log rotation is disabled

10

minidump_path

The directory for storing the Catalog Service daemon Breakpad dumps

/var/log/impala-minidumps

max_minidumps

Maximum number of Breakpad dump files stored by Catalog Service. A negative value or 0 is interpreted as an unlimited number

9

hms_event_polling_interval_s

When this parameter is set to a positive integer, Catalog Service fetches new notifications from Hive Metastore at the specified interval in seconds. If hms_event_polling_interval_s is set to 0, the automatic metadata invalidation and updates are disabled. See Metadata management

2

load_auth_to_local_rules

If checked (True) and Kerberos is enabled for Impala, Impala uses the auth_to_local option from hadoop.security.auth_to_local rules of the HDFS configuration

True

load_catalog_in_background

If it is set to True, the metadata is loaded in the background, even if that metadata is not required for any query. If False, the metadata is loaded when it is referenced for the first time

False

catalog_topic_mode

Granularity of on-demand metadata fetches between the Impala Daemon coordinator and Impala Catalog Service. See Metadata management

minimal

statestore_subscriber_timeout_seconds

Timeout in seconds for Impala Daemon and Catalog Server connections to Statestore

30

state_store_subscriber_port

The port where StateStoreSubscriberService is running. StateStoreSubscriberService listens on this port for updates from the Statestore daemon

23020

enable_statestored_ha

Indicates whether two Impala Statestore components are present in the cluster, which allows them to work in the high availability mode

false

state_store_2_host

The host where the second Impala Statestore component is running (high availability mode)

 — 

state_store_2_port

The port on which the second Impala Statestore component is running (high availability mode)

24000

enable_catalogd_ha

Indicates whether two Impala Catalog Service components are present in the cluster, which allows them to work in the high availability mode

false

kerberos_reinit_interval

Number of minutes between reestablishing the ticket with the Kerberos server

60

principal

Kerberos service principal

 — 

keytab_file

Kerberos service keytab file

 — 

ssl_server_certificate

Path to the TLS/SSL file with the server certificate key used for TLS/SSL. It is used when Impala operates as a TLS/SSL server. The certificate file must be in the PEM format

 — 

ssl_private_key

Path to the TLS/SSL file with the private key used for TLS/SSL. It is used when Impala operates as a TLS/SSL server. The file must be in the PEM format

 — 

ssl_client_ca_certificate

Path to the certificate, in the PEM format, used to confirm the authenticity of SSL/TLS servers that the Impala daemons can connect to. Since the Impala daemons connect to each other, it should also include the CA certificate used to sign all the SSL/TLS certificates. SSL/TLS between Impala daemons cannot be enabled without this parameter

 — 

webserver_certificate_file

Path to the TLS/SSL file with the server certificate key used for TLS/SSL. It is used when the Catalog Service web server operates as a TLS/SSL server. The certificate file must be in the PEM format

 — 

webserver_private_key_file

Path to the TLS/SSL file with the private key used for TLS/SSL. It is used when the Catalog Service web server operates as a TLS/SSL server. The certificate file must be in the PEM format

 — 

ssl_minimum_version

Minimum version of TLS

TLSv1.2

Others
Parameter Description Default value

Custom catalogstore.conf

In this section you can define values for custom parameters that are not displayed in ADCM UI, but are allowed in the configuration file catalogstore.conf

 — 

Enable custom ulimits

Switch on the corresponding toggle button to specify resource limits (ulimits) for the current process. If you do not set these values, the default system settings are used. Ulimit settings are described in the table below

[Service]
LimitCPU=
LimitFSIZE=
LimitDATA=
LimitSTACK=
LimitCORE=
LimitRSS=
LimitNOFILE=
LimitAS=
LimitNPROC=
LimitMEMLOCK=
LimitLOCKS=
LimitSIGPENDING=
LimitMSGQUEUE=
LimitNICE=
LimitRTPRIO=
LimitRTTIME=
Ulimit settings
Parameter Description Corresponding option of the ulimit command in CentOS

LimitCPU

A limit in seconds on the amount of CPU time that a process can consume

cpu time ( -t)

LimitFSIZE

Maximum size of files that a process can create, in 512-byte blocks

file size ( -f)

LimitDATA

Maximum size of a process’s data segment, in kilobytes

data seg size ( -d)

LimitSTACK

Maximum stack size allocated to a process, in kilobytes

stack size ( -s)

LimitCORE

Maximum size of a core dump file allowed for a process, in 512-byte blocks

core file size ( -c)

LimitRSS

The maximum amount of RAM memory (resident set size) that can be allocated to a process, in kilobytes

max memory size ( -m)

LimitNOFILE

Maximum number of open file descriptors allowed for the process

open files ( -n)

LimitAS

Maximum size of the process virtual memory (address space), in kilobytes

virtual memory ( -v)

LimitNPROC

Maximum number of processes

max user processes ( -u)

LimitMEMLOCK

Maximum memory size that can be locked for the process, in kilobytes. Memory locking ensures the memory is always in RAM and a swap file is not used

max locked memory ( -l)

LimitLOCKS

Maximum number of files locked by a process

file locks ( -x)

LimitSIGPENDING

Maximum number of signals that are pending for delivery to the calling thread

pending signals ( -i)

LimitMSGQUEUE

Maximum number of bytes in POSIX message queues. POSIX message queues allow processes to exchange data in the form of messages

POSIX message queues ( -q)

LimitNICE

Maximum NICE priority level that can be assigned to a process

scheduling priority ( -e)

LimitRTPRIO

Maximum real-time scheduling priority level

real-time priority ( -r)

LimitRTTIME

Maximum pipe buffer size, in 512-byte blocks

pipe size ( -p)

The Impala Client component
Parameter Description Default value

impala-shell-env.sh

The contents of the impala-shell-env.sh file that sets up necessary environment variables

impalarc

The contents of the impalarc file with the LDAP settings for impala-shell

The Impala Daemon component
impalastore.conf
Parameter Description Default value

hostname

The hostname to use for the Impala daemon. If Kerberos is enabled, it is also used as a part of the Kerberos principal. If this option is not set, the system default is used

 — 

beeswax_port

The port on which Impala daemons serve Beeswax client requests

21000

fe_port

The frontend port of the Impala daemon

21000

be_port

Internal use only. Impala daemons use this port for Thrift-based communication with each other

22000

krpc_port

Internal use only. Impala daemons use this port for KRPC-based communication with each other

27000

hs2_port

The port on which Impala daemons serve HiveServer2 client requests

21050

hs2_http_port

The port is used by client applications to transmit commands and receive results over HTTP via the HiveServer2 protocol

28000

enable_webserver

Enables or disables the Impala daemon web server. Its Web UI contains information about configuration settings, running and completed queries, and associated resource usage for them. It is primarily used for diagnosing query problems that can be traced to a particular node

True

webserver_require_spnego

Enables the Kerberos authentication for Hadoop HTTP web consoles for all roles of this service using the SPNEGO protocol. Use this option only if Kerberos is enabled for the HDFS service

False

webserver_port

The port where the Impala daemon web server is running

25000

catalog_service_host

The host where the Impala Catalog Service component is running

 — 

catalog_service_port

The port on which the Impala Catalog Service component listens

26000

state_store_host

The host where the Impala Statestore component is running

 — 

state_store_port

The port on which the Impala Statestore component is running

24000

state_store_subscriber_port

The port where StateStoreSubscriberService is running. StateStoreSubscriberService listens on this port for updates from the Statestore daemon

23030

scratch_dirs

The directory where Impala Daemons writes data to free up memory during large sort, join, aggregation, and other operations. The files are removed when the operation finishes. This can potentially be large amounts of data

/srv/impala/

log_dir

The directory where an Impala daemon places its log files

/var/log/impala/impalad/

profile_log_dir

The directory to which the profile log files are written

/var/log/impala/impalad/profiles

log_filename

The Prefix of the log filename — the full path is <log_dir>/<log_filename>

impalad

max_log_files

Number of log files that are kept for each severity level (INFO, WARNING, ERROR, and FATAL) before older log files are removed. The number should be greater than 1 to keep at least the current log file to remain open. If set to 0, all log files are retained and log rotation is disabled

10

audit_event_log_dir

The directory in which Impala daemon audit event log files are written if the Impala Audit Event Generation property is enabled

/var/log/impala/impalad/audit

minidump_path

The directory for storing Impala daemon Breakpad dumps

/var/log/impala-minidumps

lineage_event_log_dir

The directory in which the Impala daemon generates its lineage log files if the Impala Lineage Generation property is enabled

/var/log/impala/impalad/lineage

local_library_dir

The local directory into which an Impala daemon copies user-defined function (UDF) libraries from HDFS

/usr/lib/impala/udfs

max_lineage_log_file_size

Maximum size (in entries) of the Impala daemon lineage log file. When the size is exceeded, a new file is created

5000

max_audit_event_log_file_size

Maximum size (in queries) of the Impala Daemon audit event log file. When the size is exceeded, a new file is created

5000

fe_service_threads

Maximum number of concurrent client connections allowed. The parameter determines how many queries can run simultaneously. When more clients try to connect to Impala, the later arriving clients have to wait until previous clients disconnect. Setting the fe_service_threads value too high could negatively impact query latency

64

mem_limit

The memory limit (in bytes) for an Impala daemon enforced by the daemon itself. This limit does not include memory consumed by the daemon’s embedded JVM. The Impala daemon uses up this amount of memory for query processing, cached data, network buffers, background operations, etc. If the limit is exceeded, queries will be killed until the used memory becomes under the limit

1473249280

idle_query_timeout

Time in seconds after which an idle query (no processing work is done and no updates are received from the client) is cancelled. If set to 0, idle queries are never expired

0

idle_session_timeout

Time in seconds after which Impala closes an idle session and cancels all running queries. If set to 0, idle sessions never expire

0

max_result_cache_size

Maximum number of query results a client can request to be cached on a per-query basis to support restarting fetches. This option guards against unreasonably large result caches. Requests exceeding this maximum are rejected

100000

max_cached_file_handles

Maximum number of cached HDFS file handles. Caching HDFS file handles reduces the number of new file handles opened and thus reduces the load on a HDFS NameNode. Each cached file handle consumes a small amount of memory. If set to 0, the file handle caching is disabled

20000

unused_file_handle_timeout_sec

Maximum time in seconds during which an unused HDFS file handle remains in the HDFS file handle cache. When the underlying file for a cached file handle is deleted, the disk space may not be freed until the cached file handle is removed from the cache. This timeout allows the disk space occupied by deleted files to be freed in a predictable period of time. If set to 0, unused cached HDFS file handles are not removed

21600

statestore_subscriber_timeout_seconds

Timeout in seconds for Impala Daemon and Catalog Server connections to Statestore

30

default_query_options

A list of key/value pairs representing additional query options to pass to the Impala Daemon command line, separated by commas

default_file_format=parquet,default_transactional_type=none

load_auth_to_local_rules

If checked (True) and Kerberos is enabled for Impala, Impala uses the auth_to_local option from hadoop.security.auth_to_local rules of the HDFS configuration

True

catalog_topic_mode

Granularity of on-demand metadata fetches between the Impala Daemon coordinator and Impala Catalog Service. See Metadata management

minimal

use_local_catalog

Allows coordinators to cache metadata from Impala Catalog Service. If this is set to True, coordinators pull metadata as needed from catalogd and cache it locally. The cached metadata is automatically removed under memory pressure or after an expiration time. See Metadata management

True

abort_on_failed_audit_event

Specifies whether shutdown Impala if there is a problem with recording an audit event

False

max_minidumps

Maximum number of Breakpad dump files stored by the Impala daemon. A negative value or 0 is interpreted as an unlimited number

9

authorized_proxy_user_config

Specifies the set of authorized proxy users (the users who can impersonate other users during authorization), and users who they are allowed to impersonate. The example of syntax for the option is: authenticated_user1=delegated_user1,delegated_user2;authenticated_user2=*. See Configuring Impala delegation for clients. The list can contain short usernames or * to indicate all users

knox=*;zeppelin=*

queue_wait_timeout_ms

Maximum amount of time (in milliseconds) that a request waits to be admitted before timing out. Must be a positive integer

60000

disk_spill_encryption

Specifies whether to encrypt and verify the integrity of all data spilled to the disk as part of a query

False

abort_on_config_error

Specifies whether to abort Impala startup if there are incorrect configs or Impala is running on unsupported hardware

True

llama_site_path

Path to the llama-site.xml configuration file

/etc/impala/conf/llama-site.xml

fair_scheduler_allocation_path

Path to the fair-scheduler.xml configuration file

/etc/impala/conf/fair-scheduler.xml

enable_statestored_ha

Indicates whether two Impala Statestore components are present in the cluster, which allows them to work in the high availability mode

false

state_store_2_host

The host where the second Impala Statestore component is running (high availability mode)

 — 

state_store_2_port

The port on which the second Impala Statestore component is running (high availability mode)

24000

kerberos_reinit_interval

Number of minutes between reestablishing the ticket with the Kerberos server

60

principal

Kerberos service principal

 — 

keytab_file

Kerberos service keytab file

 — 

ssl_server_certificate

Path to the TLS/SSL file with the server certificate key used for TLS/SSL. It is used when Impala operates as a TLS/SSL server. The certificate file must be in the PEM format

 — 

ssl_private_key

Path to the TLS/SSL file with the private key used for TLS/SSL. It is used when Impala operates as a TLS/SSL server. The file must be in the PEM format

 — 

ssl_client_ca_certificate

Path to the certificate, in the PEM format, used to confirm the authenticity of SSL/TLS servers that the Impala daemons can connect to. Since the Impala daemons connect to each other, it should also include the CA certificate used to sign all the SSL/TLS certificates. SSL/TLS between Impala daemons cannot be enabled without this parameter

 — 

webserver_certificate_file

Path to the TLS/SSL file with the server certificate key used for TLS/SSL. It is used when the Impala daemon web server operates as a TLS/SSL server. The certificate file must be in the PEM format

 — 

webserver_private_key_file

Path to the TLS/SSL file with the private key used for TLS/SSL. It is used when the Impala daemon web server operates as a TLS/SSL server. The certificate file must be in the PEM format

 — 

ssl_minimum_version

Minimum version of TLS

TLSv1.2

Others
Parameter Description Default value

Custom impalastore.conf

In this section you can define values for custom parameters that are not displayed in ADCM UI, but are allowed in the configuration file impalastore.conf

 — 

log4j.properties

Apache Log4j utility settings

log.threshold=INFO
main.logger=FA
impala.root.logger=DEBUG,FA
log4j.rootLogger=DEBUG,FA
log.dir=/var/log/impala/impalad
max.log.file.size=200MB
log4j.appender.FA=org.apache.log4j.FileAppender
log4j.appender.FA.File=/var/log/impalad/impalad.INFO
log4j.appender.FA.layout=org.apache.log4j.PatternLayout
log4j.appender.FA.layout.ConversionPattern=%p%d{MMdd HH:mm:ss.SSS'000'} %t %c] %m%n
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.target=System.err
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{2}: %m%n

llama-site.xml

Resource pools configuration

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
</configuration>

fair_scheduler.xml

Resource pools configuration

<allocations>

</allocations>

Enable custom ulimits

Switch on the corresponding toggle button to specify resource limits (ulimits) for the current process. If you do not set these values, the default system settings are used. Ulimit settings are described in the table below

[Service]
LimitCPU=
LimitFSIZE=
LimitDATA=
LimitSTACK=
LimitCORE=
LimitRSS=
LimitNOFILE=
LimitAS=
LimitNPROC=
LimitMEMLOCK=
LimitLOCKS=
LimitSIGPENDING=
LimitMSGQUEUE=
LimitNICE=
LimitRTPRIO=
LimitRTTIME=
Ulimit settings
Parameter Description Corresponding option of the ulimit command in CentOS

LimitCPU

A limit in seconds on the amount of CPU time that a process can consume

cpu time ( -t)

LimitFSIZE

Maximum size of files that a process can create, in 512-byte blocks

file size ( -f)

LimitDATA

Maximum size of a process’s data segment, in kilobytes

data seg size ( -d)

LimitSTACK

Maximum stack size allocated to a process, in kilobytes

stack size ( -s)

LimitCORE

Maximum size of a core dump file allowed for a process, in 512-byte blocks

core file size ( -c)

LimitRSS

The maximum amount of RAM memory (resident set size) that can be allocated to a process, in kilobytes

max memory size ( -m)

LimitNOFILE

Maximum number of open file descriptors allowed for the process

open files ( -n)

LimitAS

Maximum size of the process virtual memory (address space), in kilobytes

virtual memory ( -v)

LimitNPROC

Maximum number of processes

max user processes ( -u)

LimitMEMLOCK

Maximum memory size that can be locked for the process, in kilobytes. Memory locking ensures the memory is always in RAM and a swap file is not used

max locked memory ( -l)

LimitLOCKS

Maximum number of files locked by a process

file locks ( -x)

LimitSIGPENDING

Maximum number of signals that are pending for delivery to the calling thread

pending signals ( -i)

LimitMSGQUEUE

Maximum number of bytes in POSIX message queues. POSIX message queues allow processes to exchange data in the form of messages

POSIX message queues ( -q)

LimitNICE

Maximum NICE priority level that can be assigned to a process

scheduling priority ( -e)

LimitRTPRIO

Maximum real-time scheduling priority level

real-time priority ( -r)

LimitRTTIME

Maximum pipe buffer size, in 512-byte blocks

pipe size ( -p)

The Impala Statestore component
statestore.conf
Parameter Description Default value

hostname

The hostname to use for the Statestore daemon. If Kerberos is enabled, it is also used as a part of the Kerberos principal. If this option is not set, the system default is used

 — 

state_store_host

The host where the Impala Statestore component is running

 — 

state_store_port

The port on which the Impala Statestore component is running

24000

catalog_service_host

The host where the Impala Catalog Service component is running

 — 

catalog_service_port

The port on which the Impala Catalog Service component listens

26000

enable_webserver

Enables or disables the Statestore daemon web server. Its Web UI contains information about memory usage, configuration settings, and ongoing health checks performed by Statestore

True

webserver_require_spnego

Enables the Kerberos authentication for Hadoop HTTP web consoles for all roles of this service using the SPNEGO protocol. Use this option only if Kerberos is enabled for the HDFS service

False

webserver_port

The port on which the Statestore web server is running

25010

log_dir

The directory where the Statestore daemon places its log files

/var/log/impala/statestored/

log_filename

The Prefix of the log filename — the full path is <log_dir>/<log_filename>

statestored

max_log_files

Number of log files that are kept for each severity level (INFO, WARNING, ERROR, and FATAL) before older log files are removed. The number should be greater than 1 to keep at least the current log file to remain open. If set to 0, all log files are retained and log rotation is disabled

10

minidump_path

The directory for storing Statestore daemon Breakpad dumps

/var/log/impala-minidumps

max_minidumps

Maximum number of Breakpad dump files stored by Statestore daemon. A negative value or 0 is interpreted as an unlimited number

9

state_store_num_server_worker_threads

Number of worker threads for the thread manager of the Statestore Thrift server

4

state_store_pending_task_count_max

Maximum number of tasks allowed to be pending by the thread manager of the Statestore Thrift server. The 0 value allows an infinite number of pending tasks

0

enable_statestored_ha

Indicates whether two Impala Statestore components are present in the cluster, which allows them to work in the high availability mode

false

state_store_ha_port

RPC port of the peer Statestore instance in the high availability mode

24020

kerberos_reinit_interval

Number of minutes between reestablishing the ticket with the Kerberos server

60

principal

Kerberos service principal

 — 

keytab_file

Kerberos service keytab file

 — 

ssl_server_certificate

Path to the TLS/SSL file with the server certificate key used for TLS/SSL. It is used when Impala operates as a TLS/SSL server. The certificate file must be in the PEM format

 — 

ssl_private_key

Path to the TLS/SSL file with the private key used for TLS/SSL. It is used when Impala operates as a TLS/SSL server. The file must be in the PEM format

 — 

ssl_client_ca_certificate

Path to the certificate, in the PEM format, used to confirm the authenticity of SSL/TLS servers that the Impala daemons can connect to. Since the Impala daemons connect to each other, it should also include the CA certificate used to sign all the SSL/TLS certificates. SSL/TLS between Impala daemons cannot be enabled without this parameter

 — 

webserver_certificate_file

Path to the TLS/SSL file with the server certificate key used for TLS/SSL. It is used when the Statestore web server operates as a TLS/SSL server. The certificate file must be in the PEM format

 — 

webserver_private_key_file

Path to the TLS/SSL file with the private key used for TLS/SSL. It is used when the Statestore web server operates as a TLS/SSL server. The certificate file must be in the PEM format

 — 

ssl_minimum_version

Minimum version of TLS

TLSv1.2

Others
Parameter Description Default value

Custom statestore.conf

In this section you can define values for custom parameters that are not displayed in ADCM UI, but are allowed in the configuration file statestore.conf

 — 

Enable custom ulimits

Switch on the corresponding toggle button to specify resource limits (ulimits) for the current process. If you do not set these values, the default system settings are used. Ulimit settings are described in the table below

[Service]
LimitCPU=
LimitFSIZE=
LimitDATA=
LimitSTACK=
LimitCORE=
LimitRSS=
LimitNOFILE=
LimitAS=
LimitNPROC=
LimitMEMLOCK=
LimitLOCKS=
LimitSIGPENDING=
LimitMSGQUEUE=
LimitNICE=
LimitRTPRIO=
LimitRTTIME=
Ulimit settings
Parameter Description Corresponding option of the ulimit command in CentOS

LimitCPU

A limit in seconds on the amount of CPU time that a process can consume

cpu time ( -t)

LimitFSIZE

Maximum size of files that a process can create, in 512-byte blocks

file size ( -f)

LimitDATA

Maximum size of a process’s data segment, in kilobytes

data seg size ( -d)

LimitSTACK

Maximum stack size allocated to a process, in kilobytes

stack size ( -s)

LimitCORE

Maximum size of a core dump file allowed for a process, in 512-byte blocks

core file size ( -c)

LimitRSS

The maximum amount of RAM memory (resident set size) that can be allocated to a process, in kilobytes

max memory size ( -m)

LimitNOFILE

Maximum number of open file descriptors allowed for the process

open files ( -n)

LimitAS

Maximum size of the process virtual memory (address space), in kilobytes

virtual memory ( -v)

LimitNPROC

Maximum number of processes

max user processes ( -u)

LimitMEMLOCK

Maximum memory size that can be locked for the process, in kilobytes. Memory locking ensures the memory is always in RAM and a swap file is not used

max locked memory ( -l)

LimitLOCKS

Maximum number of files locked by a process

file locks ( -x)

LimitSIGPENDING

Maximum number of signals that are pending for delivery to the calling thread

pending signals ( -i)

LimitMSGQUEUE

Maximum number of bytes in POSIX message queues. POSIX message queues allow processes to exchange data in the form of messages

POSIX message queues ( -q)

LimitNICE

Maximum NICE priority level that can be assigned to a process

scheduling priority ( -e)

LimitRTPRIO

Maximum real-time scheduling priority level

real-time priority ( -r)

LimitRTTIME

Maximum pipe buffer size, in 512-byte blocks

pipe size ( -p)

Found a mistake? Seleсt text and press Ctrl+Enter to report it