Hive configuration parameters

Sources

A list of sources that will be written into hive-env.sh

 — 

HADOOP_CLASSPATH

HIVE_HOME

Hive home directory

/usr/lib/hive

METASTORE_PORT

Hive Metastore port

9083

HADOOP_CLIENT_OPTS

Hadoop client options. For example, JVM startup parameters

$HADOOP_CLIENT_OPTS -Djava.io.tmpdir={{ cluster.config.java_tmpdir | d('/tmp') }}

Sources

A list of sources that will be written into hive-server2-env.sh

 — 

HADOOP_CLIENT_OPTS

Hadoop client options for HiveServer2

-Xms256m -Xmx256m

HIVE_AUX_JARS_PATH

 — 

Final HIVE_SERVER2_ENV_OPTS

Final value of the HIVE_SERVER2_ENV_OPTS parameter in hive-server2-env.sh

 — 

Sources

A list of sources that will be written into hive-metastore-env.sh

 — 

HADOOP_CLIENT_OPTS

Hadoop client options for Hive Metastore

-Xms256m -Xmx256m

Final HIVE_METASTORE_ENV_OPTS

Final value of the HIVE_METASTORE_ENV_OPTS parameter in hive-metastore-env.sh

 — 

Encryption enable

Enables or disables the credential encryption feature. When enabled, Hive stores configuration passwords and credentials required for interacting with other services in the encrypted form

false

Credential provider path

Path to a keystore file with secrets

jceks://file/etc/hive/conf/hive.jceks

Ranger plugin credential provider path

Path to a Ranger keystore file with secrets

jceks://file/etc/hive/conf/ranger-hive.jceks

Custom jceks

Set to true to use a custom JCEKS file. Set to false to use the default auto-generated JCEKS file

false

Password file name

Name of the file in the service’s classpath that stores passwords

hive_credstore_pass

hive.cbo.enable

When set to true, enables the cost-based optimizer that uses the Calcite framework

true

hive.compute.query.using.stats

When set to true, Hive will answer a few queries like min, max, and count(1) purely using statistics stored in the Metastore. For basic statistics collection, set the configuration property hive.stats.autogather to true. For more advanced statistics collection, run the ANALYZE TABLE queries

false

hive.execution.engine

Selects the execution engine. Supported values are: mr (Map Reduce, default), tez (Tez execution, for Hadoop 2 only), or spark (Spark execution, for Hive 1.1.0 onward)

Tez

hive.log.explain.output

When enabled, logs the EXPLAIN EXTENDED output for the query at log4j INFO level and in the HiveServer2 web UI (Drilldown → Query Plan). Starting Hive 3.1.0, this configuration property only logs as the log4j INFO. To log the EXPLAIN EXTENDED output in WebUI/Drilldown/Query Plan in Hive 3.1.0 and later, use hive.server2.webui.explain.output

true

hive.metastore.event.db.notification.api.auth

Defines whether the Metastore should perform the authorization against database notification related APIs such as get_next_notification. If set to true, then only the superusers in proxy settings have the permission

false

hive.metastore.uris

Metastore URI used to access metadata in a remote metastore setup. For a remote metastore, you should specify the Thrift metastore server URI: thrift://<hostname>:<port> where <hostname> is a name or IP address of the Thrift metastore server, <port> is the port, on which the Thrift server is listening

 — 

hive.metastore.warehouse.dir

Absolute HDFS file path of the default database for the warehouse, that is local to the cluster

/apps/hive/warehouse

hive.server2.enable.doAs

Impersonate the connected user

false

hive.stats.fetch.column.stats

Annotation of the operator tree with statistics information requires column statistics. Column statistics are fetched from the Metastore. Fetching column statistics for each needed column can be expensive, when the number of columns is high. This flag can be used to disable fetching of column statistics from the Metastore

false

hive.tez.container.size

By default, Tez will spawn containers of the size of a mapper. This parameter can be used to overwrite the default value

1024

hive.support.concurrency

Defines whether Hive should support concurrency or not. A ZooKeeper instance must be up and running for the default Hive Lock Manager to support read/write locks

false

hive.txn.manager

Set this to org.apache.hadoop.hive.ql.lockmgr.DbTxnManager as part of turning on Hive transactions. The default DummyTxnManager replicates pre-Hive-0.13 behavior and provides no transactions

org.apache.hadoop.hive.ql.lockmgr.DummyTxnManager

hive.users.in.admin.role

A comma-separated list of users to be assigned the administrator role when the Metastore starts

 — 

javax.jdo.option.ConnectionUserName

Metastore database user name

APP

javax.jdo.option.ConnectionPassword

Password for the metastore user name

 — 

javax.jdo.option.ConnectionURL

jdbc:postgresql://{{ groups['adpg.adpg'][0] | d(omit) }}:5432/hive

javax.jdo.option.ConnectionDriverName

JDBC driver class name used to access Hive Metastore

org.postgresql.Driver

hive.server2.transport.mode

Sets the transport mode

binary

hive.server2.thrift.port

Port number used for the binary connection with Thrift Server2

10000

hive.server2.thrift.http.port

Port number used for the HTTP connection with Thrift Server2

10001

hive.server2.thrift.http.path

HTTP endpoint of the Thrift Server2 service

cliservice

hive.metastore.transactional.event.listeners

Listener class that stores events in a database

org.apache.hive.hcatalog.listener.DbNotificationListener

hive.metastore.dml.events

Indicates whether Hive should track DML events

true

hive.server2.authentication.kerberos.principal

HiveServer2 Kerberos principal

 — 

hive.server2.authentication.kerberos.keytab

Path to the Kerberos keytab file containing the HiveServer2 principal

 — 

hive.server2.authentication.spnego.principal

SPNEGO Kerberos principal

 — 

hive.server2.webui.spnego.principal

SPNEGO Kerberos principal to access Web UI

 — 

hive.server2.webui.spnego.keytab

SPNEGO Kerberos keytab file to access Web UI

 — 

hive.server2.webui.use.spnego

Defines whether to use Kerberos SPNEGO for Web UI access

false

hive.server2.authentication.spnego.keytab

Path to SPNEGO principal

 — 

hive.server2.authentication

Sets the authentication mode

NONE

hive.metastore.sasl.enabled

If true, the Metastore Thrift interface will be secured with SASL. Clients must authenticate with Kerberos

false

hive.metastore.kerberos.principal

Service principal for the metastore Thrift server. The _HOST token will be automatically replaced with the appropriate host name

 — 

hive.metastore.kerberos.keytab.file

Path to the Kerberos keytab file containing the metastore Thrift server’s service principal

 — 

hive.server2.use.SSL

Defines whether to use SSL for HiveServer2

false

hive.server2.keystore.path

Keystore to be used by HiveServer2

 — 

hive.server2.keystore.password

Password to the HiveServer2 keystore

 — 

hive.server2.truststore.path

Truststore to be used by HiveServer2

 — 

hive.server2.truststore.password

Password to the HiveServer2 truststore

 — 

hive.server2.webui.use.ssl

Defines whether to use SSL for the Hive web UI

false

hive.server2.webui.keystore.path

Path to the keystore file used to access the Hive web UI

 — 

hive.server2.webui.keystore.password

Password to the keystore file used to access the Hive web UI

 — 

hive.ssl.protocol.blacklist

A comma-separated list of TLS versions that cannot be used by Hive

SSLv2Hello,SSLv3,TLSv1,TLSv1.1

metastore.keystore.path

Path to the Hive Metastore keystore

 — 

metastore.keystore.password

Password to the Hive Metastore keystore

 — 

metastore.truststore.path

Path to the Hive Metastore truststore

 — 

metastore.truststore.password

Password to the Hive Metastore truststore

 — 

metastore.use.SSL

Defines whether to use SSL for interaction with Hive Metastore

false

metastore.ssl.protocol.blacklist

A comma-separated list of TLS versions that cannot be used for communication with Hive Metastore

SSLv2Hello,SSLv2,SSLv3,TLSv1,TLSv1.1

iceberg.engine.hive.enabled

Enables Iceberg tables support

true

hive.security.authorization.sqlstd.confwhitelist.append

A regex to append configuration properties to the white list in addition to hive.security.authorization.sqlstd.confwhitelist

kyuubi\.operation\.handle|kyuubi\.client\.version|kyuubi\.client\.ipAddress|tez\.application\.tags

hive.server2.support.dynamic.service.discovery

Defines whether to support dynamic service discovery via ZooKeeper

true

hive.zookeeper.quorum

A comma-separated list of ZooKeeper servers (<host>:<port>) running in the cluster

 — 

hive.server2.zookeeper.namespace

Specifies the root namespace on ZooKeeper

hiveserver2

hive.cluster.delegation.token.store.class

Name of the class that implements the delegation token store system

org.apache.hadoop.hive.metastore.security.ZooKeeperTokenStore

Enable Housekeeping Thread

Defines whether background housekeeping tasks are performed by the Hive Metastore

true

Cleaner Interval

Frequency at which the background cleaning threads for ACID transactions are executed

10s

Leader Election Mode

Defines whether to use the host specified by the Housekeeping Leader Hostname parameter as the housekeeping leader (the host option). The other option is lock, which means that Hive Metastore must use ZooKeeper locks to elect a leader

host

Housekeeping Leader Hostname

FQDN of the housekeeping leader

 — 

ZK Lock Namespace

Specifies the lock namespace on ZooKeeper

 — 

Compactor Initiator

Defines whether to start the compaction-initiator thread

true

Compactor Cleaner

Defines whether to start the compaction-cleaner thread

true

HiveServer2 hive-log4j.properties

Log4j configuration used for logging HiveServer2’s activity

hive-log4j.properties

Hive Metastore hive-log4j2.properties

Log4j2 configuration used for logging Hive Metastore’s activity

hive-log4j2.properties

Hive Beeline beeline-log4j2.properties

Log4j2 configuration used for logging Hive Beeline’s activity

beeline-log4j2.properties

xasecure.audit.destination.solr.batch.filespool.dir

Spool directory path

/srv/ranger/hdfs_plugin/audit_solr_spool

xasecure.audit.destination.solr.urls

A URL of the Solr server to store audit events. Leave this property value empty or set it to NONE when using ZooKeeper to connect to Solr

 — 

xasecure.audit.destination.solr.zookeepers

Specifies the ZooKeeper connection string for the Solr destination

 — 

xasecure.audit.destination.solr.force.use.inmemory.jaas.config

Whether to use in-memory JAAS configuration file to connect to Solr

 — 

xasecure.audit.is.enabled

Enables Ranger audit

true

xasecure.audit.jaas.Client.loginModuleControlFlag

Specifies whether the success of the module is required, requisite, sufficient, or optional

 — 

xasecure.audit.jaas.Client.loginModuleName

Name of the authenticator class

 — 

xasecure.audit.jaas.Client.option.keyTab

Name of the keytab file to get the principal’s secret key

 — 

xasecure.audit.jaas.Client.option.principal

Name of the principal to be used

 — 

xasecure.audit.jaas.Client.option.serviceName

Name of a user or a service that wants to log in

 — 

xasecure.audit.jaas.Client.option.storeKey

Set this to true if you want the keytab or the principal’s key to be stored in the subject’s private credentials

false

xasecure.audit.jaas.Client.option.useKeyTab

Set this to true if you want the module to get the principal’s key from the keytab

false

ranger.plugin.hive.policy.rest.url

 — 

ranger.plugin.hive.service.name

 — 

ranger.plugin.hive.policy.cache.dir

/srv/ranger/hive/policycache

ranger.plugin.hive.policy.pollIntervalMs

30000

ranger.plugin.hive.policy.rest.client.connection.timeoutMs

Hive Plugin RangerRestClient connection timeout (in milliseconds)

120000

ranger.plugin.hive.policy.rest.client.read.timeoutMs

Hive Plugin RangerRestClient read timeout (in milliseconds)

30000

xasecure.hive.update.xapolicies.on.grant.revoke

Controls Hive Ranger policy update from SQL Grant/Revoke commands

true

ranger.plugin.hive.policy.rest.ssl.config.file

Path to the RangerRestClient SSL config file for the Hive plugin

/etc/hive/conf/ranger-hive-policymgr-ssl.xml

xasecure.policymgr.clientssl.keystore

Path to the keystore file used by Ranger

 — 

xasecure.policymgr.clientssl.keystore.credential.file

Path to the keystore credentials file

/etc/hive/conf/ranger-hive.jceks

xasecure.policymgr.clientssl.truststore.credential.file

Path to the truststore credentials file

/etc/hive/conf/ranger-hive.jceks

xasecure.policymgr.clientssl.truststore

Path to the truststore file used by Ranger

 — 

xasecure.policymgr.clientssl.keystore.password

Password to the keystore file

 — 

xasecure.policymgr.clientssl.truststore.password

Password to the truststore file

 — 

tez.am.resource.memory.mb

Amount of memory in MB, that YARN will allocate to the Tez Application Master. The size increases with the size of the DAG

1024

tez.history.logging.service.class

Enables Tez to use the Timeline Server for History Logging

org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService

tez.lib.uris

HDFS paths containing the Tez JAR files

${fs.defaultFS}/apps/tez/tez-0.10.3.tar.gz

tez.task.resource.memory.mb

Amount of memory used by launched tasks in TEZ containers. Usually this value is set in the DAG

1024

tez.tez-ui.history-url.base

URL where the Tez UI is hosted

 — 

tez.use.cluster.hadoop-libs

Specifies, whether Tez will use the cluster Hadoop libraries

true

ssl_certificate

Path to the SSL certificate for Nginx

/etc/ssl/certs/host_cert.cert

ssl_certificate_key

Path to the SSL certificate key for Nginx

/etc/ssl/host_cert.key

ssl_protocols

A list of allowed TLS protocols to set up SSL connection

TLSv1.2

nginx_http_port

Nginx HTTP port

8089

nginx_https_port

Nginx HTTPS port

9999

ACID Transactions

Defines whether to enable ACID transactions

false

Database type

Type of the external database used for Hive Metastore

postgres

Custom hive-site.xml

 — 

Custom hive-env.sh

 — 

Ranger plugin enabled

false

Custom ranger-hive-audit.xml

 — 

Custom ranger-hive-security.xml

 — 

Custom ranger-hive-policymgr-ssl.xml

 — 

Custom tez-site.xml

 — 

beeline-hs2-connection.xml

An XML template to generate property-value pairs from the hive_beeline_connection_conf object

beeline-hs2-connection.xml

Java agent path

Path to the JMX Prometheus Java agent

/usr/lib/adh-utils/jmx/jmx_prometheus_javaagent.jar

Prometheus metrics port

Port on which to display Hive HiveServer2 metrics in the Prometheus format

9208

Mapping config path

Path to the metrics mapping configuration file

/etc/hive/conf/jmx_hive_server_metric_config.yml

Mapping config

Metrics mapping configuration file

Java agent path

Path to the JMX Prometheus Java agent

/usr/lib/adh-utils/jmx/jmx_prometheus_javaagent.jar

Prometheus metrics port

Port on which to display Hive Metastore metrics in the Prometheus format

9207

Mapping config path

Path to the metrics mapping configuration file

/etc/hive/conf/jmx_hive_server_metric_config.yml

Mapping config

Metrics mapping configuration file