Kafka REST Proxy configuration parameters

Olga Semme

To configure the service, use the following configuration parameters in ADCM.

NOTE

  • Some of the parameters become visible in the ADCM UI after the Advanced flag has been set.

  • The parameters that are set in the Custom group will overwrite the existing parameters even if they are read-only.
Main

 

Parameter Description Default value

rest_listener_port

REST Proxy listener port. Specified as listeners in kafka-rest.properties file

8082
kafka-env.sh

 

Parameter Description Default value

LOG_DIR

Directory to store the logs

/var/log/kafka-rest

JMX_PORT

Port on which Kafka REST Proxy sends JMX metrics

9998

KAFKAREST_OPTS

Kafka REST Proxy environment variable name

-Djava.security.auth.login.config=/etc/kafka-rest/jaas_config.conf

KAFKAREST_JMX_OPTS

JVM options in terms of JMX options (authorization, connection, SSL)

-Dcom.sun.management.jmxremote

-Dcom.sun.management.jmxremote.authenticate=false

-Dcom.sun.management.jmxremote.ssl=false

KAFKAREST_HEAP_OPTS

Heap size allocated to the Kafka REST Proxy process

-Xmx1024M

Add property

Parameters and their values ​​entered in this field override the parameters specified in the ADCM user interface. This field also allows you to set values ​​for all user parameters that are not displayed in the interface, but are allowed in the configuration file kafka-rest-env.sh

 — 
Basic Auth properties

 

Parameter Description Default value

authentication.method

Authentication method

BASIC

authentication.roles

Defines a comma-separated list of user roles. To log in to the Kafka REST Proxy server, the authenticated user must belong to at least one of these roles. For more information, see Basic authentication

admin

authentication.realm

Corresponds to a section in the jaas_config.file that defines how the server authenticates users and must be passed as a parameter to the JVM during server startup

KafkaRest
kafka-rest.properties

 

Parameter Description Default value

id

Unique ID for this REST server instance

kafka-rest-server

consumer.threads

Minimum number of threads to run consumer request on. You must set this value higher than the maximum number of consumers in a single consumer group

50

consumer.request.timeout.ms

Maximum total time to wait for messages for a request in the maximum request size has not yet been reached (in ms)

100

consumer.request.max.bytes

Maximum number of bytes in message keys and values returned by a single request

67108864

fetch.min.bytes

Minimum number of bytes in message keys and values returned by a single request

-1

ssl.endpoint.identification.algorithm

Endpoint identification algorithm to validate the server hostname using the server certificate

 — 

ssl.keystore.location

Used for HTTPS. Location of the keystore file to use for SSL

 — 

ssl.keystore.type

File format of the keystore

 — 

ssl.keystore.password

Used for HTTPS. The store password for the keystore file

 — 

ssl.key.password

Used for HTTPS. The password of the private key in the keystore file

 — 

ssl.truststore.location

Used for HTTPS. Location of the truststore. Required only to authenticate HTTPS clients

 — 

ssl.truststore.type

File format of the truststore

 — 

ssl.truststore.password

Used for HTTPS. The store password for the truststore file

 — 

client.ssl.keystore.location

Location of the SSL keystore file

 — 

client.ssl.keystore.password

Password to access the keystore

 — 

client.ssl.key.password

Password of the key contained in the keystore

 — 

client.ssl.keystore.type

File format of the keystore

 — 

client.ssl.truststore.location

Location of the SSL truststore file

 — 

client.ssl.truststore.password

Password to access the truststore

 — 

client.ssl.truststore.type

File format of the truststore

 — 

client.ssl.endpoint.identification.algorithm

Endpoint identification algorithm to validate the server

 — 

Add key, value

Parameters and their values ​​entered in this field override the parameters specified in the ADCM user interface. This field also allows you to set values ​​for all user parameters that are not displayed in the interface, but are allowed in the configuration file kafka-rest.properties

 — 
JAAS template

 

    The user file template jaas.conf is intended for specifying user data for connecting clients of other services to the current service (paths to keytab files, the useTicketCache parameter, and others). For more information, see Configure a custom jaas.conf.

    Default value:

{% if cluster.config.basic_auth_default_config is not none %}
{{ services.kafka_rest.config.basic_auth_properties_content['authentication.realm'] }} {
  org.eclipse.jetty.jaas.spi.PropertyFileLoginModule required
  file="{{ rest_home_path }}/config/password.properties"
  debug="false";
};
{% endif %}
{% if cluster.config.kerberos_client and cluster.config.kerberos_client.enable_kerberos %}
KafkaClient {
    com.sun.security.auth.module.Krb5LoginModule required
    useKeyTab=true
    storeKey=true
    useTicketCache=false
    serviceName="kafka"
    keyTab="{{ cluster.config.kerberos_client.keytab_dir }}/kafka-rest.service.keytab"
    principal="kafka-rest/{{ ansible_fqdn }}@{{ cluster.config.kerberos_client.realm }}";
};
{%- elif cluster.config.sasl_plain_auth_default_config is not none %}
    {%- set credential = cluster.config.sasl_plain_auth_default_config.sasl_plain_users_data %}
KafkaClient {
  org.apache.kafka.common.security.plain.PlainLoginModule required
  username="kafka-rest"
  password="{{ credential['kafka-rest'] }}";
};
{% endif %}
Principal Propagation

 

Parameter Description Default value

JAAS Entry

Section of the user jaas.conf file that specifies the list of users to authenticate to Kafka. For details, see Work with kafka-rest-security

KafkaClient

 

Kafka REST Proxy component configuration parameter:

log4j properties configuration

 

Parameter Description Default value

log4j.rootLogger

Logging level

INFO
log4j_properties_template

 

    The user file template log4j_properties is intended for specifying custom logging parameters.

    Default value:

{%- set kafka_rest_server_log4j_properties_configuration = services.kafka_rest.SERVER.config.log4j_properties_configuration %}

log4j.rootLogger={{ kafka_rest_server_log4j_properties_configuration['log4j.rootLogger'] }}, stdout, file

log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=[%d] %p %m (%c:%L)%n

log4j.appender.file=org.apache.log4j.RollingFileAppender
log4j.appender.file.maxBackupIndex=10
log4j.appender.file.maxFileSize=100MB
log4j.appender.file.File=${kafka-rest.log.dir}/kafka-rest.log
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=[%d] %p %m (%c)%n
JMX Monitoring
Parameter Description Default value

jmx_port

Port to which JMX metrics are sent

9998

JMX Exporter Port

Port for connecting to Prometheus JMX Exporter

11204
JMX Authentication

 
Enables authentication for JMX in the service (used when access to the JMX port needs to be protected).

Parameter Description Default value

Username

Username for authentication in JMX

monitoring

Password

User password for authentication in JMX

 — 
Found a mistake? Seleсt text and press Ctrl+Enter to report it