Work with kafka-rest-security

Overview

By default, a Kafka REST Proxy user can only authenticate to Kafka with one predefined principal created when Kerberos is enabled.

The kafka-rest-security module provides the ability to authenticate in Kafka with different principals when connecting to Kafka via Kafka REST Proxy.

The request is filtered through the kafka-rest-security module to obtain the username and match it against a list of predefined users for Kafka authentication.

If the Kafka REST Proxy username matches one of the Kafka users, a new security context is created using that principal and used for authentication to Kafka.

Prerequisites

To enable the kafka-rest-security module, ensure the following conditions are met:

  • The ADS cluster has authentication enabled using the Kerberos protocol with any of the authentication systems:

  • A user has been created in the authentication system with a principal and a keytab file for authentication in Kafka.

  • The ADS cluster has authentication enabled for the Kafka REST Proxy service, for example, via the basic authentication or using a jaas file, a user has been created and a role has been assigned to it.

Enable the kafka-rest-security module

To enable the module on the configuration page of the Kafka REST Proxy service, do the following:

  1. Set the Show advanced switch to active.

  2. Open the JAAS template template, where you specify user data in the KafkaClient section:

    • userNames — an option for recording the username to map the Kerberos principal with the kafka-rest principal. The username must match the name entered when creating a new user during basic authentication. Multiple usernames can be specified, separated by a space.

    • keyTab — an option to specify the Kerberos principal’s keytab file.

    • principal — an option to specify the Kerberos principal.

    For each user, the entire section must be specified. For example, below are two entries for a user, the first of which matches the Kerberos principal parameters for the Kafka REST Proxy user username, and the second of which does not:

    KafkaClient {
        com.sun.security.auth.module.Krb5LoginModule required
        useKeyTab=true
        storeKey=true
        useTicketCache=false
        serviceName="kafka"
        userNames="username"
        keyTab="/etc/security/keytabs/kafka-rest.service.keytab"
        principal="kafka-rest/<hostname>@<realm>";
    
        com.sun.security.auth.module.Krb5LoginModule required
        useKeyTab=true
        storeKey=true
        useTicketCache=false
        serviceName="kafka"
        keyTab="/etc/security/keytabs/kafka.service.keytab"
        principal="kafka/<hostname>@<realm>";
    };
    NOTE
    • For more information about the JAAS template template, see Configure a custom jaas.conf.

    • The following are out-of-the-box authentication methods that can be configured in a JAAS file and can be used with the kafka-rest-security module for mapping Kafka REST Proxy principals (usernames) to Kafka principals:

    • mTLS authentication can be used in conjunction with the above methods. This authentication method cannot be used alone because it does not provide REST principal information.

  3. Set the Principal Propagation switch to active. In the JAAS Entry field, specify the section of the JAAS template user file, which contains the data of new users.

  4. Save the settings by clicking Save.

Configuring the kafka-rest-security module
Configuring the kafka-rest-security module

After changing the settings using the ADCM interface, restart the Kafka REST Proxy service. To do this, apply the Restart action by clicking actions default dark actions default light in the Actions column.

Make sure that the following parameters appear in the /etc/kafka-rest/kafka-rest.properties configuration file:

kafka.rest.resource.extension.class=io.arenadata.kafkarest.security.KafkaRestSecurityResourceExtension
kafka.rest.auth.impersonation.enabled=true
kafka.rest.auth.impersonation.jaas.entry=KafkaClient

Where:

  • kafka.rest.resource.extension.class — a list of classes that will be used to implement user resources;

  • kafka.rest.auth.impersonation.enabled — enables user name recognition in Kafka REST Proxy;

  • kafka.rest.auth.impersonation.jaas.entry — specifies the JAAS file section for name resolution.

The mTLS authentication method is enabled by adding a custom property parameter in the kafka-rest.properties file (via the Add key, value field on the configuration page of the Kafka REST Proxy service):

  • Enter field name — ssl.client.authentication

  • Enter field value — REQUIRED

CAUTION

For Kafka REST, you should configure SSL before enabling mTLS authentication.

mTLS needs to be combined with one of the JAAS login modules for use with kafka-rest-security.

List of ready-made modules that can be configured in the JAAS template:

  • org.eclipse.jetty.security.jaas.spi.JDBCLoginModule

  • org.eclipse.jetty.security.jaas.spi.PropertyFileLoginModule

  • org.eclipse.jetty.security.jaas.spi.DataSourceLoginModule

  • org.eclipse.jetty.security.jaas.ldap.LdapLoginModule

Example of working with an authenticated user

Below is a request to write the key/value pair {"foo":"bar"} to the test topic with user authentication, where username:password is the username and password entered when creating a new user during basic authentication:

$ curl -u username:password -X POST -H "Content-Type: application/vnd.kafka.json.v2+json" --data '{"records":[{"value":{"foo":"bar"}}]}' "http://hostname:8082/topics/test"
NOTE

More information about working with Kafka REST Proxy using cURL is described in the Kafka REST Proxy usage example article.

Found a mistake? Seleсt text and press Ctrl+Enter to report it