LDAP client side in Kafka

Contents

Client connection to Kafka with LDAP authentication
Creating a JAAS file for a user
Creating a configuration file .properties for the user
Connecting a user to Kafka (creating tickets) and working with .sh files (scripts)

This article describes the first steps in Kafka with Kerberos LDAP installed.

Client connection to Kafka with LDAP authentication

By default, every user in the Active Directory database that has a principal in a given realm has rights to connect to the Kafka cluster and perform actions on topics.

Creating a JAAS file for a user

A JAAS file (Java Authentication and Authorization Service) must be created for all principals. It specifies how tickets for a particular principal will be used.

NOTE

A description of the JAAS file and its assignments are given in the article Krb5LoginModule.

For broker principals, a kafka-jaas.conf file is automatically created after kerberization. To view the contents of a file, enter the following command:

$ sudo vim /usr/lib/kafka/config/kafka-jaas.conf

For client principals, you need to create the JAAS file yourself.

Run command:
```
$ sudo vim /tmp/client.jaas
```
Write data to the file:
```
KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
useTicketCache=true;
};
```
— where useTicketCache is a parameter specifying whether a ticket for this user will be obtained from the ticket cache. If you set this parameter to true, you must create a user ticket before connecting to Kafka.

Creating a configuration file .properties for the user

To create a configuration file .properties for the user, run the command:

$ sudo vim /tmp/client.properties

Fill the file with data:

security.protocol=SASL_PLAINTEXT
sasl.mechanism=GSSAPI
sasl.kerberos.service.name=kafka

Connecting a user to Kafka (creating tickets) and working with .sh files (scripts)

Open a terminal session and connect to one of the Kafka brokers.

Create a ticket for a user by entering a password:

$ kinit -p admin-kafka@AD.RANGER-TEST

Password for admin-kafka@AD.RANGER-TEST:

NOTE

In this example, admin-kafka is a user with an entry in the Active Directory database and a principal for the AD.RANGER-TEST realm.
For a complete description of the kinit command functions and applicable options, see kinit.

Check ticket:

$ klist

Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: admin-kafka@AD.RANGER-TEST

Valid starting       Expires              Service principal
08/19/2022 14:18:33  08/20/2022 00:18:33  krbtgt/AD.RANGER-TEST@AD.RANGER-TEST
	renew until 08/20/2022 14:17:35

NOTE

For a complete description of the kinit command functions and applicable options, see kinit.

Export the generated client.jaas file as a JVM option for the given user using the KAFKA_OPTS environment variable:
```
$ export  KAFKA_OPTS="-Djava.security.auth.login.config=/tmp/client.jaas"
```

Create a topic by specifying the path to the created client.properties file:

$ /usr/lib/kafka/bin/kafka-topics.sh --create --topic test-topic --bootstrap-server sov-ads-test-1.ru-central1.internal:9092,sov-ads-test-2.ru-central1.internal:9092,sov-ads-test-3.ru-central1.internal:9092 --command-config /tmp/client.properties

Get a confirmation:

Created topic test-topic.

Write a message to the topic, specifying the path to the created client.properties file:

$ /usr/lib/kafka/bin/kafka-console-producer.sh --topic test-topic --bootstrap-server sov-ads-test-1.ru-central1.internal:9092,sov-ads-test-2.ru-central1.internal:9092,sov-ads-test-3.ru-central1.internal:9092 --producer.config /tmp/client.properties

>One
>Two
>Three
>Four
>Five

Open terminal session 2 and connect to one of the Kafka brokers.

Create a ticket for user reader:

$ kinit -k reader@ADS-KAFKA.LOCAL -t /tmp/reader.user.keytab

Check ticket:

$ klist

Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: reader@ADS-KAFKA.LOCAL

Valid starting       Expires              Service principal
08/10/2022 21:30:47  08/11/2022 21:30:47  krbtgt/ADS-KAFKA.LOCAL@ADS-KAFKA.LOCAL

Export the generated client.jaas file as a JVM option for the given user using the KAFKA_OPTS environment variable:
```
$ export  KAFKA_OPTS="-Djava.security.auth.login.config=/tmp/client.jaas"
```

Read messages from a topic by specifying the path to the created client.properties file:

$ /usr/lib/kafka/bin/kafka-console-consumer.sh --topic test-topic --from-beginning  --bootstrap-server sov-ads-test-1.ru-central1.internal:9092,sov-ads-test-2.ru-central1.internal:9092,sov-ads-test-3.ru-central1.internal:9092 --consumer.config /tmp/client.properties

Messages received:

One
Two
Three
Four
Five

Verify that the received messages are correct.

Found a mistake? Seleсt text and press Ctrl+Enter to report it