Use SASL PLAIN in Kafka

Olga Semme

Contents

Checking the installed SASL PLAIN
Managing users with PLAIN authentication in Kafka
- Working with .sh files (scripts) with participation of the created user

NOTE

Enabling SASL PLAIN authentication is done in accordance with the article SASL PLAIN.

Checking the installed SASL PLAIN

Check the security and authentication configuration of the Kafka brokers.

On each host with a Kafka broker, issue the command:

$ vim /etc/kafka/conf/server.properties

Verify that in the server.properties file for each Kafka broker, the lines defining the security protocol are changed to SASL_PLAINTEXT, strings defining the authentication mechanism — to PLAIN, and also passwords for all ADS cluster services are generated:

security.inter.broker.protocol=SASL_PLAINTEXT


sasl.mechanism.inter.broker.protocol=PLAIN
sasl.enabled.mechanisms=PLAIN

listener.name.sasl_plaintext.plain.sasl.jaas.config=org.apache.kafka.common$
  username=kafka \
  password="123456" \
  user_kafka="123456" \
  user_ksql-server="ksql-server-password" \
  user_kafka-rest="kafka-rest-password" \
  user_schema-registry="schema-registry-password" \
  user_kafka-manager="kafka-manager-password" \
  user_kafka-connect="kafka-connect-password";

Verify that after installing SASL PLAIN, the listeners parameter in the Kafka service settings has changed from PLAINTEXT://:9092 to SASL_PLAINTEXT://:9092.

Kafka service settings

Managing users with PLAIN authentication in Kafka

In order to create a user, do the following:

In Actions of the cluster select Create/Update/Delete user. This action should appear in Actions after enabling Enable PLAIN successfully.

Creating a new user
Select the Create user action by enabling the eponymous switch.

Selecting an action on the user
Next, enter a username and password.

The username must contain lowercase letters, numbers, dots, underscores, and/or hyphens.
After entering the data, click Run.

Start user creation
Wait for the user creation to complete. Analyze and correct errors if they occur.

New user creation process

Make sure the server.properties file for each Kafka broker has an entry for the new user:

listener.name.sasl_plaintext.plain.sasl.jaas.config=org.apache.kafka.common$
  username=kafka \
  password="123456" \
  user_kafka="123456" \
  user_ksql-server="ksql-server-password" \
  user_kafka-rest="kafka-rest-password" \
  user_schema-registry="schema-registry-password" \
  user_kafka-manager="kafka-manager-password" \
  user_kafka-connect="kafka-connect-password" \
  user_user_5="password";

where:

user_5 — the name entered when the user was created;
password — the given password for the user.

Using the action Create/Update/Delete user it is also possible to make changes to the user data (password) and delete the user.

To authenticate a user, you need to create a special configuration file client.properties. For this you need to do:

Open the stub file by running the command:

$ sudo vim /etc/kafka/conf/client.properties

Make changes to the file by uncommenting the line with the sasl.jaas.config parameter and entering the username and password of the user as follows:
```
security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
# Uncomment and set necessary username/password
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
 username="user_5" \
 password="password";
```
where the username and password must be enclosed in quotation marks, as shown in the example.

Working with .sh files (scripts) with participation of the created user

Create a topic by specifying the path to the user configuration file client.properties described above using the --command-config option:
```
$ /usr/lib/kafka/bin/kafka-topics.sh --create --topic test-topic --bootstrap-server localhost:9092 --command-config /etc/kafka/conf/client.properties
```
If the topic was successfully created, the following message is displayed:
```
Created topic test-topic.
```

Write messages to the topic by specifying the path to the user configuration file client.properties described above using the --producer.config option:

$ /usr/lib/kafka/bin/kafka-console-producer.sh --topic test-topic --bootstrap-server localhost:9092 --producer.config /etc/kafka/conf/client.properties

Read messages from a topic by specifying the path to the user configuration file client.properties described above using the --consumer.config option:

$/usr/lib/kafka/bin/kafka-console-consumer.sh --topic test-topic --from-beginning  --bootstrap-server sov-ads-test-4.ru-central1.internal:9092 --consumer.config /etc/kafka/conf/client.properties

Verify the read messages are correct.

An attempt to run any script as a user that is not included in the server.properties file causes an authentication error, for example:

[2022-12-01 13:42:25,572] ERROR [AdminClient clientId=adminclient-1] Connection to node -1 (sov-ads-test-4.ru-central1.internal/10.92.16.223:9092) failed authentication due to: Authentication failed: Invalid username or password (org.apache.kafka.clients.NetworkClient)
[2022-12-01 13:42:25,574] WARN [AdminClient clientId=adminclient-1] Metadata update failed due to authentication error (org.apache.kafka.clients.admin.internals.AdminMetadataManager)
org.apache.kafka.common.errors.SaslAuthenticationException: Authentication failed: Invalid username or password
Error while executing topic command : Authentication failed: Invalid username or password
[2022-12-01 13:42:25,579] ERROR org.apache.kafka.common.errors.SaslAuthenticationException: Authentication failed: Invalid username or password

Found a mistake? Seleсt text and press Ctrl+Enter to report it