Use SASL PLAIN in Kafka

NOTE
Enabling SASL PLAIN authentication is done in accordance with the article SASL PLAIN.

Checking the installed SASL PLAIN

  1. Check the security and authentication configuration of the Kafka brokers.

    On each host with a Kafka broker, issue the command:

    $ vim /etc/kafka/conf/server.properties

    Verify that in the server.properties file for each Kafka broker, the lines defining the security protocol are changed to SASL_PLAINTEXT, strings defining the authentication mechanism — to PLAIN, and also passwords for all ADS cluster services are generated:

    security.inter.broker.protocol=SASL_PLAINTEXT
    
    
    sasl.mechanism.inter.broker.protocol=PLAIN
    sasl.enabled.mechanisms=PLAIN
    
    listener.name.sasl_plaintext.plain.sasl.jaas.config=org.apache.kafka.common$
      username=kafka \
      password="123456" \
      user_kafka="123456" \
      user_ksql-server="ksql-server-password" \
      user_kafka-rest="kafka-rest-password" \
      user_schema-registry="schema-registry-password" \
      user_kafka-manager="kafka-manager-password" \
      user_kafka-connect="kafka-connect-password";
  2. Verify that after installing SASL PLAIN, the listeners parameter in the Kafka service settings has changed from PLAINTEXT://:9092 to SASL_PLAINTEXT://:9092.

    ads kerberos 08
    Kafka service settings

Managing users with PLAIN authentication in Kafka

In order to create a user, do the following:

  1. In Actions of the cluster select Create/Update/Delete user. This action should appear in Actions after enabling Enable PLAIN successfully.

    ads plain 05
    Creating a new user
  2. Select the Create user action by enabling the eponymous switch.

    cluster actions 10
    Selecting an action on the user
  3. Next, enter a username and password.

    The username must contain lowercase letters, numbers, dots, underscores, and/or hyphens.

  4. After entering the data, click Run.

    cluster actions 11
    Start user creation
  5. Wait for the user creation to complete. Analyze and correct errors if they occur.

    ads plain 06
    New user creation process
  6. Make sure the server.properties file for each Kafka broker has an entry for the new user:

    listener.name.sasl_plaintext.plain.sasl.jaas.config=org.apache.kafka.common$
      username=kafka \
      password="123456" \
      user_kafka="123456" \
      user_ksql-server="ksql-server-password" \
      user_kafka-rest="kafka-rest-password" \
      user_schema-registry="schema-registry-password" \
      user_kafka-manager="kafka-manager-password" \
      user_kafka-connect="kafka-connect-password" \
      user_user_5="password";

    where:

    • user_5 — the name entered when the user was created;

    • password — the given password for the user.

    Using the action Create/Update/Delete user it is also possible to make changes to the user data (password) and delete the user.

 

To authenticate a user, you need to create a special configuration file client.properties. For this you need to do:

  1. Open the stub file by running the command:

    $ sudo vim /etc/kafka/conf/client.properties
  2. Make changes to the file by uncommenting the line with the sasl.jaas.config parameter and entering the username and password of the user as follows:

    security.protocol=SASL_PLAINTEXT
    sasl.mechanism=PLAIN
    # Uncomment and set necessary username/password
    sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
     username="user_5" \
     password="password";

    where the username and password must be enclosed in quotation marks, as shown in the example.

Working with .sh files (scripts) with participation of the created user

  1. Create a topic by specifying the path to the user configuration file client.properties described above using the --command-config option:

    $ /usr/lib/kafka/bin/kafka-topics.sh --create --topic test-topic --bootstrap-server localhost:9092 --command-config /etc/kafka/conf/client.properties

    If the topic was successfully created, the following message is displayed:

    Created topic test-topic.
  2. Write messages to the topic by specifying the path to the user configuration file client.properties described above using the --producer.config option:

    $ /usr/lib/kafka/bin/kafka-console-producer.sh --topic test-topic --bootstrap-server localhost:9092 --producer.config /etc/kafka/conf/client.properties
  3. Read messages from a topic by specifying the path to the user configuration file client.properties described above using the --consumer.config option:

    $/usr/lib/kafka/bin/kafka-console-consumer.sh --topic test-topic --from-beginning  --bootstrap-server sov-ads-test-4.ru-central1.internal:9092 --consumer.config /etc/kafka/conf/client.properties

    Verify the read messages are correct.

An attempt to run any script as a user that is not included in the server.properties file causes an authentication error, for example:

[2022-12-01 13:42:25,572] ERROR [AdminClient clientId=adminclient-1] Connection to node -1 (sov-ads-test-4.ru-central1.internal/10.92.16.223:9092) failed authentication due to: Authentication failed: Invalid username or password (org.apache.kafka.clients.NetworkClient)
[2022-12-01 13:42:25,574] WARN [AdminClient clientId=adminclient-1] Metadata update failed due to authentication error (org.apache.kafka.clients.admin.internals.AdminMetadataManager)
org.apache.kafka.common.errors.SaslAuthenticationException: Authentication failed: Invalid username or password
Error while executing topic command : Authentication failed: Invalid username or password
[2022-12-01 13:42:25,579] ERROR org.apache.kafka.common.errors.SaslAuthenticationException: Authentication failed: Invalid username or password
Found a mistake? Seleсt text and press Ctrl+Enter to report it