Use SASL PLAINTEXT in Kafka

Olga Semme
NOTE
Enabling SASL PLAINTEXT authentication is done in accordance with the article SASL PLAINTEXT.

Check the installed SASL PLAINTEXT

  1. Check the security and authentication configuration of the Kafka brokers.

    Verify that in the /etc/kafka/conf/server.properties file for each Kafka broker, the lines defining the security protocol are changed to SASL_PLAINTEXT, strings defining the authentication mechanism — to PLAIN:

    security.inter.broker.protocol=SASL_PLAINTEXT


sasl.mechanism.inter.broker.protocol=PLAIN
sasl.enabled.mechanisms=PLAIN
sasl.mechanism.controller.protocol=PLAIN

    Verify that the /etc/kafka/conf/kafka-jaas.conf file specifies the authentication module and contains the passwords for all ADS cluster services related to Kafka:

    KafkaServer {
  org.apache.kafka.common.security.plain.PlainLoginModule required
  username="kafka"
  password="kafka-password"
  user_adscc="adscc-kafka-password"
  user_kafka="kafka-password"
  user_kafka-rest="kafka-rest-password"
  user_ksql-server="ksql-server-password"
  user_kafka-connect="kafka-connect-password"
  user_schema-registry="schema-registry-password";
};

  2. Verify that after installing SASL PLAINTEXT, on the Kafka service configuration page in the server.properties group, the listeners parameter value has changed from PLAINTEXT://:9092 to SASL_PLAINTEXT://:9092.

Manage users with SASL PLAINTEXT authentication in Kafka

Create a user

In order to create a user, do the following:

  1. On the cluster configuration tab in the configuration settings tree, expand the SASL_PLAINTEXT Authentication node, then the SASL_PLAINTEXT Auth username:password map node and click + Add property to create a new user.

    Creating a new user
    Creating a new user

  2. In the window that opens, enter the user name (field name), password (secret), confirm the password, and click Apply.

    The username must contain lowercase letters, numbers, dots, underscores, and/or hyphens.

    Name and password for the new user
    Name and password for the new user

  3. Save the configuration by clicking Save.

  4. Restart the Kafka service. To do this, apply the action Restart by clicking on the actions default dark actions default light in the Actions column.

  5. Make sure the /etc/kafka/conf/kafka-jaas.conf file for each Kafka broker has an entry for the new user:

    KafkaServer {
  org.apache.kafka.common.security.plain.PlainLoginModule required
  username="kafka"
  password="kafka-password"
  user_adscc="adscc-kafka-password"
  user_kafka="kafka-password"
  user_new_user="new_user-password"
  user_kafka-rest="kafka-rest-password"
  user_ksql-server="ksql-server-password"
  user_kafka-connect="kafka-connect-password"
  user_schema-registry="schema-registry-password";
};

    where user_new_user="new_user-password" contains:

    • new_user — the name entered when the user was created;

    • new_user-password — the given password for the user.

Delete and change user data

In order to delete a user, on the cluster configuration tab in the configuration settings tree, expand the SASL_PLAINTEXT Authentication node, then the SASL_PLAINTEXT Auth username:password map node and click delete opposite the user you want to delete.

To change user data, select the desired user, enter new data and click Apply.

After deleting a user or changing data:

  1. Save the configuration by clicking Save.

  2. Restart the Kafka service. To do this, apply the action Restart by clicking on the actions default dark actions default light in the Actions column.

User authentication in the Kafka environment

To authenticate each user, including the kafka user that is automatically created by the system, you need to create a special configuration file client.properties. For this you need to do:

  1. Open the stub file by running the command:

    $ sudo vim /etc/kafka/conf/client.properties

  2. Make changes to the file by uncommenting the line with the sasl.jaas.config parameter and entering the username and password of the user as follows:

    security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
# Uncomment and set necessary username/password
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
 username="new_user" \
 password="new_user-password";

    where the username and password must be enclosed in quotation marks, as shown in the example.

Work with .sh files (scripts) with participation of the created user

  1. Create a topic by specifying the path to the user configuration file client.properties described above using the --command-config option:

    $ /usr/lib/kafka/bin/kafka-topics.sh --create --topic test-topic --bootstrap-server localhost:9092 --command-config /etc/kafka/conf/client.properties

    If the topic was successfully created, the following message is displayed:

    Created topic test-topic.

  2. Write messages to the topic by specifying the path to the user configuration file client.properties described above using the --producer.config option:

    $ /usr/lib/kafka/bin/kafka-console-producer.sh --topic test-topic --bootstrap-server localhost:9092 --producer.config /etc/kafka/conf/client.properties

  3. Read messages from a topic by specifying the path to the user configuration file client.properties described above using the --consumer.config option:

    $ /usr/lib/kafka/bin/kafka-console-consumer.sh --topic test-topic --from-beginning  --bootstrap-server sov-ads-test-4.ru-central1.internal:9092 --consumer.config /etc/kafka/conf/client.properties

    Verify the read messages are correct.

An attempt to run any script as a user that is not included in the server.properties file causes an authentication error, for example:

[2022-12-01 13:42:25,572] ERROR [AdminClient clientId=adminclient-1] Connection to node -1 (sov-ads-test-4.ru-central1.internal/10.92.16.223:9092) failed authentication due to: Authentication failed: Invalid username or password (org.apache.kafka.clients.NetworkClient)
[2022-12-01 13:42:25,574] WARN [AdminClient clientId=adminclient-1] Metadata update failed due to authentication error (org.apache.kafka.clients.admin.internals.AdminMetadataManager)
org.apache.kafka.common.errors.SaslAuthenticationException: Authentication failed: Invalid username or password
Error while executing topic command : Authentication failed: Invalid username or password
[2022-12-01 13:42:25,579] ERROR org.apache.kafka.common.errors.SaslAuthenticationException: Authentication failed: Invalid username or password
Found a mistake? Seleсt text and press Ctrl+Enter to report it