Ranger client side in Kafka

After activating the Ranger plugin, all actions in the Kafka service (creating topics, writing messages, reading messages, changing the configuration of topics, etc.) are possible only after setting up an authorization policy for the user.

Setting authorization policy

  • Before configuring authorization for this user, you should create a principal in the Kerberos SASL database or an account in the Active Directory database LDAP.

  • For more details on authorization in Kafka via Ranger, you can refer to Adding KAFKA Policies.

In order to add a new policy for a user to the created Kafka service, perform the following steps:

  1. Create a user in Ranger. To do this, in the top menu of the Ranger interface, select SettingsUsers/Groups/Roles and click Add new user.

    ads ranger 13 dark
    Creating a user in Ranger
    ads ranger 13 light
    Creating a user in Ranger
  2. In the window that opens, fill in the required data for the user:

    • User Name — username. This name cannot be duplicated in the system. The name must match the username in the KDC or LDAP database. For example, for a user with principal writer@ADS-KAFKA.LOCAL, the name in Ranger would be writer.

    • New Password — user password.

    • Password Confirm — user password confirmation.

    • First Name — personal username.

    • Last Name — last name of the user.

    • Email Address — user’s email address.

    • Select Role — user role selection (Admin, User). This is a required field.

    • Group — select the group/groups the user belongs to.

      ads ranger 14 dark
      Creating a user in Ranger
      ads ranger 14 light
      Creating a user in Ranger
  3. Click Save.

  4. In the Service Manager window click on the name of the Kafka service.

    ads ranger 08 dark
    Go to the created policy service
    ads ranger 08 light
    Go to the created policy service
  5. In the List of Policies window that opens, click Add new policy to add a new policy for the user.

    ads ranger 12 dark
    Creating an authorization policy
    ads ranger 12 light
    Creating an authorization policy
    • Policy Name — policy name. This name cannot be duplicated for the same service (Kafka) in the system. This field is required.

    • Policy Label — provides the following features:

      • Allows the user to group policy sets using one or more labels.

      • User can search policies by label names. You can search both on the list of policies page and on the report page.

      • Helps the user to export/import policies. If the user needs to export a specific set of policies, they can find the policy label and export the specific set of policies.

    • Topic — the name of the topic in which messages are published.

    • Audit Logging — select whether a particular policy will be audited.

    • Enable/Disable — policy is enabled by default. You can disable a policy to restrict user/group access to that policy.

    • Include/Exclude — the include flag means that the values ​​entered in the field will be taken into account. The default value is set to include. The exclude flag will exclude all table or column names entered in that particular field.

    • Discription — the target of the policy. This field is optional.

      ads ranger 15 dark
      The Policy Details section of the Create Policy window
      ads ranger 15 light
      The Policy Details section of the Create Policy window
  6. In the Allow Condition section of the Create Policy window, fill in the required fields:

    • Select Role — the role this policy applies to. A role is a set of permissions. Roles are an easier way to manage a set of permissions based on specific access criteria.

    • Select Group — the group that this policy applies to. To promote a user to an administrator, select the Delegate Admin check box. Administrators can edit or delete a policy and create child policies. A public group contains all users, so granting access to a public group gives access to all users.

    • Select User — the user this policy applies to (outside the group already specified). You can make the user an administrator of this policy. Administrators can create child policies based on existing policies.

    • Permissions — allows to add or remove permissions:

      • Publish — process that publishes producer messages into Kafka topic.

      • Consume — process that reads and processes producers messages from Kafka topic .

      • Configure — Kafka broker/cluster configuration.

      • Describe — getting metadata for a topic in Kafka.

    • Delegate Admin — allows to assign admin privileges to users or groups specified in the policy.

    • Policy Conditions — by clicking + under Add conditions you can add additional conditions.

      ads ranger 16 dark
      The Allow Condition section of the Create Policy window
      ads ranger 16 light
      Раздел Allow Condition окна Create Policy
      Conditions are evaluated in the order specified in the policy. The condition at the beginning of the list is applied first, then the second, third, and so on.
  7. Click Add.

How an authorized user works in Kafka

After activating Ranger and setting the authorization policy for the user, work in Kafka proceeds in accordance with the steps described in the articles:

Any action by an unauthorized user will cause an error. For example, an attempt to write messages to the new-topic topic for a user whose authorization policy does not specify the Publish permission will result in the following errors:

[2022-08-29 10:21:10,094] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 3 : {new-topic=TOPIC_AUTHORIZATION_FAILED} (org.apache.kafka.clients.NetworkClient)
[2022-08-29 10:21:10,097] ERROR [Producer clientId=console-producer] Topic authorization failed for topics [new-topic] (org.apache.kafka.clients.Metadata)
[2022-08-29 10:21:10,098] ERROR Error when sending message to topic new-topic with key: null, value: 1 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [new-topic]
Found a mistake? Seleсt text and press Ctrl+Enter to report it