Use Ranger in Kafka

Olga Semme

After activating Ranger Kafka plugin, creating a Ranger service, or connecting to an existing service (depending on the value of the Override service policies parameter), all actions in the Kafka service (creating topics, writing messages, reading messages, changing the configuration of topics, etc.) are possible only after setting up an authorization policy for the user.

Set an authorization policy

NOTE

In order to add a new policy for a user to the created Kafka service, perform the following steps:

  1. Create a user in Ranger. To do this, in the top menu of the Ranger interface, select Settings → Users/Groups/Roles and click Add new user.

    Creating a user in Ranger
    Creating a user in Ranger
    Creating a user in Ranger
    Creating a user in Ranger

  2. In the window that opens, fill in the required data for the user:

    • User Name — username. This name cannot be duplicated in the system. The name must match the username in the KDC or LDAP database. For example, for a user with principal writer@ADS-KAFKA.LOCAL, the name in Ranger would be writer.

    • New Password — user password.

    • Password Confirm — user password confirmation.

    • First Name — personal username.

    • Last Name — last name of the user.

    • Email Address — user’s email address.

    • Select Role — user role selection (Admin, User, Auditor).

    • Group — the group/groups the user belongs to.

      Creating a user in Ranger
      Creating a user in Ranger
      Creating a user in Ranger
      Creating a user in Ranger

  3. Click Save.

  4. In the Service Manager window, click on the name of the Kafka service.

    Go to the created policy service
    Go to the created policy service
    Go to the created policy service
    Go to the created policy service

  5. In the List of Policies window that opens, click Add new policy to add a new policy for the user.

    Creating an authorization policy
    Creating an authorization policy
    Creating an authorization policy
    Creating an authorization policy

  6. In the Create Policy window that opens:

    • fill in the required fields in the Policy Details section:

      • Policy Name — policy name. This name cannot be duplicated for the same service in the system. This field is required.

      • Policy Label — provides the following features:

        • Allows the user to group policy sets using one or more labels.

        • User can search policies by label names. You can search both on the list of policies page and on the report page.

        • Helps the user to export/import policies. If the user needs to export a specific set of policies, they can find the policy label and export the specific set of policies.

      • Description — the purpose of the policy. This field is optional.

      • Policy Conditions — by clicking + under Add conditions, you can add additional conditions.

      • Add Validity Period — start and end time of the policy.

    • in the Policy Details section, set the desired switch positions:

      • Audit Logging — determines whether a specific policy will be audited.

      • Enable/Disabled — by default, the policy is enabled. You can disable a policy to restrict user/group access for that policy.

      • Override/Normal — specifies the override policy. If Override is selected, access permissions in the policy override access permissions in existing policies. This feature can be used in conjunction with Add Validity Period to create temporary access policies that override existing policies.

    • in the Resources section:

      • If necessary, fill in the field that determines the value of the parameter (topic, transactionalid, cluster, delegationtoken, consumergroup).

      • Set the desired position of the Include/Exclude switch. The Include position means that the values ​​​​entered in the field will be taken into account. The Exclude position will exclude all table or column names entered in that particular field.

        Policy Details and Resources sections of the Create Policy window
        Policy Details and Resources sections of the Create Policy window
        Policy Details and Resources sections of the Create Policy window
        Policy Details and Resources sections of the Create Policy window

    • fill in the fields in the Allow Condition section:

      • Select Role — the role this policy applies to. A role is a set of permissions. Roles are an easier way to manage a set of permissions based on specific access criteria.

      • Select Group — the group this policy applies to. To promote a user to an administrator, select the Delegate Admin check box. Administrators can edit or delete a policy and create child policies. A public group contains all users, so granting access to a public group gives access to all users.

      • Select User — the user this policy applies to (outside the group already specified). You can make the user an administrator of this policy. Administrators can create child policies based on existing policies.

      • Permissions — adds or removes permissions:

        • Publish — process that publishes producer messages into Kafka topic.

        • Consume — process that reads and processes producer messages from Kafka topic.

        • Configure — Kafka broker/cluster configuration.

        • Describe — getting metadata for a topic in Kafka.

      • Delegate Admin — allows you to assign administrator privileges to users or groups specified in the policy.

      • Deny All Other Accesses — denies all other accesses.

      The Allow Condition section of the Create Policy window
      The Allow Condition section of the Create Policy window
      The Allow Condition section of the Create Policy window
      The Allow Condition section of the Create Policy window
    NOTE
    The conditions are met in the order specified in the policy. The condition at the beginning of the list is applied first, then the second, third, and so on.

  7. Click Add.

How an authorized user works in Kafka

After activating Ranger and setting the authorization policy for the user, work in Kafka proceeds in accordance with the steps described in the articles:

Any action by an unauthorized user will cause an error. For example, an attempt to write messages to the new-topic topic for a user whose authorization policy does not specify the Publish permission will result in the following errors:

[2022-08-29 10:21:10,094] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 3 : {new-topic=TOPIC_AUTHORIZATION_FAILED} (org.apache.kafka.clients.NetworkClient)
[2022-08-29 10:21:10,097] ERROR [Producer clientId=console-producer] Topic authorization failed for topics [new-topic] (org.apache.kafka.clients.Metadata)
[2022-08-29 10:21:10,098] ERROR Error when sending message to topic new-topic with key: null, value: 1 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [new-topic]
Found a mistake? Seleсt text and press Ctrl+Enter to report it