Use Ranger in Kafka
After activating the Ranger plugin, all actions in the Kafka service (creating topics, writing messages, reading messages, changing the configuration of topics, etc.) are possible only after setting up an authorization policy for the user.
Set an authorization policy
|
NOTE
|
In order to add a new policy for a user to the created Kafka service, perform the following steps:
-
Create a user in Ranger. To do this, in the top menu of the Ranger interface, select Settings → Users/Groups/Roles and click Add new user.
Creating a user in Ranger
Creating a user in Ranger -
In the window that opens, fill in the required data for the user:
-
User Name — username. This name cannot be duplicated in the system. The name must match the username in the KDC or LDAP database. For example, for a user with principal
writer@ADS-KAFKA.LOCAL, the name in Ranger would bewriter. -
New Password — user password.
-
Password Confirm — user password confirmation.
-
First Name — personal username.
-
Last Name — last name of the user.
-
Email Address — user’s email address.
-
Select Role — user role selection (Admin, User). This is a required field.
-
Group — select the group/groups the user belongs to.
Creating a user in Ranger
Creating a user in Ranger
-
-
Click Save.
-
In the Service Manager window, click on the name of the Kafka service.
Go to the created policy service
Go to the created policy service -
In the List of Policies window that opens, click Add new policy to add a new policy for the user.
Creating an authorization policy
Creating an authorization policy -
In the Create Policy window that opens, fill in the required fields in the Policy Details section:
-
Policy Name — policy name. This name cannot be duplicated for the same service in the system. This field is required.
-
Policy Label — provides the following features:
-
Allows the user to group policy sets using one or more labels.
-
User can search policies by label names. You can search both on the list of policies page and on the report page.
-
Helps the user to export/import policies. If the user needs to export a specific set of policies, they can find the policy label and export the specific set of policies.
-
-
Topic — the name of the topic in which messages are published.
-
Audit Logging — select whether a particular policy will be audited.
-
Enable/Disable — policy is enabled by default. You can disable a policy to restrict user/group access to that policy.
-
Include/Exclude — the include flag means that the values entered in the field will be taken into account. The default value is set to
include. The exclude flag will exclude all table or column names entered in that particular field. -
Discription — the target of the policy. This field is optional.
The Policy Details section of the Create Policy window
The Policy Details section of the Create Policy window
-
-
In the Allow Condition section of the Create Policy window, fill in the required fields:
-
Select Role — the role this policy applies to. A role is a set of permissions. Roles are an easier way to manage a set of permissions based on specific access criteria.
-
Select Group — the group this policy applies to. To promote a user to an administrator, select the Delegate Admin check box. Administrators can edit or delete a policy and create child policies. A public group contains all users, so granting access to a public group gives access to all users.
-
Select User — the user this policy applies to (outside the group already specified). You can make the user an administrator of this policy. Administrators can create child policies based on existing policies.
-
Permissions — adds or removes permissions:
-
Publish— process that publishes producer messages into Kafka topic. -
Consume— process that reads and processes producers messages from Kafka topic . -
Configure— Kafka broker/cluster configuration. -
Describe— getting metadata for a topic in Kafka.
-
-
Delegate Admin — assignes admin privileges to users or groups specified in the policy.
-
Policy Conditions — by clicking + under Add conditions you can add additional conditions.
NOTEThe conditions are met in the order specified in the policy. The condition at the beginning of the list is applied first, then the second, third, and so on.
-
-
Click Add.
The Allow Condition section of the Create Policy window
The Allow Condition section of the Create Policy window
How an authorized user works in Kafka
After activating Ranger and setting the authorization policy for the user, work in Kafka proceeds in accordance with the steps described in the articles:
-
Use MIT Kerberos in Kafka — for a user that has a principal in a Kerberos SASL secured environment.
-
SSL channel security in Kafka — for a user that has a principal in a Kerberos SASL secured environment with SSL channel security.
-
Use Kerberos with MS Active Directory in Kafka — for a user that has an Active Directory account and a corresponding LDAP server SSL certificate.
Any action by an unauthorized user will cause an error. For example, an attempt to write messages to the new-topic topic for a user whose authorization policy does not specify the Publish permission will result in the following errors:
[2022-08-29 10:21:10,094] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 3 : {new-topic=TOPIC_AUTHORIZATION_FAILED} (org.apache.kafka.clients.NetworkClient)
[2022-08-29 10:21:10,097] ERROR [Producer clientId=console-producer] Topic authorization failed for topics [new-topic] (org.apache.kafka.clients.Metadata)
[2022-08-29 10:21:10,098] ERROR Error when sending message to topic new-topic with key: null, value: 1 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [new-topic]