SSL channel security
This article describes the steps to enable SSL channel security for Kafka authentication.
Create SSL certificate for Certification Authority
Certificate generation sequence
-
Generate an RSA key pair for each host.
-
Create a keystore.jks keystore on each host.
-
Create an SSL certificate for each host.
-
Collect all certificates in one place.
-
Copy all certificates for each host to the keystore.jks store.
-
Sign the certificates with a key that belongs to one of the trusted root certificates included in the Java certificate store.
-
Create a trustore.jks store containing the required certificates.
-
Import trustore.jks into Java CA store.
-
Create and import an OpenSSL certificate for Nginx.
Creating an SSL certificate for Certification Authority using a script
For test and reference purposes, when generating certificates for each host and importing certificates into the Java keystore, you can use the special script generate.sh.
It is launched on a host that has access rights to cluster hosts via FQDN and internal IP.
Below there is the sequence of creating the generate.sh script.
-
Enter the command to create the script file:
$ sudo vim /tmp/ssl/generate.sh
-
Replace for the following script lines (only for them!) the following values with values that apply to your case:
-
NUMHOSTS=
— in this line specify the values (letters or numbers), individual for each host. The values (1 2 3)
are given as an example for a list of hosts:-
sov-ads-test-1.ru-central1.internal;
-
sov-ads-test-2.ru-central1.internal;
-
sov-ads-test-3.ru-central1.internal.
-
-
HOSTS=
— specify the common part of the FQDN of the host in this line, replacing$i
with the value (letters or numbers) specified in theNUMHOSTS
parameter. The valuesov-ads-test-$i.ru-central1.internal
is given as an example. -
bigdata
— password for stores, recommended to change.
-
-
Fill in generate.sh with the script text given below, taking into account individual parameters.
#!/bin/bash # Script for generating and import self-signed certificates to java keystore and openssl ca-bundle # Edit NUMHOSTS and HOSTS as it will be suitable for your case. declare -a NUMHOSTS declare -a HOSTS SSH_OPTS='-o GlobalKnownHostsFile=/dev/null -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' NUMHOSTS=(1 2 3) HOSTS=$(for i in "${NUMHOSTS[@]}"; do echo "sov-ads-test-$i.ru-central1.internal"; done) echo Generate keystore.jks on each host for HOST in $HOSTS; do echo "Generating keypair" ssh $SSH_OPTS $HOST "keytool -genkeypair -noprompt -keyalg RSA -alias $HOST -dname \"CN=$HOST, OU=AD, O=AD, L=MSK, S=MO, C=RU\" -keystore /tmp/keystore.jks -storepass bigdata -keypass bigdata -validity 360 -keysize 2048"; done echo echo Export certificates for HOST in $HOSTS;do ssh $SSH_OPTS $HOST "keytool -exportcert -file /tmp/$HOST.crt -keystore /tmp/keystore.jks -storepass bigdata -alias $HOST -rfc"; done echo echo Collect all certificates for HOST in $HOSTS; do scp $SSH_OPTS $HOST:/tmp/$HOST.crt /tmp/ done echo echo Transfer certificates on hosts for HOST in $HOSTS; do scp $SSH_OPTS /tmp/*.crt $HOST:/tmp/ done echo echo Import certificates on each host for HOST in $HOSTS; do ssh $SSH_OPTS $HOST "for CERT in $(echo ${HOSTS[*]}); do keytool -importcert -noprompt -alias \$CERT -file /tmp/\$CERT.crt -keystore /tmp/truststore.jks -storepass bigdata; sudo bash -c \"cat /tmp/\$CERT.crt >> /etc/pki/tls/certs/ca-bundle.crt\"; done"; done echo echo Import truststore to Java CA store for HOST in $HOSTS; do ssh $SSH_OPTS $HOST "sudo keytool -importkeystore -noprompt -srckeystore /tmp/truststore.jks -destkeystore /etc/pki/java/cacerts -deststorepass changeit -srcstorepass bigdata" done echo echo Create and import OpenSSL cert for Nginx for HOST in $HOSTS; do ssh $SSH_OPTS $HOST "sudo openssl req -new -newkey rsa:4096 -days 365 -nodes -x509 -subj \"/C=RU/ST=Denial/L=MSK/O=AD/CN=$HOST\" -keyout /etc/ssl/host_cert.key -out /etc/ssl/certs/host_cert.cert" ssh $SSH_OPTS $HOST "sudo bash -c \"cat /etc/ssl/certs/host_cert.cert >> /etc/pki/tls/certs/ca-bundle.crt\"" done
-
Allow script execution:
$ sudo chmod +x /tmp/ssl/generate.sh
-
Run the script:
$ /tmp/ssl/generate.sh
-
Monitor script execution.
-
Check each host for storages and files:
-
/etc/ssl/host_cert.key
-
/tmp/truststore.jks
-
/tmp/keystore.jks
-
Start SSL on an ADS Cluster using ADCM
-
Initiate enabling SSL from the ADCM panel in the Actions cluster.
Enabling SSL on the ADS cluster -
In the window that opens, enter the appropriate configurations:
-
Keystore path — path to keystore.jks storage on hosts with Kafka service.
-
Keystore password — password for keystore.jks storage, script uses
bigdata
. -
Truststore path — path to truststore.jks storage on hosts with Kafka service.
-
Truststore password — password for truststore.jks storage, script uses
bigdata
.Setting configurations to run SSL
-
-
Click Run.
-
Wait until SSL is enabled. Analyze and correct errors if they occur.
SSl startup process
Checking the installed SSL
-
Verify that the listeners parameter in the Kafka service settings changed from
SASL_PLAINTEXT://:9092
toSASL_SSL://:9092
.Kafka service settings -
Check for a change in the server.properties file on each Kafka server.
Open file server.properties:
$ sudo vim /usr/lib/kafka/config/server.properties
-
Make sure that for each Kafka broker the line defining the security protocol is changed to
SASL_SSL
and there are lines defining the server RSA keystores and passwords for them:
security.inter.broker.protocol=SASL_SSL ... ... ... ssl.keystore.location=/tmp/keystore.jks ssl.keystore.password=bigdata ssl.key.password=bigdata ssl.keystore.type=JKS ssl.truststore.location=/tmp/truststore.jks ssl.truststore.password=bigdata ssl.truststore.type=JKS
Connecting to Kafka (creating tickets) and working with .sh files (scripts) with participation of different users after installing SSL
-
To create a configuration file .properties for a user with SSL in mind, run the command:
$ sudo vim /tmp/client_ssl.properties
Fill the file with data:
security.protocol=SASL_SSL sasl.mechanism=GSSAPI sasl.kerberos.service.name=kafka
-
Follow the steps to connect to Kafka as user
writer
from the article Kerberos SASL client side in Kafka:-
Open terminal session 1 and connect to one of the Kafka brokers.
-
Create a ticket for user
writer
. -
Check ticket.
-
Export the client.jaas file as a JVM option for the given user using the
KAFKA_OPTS
environment variable.
-
-
Create a topic by specifying the path to the created client_ssl.properties file:
$ /usr/lib/kafka/bin/kafka-topics.sh --create --topic test-topic1 --bootstrap-server sov-ads-test-1.ru-central1.internal:9092,sov-ads-test-2.ru-central1.internal:9092,sov-ads-test-3.ru-central1.internal:9092 --command-config /tmp/client_ssl.properties
NOTEThe client.jaas file does not need to be changed after installing SSL. -
Write a message to a topic:
$ /usr/lib/kafka/bin/kafka-console-producer.sh --topic test-topic1 --bootstrap-server sov-ads-test-1.ru-central1.internal:9092,sov-ads-test-2.ru-central1.internal:9092,sov-ads-test-3.ru-central1.internal:9092 --producer.config /tmp/client_ssl.properties
-
Follow the steps to connect to Kafka as the user
reader
from the article Kerberos SASL client side in Kafka:-
Open terminal session 2 and connect to one of the Kafka brokers.
-
Create a ticket for user
reader
. -
Check ticket.
-
Export the generated client.jaas file as a JVM option for the given user using the
KAFKA_OPTS
environment variable.
-
-
Read messages from a topic by specifying the path to the created client_ssl.properties file:
$ /usr/lib/kafka/bin/kafka-console-consumer.sh --topic test-topic1 --from-beginning --bootstrap-server sov-ads-test-1.ru-central1.internal:9092,sov -ads-test-2.ru-central1.internal:9092,sov-ads-test-3.ru-central1.internal:9092 --consumer.config /tmp/client_ssl.properties
Make sure messages are read correctly.