SSL channel security
This article describes the steps to enable SSL channel security for Kafka authentication.
Prerequisites for enabling SSL
-
Every cluster host must have the keystore.jks key and certificate store (unique for each cluster host), know its path and password.
-
Every cluster host must have the truststore.jks key and certificate store (common for all cluster hosts), the path to it and its password must be known.
-
The list of DNs of all cluster hosts (like
CN=ads-host-1.example.com, OU=AD, O=AD, L=MSK, ST=MO, C=RU
) generated when creating certificates SSL must be known. -
The name of the user who has an entry in the Active Directory that will act as the NiFi administrator and his password must be known.
Create an SSL certificate for Certification Authority
Certificate generation sequence
-
Generate an RSA key pair for each host.
-
Create a keystore.jks keystore on each host.
-
Create an SSL certificate for each host.
-
Collect all certificates in one place.
-
Copy all certificates for each host to the keystore.jks store.
-
Sign the certificates with a key that belongs to one of the trusted root certificates included in the Java certificate store.
-
Create a trustore.jks store containing the required certificates.
-
Import trustore.jks into Java CA store.
-
Create and import an OpenSSL certificate for Nginx.
Create an SSL certificate for Certification Authority using a script
For test and reference purposes, when generating certificates for each host and importing certificates into the Java keystore, you can use the special script generate.sh.
It is launched on a host that has access rights to cluster hosts via FQDN and internal IP.
Below there is the sequence of creating the generate.sh script.
-
Enter the command to create the script file:
$ sudo vim /tmp/ssl/generate.sh
-
Replace for the following script lines (only for them!) the following values with values that apply to your case:
-
NUMHOSTS=
— in this line specify the values (letters or numbers), individual for each host. The values (1 2 3)
are given as an example for a list of hosts:-
sov-ads-test-1.ru-central1.internal;
-
sov-ads-test-2.ru-central1.internal;
-
sov-ads-test-3.ru-central1.internal.
-
-
HOSTS=
— specify the common part of the FQDN of the host in this line, replacing$i
with the value (letters or numbers) specified in theNUMHOSTS
parameter. The valuesov-ads-test-$i.ru-central1.internal
is given as an example. -
bigdata
— password for stores, recommended to change.
-
-
Fill in generate.sh with the script text given below, taking into account individual parameters.
generate.sh#!/bin/bash # Script for generating and import self-signed certificates to java keystore and openssl ca-bundle # Edit NUMHOSTS and HOSTS as it will be suitable for your case. declare -a NUMHOSTS declare -a HOSTS SSH_OPTS='-o GlobalKnownHostsFile=/dev/null -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' NUMHOSTS=(1 2 3) HOSTS=$(for i in "${NUMHOSTS[@]}"; do echo "sov-ads-test-$i.ru-central1.internal"; done) echo Generate keystore.jks on each host for HOST in $HOSTS; do echo "Generating keypair" ssh $SSH_OPTS $HOST "keytool -genkeypair -noprompt -keyalg RSA -alias $HOST -dname \"CN=$HOST, OU=AD, O=AD, L=MSK, S=MO, C=RU\" -keystore /tmp/keystore.jks -storepass bigdata -keypass bigdata -validity 360 -keysize 2048"; done echo echo Export certificates for HOST in $HOSTS;do ssh $SSH_OPTS $HOST "keytool -exportcert -file /tmp/$HOST.crt -keystore /tmp/keystore.jks -storepass bigdata -alias $HOST -rfc"; done echo echo Collect all certificates for HOST in $HOSTS; do scp $SSH_OPTS $HOST:/tmp/$HOST.crt /tmp/ done echo echo Transfer certificates on hosts for HOST in $HOSTS; do scp $SSH_OPTS /tmp/*.crt $HOST:/tmp/ done echo echo Import certificates on each host for HOST in $HOSTS; do ssh $SSH_OPTS $HOST "for CERT in $(echo ${HOSTS[*]}); do keytool -importcert -noprompt -alias \$CERT -file /tmp/\$CERT.crt -keystore /tmp/truststore.jks -storepass bigdata; sudo bash -c \"cat /tmp/\$CERT.crt >> /etc/pki/tls/certs/ca-bundle.crt\"; done"; done echo echo Import truststore to Java CA store for HOST in $HOSTS; do ssh $SSH_OPTS $HOST "sudo keytool -importkeystore -noprompt -srckeystore /tmp/truststore.jks -destkeystore /etc/pki/java/cacerts -deststorepass changeit -srcstorepass bigdata" done echo echo Create and import OpenSSL cert for Nginx for HOST in $HOSTS; do ssh $SSH_OPTS $HOST "sudo openssl req -new -newkey rsa:4096 -days 365 -nodes -x509 -subj \"/C=RU/ST=Denial/L=MSK/O=AD/CN=$HOST\" -keyout /etc/ssl/host_cert.key -out /etc/ssl/certs/host_cert.cert" ssh $SSH_OPTS $HOST "sudo bash -c \"cat /etc/ssl/certs/host_cert.cert >> /etc/pki/tls/certs/ca-bundle.crt\"" done
-
Allow script execution:
$ sudo chmod +x /tmp/ssl/generate.sh
-
Run the script:
$ /tmp/ssl/generate.sh
-
Monitor script execution.
-
Check each host for storages and files:
-
/etc/ssl/host_cert.key
-
/tmp/truststore.jks
-
/tmp/keystore.jks
-
NOTE
The NiFi service requires additional configurations in accordance with the article NiFi service authentication.
|
Start SSL on an ADS cluster using ADCM
-
Initiate enabling SSL from the ADCM panel in the Actions
of the cluster.
Enabling SSL on the ADS cluster -
In the window that opens, enter the appropriate configurations:
-
Keystore path — path to keystore.jks storage on hosts.
-
Keystore password — password for the keystore.jks store, default is
bigdata
. -
Truststore path — path to truststore.jks storage on hosts.
-
Truststore password — password for the truststore.jks store, default is
bigdata
. -
Force rewrite SSL configuration of services — enable if it is necessary to rewrite the configuration for all services. Use if you entered the wrong password or truststore path when you first enabled SSL.
-
(DANGEROUS!) Clean nifi user and authorizers — enable if you want to remove previously created users and previous authorization settings.
CAUTION-
Enabling SSL for NiFi service is available starting from version 1.7.0.b1 of the ADS cluster.
-
Option (DANGEROUS!) Clean nifi user and authorizers is available starting from version 1.7.2.b2 of the ADS cluster.
-
Option Force rewrite services SSL configuration is available starting from version 1.8.1.b1 of the ADS cluster.
Setting configurations to run SSL -
-
-
Click Run.
-
Wait until SSL is enabled. Analyze and correct errors if they occur.
SSl startup process
NOTE
Work in services after starting Enable SSL is carried out in accordance with the articles: |
Disable SSL on an ADS cluster using ADCM
-
Initiate disable SSL. To do this, click on the cluster icon Actions
and select Disable SSL.
Disabling SSL on an ADS Cluster -
Enable the (DANGEROUS!) Clean nifi user and authorizers option if you want to remove previously created users and previous authorization settings.
Starting disable SSL on an ADS cluster -
Click Run.
-
Wait for the disable SSL to complete. Analyze and correct errors if they occur.
SSL disable process