SSL channel security

Prerequisites for enabling SSL

  • Every cluster host must have the keystore.jks key and certificate store (unique for each cluster host), know its path and password.

  • Every cluster host must have the truststore.jks key and certificate store (common for all cluster hosts), the path to it and its password must be known.

  • The list of DNs of all cluster hosts (like CN=ads-host-1.example.com, OU=AD, O=AD, L=MSK, ST=MO, C=RU) generated when creating certificates SSL must be known.

  • The name of the user who has an entry in the Active Directory that will act as the NiFi administrator and his password must be known.

Create an SSL certificate for Certification Authority

Certificate generation sequence

  1. Generate an RSA key pair for each host.

  2. Create a keystore.jks keystore on each host.

  3. Create an SSL certificate for each host.

  4. Collect all certificates in one place.

  5. Copy all certificates for each host to the keystore.jks store.

  6. Sign the certificates with a key that belongs to one of the trusted root certificates included in the Java certificate store.

  7. Create a trustore.jks store containing the required certificates.

  8. Import trustore.jks into Java CA store.

  9. Create and import an OpenSSL certificate for Nginx.

CAUTION

To correctly enable authentication in the NiFi service when creating certificates, specify and fill in the san (subjectAltNames) field using the -ext option (for example, for self-signed certificates created below, the filled field looks like this: -ext san=dns: $HOST).

Create an SSL certificate for Certification Authority using a script

For test and reference purposes, when generating certificates for each host and importing certificates into the Java keystore, you can use the special script generate.sh.

Using this script, it is possible to create new certificates and overwrite existing ones.

It is launched on a host that has access rights to cluster hosts via FQDN and internal IP.

Below there is the sequence of creating the generate.sh script.

  1. Enter the command to create the script file:

    $ sudo vim /tmp/ssl/generate.sh
  2. Replace for the following script lines (only for them!) the following values ​​with values ​​that apply to your case:

    • NUMHOSTS= — in this line specify the values ​​(letters or numbers), individual for each host. The values ​​(1 2 3) are given as an example for a list of hosts:

      • sov-ads-test-1.ru-central1.internal;

      • sov-ads-test-2.ru-central1.internal;

      • sov-ads-test-3.ru-central1.internal.

    • HOSTS= — specify the common part of the FQDN of the host in this line, replacing $i with the value (letters or numbers) specified in the NUMHOSTS parameter. The value sov-ads-test-$i.ru-central1.internal is given as an example.

    • bigdata — password for stores, recommended to change.

  3. Fill in generate.sh with the script text given below, taking into account individual parameters.

    generate.sh

     

    #!/bin/bash
    
    # Script that is used to generate and import self-signed certificates to java keystore
    # Edit NUMHOSTS and HOSTS as it will be suitable for your case.
    
    declare -a NUMHOSTS
    declare -a HOSTS
    
    SSH_OPTS='-o GlobalKnownHostsFile=/dev/null -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no'
    NUMHOSTS=(1 2 3)
    HOSTS=$(for i in "${NUMHOSTS[@]}"; do echo "sov-ads-test-$i.ru-central1.internal"; done)
    
    
    for HOST in $HOSTS; do
      	echo "update-ca-trust"
    	ssh $SSH_OPTS $HOST "sudo update-ca-trust extract"
    done
    
    echo Generate keystore.jks on each host
    for HOST in $HOSTS; do
      	echo "remove old alias"
      	ssh $SSH_OPTS $HOST "keytool -delete -alias $HOST -keystore /tmp/keystore.jks -storepass bigdata";
    
      	echo "Generating keypair"
    	ssh $SSH_OPTS $HOST "keytool 	-genkey \
    	                                -noprompt \
    	                                -keyalg RSA \
    	                                -ext san=dns:$HOST \
    	                                -alias $HOST \
    	                                -dname \"CN=$HOST, OU=AD, O=AD, L=MSK, S=MO, C=RU\" \
    	                                -keystore /tmp/keystore.jks \
    	                                -storepass bigdata \
    	                                -keypass bigdata \
    	                                -validity 360 \
    	                                -keysize 2048";
    done
    
    echo
    echo Export certificates
    for HOST in $HOSTS;do
    	ssh $SSH_OPTS $HOST "keytool 	-export \
    	                                -file /tmp/$HOST.crt \
    	                                -keystore /tmp/keystore.jks \
    	                                -storepass bigdata \
    	                                -alias $HOST \
    	                                -rfc";
    done
    
    echo
    echo Collect all certificates
    for HOST in $HOSTS; do
     	scp $SSH_OPTS $HOST:/tmp/$HOST.crt /tmp/
    
    #        scp $SSH_OPTS $HOST:/tmp/*.crt /tmp/
    done
    
    echo
    echo Transfer certificates on hosts
    for HOST in $HOSTS; do
    	for CERT in $HOSTS; do
    	    scp $SSH_OPTS /tmp/$CERT.crt $HOST:/tmp/
    	done
    
    #	scp $SSH_OPTS /tmp/*.crt $HOST:/tmp/
    done
    
    echo
    echo Import certificates on each host
    for HOST in $HOSTS; do
    
    	ssh $SSH_OPTS $HOST "for CERT in $(echo ${HOSTS[*]}); do
    
    		echo "remove old alias before import"
    		keytool -delete -noprompt -alias \$CERT -keystore /tmp/truststore.jks -storepass bigdata
    
    		echo "Import Cert"
    		keytool 	-import \
    		 	        -noprompt \
    		 	        -alias \$CERT \
    		 	        -file /tmp/\$CERT.crt \
    		 	        -keystore /tmp/truststore.jks \
    		 	        -storepass bigdata;
    		sudo bash -c \"cat /tmp/\$CERT.crt >> /etc/pki/tls/certs/ca-bundle.crt\";
    		done";
    done
    
    echo
    echo Import truststore to Java CA store
    for HOST in $HOSTS; do
    	ssh $SSH_OPTS $HOST "sudo keytool 	-importkeystore \
    	 	                                -noprompt \
    	 	                                -srckeystore /tmp/truststore.jks \
    	 	                                -destkeystore /etc/pki/java/cacerts \
    	 	                                -deststorepass changeit \
    	 	                                -srcstorepass bigdata"
    done
    
    echo
    echo Create directories for certs
    for HOST in $HOSTS; do
    	ssh $SSH_OPTS $HOST "[ -d /etc/ssl ] || sudo mkdir /etc/ssl;
    			     [ -d /etc/ssl/certs ] || sudo mkdir /etc/ssl/certs"
    done
    
    echo
    echo Create and import cert for Nginx
    for HOST in $HOSTS; do
    	ssh $SSH_OPTS $HOST "sudo openssl req -new -newkey rsa:4096 -days 365 -nodes -x509 -subj \"/C=RU/ST=Denial/L=MSK/O=AD/CN=$HOST\" -keyout /etc/ssl/host_cert.key  -out /etc/ssl/certs/host_cert.cert"
    	ssh $SSH_OPTS $HOST "sudo bash -c \"cat /etc/ssl/certs/host_cert.cert >> /etc/pki/tls/certs/ca-bundle.crt\""
    done
  4. Allow script execution:

    $ sudo chmod +x /tmp/ssl/generate.sh
  5. Run the script:

    $ /tmp/ssl/generate.sh
  6. Monitor script execution.

  7. Check each host for storages and files:

    • /etc/ssl/host_cert.key

    • /tmp/truststore.jks

    • /tmp/keystore.jks

NOTE
The NiFi service requires additional configurations in accordance with the article NiFi service authentication.

Start SSL on an ADS cluster using ADCM

To enable SSL, do the following:

  1. Apply the cluster action Manage SSL by clicking actions default dark actions default light in the Actions column of the ADS cluster.

  2. In the window that opens, activate the Enable SSL switch, expand the configuration tree, and enter the required parameter values.

    Enabling SSL on the ADS cluster
    Enabling SSL on the ADS cluster
    Parameters for enabling SSL
    Parameter Description

    Keystore path

    Path to the keystore.jks storage on hosts

    Keystore password

    Password for the keystore.jks store, default is bigdata

    Truststore path

    Path to the truststore.jks storage on hosts

    Truststore password

    Password for the truststore.jks store, default is bigdata

    Force rewrite services SSL configuration

    If set to true, the configuration for all services will be overwritten. Use if you entered the wrong password or trust store path when you first enabled SSL

    (DANGEROUS!) Clean nifi user and authorizers

    If set to true, previously created users and previous authorization settings will be deleted

  3. Click Run.

  4. Verify the action in the opened window.

    Verify the action
    Verify the action
  5. Wait until SSL is enabled. Analyze and correct errors if they occur.

    SSl startup process
    SSl startup process
NOTE

Work in services after enabling SSL is carried out in accordance with the articles:

Disable SSL on an ADS cluster using ADCM

To disable SSL, do the following:

  1. Apply the cluster action Manage SSL by clicking actions default dark actions default light in the Actions column of the ADS cluster.

  2. In the window that opens, deactivate the Enable SSL switch.

    Disabling SSL on an ADS cluster
    Disabling SSL on an ADS cluster
  3. Click Run.

  4. Verify the action in the opened window.

    Verify the action
    Verify the action
  5. Wait for the disable SSL to complete. Analyze and correct errors if they occur.

Found a mistake? Seleсt text and press Ctrl+Enter to report it