SSL channel security

Prerequisites for enabling SSL

  • Every cluster host must have the keystore.jks key and certificate store (unique for each cluster host), know its path and password.

  • Every cluster host must have the truststore.jks key and certificate store (common for all cluster hosts), the path to it and its password must be known.

  • The list of DNs of all cluster hosts (like CN=ads-host-1.example.com, OU=AD, O=AD, L=MSK, ST=MO, C=RU) generated when creating certificates SSL must be known.

  • The name of the user who has an entry in the Active Directory that will act as the NiFi administrator and his password must be known.

Create an SSL certificate for Certification Authority

Certificate generation sequence

  1. Generate an RSA key pair for each host.

  2. Create a keystore.jks keystore on each host.

  3. Create an SSL certificate for each host.

  4. Collect all certificates in one place.

  5. Copy all certificates for each host to the keystore.jks store.

  6. Sign the certificates with a key that belongs to one of the trusted root certificates included in the Java certificate store.

  7. Create a trustore.jks store containing the required certificates.

  8. Import trustore.jks into Java CA store.

  9. Create and import an OpenSSL certificate for Nginx.

CAUTION

To correctly enable authentication in the NiFi service when creating certificates, specify and fill in the san (subjectAltNames) field using the -ext option (for example, for self-signed certificates created below, the filled field looks like this: -ext san=dns: $HOST).

Create an SSL certificate for Certification Authority using a script

For test and reference purposes, when generating certificates for each host and importing certificates into the Java keystore, you can use the special script generate.sh.

Using this script, it is possible to create new certificates and overwrite existing ones.

It is launched on a host that has access rights to cluster hosts via FQDN and internal IP.

Below there is the sequence of creating the generate.sh script.

  1. Enter the command to create the script file:

    $ sudo vim /tmp/ssl/generate.sh
  2. Replace for the following script lines (only for them!) the following values ​​with values ​​that apply to your case:

    • NUMHOSTS= — in this line specify the values ​​(letters or numbers), individual for each host. The values ​​(1 2 3) are given as an example for a list of hosts:

      • sov-ads-test-1.ru-central1.internal;

      • sov-ads-test-2.ru-central1.internal;

      • sov-ads-test-3.ru-central1.internal.

    • HOSTS= — specify the common part of the FQDN of the host in this line, replacing $i with the value (letters or numbers) specified in the NUMHOSTS parameter. The value sov-ads-test-$i.ru-central1.internal is given as an example.

    • bigdata — password for stores, recommended to change.

  3. Fill in generate.sh with the script text given below, taking into account individual parameters.

    generate.sh

     

    #!/bin/bash
    
    # Script that is used to generate and import self-signed certificates to java keystore
    # Edit NUMHOSTS and HOSTS as it will be suitable for your case.
    
    declare -a NUMHOSTS
    declare -a HOSTS
    
    SSH_OPTS='-o GlobalKnownHostsFile=/dev/null -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no'
    NUMHOSTS=(1 2 3)
    HOSTS=$(for i in "${NUMHOSTS[@]}"; do echo "sov-ads-test-$i.ru-central1.internal"; done)
    
    
    for HOST in $HOSTS; do
      	echo "update-ca-trust"
    	ssh $SSH_OPTS $HOST "sudo update-ca-trust extract"
    done
    
    echo Generate keystore.jks on each host
    for HOST in $HOSTS; do
      	echo "remove old alias"
      	ssh $SSH_OPTS $HOST "keytool -delete -alias $HOST -keystore /tmp/keystore.jks -storepass bigdata";
    
      	echo "Generating keypair"
    	ssh $SSH_OPTS $HOST "keytool 	-genkey \
    	                                -noprompt \
    	                                -keyalg RSA \
    	                                -ext san=dns:$HOST \
    	                                -alias $HOST \
    	                                -dname \"CN=$HOST, OU=AD, O=AD, L=MSK, S=MO, C=RU\" \
    	                                -keystore /tmp/keystore.jks \
    	                                -storepass bigdata \
    	                                -keypass bigdata \
    	                                -validity 360 \
    	                                -keysize 2048";
    done
    
    echo
    echo Export certificates
    for HOST in $HOSTS;do
    	ssh $SSH_OPTS $HOST "keytool 	-export \
    	                                -file /tmp/$HOST.crt \
    	                                -keystore /tmp/keystore.jks \
    	                                -storepass bigdata \
    	                                -alias $HOST \
    	                                -rfc";
    done
    
    echo
    echo Collect all certificates
    for HOST in $HOSTS; do
     	scp $SSH_OPTS $HOST:/tmp/$HOST.crt /tmp/
    
    #        scp $SSH_OPTS $HOST:/tmp/*.crt /tmp/
    done
    
    echo
    echo Transfer certificates on hosts
    for HOST in $HOSTS; do
    	for CERT in $HOSTS; do
    	    scp $SSH_OPTS /tmp/$CERT.crt $HOST:/tmp/
    	done
    
    #	scp $SSH_OPTS /tmp/*.crt $HOST:/tmp/
    done
    
    echo
    echo Import certificates on each host
    for HOST in $HOSTS; do
    
    	ssh $SSH_OPTS $HOST "for CERT in $(echo ${HOSTS[*]}); do
    
    		echo "remove old alias before import"
    		keytool -delete -noprompt -alias \$CERT -keystore /tmp/truststore.jks -storepass bigdata
    
    		echo "Import Cert"
    		keytool 	-import \
    		 	        -noprompt \
    		 	        -alias \$CERT \
    		 	        -file /tmp/\$CERT.crt \
    		 	        -keystore /tmp/truststore.jks \
    		 	        -storepass bigdata;
    		sudo bash -c \"cat /tmp/\$CERT.crt >> /etc/pki/tls/certs/ca-bundle.crt\";
    		done";
    done
    
    echo
    echo Import truststore to Java CA store
    for HOST in $HOSTS; do
    	ssh $SSH_OPTS $HOST "sudo keytool 	-importkeystore \
    	 	                                -noprompt \
    	 	                                -srckeystore /tmp/truststore.jks \
    	 	                                -destkeystore /etc/pki/java/cacerts \
    	 	                                -deststorepass changeit \
    	 	                                -srcstorepass bigdata"
    done
    
    echo
    echo Create directories for certs
    for HOST in $HOSTS; do
    	ssh $SSH_OPTS $HOST "[ -d /etc/ssl ] || sudo mkdir /etc/ssl;
    			     [ -d /etc/ssl/certs ] || sudo mkdir /etc/ssl/certs"
    done
    
    echo
    echo Create and import cert for Nginx
    for HOST in $HOSTS; do
    	ssh $SSH_OPTS $HOST "sudo openssl req -new -newkey rsa:4096 -days 365 -nodes -x509 -subj \"/C=RU/ST=Denial/L=MSK/O=AD/CN=$HOST\" -keyout /etc/ssl/host_cert.key  -out /etc/ssl/certs/host_cert.cert"
    	ssh $SSH_OPTS $HOST "sudo bash -c \"cat /etc/ssl/certs/host_cert.cert >> /etc/pki/tls/certs/ca-bundle.crt\""
    done
  4. Allow script execution:

    $ sudo chmod +x /tmp/ssl/generate.sh
  5. Run the script:

    $ /tmp/ssl/generate.sh
  6. Monitor script execution.

  7. Check each host for storages and files:

    • /etc/ssl/host_cert.key

    • /tmp/truststore.jks

    • /tmp/keystore.jks

NOTE
The NiFi service requires additional configurations in accordance with the article NiFi service authentication.

Start SSL on an ADS cluster using ADCM

  1. Initiate enabling SSL. To do this, apply the Enable SSL action by clicking the actions default dark actions default light icon in the Actions column.

    Enabling SSL on the ADS cluster
    Enabling SSL on the ADS cluster
  2. In the window that opens, enter the appropriate configurations and click Run:

    • Keystore path — path to keystore.jks storage on hosts.

    • Keystore password — password for the keystore.jks store, default is bigdata.

    • Truststore path — path to truststore.jks storage on hosts.

    • Truststore password — password for the truststore.jks store, default is bigdata.

    • Force rewrite SSL configuration of services — enable if it is necessary to rewrite the configuration for all services. Use if you entered the wrong password or truststore path when you first enabled SSL.

    • (DANGEROUS!) Clean nifi user and authorizers — enable if you want to remove previously created users and previous authorization settings.

      CAUTION
      • Enabling SSL for NiFi service is available starting from version 1.7.0.b1 of the ADS cluster.

      • Option (DANGEROUS!) Clean nifi user and authorizers is available starting from version 1.7.2.b2 of the ADS cluster.

      • Option Force rewrite services SSL configuration is available starting from version 1.8.1.b1 of the ADS cluster.

      Setting configurations to run SSL
      Setting configurations to run SSL
  3. Verify the action in the opened window.

    Action confirmation
    Verify the action
  4. Click Run.

  5. Wait until SSL is enabled. Analyze and correct errors if they occur.

    SSl startup process
    SSl startup process
NOTE

Work in services after starting Enable SSL is carried out in accordance with the articles:

Disable SSL on an ADS cluster using ADCM

  1. Initiate disable SSL. To do this, apply the Disable SSL action by clicking the actions default dark actions default light icon in the Actions column.

    Disabling SSL on an ADS Cluster
    Disabling SSL on an ADS Cluster
  2. Enable the (DANGEROUS!) Clean nifi user and authorizers option if you want to remove previously created users and previous authorization settings, ang click Run.

    Starting disable SSL on an ADS cluster
    Starting disable SSL on an ADS cluster
  3. Verify the action in the opened window.

    Verify the action
    Verify the action
  4. Wait for the disable SSL to complete. Analyze and correct errors if they occur.

    SSL disable process
    SSL disable process
Found a mistake? Seleсt text and press Ctrl+Enter to report it