Конференция Arenadata
Новое время — новый Greenplum
Мы приглашаем вас принять участие в конференции, посвященной будущему Open-Source Greenplum 19 сентября в 18:00:00 UTC +3. Встреча будет проходить в гибридном формате — и офлайн, и онлайн. Онлайн-трансляция будет доступна для всех желающих.
Внезапное закрытие Greenplum его владельцем — компанией Broadcom - стало неприятным сюрпризом для всех, кто использует или планирует начать использовать решения на базе этой технологии. Многие ожидают выхода стабильной версии Greenplum 7 и надеются на её дальнейшее активное развитие.
Arenadata не могла допустить, чтобы разрабатываемый годами Open-Source проект Greenplum прекратил своё существование, поэтому 19 сентября мы представим наш ответ на данное решение Broadcom, а участники сообщества получат исчерпывающие разъяснения на все вопросы о дальнейшей судьбе этой технологии.

На конференции вас ждёт обсуждение следующих тем:

  • План возрождения Greenplum;
  • Дорожная карта;
  • Экспертное обсуждение и консультации.
Осталось до события

Configure an external PostgreSQL database for Ranger metadata

Sergei Tikhomirov

Overview

The Ranger’s Migrate DB schema action allows you to migrate a database schema and service users from an internal to an external database. The migration is available for the Ranger KMS and Ranger Admin service components.

Follow the steps below to configure the database server and cluster hosts for the migration.

NOTE
Before migrating to an external database, make sure that it’s set up and running.

PostgreSQL server configuration

The example below demonstrates how to set up a PostgreSQL server on a CentOS 7 host:

  1. To prevent a faulty dependency resolution, edit the /etc/yum.repos.d/CentOS-Base.repo file by adding the following line in the [base] and [updates] sections:

    exclude=postgresql*

  2. Install the PostgreSQL configuration package.

    $ sudo yum install https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm

  3. Install the necessary version of the PostgreSQL server.

    $ sudo yum install postgresql<N>-server

    where <N> is a supported version of the server (12 is used from here on out).

  4. Create a new database cluster.

    $ sudo /usr/pgsql-12/bin/postgresql-12-setup initdb

  5. Start the PostgreSQL service using systemctl.

    $ sudo systemctl enable postgresql-12 --now

  6. See if you can access the psql console.

    $ sudo -i -u postgres; psql

From this point, refer to the relevant tab below. The commands should be launched in the psql console.

  • Ranger KMS

  • Ranger Admin

  1. Create a user for the Ranger KMS service.

    CREATE USER rangerkms WITH PASSWORD '<password>';

    where <password> is a password for the rangerkms user.

  2. Create a database for the service.

    CREATE DATABASE rangerkms;

  3. Grant permissions to the database for the created user.

    GRANT ALL PRIVILEGES ON DATABASE rangerkms TO rangerkms;

  4. Add the following line to the /var/lib/pgsql/12/data/pg_hba.conf file:

    host    rangerkms          rangerkms     <ranger_kms_host>/32              trust

    where <ranger_kms_host> is a host with an installed Ranger KMS component. On your system, the file may be located in a different directory.

  1. Create a user for the Ranger Admin service.

    CREATE USER rangeradmin WITH PASSWORD '<password>';

    where <password> is a password for the rangeradmin user.

  2. Create a database for the service.

    CREATE DATABASE ranger;

  3. Grant permissions to the database for the created user.

    GRANT ALL PRIVILEGES ON DATABASE ranger TO rangeradmin;

  4. Add the following line to the /var/lib/pgsql/12/data/pg_hba.conf file:

    host    ranger          rangeradmin     <ranger_admin_host>/32              trust

    where <ranger_admin_host> is a host with an installed Ranger Admin component. On your system, the file may be located in a different directory. Also, if there are several Ranger Admin components in your cluster, you should add a similar record for each Ranger Admin host.

To finish off, edit the /var/lib/pgsql/12/data/postgresql.conf file by setting the listen_addresses parameter to * and the port parameter to 5432.

Once it’s done, restart the PostgreSQL service:

$ sudo systemctl restart postgresql-12

Host setup

If you haven’t previously, you need to install a JDBC driver on the Ranger KMS and/or Ranger Admin hosts. To do that, follow the steps below:

  1. Download the driver. You can also obtain it from the PostgreSQL website.

    $ wget https://downloads.adsw.io/ADH/3.2.4_arenadata2/centos/7/community/x86_64/jdbc-postgresql-42.5.4-1036.el7.noarch.rpm

  2. Install the driver using yum.

    $ yum localinstall jdbc-postgresql-42.5.4-1036.el7.noarch.rpm

  3. See if the jdbc-postgresql.jar file has appeared in the /usr/share/java/ directory.

  4. Configure the service in ADCM.

  • Ranger KMS

  • Ranger Admin

  1. On the Clusters page, find your ADPS cluster and click its name.

  2. On the Services tab, click Ranger.

  3. On the opened configuration page, expand the dbks-site.xml parameter group and fill in the following parameters:

    • ranger.ks.jpa.jdbc.password — password for a user in an external database.

    • ranger.ks.jpa.jdbc.url — a JDBC connection URL, e.g. jdbc:postgresql://<database_host>:5432/rangerkms.

    • ranger.ks.jpa.jdbc.driver — a classname for a JDBC driver. For PostgreSQL, it’s org.postgresql.Driver.

    • ranger.ks.jdbc.sqlconnectorjar — a path to an SQL connector JAR, e.g. /usr/share/java/jdbc-postgresql.jar.

    • ranger.ks.jpa.jdbc.user — name of a user in an external database, e.g. rangerkms.

  4. In the Ranger KMS install.properties parameter group, change the DB_FLAVOR to POSTGRES.

  5. Click Save.

  1. On the Clusters page, find your ADPS cluster and click its name.

  2. On the Services tab, click Ranger.

  3. On the opened configuration page, expand the ranger-admin-site.xml parameter group and fill in the following parameters:

    • ranger.jpa.jdbc.password — password for a user in an external database.

    • ranger.jpa.jdbc.url — a JDBC connection URL, e.g. jdbc:postgresql://<database_host>:5432/ranger.

    • ranger.jpa.jdbc.driver — a classname for a JDBC driver. For PostgreSQL, it’s org.postgresql.Driver.

    • ranger.jdbc.sqlconnectorjar — a path to an SQL connector JAR, e.g. /usr/share/java/jdbc-postgresql.jar.

    • ranger.jpa.jdbc.user — name of a user in an external database, e.g. rangeradmin.

  4. In the Ranger Admin install.properties parameter group, change the DB_FLAVOR to POSTGRES.

  5. Click Save.

Migration

  1. Launch the Migrate DB schema action for Ranger.

  2. Choose the components for which you want to launch the migration.

    Options for the Migrate DB schema action
    The Migrate DB schema action

  3. Confirm the action.

Found a mistake? Seleсt text and press Ctrl+Enter to report it