Kafka to ADB Connector installation
To install Kafka to ADB Connector, follow the steps:
-
Make sure that the Kafka to ADB service is installed. For information on how to add services to ADB clusters, see Add services.
-
Connect to the database via
psql(or any other client program):$ psql adb -
Check that the new extension
kadb_fdwis available — via thepsqlmeta-command\dx:\dxThe output contains the
kadb_fdwextension:List of installed extensions Name | Version | Schema | Description -------------------+---------+------------+--------------------------------------------------------------------- adb_fdw | 1.0 | public | foreign-data wrapper for remote ADB servers arenadata_toolkit | 1.0 | public | extension is used for manipulation of objects created by adb-bundle dblink | 1.2 | public | connect to other PostgreSQL databases from within a database gp_exttable_fdw | 1.0 | pg_catalog | External Table Foreign Data Wrapper for Greengage gp_toolkit | 1.8 | gp_toolkit | various GPDB administrative views/functions kadb_fdw | 0.16 | kadb | Kafka-ADB foreign data wrapper plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language pxf | 2.1 | public | Extension which allows to access unmanaged data pxf_fdw | 1.0 | public | PXF Foreign Data Wrapper for Greengage tkh_fdw | 1.0 | public | ClickHouse Foreign Data Wrapper for Greenplum (10 rows) -
You can also check that the new foreign data wrapper
kadb_fdwis available using the following SQL query against thepg_catalog.pg_foreign_data_wrappersystem table:SELECT * FROM pg_catalog.pg_foreign_data_wrapper;Result:
oid | fdwname | fdwowner | fdwhandler | fdwvalidator | fdwacl | fdwoptions -------+-----------------+----------+------------+--------------+--------+-------------------------------------------------------------------------------------------- 13702 | gp_exttable_fdw | 10 | 13700 | 13701 | | 17124 | dblink_fdw | 10 | 0 | 17123 | | 17175 | adb_fdw | 10 | 17173 | 17174 | | 17187 | jdbc_pxf_fdw | 10 | 17185 | 17186 | | {protocol=jdbc,"mpp_execute=all segments"} 17188 | hdfs_pxf_fdw | 10 | 17185 | 17186 | | {protocol=hdfs,"mpp_execute=all segments"} 17189 | hive_pxf_fdw | 10 | 17185 | 17186 | | {protocol=hive,"mpp_execute=all segments"} 17190 | hbase_pxf_fdw | 10 | 17185 | 17186 | | {protocol=hbase,"mpp_execute=all segments"} 17191 | s3_pxf_fdw | 10 | 17185 | 17186 | | {protocol=s3,"mpp_execute=all segments"} 17192 | gs_pxf_fdw | 10 | 17185 | 17186 | | {protocol=gs,"mpp_execute=all segments"} 17193 | adl_pxf_fdw | 10 | 17185 | 17186 | | {protocol=adl,"mpp_execute=all segments"} 17194 | wasbs_pxf_fdw | 10 | 17185 | 17186 | | {protocol=wasbs,"mpp_execute=all segments"} 17195 | file_pxf_fdw | 10 | 17185 | 17186 | | {protocol=file,"mpp_execute=all segments"} 17201 | tkh_fdw | 10 | 17200 | 17199 | | {protocol=tkh_clickhouse,"mpp_execute=all segments",format=text} 17210 | kadb_fdw | 10 | 17209 | 17211 | | {"mpp_execute=all segments",k_initial_offset=0,k_automatic_offsets=true,k_latency_ms=2000} (14 rows) -
Ensure that the kadb schema is available — via the
psqlmeta-command\dnS+:\dnS+Result:
Name | Owner | Access privileges | Description --------------------+---------+--------------------+------------------------------------------------------------- arenadata_toolkit | gpadmin | gpadmin=UC/gpadmin+| | | =U/gpadmin | diskquota | gpadmin | | gp_toolkit | gpadmin | gpadmin=UC/gpadmin+| | | =U/gpadmin | information_schema | gpadmin | gpadmin=UC/gpadmin+| | | =U/gpadmin | kadb | gpadmin | | pg_aoseg | gpadmin | | Reserved schema for Append Only segment list and eof tables pg_bitmapindex | gpadmin | | Reserved schema for internal relations of bitmap indexes pg_catalog | gpadmin | gpadmin=UC/gpadmin+| system catalog schema | | =U/gpadmin | pg_toast | gpadmin | | reserved schema for TOAST tables public | gpadmin | gpadmin=UC/gpadmin+| standard public schema | | =UC/gpadmin | (10 rows)